SD reading problems with certain block sizes

NETMF & Gadgeteer
1

I found this issue on the Netduino Plus product also – it may be an issue in NETMF. They have fixed it, but I don’t know if they feed the code back to Microsoft, or how bugfixes work in the framework. The report for that platform is at [url]Reading problems on SD card - Netduino Plus 2 (and Netduino Plus 1) - Netduino Forums

I created a file on an SD with the bytes from 0x00…0xFF repeating 8 times, and then read it in chunks and verified the contents.

When the chunk size is 256 or 512, it works fine. If the chunk size is 264, it goes wrong.

using System.IO;
using GHIElectronics.NETMF.IO;
using Microsoft.SPOT;

namespace FezFileReadTest
{
    public class Program
    {
        public static void Main()
        {
            var persistentStorage = new PersistentStorage("SD");
            persistentStorage.MountFileSystem();
            using (var inputStream = new FileStream(@ "\SD\dummydata", FileMode.Open))
            {
                byte nextByte = 0;
                var readBuffer = new byte[264];
                int offset = 0;
                int errors = 0;
                while (true)
                {
                    var bytesRead = inputStream.Read(readBuffer, 0, readBuffer.Length);
                    if (bytesRead == 0)
                    {
                        Debug.Print("Done! " + errors + " errors detected.");
                        break;
                    }
                    for (int counter = 0; counter < bytesRead; ++counter)
                    {
                        if (readBuffer[counter] != nextByte)
                        {
                            Debug.Print("Read error! Offset " + offset + " expected " + nextByte + ", received " + readBuffer[counter]);
                            ++errors;
                        }
                        ++nextByte;
                        ++offset;
                    }
                }
            }
        }
    }
}

When I run this, I see:

Read error! Offset 914 expected 146, received 33
Read error! Offset 915 expected 147, received 58
Read error! Offset 1426 expected 146, received 33
Read error! Offset 1427 expected 147, received 58
Read error! Offset 1938 expected 146, received 33
Read error! Offset 1939 expected 147, received 58
Done! 6 errors detected.

I can provide the datafile on request.

2

Thanks for feedback. We will dig into this and get back to you

3

I think this problem has been discussed before, and a trouble ticket was opened by GHI with Microsoft?

4

Thanks for the quick follow-up – the Netduino folks may have submitted a patch, as they’ve fixed this in their implementation. Not sure if it’s in a NETMF tracking system or anything.

I was able to implement a workaround – basically I just wrap the underlying SD stream in a stream that will only do 256-byte reads. This fixes my problem, but it may not be terribly general purpose. I’m including it here.

Thanks for the help!

// There are some stream implementations that behave poorly when given
// random read sizes. In order to mitigate this, FixedBufferReadStream
// will wrap an underlying stream, and only do reads of a fixed size.
// Note that this will "over-read" from the underlying stream by as
// much as the buffer size.
//
// This class does not take ownership of the underlying stream. Most
// notably, Dispose will not do anything to it.

class FixedBufferReadStream : Stream
{
    private Stream stream;
    private byte[] buffer;
    private int lastBytesRead;
    private int offset;

    public FixedBufferReadStream(Stream stream, int bufferSize)
    {
        this.stream = stream;
        this.buffer = new byte[bufferSize];
        RefillBuffer();
    }

    private void RefillBuffer()
    {
        this.lastBytesRead = stream.Read(buffer, 0, buffer.Length);
        this.offset = 0;
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        int bytesToCopy = (count < (lastBytesRead - this.offset)) ? count : (lastBytesRead - this.offset);

        for (int counter = 0; counter < bytesToCopy; ++counter)
        {
            buffer[offset + counter] = this.buffer[this.offset + counter];
        }
        this.offset += bytesToCopy;

        if (this.offset == this.lastBytesRead)
        {
            RefillBuffer();
        }

        return bytesToCopy;
    }

    #region Unimplemented Stream members
    public override bool CanRead
    {
        get { throw new System.NotImplementedException(); }
    }

    public override bool CanSeek
    {
        get { throw new System.NotImplementedException(); }
    }

    public override bool CanTimeout
    {
        get
        {
            return base.CanTimeout;
        }
    }

    public override bool CanWrite
    {
        get { throw new System.NotImplementedException(); }
    }

    public override void Flush()
    {
        throw new System.NotImplementedException();
    }

    public override long Length
    {
        get { throw new System.NotImplementedException(); }
    }

    public override long Position
    {
        get
        {
            throw new System.NotImplementedException();
        }
        set
        {
            throw new System.NotImplementedException();
        }
    }

    public override void Write(byte[] buffer, int offset, int count)
    {
        throw new System.NotImplementedException();
    }

    public override void SetLength(long value)
    {
        throw new System.NotImplementedException();
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        throw new System.NotImplementedException();
    }
    #endregion
}
5

We have a firmware in the beta page but there is no access for “Newbie”. Please send us an email with link to this post so we can email you the firmware.