Filestream.read not returning all bytes

Archive
#1

I am trying to read a jpg from an sdcard. Card opens fine and I can see the length of the file. The issue is the fStream.Read only returns a small fraction of the bytes that should have been read. I have tried multiple files. In one jpg file it is always 177 bytes, and in a different jpg file it is always 108 bytes. The files are >200kb so from samples I realize it must be done iteratively.

The issue is that it doesn’t fill the buffer on the first read, but even when I shrink the buffer to < 100 bytes the second time calling these lines I get the exception below.

fStream = new FileStream(rootDirectory + @ “\08.jpg”, FileMode.Open,FileAccess.Read);
n = fStream.Read(bBuffer, numBytesRead, 1024);
fStream.Close();

Exception System.ArgumentException - 0xfd000000 (1)

#### GadgeteerApp1.Program::GetImage [IP: 0070] ####
#### GadgeteerApp1.Program::ProgramStarted [IP: 000d] ####

A first chance exception of type ‘System.ArgumentException’ occurred in Microsoft.SPOT.IO.dll

Any thoughts?

#2

At what exact line do you get the exception (try by stepping through the code).
Is it possible that numBytesRead + 1024 might get larger than your buffer?
This call should look like

n = fStream.Read(bBuffer, numBytesRead, bBuffer.bBuffer.Length - numBytesRead);
numBytesRead += n;

If you can not find a solution I would start doing the following:
[ol]Check cabling, may be change Gadgeteer cable
Reformat SD card (FAT32). Just because it’s working fine on your PC does not mean NETMF works as well. The error redundancy is not as good.
Try a different SD Card, may be even a different type/brand. If a different SD card works, tell GHI which exact brand/type does not work, then they might be able to improve the driver.[/ol]

#3

It seems to be an issue with the jpg files. I created a .dat (Unicode) and filled it with 1s. The first Read actually gets 1024 bytes correctly. The second read throws the exception. jpg files seem to only be reading a header or something. The exception occurs on

n = fStream.Read(bBuffer, numBytesRead, bBuffer.bBuffer.Length - numBytesRead);

#4

What is the size of your buffer?

#5

what type is fStream?
FileStream, StreamReader, …?

#6

This is the test function. Ultimately the file could be larger than buffer so the end goal is to assign each buffer to LargeBuffer. But have not gotten that far yet. The Test byte[] will not fill with jpg bytes for some reason.


 public void GetImage()
        {
            String rootDirectory = sdCard.GetStorageDevice().RootDirectory;
            FileStream fStream = null;
            FileInfo fi = new FileInfo(rootDirectory + @ "\008.jpg");
            //_lb = new LargeBuffer((int)fi.Length);
            int numBytesToRead = (int)fi.Length;
            int numBytesRead = 0;
            byte[] test = new byte[1024];
            int t = 1024;

            fStream = new FileStream(rootDirectory + @ "\008.jpg", FileMode.Open, FileAccess.Read, FileShare.Read, t);
                  
             while (numBytesToRead > 0)
           {
           
                
                 int n = 0;
                 try
                 {
                     //fStream = new FileStream(rootDirectory + @ "\008.jpg", FileMode.Open, FileAccess.Read, FileShare.Read, t);
                     n = fStream.Read(test, numBytesRead, t);
                    // fStream.Close();
            
                    //test.CopyTo(_lb.Bytes, n);
                    numBytesRead += n;
                    numBytesToRead -= n;
                    }
                    catch (Exception ex)
                    {
                    }
           }
             fStream.Close();
            
        }
#7

Not sure why it wont work on jpg files, but as I have written before:
you must reduce t by numBytesRead or you get the exception on 2nd call!



offset + count <= buffer.Length !!!
#8

Hi,
form me this code worked.
Perhaps the reason is the constructor of the FileStream



byte[] b = new byte[2048];
FileStream iFile;
iFile = new FileStream(sFile, FileMode.Open, FileAccess.Read);
Int lRemain = iFile.Length;

while (lRemain > 0)
{
       if (lRemain < 2048)
       b = new byte[lRemain];

       iFile.Read(b, 0, b.Length);
       // do something with the contents oft he buffer;
       lRemain -= b.Length;
}
iFile.Close();

Merry Christmas
Roland
#9

Reinhard…The issue is that t - numBytesRead is too large and throws and exception. Shouldn’t the count be 1024 each time I read and the offset be numBytesRead?

RoSchmi…In this example b.Length or from my code:
n = fStream.Read(test, numBytesRead, t);

n is always the length that test buffer was initialized to…even if only 103 bytes were filled.

Ok so JPGs are a no go.

I converted the JPG to .bmp and now the test buffer fills with values on the read, but I still have an issue with the size and iterating.

Even if I reinitialize fStream each time, I still get an exception on the second pass.


   while (numBytesToRead > 0)
           {

               byte[] test = new byte[1024];
               int t = 1024;

                 int n = 0;
                 try
                 {
                     fStream = new FileStream(rootDirectory + @ "\008.bmp", FileMode.Open, FileAccess.Read, FileShare.Read, t);
                     n = fStream.Read(test, numBytesRead, t);
                     fStream.Close();
            
                    // fStream.Close();
                    n = test.Length;
                    //test.CopyTo(_lb.Bytes, n);
                    numBytesRead += n;
                    numBytesToRead -= n;
                    }
                    catch (Exception ex)
                    {
                    }
           }
#10

@ Raptor User 702 - What is the exception?

#11

Exception System.ArgumentException - 0xfd000000 (1)

#### Message: 
#### Microsoft.SPOT.IO.NativeFileStream::Read [IP: 0000] ####
#### System.IO.FileStream::Read [IP: 0028] ####
#### GadgeteerApp1.Program::GetImage [IP: 0050] ####
#### GadgeteerApp1.Program::ProgramStarted [IP: 000d] ####

A first chance exception of type ‘System.ArgumentException’ occurred in Microsoft.SPOT.IO.dll
The program ‘[17] Micro Framework application: Managed’ has exited with code 0 (0x0).

#12

The first read you do is of length numBytesRead which starts out at zero?

#13

When this happens your numBytesRead points past the length of the test array.

What are you trying to do with the data once you have read it from the stream?

#14

RoSchmi (and the rest of you) were right…this works. My fundamental misunderstanding was how the read method of FileStream worked. It turns out the offset is not how far into the file you want to retrieve from :slight_smile:

Last piece is what to do with each buffer. This does not seem to be the answer for each iteration… b.CopyTo(_lb.Bytes, b.Length);

The total file size is 2880054 bytes.

Thanks!


String rootDirectory = sdCard.GetStorageDevice().RootDirectory;                               
FileInfo fi = new FileInfo(rootDirectory + @ "\008.bmp");
            _lb = new LargeBuffer((int)fi.Length);

            byte[] b = new byte[2048];
             FileStream iFile;
             iFile = new FileStream(rootDirectory + @ "\008.bmp", FileMode.Open, FileAccess.Read);
            int lRemain = (int)iFile.Length;

            while (lRemain > 0)
            {
                if (lRemain < 2048)
                   b = new byte[lRemain];

                iFile.Read(b, 0, b.Length);
                b.CopyTo(_lb.Bytes, b.Length);
                 lRemain -= b.Length;
            }
            iFile.Close();
#15

It works! CopyTo requires an index not the number of bytes!!!

Thanks again!

 

 public void GetImage()
        {
            String rootDirectory = sdCard.GetStorageDevice().RootDirectory;
            //FileStream fStream;
            FileInfo fi = new FileInfo(rootDirectory + @ "\008.bmp");
            _lb = new LargeBuffer((int)fi.Length);

            byte[] b = new byte[2048];
             FileStream iFile;
             int indexOfData = 0;


             iFile = new FileStream(rootDirectory + @ "\008.bmp", FileMode.Open, FileAccess.Read);
            int lRemain = (int)iFile.Length;

            while (lRemain > 0)
            {
                if (lRemain < 2048)
                   b = new byte[lRemain];

                iFile.Read(b, 0, b.Length);
                Thread.Sleep(1);
                
                b.CopyTo(_lb.Bytes, indexOfData );
                 lRemain -= b.Length;
                 indexOfData += b.Length;

            }
            iFile.Close();
      
}
#16

Even you solved your problem already.

The 3rd parameter (number of bytes to read) must fit into the buffer, starting at offset.
so Bytes to read = buffer.Length - offset (or smaller).