Hello,
I need to read some small log files from SD and send them to a Web Server.
When I try to create an instance of streamreader using this cose, I get an OutOfMemory exception:
using (System.IO.StreamReader reader = new System.IO.StreamReader ("filepath") )
If I run
Debug.GC(true);
before that code, it reports 10/12 KB free
How can I read the file? Basically it is a log, so I need the read it line-by-line.
Thanks
It could be interesting to know what is the hardware target. I mean if your application runs on USBizi, 10/12kb of free mem can be normal depending on your code, but if your code runs on EMX, 10/12kb left is probably due to a strong memory leakage in your code.
Sorry: I’m running Panda II
This can be the reason. There’s not a lot of RAM memory on such platform. Be sure to Dispose unusefull object before dealing with filesystem. You should also try to move this part of code at an early stage of your algorithm. If you don’t run other part that can be memory consumming does your code could work?
I’m not running “heavy” code, and in fact GC reports 10/12 KB ram, that I think is a reasonable value on the Panda II
I’ve found a solution using
using (FileStream fs = new FileStream(item, FileMode.Open, FileAccess.Read, FileShare.None, 80))
{
using (System.IO.StreamReader reader = new System.IO.StreamReader(fs))
to force FileStream to use a small buffer to read data. However it seems strange I’ve this problem with a program that is really making just a little work… 
It depends on how big/small the file is you’re reading
As Eric said, this depends on how big is your file. Even if your code does not seem to be heavy, some of the assembly reference can be very memory consumming. On a previous target (USBisy), loading the all the assembly for our application didn’t let us enough free mem to run the application.10/12kb can be enough in a lot of cases but in other such as the one you mention may be not.
You are doing it right, just use the buffer of smallest possible size when needed.
Try also to avoid as much as possible string (even for Debug.Print purpose). There’s also some bugs in netmf such as HashTable that is growing undefinitely till crashing the platform.
The problem is somewhere else than just streamreader code… There’s something that eats memory (double check the assembly references, the use of string and so on…)
Regards
Thank you
Just for reference: the file I try to read is 1KB…
Oh my, is there a known issue tracker with this information? I swear the memory management on my Panda is awful. I have to keep way too much memory free to keep it from crashing. I strongly believe something is wrong @ a fundamental level.
I have a Panda running for over a year now without any issues. If you keep running out of memory, something in your code ain’t right (with the exception of the Hashtable)
I am using a hashtable…
Well I don’t think I have the issue documented in that work item.
What about strings in general?
New version of .NET MF has some memory management fixes i think so I’m waiting to test it (we need to wait for GHI to update the premium libs). This problem is more often seen on small devices like Panda.
However I still have outOfMemory also with 12-13KB free executing
using (System.IO.StreamReader reader = new System.IO.StreamReader(fs))
Baz, I am glad you posted this because I asked myself “Didn’t I try this with no luck??”
I dealt with this issue a month or so back. I had more than enough free memory but StreamReader was not happy regardless. I ended up using FileStream (with a small buffer) only and don’t use stream reader at all.
I’m “not so happy” to know that there is some real problem not caused by my code
And how did you complete a ReadLine from the file without problem?
I am not confirming that StreamReader is broken, but it did not work for me.
To answer your question you can do the following:
fs.Read(buf, 0, buf.Length);
Streamreader allocates a buffer of about 4k up front before reading anything. The problem is it needs 4k of contiguous space… One large block with no fragmentation. This is ususually not available by the time you open it and it seems to allocate inside the Read method.
Try and create a streamreader very early in the program and read something immediately and see if makes a difference.
I have also just used Filestream.Read in the end.
There used to be a StreamReaderEx on die code site. Have a look if you can find it.
it seems that reading byte-after-byte with simple ReadByte solves the problem.
Is not so easy to re-implement a ReadLine using this approach, but at least I am not “OutOfMemory”…