Understanding GC details and OutOfMemoryException

Ronny_Hansen · January 24, 2012, 4:43pm

Hi,

We have met a problem which is probably related to memory leak, but it could also be that our application is getting quite large.

The app is currently running on the Embedded Master module which should have 8 MB of RAM, but the GC reports a total of 5242632 bytes (about 5MB) Where does the remaining 3MB go ?

Will moving all strings (like in the exemple below) to resource file reduce total RAM usage ?


Log.Info("This is a string, but is it taking up RAM or is it stored in Flash ?");

The problem seems to be “SZARRAY” which grows until we break. What kind of data/objects go to type 13 (SZARRAY) ? Need input to help me debug my code…

Why does the allocation fail (Failed allocation for 53 blocks, 636 bytes), we still have more than 636 bytes available…

Does there exists some way to query the GC or other system objects about total memory usage during runtime, and is there a way to debug/show the total objects of each type ?

GC: 638msec 5218320 bytes used, 24312 bytes available <= Giving a total of 5242632 bytes (!?)
Type 0F (STRING ): 110772 bytes
Type 11 (CLASS ): 1451760 bytes
Type 12 (VALUETYPE ): 4632 bytes
Type 13 (SZARRAY ): 2829900 bytes
Type 15 (FREEBLOCK ): 24312 bytes
Type 17 (ASSEMBLY ): 71484 bytes
Type 18 (WEAKCLASS ): 432 bytes
Type 19 (REFLECTION ): 204 bytes
Type 1B (DELEGATE_HEAD ): 7200 bytes
Type 1C (DELEGATELIST_HEAD ): 120 bytes
Type 1D (OBJECT_TO_EVENT ): 696 bytes
Type 1E (BINARY_BLOB_HEAD ): 481728 bytes
Type 1F (THREAD ): 4224 bytes
Type 20 (SUBTHREAD ): 432 bytes
Type 21 (STACK_FRAME ): 4752 bytes
Type 22 (TIMER_HEAD ): 144 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD): 48 bytes
Type 27 (FINALIZER_HEAD ): 242568 bytes
Type 31 (IO_PORT ): 468 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 6684 bytes
Failed allocation for 53 blocks, 636 bytes

Gus_Issa · January 24, 2012, 5:01pm

This just came up here this week. There is no official explanation of these messages anywhere.

You can add debug.gc(true) in different places in your app to see where does memory start increasing.

Ronny_Hansen · January 24, 2012, 5:13pm

Ok, but do you have any feedback on the other questions ?

Like - will total RAM usage be reduced if we put strings into Resource files ?

Thanks.

Gus_Issa · January 24, 2012, 5:21pm

It will and will not. In resource will be less as long as you do not use them. Then after use you can dispose to free the memory.

Ronny_Hansen · January 24, 2012, 5:25pm

Ok, so if I understand you correct:

All my “strings” as in my example is actually taking RAM when running, even if the class is not initialized.
Many such strings will add up to a large amount of RAM.

Moving the strings to Resource file will only use RAM when i use them, after that they will be disposed, and total memory usage will be less ?

Just to be 100% sure before I modify 1000+ of code lines…

// Ronny

Gus_Issa · January 24, 2012, 5:31pm

Yes but there is a thing here. EMX or EM use XiP memory so netmf maybe smarter about this and you will actually save ram if not using resources!

I would make a test app and run debug.gc to check best cases.

Anyways, you have 8mb which is very large if we are talking strings. You shouldn’t need to do any of the above.

Ronny_Hansen · January 24, 2012, 5:34pm

Thanks.

Strangely the GC reports:

GC: 638msec 5218320 bytes used, 24312 bytes available <= Giving a total of 5242632 bytes (!?)

This is just 5MB, where is my remaining 3MB ?

Mike · January 24, 2012, 5:38pm

If GC is taking 638 milliseconds to complete, you have a LOT of objects floating around.

Gus_Issa · January 24, 2012, 5:42pm

Not all 8 is heap. Where is the video buffer for example?

Ronny_Hansen · January 24, 2012, 5:42pm

Yes,

The application is HUGE.

I would really like to have some way to traverse the allocated objects to see what they are to understand more about what thay are and who owns them (or not…).

I am afraid I am having a memory leak, and it’s difficult to even have an idea where the error is since tha app is so big.

Maybe in .NET MF 5… :o

Ronny_Hansen · January 24, 2012, 5:46pm

Ok, I just added support for Video (currently using the VGA Extension adapter).

I am using WPF for the UI (which is pretty darn slow…)

Is this change in the code causing the EM to “grab” 3MB for Video?
And without the WPF no video buffer ? (ie 3MB RAM more?)

Gus_Issa · January 24, 2012, 5:49pm

I think you are looking in the wrong place. If you have memory leak then none of this would matter right?

First, run the debug.gc as suggested in different places. For example, what do you see when you run this at the beginning of your huge application?

Ronny_Hansen · January 24, 2012, 6:07pm

It apears that there is a GC glitch in NETMF 4.1

Maybe only on Netduino, but maybe not…

Here is my GC dump just after bootup:


GC: 213msec 1154628 bytes used, 4088004 bytes available
Type 0F (STRING              ):   8724 bytes
Type 11 (CLASS               ): 462972 bytes
Type 12 (VALUETYPE           ):   4680 bytes
Type 13 (SZARRAY             ):  96060 bytes
Type 15 (FREEBLOCK           ): 4088004 bytes
Type 16 (CACHEDBLOCK         ):   2328 bytes
Type 17 (ASSEMBLY            ):  71868 bytes
Type 18 (WEAKCLASS           ):    432 bytes
Type 19 (REFLECTION          ):    204 bytes
Type 1B (DELEGATE_HEAD       ):   7956 bytes
Type 1C (DELEGATELIST_HEAD   ):    144 bytes
Type 1D (OBJECT_TO_EVENT     ):    792 bytes
Type 1E (BINARY_BLOB_HEAD    ): 478056 bytes
Type 1F (THREAD              ):   3840 bytes
Type 20 (SUBTHREAD           ):    480 bytes
Type 21 (STACK_FRAME         ):   7608 bytes
Type 22 (TIMER_HEAD          ):    144 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD):    108 bytes
Type 27 (FINALIZER_HEAD      ):    816 bytes
Type 31 (IO_PORT             ):    612 bytes
Type 34 (APPDOMAIN_HEAD      ):     72 bytes
Type 36 (APPDOMAIN_ASSEMBLY  ):   6732 bytes
End parsing ...

It appears that I have plenty of free RAM (4088004 bytes) after I boot up, there must be some leak somewhere…

I have added a new console command so I can force GC to run. I will try to isolate the leak. :

Gus_Issa · January 24, 2012, 6:21pm

Yep I told you, even your HUGE application is using only about 1MB of RAM So do not worry about strings or about “saving memory” as you have plenty of memory. You still got 4MB free to go.

Ronny_Hansen · January 31, 2012, 5:19pm

Ok, so I do [italic]think[/italic] I have found out of the problem…

When I added a “console” command to run GC and report the free memory after a GC (both to serial console and to the Debug window in VS) it seems like there is no more memory leak anymore…

I have done some changes to the code as well to minimize the potential problem you can get if you dont unhook your event handlers.

I have also stopped using the internal DNS and WebService functions, and are using a special class for socket connections that I have implemented to get around the bug that the Connect() may NEVER return if the ACK/NACK sequence is not correct (in case of GPRS with packet loss or Firewalls).

I got a feedback from Colin Miller at Microsoft informing me this:

[quote]Call Dispose on all types that support it (eg bitmap)

            Dont write any busy loops (tight looping without releasing resources.  There was a bug in 4.1 where if there is always an active thread in the managed app, objects with Dispose or a finalize will not get GCd.  You want to avoid busy loops in any event.[/quote]

This may be the main source of my problem, busy loops in threads - or busy socket connections on threads that I try to kill with Thread.Abort() after a failed Socket.Connect.