Can I tell if GC has run

Archive
1

I know I get a message in the debug output window whenever the GC runs and VS is connected to my system via USB. Now I need to run some long term tests where I can’t be connected and I’d like to know if the GC has run and if so, how often and how long it takes. Is there any way to do this from inside my program?

I know I can run Debug.GC(false) in my program but don’t know how to interpret the results to tell if GC has run or not.

Thanks - Gene.

2

I don’t think you can get any information about if/when GC runs. Given GC is guaranteed to run sometime (it is after all garbage collection and must run when the situation requires). You might be better off just looking at memory usage over time to make sure your coding practices don’t cause undue need for GC, as that will minimise the impact for you.

Is your app sensitive to when GC runs? Or are you just trying to gauge the impact you’re having?

3

Just trying to gauge the impact of GC on my app and learn a little more about it. Also, I’ve made some dumb mistakes in the past that caused GC to run for minutes at a time and I need to make sure that won’t happen.

Here is what I’ve been able to gather so far about GC. I don’t know if any of this is actually correct but it is my current understanding. Corrections and/or additions will be greatly appreciated.

  1. Anything allocated private static outside a method will not be GC’ed
  2. If everything in my app is declared private static outside any method, the GC should never run.
  3. Declaring value type variables (int, double, long, etc.) inside a method should not cause the GC to run.
  4. Declaring any variable that requires “new” inside a method will eventually cause GC to run.
  5. Every string operation (like append, split, substring) will eventually cause GC.
  6. Some of the StringBuilder operations (like clear and append) do not cause GC.
  7. It is generally not a good idea to force GC with Debug.GC(true) but maybe in my situation where I do several hours of moderately time critical stuff and then sit idle for a while, it might be a good idea to force a GC during the idle time.

Again, if I’ve got any of this wrong or you can add more to my inadequate understanding of GC, I’d greatly appreciate it. [em][/em]

2 Likes
4
  1. Anything allocated private static outside a method will not be GC’ed
  • Correct as long as you are not dynamically changing the reference value that the static is holding while the application is running. The GC runs it finds all objects that are no longer accessible either directly accessible or indirectly via some other root. If you change the reference value of the private static and the previous reference no longer has any roots then that instance will be eligible for GC if/when the GC kicks in.
  1. If everything in my app is declared private static outside any method, the GC should never run.
  • Assuming that you do not dynamically allocate objects and assign the references to the private static variable. See previous answer.
  1. Declaring value type variables (int, double, long, etc.) inside a method should not cause the GC to run.
  • Correct, but beware that user defined value types (struct) in .NETMF are treated like reference types from the perspective of the GC so this only holds true for .NETMF native value types. This is different to the desktop .NET Framework.
  1. Declaring any variable that requires “new” inside a method will eventually cause GC to run.
  • Correct, assuming the method or other allocations in the system occur enough times so that the memory threshold is exceeded and the GC is educed. Just allocating an object will not mean that the GC will execute.
  1. Every string operation (like append, split, substring) will eventually cause GC.
  • Correct, because strings are immutable any changes to a string actually creates a new string and orphans the previous instance, but only if/when the GC is actually induced.
  1. Some of the StringBuilder operations (like clear and append) do not cause GC.
  • Clear would not cause GC, but of course eventually the StringBuilder probably goes out of scope (unless it is global) and then it and the internal buffer etc. are eligible for GC. Append however could result in GC, the internal buffer has to eventually grow if what you append exceeds the current buffer size. To grow the buffer an new larger buffer is created and the old one will be orphaned and potentially eventually GC’d. However, using StringBuilder is good because GC will happen much less frequently than with straight string concatenation.

I have tried to be reasonably accurate without being overly fussy about the terminology, but if required I would be happy to delve a little deeper. I hope this helps.

  1. It is generally not a good idea to force GC with Debug.GC(true) but maybe in my situation where I do several hours of moderately time critical stuff and then sit idle for a while, it might be a good idea to force a GC during the idle time.
  • Well, since .NETMF GC is not auto-tuning calling Debug.GC(true) is not as much of a “sin” as it is in the desktop framework. I call it after allocating large amounts of data that I am going to discard and do not want to pay the price for later. Normally I try to allocate everything I need up front so that I do not have any (or as few as possible) allocation during the run time of the application. Of course some of the module drivers do not follow this approach and therefore you sometimes incur additional GCs. When doing things that I know the GC might throw my routine for a loop, I check the driver code for the modules I am using to assess the potential “hidden” impact.
3 Likes
5

@ taylorza - Thanks, that is incredibly helpful. Can I ask where you got all that detail? I’ve found some references that describe the full .Net GC and even a reference that talks a little about the Compact .Net GC but nothing of any substance for the .Net Micro GC.

6

I also find this thread to be incredibly helpful in understanding GC and hope that the thread will grow with time as more information is added.
This is the kind of thread that should perhaps be linked in the documents index that GHI maintains, or in some other fashion that makes it stand out in value as a reference document. :slight_smile:

7

@ Gene - I am glad that was helpful.

To answer your question on getting to know the GC, every opportunity I get I study the source code of the framework. When MS originally released ‘ROTOR’ or SSCLI ‘Shared Source Common Language Infrastructure’ I studied that code till it made some limited sense to me. To a lesser extent I have tried to do the same with the .NETMF implementation.

I always start with the GC code because I think it is the module that gives the most coverage of the code everything about the framework and type system ties into the GC.

8

@ taylorza - I guess I’ll have to bite the bullet and take a look at the source code. And BTW, thanks again for your byte queue. I made it into a byte[] queue and it is a crucial part of the architecture for my current project.

Cheers - Gene

9

Since this thread was requested to become a kind of reference for the GC, I want to ask what exactly is the thinking behind why forcing GC is a bad idea. Is it mainly due to the unknown amount of time needed to execute it or is it something deeper such as the OS knows “better” when to execute it. I realize the NETMF is different from Desktop applications but I want to understand the rational behind it.

The reason I ask is that when using CP7 and Glide, unless I force a GC after disposing of a window, my code will generate an out of memory exception.

I see the value in minimizing the GC to limit CPU use, but in this case, I cant see any other way of having my code run correctly unless I switch to Tinkr or something like it, that does not use so much memory.

Any thoughts on this would be appreciated.

10

You don’t mention what mainboard you’re using. If you’re hitting OOM exceptions on any of the GHI Premium boards (EMX, G120, G400 based - or even a Hydra for that matter) then you’re doing something wrong… those boards have shedloads of memory so it would be hard to do that. If you’re hitting that on an STM32F4 board (Cerb family) then you’re lucky you’ve got that far :slight_smile: The resolution on the CP7 is a challenge for such a memory constrained device.

There’s really no reason to be afraid of GC, unless you have timing critical code that you’re trying to avoid disrupting for an unknown length of time. Typically a UI based app is not going to be timing critical, so just forcing GC isn’t an issue.

11

@ khalilm - In my response to @ Gene I stated that forcing a GC is not an issue for .NETMF like it is on the Desktop.

So there might still be a question as to why it is a bad idea to force a GC on the desktop. The key reason in my mind is that it messes with the GC statistics and that in turn limits the frameworks ability to tune the GC for the runtime characteristics of your application. When the CLR is spun up into a process, it starts out with a fixed thresholds for the Gen 0, Gen 1 memory, these startup thresholds might be ideal for one application and totally off for another.

Starting with Framework 2.0, the GC is able to resize the generations based on the application memory utilization characteristics, for example if the GC runs frequently and most objects are surviving the Gen 0 collection and being promoted, then the framework could increase the Gen 0 threshold so that the GC runs less often since there is a high likelihood that the objects are not going to be collected anyway. On the desktop it is much better to let the framework adjust itself to run optimally based on your application characteristics.

There is one exception, I will always force a GC and ensure that the finalizes have run if I am writing code that does some micro benchmarking, but that is a specialized case and does not fall into the realm of everyday server or desktop application.

NB: The above is just an example and is not necessarily how the current GC actually works, I do not have access to the code for the desktop version so most of the info I have is either from the investigation done by people much smarter than me, or from tests that I have carried out and has led me to some simplistic conclusions.

12

For those that are interested, I did a quick google and found a really nice MSDN article that goes into much more detail on the desktop GC. This of course does not apply to .NETMF, but having the depth of understanding makes the .NETMF GC much easier to understand.

And here is a quote that supports my example from earlier.

13

@ brett Thanks for your reply. I am using the Spider/EMX board and while I am quite possibly doing something wrong, I am unsure as how to fix it.

From what I read in the Glide forums, Glide keeps a bitmap for each windows.
https://www.ghielectronics.com/community/forum/topic?id=14411

Since I dont release my main window, I swap in and out the other windows disposing them as I go.

However, even disposing them does not seem to force the GC in time before it generates an OOM exception. Thus my forcing GC allows my program to continue.

I have optimized the graphics. So I dont see what else can be done to prevent this problem as I am using string resources to load the graphics using Glide. I can post some code if that will help but if you think this is not correct please let me know.

@ taylorza I did read that caveat in your reply and was just looking for more input into why. And boy did you give me details…way beyond my current knowledge but it gives me a good starting point for understanding it in more detail. I did do some reading on the desktop GC and saw everyone was saying it was a bad idea but without saying exactly why.

Thanks for the link, I will spend some time to read it.

14

That is only because you guys are discussing one aspect of .NET and .NETMF that interests me tremendously… My drug of choice :slight_smile:

15

Are you really getting OOMs or are you just seeing issues with contiguous blocks not being available?

How many screens do you have?

Perhaps you’re hitting the ~700k limit (Taylorza will step in and use the right terminology here :slight_smile: ) that the default stack? allocation is.

16

The ~700Kb limit is a limit for a single object allocated on the heap, however Bitmaps are special cased, if the Bitmap exceeds a set threshold it is automatically allocated from the custom heap which only limited by the amount of memory assigned to the custom heap. This is the same heap that the LargeBuffer uses for example.

I took a quick peek at the NETMF Bitmap code, and I stand to be corrected here since I have only taken a cursory look at the code, but it seems that calling Dispose on the Bitmap does not actually free the underlying memory that is allocated, it only marks it available to be collected if/when the GC runs. This is not typical of what would happen on the Desktop framework where the unmanaged memory of a Bitmap would be reclaimed deterministically when calling Dispose. So, if I am correct in my reading of the code (I will take a more detailed look this weekend) it might just be good to force a GC after disposing bitmaps.

17

Well that is good news for the rest of us who can learn from all that you find out. :slight_smile:

@ brett My hardware is currently be used in a demo so I cannot give a screenshot of the error but yes it was a OOM exception. I will it once I get it back.

When I first developed my code, I was running it on the TE35. Using that screen, I could allocate all windows in memory on startup and call them as needed. This resulted in a very quick interface especially with the screen animations.

However, when I tried this same procedure using the CP7, nothing worked. After running the debugger, I saw the OOM exception and did some reading through the forums as I mentioned before. So I switched the code to keep my one main window in memory and call the others as needed. In this particular case I am just loading a 17KB jpg image to demonstrate the resolution of the screen. If I dont call GC after I dispose of this window the OOM is generated. So to answer you question, I load only 2 windows at a time.

I did see the documentation on the allocating heap but then saw that it was changed in 4.2 and is no longer an API option???

However whether this is a stack issue or contiguous block issue, I cannot answer. How can I determine that?

Also if it is a stack issue as @ taylorza suggested, then is it possible or necessary to change the size of the custom heap?

Again thank you both for the input. Coming from the desktop it has been a long time since the Windows/DOS days where you needed to worry about memory and the 64K memory limits. I just need to remind myself of this constraint as I code in NETMF. But I bought the extra FLASH memory just in case :slight_smile:

18

You can get two types of error in low memory conditions (not that it actually makes a difference here, but in other situations it may). The first message can be a failure to find contiguous blocks but it manages to clean up and you still continue on - the error is something like “failed to allocate X blocks” (does the GC just get invoked here to do the clean up? I suspect so); then you can get situations where there isn’t anything that’s pending cleanup and you can’t allocate the required memory.

Certainly the size of the CP7 screen is the core reason you’re seeing these issues in your situation - didn’t happen when the images were only T35 sized. I do wonder if you might be better off using real bitmaps instead of JPGs so you don’t store them twice, or whether the way you’re using them is actually a problem. Are these static files and loaded in as resources? Perhaps if you can show us code someone (taylorza :stuck_out_tongue: ) might be able to suggest optimisations.

19

I am (almost) 100% sure it was an OOM exception, as that is what started me to look into modifying my code.

The JPEGs themselves are on an SD-Card and loaded into memory when the window needs to be created.

I did upload the code due to another issue in this post
https://www.ghielectronics.com/community/forum/topic?id=15213

So that same code can be used here but it is a slimmed down and modified version as it does no call the GC and does not have the code for all the other windows. Later too I modified it to just allocate two Windows instead of the 8 in display.cs

https://drive.google.com/file/d/0B33xtdsvpTTSM1lmZlcydVRyVW8/edit?usp=sharing

But the idea of how I load the bitmaps is there. I basically followed these two examples.

https://netmfglide.codeplex.com/SourceControl/latest#Glide/Examples/DisplayCP7TouchExample.xml

https://netmfglide.codeplex.com/SourceControl/latest#Glide/Examples/ImageExample.xml

I am curious though what you mean by “I do wonder if you might be better off using real bitmaps instead of JPGs so you don’t store them twice”. I dont really follow that.

Thanks,
Khalil

20

Actually, I explained that the limit is a heap allocation limit. In .NETMF it is unlikely that this is a stack allocation issue since almost all data types are allocated on the heap. A stack issue would most likely result in a stack overflow, which would probably present as the board dumping the register values to the display, rather than an out of memory exception.