GC dump and OutOfMemoryException

Matt5 · January 6, 2014, 2:50pm

I’m wondering what the GC dump reveals when I’m experiencing an OutOfMemoryException (Panda II SDK4.1).

I’ve got some problems that i’ll be chasing that causes the error, but, meanwhile, the GC dump seems to say I’ve got 20k memory available but get the exception when attempting to alloc 780 bytes. What is really going on?

Dump (preceded by 3 Debug.Print lines showing my arraylist is getting larger and larger with each call to Enqueque()):

Enqueque Del declared by CycleTimedActuator priority 4 Count 40
Enqueque Del declared by CycleTimedActuator priority 4 Count 41
Enqueque Del declared by CycleTimedActuator priority 4 Count 42
GC: 5msec 43908 bytes used, 20472 bytes available
Type 0F (STRING ): 660 bytes
Type 11 (CLASS ): 13200 bytes
Type 12 (VALUETYPE ): 1992 bytes
Type 13 (SZARRAY ): 6000 bytes
Type 15 (FREEBLOCK ): 20472 bytes
Type 17 (ASSEMBLY ): 13464 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 19 (REFLECTION ): 24 bytes
Type 1B (DELEGATE_HEAD ): 1728 bytes
Type 1D (OBJECT_TO_EVENT ): 288 bytes
Type 1E (BINARY_BLOB_HEAD ): 384 bytes
Type 1F (THREAD ): 768 bytes
Type 20 (SUBTHREAD ): 96 bytes
Type 21 (STACK_FRAME ): 2856 bytes
Type 22 (TIMER_HEAD ): 360 bytes
Type 23 (LOCK_HEAD ): 120 bytes
Type 24 (LOCK_OWNER_HEAD ): 48 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD): 48 bytes
Type 27 (FINALIZER_HEAD ): 264 bytes
Type 31 (IO_PORT ): 216 bytes
Type 33 (I2C_XACTION ): 48 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1224 bytes
Failed allocation for 65 blocks, 780 bytes

#### Exception System.OutOfMemoryException - CLR_E_OUT_OF_MEMORY (3) ####
#### Message: 
#### System.Collections.ArrayList::Insert [IP: 0000] ####
#### MFCore.PriorityQueue::Enqueue [IP: 0021] ####
#### MFCore.WorkItemQueue::Enqueue [IP: 0017] ####
#### MFCore.Timer::OnPeriod [IP: 0018] ####

A first chance exception of type ‘System.OutOfMemoryException’ occurred in mscorlib.dll
An unhandled exception of type ‘System.OutOfMemoryException’ occurred in mscorlib.dll

Brett · January 6, 2014, 2:54pm

The one thing it shows is that you’re failing to allocate 780 bytes when you have 20k bytes free. You should try to force compaction. A more useful approach would be to show the GC output multiple times in the lead up to this, so you can see what’s changing.

Matt5 · January 6, 2014, 3:39pm

Brett, so the problem is fragmented memory? A quick look for compaction techniques didn’t seem to reveal anything basic like GC.Compact(). How would this be done?

Following is are multiple dumps leading up to the exception. Nothing much seems to be changing other than my array list getting larger due to Dequeue() not being called after the first few moments.

Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Enqueque Del declared by CycleTimedActuator priority 4 Count 2
Dequeque Del declared by CycleTimedActuator priority 4 Count 1
Dequeque Del declared by CycleTimedActuator priority 4 Count 0
Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Dequeque Del declared by CycleTimedActuator priority 4 Count 0
Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Dequeque Del declared by CycleTimedActuator priority 4 Count 0
Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Dequeque Del declared by CycleTimedActuator priority 4 Count 0
Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Dequeque Del declared by CycleTimedActuator priority 4 Count 0
GC: 5msec 46740 bytes used, 17640 bytes available
Type 0F (STRING ): 732 bytes
Type 11 (CLASS ): 13200 bytes
Type 12 (VALUETYPE ): 1008 bytes
Type 13 (SZARRAY ): 4980 bytes
Type 15 (FREEBLOCK ): 17640 bytes
Type 16 (CACHEDBLOCK ): 48 bytes
Type 17 (ASSEMBLY ): 13464 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 19 (REFLECTION ): 24 bytes
Type 1B (DELEGATE_HEAD ): 1728 bytes
Type 1D (OBJECT_TO_EVENT ): 288 bytes
Type 1E (BINARY_BLOB_HEAD ): 6000 bytes
Type 1F (THREAD ): 768 bytes
Type 20 (SUBTHREAD ): 96 bytes
Type 21 (STACK_FRAME ): 2040 bytes
Type 22 (TIMER_HEAD ): 360 bytes
Type 23 (LOCK_HEAD ): 60 bytes
Type 24 (LOCK_OWNER_HEAD ): 24 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD): 48 bytes
Type 27 (FINALIZER_HEAD ): 264 bytes
Type 31 (IO_PORT ): 216 bytes
Type 33 (I2C_XACTION ): 48 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1224 bytes
Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Dequeque Del declared by CycleTimedActuator priority 4 Count 0
Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Dequeque Del declared by CycleTimedActuator priority 4 Count 0
Enqueque Del declared by CycleTimedActuator priority 4 Count 1
Enqueque Del declared by Data priority 0 Count 2
Enqueque Del declared by Data priority 1 Count 3
Enqueque Del declared by Data priority 2 Count 4
Enqueque Del declared by Data priority 2 Count 5
Enqueque Del declared by Data priority 1 Count 6
Enqueque Del declared by Data priority 2 Count 7
Enqueque Del declared by Data priority 2 Count 8
Dequeque Del declared by CycleTimedActuator priority 4 Count 7
Dequeque Del declared by Data priority 2 Count 6
Enqueque Del declared by CycleTimedActuator priority 4 Count 7
Enqueque Del declared by CycleTimedActuator priority 4 Count 8
Enqueque Del declared by CycleTimedActuator priority 4 Count 9
GC: 5msec 43788 bytes used, 20592 bytes available
Type 0F (STRING ): 660 bytes
Type 11 (CLASS ): 13200 bytes
Type 12 (VALUETYPE ): 1176 bytes
Type 13 (SZARRAY ): 4956 bytes
Type 15 (FREEBLOCK ): 20592 bytes
Type 17 (ASSEMBLY ): 13464 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 19 (REFLECTION ): 24 bytes
Type 1B (DELEGATE_HEAD ): 1728 bytes
Type 1D (OBJECT_TO_EVENT ): 288 bytes
Type 1E (BINARY_BLOB_HEAD ): 2124 bytes
Type 1F (THREAD ): 768 bytes
Type 20 (SUBTHREAD ): 96 bytes
Type 21 (STACK_FRAME ): 2856 bytes
Type 22 (TIMER_HEAD ): 360 bytes
Type 23 (LOCK_HEAD ): 120 bytes
Type 24 (LOCK_OWNER_HEAD ): 48 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD): 48 bytes
Type 27 (FINALIZER_HEAD ): 264 bytes
Type 31 (IO_PORT ): 216 bytes
Type 33 (I2C_XACTION ): 48 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1224 bytes
Enqueque Del declared by CycleTimedActuator priority 4 Count 10
Enqueque Del declared by CycleTimedActuator priority 4 Count 11
Enqueque Del declared by CycleTimedActuator priority 4 Count 12
Enqueque Del declared by CycleTimedActuator priority 4 Count 13
Enqueque Del declared by CycleTimedActuator priority 4 Count 14
Enqueque Del declared by CycleTimedActuator priority 4 Count 15
Enqueque Del declared by CycleTimedActuator priority 4 Count 16
Enqueque Del declared by CycleTimedActuator priority 4 Count 17
Enqueque Del declared by Data priority 0 Count 18
Enqueque Del declared by Data priority 1 Count 19
Enqueque Del declared by Data priority 2 Count 20
Enqueque Del declared by Data priority 2 Count 21
Enqueque Del declared by Data priority 1 Count 22
Enqueque Del declared by Data priority 1 Count 23
Enqueque Del declared by Data priority 1 Count 24
Enqueque Del declared by Data priority 0 Count 25
Enqueque Del declared by Data priority 1 Count 26
Enqueque Del declared by Data priority 2 Count 27
Enqueque Del declared by Data priority 2 Count 28
GC: 5msec 42960 bytes used, 21420 bytes available
Type 0F (STRING ): 660 bytes
Type 11 (CLASS ): 13200 bytes
Type 12 (VALUETYPE ): 1632 bytes
Type 13 (SZARRAY ): 5412 bytes
Type 15 (FREEBLOCK ): 21420 bytes
Type 17 (ASSEMBLY ): 13464 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 19 (REFLECTION ): 24 bytes
Type 1B (DELEGATE_HEAD ): 1728 bytes
Type 1D (OBJECT_TO_EVENT ): 288 bytes
Type 1E (BINARY_BLOB_HEAD ): 384 bytes
Type 1F (THREAD ): 768 bytes
Type 20 (SUBTHREAD ): 96 bytes
Type 21 (STACK_FRAME ): 2856 bytes
Type 22 (TIMER_HEAD ): 360 bytes
Type 23 (LOCK_HEAD ): 120 bytes
Type 24 (LOCK_OWNER_HEAD ): 48 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD): 48 bytes
Type 27 (FINALIZER_HEAD ): 264 bytes
Type 31 (IO_PORT ): 216 bytes
Type 33 (I2C_XACTION ): 48 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1224 bytes
Enqueque Del declared by Data priority 1 Count 29
Enqueque Del declared by Data priority 2 Count 30
Enqueque Del declared by Data priority 2 Count 31
Enqueque Del declared by CycleTimedActuator priority 4 Count 32
Enqueque Del declared by CycleTimedActuator priority 4 Count 33
Enqueque Del declared by CycleTimedActuator priority 4 Count 34
Enqueque Del declared by CycleTimedActuator priority 4 Count 35
Enqueque Del declared by CycleTimedActuator priority 4 Count 36
Enqueque Del declared by CycleTimedActuator priority 4 Count 37
Enqueque Del declared by CycleTimedActuator priority 4 Count 38
Enqueque Del declared by CycleTimedActuator priority 4 Count 39
Enqueque Del declared by CycleTimedActuator priority 4 Count 40
Enqueque Del declared by CycleTimedActuator priority 4 Count 41
Enqueque Del declared by CycleTimedActuator priority 4 Count 42
GC: 5msec 43908 bytes used, 20472 bytes available
Type 0F (STRING ): 660 bytes
Type 11 (CLASS ): 13200 bytes
Type 12 (VALUETYPE ): 1992 bytes
Type 13 (SZARRAY ): 6000 bytes
Type 15 (FREEBLOCK ): 20472 bytes
Type 17 (ASSEMBLY ): 13464 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 19 (REFLECTION ): 24 bytes
Type 1B (DELEGATE_HEAD ): 1728 bytes
Type 1D (OBJECT_TO_EVENT ): 288 bytes
Type 1E (BINARY_BLOB_HEAD ): 384 bytes
Type 1F (THREAD ): 768 bytes
Type 20 (SUBTHREAD ): 96 bytes
Type 21 (STACK_FRAME ): 2856 bytes
Type 22 (TIMER_HEAD ): 360 bytes
Type 23 (LOCK_HEAD ): 120 bytes
Type 24 (LOCK_OWNER_HEAD ): 48 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD): 48 bytes
Type 27 (FINALIZER_HEAD ): 264 bytes
Type 31 (IO_PORT ): 216 bytes
Type 33 (I2C_XACTION ): 48 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1224 bytes
Failed allocation for 65 blocks, 780 bytes

#### Exception System.OutOfMemoryException - CLR_E_OUT_OF_MEMORY (3) ####
#### Message: 
#### System.Collections.ArrayList::Insert [IP: 0000] ####
#### MFCore.PriorityQueue::Enqueue [IP: 0021] ####
#### MFCore.WorkItemQueue::Enqueue [IP: 0017] ####
#### MFCore.Timer::OnPeriod [IP: 0018] ####

A first chance exception of type ‘System.OutOfMemoryException’ occurred in mscorlib.dll
An unhandled exception of type ‘System.OutOfMemoryException’ occurred in mscorlib.dll

Matt5 · January 6, 2014, 4:07pm

Looks like this would do the job. Will try it to see what happens.

“Run Debug.GC(true) to compact your heap”

Brett · January 6, 2014, 4:22pm

affirmative, that’s what you want to do in these situations. GC will do it, but when you’re churning fast (As you appear to) the regular GC process may be too slow to meet your needs

Matt5 · January 6, 2014, 5:04pm

Works like a hot damn. Thanks, Brett. (I should have been able to find ref to Debug.GC() but didn’t find it initially.) Now to fix what is causing the fragmentation…

Matt5 · January 7, 2014, 2:20am

The fragmentation is directly related to my OneWire code that I took from GHI’s SDK 4.1 onewire tutorial. The old code uses a Thread.Sleep(750) which I believe slows/blocks the thread executing the onewire read which, for some reason, leads to the fragging. See my post requesting help to remove the Thread.Sleep():

https://www.ghielectronics.com/community/forum/topic?id=14543

Matt5 · January 7, 2014, 3:15pm

When I replaced the Thread.Sleep(750) with a while loop polling the Dallas thermometer (as well as fixing a wiring problem), fragmentation stopped and my code that uses a priority queue to handle timer events (with user defined priorities and can optionally use a modification of R. Ostermeier’s threadpool) runs just fine with no memory alloc exceptions.