So, I recently added some nice looking code. I have a class with a variety of properties. This class represents all the values in a message that I send over the wire. So in this class I have a ToBytes() method that serializes all of my properties into a byte array. It’s much easier to read than the stuff I did in mIP where I was obsessed with memory usage and would reuse static arrays. No, this time I used a ArrayList to store individual bytes. The code looked something like this:
ToBytes()
{
Debug.Print("ToBytes. Free Mem at start: " + Debug.GC(false));
var header = new ArrayList();
header.AddRange(new byte[4]); // blank size
header.AddRange(Encoding.UTF8.GetBytes(this.DeviceID)); // 32 bytes
header.AddRange(DateTime.Now.ToBytes()); // Local DateTime -- 8 bytes
header.AddRange(((uint)this.Command.Length).ToBytes()); // Command Length -- 4 bytes
header.AddRange(Encoding.UTF8.GetBytes(this.Command)); // Command -- variable length
header.AddRange(((uint)ByteContent.Length).ToBytes());
header.AddRange(ByteContent);
var headerBytes = (byte[])header.ToArray(typeof(byte));
headerBytes.Overwrite(0, ((uint)headerBytes.Length).ToBytes()); // Write entire message size
Debug.Print("ToBytes. Free Mem at end: " + Debug.GC(false));
return headerBytes;
}
So after a day of running, I found my code had failed with an out of memory exception on one of the AddRange methods. Now, the AddRange method is just an extension method that loops over an ArrayList to add all the items, just like regular .NET. Here is that code:
public static void AddRange(this ArrayList theList, Array values)
{
foreach (var aValue in values) theList.Add(aValue);
}
So, my general rule of thumb is that when you have an out of memory exception, usually, your program will break on the memory hogging line of code… So, how bad is it? Well, I added a Debug.Print line at the beginning and end of the ToBytes() method to see and here is what the result was:
ToBytes. Free Mem at start: 51408
Failed allocation for 713 blocks, 8556 bytes
ToBytes. Free Mem at end: 28248
Free Memory: 48600
Whoa! A single call to this method, which constructs a 150 byte array, used 23,160 bytes of RAM! So what happened? Here is what I think is going on. First, when you use ArrayList, everything is stored in an object array inside the ArrayList class. Because of that alone, for each byte, ArrayList would now use 12 bytes because of the overhead in every object class. Then on top of that, I am essentially calling (through my AddRange method) the ArrayList.Add method for every byte. Internally, the ArrayList has to allocate a new object array everytime I outgrow the previous object array, creating potentially a multitude of object arrays… So, to avoid all that, I basically wrote a type specific version of ArrayList called ListOfByte which I will now share on CodeShare. https://www.ghielectronics.com/community/codeshare/entry/875
After this change the output was this:
ToBytes. Free Mem at start: 50628
ToBytes. Free Mem at end: 49524
Free Memory: 50136
Now, with ListOfByte I used only 1104 bytes, compared to the 23,160 I was consuming with ArrayList.
That makes the new method 20 TIMES MORE EFFICIENT with memory!
But, I still have all the conveniences of being able to add to the collection at will. So the code still looks clean, and it’s much more memory efficient.
Incidentally, the Failed allocation statement is a warning from the runtime telling you the memory had to be compacted to get enough space to continue. The Out of Memory exception came later. Also, the “Free Memory:” output was another call to Debug.GC(false) to see how much memory was free after the objects were disposed.
Happy coding!