Okay, so as I said I’m going to put down more information on my Memory Management problem.
- Take into a byte array a serialized version of some data
- Deserialize the array and reconstruct the fleshed out data structure
- Use the data structure in processing future events
When the Serialized Data is > ~2kB, then the reconstruction of the data structure will result in OutOfMemoryExceptions, even though there is a reported 20kB of free memory AFTER the serialized array is already established in memory.
This is the way I will state the problem. I understand there are other ways to explore in order to reach my actual goal as prescribed by my job, but I feel a strong need to understand what is happening in my current implementation. Why does deserializing this data cost so much memory?
I will give:
- A diagram of the data structure.
- An example of what a fleshed data structure might look like at the end of the process, replacing the notion of arrays with each element in the array being a child of the object which holds a reference to the array.
- Code representing my implementation
The data structure consists of 7 classes. A class may contain primitives, an array of primitives, a reference to an object of another class, or a reference to array of objects of another class.
Below is a diagram of the data structure.
Below is an example of how the structure might flesh out at runtime:
Finally, a code example of how the structure is deserialized and reconstructed is below.
// Assume that RawData is a filled byte array containing the serialized data
// reference to the top level object
private static A objectA;
// Calling Code
objectA = CreateStructure();
// Deserializes RawData and returns reference to the top
// level of the data structure
private static A CreateStructure()
A returnObj = new A();
ushort NumberOf_B_Objects = GetUShortFromRawData();
returnObj.ArrayOf_B_Objects = new B[NumberOf_B_Objects];
for ( int i = 0; i < returnObj.ArrayOf_B_Objects.Length; i++ )
returnObj.ArrayOf_B_Objects[ i ] = Create_B_Object();
ushort NumberOf_G_Objects = GetUShortFromRawData();
returnObj.ArrayOf_G_Objects = new G[NumberOf_G_Objects];
for ( int i = 0; i < returnObj.ArrayOf_G_Objects.Length; i++ )
returnObj.ArrayOf_G_Objects[ i ] = Create_G_Object();
// Create and return a B object
private static B Create_B_Object()
B returnObj = new B();
// example of one primitive datum
returnObj.data = Get_B_Data_FromRawData();
ushort NumberOf_C_Objects = GetUShortFromRawData();
returnObj.ArrayOf_C_Objects = new C[NumberOf_C_Objects];
for( int i = 0; i < returnObj.ArrayOf_C_Objects.Length; i++ )
returnObj.ArrayOf_C_Objects[ i ] = Create_C_Object();
// Create and return a C Object
private static C Create_C_Object()
C returnObj = new C();
returnObj.Data = GetDataFromRawData();
bool Has_D_Obj = GetBoolFromRawData();
if( Has_D_Obj )
returnObj.RefTo_D_Obj = Create_D_Object();
I will stop there. I hope I have made it clear that the remaining functions will be very similar to the previous functions.
I would greatly appreciate anyone who took the time to understand this and provide insight into how I should allocate memory in another manner, or how I could see that the structure is just truly much larger than the serialized data.