Main Site Documentation

BINARY_BLOB_HEAD eats up memory


#1

Type 0F (STRING ): 456 bytes
Type 11 (CLASS ): 2232 bytes
Type 12 (VALUETYPE ): 84 bytes
Type 13 (SZARRAY ): 1404 bytes
Type 15 (FREEBLOCK ): 1188 bytes
Type 17 (ASSEMBLY ): 13056 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 1B (DELEGATE_HEAD ): 432 bytes
Type 1C (DELEGATELIST_HEAD ): 48 bytes
Type 1D (OBJECT_TO_EVENT ): 120 bytes
Type 1E (BINARY_BLOB_HEAD ): 41316 bytes
Type 1F (THREAD ): 1152 bytes
Type 20 (SUBTHREAD ): 144 bytes
Type 21 (STACK_FRAME ): 1188 bytes
Type 27 (FINALIZER_HEAD ): 192 bytes
Type 31 (IO_PORT ): 180 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1068 bytes
Failed allocation for 10 blocks, 120 bytes

I’m back with a problem again. :slight_smile: It seems this code eats up all Fez’s memory.
This code is for writing files from the computer to the USB flash drive in the Domino. Can somebody tell what needs to be optimized.

static void receivefile()
        {
            string rootDirectory = VolumeInfo.GetVolumes()[0].RootDirectory;
            string originalFile = rootDirectory + @ "\received.txt";
            FileStream fs3 = new FileStream(originalFile, FileMode.Append, FileAccess.Write);
            byte[] size = new byte[20];
            UART.Read(size, 0, 20); //Get the number of bytes that the computer will send. (Ex. "2048                ")
            System.Text.Encoding enc1 = System.Text.Encoding.UTF8;
            string mysize = new string(enc1.GetChars(size));
            int newsize = System.Convert.ToInt32(mysize.Trim()); //removes trailing spaces after the file size.
            Debug.Print(newsize.ToString());
            int ctr=0;
            int hold = 500;
            byte[] filesize = new byte[hold];
            if (newsize > hold)
            {
                while (ctr < (newsize / hold))
                {
                    
                    UART.Read(filesize, 0, hold);
                    fs3.Write(filesize, 0, hold);
                }
                ctr++;
                if (ctr == ((newsize / hold) - 1))
                {
                       
                    UART.Read(filesize, 0, (newsize % hold));
                    UART.Write(filesize, 0, (newsize % hold));
                }
            }
            if (newsize < hold)
            {
                      
                UART.Read(filesize, 0, newsize);    
            }

            Debug.Print("done");
            Thread.Sleep(3000);
            fs3.Close();
        }


#2

Do you get out of memory exceptions? When you exit this method the memory level should go back to where it was, right?

You can use Debug.GC(true) to force the GC to compact the heap


#3

I get no errors. I force stop the program because that GC output seems it will go on forever. It just repeats.

EDIT:

    #### Exception System.OutOfMemoryException - CLR_E_OUT_OF_MEMORY (3) ####
    #### System.IO.FileStream::Write [IP: 001d] ####
    #### MFConsoleApplication1.Program::receivefile [IP: 008b] ####
    #### MFConsoleApplication1.Program::DataReceived [IP: 0050] ####
    #### System.IO.Ports.SerialPort::DataEventHandler [IP: 0012] ####

This shows up after the binary_blob_head reaches it’s limit.

And this pops up after waiting for the repeating GC output to end.

An unhandled exception of type ‘System.OutOfMemoryException’ occurred in System.IO.dll


#4

Why do you read the whole file in one run instead of reading chunks? Also your function doesn’t loop so where to you see the multiple gc outputs?


#5

I’m not sure if I understood this right. The domino receives 250 bytes every 1 second so every 2 seconds the 500 byte array of the Domino will be fiiled and it will write it now to the file in the USB.

A VB program is used to send those 250 bytes every second from a file( placed a 1 second delay so that the domino will not be overwhelmed).

The function runs only once but there is a while loop inside that waits for its 500 byte array to be filled then writes it to the file, it will loop until the file size that the VB prog sent earlier is achieved.

            if (newsize > hold) //newsize = filesize || hold = 500
            {
                while (ctr < (newsize / hold)) //ctr=0
                {
                   
                    UART.Read(filesize, 0, hold);
                    fs3.Write(filesize, 0, hold);
                
                    ctr++;
                }
               
                if (ctr == ((newsize / hold) - 1))
                {
                       
                    UART.Read(filesize, 0, (newsize % hold));
                    fs3.Write(filesize, 0, (newsize % hold));
                }
            }

#6

Ok I see it better now. What you are doing should be fine.


#7

But I still don’t know what part eats up Fez’s memory.

I’m suspecting that .NETMF is storing those received data in a separated buffer. :frowning:

Because my sendfile() function that uses the same technique to send files to the computer is working fine even if I used 8000bytes buffer compared to the 500bytes buffer of the receive function.

Here is the sendfile function.

   int xx = 0, bytes = 8000;
            if (fs1.Length > bytes)
            {
                while (xx < (fs1.Length / bytes))
                {
                    byte[] ByteArray = new byte[bytes];
                    fs1.Read(ByteArray, 0, bytes);
                    UART.Write(ByteArray, 0, bytes);
                    xx++;
                    if (xx == ((fs1.Length / bytes) - 1))
                    {
                        fs1.Read(ByteArray, 0, ((int)fs1.Length % bytes));
                        UART.Write(ByteArray, 0, ((int)fs1.Length % bytes));
                    }
                }
            }

I’ve tested this code and managed to send a single 47MB .exe file via bluetooth connection. I’d tried to send a 700MB .iso file but my Bluetooth module burned out before it was finished (managed to send 400mb+ in 7hrs before in burned :frowning: ).

Now I’m just using a serial level converter.


#8

Are you sure there is no other threads or some other code producing this? Because your code shouldn’t use all memory…


#9

You might want to add a small sleep?
I have noticed, with my hexapod, that sending a lot of data over serial eats up memory. Adding a small sleep fixed this (example: Thread.Sleep(10); )


#10

The sleeps gives time got GC to clean up the heap.

My first respond was to try to add Debug.GC(true) to clean and compact the heap


#11

It only happens when I run that function so it was the first thing I suspected. I will try to isolate that function and create another program for it and try again.

[quote]The sleeps gives time got GC to clean up the heap.

My first respond was to try to add Debug.GC(true) to clean and compact the heap[/quote]

Ok I’ll try this also. I used Sleep on my vb program that sends the bytes I didn’t add it to the Fez because I tried to reduce codes as much as possible to reduce a few byte and cleaner look. :stuck_out_tongue:

I probably have a time to try to report tomorrow evening.

Thank you all for your help. :slight_smile:


#12

Have you tried to make a program with only that function to eliminate interactions with other things happening in your program?


#13

I’ve found the problem.


// Hold a static reference in case the GC kicks in and disposes it automatically, note that we only
support one in this example!
Static SerialPort UART = new SerialPort("COM1", 115200);
static PersistentStorage ps;

:naughty:

As I understand from the comment above, static by-passes the GC and maybe the reason why the Binary_blob_head eats up all the memory. What I did is I put “SerialPort UART = new SerialPort(“COM1”, 115200);” in every function that needs it.

Now all I have to do is

  1. fix the corruption of files after sending (exactly the same byte size). - maybe a simple encoding or misplaced bytes problem.
  2. And also create a VB program that would get the folder and file directory from the domino and then create a GUI for it. - NO IDEA. :wall:

Problem solved


#14

You do not need to do this! Just add

static SerialPort UART = new SerialPort("COM1", 115200);

In your program namespace (above main) and you can access it from every method!


#15

[quote]You do not need to do this! Just add

static SerialPort UART = new SerialPort(“COM1”, 115200);

In your program namespace (above main) and you can access it from every method![/quote]

You didn’t read my WHOLE post. That code you posted is already in my code. It was causing problem because if I declare it as STATIC. The Garbage Collector would ignore it, causing the Binary_Blob_Head to consume all memory.


#16

Uhm. There is only one page and I did not see it? If I did something wrong/did not see something please do not mind.


#17

It was just above your post. :slight_smile:

Its ok, it always happens with my post. Maybe I posted to long plus the fact that I’m not a native English speaker/writer so maybe my grammar and choice of words are not that effective. I always hated English class at school. :wall:


#18

Ok sorry! I really did not see that!

It does not matter that your english is not perfect, neither is mine. :stuck_out_tongue: