SerialPort Event eats up memory

NETMF & Gadgeteer
1

Hi !
I have tested to receive 3 byte of serial data to a FEZ Domino in an Event and that eats up my memory, and it ends with program crash.

GC: 3msec 28104 bytes used, 36276 bytes available
GC: 3msec 33348 bytes used, 31032 bytes available
GC: 3msec 37392 bytes used, 26988 bytes available
GC: 3msec 37224 bytes used, 27156 bytes available
GC: 3msec 39552 bytes used, 24828 bytes available
GC: 3msec 48240 bytes used, 16140 bytes available
GC: 3msec 61548 bytes used, 2832 bytes available

Failed allocation for 17 blocks, 204 bytes

I have come to the conclusion that when i use serialPort.Read(RX_buffer, 0, 3); it reduce the memory, but when i use serialPort.BytetoRead i got 1 char at time and memory is not reducing.

Here is a code sample that i use

I have also seen that if i use different Thread.sleep times in the end of the main do…while loop
it changes the time it takes to eat up memory, the fates way is to use Timout.Infinite

Any one who have a clue ??


using System;
using System.Threading;
using System.IO;
using System.Text;
using System.IO.Ports;

using Microsoft.SPOT;
using Microsoft.SPOT.IO;

using Microsoft.SPOT.Hardware;

using GHIElectronics.NETMF.Hardware;
using GHIElectronics.NETMF.FEZ;
using GHIElectronics.NETMF.IO;

namespace FezDominoTest1
{


    public class Program
    {

        public static SerialPort serialPort;

        static OutputPort LED;

        static byte[] RX_buffer = new byte[50];



        static void RunMyTimer(object o)
        {
            Debug.Print("Timer evnt");
            LED.Write(!LED.Read());
        }

        static void serialPort_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {

            int count = serialPort.Read(RX_buffer, 0, 3);


            char[] chars = Encoding.UTF8.GetChars(RX_buffer);
            string str = new string(chars, 0, count);

            Debug.Print(str);

            //  RX_buffer = null;

        }

        public static void Main()
        {

            serialPort = new SerialPort("COM2", 9600, Parity.None);
            serialPort.Open();
            serialPort.DataReceived += new   SerialDataReceivedEventHandler(serialPort_DataReceived);
                      

            LED = new OutputPort((Cpu.Pin)FEZ_Pin.Digital.LED, true);

            Timer MyTimer = new Timer(new TimerCallback(RunMyTimer), null, 0, 5000);




            //do
            //{
                 Thread.Sleep(Timeout.Infinite);

            //}
            //while (true);

        }

    }
}
2

You are assuming that when you get the data ready event that there are only three chars in the internal buffer. This may not be the case. You should use the data ready method to determine how many characters are ready to read. I suspect there are more than 3 chars ready when you get the event.

Search the forums for discussion on serialport usage. There have been lots of threads on the best way to handle serialports.

3

I have a serial terminal program that i can send bytes with automatic and i only send 3 bytes
and that is what i receive i.e Debug.Print out

If a change to serialPort.Read(RX_buffer, 0, 1) “Read 1 byte” and send the same 3 bytes
Debug.Print out every 1 bytes of the 3 sent, and the memory is not reducing.

How is Read actually work, is it waiting for 3 bytes until timeout, but if no timeout is set.

A workaround could be to receive 1 byte at a time and then store them in another array.

//

4

I found that if i add this code it works fine without memory loss.


            int count = serialPort.BytesToRead;

            serialPort.Read(RX_buffer, 0, count);

But still not sure why i can´t use 3 byte reading if i only send 3 byte at a time, maybe the chip cant handle the amount of data and the serial buffer is filling up…

5

Using BytesToRead is the correct way to handle the event.

The chip can handle 9600 without any problem.

While you write 3 bytes of data at the PC end, you can not assume that three will arrive in a data ready event. Print out the value of count, and see what is coming in.

6

The right way to handle serial port is by having a separate thread that always reads all data available in UART.

You are only reading some data in the event so there is some left in the internal FIFO…few seconds and you are out of memory

7

Hello again.

I have tested to print out how many chars there are to read when the Event occurs, using the
serialPort.BytesToRead and that always give me 1 even if i send 3 bytes, if i always read 1 byte
when Event occurs the memory is not reducing, but if i set the number to 2 or more and send 2 or more one time , GC shows reduce free memory for every 3 bytes i send.

A question is, does BytesToRead relay tell how many bytes there are in serial buffer ?? why does it always give me 1 when sending more chars, even if i don´t do any Read the ByteToRead should increase for every event it receivs bytes.
Maybe a Microsoft .Net bug ? or what’s eating memory then, not the serial buffer,
it can´t reduce 35000 bytes of memory when i only send say about 300 bytes?

If i use the normal serialPort.Read without Event and say i want to read 10 bytes i got all bytes without problem.

Any clue ?

8

[quote]Any clue ?
[/quote]

If the event says there is one byte available, then there is one byte in the internal buffer. At 9600 you are sending about one byte per millisecond. At the hardware level, that is a lot of time between bytes. At speeds higher than 9600 you will find there are more bytes available. Do a loopback test at a higher speed.

In a data ready event, you should only read the number of bytes that are available, process them, and get out as fast as possible.

Reading more bytes than are available, in the event, seems to make bad things happen.

Why? The answer is in the framework code, which you can get by downloading the MS porting kit for MF. I would never want to ready more bytes than are ready in an event, so research the questions

If you want do a read for a specific number of bytes, then start a thread and do the read in the thread. Don’t use the data ready event.

9

I think what is happening is buffering side effects and using event.
When you use event, SP has to cut off read after some internal time metric and post notification of bytes ready in that buffer at that time. And create another buffer internal to keep reading on. So if it posts 1, you have to read 1, not 2 - to pop that first buffer (it continues to read on another buffer). I don’t really want to disect this method deeper in MS code, as I feel it is not right method to use anyway.

In contrast, when you post a Read buffer first, it fills that buffer as fast as it can without all the buffer juggling as above. I recommend reading on another thread and post read buffers first. This is same situation in reading from sockets.

Based on experience, I note the following:

  1. Don’t use event.
  2. Don’t use BytesToRead
  3. Make your protocol message based and send Len bytes first if possible. So you always know how many bytes to read for the enitire message. |len|Data … |len|Data.
  4. Read on a seperate thread (or async) and post complete messages to a Queue or kick off message handler in yet another thread so as not to block the read Loop thread. In MF, read on seperate thread because no Async methods as in big .Net.
  5. If possible, always have a read buffer posted as large as possible.

Example read method. Method not return until x Read. Call this in a loop. Another method would be to force a fixed message size in your protocol and aways read that amount.

private void ReadLoop()
{ // On read thread.
  while(isRunning)
  {
    byte[] header = ReadBytes(4);
    int len = //parse Int32
    byte[] msg = ReadBytes(len);
    q.Enqueue(msg); // Post to blocking queue or another thread for work.
  }
}


private byte[] ReadBytes(int len)
{
    byte[] buf = new byte[len];
    int received = 0;
    while (received < len)
    {
        int toRead = len - received;
        int read = comPort.Read(buf, received, toRead);
        received += read;
    }
    return buf;
}

hth

10

Ok !

That sounds like it could be the memory reducing problem.

I will try to use enqueue method later, it seem to be a nice feature in C# to handle cued data.

Anyway i have rewrite my code so I only read 1 byte at every event and then I store it away in another array, that works OK with no memory loss.

//

11

You should not always read one byte in an event!

You should read the number of bytes that are ready to be read, if you don’t, you will lose memory.

12

Also, reading only 1 byte is not efficient. Are you pushing max memory now? If you not, what is the concern of doing other way? Should be no more memory then buffer size and handful of bytes for the thread.

13

[quote]You should not always read one byte in an event!

You should read the number of bytes that are ready to be read, if you don’t, you will lose memory.
[/quote]

But as i said the BytesToRead after “event” is always 1, even if i send 2 or more bytes, so
i read that byte and store it in another array. Works fine.

14

[quote]But as i said the BytesToRead after “event” is always 1, even if i send 2 or more bytes, so
i read that byte and store it in another array. Works fine.[/quote]

It will work until it does not. What happens if garbage collection occurs and delays an event long enough for two chars to be in the internal buffer?

15

You should have decent size buffer, let us say 200 bytes. Then in the event, read 200 bytes. You do not care if UART has 1 or 100 bytes, you are basically reading the whole UART buffer. Do not forget to set the read timeout to 0.

I said 200 as an example, it depends on the baud-rate you are using and on how busy your system is. You need to experiment with it to find a good buffer size.

See this http://www.fezzer.com/project/47/gps-extension/