UART buffer size

Which is the buffer size of Fez Cobra’s UART? I’m using the same code in a Net Framework 4 windows app and in a NETMF 4.1 app, and when I send lots of datas in a very short time, windows app is working properly, but in my Fez doesn’t it.

I know that the serial port buffer in Windows (and Mac OS) is limited in size to 10 Kbytes, but what about Fez Cobra?

Thanks!!

PD: I’ve been looking for it in EMX User Manual, and have not found anything.

4k

Thanks!!!

What is lots of data? Quantify.

Are you losing data?

Done correctly, the serial port on MF is very fast. If you are having trouble, pl;ease post you code,

I’m sending 25bytes frames, with 50ms between each other. If I wait 150ms between frames, all works ok.

Code is about 150 lines, if you give me your email, I could send you code.

Please isolate the serial port portion of your code. Show me the code that reads the frames only,.

You still have not told me what is not working correctly.

The problem is that I’m not receiving some frames. I also have to say that I check that all frames begin by a special byte and have the right checksum.

All this is in the same method so this is because the 150 lines.

In the past, when others have had difficulty with framed serial data, it has usually been a problem with how they were handling the data stream.

It should not take 150 lines to just read a frame. Refactor your code to do the frame reading in a standalone method, and post that method. It is good programming practice to encapsulate specific functionality instead of having a large monolith program.

While I’m refactoring code, I give you some more info. Sometimes, I get this exception:

[quote]Excepción del tipo ‘System.ArgumentOutOfRangeException’ en Microsoft.SPOT.Hardware.SerialPort.dll
#### Exception System.ArgumentOutOfRangeException - CLR_E_OUT_OF_RANGE (6) ####
#### Message:
#### System.IO.Ports.SerialPort::Read [IP: 0000] ####
#### EPI.Net.ZigBee.EpiZB20::ReceiveData [IP: 0080] ####[/quote]

Are you operating on raw bytes or maybe you are converting all you receive to string ?

@ Manu: can you post the code that processes the incomming data?

Here it goes:

        private void ReceiveData()
        {
            try
            {
                int bytesToRead = _serialPort.BytesToRead;

                while (!_stopThd)
                {
                    if (bytesToRead == 0)
                    {
                        ;// Thread.Sleep(20);
                    }
                    else
                    {
                        byte[] bytes = new byte[1024];

                        if (_serialPort == null || !_serialPort.IsOpen)
                        {
                            if (_serialPort == null)
                            {
                                _serialPort = new SerialPort(_port, _baudRate);

                                _serialPort.Open();

                                bytesToRead = _serialPort.BytesToRead;
                                _readBuffer.SetLength(0);
                                continue;
                            }
                        }

                        try
                        {
                            int bytesRead = _serialPort.Read(bytes, 0, bytesToRead);

                            for (int i = 0; i < bytesRead; i++)
                            {
                                _readBuffer.WriteByte(bytes[i]);
                            }

                            bool startOK = false;
                            bool lengthOk = false;
                            bool checksumOk = false;

                            do
                            {
                                _readBuffer.Position = 0;

                                // startOK should be always true if there is at least on byte
                                startOK = ((byte)_readBuffer.ReadByte() == EpiZBResponse.PACKET_STARTBYTE);

                                if (!startOK)
                                {
                                    bytes = _readBuffer.ToArray();
                                    _readBuffer = new MemoryStream();

                                    startOK = false;
                                    for (int i = 0; i < bytes.Length; i++)
                                    {
                                        if (!startOK && bytes[i] != EpiZBResponse.PACKET_STARTBYTE)
                                            continue;

                                        startOK = true;
                                        _readBuffer.Write(bytes, i, bytes.Length - i);
                                        _readBuffer.Position = 0;
                                    }
                                }                                

                                int l = GetLength();
                                
                                lengthOk = (l > -1);
                                if (lengthOk) checksumOk = CheckSum(l);

                                bytes = _readBuffer.ToArray();
                                _readBuffer.SetLength(0);

                                ByteReader br = new ByteReader(bytes, ByteOrder.BigEndian);

 #if(DEBUG && !MF && !WindowsCE)
                                Debug.WriteLine(String.Format("<<({0})\t{1}", bytes.Length, ByteUtil.PrintBytes(bytes, false)));
 #endif
                                if (startOK && lengthOk && checksumOk)
                                {
                                    byte startByte = br.ReadByte();     // start byte
                                    short length = br.ReadByte();

                                    Stopwatch stopwatch = Stopwatch.StartNew();
                                    CheckFrame(length, br);                                    

                                    if (br.AvailableBytes > 1)
                                    {
                                        br.ReadByte();  // checksum of current API message

                                        // ok, there are more bytes for an additional frame packet
                                        byte[] available = br.GetAvailableBytes();
                                        _readBuffer.Write(available, 0, available.Length);
                                    }
                                }
                                else if (startOK && lengthOk && !checksumOk)
                                {
 #if(DEBUG && !MF && !WindowsCE)
                                    Debug.WriteLine("<< ++ CheckSum ERROR!");
 #endif
                                    Debug.Print("CheckSum ERROR!");
                                    startOK = false;
                                    for (int i = 1; i < bytes.Length; i++)
                                    {
                                        if (bytes[i] != EpiZBResponse.PACKET_STARTBYTE)
                                            continue;

                                        startOK = true;
                                        _readBuffer.Write(bytes, i, bytes.Length - i);
                                        _readBuffer.Position = 0;
                                    }
                                }
                                else
                                {
                                    _readBuffer.Write(bytes, 0, bytes.Length);
                                }
                            }
                            while (startOK & lengthOk & (_readBuffer.Length > 4));
                        }
                        catch (Exception ex)
                        {
                            _readBuffer.SetLength(0);

                            if (_serialPort != null && _serialPort.IsOpen)
                            {
                                bytesToRead = _serialPort.BytesToRead;
                            }
                        }
                    }

                    if (_serialPort != null && _serialPort.IsOpen)
                        bytesToRead = _serialPort.BytesToRead;
                }
            }
 #if(!MF)
            catch (ThreadAbortException ex)
            {
 #if(DEBUG && !MF && !WindowsCE)
                // Display a message to the console.
                Debug.WriteLine("{0} : DisplayMessage thread terminating - {1}",
                    DateTime.Now.ToString("HH:mm:ss.ffff"),
                    (string)ex.ExceptionState);
 #endif
                throw new Exception(ex.Message, ex);
            }
 #else
            catch (Exception ex)
            {
                throw new Exception(ex.Message, ex);
            }
 #endif
        }

Sorry. Double post!

Wow :slight_smile:

That loop is constantly newing streams and arrays. Seems like you make it more complex then needed.
You have a description of that protocol somewhere? Because it’s hard to figure out from the code…

I have made some generic example in notepad, so it will not compile but it gives you a total different view on how you could handle it more efficient. I didn’t implement the ProcessPacket method called by the ReceivedData thread proc, I’ll leave that up to you.

In the example code here I detect and capture packets without newing anything. Also, open the serial port before the loop and close it after the loop. Don’t do this in the receive loop that needs to be as tight as possible.


private void ReceiveData()
{
	byte[] buffer = new byte[1024];	// Make sure this buffer is large enough to hold a full packet
	int bytesInBuffer = 0;
	int startOfPacket = -1;
	
	_serialPort = new SerialPort(_port, _baudRate);
    _serialPort.Open();
    
    while (!_stopThd)
    {
    	while (_serialPort.BytesToRead > 0)
    	{
    		int bytesRead = _serialPort.Read(buffer, bytesInBuffer, buffer.Length - bytesInBuffer);
  		
    		if (bytesRead <= 0)
    			break;

			// Search PACKET_STARTBYTE in newly received data
			if (startOfPacket == -1)
				startOfPacket = Array.IndexOf(buffer, EpiZBResponse.PACKET_STARTBYTE, bytesInBuffer);

			// Update number of bytes in buffer
    		bytesInBuffer += bytesRead;

			// Throw away data before PACKET_STARTBYTE
    		if (startOfPacket > 0)
    		{
    			bytesInBuffer -= startOfPacket;
    			if (bytesInBuffer > 0)
    				Array.Copy(buffer, startOfPacket, buffer, 0, bytesInBuffer);
    		}
    		
    		if ((startOfPacket == 0) && (bytesInBuffer >= ACCEPTABLE))
    		{
    			int bytesConsumed;
    			ProcessPacket(buffer, bytesInBuffer, out bytesConsumed);
    			
    			// Throw away consumed bytes by ProcessPacket
    			bytesInBuffer -= bytesConsumed;
    			if (bytesInBuffer > 0)
    			{
    				Array.Copy(buffer, startOfPacket, buffer, 0, bytesInBuffer);
    				startOfPacket = Array.IndexOf(buffer, EpiZBResponse.PACKET_STARTBYTE, bytesInBuffer);
    			}
    			else
    				startOfPacket = -1;
    		}
    	}
    	
    	Thread.Sleep(20);
    }
    
    _serialPort.Close();
}

I have also refactored your code, now you can analizy my sample and Wouters. I also don’t create any new buffers, just operate on one byte array. My sample should work if you add your protocol logic.

using System;
using System.IO.Ports;
using Microsoft.SPOT;

namespace FEZ_Domino_Application1
{
    public class Program
    {
        private static SerialPort _serialPort;
        private static string _port;
        private static int _baudRate;
        private static int _bufferCount;
        private static byte[] _buffer;

        public static void Main()
        {
            _port = "COM1";
            _baudRate = 9600;
            _buffer = new byte[1024];
            _serialPort = new SerialPort(_port, _baudRate);
            _serialPort.Open();
            _serialPort.DataReceived += (sender, args) => ReceiveData();
        }

        private static class EpiZBResponse
        {
            public const byte PACKET_STARTBYTE = 0x01;
        }

        private static void ReceiveData()
        {
            try
            {
                while (_serialPort.BytesToRead > 0)
                    _bufferCount += _serialPort.Read(_buffer, _bufferCount, _serialPort.BytesToRead);

                var offset = -1;

                while (true)
                {
                    offset = Array.IndexOf(_buffer, EpiZBResponse.PACKET_STARTBYTE, offset + 1);

                    // no start byte
                    if (offset == -1)
                    {
                        _bufferCount = 0;
                        return;
                    }

                    var remainingCount = _bufferCount - offset;

                    if (!IsLengthOk(_buffer, offset, remainingCount))
                    {
                        Debug.Print("Not enough bytes, waiting for more");
                        return;
                    }

                    var length = GetLength(_buffer, offset);

                    Debug.Print(CheckSum(_buffer, offset)
                                    ? HexHelper.ToHex(_buffer, offset, length)
                                    : "Invalid checksum");

                    offset += length - 1;
                }
            }
            catch (Exception e)
            {
                Debug.Print("Exception while handling received data. " + e);
            }
        }

        private static bool IsLengthOk(byte[] data, int offset, int avilableBytes)
        {
            // length is ok if the value is less or equal to available byte avilableBytes
            return avilableBytes > 1 && GetLength(data,offset) <= avilableBytes;
        }

        private static int GetLength(byte[] data, int offset)
        {
            // i assume the second byte is the length of whole frame
            return data[offset + 1];
        }

        private static bool CheckSum(byte[] data, int offset)
        {
            // i assume the checksum is the last byte and value equals to 0xFF
            return data[offset + GetLength(data,offset) - 1] == 0xFF;
        }

        public static class HexHelper
        {
            const string Hex = "0123456789ABCDEF";

            public static string ToHex(byte[] data, int offset, int count)
            {
                var result = string.Empty;
                for (var i = 0; i < count; i++)
                    result += ToHex(data[offset + i]);
                return result;
            }

            public static string ToHex(byte b)
            {
                return new string(new[] { Hex[b >> 4], Hex[b & 0x0F] });
            }
        }
    }
}

@ Gralin: your buffer will overflow when serial data keeps comming. Those return statements will exit the thread instead of breaking the while (true) loop

@ Wouter: I don’t see it. The return keyword will break the loop the same way break does. Can you give me sample data set that this will now work for?

@ Gralin: the two return statements in the ReceiveData method will return from your ReceiveData method and not just break the while loop.

@ Wouter: and that’s exactly what i want them to do :wink: if there was more code to follow i would use break. How come can this cause overflow ?