SerialPort drops input or output bytes

I’m trying to receive serial data at a baudrate of 19200 (1 stopbit, no parity, RTS-CTS handshaking).
The problem is that the FEZ frequently drops one or more incomming bytes.

In short, this is what my code should does:
send a code to request a table (= 3 bytes)
receive the table byte by byte and verify sync and crc ccitt at the end.

In my first tryout I called SerialPort.Read for each byte. I thought this was the problem of the dropped bytes (maybe the FEZ couldn’t follow).

So I created a thread that always receives as much data as possible.

Some declarations:


private byte[] inputBuffer = new byte[1024];
private int inputBufferUsed = 0;
private int readTimeout = 5000;
private ManualResetEvent dataReceived = new ManualResetEvent(false);

This is the thread function:


private void SerialThreadProc()
{
    byte[] buffer = new byte[1];
    while (!terminate)
    {
        int inputBufferFree = inputBuffer.Length - inputBufferUsed;

        if (inputBufferFree > 0)
        {
            if (serialPort.Read(buffer, 0, 1) == 1)
            {
                lock (inputBuffer)
                {
                    inputBuffer[inputBufferUsed++] = buffer[0];
                    dataReceived.Set();
                }
            }
        }
        else
            Thread.Sleep(100);  // Buffer full
    }
}

This is how I receive a byte:


public bool WaitByte(out byte data)
{
    bool mustWait = false;
    data = 0;

    try
    {
        lock (inputBuffer)
        {
            if (inputBufferUsed > 0)
            {
                data = inputBuffer[0];
                inputBufferUsed--;
                Array.Copy(inputBuffer, 1, inputBuffer, 0, inputBufferUsed);
                return true;
            }
            else
            {
                dataReceived.Reset();
                mustWait = true;
            }
        }

        if (mustWait && dataReceived.WaitOne(readTimeout, false))
        {
            lock (inputBuffer)
            {
                data = inputBuffer[0];
                inputBufferUsed--;
                Array.Copy(inputBuffer, 1, inputBuffer, 0, inputBufferUsed);
                return true;
            }
        }
    }
    catch (Exception e)
    {
        client.OnError(Client.ErrorCause.SerialReadWriteError, e.Message);
    }
            
    return false;
}

What I have now is lost bytes but my inputBuffer never gets completly full, so:

  1. the ‘lost’ bytes never get in
  2. the serialport stops working

Is there something else I should try?

If you use RTS/CTS and you have pins connected properly and the other side of the serial cable is conforming to the CTS/RTS then you should never miss any data.

Subscribe to the error event and you will know if you are having any errors

Check your cable and your other side for CTS/RTS

I don’t get any error through the handler. Would I get an error if the internal buffer of the SerialPort ran out of space?

Because maybe the thread function also can’t follow because I use lock() to synchronise inputBuffer access.

Hmm that should be no problem either since, when there is no byte, the WaitByte function uses WaitOne with a timeout of 5000ms, so the serial thread function can run at full speed.

I measured RTS/CTS and it seems to be connected just fine. I also added my schematic.

Maybe your are losing data because of the transceiver you have and the switching on/off mechanism.

Or your other side is not running with RTS CTS properly. You can easily test this but using RTS as an output port and set the pin to stop data flow, then send data from the other side. If data is still moved then there is an error.

…or connect a scope to RX and RTS pins and check that it data flow is actually stopping when EMX asserts the RTS pin

Ok ,so I measured the RTS output of the FEZ and I see some weird behavior.

The FEZ is putting RTS high after the stopbit of EACH received byte for 24µS. Also I don’t expect that the FEZ puts its RTS high at the moment the buffer is full. The RTS must be put high at what they call a “high water” level (this level is lower then the total capacity of the internal buffer).

Can I review the native code of the serial port in the NETMF PK or did GHI modify that code?

When RTS is high, your side shouldn’t send any more data. and internally, your code should be emptying the received data continuously.

This is easy to test, run a simple program that reads serial in a loop and send all data you want to EMX, you will never miss any data.

There is something wrong with the handshaking used on the FEZ.

Those 24µs spikes on RTS are not a normal behavior. RTS must work with a high and low ‘water’ level. Is RTS/CTS implemented at hardware level or at driver level?

See attached image.
Yellow signal is measured on RXD, blue signal is measured on RTS.
Time markers are set on startbit end end-of-stopbit.

Write couple lines of code to open the port and then do nothing. Now on the other side, send data till RTS goes high. It must stay high

After you verified that on scope, change your code to read the uart and dump it when a button is pressed. Try last test and when RTS is high, press the button and observe the signals on scope

I just ran this test code:


using System;
using System.Threading;

using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;

using GHIElectronics.NETMF.FEZ;
using System.IO.Ports;

namespace HandshakeTest
{
    public class Program
    {
        public static void Main()
        {
            byte[] data = new byte[1];
            InputPort selectButton = new InputPort((Cpu.Pin)FEZ_Pin.Digital.ButtonSelect, true, Port.ResistorMode.PullUp);
            InputPort downButton = new InputPort((Cpu.Pin)FEZ_Pin.Digital.ButtonDown, true, Port.ResistorMode.PullUp);

            SerialPort port = new SerialPort("COM2", 19200, Parity.None, 8, StopBits.One);
            port.ReadTimeout = 5000;
            port.WriteTimeout = 5000;
            port.Handshake = Handshake.RequestToSend;
            port.Open();

            while (true)
            {
                // wait until select button pressed & released
                while (selectButton.Read()) Thread.Sleep(100);
                while (!selectButton.Read()) Thread.Sleep(100);
                    
                // ask for table
                Utility.Piezo(1000, 500);
                port.Write(new byte[] { 0, 0, 0 }, 0, 3);
                
                int avail = port.BytesToRead;

                // wait until select button pressed & released
                while (selectButton.Read())
                {
                    if (avail != port.BytesToRead)
                    {
                        avail = port.BytesToRead;
                        Debug.Print(avail.ToString());
                    }

                    // redownload table if down button pressed.
                    if (!downButton.Read())
                    {
                        Utility.Piezo(1000, 500);
                        port.Write(new byte[] { 0, 0, 0 }, 0, 3);
                        while (!downButton.Read()) Thread.Sleep(100);
                    }

                    Thread.Sleep(100);
                }
                while (!selectButton.Read()) Thread.Sleep(100);

                // start reading
                Utility.Piezo(2000, 250);
                while (port.Read(data, 0, 1) == 1) ;
                Utility.Piezo(4000, 250);
            }
        }
    }
}

The RTS line goes high for 24µs after each byte received, starting from the very first byte. -> This is a driver issue that should be solved by GHI.

After 12288 bytes the RTS goes high and stays high until I read out the serialport. While RTS is high, the black box stops sending immediately so this part is functioning correctly.

What I see on the scope is that the table I receive varies in length (while it always should be the of the same length). I think the blackbox has a problem with the RTS line going high every time. I’ll see if I can take a look at the source code of that device.

You must know that, when FEZ toggles the RTS line on each byte, this is generating interrupt requests at the side of the blackbox. So I almost beg GHI to solve this issue ;D

Be definition, this shouldn’t effect the flow control to lose bytes like you earlier stated. Maybe it is the interrupt in your code that is having problems so the fix on your side.

Now, that pulse shouldn’t be there and will take of this immediately.

Idd Gus, it should not affect the communication. And indeed, there is a bug on my side too in the modem status interrupt.

I’ll see if I can recompile the firmware of that device :slight_smile:

Thanks for your advise

I was able to recompile the firmware of my black box. There was an issue in the interrupt handler of the modem status register (transmission did not continue on CTS).

Communication with FEZ Cobra now works like a charm! ;D

But those pulses are still there right?

Yes they are. Thanks to those pulses I’ve found this 20yrs old bug!