Buffer Overrun Error

NETMF & Gadgeteer
1

I am having an issue where when using the ChipworkX with a Serial to Ethernet Converter and using the Ethernet controller. I am using a Lantronix UDS1100 Serial to Ethernet Device, if I run without using any sockets it will run days without any issue. Once I open a socket while using the serial to Ethernet device, eventually I will get a buffer overrun error on the serial port. I’ve tried multiple different baud rates 9600 to 230400, hasn’t made a difference.

I am using the latest firmware v4.1.8.0.

2

Are you emptying your incoming serial data correctly?

3

Yes, I believe that I am. I’ve never had any issues with buffer overflow.

4

Here is the code i am using.
I am taking the data from the buffer in the DataReceived Event of the serial port and puts the data in a blocking queue.
I have added a thread that then consumes the data from the blocking queue.


private void mSerialPort_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
      {
         try
         {
            int BytesToRead = mSerialPort.BytesToRead;
            if (BytesToRead > 0)
            {
               byte[] Buffer = new byte[BytesToRead];
               mSerialPort.Read(Buffer, 0, BytesToRead);
               mDataBuffer.Enqueue(ArrayEx.Clone(Buffer), 100);//base.OnDataRx(Buffer, BytesToRead);
            }
         }
         catch (Exception ex)
         {
            Debug.Print("Exception in ReaderCommunication.mSerialPort_DataReceived: " + ex.Message);                }
      }

      private void RxDataConsumerThread()
      {

         System.Threading.WaitHandle[] WaitHandleList = { mShutdownConsumer, mDataBuffer.WaitDequeue };
         int WaitIndex = -1;

         try
         {

            byte[] RxData;// = new byte[8912];
            bool IsDirty = false;

            mRunning = true;
            mShutdownConsumer.Reset();

            while (mRunning)
            {
               RxData = new byte[0];
               while (!mDataBuffer.IsEmpty)
               {
                  byte[] Buffer = (byte[])mDataBuffer.Dequeue(1000);
                  if (Buffer != null)
                  {
                     ArrayEx.Append(Buffer, ref RxData);
                     IsDirty = true;
                  }
               }
               if (IsDirty)
               {
                  //Debug.Print("Rx: {" + RxData.Length + "} - " + ArrayEx.AsStringHex(RxData, " "));
                  base.OnDataRx(RxData, RxData.Length);
                  IsDirty = false;
               }
               WaitIndex = System.Threading.WaitHandle.WaitAny(WaitHandleList, -1, false);
               if (WaitIndex == 0) { mRunning = false; }
            }
         }
         catch (Exception ex)
         {
            Debug.Print("Exception in SerialConnection.RxDataConsumerThread: " + ex);  
         }
         finally
         { }
      }
5

Try to lessen object allocation to speed things up. In the embedded world, we try to only allocate/free objects when absolutely necessary.

This should help http://wiki.tinyclr.com/index.php?title=Thinking_Small

6

I have created an extremely simple app. It has 2 main parts,

  1. creates a socket and connects to host, once connected it does nothing but wake up every 15 seconds and makes sure it is still connected.
  2. It opens a serial port and waits for data to be received. Once the data is received it waits 500ms and sends it back out.

There is a desktop app that accepts the socket and then does nothing. The same app also sends data to the serial port and when data is received on the serial port waits 500ms and sends it back out. effectively making an endless loop between desktop and device.

There is only about 42 bytes in the data packet with 2 packets being sent from the desktop to device.

After 1 hour and 21 minutes I get a buffer overrun error on the device serial port, this to me seems like an issue as the device should be able to handle such a simple and small task as this.
I have also ran this same test overnight without the socket connected and it did not give me a buffer overrun error.

SerialPort Class


using System;
using Microsoft.SPOT;
using System.IO.Ports;
using System.Threading;

namespace ChipworkX_Application1
{
   class SerialDriver
   {

      private SerialPort mSerialPort;
      private ManualResetEvent ShutdownEvent;

      public SerialDriver()
      {
         mSerialPort = new SerialPort("COM3", 115200, Parity.None, 8, StopBits.One);
         mSerialPort.DataReceived += new SerialDataReceivedEventHandler(mSerialPort_DataReceived);
         mSerialPort.ErrorReceived += new SerialErrorReceivedEventHandler(mSerialPort_ErrorReceived);
         ShutdownEvent = new ManualResetEvent(false);
      }

      void mSerialPort_ErrorReceived(object sender, SerialErrorReceivedEventArgs e)
      {
         Debug.Print(DateTime.Now.ToString() + " - Serial Port Error: " + e.EventType);
      }

      void mSerialPort_DataReceived(object sender, SerialDataReceivedEventArgs e)
      {
         try
         {
            int BytesToRead = mSerialPort.BytesToRead;
            if (BytesToRead > 0)
            {
               byte[] Buffer = new byte[BytesToRead];
               mSerialPort.Read(Buffer, 0, BytesToRead);
               ProcessRxData(Buffer, BytesToRead);
            }
         }
         catch (Exception ex)
         {
            Debug.Print("Exception in SerialDriver.mSerialPort_DataReceived: " + ex);
         }

      }

      void ProcessRxData(byte[] Data, int Count)
      {
         Thread.Sleep(500);
         if (Count > 0)
         {
            mSerialPort.Write(Data, 0, Data.Length);
         }
         else
         { }
      }

      public void ThreadMain()
      {

         System.Threading.WaitHandle[] WaitHandleList = { ShutdownEvent };
         bool ShutdownRequested = false;


         do
         {
            try
            {
               if (ShutdownEvent.WaitOne(0, false))
               {
                  ShutdownRequested = true;
               }
               else
               {
                  if (!mSerialPort.IsOpen)
                  {
                     if (TryOpenPort())
                     { }
                  }
                  else
                  { }
               }

               int WaitAnyTimeout = 60 * 1000;
               System.Threading.WaitHandle.WaitAny(WaitHandleList, WaitAnyTimeout, false);

            }
            catch (Exception ex)
            {
               Debug.Print("Exception in SerialDriver.ThreadMain: " + ex);
            }

         } while (!(ShutdownRequested));

      }

      protected bool TryOpenPort()
      {
         if (mSerialPort.IsOpen)
            return true;
         try
         {
            mSerialPort.Open();
         }
         catch (Exception ex)
         {
            Debug.Print("Exception in SerialDriver.TryToOpenPort: " + ex);
         }
         return mSerialPort.IsOpen;
      }


   }
}

Socket Class



using System;
using System.Net;
using Microsoft.SPOT;
using System.Threading;
using System.Net.Sockets;

namespace ChipworkX_Application1
{
   class SocketDriver
   {

      private static IPAddress remoteIP = IPAddress.Parse("192.168.0.50");
      private IPEndPoint remoteEP = new IPEndPoint(remoteIP, 14220);
      private Socket mSocket;
      private ManualResetEvent ShutdownEvent;

      public SocketDriver()
      {
         mSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
         ShutdownEvent = new ManualResetEvent(false);
      }

      public void ThreadMain()
      {

         System.Threading.WaitHandle[] WaitHandleList = { ShutdownEvent };
         bool ShutdownRequested = false;


         do
         {
            try
            {
               if (ShutdownEvent.WaitOne(0, false))
               {
                  ShutdownRequested = true;
               }
               else
               {
                  if (!IsConnected)
                  {
                     if (TryConnectSocket())
                     { }
                  }
                  else
                  { }
               }

               int WaitAnyTimeout = 15 * 1000;
               System.Threading.WaitHandle.WaitAny(WaitHandleList, WaitAnyTimeout, false);

            }
            catch (Exception ex)
            {
               Debug.Print("Exception in SocketDriver.ThreadMain: " + ex);
            }

         } while (!(ShutdownRequested));

      }

      public bool IsConnected
      {
         get
         {
            try
            {
               return ((mSocket != null) && mSocket.Poll(1 * c_microsecondsPerSecond, SelectMode.SelectWrite)); 
            }
            catch { return false; }
         }
      }

      const Int32 c_microsecondsPerSecond = 1000000;
      protected bool TryConnectSocket()
      {
         try { mSocket.Connect(remoteEP); }
         catch (SocketException ex)
         {
            if (ex.ErrorCode == 10035 || ex.ErrorCode == 10056)
            {
               if (mSocket.Poll(1 * c_microsecondsPerSecond, SelectMode.SelectError)) { return false; }
               if (!mSocket.Poll(1 * c_microsecondsPerSecond, SelectMode.SelectWrite)) { return false; }
            }
            else
            {
               Debug.Print("SocketException in SocketDriver.TryConnectSocket: " + ex);
               return false;
            }
         }
         return true;
      }


   }
}
7

The first thing you have to do is not spend 500ms in an event handler each time you receive one or more serial characters. The event handler thread is special, and putting a wait into it could cause funny things to happen.

8

If I take out the 500ms sleep, then it only takes about 5 minutes before I get buffer overruns. and I got 9 of them in a row, which equals 9 lost bytes.

I understand that the code given is not optimized, but I would think it should be able to handle simple comm traffic like this with out an issue. So I’m trying to see if the issue is less about code optimization and more if there is a bug lower down the pipe.

9

You have not included all of your code. Makes it hard to see what is happening.

You need to develop a small standalone example that just shows the problem you are having.

10

Here is the rest of the Device Code. 2 more classes 4 total:
Program.cs


using System;
using Microsoft.SPOT;
using Microsoft.SPOT.Net.NetworkInformation;

using GHIElectronics.NETMF.Net;
using GHIElectronics.NETMF.Hardware;

namespace ChipworkX_Application1
{
   public class Program
   {
      private static SystemManager mSysMgr;

      public static void Main()
      {
         EnableEnet();
         ConfigureEnet();

         mSysMgr = new SystemManager();
         mSysMgr.Start();

         System.Threading.Thread.Sleep(-1);
      }

      public static NetworkInterface EnetSettings;
      public static byte[] MAC_Address = new byte[] { 0x00, 0x21, 0x03, 0x80, 0x34, 0x0D };
      public static string Gateway = "0.0.0.0";
      public static string[] DNS_Entries = new string[] { "0.0.0.0", "0.0.0.0" };
      public static string StationIPAdrs = "192.168.0.201";
      public static string StationSubnet = "255.255.255.0";

      public static void ConfigureEnet()
      {
         if (EnetSettings != null)
         {
            // Store MAC
            Array.Clear(EnetSettings.PhysicalAddress, 0, EnetSettings.PhysicalAddress.Length);
            Array.Copy(MAC_Address, EnetSettings.PhysicalAddress, MAC_Address.Length);

            // Store IP
            EnetSettings.EnableStaticIP(StationIPAdrs, StationSubnet, Gateway);
            Debug.Print("Setting:\n\t IP: " + StationIPAdrs + "\n\tSUB: " + StationSubnet + "\n\tGWY: " + Gateway);
            // Store DNS
            EnetSettings.EnableStaticDns(DNS_Entries);
            Debug.Print("Setting:\n\tDNS1: " + DNS_Entries[0].ToString() + "\n\tDNS2: " + DNS_Entries[1].ToString());
         }
      }

      public static void EnableEnet()
      {
         Debug.Print("Ethernet Enabled: " + GHIElectronics.NETMF.Net.Ethernet.IsEnabled.ToString());
         if (!Ethernet.IsEnabled)
         {
            // Switch to Ethernet interface
            Ethernet.Enable();
         }

         Debug.Print("Ethernet Enabled - Retrieving Interface...");
         // Get Enet Interface
         NetworkInterface[] Interfaces = NetworkInterface.GetAllNetworkInterfaces();
         for (int index = 0; index < Interfaces.Length; ++index)
         {
            if (!(Interfaces[index] is Wireless80211))
            {
               EnetSettings = Interfaces[index];
               break;
            }
         }

      }


   }
}

SystemManager.cs


using System;
using Microsoft.SPOT;
using System.Threading;

namespace ChipworkX_Application1
{
   class SystemManager
   {

      private Thread mSerialThread;
      private Thread mSocketThread;
      private SerialDriver mSerial;
      private SocketDriver mSocket;

      public void Start()
      {
         mSocket = new SocketDriver();
         mSerial = new SerialDriver();
         mSerialThread = new Thread(mSerial.ThreadMain);
         mSerialThread.Start();
         mSocketThread = new Thread(mSocket.ThreadMain);
         mSocketThread.Start();
      }

   }
}

Desktop Code: Simple VB Form App with 1 button


Imports System.IO.Ports
Imports System.Net.Sockets

Public Class Form1


   Private mSerialPort As SerialPort
   Private mTCPListener As TcpListener
   Private mClient As TcpClient

   Protected Overrides Sub OnLoad(e As System.EventArgs)
      MyBase.OnLoad(e)

      mSerialPort = New SerialPort("COM1", 115200, Parity.None, 8, StopBits.One)
      AddHandler mSerialPort.DataReceived, AddressOf mSerialPort_DataReceived
      AddHandler mSerialPort.ErrorReceived, AddressOf mSerialPort_ErrorReceived
      mSerialPort.Open()

      mTCPListener = New TcpListener(System.Net.IPAddress.Any, 14220)
      mTCPListener.Start()
      mTCPListener.BeginAcceptTcpClient(New AsyncCallback(AddressOf AcceptTCPClient), mTCPListener)

   End Sub

   Protected Overrides Sub OnFormClosing(e As System.Windows.Forms.FormClosingEventArgs)
      MyBase.OnFormClosing(e)

      If mSerialPort IsNot Nothing Then
         mSerialPort.Close()
         mSerialPort.Dispose()
      End If
      If mTCPListener IsNot Nothing Then
         mTCPListener.Stop()
      End If

   End Sub

   Private Sub AcceptTCPClient(ar As IAsyncResult)
      Dim Listener As TcpListener = CType(ar.AsyncState, TcpListener)

      mClient = Listener.EndAcceptTcpClient(ar)

      Console.WriteLine("Client Connected: " & mClient.Client.LocalEndPoint.ToString())
   End Sub


   Private Sub mSerialPort_ErrorReceived(sender As Object, e As SerialErrorReceivedEventArgs)
      Debug.Print("{0} - Serial Port Error: {1}", Now.ToString(), e.EventType)
   End Sub

   Private Sub mSerialPort_DataReceived(sender As Object, e As SerialDataReceivedEventArgs)
      Try
         Dim BytesToRead As Integer = mSerialPort.BytesToRead
         If BytesToRead > 0 Then
            Dim Buffer(BytesToRead - 1) As Byte
            mSerialPort.Read(Buffer, 0, BytesToRead)
            ProcessRxData(Buffer, BytesToRead)
         End If
      Catch ex As Exception
         Debug.Print("Exception in mSerialPort_DataReceived: {0}", ex)
      End Try
   End Sub

   Private Sub ProcessRxData(Data As Byte(), Count As Integer)
      Debug.Print("{0} - Processing Data: ({1})", Now.ToString, Count)
      If (Count > 0) Then
         mSerialPort.Write(Data, 0, Data.Length)
      Else
      End If
   End Sub


   Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
      mSerialPort.Write("12345678901234567890123456789012345678901234567890")
   End Sub
End Class
11

Too much code for me to look at… sorry

12

@ Andy.NApex Did you ever solve this?

13

I have not resolved this issue yet.