Panda II not receiving serial input

Just received my Panda II and I have the following problem.
This happens on both COM1 and COM4.
COM1 is connected to an XBee.
COM4 is connected to a PC.
I can send data with no problem on both ports.
I could not receive data from the PC. I commented out the setup for COM4 and programed the data input pin as in input and it worked fine. Went back to connection to the PC and could not receive any data. In my power up routine. I now setup the data input pin as an input, read it, then dispose it. I then setup COM4 and now I can receive data from the PC.
I did the same for COM1 but still can not receive any data. The XBee works because it was connected to a Netduino and was workiing fine. The XBee is receiving data as I can see the receive LED come on.

There must be something simple that I am missing but I’m sure not seeing it today.

Thanks for any help in this matter,
Dave

This is in main()
(Also can not receive if ledTest2 is commented out.)


   ledTest2 = new InputPort((Cpu.Pin)FEZ_Pin.Digital.Di0, false, Port.ResistorMode.Disabled);
   ledTest2.Read();
   ledTest2.Dispose();
   XBeeControl = new XBee.XBee("COM1", 2400);


This is the XBee port open method.


private void OpenPort()
   {
   comPort = new SerialPort(s_Port, i_BaudRate, Parity.None, 8, StopBits.One);
   if(comPort != null)
      {
      comPort.Open();
      if(comPort.IsOpen)
         comPort.DataReceived += new SerialDataReceivedEventHandler(comPort_DataReceived);
      }
}				// end-of private void OpenPort()

How are you connecting the Panda to the PC?

I’m using a serial port via USB to the PC (no handshaking). I have a USB-serial converter.
The com port to the XBree is straight wired.

Hi dcas, welcome

I would ask you to test in isolation. Test a simple input and output to each serial port when it’s connected to a PC, and prove that you can send and recieve data that way.

I wasn’t sure what you meant by [quote]and programed the data input pin as in input and it worked fine. [/quote] so perhaps that’s relevant and telling us something but I can’t be sure.

Can you also confirm you have a TTL or RS232 serial to USB converter? Panda only has TTL level and if your serial->USB converter is expecting RS232 (like mine does) you also need an RS232 circuit (MAX232 or similar)

What I meant was that I tested the serial input pin as an input (com port commented out). When I saw that it was working like that I went back and setup it up as a COM port. I did each setup one at a time.
The converter (USB-TTL) is my own. I lowered the pull up resistor going to the serial input of the Panda and now the PC send and receive seems to be working a lot better.
The level from the converter to the Panda II is 3.29v.
The data from the XBee is connected direct as it puts out 3.29v also.
I modified the code to run a test as follows. I send a message to the XBee to another XBee out at the well (monitoring the temperature and heating lamp) about every minute. It will then respond with sensor data. The test is that, everytime I go to send the request, I close the com port and reopened it. This has resulted in a valid response on the serial input most of the time. I still get a few no answers by the serial port (data is being returned by the remote XBee as indicated by an LED on the XBee).
I would think the serial data from the XBee would be the one that works the best as there is no converter as it is wired direct with two wires (plus ground of course).

Do the levels need to be 5v?

I would like to buy a converter but being on Social Security it took 2 months to get the Panda II. It could be another couple of months before I can get anything else. So I have to make do with what I have laying around. My home built converter has been working fine for months on the Netduino.
If I can’t get this working better by tomorrow I will have to go back to the Netduino because of a big freeze coming in this week.

Thanks for your time on this

hi again

ok, that kind of makes sense.

Voltage levels on all GHI products are 3v3, but on a Panda most inputs are 5v tolerant if you’re interfacting with 5v outputs from something you can’t adjust.

I have never seen the need to close and re-open the port to get data. I don’t know any other way to say this - I think you have a coding issue so it’s time to show us your full code.

You also need to be real sure that you need a datarecieved event handler, as sometimes you are better off with a polling loop - the event handler doesn’t guarantee you get all the data at any one point in time, so you still have to load that into a buffer and extract data out of it.

I’d still like to see a very generic serial port test that you do on COM1 and COM4 and connected to the PC, to make sure we have all the components working before we add xbee; I’ll put together some code and test on my Panda II and share it here for you to test.

Give this a whirl… it’s overkill and takes some code from one of my projects, but it should prove that you have comms working on a port.

using System;
using System.IO;
using System.IO.Ports;
using System.Text;
using System.Threading;

using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using Microsoft.SPOT.IO;

using GHIElectronics.NETMF.FEZ;

using GHIElectronics.NETMF.Hardware;
using GHIElectronics.NETMF.IO;
using GHIElectronics.NETMF; 

namespace FEZ_Panda_II_Application1
{
    public class Program
    {
        static SerialBuffer serialBuffer = new SerialBuffer(256);
        static SerialPort com1;
        static int delay_1interval = 1000;
        static int delay_2interval = delay_1interval * 2;
        static int myserialtimeout = 2000;


        public static void Main()
        {
            // Blink board LED

            bool ledState = false;
            string command;
            string response;
            
            com1 = new SerialPort("COM1", 9600, Parity.None, 8, StopBits.One);
            com1.ReadTimeout = myserialtimeout;
            com1.Open();

            OutputPort led = new OutputPort((Cpu.Pin)FEZ_Pin.Digital.LED, ledState);

            while (true)
            {
                // Sleep for 5000 milliseconds
                Thread.Sleep(5000);

                // toggle LED state
                ledState = !ledState;
                led.Write(ledState);
                command = "Hello PC, are you awake??\r\n";
                Debug.Print("about to send " + command);
                response = SendReceive(com1, command);
                Debug.Print("Got back " + response);

            }
        }
        private static string SendReceive(SerialPort port, string request)
        {
            port.Write(Encoding.UTF8.GetBytes(request), 0, request.Length);
            string line = ReadLine(port, 13, 300); // \r is line seperator.
            // if line starts with ", then keep reading lines until line that ends with 
            return line;
        }

        public static string ReadLine(SerialPort port, int delim = 10, int maxBufSize = 300)
        {
            //CR 13 LF 10 = "\r\n"
            if (maxBufSize <= 0) throw new ArgumentOutOfRangeException("maxBufSize");
            byte[] buf = new byte[maxBufSize];
            int curPos = 0;

            while (curPos < maxBufSize)
            {
                int read = port.Read(buf, curPos, 1);
                if (read == 0) break;
                if (buf[curPos] == delim) //this won't work for multiline responses since this flushes anything left unread
                {
                    port.Flush();
                    break;
                }
                curPos += read;
            }

            // curPos will be zero if first read is zero or first read is delim byte.
            if (curPos == 0) return null;
            if (curPos == maxBufSize)
                throw new InvalidOperationException("Line size > maxBufSize.");
            if (buf[curPos] != delim)
            {
                port.Flush();
                return null; // if no delim found, assume comms failure. drop the current reading and move on
            }

            buf[curPos] = 0;
            char[] ca = Encoding.UTF8.GetChars(buf);
            return new string(ca);
        }


    }

    public class SerialBuffer
    {
        private System.Text.Decoder decoder = System.Text.UTF8Encoding.UTF8.GetDecoder();
        private byte[] buffer;
        private int startIndex = 0;
        private int endIndex = 0;
        private char[] charBuffer;
        public SerialBuffer(int initialSize)
        {
            buffer = new byte[initialSize];
            charBuffer = new char[256];
        }
        public void LoadSerial(SerialPort port)
        {
            int bytesToRead = port.BytesToRead;
            //Debug.Print("to read:" + bytesToRead);
            if (bytesToRead > 0)
            {
                if (buffer.Length < endIndex + bytesToRead) // do we have enough buffer to hold this read? 
                {
                    // if not, look and see if we have enough free space at the front                    
                    if (buffer.Length - DataSize >= bytesToRead)
                    {
                        ShiftBuffer();
                    }
                    else
                    {
                        // not enough room, we'll have to make a bigger buffer                        
                        ExpandBuffer(DataSize + bytesToRead);
                    }
                }
                //Debug.Print("serial buffer load " + bytesToRead + " bytes read, " + DataSize + " buffer bytes before read");
                port.Read(buffer, endIndex, bytesToRead);
                endIndex += bytesToRead;
                //Debug.Print("serial buffer loaded " + bytesToRead + " bytes read, " + DataSize + " buffer bytes after read");
            }
            else
            {
                Debug.Print("serial read empty");
            }
        }

        private void ShiftBuffer()
        {
            // move the data to the left, reclaiming space from the data already read out                
            Array.Copy(buffer, startIndex, buffer, 0, DataSize);
            endIndex = DataSize;
            startIndex = 0;
        }
        private void ExpandBuffer(int newSize)
        {
            byte[] newBuffer = new byte[newSize];
            Array.Copy(buffer, startIndex, newBuffer, 0, DataSize);
            buffer = newBuffer;
            endIndex = DataSize;
            startIndex = 0;
        }
        public byte[] Buffer
        {
            get
            {
                return buffer;
            }
        }
        public int DataSize
        {
            get
            {
                return endIndex - startIndex;
            }
        }
        public string ReadLine()
        {
            lock (buffer)
            {
                int lineEndPos = Array.IndexOf(buffer, '\r', startIndex, DataSize);
                // HACK: not looking for \r, just assuming that they'll come together                            
                if (lineEndPos > 0)
                {
                    int lineLength = lineEndPos - startIndex;
                    if (charBuffer.Length < lineLength)  // do we have enough space in our char buffer?                        
                    {
                        charBuffer = new char[lineLength];
                    }
                    int bytesUsed, charsUsed;
                    bool completed;
                    decoder.Convert(buffer, startIndex, lineLength, charBuffer, 0, lineLength, true, out bytesUsed, out charsUsed, out completed);
                    string line = new string(charBuffer, 0, lineLength);
                    startIndex = lineEndPos + 1;
                    Debug.Print("found string length " + lineLength + "; new buffer = " + startIndex + " to " + endIndex);
                    return line;
                }
                else
                {
                    return null;
                }
            }
        }
        public bool GotEOL(char terminator) //
        {
            if (startIndex != endIndex)
            {
                if (startIndex + 1 != endIndex)
                {
                    return true;
                }
                else
                {
                    return true;
                }
            }
            else
            {
                return false;
            }
        }
        public bool GotEOL(byte terminator) //
        {
            if (startIndex != endIndex)
            {
                if (buffer[endIndex - 1] == terminator)
                {
                    return true;
                }
                else
                {
                    return false;
                }
            }
            else
            {
                return false;
            }
        }
        public bool GotEOL(byte[] terminator) //
        {
            if (terminator.Length > 2) throw new IndexOutOfRangeException("terminator must be 2 or less bytes");

            if (startIndex != endIndex)
            {
                if (endIndex - startIndex > terminator.Length)
                {
                    return false;
                }
                else
                {
                    if (buffer[endIndex - 1] == 0x0D && buffer[endIndex - 2] == 0x22)
                    {
                        return true;
                    }
                    else
                    {
                        return false;
                    }
                }
            }
            else
            {
                return false;
            }
        }

        public void DumpBuf()
        {
            string hexline = "0123456789ABCDEF";
            if (startIndex != endIndex)
            {
                lock (buffer)
                {
                    int lineLength = endIndex - startIndex;
                    for (int i = 0; i < lineLength; i++)
                    {
                        byte j = buffer[i + startIndex];
                        Debug.Print("char=" + i + " Hex=" + hexline[(j & 0xF0) >> 4] + hexline[j & 0x0F]);
                    }
                    startIndex = endIndex;
                }
            }
        }

    }

}

Your code appeared to work on both COM1 & COM4.
I’m now looking into using your method for my com’s.
Will post later today how it works out.

I want to thank you again for your time and trouble in this matter,
Dave

What I did was setup polling for the well receive com port. It is in it’s own thread and appears to be working better then the event based method I was using. I still do not understand why the event based receive method did not work correctly as it is hard to mess that code up and it was the same code I have been using for a long time. In any case, I don’t have time to work on that right now.
I left the PC com port as is, event based on the receive side as it has been running OK since I lowerd the pull up resister on the serial-3.3v converter.
What I did not make clear before is that I have a cable that is a USB-serial converter and I have my own circuit that converts RS232 to 3.3v.

A BIG thanks for your help,
Dave

great news.

Let me just say, event based serial port handling can be confusing because an event just means there is data in the buffer to collect; it doesn’t mean the data is complete, or all you would expect at any point, just that there is “some” data available to be extracted. You then still need to stuff that into a buffer and figure out when to parse it / deal with it, which is sometimes harder than getting the data alone. I looked at my specific scenario and came to the conclusion (along with some guidance from the others here too) that my needs were best suited to polling and that is what I implemented. If you can figure out a good algorithm to handle the event based data then I am sure it too can be made to work fine.

But right now the best thing is your project is back on the rails and before the snow too :slight_smile: (while I sit here in 30deg C heat at 8:30am :smiley: )

Brett,

here’s how I do it with events. The ieda is somehow simple : I’m using Start/End delimiter chars and buffering starts only when a Start char has been received and of course stops when the End char is received. All other received chars or bytes are ignored.

This give code like this one :


static char SerialStart='#', SerialEnd='$';
static void Serial_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            int Nb = Serial.BytesToRead;
            byte[] Buf = new byte[Nb];
            Serial.Read(Buf, 0, Nb);
            char[] ReceivedChars = Encoding.UTF8.GetChars(Buf);
            foreach (char c in ReceivedChars)
            {
                if (BufferingSerial && c != SerialEnd) { SerialString = string.Concat(SerialString, c); }
                if (c == SerialStart)
                {
                    BufferingSerial = true;
                    SerialString = string.Empty;
                }
                if (c == SerialEnd && BufferingSerial)
                {
                    BufferingSerial = false;
                    DealWithSerialString(SerialString);
                    SerialString = string.Empty;
                }
            }
        }

        private static void DealWithSerialString(string pStr)
        {
            Debug.Print("Result : " + pStr);
        }

It’s working pretty well.
The only thing is that you can’t use your Start/End chars for other purpose in your communications, of course.

BecA, that scenario is as I think I mentioned earlier, stuffing it into a buffer in the handler.

You take that a little further and deal with the delimeters within the handler, which is ok when you have control of the sending end of the data and can add these delimeters appropriately. The scenario where you can’t (say for instance when the sending device is a commercial product that can’t be modified) you have to handle that differently - for example, where the terminator is a two byte for pair you need to look forward and back in the full string not just the buffered data you’ve read in the handler.

I’ve been planning to start this thread for some time. I would really love for us to all put our heads together and come up with an optimal algorithm for solving this problem. I’ve been basically doing it like Bec and I’ve run into the problems Brett is referring to. More importantly, I’m wearing out the GC in the process. I think I know where to solve most of that but handling data that is streaming in very fast and needs to be parsed is a tough nut to crack. Maybe we should start a new thread or Wiki page for consolidating ideas and optimizing. My data is XML-like in nature and has proven to be difficult to keep up with the parsing as its coming in.

@ Brett : yes, it is exactly like what you wrote ! I was only showing the code instead of saying " I’m also doing like this" :wink:

And of course, you’re right on the fact that this way of doing things relies on a particular structure of data or protocal on the line.

My intent with this parser was that it can be used anywhere without big modifications. For example, I’m using it in a TCP client/server application (commercial, so I can’t show code here, sorry) with clientStream.Read() method.
Here again, I have control on the Start/End sequence, but the main method is the same.

I will try this to confirm, but I think it can also be used with NMEA sequences, where there’s a Start char (’$’) but no End char. But ‘$’ can be considered as a End char too : if string is not empty, then it’s an End char else it’s a Start char. To be tested, though.

@ ianlee74 : then start a new thread. I will try to help.
But I have to say that I wouldn’t like to have to parse XML in such a situation ??? I think that this not the right kind of data to be transmitted in a serial way [italic]if it has to be parsed at the same time[/italic].
Or, I would wait till the complete end of the XML structure and then do the parsing in another thread or method. But never while receiving data, which is by definition almost always incomplete.

Edit: as you can see, I don’t do any parsing in the receive method. In my sample, this would be done in the DealWithSerialString() method, where you could use string.Split() or string.Substring() as needed.

the other consideration is that if you want to make this truly connection agnostic, you can’t use a handler like this; if you’re using USB, the serial port has to be polled, so this method really only works for true UARTs.

Ok. I’m going to start this thread in a couple weeks once the piezo contest is over and I have time to setup that project again. It’s an XBee-based universal remote for controlling my quadcopter and eventually other things. The messaging isn’t exactly XML. It’s trimmed down but it is rather dynamic since the idea is that the remote can send and receive data from multiple sources continuously. I’m doing the parsing outside of the receive but it gets nasty when a message spans multiple packets. The message format is still a work in progress but I think it will eventually be something similar to JSON.

I believe this is what is causing my constant GC. My next steps were to try and eliminate splits or at least build a better buffer.

Strings in .NET are immutable, meaning, whenever you change one (via .Split(), .Substring(), .Replace(), .Concat() or anything like that) you are actually creating a new string.

String manipulation is a very common reason for high GC times, even in the full desktop framework.

Yea, I knew this was going to be a problem when I did it but there’s really no easy alternative. Perhaps an RLP based String object…