Trouble correctly interpreting serial data

Archive
1

Hi,

I’m trying to read the serial data coming from a GPS module but I’m having trouble getting all of the message. I’m pretty sure that it’s my code that is wrong somewhere so I thought I’d check with you guys if that’s OK.

Here’s what I’m doing during the serial data received event;


static void EM406_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            Debug.Print("Data Received");
            data_received = EM406.Read(buffer, 0, buffer.Length);
            buffer_i = 0;

            while (buffer_i < data_received)
            {
                gps_data[buffer_i] = (char)buffer[buffer_i];
                buffer_i++;
            }

            string sentence_string = new string(gps_data, 0, gps_data.Length);

            Debug.Print(sentence_string.Substring(0, sentence_string.Length));
        }

Here is an example of the output in the debug window;

So I’m not getting the $ sign correctly and the rest of the data after the GPGGA header which should look like this;

Any help is greatly appreciated.

2

You’re overwriting your “gps_data” buffer. Move this line out:


buffer_i = 0;

and try again.

3

@ Hyperlisk - I’m not sure I get what you’re saying.

buffer_i

is just a counter to iterate through the serial port read buffer and allocate the characters to the character array

gps_data

.

Maybe the rest of the code makes it easier to see what I’m trying to do;


public class Program
    {
        static SerialPort EM406 = new SerialPort("COM3", 4800, Parity.None, 8, StopBits.One);
        static byte[] buffer = new byte[100];
        static char[] gps_data = new char[100];
        static int data_received;
        static int buffer_i;
        
        public static void Main()
        {
            EM406.ReadTimeout = 0;
            EM406.Open();
            EM406.DataReceived += new SerialDataReceivedEventHandler(EM406_DataReceived);
            EM406.ErrorReceived += new SerialErrorReceivedEventHandler(EM406_ErrorReceived);

            Thread.Sleep(Timeout.Infinite);
        }

        static void EM406_ErrorReceived(object sender, SerialErrorReceivedEventArgs e)
        {
            Debug.Print("Error");
        }

        static void EM406_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            Debug.Print("Data Received");
            data_received = EM406.Read(buffer, 0, buffer.Length);
            buffer_i = 0;

            while (buffer_i < data_received)
            {
                gps_data[buffer_i] = (char)buffer[buffer_i];
                buffer_i++;
            }

            string sentence_string = new string(gps_data, 0, gps_data.Length);

            Debug.Print(sentence_string.Substring(0, sentence_string.Length));
        }
    }
4

Do I even need the while loop, instead just using this?

gps_data = Encoding.UTF8.GetChars(buffer);
5 
            while (buffer_i < data_received)
            {
                gps_data[buffer_i] = (char)buffer[buffer_i];
                buffer_i++;
            }

You always start at 0 and therefore you always overwrite previous data. You need to keep last position somewhere until you reach end of the line. When you reach it you reset that value to 0. Something like this:

static int position = 0;
static void EM406_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            Debug.Print("Data Received");
            data_received = EM406.Read(buffer, 0, buffer.Length);
            buffer_i = 0;
 
            while (buffer_i < data_received)
            {
                gps_data[position+buffer_i] = (char)buffer[buffer_i];
                buffer_i++;
                position++;
            }

            string sentence_string = new string(gps_data, 0, position);
 
            Debug.Print(sentence_string);
        }

This is still not complete because you might get * in between reads. In your while loop you need to check each char if it’s * and then process two more bytes and reset position to 0. I hope you get the idea.

6

@ wolfbuddy

You are also assuming that an entire GPS message arrives with one read. This does not occur. You should be assembling the message over multiple events. Each sentence begins with a ‘$’ and ends with a carriage return/line feed sequence and can be no longer than 80 characters of visible text (plus the line terminators).

When you receive the DateReceived event, you must determine how many bytes are available to read. I suggest you follow the code below:


static void On_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
     while (serialPort.BytesToRead > 0)
    {
         byte[] buffer = new byte[serialPort.BytesToRead];
         serialPort.Read(buffer, 0, buffer.Length);
         ..... process data

    }
}

This code is inefficient. You would not want to constantly allocate a buffer to read into, but it illustrates the proper handling of the event. The while loop is there to allow for handling any additional bytes that arrive while processing the prior read.

7

@ wolfbuddy -
just had the same thing to do:
https://www.ghielectronics.com/community/codeshare/entry/763
see in Program.cs

void bluetooth_DataByteArrayReceived(Bluetooth sender, byte[] data, int startIndex, int ByteCount)

I used a modified class serialBuffer.cd by andrejk: SerialBuffer: simple, fast … from this Forum

Another thing: I think, you must get the data from the eventargs
(e.g. e.data ) or whatever datatype you will find there.
Edit: This was a wrong advice, eventargs from serial port do not contain data

8

[quote=“Mike”]
@ wolfbuddy

You are also assuming that an entire GPS message arrives with one read. This does not occur. You should be assembling the message over multiple events. [/quote]

This is the thing I’ve missed, I wasn’t aware of that. Is this because there is a limit to the number of bytes that can be sent/received at a time? Please excuse my limited knowledge of serial data comms.

9

[quote=“wolfbuddy”]

There are internal buffers of limited size within the framework for the serial port. The goal is to get the data out of these buffer as fast as possible to avoid overrun.

A serial port is a “stream” device. With stream devices, there is no concept of a “record”. It just receives a stream of bytes. TCP sockets work in a similar manner.

When sending to a stream device, you usually do not need to be concerned about the size of the internal buffer. If the data size exceeds the internal tx buffer size, your program will block until all the data has sent.

10

I think I see how the serial data work now (hopefully!).

Every data event received is 6 characters (so 6 bytes too?), except for the gap between each gps data array which is a new line followed by the 5 characters ‘GPGGA’, which appears to be repeated. The first character in each message is the next character in the gps data array and, as mentioned by Mike, the first one is always a ‘$’ and the last is a checksum denoted with the ‘*’.

So what I collected yesterday as;

Translates as (except for one missing comma which is worrying);

From which you should all be able to see where I was in the world and how fast I was travelling! :smiley:

Edit; also just re-read Hyperlisk and RoSchmi’s comments and they make much more sense now, thank you guys.

11

you can never be sure how many characters you’ll get - if you’re “consistently” getting 6, that’s great, but probably a fluke. Sometime soon you’ll get 4 or 7, so just don’t assume anything.

12

Yep, I think I understand that now.

I think my code does that anyway though because the SerialPort.Read method returns an integer that is the number of bytes from the SerialPort input buffer, and I use that already to set the number of times my loop iterates. I need to do as Hyperlisk suggested and set a positional counter to stop myself overwriting the gps data array each time, but instead adding to it with the first character of each message to build up the complete character array of gps data.

I also need to add checks to find the $ and * signs as well as the gps headers, so I’ve got a bit of work to do now!

13

@ wolfbuddy - First, sync by waiting for a ‘$’. Then assemble a sentence until I get a ‘\r’, throwing away ‘\n’.

With a serial port, attached to a device like a GPS which constantly outputs, you need to perform a sync operation. It is possible that you could open your serial port in the middle of a sentence and get half a sentence.

It is also important to know that if the serial port hardware senses any fear on your part, it will mess you up.

1 Like
14

LOL

15

@ wolfbuddy - You can also use the Deserialize() method in my SimpleSerial class, which automatically parses streaming NMEA messages such as GPS and stores them in an array that you can then iterate over. Synchronization and dangling sentences are handled automatically in the background. Be sure to check out the usage example. It should be pretty straightforward.

https://www.ghielectronics.com/community/codeshare/entry/644

16

Bring it on!! :smiley:

Thanks bud, I’ll take a look.

I do already have a class that will handle the gps data, but if I just use that I won’t really learn anything about handling serial data!

17

@ Mike -

Had a good belly laugh on that one!

18

Hi,

I’ve changed my data received event to the following;

static void EM406_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            //Debug.Print("Data Received");
            data_received = EM406.Read(buffer, 0, buffer.Length);

            if ((char)buffer[0] == '$')
            {
                gps_message_start = true;
                gps_data[0] = (char)buffer[0];
                gps_data_i = 1;
            }
            else if ((char)buffer[0] == '*')
            {
                if (gps_message_start)
                {
                    sentence_string = new string(gps_data, 0, gps_data.Length);
                    Debug.Print(sentence_string.Substring(0, sentence_string.Length));
                    gps_data_i = 0;
                }
                gps_message_start = false;
            }
            else
            {
                if (gps_message_start)
                {
                    gps_data[gps_data_i] = (char)buffer[0];
                    gps_data_i++;
                }
            }
        }

and I’m now getting lots of…

So now my plan is to save the messages in separate variables based on their headers, so that I can then pluck out the information that I want from them.

Any comments are very welcome, but please remember I’m a noob!! :smiley:

19

I’d use instead of buffer.length I’d use BytesToRead(). By trying to read more than there is available, you might then wait (and give you less time to process the data).

20

@ Brett - Do you mean how Mike suggested before;


byte[] buffer = new byte[serialPort.BytesToRead];
serialPort.Read(buffer, 0, buffer.Length);

So this is setting the size of the buffer array to the number of bytes available to read from the serial port, before writing them into the array?

Wouldn’t creating a new byte array, every time there is a data received event, be an inefficient thing to do? Should I be creating a ‘Static byte[] buffer;’ at the top of my class?

Edit; actually, I just realised that I’m creating a ‘Static byte[] buffer = new byte[100];’ at the top of my class. This seemed more efficient to me.