NMEA parser without strings

Hello,

Here is a NMEA parser that does not involve strings for decoding. Not a single one.

It takes an NMEA sentence as a byte array, as it would come from the GPS/UART, and then decode it to different static variables.

The caller only has to periodically check those variables to get the latest data :

class Program
    {
        private static readonly String GGAString = "$GPGGA,123519,4807.038,N,01131.000,E,1,08,0.9,545.4,M,46.9,M,,*47";
        private static readonly String GSAString = "$GPGSA,A,3,04,05,,09,12,,,24,,,,,2.5,1.3,2.1*39";
        private static readonly String GSVString1 = "$GPGSV,3,1,11,03,03,111,00,04,15,270,00,06,01,010,00,13,06,292,00*74";
        private static readonly String GSVString2 = "$GPGSV,3,3,11,22,42,067,42,24,14,311,43,27,05,244,00,,,,*4D";
        private static readonly String RMCString = "$GLRMC,221030,A,4807.038,N,01131.000,E,022.4,084.4,101120,003.1,W*6A";

        static void Main()
        {
            NMEAParser.Parse(Encoding.UTF8.GetBytes(RMCString));
            
            Debug.WriteLine($"Latest RMC sentence");
            Debug.WriteLine($"Fix time : {NMEAParser.RMCSentence.FixTime}");
            Debug.WriteLine($"Status : {(NMEAParser.RMCSentence.Status == 'A' ? "valid" : "Invalid")}");
            Debug.WriteLine($"Latitude : {NMEAParser.RMCSentence.Latitude,8:N3} {NMEAParser.RMCSentence.LatitudeHemisphere}");
            Debug.WriteLine($"Longitude : {NMEAParser.RMCSentence.Longitude,8:N3} {NMEAParser.RMCSentence.LongitudePosition}");

            Thread.Sleep(Timeout.Infinite);
        }
}

The strings you are seeing here are only for readability. As you can see, they are sent to the parser as Byte arrays.

Result :

Latest RMC sentence
Fix time : 11/10/2020 22:10:30
Status : valid
Latitude : 48.070 N
Longitude : 11.310 E

You know where I live, now :wink:

So far, only four sentences are handled : RMC, GGA, GSA and GSV. But it’s very easy to add new ones.

Why this parser ? Because strings in C# can easily lead to memory issues and because GPS data can come in huge amounts or numerous chunks in a short time.

This code does not consume much memory. So far, parsing 5 sentences every second during 2 min has only consumed 13KB.
Whereas using strings to decode sentences with the same timings eats up 268KB of memory…

> Before loop BYTES :
> +-----------+------------+------------+
> | Memory    | Used       | Free       |
> +-----------+------------+------------+
> | Managed   |    156,080 |    351,712 |
> | Unmanaged |          0 | 33,554,404 |
> +-----------+------------+------------+
> 
> After loop BYTES :
> +-----------+------------+------------+
> | Memory    | Used       | Free       |
> +-----------+------------+------------+
> | Managed   |    169,616 |    338,176 |
> | Unmanaged |          0 | 33,554,404 |
> +-----------+------------+------------+
> 
> Before loop STRINGS :
> +-----------+------------+------------+
> | Memory    | Used       | Free       |
> +-----------+------------+------------+
> | Managed   |    155,504 |    352,288 |
> | Unmanaged |          0 | 33,554,404 |
> +-----------+------------+------------+
> 
> After loop STRINGS :
> +-----------+------------+------------+
> | Memory    | Used       | Free       |
> +-----------+------------+------------+
> | Managed   |    423,696 |     84,096 |
> | Unmanaged |          0 | 33,554,404 |
> +-----------+------------+------------+

It will replace the current MBN parser in the GPS utilities driver.

But remember : it’s a parser. Only a parser. The GPS and UART logic are separate things not handled in this code.

9 Likes

I call BS ~600km wrong :yum:

3 Likes

The NMEA samples were taken from a webpage, there are not actually real sentences from my GPS.

But now everybody knows the real location :smiley:

1 Like

This is awesome!

And it came at just the right time. I am working on a project where I’ll need to be able to decode 6 UARTS full of NMEA soon, this library will be an AWESOME addition to it! Thank you so much!!

1 Like

Thank you !

Do not hesitate to tell me which NMEA sentences are currently missing for your project. I can easily add them.

For now, HDT and VTG would make me happy!

I also have some custom sentences coming soon, but I will add it myself :slight_smile:

VTG and HDT added.

1 Like

Have you seen this? GPS library · Issue #575 · ghi-electronics/TinyCLR-Libraries · GitHub

I think what you did is needed by many, so thank you. Are you going to turn it into a NuGet and maintain regularly? Even better, contribute it to the drivers repo and then we would maintain it going forward. Assuming this lib solves the issue above.

Thank you so much! Much appreciated.


I agree with Gus. It would be awesome if you could make this a NuGet package, but that might make adding new sentences into it slightly harder.

The code can always be available for those with special needs but then NuGet for everyone else.

I did not see this issue but it looks like this parser can be a solution.

I will contribute to your driver repo with it in a few days, time for me to strengthen it a bit with the TalkerID and some (little) more NMEA sentences.
If you could handle the NuGet package, it would be nice :wink:

4 Likes

You are awesome :slight_smile:

There will be a significant change in the driver. I have used floats for latitude, longitude and such fields but there are sentences that send very precise informations and float is not accurate enough.

This is for example the case for the GLL sentence :

$GNGLL,4404.14012,N,12118.85993,W,001037.00,A,A*67

So I will have to use doubles instead of singles to maintain the precision of the received data. At the cost of increased structs’ sizes.

Hello,

Major overhaul on the NMEA parser…

After some tests, I’ve noticed that at some time available memory started to decrease. Slowly but it was decreasing regularly. Not good :frowning:

So I’ve changed many things : no more ArrayLists, no more Array.Split(). Those were the cause of the “memory leak”.

The code is also now very specific to parsing NMEA sentences. Before the rework, some methods were kind of generic but they were consuming too much memory.

Now the code is not as beautiful as before but it’s way more efficient. And memory consumption is minimal and steady.

After 10.000 loops, each one parsing 8 different NMEA sentences every 10ms, only 18KB have been eaten :

> Before loop 10.000 x 8 sentences
> +-----------+------------+------------+
> | Memory    | Used       | Free       |
> +-----------+------------+------------+
> | Managed   |    157,168 |    350,624 |
> | Unmanaged |          0 | 33,554,404 |
> +-----------+------------+------------+
> 
> After loop 10.000 x 8 sentences
> +-----------+------------+------------+
> | Memory    | Used       | Free       |
> +-----------+------------+------------+
> | Managed   |    175,536 |    332,256 |
> | Unmanaged |          0 | 33,554,404 |
> +-----------+------------+------------+

It’s the same memory consumption for 10 loops or 1000 loops. So now I know that memory doesn’t “leak” anymore.

Also, as said above, I have added the talker Id and changed Singles to Doubles.

Edit PR created in TinyCLR-Drivers repo.

1 Like

Please do not use this parser yet.

It still needs some fixes as different GPS devices do not send the same data format for the same sentences
e.g. some GPS are sending Latitude or Fix time as xxxxxx.xx while others are sending xxxxxxx only, without decimals.

Also, checksums are invalid at the moment for sentences that end with CRLF.

PR has been closed for now.

Please take your time

I will need some time, indeed…

Same device (SAM-M8Q) is sending different GSV sentences for the last satellite in view, depending on the talker :

> GPS : $GPGSV,4,4,13,30,31,066,28*40
> GLONASS : $GLGSV,3,3,10,84,06,036,,,,,18*52

This is quite challenging.

It’s the last pebble in my shoe. But it hurts :cry:

Writing a NMEA parser is easy, writhing a proper parser that is efficient is not so easy. I have seen so many implementations that work but they just kill the system with strings manipulation.

One of the thought I had was to implement a native parser or at least implement a “helper” in this case. Like a bit converter class that takes byte arrays and breaks it into values. Like you search for $ and \n in C# and then pass that byte array to a native parser that returns array values found between commas.

BitConverter.ParseCSV(byte[] dawdata, start, size, double[] returnedvalues)

We are open for ideas.

That’s exactly how this parser is working.
Problem is that not all devices are sending excactly the sames sentences and different talkers do not send “standard” sentences either.

The GSV example above is a “good” example of what can happen :frowning:

About the helper, it may be useful but, again, there would be memory issues at some time because of the arrays you would have to return. Each one would have a different size, thus needing a new one to be allocated at each call.

That’s why I’m using an array containing commas positions. It’s a small array of fixed size. This makes code less “beautiful” or readable, but it does not consume much memory. And it’s fast. Even in C#.

I was wrong in my analysis… This is an almost well-formed sentence, in fact.
Indeed, there are 10 satellites in view. Each GSV containing 4 satellites, the last GSV should contain 2 satellites. Which is the case for GLONASS.
But it turns out that the 10th satellite has empty data :frowning: