Chris, you do realise it is possible to write code yourself right? 
Yeah, I do, itâs just Iâm really strapped for time at the moment.
Iâm working on an overhaul of the parsing logic here. My 10hz gpz is putting out data way too quickly. The first step is a new buffering class that uses a rolling byte buffer instead of the message queue. so the SerialPortDataReceived code looks like this (some code removed for clarity):
serialBuffer.LoadSerial(serialPort);
string dataLine;
while((dataLine = serialBuffer.ReadLine()) != null)
{
// parse the NMEA data sentence here
}
Iâm still working on performance improvements, but at the current data rate (GPGLL @ 2hz, GPRMC @ 5hz, everything else off), the SerialPortDataReceived event was taking up 23% of my CPU time. Now itâs just over 10%. I still have a few major performance improvements to make and clean-up work before I check it in.
I also added a method to send a NMEA message to the GPS. This is how Iâm configuring my GPS to send a little less data until I can tune this baby up.
I will share my SerialBuffer class here⌠it seems pretty complete, except for a few hacks.
public class SerialBuffer
{
private System.Text.Decoder decoder = System.Text.UTF8Encoding.UTF8.GetDecoder();
private byte[] buffer;
private int startIndex = 0;
private int endIndex = 0;
private char[] charBuffer;
public SerialBuffer(int initialSize)
{
buffer = new byte[initialSize];
charBuffer = new char[256];
}
public void LoadSerial(SerialPort port)
{
int bytesToRead = port.BytesToRead;
if (buffer.Length < endIndex + bytesToRead) // do we have enough buffer to hold this read?
{
// if not, look and see if we have enough free space at the front
if (buffer.Length - DataSize >= bytesToRead)
{
ShiftBuffer();
}
else
{
// not enough room, we'll have to make a bigger buffer
ExpandBuffer(DataSize + bytesToRead);
}
}
//Debug.Print("serial buffer load " + bytesToRead + " bytes read, " + DataSize + " buffer bytes before read");
port.Read(buffer, endIndex, bytesToRead);
endIndex += bytesToRead;
}
private void ShiftBuffer()
{
// move the data to the left, reclaiming space from the data already read out
Array.Copy(buffer, startIndex, buffer, 0, DataSize);
endIndex = DataSize;
startIndex = 0;
}
private void ExpandBuffer(int newSize)
{
byte[] newBuffer = new byte[newSize];
Array.Copy(buffer,startIndex,newBuffer,0,DataSize);
buffer = newBuffer;
endIndex = DataSize;
startIndex = 0;
}
public byte[] Buffer
{
get
{
return buffer;
}
}
public int DataSize
{
get
{
return endIndex - startIndex;
}
}
public string ReadLine()
{
lock (buffer)
{
int lineEndPos = Array.IndexOf(buffer,'\n',startIndex,DataSize); // HACK: not looking for \r, just assuming that they'll come together
if (lineEndPos > 0)
{
int lineLength = lineEndPos - startIndex;
if (charBuffer.Length < lineLength) // do we have enough space in our char buffer?
{
charBuffer = new char[lineLength];
}
int bytesUsed,charsUsed;
bool completed;
decoder.Convert(buffer, startIndex, lineLength, charBuffer, 0, lineLength, true, out bytesUsed, out charsUsed, out completed);
string line = new string(charBuffer,0,lineLength);
startIndex = lineEndPos + 1;
//Debug.Print("found string length " + lineLength + "; new buffer = " + startIndex + " to " + endIndex);
return line;
}
else
{
return null;
}
}
}
}
This is probably great code but why all this buffer stuff with char arrayâs.
As we are working with strings so why not just convert what you read to a string and append it to a string buffer. The GC will handle the rest.
@ geir
the binary manipulation methods are much faster and more flexible. I avoid a ton of reallocations (very few FC calls), all of the looping is done in fast native code instead of interpreted c# and the code is much simpler.
When new data comes in, instead of cocatenenating two strings (which in .NET actually creates a new third string and letâs the GC deal with cleaing up the other two), the SerialPort class writes to an offset to a buffer we already allocated. When reading data out, the UTF8Decoder reads a range of bytes of a buffer, after which a pointer is incremented instead of creating another string.
If the MF had a StringBuffer class like the big framework, then doing this in strings would be more practical. (And the community-developed StringBuffer doesnât countâitâs written in managed code so itâs not nearly as fast.)
Again, is probably great code.
but one thing confuses me;
How is this native code?
Can you elaborate on this, -what is native code and what is âinterpeted c#â code in this setting?
Has anyone ever seen or use âArray.BinarySearchâ?
This looks very promising for things like GPS parsing, specially if you are looking for the beginning/end of the line.
I wanted to ask the GHI developers to add something for that but then I found this built in method that maybe exactly what we need?
@ geir Iâm using native .net framework methods on all iterative operations (e.g. Array.IndexOf, Array.Copy, decoder.Convert) which are written in low-level code tuned to the hardware. The same code written in c# would be interpreted and an order of magnitude slower. There are some native string operations but they all create new strings, not modify existing strings (strings are immutable in .NET) so they end up making work for the GC.
@ gus Array.BinarySearch only works if the array data is sorted, I believe. But Array.IndexOf is a very fast native function. Itâs what I use to find the â\nâ line break in SerialBuffer.ReadLine.
IndexOfâŚlooks perfect for parsing data like GPS streams. I havenât used it before.
I am still curious on how BinarySearch works and how it is different than IndexOf. They seem to be the same to me!
BinarySearch requires a sorted array to work. (Very fast!)
IndexOf is a traditional item at a time search
-Eric
Does anyone know if all binary outputting GPSes use the same protocol? I canât seem to find a name of the standard, if they do.
UBX is different to others