Hydra Performance

Archive
1

I now have my Fez Hydra receiving GPS data. I am using a 5Hz GPS Rx that is sending two lines of text, totalling about 130 characters every 200 ms.

I can receive the GPS data, perform some basic calculations like compute distance traveled and output 3 values to the LCD screen using SimpleGraphics.DisplayText.

If I display anything else, I get buffer overflows, I assume because the processor cannot keep up.

Does this seem reasonable? My plans are to run Glide to display the data and support the touch screen for user input, but I am guessing that this will need more CPU than the simple graphics. Is that correct?

Does anyone have any pointers on how I can improve the performance? Will multiple threads help?

I did have a prototype of the UI running that used a timer to generate the GPS data (with no calculations) and that worked fine, but I am doubting that I will be able to run that while handling real GPS data.

2

Without seeing your code, we can only guess what is wrong.

I doubt that the processor is too slow for the application.

Reduce your code to the smallest possible size, while still displaying the issue, and post it here. Then, we can properly comment.

3

String operations, on 10 lines of text per second, will kill you if you’re not careful, so this would be my guess as to an area of focus in your code optimisation

One of the first things I would suggest is testing to see if you can sustain things at a 1Hz sample rate - and I’d ask, is 5Hz actually needed, or is a lower rate sufficient ?

4

Here is the code. I am using a modifies serial.cs as the standard one has memory problems with line received. The only change is to comment out the definition of the datareceived event so that it never gets called.

using System;
using System.Text;
using System.Collections;
using System.IO.Ports;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Presentation;
using Microsoft.SPOT.Presentation.Controls;
using Microsoft.SPOT.Presentation.Media;
using Microsoft.SPOT.Touch;

using Gadgeteer.Networking;
using GT = Gadgeteer;
using GTM = Gadgeteer.Modules;
using Gadgeteer.Modules.GHIElectronics;
using GTI = Gadgeteer.Interfaces;

namespace GadgeteerApp1
{
    public partial class Program
    {
        private static GTI.Serial serial { get; set; }
        static GTI.DigitalOutput ResetPin;
        private const string DELIMITER = "\r\n";
        private const double EarthRadiusMetres = 6376500;
        private const double deg2rad = 0.01745329251994329576923690768489;
        private string VTGString, GGAString;

        static double odo, tod, latitude, longitude;
        static double distance, totaldist = 0;
        static double lastlat, lastlong;
        static double speed;
        private static bool gpsOK = false;

        // Calibration and units
        double SecInDay = 3600 * 24;

        
        void ProgramStarted()
        {
            
            display_T35.SimpleGraphics.Clear();
            display_T35.SimpleGraphics.DisplayText("Rally GPS initializing...", Resources.GetFont(Resources.FontResources.NinaB), GT.Color.Magenta, 10, 10);
            GT.Socket socket = GT.Socket.GetSocket(7, true, null, "U");

            ResetPin = new GTI.DigitalOutput(socket, GT.Socket.Pin.Three, true, null);

            serial = new GTI.Serial(socket, 115200, GTI.Serial.SerialParity.None, GTI.Serial.SerialStopBits.One, 8, GTI.Serial.HardwareFlowControl.NotRequired, null);
            serial.LineReceivedEventDelimiter = DELIMITER;
            serial.AutoReadLineEnabled = true;
            Thread.Sleep(500);
            //serial.DiscardInBuffer();
            
            serial.Open();   
            serial.LineReceived += new GTI.Serial.LineReceivedEventHandler(serial_LineReceived);
 
        }

        void serial_LineReceived(GTI.Serial sender, string line)
        {
            if (line.Substring(4, 3) == "VTG")
            {
                VTGString=line;
                processgps();
            }
            else if(line.Substring(4, 3) == "GGA")
            {
                GGAString = line;
            }
        }
        void processgps()
        {
            string VTG, GGA, ts;
            int hh, mm;
            double sec;
            VTG = VTGString;
            GGA = GGAString;
            speed = double.Parse(VTG.Substring(33, 7));
            hh = int.Parse(GGA.Substring(8,2));
            mm = int.Parse(GGA.Substring(10, 2));
            sec = double.Parse(GGA.Substring(12, 4));
            tod = (3600 * hh) + (60 * mm) + sec;
            ts = sec2HMS(tod);
            lastlat = latitude;
            lastlong = longitude;
            latitude=latitudefromgps(GGA.Substring(17,10));
            longitude = longitudefromgps(GGA.Substring(30, 10));
            if (gpsOK)   //gpsOK is set after the first pass through this routine and means tha lastlat and lastlong have valid locations so we can start tracking.
            {
                distance = gpsdistance(lastlat, lastlong, latitude, longitude);
                totaldist += distance;
                odo = totaldist/1000.0;
            }
            display_T35.SimpleGraphics.Clear();
            display_T35.SimpleGraphics.DisplayText(speed.ToString("F2"), Resources.GetFont(Resources.FontResources.NinaB), GT.Color.Magenta, 10, 10);
            display_T35.SimpleGraphics.DisplayText(ts, Resources.GetFont(Resources.FontResources.NinaB), GT.Color.Magenta, 10, 40);
            //display_T35.SimpleGraphics.DisplayText(latitude.ToString("F8"), Resources.GetFont(Resources.FontResources.NinaB), GT.Color.Magenta, 10, 70);
            //display_T35.SimpleGraphics.DisplayText(longitude.ToString("F8"), Resources.GetFont(Resources.FontResources.NinaB), GT.Color.Magenta, 10, 100);
            display_T35.SimpleGraphics.DisplayText(odo.ToString("F3"), Resources.GetFont(Resources.FontResources.NinaB), GT.Color.Magenta, 10, 130);
            gpsOK = true;

        }
        String sec2HMS(double t)
        {
            double thetime;
            string hms, tmp;
            int h, m;
            double s;

            thetime = t;
            s = thetime / 3600.0;
            h = (int)s;
            thetime = thetime - (h * 3600);
            s = thetime / 60.0;
            m = (int)s;
            s = thetime - (m * 60);
            tmp = h.ToString("N0");
            if (tmp.Length == 1) tmp = "0" + tmp;
            hms = tmp + ":";
            tmp = m.ToString("N0");
            if (tmp.Length == 1) tmp = "0" + tmp;
            hms = hms + tmp + ":";
            tmp = s.ToString("F1");
            if (tmp.Length == 3) tmp = "0" + tmp;
            hms = hms + tmp;
            return hms;
        }
        double latitudefromgps(string s)
        {
            double degrees=double.Parse(s.Substring(0,2))+(double.Parse((s.Substring(2)))/60);
            return degrees;
        }
        double longitudefromgps(string s)
        {
            double degrees = double.Parse(s.Substring(0, 3)) + (double.Parse((s.Substring(3))) / 60);
            return degrees;
        }
        double gpsdistance(double lla, double llo, double cla, double clo)
            // lla= last latitude, cla=currenlt latitude, llo and clo and longitude equivalents.
            // llar=lla expressed in radianns etc.
        {
            double llar = lla * deg2rad;
            double llor = llo * deg2rad;
            double clar = cla * deg2rad;
            double clor = clo * deg2rad;
            double dlat = clar - llar;
            double dlong = clor - llor;
            double a = System.Math.Pow(System.Math.Sin(dlat / 2.0), 2) + (System.Math.Cos(lla) * System.Math.Cos(cla) * System.Math.Pow(System.Math.Sin(dlong / 2.0), 2));
            double c = 2.0 * System.Math.Atan2(System.Math.Sqrt(a), System.Math.Sqrt(1 - a));
            double d = c*EarthRadiusMetres;
            return d;
        }


    }
}
5

Using code tags will make your post more readable. This can be done in two ways:[ol]
Click the “101010” icon and paste your code between the

 tags or...
Select the code within your post and click the "101010" icon.[/ol]
(Generated by QuickReply)
6

@ Brett

Yes, 5Hz is needed for this application.

7

@ Architect: Done! That definitely helps.

8

Have you considered doing away with strings and going back to byte arrays? May be worth a test to see if that will improve things for you. I’d suggest you poll the serial port into a circular buffer of known size and wrap your own end-of-line detect logic into there. There’s a post from many moons ago from William that has a good routine for doing this; no promises but I’ll try to find a reference to it.

Strings are bad because they cause memory churn and GC cycles. Using byte arrays of fixed sizes is more efficient. (Personal comment: Gadgeteer’s serial port handlers are there to make it easy to make stuff easy to handle, not necessarily optimised for throughput, and in your case throughput will be king)

9

Looking quickly, I would try to avoid substring() and pre-fetch the fonts instead of constantly going to the resources.

I would consider adding a queue, and in the event handler only add the line to the queue. I would then have a thread that processes the queue. If there are more than one entry in the queue, then I would discard the older entries. This would allow you to get things running, and then work on optimizing the code.

10

So there is no GPS data everything is working well? Makes me wonder what happens if GPS is done natively.

In other words, how much processing power does GPS decoding needs.

11

@ pi - My suggestions:

1.) Use string.Split(’,’) instead of string.Substring() to extract the raw fields from your GPS sentence. NMEA sentences are always comma-delimited, and the number of commas is always fixed, so extracting the fields is easy if you partition along commas. In my tests, string.Substring() is [em]3X slower[/em] than string.Split(’,’) when parsing the same number of sentences.

2.) Avoid string operations as much as possible. For GPS parsing, the most string-intensive part is extracting the fields from the commas (which can easily be done using the Split() method). Once you have the fields, convert them to numeric variables as soon as possible and manipulate them as numerics. Don’t go back to strings at all. String manipulation will kill your speed.

3.) When at all possible, avoid string.Substring() like the plague. It is slooooow. If you find yourself using Substring(), ask yourself if there’s something else you can use in its place.

Here’s a concrete example:

These 2 functions are slow because they use string manipulation twice per call:

double latitudefromgps(string s)
        {
            double degrees=double.Parse(s.Substring(0,2))+(double.Parse((s.Substring(2)))/60);
            return degrees;
        }
        double longitudefromgps(string s)
        {
            double degrees = double.Parse(s.Substring(0, 3)) + (double.Parse((s.Substring(3))) / 60);
            return degrees;
        }

I would suggest something that is as numerically-based as possible, such as:

/// <summary>
        /// Converts minutes to degrees.
        /// </summary>
        /// <param name="minutes">Minutes to convert.</param>
        /// <returns>double</returns>
        private static double MinToDeg(double minutes)
        {
            return minutes / 60.0;
        }

        /// <summary>
        /// Converts the raw latitude/longitude string from a GGA message to decimal degrees.
        /// </summary>
        /// <param name="latLon">
        /// Latitude/longitude string of the form DDMM.MMMM (for latitude) or DDDMM.MMMM (for longitude) taken directly from a GGA sentence.
        /// </param>
        /// <param name="dir">
        /// Latitude/longitude direction taken directly from a GGA sentence: "N", "S", "E", or "W" (case-sensitive).
        /// </param>
        /// <returns>
        /// Latitude or longitude as decimal degrees.
        /// </returns>
        private static double RawGGALatLonToDeg(string latLon, string dir)
        {
            double dblRaw = double.Parse(latLon);           // numeric version of the raw input string
            double deg = MathEx.Truncate(dblRaw / 100.0);   // whole degrees portion
            double minutes = dblRaw % 100.0;                // modulus 100 to extract minutes portion (the remainder)
            deg = deg + GPS.MinToDeg(minutes);              // convert minutes to decimal form and add to whole degrees portion

            if (dir == "S" || dir == "W")
            {
                deg = -1.0 * deg; // southern and western latitudes are negative by convention
            }

            return deg;
        }

My tests show that the latter numeric-based methods are 21.43% faster than the string-based ones.

12

I would suggest to use as often as you can static variables. Then I would remove the event handler and would perform polling. My opinion is that event is an easy way to manage serial port but experiments shows that for continuous communication it awakes too often.

13

[quote=“pi”]I can receive the GPS data, perform some basic calculations like compute distance traveled and output 3 values to the LCD screen using SimpleGraphics.DisplayText.

If I display anything else, I get buffer overflows, I assume because the processor cannot keep up.
[/quote]
I think it is the drawing code, rather than the GPS string processing, which is your bottleneck. By default, SimpleGraphics redraws the screen every time anything on it changes, which is very bad for performance.

Try setting SimpleGraphics to not redraw on every drawing command and instead require an explicit Redraw command. This involves setting SimpleGraphics.AutoRedraw to false, replacing SimpleGraphics.Clear() with SimpleGraphics.ClearNoRedraw(), and adding a call to SimpleGraphics.Redraw() after you’ve done your drawing. Also, pre-fetch your NinaB font into a variable just once and hang on to it, instead of fetching it on every call to DisplayText. This should speed up your display code significantly.

14

I think it is the drawing code, rather than the GPS string processing, which is your bottleneck. By default, SimpleGraphics redraws the screen every time anything on it changes, which is very bad for performance.

Try setting SimpleGraphics to not redraw on every drawing command and instead require an explicit Redraw command. This involves setting SimpleGraphics.AutoRedraw to false, replacing SimpleGraphics.Clear() with SimpleGraphics.ClearNoRedraw(), and adding a call to SimpleGraphics.Redraw() after you’ve done your drawing. Also, pre-fetch your NinaB font into a variable just once and hang on to it, instead of fetching it on every call to DisplayText. This should speed up your display code significantly.
[/quote]

Totally agree, ive messed arround before with GPS and the display. I ended up using WPF (which you would think would be slower) but if i remember corectly allows you to redraw individual items (or text items) rather than the whole screen. Simple Graphics always flushes the whole screen.

15

@ pi - Maybe this will help you find the bottleneck?

http://www.ghielectronics.com/community/codeshare/entry/705

16

That’s a lot of useful input.

So far I have modified the SimpleGraphics commands to turn off AutoRedraw and only redraw the screen once all the data is written. This is a big improvement, I can write a lot more text with no sign of a buffer overflow.

My understanding is that Glide only updates the screen when you tell it, so maybe that will work out for me.

My next step is to improve the GPS processing with the Split option to replace the substring usage.

17

@ pi - I think you can optimise the time function.
first you parse the timestring to seconds and then recalculate a string?
Also your time calculation mixes int and and double values and so many converting functions where used
If you realy need the parsed time then use the SPOT.Time objekt and use the .ToString function.

18

I parse the time string to seconds because I need to manipulate the time value. I need to supply a time zone offset and then a user supplied offset, which is usually a few seconds. Car rallies always have a master clock that everyone has to synchronize to. Organizers usually try to make it accurate but I find there is usually 1 or 2 seconds difference between rally time and GPS time.

I have made the time decoding more efficient by doing it with math rather than string manipulation.

I now have the system running with all the math I need included, so now I will start to add the UI.

19

Thought I would publish the current status of this.

I implemented the suggested changes but when I added the Glide UI to display the information I needed I was still getting buffer Overflows.

The solution was to update the display at 1Hz. The data from the GPS is received and processed at 5Hz. The display updating is performed in a separate thread that just loops with a ‘while(true)’ statement and suspends itself when the display has been updated.

The GPS precessing thread resumes the display thread when the decimal seconds is zero.

Some code snippets:



        static Thread DispThread;


            DispThread = new Thread(UpdateDisplay);


// In gps processing code:

                if (Decimalseconds == 0)
                {
                        DispThread.Resume();
                }

//display thread:
       void UpdateDisplay()
        {
            while(true)
            {
                //display stuff here
                Glide.MainWindow = windows[0];
                Thread.CurrentThread.Suspend();
            }
        }
20

@ pi - Perhaps you also want to look into System.Threading.AutoResetEvent and System.Threading.ManualResetEvent? They’re typically used to have task(s) wait for something to happen before proceeding.