Trying to understand GHI SplitToArray Utility

I’m running the example shown at https://www.ghielectronics.com/community/forum/topic?id=11534&page=1 with my attempt at using this utility tacked on at the end. I’m not getting the response I expect nor does it look like the GHI code gives the expected results.


using System;
using Microsoft.SPOT;

using System.Text;

namespace GHISplitToArray
{
    public class Program
    {
        public static void Main()
        {
            Debug.Print("Program Started");

            //Start of example from GHI Forum
                    string gpss = 
                ".3372,W,1,8,1.03,61.7,M,55.2,M,,*76" +
                "$GPGGA,092750.000,5321.6802,N,00630.3372,W,1,8,1.03,61.7,M,55.2,M,,*76" +
                "$GPGGA,092750.000,5321.6802,N";

            byte[] gps = System.Text.UTF8Encoding.UTF8.GetBytes(gpss);//we are faking GPS stream for this test

            byte[] CSVLine = new byte[256];//our buffer to help a line CSV comma separated values
            float[] values = new float[100];


            ///////////////// GPS code //////////////////
            // find the start and end of the line we care about
            int lineStart = Array.IndexOf(gps, '$');
            int firstComma = Array.IndexOf(gps, ',', lineStart);
            firstComma++;//skip the comma
            int lineEnd = Array.IndexOf(gps, '*', lineStart);
            int length = lineEnd - firstComma;

            Array.Copy(gps, firstComma, CSVLine, 0, length);

            // "CSVLine" now holds a single line of the length "length"...looks like this
            // 092750.000,5321.6802,N,00630.3372,W,1,8,1.03,61.7,M,55.2,M,,
            
            // GHI will add this
            // a comma separated values CSV will be extracted and put in the float array
            // in this example it use ',' but this can be anything!
            int floatCount = GHI.Utilities.Arrays.SplitToArray(CSVLine, length, ',', values);
            /* from values above it will be
             * values[0] = 092750.000;
             * values[1] = 5321.6802;
             * values[2] = (float)'N';
             * ...etc.
            */
            ///////////////////////////////////////////////////
            
            
            ///debug test
            /////show the byte array
            string extracted = new string(System.Text.UTF8Encoding.UTF8.GetChars(CSVLine));
            Debug.Print("GHI forum example: https: //www.ghielectronics.com/community/forum/topic?id=11534&page=1");
            //show the decoded values
            for (int i = 0; i < floatCount; i++)
            {
                Debug.Print(values[i].ToString());
            }
            Debug.Print(extracted);
            //End of GHI example from forum
            
            Debug.Print("\r\nNow my turn");

            String stringArray = "1.0,2.0,3.5,5.0,10.123";
            byte[] byteArray = UTF8Encoding.UTF8.GetBytes(stringArray.ToString());
            int returnValue = -1;
            float[] splitValues = new float[32];

            returnValue = GHI.Utilities.Arrays.SplitToArray(byteArray, stringArray.Length, ',', splitValues);

            StringBuilder inputBytes = new StringBuilder(128);
            inputBytes.Clear();
            inputBytes.Append(UTF8Encoding.UTF8.GetChars(byteArray));

            Debug.Print("Input byte array (as characters): " + inputBytes.ToString());
            Debug.Print("Return value = " + returnValue.ToString());

            for (int i = 0; i < 5; i++)
                Debug.Print(splitValues[1].ToString());

        }

    }
}

Here’s what shows up in the output window

Program Started
GHI forum example: https: //www.ghielectronics.com/community/forum/topic?id=11534&page=1
10.1230927
5321.68018
78
630.337219
87
1
8
1.02999997
61.7000008
77
55.2000008
77
nan
nan
092750.000,5321.6802,N,00630.3372,W,1,8,1.03,61.7,M,55.2,M,

Now my turn
Input byte array (as characters): 1.0,2.0,3.5,5.0,10.123
Return value = 4
2
2
2
2
2
The thread ‘’ (0x1) has exited with code 0 (0x0).
Done.

At the very least I’d expect the first value from the GHI code to be 92750.00.

What don’t I understand?

In your example
Debug.Print(splitValues[1].ToString());
should be
Debug.Print(splitValues[i].ToString());
to display all the splitValues array contents

Not sure why the value is 10.1230927 and not 92750.000

Thanks for the correction, my usual dumb mistake. But that doesn’t explain the results for the GHI example at the top of the code.

Here’s what I would have expected vs what the code printed out. The first result is way of and all the other byte arrays with floating point values give results with small but not insignificant errors.

Code results            Expected results
10.1230927                 92750.000
5321.68018                 5321.6802
78                                  78  
630.337219                 630.3372
87                                  87
1                                    1
8                                    8
1.02999997                1.03
61.7000008                61.7
77                                 77
55.2000008               55.2
77                                 77
nan                              nan
nan                              nan

@ Gene - What version of the firmware are you using? I ran the code exactly as you posted and the values came out expected. The first result was 92750.000.

As for some numbers being slightly off, that is because we use the float data type which has less precision than a double so it cannot represent the numbers exactly.

John

Here is what FEZ Config says

Loader (TinyBooter) version information:
4.3.4.0 on this computer.
4.3.4.0 on this device.

The Loader (TinyBooter) is up to date. <<<

Firmware (TinyCLR) version information:
4.3.4.0 on this computer.
4.3.4.0 on this device.

The Firmware (TinyCLR) is up to date. <<<
Please wait for the device to reboot… Done.

Here are some more confusing results

 

            Debug.Print("\r\nNow my turn");

            String stringArray = "$dDepth,1.0,-2.0,3.5,5.0,10.123,";
            byte[] byteArray = UTF8Encoding.UTF8.GetBytes(stringArray.ToString());
            int returnValue = -1;
            float[] splitValues = new float[32];

            Debug.Print("stringArray Length = " + stringArray.Length);
            string myExtracted = new string(System.Text.UTF8Encoding.UTF8.GetChars(byteArray));
            Debug.Print("byteArray (as string) = " + myExtracted);
            
            returnValue = GHI.Utilities.Arrays.SplitToArray(byteArray, stringArray.Length, ',', splitValues);

            StringBuilder inputBytes = new StringBuilder(128);
            inputBytes.Clear();
            inputBytes.Append(UTF8Encoding.UTF8.GetChars(byteArray));

            Debug.Print("Input byte array (as characters): " + inputBytes.ToString());
            Debug.Print("Return value = " + returnValue.ToString());

            for (int i = 0; i < returnValue; i++)
                Debug.Print(splitValues[i].ToString());
       

and the results
Now my turn
stringArray Length = 32
byteArray (as string) = $dDepth,1.0,-2.0,3.5,5.0,10.123,
Input byte array (as characters): $dDepth,1.0,-2.0,3.5,5.0,10.123,
Return value = 7
nan
1
nan
3.5
5
10.1230001
nan

A couple of issues.

  1. Shouldn’t splitToarray be able to handle the “-2” field in the byteArray? Hopefully, I’m doing something wrong or this is a pretty fatal issue.

2)This is the only string2number routine I’ve ever used that rounds off like this. It may not be a problem for many applications but it would sure be nice to fix it.

  1. It appears you need the trailing token (‘,’ in this case) to get the last value (10.123) to be split and that seems to force the last nan in the results. This works with a NMEA string since the last field is usually a check sum and it is preceeded with a ‘,’ token and it should return nan. Unfortunately, this isn’t ideal for other non-NMEA strings. For example, simple JSON or JSON-like files could easily be parsed with this function if you didn’t need the trailing comma.

Is the source code available? I’d love to take a look and see if I can warp it into something that fits my requirements a little better.

Thanks for the help.

@ Gene - I doubt it’ll fix it, but you can test on 4.3.6 just to be sure?

For the rounding issue, the only way to reduce the occurrence of rounding is to use a double data type since it has much higher precision. That is something we will look into.

This function was designed for NMEA strings, so it will likely have issues with other formats but we will look into generalizing it more.

We do not make the source available, but you should be able to make a generalized version in C# by splitting on the ‘,’ and trying to parse each entry as a double with special handling for single ASCII characters if needed.

@ John

Got it. Did you look into the issue with splitting negative numbers?

@ Gene - I was able to reproduce the negative number issue. We will be reviewing the design of this function.

Thanks