Need help to speed up my code

NETMF & Gadgeteer
1

Hi Everyone,

I need to perform data logging of 8 sensors at 100Hz. These sensors send float data over ethernet as UDP streams by packet of 20 samples every 200ms. The length of each stream is of size 140 bytes. Here’s the code I wrote to extract data and write it to a file:


 public static void poll_socket_data(ref UsuVal[] tab_v)
        {
            try
            {
                if (_server_data != null)
                {
                    while (_server_data.Poll(1000, SelectMode.SelectRead))
                    {
                        
                        //_server_data.
                        var bufferin = new byte[_server_data.Available];
                        _server_data.ReceiveFrom(bufferin, ref ServerEndPoint_data);
                        vu = (bufferin[7] & 0x1E) >> 1;
                        int j = 0;
                        for (int i = 0; i < 77; )
                        {
                            tab_val[j] = ToFloat(bufferin, 18 + i);
                            j++;
                            i = i + 4;
                        }
                        v = "";
                        for (int z = 0; z < 20; z++)
                        {
                            v += ";" + tab_val[z];
                        }
                        cpt[(vu - 1) * 2 + bufferin[8] - 1]++;
                      Debug.Print(cpt[0] + "\t" + cpt[1] + "\t" + cpt[2] + "\t" + cpt[3] + "\t" + cpt[4] + "\t" + cpt[5] + "\t" + cpt[6] + "\t" + cpt[7]);
                        s+=vu+";"+ bufferin[8] +";"+ "20" + BitConverter.ToUInt16(bufferin, 10) + "\\" + bufferin[12] + "\\" + bufferin[13] + ";" + bufferin[14] + ":" + bufferin[15] + ":" + bufferin[16] + "," + (1000 - (bufferin[17] * 1000) / 256) + v;
                   /*   foreach (OpSense_Sensor o in op_sensor_list)
                        {
                            if (o.canal_op == bufferin[8] && o.mid == vu)
                            {
                                tab_v[o.channel].Affect(tab_val[19]);
                                break;
                            }
                        }*/
                    }
                    if(s!="")
                    {
                        Sdc.SDC_write_line_minimal("LOG\\OpSense.csv", s);
                        s="";
                    }
                }
            }
            catch (Exception e)
            {
                Debug.Print("Exception in Poll_socket_data" + e.Message + "\t" + e.StackTrace);

            }
        }



   public static void SDC_write_line_minimal(string fileName, string line)
        {
               try
                {
                    using (var sw = new StreamWriter("\\SD\\" + fileName, true))
                    {
                        sw.WriteLine(line);
                     }
                }
                catch (Exception e)
                {
                    Debug.Print("Exception" + e + "while writing " + line + " in " + fileName);
                }
              
        }

The applicaition is manly composed of a main loop performing other data measurement and logging tasks. In this loop I am colling the function poll_socket_data. This does not work well, only few second after starting, the number of messages available on the socket is too big and I do not succeed to write on the file fast enough.

Any help would be appreciate :wink:

2

Oops I forgot to do s="" after writing… now runtime is better but still need to be enhanced…

3

@ leforban - I see several possible gains, but without the whole context, it is difficult to tell with certainty if they are suitable.

4

@ SouthernStars - Please let me know what you want to know. The context is huge…

5

This is done on G400. 140bytes every 200ms times 8 sensors results in 5600 bytes per second, cyberh0me you’re right!

6

@ leforban - Some examples:

  • where and how are the non local variables declared
  • from where are your two methods called (use ‘find all references’ in VS)
  • how often they are called
    …
7
  • Pull writing to SD out to another task.
  • If write with append mode, keep the stream still open
    Those are my choices.
8

Maybe you could collect the sensor-values in a buffer (in RAM) for 1 second (or even more) to write more data at once on the SD-card. I don’t know details about the SD-Card-driver and the hardware behind, but maybe writing many small pieces takes a lot more time than just writing all 5600 bytes at once. There will be a delay of course, I don’t know if it’s a time-critical thing you’re coding :).
Is that used in a 24/7-project, or does it just work for some time? So maybe its possible just to store everything in RAM (the G400 does have a lot of it) and write it all at the end…