Problem with Datalogger application

I am working on a weather Datalogger project on EMX tool. My basic requirement is

  1. 1 Minute data logging from sensors (attached with RS232 port,)
  2. Store data in SD card after data logging.
  3. Represent read data on GUI.

To achieve this I have made two threads.

  1. Global thread fires every 1 minute
    1.1) Communicate with Sensors on RS232
    1.2) Get data from sensor and store in global variable.
    1.3) Set global variable flag true for other timers indication
    1.4) Write data in SD card (csv file)

  2. GUI Representation class level (local) timer fires every 5 second
    2.1) Check the global variable flag.
    2.2) If it is true then read sensor parameters form global variable and represent in GUI.
    2.3) I have used lock mechanism in both the timers while accessing global data.

The application works properly, it collects data from sensors and stores data in csv file of SD card. The problem is consistency. The data logging from sensor and storing in SD card timer (global timer) gets hang after around 9 hours. The GUI timer (local timer) is live when Global timer gets hang.

For testing, I have tried with commenting data storage on SD card and it works properly without hang condition. This means that hang cause because of SD storage. My SD storage class is simple and its AppendStringToFile function writes data in to SD Card. I have made the static object of this class in Global thread class and initialize SD card. In Global thread I am calling AppendStringToFile function for writing.

AppendStringToFile functions code is as under.



       public Boolean AppendStringToFile(String stringToWrite,string fname)
        {
            Boolean success = false;

            ProsName = "AppendStringToFile";
            Data = "";


            lock (this)
            {
                try
                {
                    filePath = rootdirectory + "\\" + fname;
                    // The using statement ensures that Dispose is called on fileHandle even if an exception occurs.                    
                    filehandle = new FileStream(filePath, FileMode.Append);                   
                    stringToWrite = stringToWrite + "\r\n";
                    byte[] databt = Encoding.UTF8.GetBytes(stringToWrite);
                    filehandle.Write(databt, 0, databt.Length);
                    filehandle.Close();
                }

                catch (Exception ex)
                {
                    Fnlstr = DateTime.Now.ToString() + "," + "Class Name:-" + ClsName + "," + "Procedure Name:-" + ProsName + "," + "Data:-" + Data + "," + "Error Msg:-" + ex.Message;
                    Glbsd.SDAppendStringToFile(Fnlstr, "ErrorLog.csv");
                    Debug.Print(ex.Message);
                }
            }
            return success;
        }


Sorry for the long messge but i think its require to know the problem and to provide solution.

I’d try someone else’s SD logging code to see if their approach is different. Here’s something that might be beneficial:

[url]http://www.fezzer.com/project/169/sd-card-program-logging/[/url]

And then I’d try adding debugging to a file to see if there’s anything specific you can find.

Do you know the line in your code where it hangs?
How big is the string you write to the file?
Do you start with empty file and then after 9 hours it happens or is there a file already that you are appending to.

9 hours * 1 string a minute = 540 strings written before it hangs. Is that correct?

Do you know the line in your code where it hangs?

Ans:- I don’t know the line but when i have comment calling “AppendStringToFile” function. it works properly. I have tried with error handling too but no error file generates in SD card.

How big is the string you write to the file?

Ans:-My string sample is “01/01/2009 00:15:15,-019.00 C,067.00 %,1079.0 hPa,05:14 hh:mm,002.00 m/s,0140.0 Deg,0523.00 mm”

Do you start with empty file and then after 9 hours it happens or is there a file already that you are appending to.

Ans:-I start with empty file, when i first time invoke the “AppendStringToFile” it will create the file in SD card.

9 hours * 1 string a minute = 540 strings written before it hangs. Is that correct?

yes, uptill now the maximum strings written in the file is 494.

I think somebody had similar issue before. Search the forum.
As a workaround you can split logging into smaller files 200 entries per file or so for example.

Monitor the debug output. Do you get any exceptions at all?
also, make sure your program is still working. I mean if some thread exits, it seems that it is hanging

I have split the loging in to smaller files as 60 entries per file. It works upto 14.30 hours and gets hang.

I have check that the one thread is still exist which represents data in GUI, But the data acquisition and storage thread gone hang.

I didn’t get exception messge but I got below list at output window of VC2010. Please help if any one get hint by this output message.

[line]
GC: 411msec 449976 bytes used, 13180860 bytes available
Type 0F (STRING ): 4128 bytes
Type 11 (CLASS ): 40716 bytes
Type 12 (VALUETYPE ): 636 bytes
Type 13 (SZARRAY ): 7296 bytes
Type 15 (FREEBLOCK ): 13180860 bytes
Type 16 (CACHEDBLOCK ): 408 bytes
Type 17 (ASSEMBLY ): 38064 bytes
Type 18 (WEAKCLASS ): 192 bytes
Type 19 (REFLECTION ): 216 bytes
Type 1B (DELEGATE_HEAD ): 2160 bytes
Type 1C (DELEGATELIST_HEAD ): 96 bytes
Type 1D (OBJECT_TO_EVENT ): 312 bytes
Type 1E (BINARY_BLOB_HEAD ): 345912 bytes
Type 1F (THREAD ): 1152 bytes
Type 20 (SUBTHREAD ): 144 bytes
Type 21 (STACK_FRAME ): 2400 bytes
Type 22 (TIMER_HEAD ): 144 bytes
Type 27 (FINALIZER_HEAD ): 960 bytes
Type 31 (IO_PORT ): 324 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 4644 bytes

It is hard to say without seeing more code.
Have you tried using different card (brand, size)?

The status information you are seeing below is fine, this is just the output that the Garbage Collector produces as it operates.

You said that the other threads have hung - how do you know? And you say the UI is still operating, does that mean you can still interact with the device?

At the ~14 hour mark, what actually happens? Can you issue a break in VS and see where the code is up to? Can you add debug.print statements at strategic points in the code to see what is going on immediately before ?

I think I know the problem! You are losing some resources when the GC kicks in and this is why…please read this microframeworkprojects.com

Thanx Gus, you are right that you got the problem.

Actually GC was removing my object which I have declared static and define in program.cs. The object is responsible for running 1 minute timer thread written in class and as the object dispose/removed by GC the whole timer thread stops and the SD storage stops.

I have checked with initializing local object in a class and defining 1 minute timer thread in same class and it works properly.

But, I want to define the 1 minute thread globally so, irrespective of any classs dependency it should acquire sensor data and store sensor data in SD card. I have tried with defining the class object in program.cs but GC is removing it though the timer is executing every 1 minute.

Please guide me how to achieve this. I may have wrong idea but can be stop GC to execute? This may help but I dont know it may occur other problems of not.

All you have to do is keep a live reference to your object

live reference? didn’t get you, can you please explain through example?

This is explained here with example microframeworkprojects.com