Log on SD card

hi,
I have a Spider device.
I want to implement a log, that will be written to a SD card.
According to: https://www.ghielectronics.com/docs/96/sd-card-module
I must call

sdCard.MountSDCard();

before using the SD card.
And I saw that if I dont call:

sdCard.UnmountSDCard();

before I’m taking out the card, the log file is not saved.
My device is connected to the network, and number of times the device has stopped responding to pings. I want to find the problem with the log file.

if I call, for every log-write the function:


        public void WriteToLog(string line)
        {
            //Mount the SD card
            _sdCard.MountSDCard();
            //Use FileStream to read a text file
            var fs = new FileStream(_logFilePath, FileMode.OpenOrCreate);
            //load the data from the stream
            fs.Position = fs.Length;
            var bytes = Encoding.UTF8.GetBytes(line + '\n');
            fs.Write(bytes, 0, bytes.Length);
            fs.Close();
            _sdCard.UnmountSDCard();
        }

It’s slow the program down. the function

 _sdCard.UnmountSDCard();

takes alot of time.
if I dont call

_sdCard.UnmountSDCard();

, I saw that when I’m taking out the card the file has not been saved.

  1. So when should I write:
 _sdCard.UnmountSDCard();

?
2. Is there an event for Reset button press? If yes, I can register to it, and call there:

 _sdCard.UnmountSDCard();

. and I will press the reset button before I want to take-out the card.
3. Do you know any-situation of the device, that which causes the device not respond to pings?
Thanks, Matan

@ Matan - Mount SD card once in for example main() When you want to log something… Open the log file, write something, Close the log file and Flush the buffer directly after your write operation

I’ve had issues where I decided it’s best to ensure the SD card is really still mounted before writing to it. Here is my logger code that logs 10 entities to a CSV file about every 5 seconds.

 while (!sdCard.IsCardMounted)
                        {
                            sdCard.MountSDCard();
                            Thread.Sleep(500);
                        }
                        Thread.Sleep(5000);

                        string rootDirectory = "\\SD";
                        string path = rootDirectory + @ "\log.csv";

                        if (!File.Exists(path))
                        {
                            // Create a file to write to. 
                            StreamWriter sw = new StreamWriter(path);
                            sw.WriteLine("Counter,Pump1Off,Pump2Off,Pump3Off,Pump4Off,Ana1,Ana2,Ana3,Ana4,DateTime");
                            sw.Flush();
                            sw.Close();
                            Thread.Sleep(250);
                        }

                        StreamWriter logFile = new StreamWriter(path, true);
                        Thread.Sleep(250);
                        logFile.WriteLine(logger());
                        logFile.Flush();
                        logFile.Close();

I create “new” streamwriters every time to ensure things are a clean slate.

Gadgeteer.

logictechs,
I tried your code. But it is still has the problem:
even if I do:

logFile.WriteLine(dataToWrite);
                        logFile.Flush();
                        logFile.Close();

and after that I remove physically the card, the data is not saved to the file.
But when I do:

logFile.WriteLine(dataToWrite);
                        logFile.Flush();
                        logFile.Close();
                        sdCard.UnmountSDCard();

and after that I remove physically the card, the data is saved to the file.
But to do so:

sdCard.UnmountSDCard();

for every log-writting, its slows down significantly the flow.

@ andre.m - now I think that I got what you asked…
In my ‘Solution’ there is a Gadeteer ‘Project’, and the log is in other micro-.Net library ‘Project’.
The run of the program starts form the Gadgeteer, and it uses few library-projects that I implement.
Did I understand you?

@ Matan - I’m not a gadgeteer user, so i am not sure if this works for you… What i do before i power down a board is



using Microsoft.SPOT.IO

void FlushVolumes()
{
        VolumeInfo[] vi = VolumeInfo.GetVolumes();
        foreach (VolumeInfo volumeInfo in vi)
        {
            volumeInfo.FlushAll();
         }
}

I’m having a similar problem with my CerbuinoBee. I’ve also found that if the SD card is not unmounted before the board is powered off (or card removed) anything that has been written to the card beforehand is lost. I’ve been having to mount the card, write what I want and unmount each time to get successful writes.

The problem is I’m using the Cerbuino as a data logger at the moment and the mount takes around 1.2 seconds (I timed the whole process last night), which is no good for a data logger that needs to log at 20hz or higher. If anyone has a solution to this I’d be really interested.

BTW, I’m using native NETMF.

@ RobvanSchelven - the .Net-micro-framework does not includes the namespace:

using Microsoft.SPOT.IO

:frowning:

@ Matan - Did you add a reference to Microsoft.SPOT.IO ?

@ wolfbuddy - With my logger code in a Cerbuino with connect microSD card module, I can reliably log 4 digital inputs and 4 analog inputs to a log.csv file every 5 seconds. The key thing is, you need to close the file after writing to it and not unmounting it. Then, when you want to see the data, tell it to stop logging so the file is not open when you take the card out or want to send it out via some sort of connection(serial, Bluetooth, cellular, etc.) to the board.

To all having problems with SD card ops, make sure you’re using the latest firmwares also! The last version was very flawed for my ops.

When you say close the file, do you mean close the file stream?

Here’s my method, please don’t laugh!


public void FileWrite(string Text, string File)
        {
            string rootDirectory = VolumeInfo.GetVolumes()[0].RootDirectory;
            FileStream TextInStream = new FileStream(rootDirectory + @ "\" + File, FileMode.Append, FileAccess.Write);
            if (TextInStream.CanWrite)
            {
                TextWriter TextIn = new StreamWriter(TextInStream);
                TextIn.WriteLine(Text);
                TextIn.Flush();
                TextInStream.Flush();
                TextIn.Close();
                TextInStream.Close();
                TextIn.Dispose();
                TextInStream.Dispose();
            }
            else
            {
                throw new Exception("Cannot write to file:" + File);
            }
            
        }

I’ve added the close methods just now.

I’ve also altered the flow of my data logger so that it checks to see if the card is mounted each time it logs, and only mounts it if it’s not mounted already. Then it unmounts when logging is stopped by a particular event. It’s now logging at nearly 30hz which is much more like it! :smiley:

@ wolfbuddy - Yes, I make sure to close the any sd card streams after reading or writing ops are done each time. For example, open the file, read what you need into an array, then close it. Open the file, write what you need, then close it. Never leave it just hanging open waiting for a disaster to screw up the file. Which brings up the idea of adding a battery backup incase of power failure.

With the 5 second rate I’ve done, that for writing a string line of comma delimited data readable by end user if they want to pop out the card for direct access. I’m kinda rethinking now to have both a CSV file and raw data file to allow faster data collection. My application involves checking if power is on and what fluid pressure is applied to a sensor being read by a 4 to 20mA board to determine gallons pumped out of a tank. So it doesn’t need to be really fast.

Thanks logictechs.

@ RobvanSchelven - your code works! thanks!
but I found better code:

sdCard.GetStorageDevice().Volume.FlushAll();

its also does the work.

@ Matan - Good to hear!.. didn’t know the one you proposed. Thanks!

hi all:

  1. I solved one problem: if I extract the SD-card - I still can the read the log file (the FlushAll method did the work).
  2. when I insert the SD-card again, in the middle of the runnig, I want it to continue the log from the insert point. BUT it stops working.
    the code:
     
               //in case the SD card has been extracted 
                if (!sdCard.IsCardInserted && IsCardWasInserted)
                {
                    IsCardWasInserted = false;
                    return;
                }

                //in case there is a SD card that was inserted 
                //(and there was not card in the last time)
                if (sdCard.IsCardInserted && !IsCardWasInserted)
                {
                    sdCard.MountSDCard();
                    storageDevice = sdCard.GetStorageDevice();
                    storageDevice.Volume.Refresh();
                    filePath = storageDevice.RootDirectory + @ "\" + LOG_FILE_NAME;
                    IsCardWasInserted = true;
                }

                //in case the SD card initialize was failed or there are no card inserted
                if (!IsCardWasInserted || storageDevice == null)
                {
                    return;
                }

Some thing strange:
if I start the Gadgeteer without a SD-Card inserted, and after that I insert a card, this code works. but if I extract, at the same running, the SD card and insert it again, the log file does not has any new log lines (Although the debugger shows that everything looked fine).

My code that writes to the file is very simple:


                var sw= new StreamWriter(filePath, true);
                sw.WriteLine(line);
                sw.Flush();
                sw.Close();
                //flush the file
                storageDevice.Volume.FlushAll();

again: this code works fine (-I can take out the card in any time, and all line that will be there), but at the second insert, the file stops to be updated with the new log lines.
(Although the debugger shows that everything looked fine).

why don’t you look at using the insert and removal events to track the status as well, and make sure the state is consistent before writing?

@ Brett - What events?

Microsoft.SPOT.IO.EjectEventHandler and Microsoft.SPOT.IO.InsertEventHandler