Hi, I referring the tutorial of Internet of Things to get the temp and light data.
Then I write it to sd card in domino by modifying the code from"Snippet — READ//WRITE FROM SD/USD CARD". http://code.tinyclr.com/project/97/readwrite-from-sdusd-card/
But I can’t get the high speed sampling and storage.
I set this,
Thread.Sleep(50);
I check the log file, but why I can only get 10 data per second?
how to improve ?
public static void Main()
{
// Blink board LED
bool ledState = false;
OutputPort led = new OutputPort((Cpu.Pin)FEZ_Pin.Digital.LED, ledState);
pStore = new PersistentStorage("SD");
pStore.MountFileSystem();
pStoreI = VolumeInfo.GetVolumes()[0];
// Initialize the eblocks
thermometer = new AnalogIn((AnalogIn.Pin)FEZ_Pin.AnalogIn.An2);
thermometer.SetLinearScale(-22, 56);
lightSensor = new AnalogIn((AnalogIn.Pin)FEZ_Pin.AnalogIn.An3);
lightSensor.SetLinearScale(0, 100);
// Begin sampling the temperature
running = true;
thread = new Thread(MainLoop);
thread.Priority = ThreadPriority.Highest;
thread.Start();
}
public static void MainLoop()
{
while (running)
{
temp = thermometer.Read();
light = lightSensor.Read();
temp = ((temp * 9) / 5) + 32;
Debug.Print("Temperature: " + temp + " F");
light = 100 - light;
Debug.Print("Brightness: " + light + "%");
string rootDirectory = VolumeInfo.GetVolumes()[0].RootDirectory;
idx = getIndex(rootDirectory);
string path = rootDirectory + @ "\log "+".csv";
DateTime time = DateTime.Now;
filePutContents(path,temp,light);
Thread.Sleep(50);
}
}
Dear Architect, Mike, thanks, I put it in the main, but the result is almost the same.
The theoretical maximum rate at 20 per second. but I only get 10, 11 or 12.
I need test it successful and then modified to sampling the 3axial accelerometer 200 per second.
It seems far from my goal.
And does netMF has Accurate timer? because I need to analyze the data using modal analysis method.
The accurate time interval is very important.
Has any code examples?
It is very important part in my project.
You need to optimize your code. Good chunk of the code can be moved out of the loop. For example you don’t need to get volume info every time. Define variables out of the loop and reuse.
If you can’t get more than 20/sec, perhaps your SD card is not very capable. You haven’t told us anything about your hardware, so please tell us more about that? Have you got a Panda II or a Domino (ie they have a fitted SD card holder)? What SD card are yuo using?
I have code doing about 48 samples/sec. That includes a debug.print or two that cause GC to go nuts so I am sure it can be improved. Edit: no debug.print gives me 63/sec. GC still goes nuts because I the encode and file IO is still in a byte array.
If you look at the portions of code that you have cobbled together to form this project, the “FilePutContents” code is potentially a time-wasting problem, it opens the file stream every time and then writes a paltry 32 bytes of data.
@ godFather89 To use your timer I can get the 20/s when setting the time interval to 50ms.
When I set time interval to 10ms, I can get the max 30/s.
@ Brett
My device is domino, the sd card is ScanDisk 1 GB micro SD card inserted to on board SD socket.
Yes, the “FilePutContents” code is potentially a time-wasting problem.
the FilePutContents method average take 30 ms.
My SD card is too slow?
I modified the code.
I can get 80/s. still not reach the 100/s.
And I don’t know why this code only can run on the debugging mode(F5).
When I re power the board.
The data logging don’t work.
Let me just say, the chance of you getting a great improvement in SD card write performance is not that great. So if you get 80 samples a second, you might be at about the limit of what the combination is capable of.
Certainly the SD card IO is going to be the slowest link in the chain I can see. For starters, you’re writing data into 4-bit chunks, so even if you were just writing raw data you’re making two writes per byte. Then you’re wrapping a file-system level call around it. Perhaps you’ll find a difference if you buffer your writes into larger chunks, because the loop you have creates and destroys the byte array each time - but handling the length then becomes a little trickier. You’re also writing ASCII, a great readable text format but pretty abysmal for data size
But lets go back to your problem. Do you really need 100 samples logged in a second to an SD card, of temperature and light? The chance of temperature reading changing in a second is pretty low, let alone in 1/100th of a second. The chance of light changing in a meaningful way in that time is also not very high. If you can’t make 100/sec, what’s the impact to your project or whatever it is?
This code is for me to learning the SD storage.
My final goal is to sample the 3 axis accelerometer.
In my application, the sampling rate should be 200 Hz.
So do I need use RLP to do that?
Only me use NETMF to do the project in my company.
I can’t get any support.
Sometimes a simple problem maybe takes me several days.
So many times feel frustrated.
Anyway, I have many good guys here to help me.
Back to my problem.
Why when I take the
FileStream fHandle = new FileStream(path, FileMode.Append, FileAccess.Write);
outside the Timer and delelet the
fHandle.Close()
,
Why this code only can run on the debugging mode(F5).
When I re power the board.
The data logging don’t work.
In this case you should write your values in a big byte array (size equal to a multiple of a reading size) that you write to the file only when it’s full. So forget ASCII.
You can see that I write the binary value of the 2 ints in the buffer. Each reading is 8 bytes long (4 bytes each int). Most sensors give 8 or 16 bit readings so this could be even better. When the buffer is full (offset == size) each 512 (4096 / 8 ) readings, the buffer is written to the file. You don’t need to save the time since you know each sample is taken after 10 ms. Also, don’t do any calculations. It’s only a logger so any calculations will be done in the log viewer. If this won’t be fast enough, only RLP can be faster but you can’t access the file system from RLP only if you go back to the managed side.
If you need to do the logging very fast for a few seconds, you could use a trigger (to start the logger) and a big buffer and only when the time is over save the actual readings to the file.
The problem is NOT something that RLP will help you with, from what I know about RLP. The problem is with the amount of data you’re trying to lay down on a relatively slow storage device in an SD card.
Can you design a solution that has high speed memory? How many samples do you actually need to sample at that rate? 1sec? 10 Sec? Build a memory solution and once full, push that to SD card for later analysis.
Onto your debugging issue. What do you want to achieve with the timer? I like simple. If all you’re trying to do is to see how fast you can push data onto SD card, then why not go back to the way you were running originally? Here’s what I would do to try and sort it out, the way I have it running it’ll give you a fast-flash LED before it starts the data write, then a slow-flash after logging for (currently 11 secs). I decided to write out time.ticks since that was something you were likely to want to see (80+ samples with the same time doesn’t really help :)) Let me know whether you get the same flashing with and without debugger attached? I’ll go give it a whirl on my Panda2 and report back
public class Program
{
static PersistentStorage pStore ;
static VolumeInfo pStoreI;
static bool running;
static AnalogIn thermometer;
static AnalogIn lightSensor;
static Thread thread;
static int temp;
static int light;
static int idx;
public static void Main()
{
// Blink board LED
bool ledState = false;
OutputPort led = new OutputPort((Cpu.Pin)FEZ_Pin.Digital.LED, ledState);
// Initialize the eblocks
thermometer = new AnalogIn((AnalogIn.Pin)FEZ_Pin.AnalogIn.An2);
thermometer.SetLinearScale(-22, 56);
lightSensor = new AnalogIn((AnalogIn.Pin)FEZ_Pin.AnalogIn.An3);
lightSensor.SetLinearScale(0, 100);
for (int i = 0; i < 100; i++)
{
if (i % 10 == 0)
{
led.Write(!ledState);
ledState = !ledState;
}
Thread.Sleep(20);
}
// Begin sampling the temperature
running = true;
thread = new Thread(MainLoop);
thread.Priority = ThreadPriority.Highest;
thread.Start();
Thread.Sleep(11000); //will leave the collection running for 11 secs.
running = false;
for (int i = 0; i < 100; i++)
{
if (i % 10 == 0)
{
led.Write(!ledState);
ledState = !ledState;
}
Thread.Sleep(60);
}
Thread.Sleep(Timeout.Infinite);
}
public static void MainLoop()
{
byte[] mystr;
pStore = new PersistentStorage("SD");
pStore.MountFileSystem();
string rootDirectory = VolumeInfo.GetVolumes()[0].RootDirectory;
FileStream FileHandle = new FileStream(rootDirectory +
@ "\log.csv", FileMode.OpenOrCreate, FileAccess.Write);
DateTime time = DateTime.Now;
long ctr = 1;
while (running)
{
temp = ((thermometer.Read() * 9) / 5) + 32;
light = 100 - lightSensor.Read();
time = DateTime.Now;
mystr = Encoding.UTF8.GetBytes(ctr + "," + time.Ticks.ToString() + "," + temp + "," + light + "\r\n");
FileHandle.Write(mystr,0,mystr.Length);
ctr++;
}
FileHandle.Flush();
FileHandle.Close();
Thread.Sleep(10);
pStore.UnmountFileSystem();
}
Could you explain why use & 0xFF, I don’t understand it.
And this only can run when debugger attached.
When I execute F5. The debugger output shows
Step into: Stepping over non-user code ‘FEZ_Domino_SD_test_godFather89_.Program…’
@ Brett
Your code works either on debugger attached or without.
I got 1426 data.
"Can you design a solution that has high speed memory? "
Do you mean SDRAM?, or I may use high level board with SDRAM?
"How many samples do you actually need to sample at that rate? 1sec? 10 Sec?"
Continuously sampling for 3-5 minutes.
Yes, you need to reset the offset. I forgot about it.
& 0xFF is used to mask only the 8 least significant bits (one byte). If you don’t use the & 0xFF, it will try to convert numbers greater than 255 to a byte which will throw an exception.
Please post the entire code you used.
5*60 * 200 (samples/sec) * 8 (bytes/sample) = @ 469 KB of memory. If you have enough memory, this would be the best solution. You could use a SPI memory chip.
If you can’t debug your app, but can mine, then there’s something weird in your code. Nothing I can help you with unfortunately.
~120 samples per second, that’s not bad. How fast can you read your accelerometer ?
500kb of memory, that’s quite a lot in microcontroller terms.
SPI; I wonder whether an SPI memory chip would be faster than the SD card; at least you don’t have a file system on top of it all which is likely to save you time, as well as not requiring you to do ASCII conversion that bloats things heavily. I’ve recently read a project where someone wrote a file system (which is a bit of an aside) on top of an SPI flash memory chip from Atmel - here’s the project Designing a simple and cheap temperature logger. Part 4: The junk in the trunk. | Pick and Place - and they’re using AT25DF081, which besides the voltage interfacing challenge might be worth looking into.
Your project sounds interesting, but I can’t really imagine what you are going to get out of 3-5 mins of accelerometer data. Is this a personal project, a work task, a uni project? How much effort you want to go into to try to solve this will affect what options are in any way relevant…
[quote]500kb of memory, that’s quite a lot in microcontroller terms.
SPI; I wonder whether an SPI memory chip would be faster than the SD card; at least you don’t have a file system on top of it all which is likely to save you time, as well as not requiring you to do ASCII conversion that bloats things heavily. I’ve recently read a project where someone wrote a file system (which is a bit of an aside) on top of an SPI flash memory chip from Atmel - here’s the project Designing a simple and cheap temperature logger. Part 4: The junk in the trunk. | Pick and Place - and they’re using AT25DF081, which besides the voltage interfacing challenge might be worth looking into. [/quote]
To keep the confusion out, I was thinking of SPI SRAM (even better FRAM) if the board you use does not have enough memory. This should be very fast.
After you write all your samples in the SPI RAM, you can read them and save them in a file on the SD card/USB memory stick, having enough time to make computations, ASCII or whatever.
I will post my code tomorrow.
I will test the adxl345 first.
I think I will change the hardware. The Hydra With 16MB of SDRAM may be a choice.
In my PhD study, I have been used the Imote2 xbow.jp
The board has 32 Mb SDRAM, I can write the 3 axis acceleration data in 250 Hz sampling rate for 20 minutes. Then wireless transmit the data to base station.
But the same problem, there are no accurate timer in NETMF.
And, the Imote2 stop produce. I 'd like to design a new wireless sensor node based on .NETMF.
I am doing the previous study.
Actually, my major is civil engineering, I learning the NETMF, embed system, and electronic related by myself. It is not easy for me, but interesting. Lucky, there are many professional here can teach me many.
Thanks.
