What’s the proper way to handle data being written to the SD over time?
Scenario: Data incoming off the canbus at 8ms per packet, processing the data, and then updating the related objects. Another thread then writes the data from those related objects to the SD card.
I have a thread that writes the data (via FileStream.Write) at 50ms intervals and then a separate thread that flushes the FileStream every 1 second (I have tried 50ms and 30 sec no change)
The problem: The Flush call is blocking all other threads for sometimes up to 1480ms which causes my CAN keep alive thread to be blocked, and the DUT stops transmitting data.
My Question, what is the correct way to handle this situation? Should I flush often? or?
Am I right assuming that the data is stored in RAM until flushed to disk?
Thanks in advance
why do not use sqlite with insert into (SDCard) and you can create mulitple db files example
logyyymmdd.db3 before insert if not exist db…
I need to output a csv file, and the date is real time and always changing. To be concise CanBus -> interpret -> output .csv
I haven’t tested this but there are interrupts and buffering that handle CAN messaging. Even if the system is busy, it should buffer the incoming messages. There are limits of course.
The issue is it blocking the outgoing keep alive message which is on a separate thread 
Do we have to call flush every so often? or can we just call it at the end of the task? (Still fuzzy of if the data is retained in RAM until you flush)
Thanks
You should flush when you know you have idle time, like right after you send the heartbeat message.
from sqlite db you can extract any format on pc csv , excel , txt …
Do you have simple project that we can reproduce flush take 1.5 seconds?