Hello,
I’m currently trying to log data from UART (without HW or SW flow control) using 921 600 baud rate, odd parity. Current byte rate is about 15 000 bytes/s, problem is that i miss about 50% of messages (lenght of one is 30 bytes). Is there anyone that was dealing with this kind of problem ?
Thanks,
Jan
Your question is missing one or more of these details:[ul]
The name of the product.
The code you’re trying to run.
Details on your setup.[/ul]
(Generated by QuickReply)
Sorry Mike for incomplete question, here reader code that i’m using to read data and write them to SD, there is no code for SD card initialization (different class).
using System;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using System.Threading;
using System.IO.Ports;
using System.IO;
using Microsoft.SPOT.IO;
namespace UARTReader
{
public class App
{
private OutputPort m_led = new OutputPort((Cpu.Pin)GHIElectronics.NETMF.FEZ.FEZ_Pin.Digital.LED, false);
private SerialPort m_serial = new SerialPort("COM1", 921600, Parity.Odd);
private const int MessageLenght = 30;
private const byte MessageStartByte = 0x83;
private byte[] m_Buffer;
private int m_counter;
private int m_messageCounter;
private TimeSpan m_RecordStart;
public void Run()
{
bool headerMatch = false;
byte[] oneByte = new byte[1];
m_counter = 0;
m_Buffer = new byte[1024];
using (FileStream file = new FileStream(GetLogFileName(), FileMode.Create))
{
using (m_serial)
{
m_serial.Open();
m_serial.DiscardInBuffer();
while (!headerMatch)
{
m_serial.Read(oneByte, 0, 1);
if (oneByte[0] == MessageStartByte)
{
m_serial.Read(m_Buffer, 0, 29);//dočtení jedné věty
headerMatch = true;
break;
}
}
m_RecordStart = Utility.GetMachineTime();
int dataCount = 0;
while (true)
{
m_led.Write(false);
dataCount = m_serial.BytesToRead;
m_serial.Read(m_Buffer, 0, dataCount);
file.Write(m_Buffer, 0, dataCount);
m_counter = m_counter + 1;
m_messageCounter = m_messageCounter + 4;
m_led.Write(true);
if (m_counter > 1000)
{
file.Flush();
m_counter = 0;
if (m_button.Read())
{
break;
}
}
}
m_led.Write(false);
m_RecordStart = Utility.GetMachineTime() - m_RecordStart;
Debug.Print("Recorded " + m_messageCounter.ToString() + ", time span=" + m_RecordStart.Hours.ToString()
+ ":" + m_RecordStart.Minutes.ToString() + ":" + m_RecordStart.Seconds.ToString() + "." + m_RecordStart.Milliseconds.ToString());
}
}
}
/// <summary>
/// Returns file name like \SD\000_rawdata.txt
/// </summary>
/// <returns></returns>
private string GetLogFileName()
{
//code made shorter(not important part)
return @ "\SD\000_rawData.txt";
}
}
}
What makes you think the serial port can handle a baud rate of 921600? The highest “standard” rate I’ve ever seen used is 115200 bit/s.
As Eric said, you might have an issue at that high of a baud rate…
The first I would do is add a SerialErrorReceivedEventHandler, and see if you are getting overrun errors. If
you are, then you might have to consider splitting off the file writing into a separate thread and/or using
DataReceived event.
and you should read the throughput threads on here about SD card writing. They may also have something to think about in them… http://www.tinyclr.com/forum/topic?id=1858 is a good start on benchmarking.
If writting to SD card is to slow(open file, write small data, flush data and close file, flush file system) you have option to “save” data from serial port to Queue and then in another thread write to file. Im using this way in my logging system because if I direct write to SD card I get corupted LOG file, missing lines, because I write to log from different threads…
The baud rate works but the system is too slow for handling data and storing to SD. GHI offers dedicated chips to do that, like ALFAT.
@ Dejan - will try your solution, but as Gus stated it might not handle such amount of data.
@ Brett - Well I’have done some test about writing data to SD, per 1 second source generates 15 000, which are stored in about 120 ms.
@ Mike - Well so far I know that the baud rate is high, but I need to store those data for one of my projects.
@ ransomhall - As Gus stated, It can handle that baud rate.
@ Gus - Problem with ALFAT is that our source of serial data is a black box for us, and we need a logger to read incomming data after power on to file with index (like 000_data.txt, 001_data.txt, etc.) Could be ALFAT programmed / configured to do such kind of action without specifing any commands ?
Thanks for reply.
Jan
You will need custom firmware for that, which can be costly. If what you are doing is volume then we can customize something to fit your exact needs.
Well we need logging only for few months, and just about 5 pieces. What about using some more powerful FEZ, what you think Gus ? Could it make some difference ? Hydra, ChipworkX ?
Thanks,
Jan
Larger devices are a bit faster but not enough for 100% but the good news is that they have a LOT of ram. You can buffer your data in RAM in a huge buffer and write when data is not coming in.
@ Gus - So for example using Hydra and recording 15 minutes (about 8 MB) to RAM and storing data ? I’m not sure how long it would take to store 8 MB of data, but I suppose it would take a while…and the other problem is that data are comming every 2 ms from source, so I suppose that would be much time for writing data.
Thanks,
Jan
JohnySeven please try my solution I think it can be improved but for me work ok:
LOG class:
class Log
{
#region Variables & Constants
private string _log_file_path;
private Queue log_messages = new Queue();
private Thread save_thread;
#endregion
#region Constructor
public Log(string log_file_path)
{
try
{
_log_file_path = log_file_path;
save_thread = new Thread(SaveToFile);
save_thread.Priority = ThreadPriority.BelowNormal;
save_thread.Start();
}
catch { }
}
#endregion
#region Public Methods
public void WriteLog(string msg)
{
try
{
log_messages.Enqueue("[\"" + DateTime.Now.ToString() + " \",\"" + msg + "\"],");
}
catch { }
}
#endregion
#region Private Methods
private void SaveToFile()
{
while (true)
{
try
{
if (log_messages.Count != 0)
{
using (FileStream stream = new FileStream(Path, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
using (TextWriter writer = new StreamWriter(stream))
{
while (log_messages.Count != 0)
{
writer.WriteLine((string)log_messages.Dequeue());
}
writer.Flush();
}
}
else
{
Thread.Sleep(500);
}
}
catch { }
}
}
#endregion
#region Public Fields
public string Path
{
get { return _log_file_path; }
set { _log_file_path = value; }
}
#endregion
}
use in code:
//Mount storage!!!
Log log = new Log(@ "SD\log\log.log");
//In your receive from serial event add: log.WriteLog(message);
log.WriteLog("Test");
You can increase Delay in SaveToFile() methode to 1000ms or more…
Let me know how work this for you…
@ Dejan - Wov, thx for code, I’ll try it tomorrow and let you know.
Jan
@ JohnySeven: with RLP you can do it. For my own project, I made a µSD FAT library which reads 4,4MByte/s and writes 1,78MBytes/s back to the card. See http://www.tinyclr.com/forum/topic?id=6849&page=2#msg67196
So logging 115K/s of serial data is just piece of cake for RLP 
JohnySeven did you try my solution? Do work?
@ Dejan - Well I switched to FEZ Hydra (more powerful and more RAM), and copying data from serial buffer works fast, so no buffer overflow at all. But when I start to write to FileStream, I see multiple data overflows per second. So I have to change strategy of storing data to SD card or making storing of data faster. So here comes WouterH with his RLP solution. But so far I have to digg in to FAT library and writing data to SD card on FEZ Hydra to try to speed up storing process.
But still, thank you all for ideas, code example and support, really appreciate that! 
Jan