Main Site Documentation

Parsing CSV


#1

Hi All

I have opened a csv file and can loop through each row.
However, what would the best way be for parsing the line out into its individual fields?

Many Thanks

Ade


#2

What does a row in your file look like?


#3

Use string functions to split things up, like so:


string[] fields = row.Split(','); //replace comma with your delimiter
//optionally remove whitespace
for (int i = 0; i < fields.Length; i++)
   fields[i] = fields[i].Trim();

where row is your record string.

[edit] based on the format you just posted, you’ll have to do some additional cleanup - remove single quotes and cast fields to appropriate object types.


#4

Hi

I was planning on using it as follows.
Unless you have some suggestions.

I need to record three int values, one double value, one text value and one datetime

25,15,‘usbin’,1,0.25,‘25/09/2011 09:25’

Cheers

Ade


#5

To make things easier I would get rid of single quotes in the string field (unless you want to use commas in the string field). Then all you need is to use proper Parse methods to get number/date values (Int.Parse, Double.Parse, DateTime.Parse).


#6

You could always use xml but that’s more advanced (just thought that I’d throw that out there)


#7

Remember that String.Split() isn’t suitable if commas can appear in text fields:


1,2,3,"this is, a single field",5,6,7


#8

Hopefully NetMF 4.2 is going to bring in RegExps for clever string analysing :smiley:


#9

Hi All

I have started to use a csv file for the logging and have noticed that I keep getting an error when I try to read from the csv file using a streamreader.readline method.
It only has a single line in it

GC: 2msec 43308 bytes used, 21072 bytes available
Type 0F (STRING ): 720 bytes
Type 11 (CLASS ): 4812 bytes
Type 12 (VALUETYPE ): 72 bytes
Type 13 (SZARRAY ): 6888 bytes
Type 15 (FREEBLOCK ): 21072 bytes
Type 17 (ASSEMBLY ): 17580 bytes
Type 18 (WEAKCLASS ): 48 bytes
Type 19 (REFLECTION ): 24 bytes
Type 1B (DELEGATE_HEAD ): 612 bytes
Type 1C (DELEGATELIST_HEAD ): 144 bytes
Type 1D (OBJECT_TO_EVENT ): 144 bytes
Type 1E (BINARY_BLOB_HEAD ): 7680 bytes
Type 1F (THREAD ): 1152 bytes
Type 20 (SUBTHREAD ): 144 bytes
Type 21 (STACK_FRAME ): 1068 bytes
Type 27 (FINALIZER_HEAD ): 312 bytes
Type 31 (IO_PORT ): 216 bytes
Type 34 (APPDOMAIN_HEAD ): 72 bytes
Type 36 (APPDOMAIN_ASSEMBLY ): 1620 bytes
Failed allocation for 685 blocks, 8220 bytes

#### Exception System.OutOfMemoryException - CLR_E_OUT_OF_MEMORY (1) ####
#### Message: 
#### System.IO.StreamReader::ReadLine [IP: 000a] ####
#### VibrationTool.Program::ReadSDLog [IP: 0057] ####
#### VibrationTool.Program::LoadSD [IP: 00f2] ####
#### VibrationTool.Program::Main [IP: 00b1] ####

A first chance exception of type ‘System.OutOfMemoryException’ occurred in System.IO.dll
Step into: Stepping over non-user code 'System.IO.TextReader.Dispose’
Step into: Stepping over non-user code 'System.IO.StreamReader.Dispose’
Step into: Stepping over non-user code 'System.IO.Stream.Close’
Step into: Stepping over non-user code 'System.IO.FileStream.Dispose’
Step into: Stepping over non-user code 'System.IO.FileStream.Dispose’
Step into: Stepping over non-user code 'System.IO.FileSystemManager.RemoveFromOpenList’
Step into: Stepping over non-user code 'System.IO.FileSystemManager.RemoveFromOpenList’
Step into: Stepping over non-user code 'System.Collections.ArrayList.Remove’
Step into: Stepping over non-user code 'System.Array.IndexOf’
Step into: Stepping over non-user code 'System.Array.IndexOf’
Step into: Stepping over non-user code 'System.Collections.ArrayList.Remove’
Step into: Stepping over non-user code 'System.Collections.ArrayList.Remove’
Step into: Stepping over non-user code 'System.IO.FileSystemManager.RemoveFromOpenList’
Step into: Stepping over non-user code 'System.IO.FileStream.Dispose’
Step into: Stepping over non-user code 'System.IO.Stream.Close’
Step into: Stepping over non-user code 'System.IO.Stream.Close’
Step into: Stepping over non-user code 'System.IO.StreamReader.Dispose’
Step into: Stepping over non-user code 'System.IO.TextReader.Dispose’
Step into: Stepping over non-user code 'System.IO.TextReader.Dispose’
Step into: Stepping over non-user code 'System.IO.Stream.Dispose’
Step into: Stepping over non-user code 'System.IO.Stream.Close’
Step into: Stepping over non-user code 'System.IO.FileStream.Dispose’
Step into: Stepping over non-user code 'System.IO.Stream.Close’
Step into: Stepping over non-user code 'System.IO.Stream.Close’
Step into: Stepping over non-user code 'System.IO.Stream.Dispose’
The program ‘[15] Micro Framework application: Managed’ has exited with code 0 (0x0).


#10

ReadLine has a bug. We used to have a thread that discussed it and has a fix. But it is not available anymore. May be GHI can dig it up. More details here

http://netmf.codeplex.com/workitem/480


#11

I’m not sure but are you attempting to read the entire file into memory? (Also, I’d like to encourage you to use XML :slight_smile:


#12

Hi

I am just using read line to read a single line from a csv file.
I was thinking about xml, but when the device starts up it reads the log file and splits off older entries (>24hours old) into an archive file.

If anyone has any examples on using xml files in this way, especially as a logging file.


#13

Have you tried to execute just that read line code by itself in a separate project. Could it be that the memory on the device is already full from running the rest of the code?


#14

It is showing that I have 19 - 21 k just before it runs.
Also, I only have a single line in the csv file.

Ade


#15

Guys,it is a known bug with ReadLine.

http://www.tinyclr.com/forum/1/4230/#/1/msg40300

William made a fix, it was posted here, but that board is not available anymore. He also explained what has to be done in the link I have provided earlier.


#16

Found it for you:

http://www.tinyclr.com/forum/2/1620/#/1/msg18520


#17

Hi All

How would I handle reading a csv file that was larger?
Lets say up to 1MB in size


#18

Read it in chunks. Only load what you really need. In other words implement caching mechanism.


#19

Thankyou for the suggestion.

How would read it bit by bit as the read line is bugged and does not work, even though this would have been the ideal way.
I would need to read it to the end of each line (preferably).

Many Thanks for your help so far.

Ade


#20

Have you tried ReadLineEx from the following link:

http://www.tinyclr.com/forum/2/1620/#/1/msg18520