Main Site Documentation

Zip/Reduce folder size for data transfer


#1

I am working with GHI FEZ cobra.

I want to zip/reduce the size of around 50MB folder and copy from SD card to USB stick attacked on USB Master.

The basic goal behind this is reducing data transfer time. Currently I am copying files from SD folder and paste it in a USB sticks folder. As the data is 50 MB, it is taking much time.

If anyone can suggest some solution/work around then its appreciable.


#2

Compression is a compute-intensive process, and probably not someting you’d want to do on a microcontroller.

What the heck do you have that’s 50mb in the embedded world?


#3

From serial port i am receiving data from other device at regular interval. I am storing the data in SD card(2 GB). I am storing the data in a folder and in .csv file format. i am making files on daily bases.

As and when I want the data backup, i will attach a USB stick and get the data.


#4

Having zip is one of the things on my personal wish list


#5

http://sharpdevelop.net/OpenSource/SharpZipLib/

Free and open-source; maybe someone should get a NETMF version going.


#6

Anyone remember using a Zip file or JPG on a 386 PC? It was really slow, make me think that the time to compress then transfer may be longer than a plain transfer. What sort of data are you collecting? Do you need to keep it all. One common method is to throw out all samples that are the same or within some margin of error, only keep and timestamp (mark sample number) when the data changes. This can drastically reduce the size of a data file.