Is there a compression library out there?

I need to send some text files via email from the device. They can be up to 2MB in size but with ZIP this can drop to around 100KB so much easier to email.

I have found #ZipLib and looking to see if this will build under NETMF.

Does anyone know of any other compression library that works with NETMF?

there is a compression library with mono

I did find one that was built with the old compact framework and I tried to see if I could build it but lots of missing references that are not in NETMF 4.3

Time to put on your RLP hat :wink: Google Code Archive - Long-term storage for Google Code Project Hosting.

1 Like

It would almost certainly be faster to send the file uncompressed (even accounting for NETMF’s slow network options) than it would be to compress using a library implemented in NETMF. Gus’ option is a valid one, though, you might want to look into that.

I am digging out the RLP tutorials as we speak. :slight_smile:

Uncompressed sending is not an option. It doesn’t matter if the process to compress takes minutes on a background thread as the files are 1.5MB uncompressed and only sent daily after midnight. With most of the locations this will be installed only offering a 2G connection, I need to get them as small as possible before sending.

If your files can be compressed so much it indicates that there is an lot of redundancy in the data. You could look at using a more efficient binary file format.

The files have to be human readable. They are simply CSV formatted ASCII data and this is why they compress down well.

My math could be off by orders of magnitude, but that’s only ~6-7 minutes?

Plus all the retries. :slight_smile:

These locations have poor connectivity. Enough to send small bytes of data. The SCADA upload for instance with about 30 bytes is pretty solid but the email sending often takes a lot longer than it should due to the retries.

I’ll try a test with a 1.5MB file and see how it goes.

@ Dave McLaughlin - So what we need is a Gzip libary for netmf

Yip. I’ve started to port the miniz one Gus pointed to. I followed Simon’s RLP guide and I have it compiling without changes. Now I need to add the code to receive the NETMF native stuff and I’ll be able to test it. No idea yet if anything will work.

Hey Dave,

If the file is about 2MB only then there is no need to compress. But if speed is not permitting , You can directly compress them online.

@ jay_rogers - The connection speed is too low so he really needs to compress the file.

It’s in a remote location so I really need to make the file as small as possible as most likely max connection will be 2G and probably Edge if I am lucky.

Hi Dave,

I currently work on a infield update library for my G120 and G400 to cover the full remote update process (download, validation, updating firrmware, app, files on SD, preupdate procedure (eg updating database or file by a local procedure)), and for thatI code an RLP for the compression algorithm LZJB to reduce the downloaded size. With my test, I reach an average compression of 50%-60% of the original file. To limit the size in RAM, I compressed by block of 64kb.

If you think it is a good solution for you, I can already try to compress one of your file and tell you the compressed size I reach with my implementation.