Persisting data

I am in need of a way to persist small amounts of data between power cycles.

Looking at the STICore hardware listing, i see the FEZ PICO having “External Flash”.
Would this already be supported in TinyCLR?
If so, how can i acquire the FEZ PICO today?

Please advise

Yes, it is supported. See : Secure Storage

That’s what I need thanks. So that config area seems to be supported on each devise as it’s available memory on the MCU itself.
So what is this ‘external flash,’ I see on the FEZ pico and others? I still want one… :hugs:

Sorry - yes, I pointed to the Secure Storage stuff, which is great for storing config data.

For using the extended flash, see : External Memory

and particularly this line:
The entire 16MB of flash, or 10MB when deployment is extended, can be used directly by reading/writing raw sectors or using Tiny File System.

2 Likes

Regarding the secure storage, Im curious about wear leveling, there doesn’t appear to be any abstraction in place for this, depending on write frequency, is it not something that needs to be taken into consideration?

If it does need to be concidered, I suppose then the question is about the durability of this particular memory. What is the expected lifetime/how do writes to one block effect the entire space or just that block?
Just asking so i know how to write a wear leveling algorithm if need be.

I thought wear leveling was implemented in tiny file system.

Always… How often are considering writing? You started out asking about small amounts of data between power cycles. Normally this does not raise an excess writing flag.

Even outside of TinyFS, implementing your own wear-leveling is relatively simple. Erased memory is filled with 0xFF bytes. So, write new blocks of config data to flash memory sequentially, starting each block of data with some sentinel value that isn’t 0xFFFF and and a word indicating the length of that block. When reading, use the last config block that you read before everything is 0xFF’s. When writing, if you run out of space, call the erase function and start over writing and reading from the beginning.

Yes, small amount of data but at an immeasurable, if not excessive, interval.

i can see writes happening many times a day for several years… if a product was success at all of course…

In regard to Secure Storage, this is a relative small partition, and it appears direct access to the blocks, i would have thought use of a file system here is not very appropriate, if possible at all.

Yes, seems that would do it, just wasnt sure if something was done at the naitive read/write level… didnt want to needlessly implement anything on top of it. thanks

@LucaP, you are correct, Tiny FileSystem does implement wear leveling as well as garbage collection which is used to compact the storage when available space is fragmented and free space is low.

2 Likes

@mcalsyn Though is memory always erased upon flash/programming?
In other words, would say even an in field update necessarily erase the flash, if in fact one doesnt want it to?

What I say below applies to the space managed by SecureStorage. Flashing your program is a different matter.

So no, nothing is erased for you in the SecureStorage space. In the api, you indicate which block to write, and the system just writes that block at the correct location in flash memory. And it will write that whether that block was erased or previously written.

The way that the flash works is that when it is erased, every bit is set to ‘1’. When you write, the 1 cells stay 1 and any 0’s that you are writing get set to 0. You can write over a block multiple times, but you can only set 1’s to 0’s. You cannot set 0’s back to 1. Only erasing does that, and it sets all bits back to 1. But it does that for the whole secure storage area (all the blocks).

So, let’s say that there are 128 blocks of 32 bytes. Calling .Erase() will set all the bits in all 128 blocks to ‘1’. That is, the whole of SecureStorage space will be set to 0xff.

Now lets say you want to write 64 bytes of data. Write that to blocks 0 and 1. Later, when you want to write another 64 bytes with different values, don’t erase - just write the next 64 bytes to blocks 2 and 3.

When reading, you want to read the last set of 64 bytes that have valid values (since those will be the most recently written).

When writing, if you reach block 127, you’ve run out of space. NOW you need to erase the whole 128 blocks with .Erase, and you can store your data in blocks 0 an 1.

My practice is to write data with a preamble of a signature and length - for example 0xAA55 [len] [data], On reading, if I see 0xAA55, I know I have a block of config data, and the next block is [len] bytes away (potentially plus some buffer to make it align with a block boundary). When the current start plus [len] is 0xffff, then I know that the block that I am on is the last in the chain, and represents the most recent config data.

Hope that helps.

4 Likes

Great description of how it works!

It would be even better if all of that was abstracted in a helper library so not everyone needs to reinvent :stuck_out_tongue:

1 Like

As someone currently reinventing this wheel, I concur.

I would certainly want to publish this along with my gpio extender but I feel a config implementation is pretty application specific… Id fear spending so much time considering what and how to do it so generically that it could be anywhere. Guess I’m not willing to give it enough thought right now…

Yep, same here, as im not even sure i would do it properly… Also it adds to precious code space so GHI will not do it either… The bane of small devices, space!

No good deed goes unpunished…

Im withdrawing this comment/question. There seems to be so many ways to implement things, one would need a spec to describe with any more detail…