I thought wear leveling was implemented in tiny file system.
Always… How often are considering writing? You started out asking about small amounts of data between power cycles. Normally this does not raise an excess writing flag.
Even outside of TinyFS, implementing your own wear-leveling is relatively simple. Erased memory is filled with 0xFF bytes. So, write new blocks of config data to flash memory sequentially, starting each block of data with some sentinel value that isn’t 0xFFFF and and a word indicating the length of that block. When reading, use the last config block that you read before everything is 0xFF’s. When writing, if you run out of space, call the erase function and start over writing and reading from the beginning.
Yes, small amount of data but at an immeasurable, if not excessive, interval.
i can see writes happening many times a day for several years… if a product was success at all of course…
In regard to Secure Storage, this is a relative small partition, and it appears direct access to the blocks, i would have thought use of a file system here is not very appropriate, if possible at all.
Yes, seems that would do it, just wasnt sure if something was done at the naitive read/write level… didnt want to needlessly implement anything on top of it. thanks
@LucaP, you are correct, Tiny FileSystem does implement wear leveling as well as garbage collection which is used to compact the storage when available space is fragmented and free space is low.
2 Likes
@mcalsyn Though is memory always erased upon flash/programming?
In other words, would say even an in field update necessarily erase the flash, if in fact one doesnt want it to?
What I say below applies to the space managed by SecureStorage. Flashing your program is a different matter.
So no, nothing is erased for you in the SecureStorage space. In the api, you indicate which block to write, and the system just writes that block at the correct location in flash memory. And it will write that whether that block was erased or previously written.
The way that the flash works is that when it is erased, every bit is set to ‘1’. When you write, the 1 cells stay 1 and any 0’s that you are writing get set to 0. You can write over a block multiple times, but you can only set 1’s to 0’s. You cannot set 0’s back to 1. Only erasing does that, and it sets all bits back to 1. But it does that for the whole secure storage area (all the blocks).
So, let’s say that there are 128 blocks of 32 bytes. Calling .Erase() will set all the bits in all 128 blocks to ‘1’. That is, the whole of SecureStorage space will be set to 0xff.
Now lets say you want to write 64 bytes of data. Write that to blocks 0 and 1. Later, when you want to write another 64 bytes with different values, don’t erase - just write the next 64 bytes to blocks 2 and 3.
When reading, you want to read the last set of 64 bytes that have valid values (since those will be the most recently written).
When writing, if you reach block 127, you’ve run out of space. NOW you need to erase the whole 128 blocks with .Erase, and you can store your data in blocks 0 an 1.
My practice is to write data with a preamble of a signature and length - for example 0xAA55 [len] [data], On reading, if I see 0xAA55, I know I have a block of config data, and the next block is [len] bytes away (potentially plus some buffer to make it align with a block boundary). When the current start plus [len] is 0xffff, then I know that the block that I am on is the last in the chain, and represents the most recent config data.
Hope that helps.
4 Likes
Great description of how it works!
It would be even better if all of that was abstracted in a helper library so not everyone needs to reinvent 
1 Like
As someone currently reinventing this wheel, I concur.
I would certainly want to publish this along with my gpio extender but I feel a config implementation is pretty application specific… Id fear spending so much time considering what and how to do it so generically that it could be anywhere. Guess I’m not willing to give it enough thought right now…
Yep, same here, as im not even sure i would do it properly… Also it adds to precious code space so GHI will not do it either… The bane of small devices, space!
No good deed goes unpunished…
Im withdrawing this comment/question. There seems to be so many ways to implement things, one would need a spec to describe with any more detail…
Another question i have, having also implemented crc8 checksum, thinking that in my particular implementation even though the data is just a mere few bytes, I would reserve and write the same data to 2 separate blocks at once for redundancy/error correction, then take the first to pass the crc check. I feel this beats something like writing the data 3 times and voting best 2 of 3.
Or is this largely unnecessary given the durability of flash? I mean apparently you cant even set a bit even if you mean to do it… Or are bit flips entirely based on something else and this does not influence durability?
Hmm and then there appears to be RTC battery backed up memory 
After reading this thread stating that flash is limited to 10,000 writes, i feel the need to perhaps spread things around a bit.
RTC memory looks good for persisting, not config data, but perhaps state data, as in after an exception/crash where you dont expect battery disconnect/drain while the device restarts…
It also appears unmanaged but without any read/write restrictions secure seems to have, Ill hope to implement core functionality that can be reused by each these memory locations…
@mcalsyn might you be able to help me figure out what is going wrong with my secure storage memory writing algorithm…
Considering your statements:
I had written a memory manager, in which on successive writes, it will use every last byte of the memory area to help reduce wear.
Meaning after writing say a byte at index 0, on the subsequent write it wouldn’t just write to the next block, it would seek the next unused byte which is at index 1 in the same block, and start writing there. But it proves I am unable to do this.
I find that once the block is not empty, i am unable to write to it getting a ArgumentException, no other definitive information given.
Now i understand the description that you will not be able to flip bits from 0 to 1, but as i mention, the function was to merely write to the unused bytes in the block that are still ‘empty’ having only been written previously with (0xff).
To confirm, might it in fact not be possible to write to the same block twice? Of course I always could be doing something entirely unrelated wrong
I can’t actually say. Generally, I have considered that rounding to the nearest 32 byte boundary is cheap, so I have not tried byte-level packing. Maybe there is a check in firmware as to whether the block is truly empty or some limitation in the STM32 architecture that you are seeing bubbled up as an exception.
How big are your config blobs? I find it kind of hard to imagine that within 10,000 write cycles, that those few bytes of rounding cost you any significant lifespan of the device.
Well i was looking to create a library so i wouldn’t ever need to do it again, hopefully it could be useful for anyone’s application, therefore it would be best if not required, to consider the application could have very little config data that may change repeatedly.
That said, personally i have applications with config data as little as 8 bytes, without the overhead of my library, that would be a 75% loss in and of itself, with over head just under 50% loss. There are other points also that would cause a great loss. Like 33 bytes, or just over 4K, and depending on the rate of use, i think you can see the impact/waste.
Now i cant see a device i create standing the test of time and wearing out due to this but…
(other than the exception im getting) where might i learn more about the particular memory used and confirm if there is a limitation?
thanks