I have done some work for the STM32F4 with the guys from Oberon. On Oberon’s codeplex project, there is a not yet released branch including experimental MFUpdate support for the Mountaineer boards. MFUpdate support was added by me:)
I have been working on enhancing that for a while now:
I wrote a BlockStorage driver for the external 8MB flash chip, so it can be used for MFUpdate (I mean a native BlockStorage driver, integrated into the Porting Kit). I also added FileSystem support for the Mountaineer boards and have been looking very closely at how the caching of FAT sectors work, and what that means for wear leveling. I must say, the FAT implementation is quite good, still, flash wear is a problem.
I am currently evaluating the changes needed to integrate the wear leveling driver available in the Porting Kit, but this is really turning out to be a nightmare. That’s over 2000 lines of (for me complex code) with almost no explanation how this stuff is supposed to work.
This has brought my attention to TinyFileSystem. I guess it would be quicker, and in the end more effective, to implement TFS in native code, than to bother any longer with FAT and wear leveling.
So, I will be taking a closer look at TFS , and would like to ask if you have a problem if your ideas are “stolen” for the creation of a new NETMF PK level file system.
What I would like to do is create a codeplex project for this, and once ready, propose to integrate it into the Porting Kit so it becomes a standard feature of NETMF.
My main worry is stability - there do not seem to be that much users who tested TFS yet, so I have no idea how stable that is.
Maybe we can chat about TFS. Maybe you can contact me at markDOTmunteATpoboxDOTcom
Please feel free to use the code, I initially developed the code in response to a request from Gus to the community for anybody interested giving it a shot. I knew nothing about flash memory and saw it as a good opportunity learn something new. I would consider it a huge success if someone was able to derive real benefit from it.
I do not know if anyone is seriously using the code, so I understand your concern regarding stability. One thing I need to do is validate the file name when the file is created and check that it is not more that 16 bytes long.
If you need to know anything about the code or why I did something in a particular way I would be happy to respond to any questions or suggestions for improvements you have along the way.
I have been using it as the file system for my web server as it was far more reliable than MicroSD implementation on the Cerberus 5 months ago (although I am about to reevaluate with the new firmware). I can tell you that my version of TFS has been running for the last 5 months continuously. But, I only write once to create the files. I did write to a log file, but when the file system filled, either my code or TFS did not handle it well. I think some work may be needed there. But, I studied it for a while and was VERY impressed at how it was written to avoid wearing out the flash with a good understanding of how the SPI flash write. I think this file system should be the basis for anyone trying to do this.
Some of the changes I made were:
allow slashes in the filename and extend the size. This allows for a primitive directory support and longer filename support.
Enhanced the driver to support several different flash chips
Detect Flash chip size and format accordingly
Partitioning
Some stuff to share the SPI.
Unfortunately, it’s a bit tangled with some other code because of an unfortunate architectural choice. If someone is serious about making this part of the PK, I would like to incorporate some of these changes first.
Great to know Valkyrie-MT.
Question: did you write any unit tests you can share? I thinks the first step before trying to port this to native code, is to agree on a final, tested, feature set under managed code.
Besides testing for stability, I’d really also like to code some tests that show how wear is leveled out by TFS. Good thing is, most (if not all) of these tests can be run using the emulator.
Does somebody have a clear idea how to test a filesystem and flash wear
I am open to any suggestions!
Some things I think that are missing (I haven’t have the time to analyze it in details yet):
I miss a volume signature structure - something that can be used to “mount” (or deny a mount)
I would reserve some bytes in the header structures, so crc or ecc can be added later
I miss some version information in the structure: I guess it is important to be able to extend the filesystem structures should newer versions of the TFS add features later
we should consider using the same naming as used by the porting kit (I’m not saying I prefer one or the other, but they should match).
smallest erasable unit: called block in PK, sector in TFS
largest, in one step programable unit: called sector in PK, called cluster in TFS
I’ve found this link some time ago. I’ll try to compare these concepts to what is implemented in TFS.
As I mentioned, I created a custom emulator to develop and test TFS rather than use a physical flash module. For testing the wear leveling it would probably be a matter of adding some counters into the emulator and then run a suite of tests.
The tests I have for the file system are not currently very well organised, but they exist. They test things like writing to the file, overwriting portions of a file and testing that reading the file back give expected content etc. writing to the file system until it is full, deleting running compaction of the file system and then verifying the files on the file system.
That is correct, there is no volume signature. The basic meta data maintained was sufficient to mount the filesystem and I did not want to have additional overhead of managing a special block for the volume. I did have this in the design at some point, but given the goal of writting a small and reasonably efficient FS it did not add much value for my purposes at the time.
The CRC/ECC was something I considered, the problem that I faced with that is that every write would potentially need to update the CRC which meant additional writes and cluster allocations due to the fact that the crc cannot be updated in place. So from a efficiency point of view I decided against this. Of course an alternate design might accommodate this better.
Version number would be very good, but because of my choice not to have a volume structure this was not easily accommodated and I did not want to add additional data to every file header entry.
The names in TFS match the names of the teminology in the documentation for the flash chip TFS initially targeted. Just to keep the code and the documentation for the target module in relative sync. I agree that moving into the PK might make more sense to adopt the naming conventions used there.
I’m working on trying to get a G120HDR to communicate with the GHI flash module via the TinyFileSystem. I’m wondering about the value pagesPerCluster that gets sent to the MX25I3206BlockDriver. What would an appropriate value for this?
Also, if I run the following once:
if (_fileSystem.CheckIfFormatted())
{
_fileSystem.Mount();
}
else
{
_fileSystem.Format();
}
it takes about 30 secs for the CheckIfFormatted() and then another 10 secs for _fileSystem.Format()
However, the next time i start the application, the CheckIfFormatted seems to get stuck (I’ve waited for at least 10 minutes for it to finish). The the “active diode” is bright so it’s attempting to something, but i dont know what.
, what do they do? I’ve never worked with SPI before so i don’t know how to interpret this, and i ha no idea why it would write 0x02 to data2[1]. It doesn’t in my case
@ rullanidag - I will check it out this weekend and see if I can find anything wrong when executing this sequence of actions.
Just to answer your question on returning of 0x02. You will see that the command being written to the device is a read status command, the command is in the write buffer (data1)
The status will be returned in the second byte of the read buffer (data2), the 2nd bit in the status indicates that the write has been enabled.
I guess your question is also why is it in the second byte and not the first. Well the way SPI works is that as data is written it can also be read, so while the first byte is being written bit by bit, there is a response being sent bit by bit, of course because the first byte has not been fully received and decoded, the byte being returned is just 0, but once the written byte has been received the device can now send a valid response which would be the second byte returned.
This is known as full duplex data transmission. each clock cycle, on SCLK, the master sends a bit on MOSI and the slave reads the bit, the slave also sends a bit on MISO and the master reads the bit.
@ rullanidag - Can you possibly share a little more of your code. I tested the code from the usage example include on the code share ling and it works correctly.
What surprises me is that you say the CheckIfFormatted takes 30s. I test on a formatted flash and on a non-formatted flash and CheckIfFormatted returns within 2 seconds in the case of a formatted flash, which is the worst case since it will scan the entire flash module. Maybe there is some other issue at play here.
Ok, so here’s the deal, and feel free to laugh: I had managed to wire to the wrong pins so any failure in communication with the flash is now quite explainable. Allthough, it’s perhaps weird that it eventually passed the format method. Sorry @ taylorzaa for having occupied your
time with this.
I started having other problems aswell, and wrote a looong text here to explain my problems, but before I had time to post it I changed the value that I asked about earlier (pagesPerCluster) to 1 and it started working. I earlier had the values 100 or 10 (I have no idea what it means so I just guessed). So i guess I’m up and running now, though I’ll probably have more questions in the future.
@ rullanidag - Not going to laugh, just relieved that there was an explanation for the problem!
The purpose of the pagesPerCluster is to group a set of flash pages into a single cluster. Basically I treat a cluster as a unit of allocation, so every time more space is needed to extend a file as it grow the additional space is allocated one cluster at a time. Of course each cluster has a certain amount of overhead, so if you are too few pages per cluster then you have more overhead due to the fact that more clusters are required. While on the other hand if you have too many pages per cluster then even writing only one byte to the file system will allocate lots of space to the file because the smallest allocation unit is a cluster.
You can think of this exactly like the relationship between sectors per cluster on a normal disk. I typically use a value of 4 pages per clusters.
Do you see any reason why it fails when I choose the value 10? What happens is that it passes CheckIfFormatted() but then the Mount() fails with the message “Not Formatted”.
@ rullanidag - No problem. Off the top of my head I can’t think of a reason why it would not work with a value of 10 but I will think through it a little an maybe put it under the debugger. If I find something I will let you know.
Answering my own question: it is possible! FRAM is pretty much the same RAM, so I used the provided MemoryBlockDriver almost as is. Works! Didn’t expect that to happen so quickly Nice job taylorza. I owe you one beer now.
