SD Card File System Corruption

Hello! I’ve a serious problem with the SD-Card.
My Application (running on FEZ Cerberus) log data on the SD Card every 10 minutes. The problem is that some times the file system become corrupted.
I think that this occur when we have a power loss during the write operation, no the Flush() function does not resolve the problem and the only thing that I can do is to place the SD card in my PC and repair file system.
I can accept the loss of data because a can write multiple version of the same file but if the file system become corrupted the board can not access SD card anymore.
Any software solution for this problem? (No backup battery please!)
Can I repair file system from the firmware?
Alternatively Can I use the SD card in raw format as a SPI flash device? If yes, how?

I think that this is the only weakness of the system, otherwise it seem to be very very stable.

Thanks.

Hi.
What the problem to use in your project spi flash instead of sd card?

Indeed it is a big problem of our devices. GHI team proudly says that it is buggy by design (FAT file system). The bad new is that it won’t change. The only thing you can do is:

1:Bufferize your data and write it when it reach a big chunk, then the probability of a power loss during the writing process will decrease.
2: Use a back up battery or like me a supercap. 5F let me enough time to flush the filesystem properly.

It is not that buggy. The only thing we need is a method to recover — similar to “Scan and fix” dialog box on windows…

Proudly?! That was unnecessary. Please Do a search online about journaling file systems and how FAT works.

@ Gus - There are plenty of embedded solutions using SD card that work reliably. Maybe you could assign some resources to it and make SD driver in your libraries a recoverable one? :wink:

@ slawek -
The only problem inusing a flash instead of a sd card is that spi pins of our board are routed to the sd-card slot. I can not redesign the entire board only because the FAT is buggy.
Is there any chance to use the sd card as a raw flash memory?
I can also accept the use of a method that fixes the filesystem corruption.

Is not possible using another file system?

Proudly? may be not :-[ but to my opinion there was not enought effort to change that in 3 years.

it may be enough reliable for hobbyists (Simon you are not considered as an hobbyists :smiley: ), this is not the case for me and for people that store critical configuration files on the SD card. You can say that there are some alternatives, using SPI flash, EWR and so on. EWR is not reliable neither, and SPI flash has to be added early in the design stage.

All of theses facts was not known to me three years ago, now that I know that , my system is better than never and I can say that it’s reliable but I have added SPI and supercaps. Unfortunately we have 50EMX in the field that come back every 6 months due to such problem on SD.

In some cases the SD can not be recovered, the file system is full of empty files with Hieroglyph names.

That is simply not possible. FAT will corrupt on ANY system and it can’t be reliable if hot ejected. That is FAT by design since our read designed very long time ago. This is not GHI and not netmf, it is how FAT works.

@ leforban - nothing can be done, beside not using FAT. See my last reply. Anytime tells you otherwise is never really understand FAT. Anyway, I feel like this is going to be an endless argument so I am going to stop here.

To be clear, everything I saied was in reference to removing the card, or power loss, or system rest with open files, especially while writing.

Hi Gus it may be an endless discussion, but be sure that it will be a usefull one. I don’t say that it is GHI fault nor NETMF but as designer, I can’t say " ok job done it will crash sometime but don’t worry it was expected by design".

As Simon says, there’s tons of end devices that uses SD and FAT and I do not observe such amount of filesystem corruption. The fact is that most of the time those embedded devices are power suppplied with battery and there’s a voltage monitoring and a cover detection system that prevent any wild power loss and sd remove during writing.

I know FAT may be corrupted but it can also be repaired. What about an other filesystem ?

Take a look for that code. It’s not a C# but well explained.

I’m new to this so please excuse my ignorance. Is it possible to reformat the SD card from the managed code. I know I’d loose the data. But the G120 has a ton of memory. I could read as much as I could to memory, reformat the card and write it back right?

If I can’t read from it at all, Can’t I just reformat and kiss the data good bye. Better than being called into the field.

Also, Worst case I need to add a super CAP. If I add one of these.

http://au.mouser.com/ProductDetail/Elna/DB-5R5D105T/?qs=sGAEpiMZZMsCu9HefNWqpo528vTw5W7MSmQ5EB6WGy4%3D

Can someone check my math also. I think this is about 15 Joules. If I’m max draining 300ma at 3.3 volts. I should get 15seconds out of this bad boy to flush the card write/right ? :smiley:

@ stotech - take a look Microsoft.SPOT.IO.VolumeInfo.Fomat in Microsoft.SPOT.IO.dll.

@ leforban - There was always Microsoft’s TFAT, kind of compatible and designed to cope with sudden power-fail scenarios. We looked at it when we were building Windows CE devices. For some reason MS never incorporated it into their Desktop OSs, which is a pity. Still, it would be nice if it was available as an option. The source is in the CE Builder kits.

[url]https://en.wikipedia.org/wiki/Transaction-Safe_FAT_File_System[/url]
[url]Microsoft Learn: Build skills that open doors in your career
or another one:
[url]http://elinux.org/images/5/54/Elc2011_munegowda.pdf[/url]

Building a crash-proof FS isn’t the real problem, it is keeping it compatible with PCs so you can move the media about.
I wrote one for one our systems years ago that was only for use on those machines, using byte-write block-erase NAND memory, dual index blocks, on systems that could be powered down at any time. Don’t know of any failures in >20 years of use.

I could live without compatibility as I use USB to transfer files from the internal SD so a bullet proof file system for NETMF would be a nice option.

Can some one point me to a usage example of VolumeInfo.Fomat. I’ve googled around and got no examples I can understand. I just want to be able to re-format to fat32 if I get a corrupted FS.

Thanks

@ stotech - A year ago I was experimenting with this with no success. The situation now may be different, but back then it was pretty much “if you’ve got your SD card corrupted then it’s dead completely”. The only way to recover was to put SD in PC and fix (or even format) it. Thus I moved away from SD cards; I use FRAM chip (contrary to the public opinion here on forums, TinyFileSystem was corrupting my data), and also gradually adding ALFAT to log big amounts of data. Oh, and I also have a supercap onboard, that gives me 1-2 seconds extra to finish file operations.

It’s a pitty, though, that we cannot have a reliable (or at least partially recoverable) SD card support just out of the box :frowning: Every time I asked for it, Gus always replied “that’s how FAT works”, although I’m still confident that the same FAT works much better on PCs. So it is not entirely FAT fault.

That being said, it makes me wonder if SD card support was any better back in 4.1 days. I have a few systems in the field working 24/7, and still no complaints… Probably, the client doesn’t use that functionality that requires accessing SD card :slight_smile: