Tiny Filesystem on 16MB QSPI hungs during flush

We are implementing a datalogger function using the 16MB of built in QSPI. Our code is based on the samples provided at https://docs.ghielectronics.com/software/tinyclr/tutorials/file-system.html

After a variable amount of writes (something between 500 and 4000), the system hungs while performing fsWrite.Flush(). Then the only option is to reboot and Format the filesystem. Sometimes we have been able to write up to 10,000 records, but most of the time the problem happens earlier.

To simplify the analysis, we wrote a small program and run it on the 20260 development board. It is easy to replicate the problem, running the program 2 or 3 times.

Here is the code:

using System;
using System.Collections;
using System.Text;
using System.Threading;
using GHIElectronics.TinyCLR.Devices.Gpio;
using GHIElectronics.TinyCLR.Pins;
using System.IO;
using System.Diagnostics;
using GHIElectronics.TinyCLR.Native;
using GHIElectronics.TinyCLR.Devices.Storage;
using GHIElectronics.TinyCLR.Devices.Storage.Provider;
using GHIElectronics.TinyCLR.IO.TinyFileSystem;
using TinyCLRApplication1;

namespace TinyCLRApplication1
{
    class Program
    {
        static void Main()
        {

            TFS_init();
            TFS_listdir();
            TFS_test(10000,100);

            while (true)
            {
                Thread.Sleep(1000);
            }
        }
        public sealed class QspiMemory : StorageDriver
        {
            public override int Capacity => 0x1000000;
            public override int PageSize => 0x400;
            public override int SectorSize => 0x1000;
            public override int BlockSize => 0x10000;
            private StorageController qspiController;
            private IStorageControllerProvider qspiDrive;

            public QspiMemory()
            {
                qspiController = StorageController.FromName(SC20260.StorageController.QuadSpi);
                qspiDrive = qspiController.Provider;
                qspiDrive.Open();
            }

            public override void EraseBlock(int block, int count)
            {
                if ((block + count) * BlockSize > Capacity) throw new ArgumentException("Invalid block + count");

                var address = block * BlockSize;

                for (var i = 0; i < count; i++)
                {
                    qspiDrive.Erase(address, BlockSize, TimeSpan.FromSeconds(100));
                    address += BlockSize;
                }
            }

            public override void EraseChip()
            {
                var block = this.Capacity / this.SectorSize;
                var address = 0;

                while (block > 0)
                {
                    qspiDrive.Erase(address, SectorSize, TimeSpan.FromSeconds(100));
                    address += SectorSize;
                    block--;
                }
            }

            public override void EraseSector(int sector, int count)
            {
                if ((sector + count) * SectorSize > Capacity) throw new ArgumentException("Invalid sector + count");

                var address = sector * SectorSize;

                for (var i = 0; i < count; i++)
                {
                    qspiDrive.Erase(address, BlockSize, TimeSpan.FromSeconds(100));
                    address += SectorSize;
                }
            }

            public override void ReadData(int address, byte[] data, int index, int count)
            {
                qspiDrive.Read(address, count, data, index, TimeSpan.FromSeconds(1));
            }

            public override void WriteData(int address, byte[] data, int index, int count)
            {
                qspiDrive.Write(address, count, data, index, TimeSpan.FromSeconds(1));
            }
        }
        public static TinyFileSystem tfs = new TinyFileSystem(new QspiMemory());
        public static long FreeMEM;
        public static void TFS_init()
        {
            Debug.WriteLine("TFS_init. Inicializa TFS");
            if (!tfs.CheckIfFormatted())
            {
                //Do Format if necessary 
                Debug.WriteLine("TFS_init. Formatea el TFS");
                tfs.Format();
            }
            else
            {
                // Mount tiny file system
                Debug.WriteLine("TFS_init. Monta el TFS");
                tfs.Mount();
            }
        }

        public static string TFS_listdir()
        {
            string rsp = "TFS Files:";
            Debug.WriteLine(rsp);

            var lista = tfs.GetFiles();
            string[] file = lista;

            for (int k = 0; k < file.Length; k++)
            {
                rsp += " " + file[k] + "(" + tfs.GetFileSize(file[k]).ToString() + ");";
                Debug.WriteLine("   " + file[k]);
            }
            return rsp;
        }

        public static void TFS_writeLine(string archivo, string data)
        {

            Debug.WriteLine("TFS_writeLine. Archivo: " + archivo);

            try
            {
                using (var fsWrite = tfs.Open(archivo, FileMode.Append))
                {
                    using (var wr = new StreamWriter(fsWrite))
                    {
                        wr.WriteLine(data);
                        Debug.WriteLine("TFS_writeLine. wr.WriteLine Ejecutado OK");
                        wr.Flush();
                        Debug.WriteLine("TFS_writeLine. wr.Flush Ejecutado OK");
                        fsWrite.Flush();
                        Debug.WriteLine("TFS_writeLine. fsWrite.Flush Ejecutado OK");
                    }
                }
            }
            catch (Exception e)
            {
                Debug.WriteLine("TFS_writeLine. ERROR: " + e.Message);
            }
        }

        public static void TFS_test(int nreg, int lreg)
        {
            Debug.WriteLine("TFS_test. INICIO. NREG=" + nreg + " LREG=" + lreg);

            var LED = GpioController.GetDefault().OpenPin(SC20260.GpioPin.PH10);
            LED.SetDriveMode(GpioPinDriveMode.Output);

            string data = "";
            int k;

            for (k = 0; k < lreg; k++)
            {
                data += "A";
            }

            FreeMEM = System.GC.GetTotalMemory(true);
            Debug.WriteLine("TFS_test. INICIO WAIT. Memoria usada= " + FreeMEM);

            for (k = 0; k < nreg; k++)
            {
                LED.Write(GpioPinValue.High);
                Thread.Sleep(100);
                
                Debug.WriteLine("   Registro: " + k.ToString("D6"));
                TFS_writeLine("TEST.txt", data);

                LED.Write(GpioPinValue.Low);
                Thread.Sleep(100);
            }
            FreeMEM = System.GC.GetTotalMemory(true);
            Debug.WriteLine("TFS_test. FIN WAIT. Memoria usada= " + FreeMEM);

        }

    }
}
1 Like

I’m experiencing the exact same thing except with the SD card… flushing takes 200-4000ms…

It’s hit and miss when it happens. Can go 1 min and then it happens, or can go 30 min…

Thanks for putting together this example, hopefully GHI can find resolution as this is a big issue

Not the same unfortunately. Data is buffered into RAM until there is need to flush. You can help by calling flush when your system is not busy.

QSPI issue mentioned by @jcplaza_wiseaccess_c is system lockup, which we will be investigating.

1 Like

Just added: https://github.com/ghi-electronics/TinyCLR-Libraries/issues/680

1 Like

I don’t think the system hang. Problem is, when free byte left is 2 * cluster size then the TinyFS does a compact automatically which take a lot of time because all these compact is in manage code.

I paused the system and seen that it is still running, inside Flush() function.

To avoid compact you may want to check the size available before call Flush()

 TinyFileSystem.DeviceStats deviceStatus = tfs.GetStats();

            if (deviceStatus.BytesFree <= 2*clusterSize)
                throw new Exception("QSPI full - Ignored Compact");

Thank you. We will try that, even though we are not even near to that QSPI memory occupation. The problem occurs when we have less than 2% of QSPI memory usage.

We will also repeat our test, this time waiting until the compact is finished and the system resumes normal processing.

I got an error from VisualStudio: “clusterSize does not exists in the current context”. Where is it defined?

Meanwhile, I added a Debug.WriteLine showing the bytesFree and realized that even if I’m writing only 100 bytes records, every write reduces bytesFree by 4096 !! So the memory fills up faster than we expect.

Does this mean we can only write physical records of 4096 bytes using this filesystem implementation?

If so, we will have to change our approach because we need to be able to write at least 10.000 datalogger records (each one of aprox. 100 bytes).

Good thing is we are same page because system doesn’t hang.

Bad thing is every Flush() reduce 4096 bytes, even few bytes, not sure it is TFS behavior or bug. Anyway we are working on it.

Our board has a 32KB F-RAM we use to store config parameters. I’m thinking of using a segment of it as a temporary buffer of 4096 bytes. When the buffer fills up we will write it to the QSPI and do the flush.

Reading the datalogger will be just a matter of reading directly from QSPI memory and then the records that may still be in the F-RAM.

We will start working on that idea in parallel with your investigation.

I don’t know which memory chip you are using but in our driver for the MikroE FRam Click, we use a 64 byte page size. Which gives a 64 x 4 pages = 256 bytes cluster size if you use the default constructor for TFS.

Perhaps you could look at this page size in your code ?

Edit : if you are using the QSPI memory, then our driver is using a 256 bytes page size, which gives 1024 byte cluster size.

Thank you. We’ll look at it.

Hi Dan. We bypassed the issue using a 4KB area in FRAM as a buffer to collect datalogger records. When the 4K area fills up, we write to QSPI.

Anyway, we would like to know if you have any news on the subject. Did you find if it’s a bug?

You can set this page size is 64 or 256 then you are good to go.

Thank you Dan

You should use the size described in the memory chip datasheet.