I/O problems due to memory?

Archive
1

I am puzzled by problems I am running into with I/O. When doing read/write for very small files (200-1000 bytes) I am randomly hitting IOException errors. The only correlation I can see is when available RAM gets below 20K.

I put together a simple project to demonstrate the problem - the code is at this GitHub repository: GitHub - DevOpsGear/GadgeteerExperimentation

using Microsoft.SPOT;
using System;
using System.IO;

namespace GadgeteerExperimentation
{
    public partial class Program
    {
        private void ProgramStarted()
        {
            Debug.Print("Program started - memory = " + Debug.GC(false));

            for (int preAlloc = 22000; preAlloc < 40000; preAlloc += 1000)
            {
                Debug.GC(true);
                RunTests(preAlloc);
            }
        }

        private static void RunTests(int preAlloc)
        {
            Debug.Print("STARTING RUN (PREALLOCATING " + preAlloc + " BYTES)");
            Debug.Print("Prior to alloc, memory = " + Debug.GC(false));

            var preAllocatedMemory = new byte[preAlloc];
            Debug.Print("Beginning - memory = " + Debug.GC(false));

            const string directory = @ "\SD\experimentation";
            if (!Directory.Exists(directory))
                Directory.CreateDirectory(directory);

            const int length = 500;
            const int countFiles = 500;
            var sizes = new int[countFiles];
            var rnd = new Random();
            for (var idx = 0; idx < countFiles; idx++)
            {
                sizes[idx] = length;
                var data = new byte[length];
                rnd.NextBytes(data);

                var fileName = "file-" + idx + ".txt";
                var path = Path.Combine(directory, fileName);
                File.WriteAllBytes(path, data);
                if (idx%25 == 0)
                    Debug.Print("Writing " + idx + "; memory = " + Debug.GC(false));
            }

            for (var idx = 0; idx < countFiles; idx++)
            {
                var fileName = "file-" + idx + ".txt";
                var path = Path.Combine(directory, fileName);
                var data = File.ReadAllBytes(path);
                if (idx%25 == 0)
                    Debug.Print("Reading " + idx + "; memory = " + Debug.GC(false));
                if (data.Length != sizes[idx])
                    Debug.Print("*** LENGTH MISMATCH FOR FILE " + idx + "!!!");
            }
            Debug.Print("Ending - memory = " + Debug.GC(false));
            Debug.Print("ENDING RUN (PREALLOCATING " + preAlloc + " BYTES)");
            Debug.Print("");
        }
    }
}

Does anyone have any ideas for further troubleshooting?

Thanks!
Rob

2

My bad! It has been removed.

3

I ran this several times, and each time I get a seemingly random result. For example, this time it ran for quite a long time, with available RAM getting down to under 12K with no I/O issues at all. It didn’t fail until I attempted to allocate 32K of RAM.

Program started - memory = 55008
STARTING RUN (PREALLOCATING 22000 BYTES)
Prior to alloc, memory = 54828
Beginning - memory = 32808
Writing 0; memory = 19164
Writing 25; memory = 15180
...
STARTING RUN (PREALLOCATING 31000 BYTES)
Prior to alloc, memory = 45384
Beginning - memory = 14364
Writing 0; memory = 11580
Writing 25; memory = 11580
...
Writing 400; memory = 11568
Writing 425; memory = 11568
Reading 0; memory = 11580
...
Reading 475; memory = 11568
Ending - memory = 11532
ENDING RUN (PREALLOCATING 31000 BYTES)

STARTING RUN (PREALLOCATING 32000 BYTES)
Prior to alloc, memory = 45384
Failed allocation for 2669 blocks, 32028 bytes

Failed allocation for 2669 blocks, 32028 bytes

A first chance exception of type 'System.OutOfMemoryException' occurred in GadgeteerExperimentation.exe
An unhandled exception of type 'System.OutOfMemoryException' occurred in GadgeteerExperimentation.exe
4

What’s the total amount of data on the sd card when it fails?

What hardware and firmware version are you using?

5

so the first thing I would also say is you’re not letting Gadgeteer do it’s thing. You need to let the ProgramStarted() method finish to then set up it’s environment - that’s probably not going to change your behaviour but it is a consistency issue I’d try to eliminate. http://blogs.msdn.com/b/net_gadgeteer/archive/2011/12/19/why-not-while-true.aspx

6

[quote=“hagster”]What hardware and firmware version are you using?
[/quote]Sorry I should have mentioned that in the original post. It’s a FEZ Cerbuino NET, using NETMF version 4.3.1.0 and Gadgeteer assembly version 2.43.1.0.

[quote=“hagster”]What’s the total amount of data on the sd card when it fails?
[/quote]It’s a 2GB card and has only a few megabytes of data on it.

[quote=“Brett”]so the first thing I would also say is you’re not letting Gadgeteer do it’s thing. You need to let the ProgramStarted() method finish to then set up it’s environment - that’s probably not going to change your behaviour but it is a consistency issue I’d try to eliminate.
[/quote]Good point, thank you – I will update the sample code. (My actual application uses timers correctly.)

7

The test code is now in a thread. Same behavior as before.

using System.Threading;
using Microsoft.SPOT;
using System;
using System.IO;

namespace GadgeteerExperimentation
{
    public partial class Program
    {
        private void ProgramStarted()
        {
            var th = new Thread(Test);
            th.Start();
        }

        private void Test()
        {
            Debug.Print("Program started - memory = " + Debug.GC(false));

            var isInserted = Mainboard.IsSDCardInserted;
            var isMounted = Mainboard.IsSDCardMounted;
            if(!isMounted)
                Mainboard.MountStorageDevice(@ "SD");

            var mb = Mainboard;
            var sd = mb.SDCardStorageDevice;
            var vi = sd.Volume;
            //vi.Format("FAT", 0, "TEST", true);
            vi.Refresh();
            var totalSize = vi.TotalSize;
            var freeSpace = vi.TotalFreeSpace;

            Debug.Print("totalSize = " + totalSize);
            Debug.Print("freeSpace = " + freeSpace);

            for (var preAlloc = 20000; preAlloc < 40000; preAlloc += 1000)
            {
                Thread.Sleep(250);
                Debug.GC(true);
                RunTests(preAlloc);
            }
        }

        private static void RunTests(int preAlloc)
        {
            Debug.Print("STARTING RUN (PREALLOCATING " + preAlloc + " BYTES)");
            Debug.Print("Prior to alloc, memory = " + Debug.GC(false));

            var preAllocatedMemory = new byte[preAlloc];
            Debug.Print("Beginning - memory = " + Debug.GC(false));

            const string directory = @ "\SD\experimentation";
            if (!Directory.Exists(directory))
                Directory.CreateDirectory(directory);

            const int length = 500;
            const int countFiles = 500;
            var sizes = new int[countFiles];
            var rnd = new Random();
            for (var idx = 0; idx < countFiles; idx++)
            {
                Thread.Sleep(20);

                sizes[idx] = length;
                var data = new byte[length];
                rnd.NextBytes(data);

                var fileName = "file-" + idx + ".txt";
                var path = Path.Combine(directory, fileName);
                File.WriteAllBytes(path, data);
                if (idx%25 == 0)
                    Debug.Print("Writing file " + idx + "; memory = " + Debug.GC(false));
            }

            for (var idx = 0; idx < countFiles; idx++)
            {
                Thread.Sleep(20);

                var fileName = "file-" + idx + ".txt";
                var path = Path.Combine(directory, fileName);
                var data = File.ReadAllBytes(path);
                if (idx%25 == 0)
                    Debug.Print("Reading file " + idx + "; memory = " + Debug.GC(false));
                if (data.Length != sizes[idx])
                    Debug.Print("*** LENGTH MISMATCH FOR FILE " + idx + "!!!");
            }
            Debug.Print("Ending - memory = " + Debug.GC(false));
            Debug.Print("ENDING RUN (PREALLOCATING " + preAlloc + " BYTES)");
            Debug.Print("");
        }
    }
}
8

You also need to be aware of the fact, that if your RAM gets fragmented because of allocations and release of resources, you might not be able to allocate large blocks of memory. This will also result in a OutOfMemoryException, even if Debug.GC() tells you that the would be enough total memory left.

9

Thanks Reinhard. The actual app is not allocating large blocks, and the example code generally does not fail with an OutOfMemoryException. Instead, I am seeing this:

Debug.GC(false) = 19152
(Reading 500-byte file)
Failed allocation for 172 blocks, 2064 bytes
A first chance exception of type 'System.IO.IOException' occurred in Microsoft.SPOT.IO.dll
An unhandled exception of type 'System.IO.IOException' occurred in Microsoft.SPOT.IO.dll

There always is a failed allocation just before the IOException, and it’s always 2064 bytes. Could the memory be fragmented enough for that to fail?

Thanks!
Rob

10

@ Spamagnet - I guess the IO function tries to allocate some memory of 2064 bytes (some internal buffer I guess) and that fails because there is not enough memory in a single block available. The IO function catches this and re-throws an IOException.

I still think this might come from memory fragmentation.
You should not allocate and release arrays and objects in your code. try to store them in fields and reuse them as often as possible.
This will improve performance and prevents memory fragmentation.

11

[quote=“Reinhard Ostermeier”]I still think this might come from memory fragmentation.[/quote]Thanks Reinhard! I will review my code and see if I am doing a lot of alloc/release.
Regards
Rob