Unmanaged heap

Righto brains trust - how do we play with large objects in the unmanaged heap?

I can happily read a 70kb file into a byte array from SD but want to then create a string from the byte[] and promptly run out of memory, so i assume we need to play in the unmanaged heap, but buggered if i know how to do it proper like. :flushed:

using System;
using System.Collections;
using System.Diagnostics;
using System.IO;
using System.Text;
using System.Threading;
using GHIElectronics.TinyCLR.Devices.Storage;
using GHIElectronics.TinyCLR.IO;
using GHIElectronics.TinyCLR.Native;

namespace USBTest
    class Program
        private static byte[] _data;

        static void Main()
            UnmanagedBuffer uBuffer = new UnmanagedBuffer(100000);
            byte[] uTable = uBuffer.Bytes;
            var sd = StorageController.FromName(@"GHIElectronics.TinyCLR.NativeApis.STM32H7.SdCardStorageController\0");            
            string file = $@"{drive.Name}BP2.xml";

            if (File.Exists(file))
                using (var fs = new FileStream(file, FileMode.Open, FileAccess.Read))
                    uTable = new byte[(int)fs.Length];
                    fs.Read(uTable, 0, (int)uTable.Length);
            string str = new string(Encoding.UTF8.GetChars(uTable)); // death by lack of ram
            uTable = null;
1 Like

Unmanaged heap is for bye arrays only. So your question is on how to create larger strings. Good question and I do but have an answer yet. We can easily enable external memory as the main heap and you would have unlimited memory but then, while difficult, someone can use a logic analyzer to see the data.

This 70kb string is a problem in managed memory?

70KB is not a problem.

Problem is:


uTable is 100,000 byte, not 70KB.

char in C# it is 2 bytes.

It cost about 300KB for converting internally.

you may want to try this:

           var c = new char[70 * 1024];

            for (var i = 0; i < c.Length; i++)
                c[i] = 'a';

            var s = new string(c);
1 Like

I started porting code but have no hardware yet but this would be an issue?

Yes it will be. You need about 2MB to process that one line! Even if we enable full heap on external memory, is it smart to use 2MB in a single line? This is an IoT device, not a server.


but the G120 has no issues with this :crazy_face:
just joking, we will see how we handle this but that kind off string size is something we need to handle.

We had a meeting about this today and for customers that want unlimited RAM and do not care if data is stored in external SDRAM, we are adding a new config to enable external heap. (G120 did it in external DRAM).

If all tested well, this will be added to preview.


Added, there is big pool 32MB, you can swim all day.