PandaII : Trying to build an API with WebServer+FezTouchScreen+microSD+nativeCode = OutOfMemoryException

Hello all,
I am trying to build a system wich:

-manages a motor with a native code PID : OK
-displays bitmaps on fez touch screen from SD card: OK
-Manage a small webserver : OK

Every part is working in standalone mode, and even when I put bo combination of both features, but when I put all together, I see at the startup application a ‘System.OutOfMemoryException’ in mscorlib.dll

Am I trying to do too much with the panda or I missed something?
Did someone already made something similar?
Any workaround/ideas?



using System;
using System.Threading;
using System.Text;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using GHIElectronics.NETMF.Native; //code natif RLP
using GHIElectronics.NETMF.FEZ;
using GHIElectronics.NETMF.Hardware;
using GHIElectronics.NETMF.Net;
using GHIElectronics.NETMF.Net.NetworkInformation;

using GHIElectronics.NETMF.IO;      //pour carte SD
using GHIElectronics.NETMF.System;  //pour carte SD


namespace OutOfMem00
{

    public class Program
    {

        public static void Main()
        {
            ///////////////////////////////////////////////////////////////////////
            //RLP
            /*RLP.Procedure TestMethod_c;
            RLP.Procedure debugProcedure_c;
            RLP.Procedure SetConsigne_c;*/
            
            RLP.Enable();
            RLP.Unlock("C.............................."});

            //the line below causes : 'System.OutOfMemoryException' in mscorlib.dll
            byte[] elf_file = Resources.GetBytes(Resources.BinaryResources.RLP_PID);    

            if (elf_file == null)
            {
                Debug.Print("Echec de chargement du fichier binaire");
                return;
            }
            
            RLP.LoadELF(elf_file);
            RLP.InitializeBSSRegion(elf_file);

            
            ///////////////////////////////////////////////////////////////////////
            //FEZ TOUCH SCREEN

            // Montage microSD
            PersistentStorage sdPS;
            sdPS = new PersistentStorage("SD");
            sdPS.MountFileSystem();


            //Sequence d'init de l'ecran FEZ TOUCH
            // This is for FEZ Panda II
            FEZ_Components.FEZTouch.LCDConfiguration lcdConfig = new FEZ_Components.FEZTouch.LCDConfiguration(
                FEZ_Pin.Digital.Di28,
                FEZ_Pin.Digital.Di20,
                FEZ_Pin.Digital.Di22,
                FEZ_Pin.Digital.Di23,
                new FEZ_Pin.Digital[8] { FEZ_Pin.Digital.Di51, FEZ_Pin.Digital.Di50, FEZ_Pin.Digital.Di49, FEZ_Pin.Digital.Di48, FEZ_Pin.Digital.Di47, FEZ_Pin.Digital.Di46, FEZ_Pin.Digital.Di45, FEZ_Pin.Digital.Di44 },
                FEZ_Pin.Digital.Di24,
                FEZ_Pin.Digital.Di26
                );

            FEZ_Components.FEZTouch lcdFEZ = new FEZ_Components.FEZTouch(lcdConfig);


            string path = @ "\SD\picts\menuStart320x240.bin";
            lcdFEZ.printLCDPictFromSD(path, 0, 0);




            ///////////////////////////////////////////////////////////////////////
            //ETH SERVER
            byte[] ip = { 172, 16, 120, 42 };
            byte[] subnet = { 255, 255, 224, 0 };
            byte[] gateway = { 172, 16, 120, 1 };
            byte[] mac = { 0x00, 0x88, 0x98, 0x90, 0xD4, 0xE0 };
            WIZnet_W5100.Enable(SPI.SPI_module.SPI1, (Cpu.Pin)FEZ_Pin.Digital.Di10,
            (Cpu.Pin)FEZ_Pin.Digital.Di7, true);
            NetworkInterface.EnableStaticIP(ip, subnet, gateway, mac);
            NetworkInterface.EnableStaticDns(gateway);
            // start server
            HttpListener listener = new HttpListener("http", 80);
            listener.Start();
            while (true)
            {
                HttpListenerResponse response = null;
                HttpListenerContext context = null;
                try
                {
                    context = listener.GetContext();
                    response = context.Response;
                    // The button is pressed
                    if (context.Request.HttpMethod == "POST")
                    {
                        ;
                    }
                    // Sends response
                    response.StatusCode = (int)HttpStatusCode.OK;
                    byte[] HTML = Encoding.UTF8.GetBytes(
                    "<html><body>" +
                    "<h1>Hosted on FEZ Panda II</h1>" +
                    "<p>Let's scare some people!</p>" +
                    "<form action=\"\" method=\"post\">" +
                    "<input type=\"submit\" value=\"Activate!\">" +
                    "</form>" +
                    "</body></html>");
                    response.ContentType = "text/html";
                    response.OutputStream.Write(HTML, 0, HTML.Length);
                    response.Close();
                }
                catch
                {
                    if (context != null)
                    {
                        context.Close();
                    }
                }
            }

        }//main

    }//program

}//namespace

Edit : I guess that the issue is related to RAM usage, but could I have also reached the max amount of assemblies that I add in the project? (82.6Kbytes if I sum the .pe files in /le directory)?

You are trying to do a lot with ~62KB of RAM. Your program size is not an issue

Might be too much for the Panda.

You could try unmounting the SD after you get the image displayed.

Thanks Mike… So I made some more trials and I am afraid that unmounting the SD will not help : I am a bit stubborn, So I will try to understand what happens to the pandaII :

The out of memory exception appears early, when calling "byte[] elf_file = Resources.GetBytes(Resources.BinaryResources.RLP_PID);“
Maybe I can reduce the size of the 10kbytes RLP. elf file? I think it is fixed in size and I process not a so big amount of native code, so it should be something to optimize…
Indeed, Garbage collector seems to tell that free space is available… and if I replace
"byte[] elf_file = Resources.GetBytes(Resources.BinaryResources.RLP_PID);” by
"byte[] elf_file = new byte[11000];"
I don’t have this memory exception … strange isn’t it?

I made a new test with just this shorter code, and it is the same result :


using System;
using System.Threading;
using System.Text;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using GHIElectronics.NETMF.Native; //code natif RLP
using GHIElectronics.NETMF.FEZ;
using GHIElectronics.NETMF.Hardware;
using GHIElectronics.NETMF.Net;
using GHIElectronics.NETMF.Net.NetworkInformation;

using GHIElectronics.NETMF.IO;      //pour carte SD
using GHIElectronics.NETMF.System;  //pour carte SD



namespace OutOfMem00
{

    public class Program
    {


        public static void Main()
        {
            ///////////////////////////////////////////////////////////////////////
            //RLP
            /*RLP.Procedure TestMethod_c;
            RLP.Procedure debugProcedure_c;
            RLP.Procedure SetConsigne_c;*/
 
            RLP.Enable();
            RLP.Unlock("C.............................."});
 
            //the line below causes : 'System.OutOfMemoryException' in mscorlib.dll
            byte[] elf_file = Resources.GetBytes(Resources.BinaryResources.RLP_PID);
        }//main

    }//program

}//namespace
 

I had some progress after stripping RLP.elf file with the arm-none-eabi-strip command included with yagarto chain : my .elf file is now just 6 kb (11 kb before) , and the application is now crashing just the next line with an unmanaged 'System.Exception’
RLP.InitializeBSSRegion(elf_file);
->i will try to use InitializeBSSRegion(byte[] elf, string start_address_name, string end_address_name);

Maybe you could use Keil’s MDK compiler for the RLP code. I am not sure but I think the free one (up to 32 KB files) can optimize the code to make it even smaller.

Great Idea! I had again some improvment just removing less than 1kb of native code without stripping… But I don’t understand why just stripping the .elf is not working and it crashes at RLP.InitializeBSSRegion(elf_file); … Is this removing some essential info? it shouldn’t… I am also really wondering what I am supposed to put in the two strings args “start_adress” and “end_adress” of the overload function RLP.InitializeBSSRegion(elf_file, “start_adress”, “end_adress”); hex adresses?

Indeed, 11 kb seems big for the small amount of native code I do…
So I will check the limited keil compiler and tell you : MDK Version 5 (bottom of the page)

I am also very surprised to get a

Exception System.OutOfMemoryException - CLR_E_OUT_OF_MEMORY (1)

just following the GC dump telling 21 kb are free …or maybe loading the .elf file consumes twice ram than the size of .elf?
“Type 15 (FREEBLOCK ): 21696 bytes”


GC: 2msec 32436 bytes used, 21696 bytes available
Type 0F (STRING              ):    396 bytes
Type 11 (CLASS               ):   2628 bytes
Type 12 (VALUETYPE           ):    588 bytes
Type 13 (SZARRAY             ):   2064 bytes
Type 15 (FREEBLOCK           ):  21696 bytes
Type 17 (ASSEMBLY            ):  16560 bytes
Type 18 (WEAKCLASS           ):     48 bytes
Type 19 (REFLECTION          ):     48 bytes
Type 1B (DELEGATE_HEAD       ):    324 bytes
Type 1D (OBJECT_TO_EVENT     ):     72 bytes
Type 1E (BINARY_BLOB_HEAD    ):   5940 bytes
Type 1F (THREAD              ):    384 bytes
Type 20 (SUBTHREAD           ):     48 bytes
Type 21 (STACK_FRAME         ):   1308 bytes
Type 22 (TIMER_HEAD          ):     72 bytes
Type 27 (FINALIZER_HEAD      ):     48 bytes
Type 31 (IO_PORT             ):     72 bytes
Type 34 (APPDOMAIN_HEAD      ):     72 bytes
Type 36 (APPDOMAIN_ASSEMBLY  ):   1764 bytes
Failed allocation for 865 blocks, 10380 bytes

    #### Exception System.OutOfMemoryException - CLR_E_OUT_OF_MEMORY (1) ####
    #### Message: 
    #### System.Resources.ResourceManager::GetObjectInternal [IP: 0000] ####
    #### System.Resources.ResourceManager::GetObjectFromId [IP: 0009] ####
    #### Microsoft.SPOT.ResourceUtility::GetObject [IP: 0000] ####
    #### PrecellysV3Panda.Resources::GetBytes [IP: 000a] ####
    #### PrecellysV3Panda.Program::Main [IP: 001f] ####
Une exception de première chance de type 'System.OutOfMemoryException' s'est produite dans mscorlib.dll
Une exception non gérée du type 'System.OutOfMemoryException' s'est produite dans mscorlib.dll

 Uncaught exception 
Le thread '<Sans nom>' (0x1) s'est arrêté avec le code 0 (0x0).

You are allocating 10K out of 20K, which is possible if the memory is not fragmented. In general, 10K array on system with little memory is not a good practice.

Thanks Gus, it makes things really more clear : So tell me if I well understand : Program is first loaded on RAM, then executed thanks to the remaining RAM…
In my case, the garbage collector dump tells that I have 21 kb left for execution of the program (and so the program itself takes about 40kb I presume)

But as you say free space can be fragmented (even if the application just started?) and I guess there is not a lot of possibility to avoid my issues:
-To use smaller buffers when possible (in my case try to get smaller .elf file, as suggested Aron)
-To get more RAM (I understand from other topics that you strongly recommand to switch to other modules instead of trying to add external ram on USBizi)
-Other tips? because apart trying to optimize code and compilation of .elf file, I guess I have no choice to allocate this 10K array (size of the file) …

But it is not explaining why I have memory exception when using getbytes and not when testing with allocating a new byte of 11kb … Maybe in the firest case space has to be contiguous, and in second case, fragmentation is not important?

[quote]Indeed, Garbage collector seems to tell that free space is available… and if I replace
"byte[] elf_file = Resources.GetBytes(Resources.BinaryResources.RLP_PID);" by
"byte[] elf_file = new byte[11000];"
I don’t have this memory exception … strange isn’t it?[/quote]

I’m just throwing this out there.

You might try to dispose elf_file after loading it. Else it is a big chunk of ram being wasted…

Thank you GMod, but out of memory exception is during loading the .elf file ; so disposing after loading doesn’t help.
According to RLP.cs documentation, I already do it in the proper way : disposing the elf file consist to put it to null and wait the GC do its job…
It gives me the idea to force the Garbage collector just before calling Resources.GetBytes :
Good news : It fixed my issue (for the moment)

    Debug.GC(true);
            byte[] elf_file = Resources.GetBytes(Resources.BinaryResources.RLP_PID);

But my code will grow again, so to help me (and other people) to save/better understand memory, I would like to answer those questions :

1)Memory is fragmented, so large buffer allocation could be a problem : is there any maximal recommanded size ?Is it possible to have visibility/understanding on how fragmentation is managed? (size of blocks… )

  1. Native code with RLP and yagarto tools can generate large .elf files (>10k with less than 100 lines)
    =>Is there some optimisation that wer can fo with yagarto? I already try to strip it, it reduces a lot the size of the . elf, but then the file is unusable with RLP…
    =>I will try MDK KEIL, but it produces .bin and not .elf files wich is less practical . Indeed, I think yagarto could generate .bin files as well, isn’t it?

I usually load the elf file at the very beginning of my code, before memory is used and fragmented. and even add Debug.GC(true) before it.

You can also make the elf file smaller by removing debug info and by using the optimizer. Check your compiler documentation on how to do that.

I already do that as soon as I can (first lines of main function)

Yes thank you for that, it fixed my issue

yagarto (gcc in general) doesn’t seem to be very good in size optimizing, (the -oXX options are useless and debug info is already removed => I switch to limited-size free keil MDK compiler wich should give better results…
The initial issue of this topic is fixed. For people interested, I create a topic here about native programming with keil on Panda… http://www.tinyclr.com/forum/topic?id=7600&page=1#msg74641