Custom Mountaineer Firmware (GCC) / Accurately Timed Pulses

nicholas_goodman · May 22, 2013, 2:50pm

@ jay,

I found out what was being placed in RAM that should not have been (and a big key difference between the Cerberus and the MountaineerEth).

From the MDK scatterfile, I noticed there is an additional Set statement:

<Set Name="Ethernet"                Value="0x2001E000"/>

and further in the file there is an additional LoadRegion:

        <ExecRegion Name="ER_ETHERNET" Base="%Ethernet%" Options="ABSOLUTE" Size="0x1E00">
          <FileMapping Name="STM32F4_ETH_rx_desc.obj" Options="(+ZI)" />
          <FileMapping Name="STM32F4_ETH_tx_desc.obj" Options="(+ZI)" />
        </ExecRegion>

The ExecRegion uses syntax that differs from what the GCC build is expecting. The

Options="ABSOLUTE"

causes a syntax error; whereas the

 Name=

attributes causes ld.exe to report “cannot find STM32F4_ETH_rx_desc.obj”.

Do you know how to convert this ExecRegion into one that EABI can process?

nicholas_goodman · May 25, 2013, 12:24pm

Okay, in the past few days I have been learning a lot, and here is an update on my progress so far:

I have gotten one successful build of the Mountaineer firmware using GCC, however, to do so required shrinking the stack size to an unacceptable amount. Note that the standard stack size on a Mountaineer is already reduced to 16k, compared to the Cerberus 32k stack.

There are two main issues remaining:

1. I am unable to get Linker (ld.exe) to map specific object files to a specified memory region

Normally, native code static variables are placed in the CCM RAM, which on the STM32F4 is only 64k in size. The static variable share this region with the stack, so more static variables in the source code, the smaller the available RAM for the stack is.

The STM32F4_ETH_lwip project, required by the Mountaineer, contains declarations for fairly large send and receive buffers which are too large for the CCR region. When using the RVDS compiler, the scatterfile tells the linker that the object files containing these buffers should be mapped to the much larger SRAM region. I currently do not know the correct syntax in the GCC scatterfile to accomplish this.

2. The remaining native driver code is still too large to fit in the CCM

When I excude the STM32F4_ETH_lwip project from the TinyCLR build, I still exceed the available RAM by 2104 bytes. In theory, I could shrink my stack from 16K to 14K, though that seems less-than-desirable. It seems the RVDS compiler with the 4.2 porting kit can fit everything in. I wonder why this is the case.

I will continue to investigate and read up on the GCC linker to see what I can do.

When I get a reasonable build, I will post instructions on the steps required.

dobova · May 25, 2013, 12:40pm

@ nicholas.goodman -
If you are not using STM32f4 Internal Ethernet MAC, you can get rid of this sections. Just comment out. Be aware that if you are using MountaneerETH .proj you need accordingly change ethernet_lwip parameters.
I had also change code size from 70000 to 90000 (for MDK but this may apply also to gcc) after adding my interops to eth fw.


    <If Name="TARGETLOCATION" In="FLASH">
        <Set Name="Config_BaseAddress"  Value="0x0800C000"/>    
        <Set Name="Config_Size"         Value="0x00004000"/>
        <Set Name="Code_BaseAddress"    Value="0x08010000"/>
        <Set Name="Code_Size"           Value="0x00090000"/> <!-- was 70000 -->
        <Set Name="Valid"               Value="true"/>
    </If>

nicholas_goodman · May 26, 2013, 3:00pm

@ dobova, I do need to use the Ethernet port on the board. My goal throughout this project has been to keep the firmware as close as possible to the official released firmware from the Mountaineer group.

After much trial-and-error and searching on the internet, I have resolved the first of my two remaining issues: putting the STM32F4_ETH_rx_desc and STM32F4_ETH_tx_desc buffers in SRAM instead of the CCM.

I am still exceeding my RAM size by roughly 2K bytes. @ dobova, were you able to get the original source code provided on Mountaineer’s codeplex site to compile using MDK without modifying the scatterfile / linker script?

If so, I would be very interested to see what symbols were actually placed in the address range 0x1000400 to 0x1001000. Certainly the lwip libraries are taking up much of my CCM ram, but I see nothing in the MDK scatterfile which moves particular symbols out of that region to make it all fit.

I will write up my complete procedure soon.

jay · May 26, 2013, 3:35pm

This is interesting to follow – I wish I had more to contribute, but I’ve never worked with the Mountaineer firmware before.

Make sure to write things up if you get it working!

nicholas_goodman · May 27, 2013, 8:29am

Success!

After much searching through the memory map and comparing the RAM usage with that given in the documentation supplied by Mountaineer, I discovered my problem: the USART buffers.

@ jay, since I was trying to build the MountineerETH in the community ports kit, I did not notice that Cerberus and Mountaineer use a different version of the STM32F4 processor_selector header file. The Cerberus declares a 1024 byte receive buffer (for each port) while the Mountaineer uses the standard 256 bytes. This difference persists in the official repositories for each platform.

IMO, this is somewhat of a design flaw in the NETMF PK. Buffer sizes for any hardware should be based on the total amount of memory available which varies by solution, not by CPU. Maintaining a porting kit capable of building both Cerberus and Mountaineer solutions would require additional modification to the STM32F4 DeviceCode headers.

Building MountaineerETH TinyCLR with GCC
(Using the Community Ports PK)

Download the source code:

http://netmf4stm32.codeplex.com/SourceControl/latest

Copy [em]\Solutions\MountaineerEth[/em] project folder to the community ports \Solutions folder

Modify [em]\Solutions\MountaineerEth\MountaineerEth.settings[/em]: Add the following XML element to the node:

<DEVICE_TYPE>cortex-m4</DEVICE_TYPE>

Add a GCC scatterfile to \Solutions\MountaineerEth\TinyCLR:
http://www.ghielectronics.com/community/forum/topic?id=12086&page=4#msg123917

Modify [em]\DeviceCode\Targets\Native\STM32F4\processor_selector.h[/em] file: change the PLATFORM_DEPENDENT_RX_USART_BUFFER_SIZE define from 1024 to 256

Using a “replace in files” utility, such as Notepad++ replace all instances of the string “lwip_1_3_2” with “LWIP”.

Navigate to [em]\Solutions\MountaineerEth\TinyCLR[/em] and build away!

dobova · May 27, 2013, 1:09pm

@ nicholas.goodman -

No I didn’t build, and after few tests I leave it out. I was just interested in Eth MAc + PHY code, but I have different PHY chip (LAN8270 RMII). I’ve my board solution using similar Cerberus settings so I didn’t go further with montaneer fw. I have used some code idea for my MAC->PHY interface code. For DMA use, I modified my scatter file to include ethernet rx/tx buffers for the DMA transfer. But looking at the hardware I found that the guys at mikroe saved few bucks getting rid of 25mhz oscillator on the 8270 so generating clock using MCO1 pin (need 50mhz) from STM32, that’s awful! … because the NETMF clock settings are not suitable for generating 100 or 150mhz to PLL MCO1 divider! … Jesus ! At the end I’m using successfully using again good-old ENC28 board.

fradav · May 28, 2013, 8:07am

@ nicholas.goodman - I remember I was unable to compile ethernet based firmware for cerberus and the patched 4.2PK for gcc because of RAM limitations. Is it possible it is because of the combined effect of gcc generating slightly bigger binaries than MDK AND the USART buffers size ?

nicholas_goodman · May 28, 2013, 10:11am

@ fradav, it is certainly possible. As far as CCM RAM usage goes, in theory, the static variables will take up the same amount of space regardless of the compiler used. The key difference is the extra variables the GCC compiler creates for certain core C / C++ tasks.

My experience with the EABI compiler is that core C/C++ functionality such as malloc is being provided by the open-source Newlib libraries. Two symbols in particular, [em]impure_data[/em] and [em]__malloc_av[/em], take up 1064 and 1032 bytes, respectively.

When I compile the Cerberus code (from the community ports project, derived from 4.3 PK) I observe that there is some 13K of CCM RAM free. If adding networking is overflowing that memory, you can either reduce your stack size or push certain network symbols to the SRAM and reduce your heap size.

I will post my scatterfile later today for GCC that shows how to push certain symbols into SRAM.

jay · May 28, 2013, 10:22am

Also be careful not to confuse overflowing flash with overflowing RAM. It’s very easy to get the flash regions to overflow – I think adding ethernet functionality and compiling with GCC (in any release mode) is enough to do it. Last I checked, Cerberus without ethernet won’t compile in GCC in debug mode – it has to be in release mode, otherwise, the firmware sector will overflow.

Of course, you can modify the deployment region sizes to accommodate a larger firmware image.

nicholas_goodman · May 28, 2013, 12:12pm

Below is the GCC scatterfile needed to build MountaineerEth. Note how the object files containing the ethernet buffers are moved to the SRAM section. This declaration has to come [em]before[/em] the rest of the RAM is declared, since the basic RAM section uses a wildcard to specify file names. The GCC linker will not look ahead to make sure a file is not targeted for another location.

The default memory map of the Mountaineer does not include an RLP section.

<?xml version="1.0"?>
<ScatterFile xmlns="http://schemas.microsoft.com/netmf/ScatterfileSchema.xsd">

  <!-- STM32 with 1M internal Flash and 192k internal RAM -->

  <Set Name="Valid" Value="false"/>

  <!-- ################################################################################ -->

  <Set Name="RAM_Start"           Value="0x20000000"/>
  <Set Name="RAM_Size"            Value="0x00030000"/> <!--192k-->
  <Set Name="CCM_Start"           Value="0x10000000"/>
  <Set Name="CCM_Size"            Value="0x00010000"/>  <!--64k-->

  <Set Name="Stack_Bottom"            Value="0x10000000"/>
  <Set Name="Stack_Size"              Value="0x00004000"/>  <!--16k-->
  <Set Name="Heap_Begin"              Value="0x20000000"/>
  <Set Name="Heap_End"                Value="0x2001DFF8"/>  <!--115k-->
  <Set Name="Ethernet"                Value="0x2001E000"/>
  <Set Name="Ethernet_Size"           Value="0x00001E00"/>
  
  <!-- Custom heap used for interrupt handler table only -->
  <Set Name="Custom_Heap_Begin"       Value="0x2001FE00"/>
  <Set Name="Custom_Heap_End"         Value="0x2001FFF8"/>

  <IfDefined Name="PROFILE_BUILD">
    <Error Message="Configuration not implemented for GCC"/>
  </IfDefined>

  <If Name="TARGETLOCATION" In="FLASH">
    <Set Name="Config_BaseAddress"  Value="0x0800C000"/>
    <Set Name="Config_Size"         Value="0x00004000"/>
    <Set Name="Code_BaseAddress"    Value="0x08010000"/>
    <Set Name="Deploy_BaseAddress"  Value="0x08080000"/>
    <Set Name="Code_Size"           Value="%Deploy_BaseAddress - Code_BaseAddress%" />
    <Set Name="Valid"               Value="true"/>
  </If>

  <!-- ################################################################################ -->

  <If Name="Valid" Value="false">
    <Error Message="Configuration not recognized"/>
  </If>

  <NamedGroup Name="MEMORY">
    <LoadRegion Name="LR_%TARGETLOCATION%" Base="%Code_BaseAddress%" Size="%Code_Size%"/>
    <IfDefined Name="Config_BaseAddress">
      <LoadRegion Name="LR_CONFIG" Base="%Config_BaseAddress%" Size="%Config_Size%"/>
    </IfDefined>
    <IfDefined Name="Data_BaseAddress">
      <LoadRegion Name="LR_DAT" Base="%Data_BaseAddress%" Size="%Data_Size%"/>
    </IfDefined>
    <LoadRegion Name="HEAP" Base="%Heap_Begin%" Size="%Heap_End - Heap_Begin% + 1"/>
    <LoadRegion Name="STACK" Base="%Stack_Bottom%" Size="%Stack_Size%"/>
    <LoadRegion Name="RAM" Base="%CCM_Start + Stack_Size%" Size="%CCM_Size - Stack_Size%"/>
  </NamedGroup>

  <EntryPoint Name="EntryPoint"/>

  <NamedGroup Name="SECTIONS">
    <!--<ExecRegion Name="ER_BOOT" Options=">LR_BOOT">
      <FileMapping Name="*" Options="(i.EntryPoint)"/>
    </ExecRegion>-->
    <ExecRegion Name="ER_%TARGETLOCATION%" Options=">LR_%TARGETLOCATION%">
      <FileMapping Name="*" Options="(i.EntryPoint)"/>
      <FileMapping Name="*" Options="(SectionForBootstrapOperations)" />
      <FileMapping Name="*" Options="(.text*)" />
      <FileMapping Name="*" Options="(i.*)" />
      <FileMapping Name="*" Options="(t.*)" />
      <FileMapping Name="*" Options="(.rodata*)" />
      <FileMapping Name="*" Options="(rodata)" />
      <FileMapping Name="*" Options="(.constdata*)" />
      <FileMapping Name="*" Options="(.conststring*)" />
       <FileMapping Name="*" Options="(.glue*)" />
      <FileMapping Name="*" Options="(tinyclr_metadata)" />
      <FileMapping Name="*" Options="(SectionForFlashOperations)" />
     <!--<IfNotDefined Name="Data_BaseAddress">
        <FileMapping Name="*" Options="(tinyclr_metadata)" />
      </IfNotDefined>
      <IfNotDefined Name="Config_BaseAddress">
        <FileMapping Name="*" Options="(SectionForConfig)" />
      </IfNotDefined>-->
      <!-- libc thinks that it needs these routines, but they are never called -->
      <Provide Name="_sbrk" />
      <Provide Name="_write" />
      <Provide Name="_close" />
      <Provide Name="_fstat" />
      <Provide Name="_lseek" />
      <Provide Name="_read" />
      <Provide Name="_exit" />
      <Provide Name="_getpid" />
      <Provide Name="_kill" />
      <Provide Name="abort" />
      <Provide Name="__errno" />
      <Provide Name="_read" />
      <Provide Name="isatty" />
      <Provide Name="_isatty" />
      <FileMapping Name="LONG(0xE12FFF1E);" />
    </ExecRegion>
	<ExecRegion Name="ER_ETHERNET" Base="%Ethernet%">
       <FileMapping Name="*" Options="(SectionForEthernet)" />
       <FileMapping Name="*STM32F4_ETH_rx_desc.obj" Options="(.bss)" />
       <FileMapping Name="*STM32F4_ETH_tx_desc.obj" Options="(.bss)" />
    </ExecRegion>
    <ExecRegion Name="ER_RAM_RO" Align="0x10" Options=">RAM AT>LR_%TARGETLOCATION%">
    </ExecRegion>
    <ExecRegion Name="ER_RAM_RW" Align="0x10" Options=">RAM AT>LR_%TARGETLOCATION%">
      <FileMapping Name="*" Options="(rwdata)" />
      <FileMapping Name="*" Options="(.data*)" />
    </ExecRegion>
    <ExecRegion Name=".bss" Align="0x10" Options=">RAM">
      <FileMapping Name="*" Options="(.bss*)" />
      <FileMapping Name="PROVIDE(__exidx_start = .);" />
      <FileMapping Name="PROVIDE(__exidx_end = .);" />
    </ExecRegion>
    <ExecRegion Name=".zidata" Align="0x10" Options=">RAM">
      <FileMapping Name="*" Options="(.zidata*)" />
    </ExecRegion>

    <ExecRegion Name="ER_HEAP_BEGIN" Base="%Heap_Begin%">
      <FileMapping Name="*" Options="(SectionForHeapBegin)" />
    </ExecRegion>
    <!-- everything between heapbegin and heapend will be allocated for a heap -->
    <ExecRegion Name="ER_HEAP_END" Base="%Heap_End%">
      <FileMapping Name="*" Options="(SectionForHeapEnd)" />
    </ExecRegion>

    <ExecRegion Name="ER_CUSTOM_HEAP_BEGIN" Base="%Custom_Heap_Begin%">
      <FileMapping Name="*" Options="(SectionForCustomHeapBegin)" />
    </ExecRegion>
    <!-- everything between heapbegin and heapend will be allocated for the unmanaged SimpleHeap -->
    <ExecRegion Name="ER_CUSTOM_HEAP_END" Base="%Custom_Heap_End%">
      <FileMapping Name="*" Options="(SectionForCustomHeapEnd)" />
    </ExecRegion>

    <ExecRegion Name="ER_STACK_BOTTOM" Base="%Stack_Bottom%">
      <FileMapping Name="*" Options="(SectionForStackBottom)" />
    </ExecRegion>
    <ExecRegion Name="ER_STACK_TOP" Base="%Stack_Bottom + Stack_Size%">
      <FileMapping Name="*" Options="(SectionForStackTop)" />
    </ExecRegion>

    <ExecRegion Name="/DISCARD/">
      <FileMapping Name="*" Options="(.ARM.exidx*)" />
      <FileMapping Name="*" Options="(.ARM.extab*)" />
    </ExecRegion>

    <IfDefined Name="Config_BaseAddress">
      <ExecRegion Name="ER_CONFIG" Options=">LR_CONFIG">
        <FileMapping Name="*" Options="(SectionForConfig)" />
      </ExecRegion>
    </IfDefined>
    <IfDefined Name="Data_BaseAddress">
      <ExecRegion Name="ER_DAT" Options=">LR_DAT">
        <FileMapping Name="*" Options="(tinyclr_metadata)" />
      </ExecRegion>
    </IfDefined>
  </NamedGroup>

  <!-- The following variables are used to simulate the ones autogenerated by RVDS -->
  <GlobalVariable Name="Load$$ER_%TARGETLOCATION%$$Base"          Value="ADDR(ER_%TARGETLOCATION%)"/>
  <GlobalVariable Name="Image$$ER_%TARGETLOCATION%$$Length"       Value="SIZEOF(ER_%TARGETLOCATION%)"/>

  <GlobalVariable Name="Image$$ER_RAM_RO$$Base"                   Value="ADDR(ER_RAM_RO)"/>
  <GlobalVariable Name="Image$$ER_RAM_RO$$Length"                 Value="SIZEOF(ER_RAM_RO)"/>
  <GlobalVariable Name="Load$$ER_RAM_RO$$Base"                    Value="LOADADDR(ER_RAM_RO)"/>
  
  <GlobalVariable Name="Image$$ER_ETHERNET$$ZI$$Base"             Value="LOADADDR(ER_ETHERNET)"/>
  <GlobalVariable Name="Image$$ER_ETHERNET$$ZI$$Length"           Value="SIZEOF(ER_ETHERNET)"/>
  
  <GlobalVariable Name="Image$$ER_RAM_RW$$Base"                   Value="ADDR(ER_RAM_RW)"/>
  <GlobalVariable Name="Image$$ER_RAM_RW$$Length"                 Value="SIZEOF(ER_RAM_RW)"/>
  <GlobalVariable Name="Load$$ER_RAM_RW$$Base"                    Value="LOADADDR(ER_RAM_RW)"/>
  <GlobalVariable Name="Image$$ER_RAM_RW$$ZI$$Base"               Value="ADDR(.bss)"/>
  <GlobalVariable Name="Image$$ER_RAM_RW$$ZI$$Length"             Value="SIZEOF(.bss)"/>
  <GlobalVariable Name="__use_no_semihosting_swi"                 Value="0"/>

</ScatterFile>

Cuno · May 28, 2013, 12:25pm

Yes, because we treat its firmware as a reference implementation and therefore avoid any non-standard extensions.

Cuno

nicholas_goodman · May 28, 2013, 1:13pm

@ Cuno,

The reason I mention RLP is that the posted GCC scatterfile is a modified version of the Cerberus scatterfile - just to point out a key difference between the original.

The ethernet buffers just happen to land on exactly the same address as the Cerberus RLP base address.

wire_dancer · July 30, 2013, 4:40am

I’d like to natively debug netmf on the Cerberus (or STM32F4 Discovery) with Crossworks, as in tinyclr.it but am running into this very issue.
What I do not understand is why the debug binary needs to fit into the flash region - I assumed that you would only need the debug symbols on the PC with the source code? At least that’s how it works in normal gdb “remote” debugging which I used in other projects.

Has no-one set up a native debugging solution for STM32F4 with Netmf 4.3 and GCC so far?

Thanks!

nicholas_goodman · July 30, 2013, 9:05am

@ wire_dancer,

I have skimmed the link you posted. It appears that using the Crossworks native debugger does not require any special binaries to be loaded on the mainboard itself, rather it uses the output of your custom msbuild so that it understands the symbol mapping of the native code. You are only required to compile TinyBooter / TinyCLR from scratch so that you will have the appropriate axf file to give the native debugger.

This is unrelated to if you can get the PK Cerberus project to compile in debug mode.

Have you tried doing a release flavor of msbuild and seeing if you can attach the Crossworks debugger in that case?

If not, you may be able to save some flash region memory by reducing your USART buffer sizes (which are unusually large on the Cerberus / STM32F4).

Otherwise, if you are deploying relatively simple managed solutions, you can reduce the size of your heap.

wire_dancer · July 30, 2013, 9:27am

just to be clear - we do not actually save any flash space by reducing buffers and heap, do we?

nicholas_goodman · July 30, 2013, 9:29am

oh right. i am familiar with handling the RAM limitations of this chip. total mistake on my part.

i should know better than to read / post to the forums before my morning coffee.

@ wire_dancer,

all that said, have you attempted a debug GCC compile of the Cerberus + Ethernet? How much are you going over? if msbuild fails at the linking stage, it should tell you by how much the memory was overflowed.

If you are barely exceeding your available flash, you can shrink the flash region that takes the deployment image (currently it is 500 kB), by adjusting the “Deploy_BaseAddress” entry in the scatterfile.

wire_dancer · July 30, 2013, 9:47am

I just tried building TinyBooter and TinyCLR by invoking msbuild inside these subdirectories of the FEZCerberus solution, but flash overflow errors occur in both of them.

Yes, this actually works but is of limited use as you can only step through the disassembled code, with no link to the sources.

Wouldn’t it be possible to build a debug version with no size constraints, for use on the PC with the debugger only, which would never be downloaded?


  c:\GCC_46\bin\..\arm-none-eabi\bin\ld.exe: C:\Users\wiredancer\Documents\netmf-community-ports\BuildOutput\THUMB2\GCC4.6\le\FLASH\debug\FEZCerberus\bin\tinyclr.axf section `ER_FLASH' will not fit in region `LR_FLASH'
  c:\GCC_46\bin\..\arm-none-eabi\bin\ld.exe: region `LR_FLASH' overflowed by 267408 bytes

looks like a little more than “barely”

nicholas_goodman · July 30, 2013, 10:44am

Well, it is less than ideal, but if you can keep your application / managed code to less than 248 kB, you can make it fit.

Adjust your scatterfile to:


  <If Name="TARGETLOCATION" In="FLASH">
    <Set Name="Config_BaseAddress"  Value="0x0800C000"/>
    <Set Name="Config_Size"         Value="0x00004000"/>
    <Set Name="Code_BaseAddress"    Value="0x08010000"/>
    <Set Name="Deploy_BaseAddress"  Value="0x080C2000"/>
    <Set Name="Code_Size"           Value="%Deploy_BaseAddress - Code_BaseAddress%" />
    <Set Name="Valid"               Value="true"/>
  </If>

jay · July 30, 2013, 10:53am

It’s also not a bad idea to go into the project and take out any unnecessary crap you don’t need (like RLPlite, USB, graphics, etc).