Filestream vs memory stream

Hi Every one.

I am receiving on my EMX board two xml files from a computer through the CDC VCOM interface. After reading byte on vcom and checking that each byte is different than null, I start dumping the received bytes (by block of 1024) into a memory stream.

The problem is that how to manage the end of file properly without zero padding the last part of the received block. Indeed, the last block is then composed of data (corresponding to the last byte of the file sent by the computer) and null character due to the fact that reading nothing on cdc returns 0x00 bytes.

Do you have a smart, elegant and runtime efficient way of shrinking the last part of the last block? I have seletect a memory stream but it can be something else… At the end I need to have the xml file available in volatile memory and on the SDCard.

If I understand your problem correctly…
I would read the data length at the computer and send this to the Micro and then use FileStream.Position or MemoryStream.Position to count characters received and then save the data.

It’s OK you can all laugh if you want too…

Nice drop willgeorge, this is a trick that would work however it involves modification on the computer application and I would like to not touch it unless there’s no way to solve that on NETMF side.

For now I am playing with Seek and position but the msdn suffers from a lack of details on how to use it…

Actually dumping the entire memory stream into filestream result in null characters at the end of the file. I need therefore to remove them. In order to detect the end of the data, i am thinking to check if the byte read from the memory stream is null or not.


Debug.Print("Position =" + ms.Position.ToString());
ms.Seek(-1024,SeekOrigin.Current);
            Debug.Print("Position =" + ms.Position.ToString());
            byte[] tmp = new byte[1];
            ms.Read(tmp,0,1);
            while(tmp[0] != (byte)0x00)
            {
                ms.Position++;
                ms.Read(tmp,0,1);
                Debug.Print("Position =" + ms.Position.ToString());
            }

The position printed first is 92161. after the seek method position is 91137. I go backward in order to evaluate byte of the last block of data. Better efficiency could be achieve by starting at the end of the file and go backward for each byte. We will see later…

The problem is that the code does not enter into the while loop… What’s wrong?

@ leforban -
Are you sure that tmp[0] is not equal to 0 ? I would try a debug.print of tmp

actually tmp[0] is equal to 0 at the first iteration. therefore, the while condition is false… whereas the last block should be composed of data. I go on with investigations and will let U know.

ok let’s go deeper. I the seek go more backward, temp[0] is different than null byte. But now the problem is that ms.position++ generate an exception. Despite the fact that the api ref for the SDK4.2 says that Position member allows to get or set the position inside the stream.

Finally, I disciver that I was writing an other empty bloc of 1024 bytes…
Next post will resume…

@ leforban -

See the following code in codeshare…

http://www.ghielectronics.com/community/codeshare/entry/305

Note line 192. public void SetPixel(UInt32 posX, UInt32 posY, UInt32 red, UInt32 green, UInt32 blue) {.

Uses Seek

Also…

Exactly, I am using seek and it seems to work. Is there’s an error in msdn doc related to Position member (get works well but it can not be set)

@ leforban -

For NET Micro 4.2 see

Hi WillGeorge. The problem in documentation is not for the seek method but in Position member : MemoryStream.Position Property | Microsoft Learn

MSDN said that position Gets or sets the current position within the stream while according to my experiment, it allows only to get, not set

@ leforban -

Sorry for the misleading information!

No pb. Do you understand like me that according to the msdn with the link UI have provided that Position should allow us to set the position within the stream?

@ leforban -

Sorry for the delay. First priority has been babysitting my grandchildren.

Anyway, I threw together some code using MemoryStream and Seek.
I’m not sure what you really need but the code ‘may’ get you started?

I am NOT a professional programmer… I just hack away until something ‘seems to work’.
I am not saying this is the ‘correct’ way to do it, just my way on the first try.

Using a Fez Spider with a CP7 display and a button so you will need to change the code for your needs.



using System;
using System.Collections;
using System.Threading;
using Microsoft.SPOT;
using Microsoft.SPOT.Presentation;
using Microsoft.SPOT.Presentation.Controls;
using Microsoft.SPOT.Presentation.Media;
using Microsoft.SPOT.Touch;

using Gadgeteer.Modules;
using Gadgeteer.Modules.GHIElectronics;
using Gadgeteer.Networking;
using GT = Gadgeteer;
using GTM = Gadgeteer.Modules;
using MSPC = Microsoft.SPOT.Presentation.Media;

using System.IO;
using System.Text;

namespace GadgeteerAppSeekTest
{
    public partial class Program
    {
        Font MyFontB = Resources.GetFont(Resources.FontResources.NinaB); //Big font to use
        Font MyFontS = Resources.GetFont(Resources.FontResources.small); //Small font to use

        // This method is run when the mainboard is powered up or reset.   
        void ProgramStarted( )
        {
            Startbutton.ButtonPressed += new GTM.GHIElectronics.Button.ButtonEventHandler(Startbutton_ButtonPressed);

            // Use Debug.Print to show messages in Visual Studio's "Output" window during debugging.
            Debug.Print("Program Started. Press start button.");
            display_CP7.SimpleGraphics.DisplayText("Press start button.", MyFontB, MSPC.Colors.White, 1, 1);
        }

        private void Startbutton_ButtonPressed(Button sender, Button.ButtonState state)
        {
            string teststring1 = "This is the first string.";
            string teststring2 = "This is the second string.";
            string teststring3 = ","; // decimal 44 - Hex 2C
            string teststring4 = " "; // decimal 32 - Hex 20

            String bytesconverted = String.Empty;

            Debug.Print("Test strings to place into MemoryStream");
            Debug.Print(teststring1 + " Length = " + teststring1.Length.ToString());
            Debug.Print(teststring2 + " Length = " + teststring2.Length.ToString());
            Debug.Print(teststring3 + " Length = " + teststring3.Length.ToString());

            byte[] byteArray1 = System.Text.Encoding.UTF8.GetBytes(teststring1);
            byte[] byteArray2 = System.Text.Encoding.UTF8.GetBytes(teststring4 + teststring2);
            byte[] byteArray3 = System.Text.Encoding.UTF8.GetBytes(teststring3);

            byte[] AReadBuffer = new byte[256];
            int BytesRead = 0;
            int BytesToRead = 0;

            //Expandable memory stream, create MemoryStream instance without parameterized constructor.
            MemoryStream AMemoryStream = null;
            AMemoryStream = new MemoryStream(); 
          
            bool canread = AMemoryStream.CanRead;
            Debug.Print("canread =  " + canread);
            bool canwrite = AMemoryStream.CanWrite;
            Debug.Print("canwrite =  " + canwrite);
            bool canseek = AMemoryStream.CanSeek;
            Debug.Print("canseek =  " + canseek);
            bool cantimeout = AMemoryStream.CanTimeout;
            Debug.Print("cantimeout =  " + cantimeout);

            //Exception System.ArgumentException
            //if you do not Seek to the beginning of the memory buffer before a new write
            AMemoryStream.Seek(0, SeekOrigin.Begin);
            AMemoryStream.Write(byteArray1, 0, teststring1.Length);
            AMemoryStream.Flush();
            //Return to origin before read
            AMemoryStream.Seek(0, SeekOrigin.Begin);
            BytesRead = AMemoryStream.Read(AReadBuffer, 0, (int)AMemoryStream.Length);
            bytesconverted = new String(System.Text.Encoding.UTF8.GetChars(AReadBuffer));
            Debug.Print("1) MemoryStream: " + bytesconverted);
            display_CP7.SimpleGraphics.DisplayText("1) MemoryStream: " + bytesconverted, MyFontS, MSPC.Colors.White, 1, 20);
            
            AMemoryStream.Seek(0, SeekOrigin.Current);
            AMemoryStream.Write(byteArray2, 0, teststring2.Length);
            AMemoryStream.Flush();
            Array.Clear(AReadBuffer, 0, 256);
            AMemoryStream.Seek(0, SeekOrigin.Begin);
            BytesToRead = BytesRead;
            BytesRead = AMemoryStream.Read(AReadBuffer, 0, (int)AMemoryStream.Length);
            bytesconverted = new String(System.Text.Encoding.UTF8.GetChars(AReadBuffer));
            Debug.Print("2) MemoryStream: " + bytesconverted);
            display_CP7.SimpleGraphics.DisplayText("2) MemoryStream: " + bytesconverted, MyFontS, MSPC.Colors.White, 1, 30);

            Array.Clear(AReadBuffer, 0, 256);
            BytesToRead = BytesRead;
            //Go to position 47 in memory
            AMemoryStream.Seek(0, SeekOrigin.Begin);
            //Writes a , at AMemoryStream buffer[0]. This overwrites the current data
            AMemoryStream.Write(byteArray3, (int)AMemoryStream.Position, teststring3.Length);
            AMemoryStream.Seek(0, SeekOrigin.Begin);
            BytesRead = AMemoryStream.Read(AReadBuffer, 0, (int)AMemoryStream.Length);
            bytesconverted = new String(System.Text.Encoding.UTF8.GetChars(AReadBuffer));
            Debug.Print("3) MemoryStream comma written to pos[0]>  " + bytesconverted);
            display_CP7.SimpleGraphics.DisplayText("3) MemoryStream comma written to pos[0]>  " + bytesconverted, MyFontB, MSPC.Colors.White, 1, 50);

            Array.Clear(AReadBuffer, 0, 256);
            //Go to position 47 in memory
            AMemoryStream.Seek(47, SeekOrigin.Begin);
            //Writes a , at AMemoryStream buffer[47]. This overwrites the current data
            AMemoryStream.Write(byteArray3, 0, byteArray3.Length);
            AMemoryStream.Seek(0, SeekOrigin.Begin);
            BytesRead = AMemoryStream.Read(AReadBuffer, 0, (int)AMemoryStream.Length);
            bytesconverted = new String(System.Text.Encoding.UTF8.GetChars(AReadBuffer));
            Debug.Print("4) MemoryStream comma written to pos[47]> " + bytesconverted);
            display_CP7.SimpleGraphics.DisplayText("4) MemoryStream comma written to pos[47]> " + bytesconverted, MyFontB, MSPC.Colors.White, 1, 64);
            display_CP7.SimpleGraphics.DisplayText("See new comma in each string", MyFontB, MSPC.Colors.White, 1, 76);
        }
    }
} 

End of code

Debug output from VS2010

Using mainboard GHI Electronics FEZSpider version 1.0
Program Started. Press start button.
The thread '<No Name>' (0x3) has exited with code 0 (0x0).
 * 
Test strings to place into MemoryStream
This is the first string. Length = 25
This is the second string. Length = 26
, Length = 1
 * 
canread =  True
canwrite =  True
canseek =  True
cantimeout =  False
 * 
1) MemoryStream: This is the first string.
2) MemoryStream: This is the first string. This is the second string
3) MemoryStream comma written to pos[0]>  ,his is the first string. This is the second string
4) MemoryStream comma written to pos[47]> ,his is the first string. This is the second st,ing



Hello will george. I am using Seek in the same manner than in your code and it works well. I was just believing that Position could be used as a normal member such that Position++ would have incremented the position within the stream. Actually this is false, Position can only be used to get the position, not to set it directly using position=value.

All is ok now, thank for your help