Send captured image to socket server

How to check the sended bytes on server side (gadgeeter project) if i dont have a length option in Wifly socket sendbinary method? On the client side i can check the recieved bytes using a:


Int bytesRecived = handler.EndRecieve(ar)


1)I am using Wifly GSX from netmf toolbox framework
2)I am using RN-XV WIFly on the GHI-XBee adapter.

All you can do is hoping that the whole array you hand over to SendBinary is really sent out. Some Send methods have an integer as return value which tells you how many bytes have been sent.

@ Alex Bilityuk - thanks for the information. I didnā€™t know that there is a driver for the WiFly Module in the NETMF Toolbox. Iā€™m just trying to adapt Duke Nukems Azure Access test to the WiFly Module, if I knew perhaps this driver could have saved me some hours.

Some months ago there was a thread of jstone05 who reported of lost data during WiFly file transfer. A I understand, in this thread the issue was not solved.

https://www.ghielectronics.com/community/forum/topic?id=14736&page=5

1 Like

:wall: :wall: :wall:
I an stuck. I have no idea what to doā€¦ :frowning: Maybe GHI guys can help me to find a solutionā€¦

This guy in thread you provided writes about couple of bytes loosing, but i have a huge amount of bytes lost, just recently printed in output window bytes sent from server (10980 bytes) and prites recieved bytes in client side (819 bytes). The difference is hugeā€¦

@ Alex Bilityuk - Please post your complete TCP server and client side code again. May be we can see something then.

Ok

Server side (based on WiFly socket. This is my gadgeteer project):


 if (serialCameraL2.NewImageReady)
                {
                    timer.Stop();
                    Debug.Print("Newimageready");
                    capturedata = serialCameraL2.GetImageData();
                    if (capturedata != null)
                    {
                        DisplayStatus(1, "Sending to PC...");
                        // Send picture to PC.
                        if (serverSocket.IsConnected)
                        {                              
                            Byte[] bytesToSend = capturedata;
                    
                            Byte[] outboundBuffer = new byte[1024];                           

                            int incomingOffset = 0;       
                
                            while (incomingOffset < bytesToSend.Length)
                            {
                                int length = System.Math.Min(outboundBuffer.Length, bytesToSend.Length - incomingOffset);                                                            
                                Array.Copy(bytesToSend, incomingOffset, outboundBuffer, 0, length);
                                incomingOffset += length;
                              
                                // Transmit outbound buffer
                                serverSocket.SendBinary(outboundBuffer); 
                                
                                Thread.Sleep(50);
                     
                             }
                  
                   
                        }
                        DisplayStatus(1, "Sended:"+capturedata.Length.ToString());

                    }
                    else
                    {
                        Debug.Print("ImageData = null");
                        throw new System.ApplicationException("Image Data is null");
                    }

                }

Client side (Windows 7 app, based on windows socket):

   
    public static void Listen()
        {
            byte[] bytes = new Byte[1024];

            IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
            IPAddress ipAddress = ipHostInfo.AddressList[0];
            IPEndPoint localEndPoint = new IPEndPoint(ipAddress, 1988);          

            try
            {
                _listener.Bind(localEndPoint);
                _listener.Listen(100);

                while (true)
                {
                    // Set the event to nonsignaled state.
                    allDone.Reset();

                    // Start an asynchronous socket to listen for connections.
                   
                    _listener.BeginAccept(
                        new AsyncCallback(AcceptCallback),
                        _listener);

                    // Wait until a connection is made before continuing.
                    allDone.WaitOne();
                                              

                }
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }           
           
        }

        public static void AcceptCallback(IAsyncResult ar)
        {
            // Signal the main thread to continue.
            allDone.Set();

            // Get the socket that handles the client request.
            _listener = (Socket)ar.AsyncState;
            handler = _listener.EndAccept(ar);

            // Create the state object.
            StateObject state = new StateObject();
            state.workSocket = handler;
            handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
                new AsyncCallback(ReadCallback), state);
        }

        public static void ReadCallback(IAsyncResult ar)
        {
            
            // Retrieve the state object and the handler socket
            // from the asynchronous state object.
            StateObject state = (StateObject)ar.AsyncState;
            handler = state.workSocket;
            handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
                  new AsyncCallback(ReadCallback), state);           

            // Read data from the client socket. 
            int bytesRead = handler.EndReceive(ar);          
            
            if (bytesRead > 0)
            {
               
                memorystream.Write(state.buffer, 0, bytesRead);// HERE IS MY GOAL I NEED THE PICTRURE DATA TO COME HERE
                        
            }
            else
            {
                // Not all data received. Get more.
                handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
                new AsyncCallback(ReadCallback), state);
            }
        }

   private static void SendCallback(IAsyncResult ar)
        {
            try
            {
                // Retrieve the socket from the state object.
                handler = (Socket)ar.AsyncState;

                // Complete sending the data to the remote device.
                int bytesSent = handler.EndSend(ar);
                //Console.WriteLine("Sent {0} bytes to client.", bytesSent);

                //handler.Shutdown(SocketShutdown.Both);
                //handler.Close();

            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }
        }

@ Alex Bilityuk - For what I know, Camera L1/L2 give you a JPG not a BMP


...
// dataImage are bytes received from the camera.
         Bitmap image = new Bitmap(dataImage,Bitmap.BitmapImageType.Jpeg);                        
...
 
  1. In this case you are sending binary for a JPG on the other side of the socket.
  2. I suggest you to provide a chunks sendbinary with each chunks length from 512 to 1024.
  3. What board are you using ?

thank you for reply

  1. It doesnā€™t matter what i am sending because not all bytes receved on the client side. Why do you think itā€™s a jpg? I saved picture from L2 camera to SD and after that i checked parameters of that picture and this is: Format24bppRgb and my picture was saved in bmp
  2. Already tried, it doesnā€™t helpā€¦ Also tried to set thread.sleep on the end of the send loop with
    different settings 1,2,5,10 seconds. doesnā€™t work too.
  3. Cerbuino bee netmf v4.3
  4. Bitmap doesnā€™t have overloads for dataImage (i think you mean byte format) and i canā€™t
    find a property BitmapImageType under Bitmap class.

@ Alex Bilityuk - In your readCallback you call BeginReceive before you call EndReceive, and also before you have actually read the data.

public static void ReadCallback(IAsyncResult ar)
         {
             
             // Retrieve the state object and the handler socket
             // from the asynchronous state object.
             StateObject state = (StateObject)ar.AsyncState;
             handler = state.workSocket;
             handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
                   new AsyncCallback(ReadCallback), state);           

             // Read data from the client socket. 
             int bytesRead = handler.EndReceive(ar);          
             
             if (bytesRead > 0)
             {
                
                 memorystream.Write(state.buffer, 0, bytesRead);// HERE IS MY GOAL I NEED THE PICTRURE DATA TO COME HERE
                         
             }
             else
             {
                 // Not all data received. Get more.
                 handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
                 new AsyncCallback(ReadCallback), state);
             }
         }

This might lead to overlappig calls of the ReceiveCallaback.
I would suggest to remove the 1st call to BeginReceive.
the 2nd time you call BeginReceive if the received number of Bytes is Zero.
But Zero Bytes means thet the Connection was closed by one of the two sides, and it does not make any sens to begin readion more data.

After writing to the Memory stream you should detect somehow, if you have received all data, and if not, call BeginRecieve again.
I would send the number of Bytes just before sending any data, so you know how many Bytes to receive in total.

Like this:

public static void ReadCallback(IAsyncResult ar)
         {
             
             // Retrieve the state object and the handler socket
             // from the asynchronous state object.
             StateObject state = (StateObject)ar.AsyncState;
             handler = state.workSocket;

             // Read data from the client socket. 
             int bytesRead = handler.EndReceive(ar);          
             
             if (bytesRead > 0)
             {
                if (state.TotalSize == 0) // 1st package
                {
                   state.TotalSize = BitConverter.ToUInt32(state.buffer, 0);
                   memorystream.Write(state.buffer, 4, bytesRead - 4);
                }
                else
                {
                   memorystream.Write(state.buffer, 0, bytesRead);
                }
                if (memorystraem.Length >= state.TotalSize)
                {
                   // create your Bitmap here
                }
                else
                {
                   handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
                       new AsyncCallback(ReadCallback), state);
                }
             }
             else
             {
                 // most likely the Connection was closed if Zero Bytes have been received.
             }
         }
1 Like

Reinhard, I did everything like you suggested, but it didnā€™t help. The same result about 10000 bytes loosing everytimeā€¦ :wall:

Here is my client code after change:


 public static void ReadCallback(IAsyncResult ar)
        {            
            
            // Retrieve the state object and the handler socket
            // from the asynchronous state object.
            StateObject state = (StateObject)ar.AsyncState;
            handler = state.workSocket;
            

            // Read data from the client socket. 
            int bytesRead = handler.EndReceive(ar);
            if (state.IsFirstPacket == 0)
            {
                state.sb.Append(Encoding.UTF8.GetString(state.buffer, 0, bytesRead - 4));
                state.bytes_check = state.sb.ToString();
                state.TotalSize = Convert.ToInt32(state.bytes_check);
                state.IsFirstPacket++;
            }

            if (bytesRead > 0)
            {
                if (state.TotalSize == 0) // 1st package
                {
                    state.TotalSize = BitConverter.ToInt32(state.buffer, 0);
                    memorystream.Write(state.buffer, 4, bytesRead - 4);
                }
                else
                {
                    memorystream.Write(state.buffer, 0, bytesRead);
                }
                if (memorystream.Length >= state.TotalSize)
                {
                    // create your Bitmap here
                    state.IsFirstPacket = 0;
                }
                else
                {
                    handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
                        new AsyncCallback(ReadCallback), state);
                }
            }
            else
            {
                // most likely the Connection was closed if Zero Bytes have been received.
            }
        }



Here is server side:


   if (serialCameraL2.NewImageReady)
                {
                    timer.Stop();
                    Debug.Print("Newimageready");
                    capturedata = serialCameraL2.GetImageData();
                    if (capturedata != null)
                    {
                        DisplayStatus(1, "Sending to PC...");
                        // Send picture to PC.
                        if (serverSocket.IsConnected)
                        {
                     
                            bytesToSend = capturedata;

                            // SEND BYTES AMOUNT TO SERVER FOR CONSISTENSY CHECK
                            if (serverSocket.IsConnected)
                            {
                                String request = bytesToSend.Length.ToString();
                                Byte[] count_check = Encoding.UTF8.GetBytes(request);
                                serverSocket.SendBinary(count_check);
                            }
                        
                            outboundBuffer = new byte[1024];

                            int incomingOffset = 0;
                          
                            while (incomingOffset < bytesToSend.Length)
                            {
                                int length = System.Math.Min(outboundBuffer.Length, bytesToSend.Length - incomingOffset);
                                Array.Copy(bytesToSend, incomingOffset, outboundBuffer, 0, length);
                                incomingOffset += length;

                                // Transmit outbound buffer
                                serverSocket.SendBinary(outboundBuffer);                               

                                Thread.Sleep(50);

                            }
                           
                        }
                        DisplayStatus(1, "Sended:"+capturedata.Length.ToString());

                    }
                    else
                    {
                        Debug.Print("ImageData = null");
                        throw new System.ApplicationException("Image Data is null");
                    }

                }

Hi Alex,
Uhm very strange you donā€™t have Bitmap.BitmapImageType ā€¦ But Iā€™m talking about NETMF 4.3, erlier fwk may not have same enumerationā€¦
The file type in any case doesā€™t change the problem with sockets, thatā€™s right.

I second the suggested code from Reinhard, but I would first take it easy and try with sync socket and not with async. You can switch later to async when you are sure everything is fine.
I would also try the code on enc28 wired network to see if WyFly is getting troubles. I use wyfly (PIC32 native env not netmf) in a very short wifi messages so I donā€™t experience any data loss.

One note: I experienced lot of problems on Cerberus board with bitmaps and sockets becouse the GC (due to memory shortage) starts over destroying the socket connection. Check in the debug if this can be the case.

Hi dobova!

  1. I do have a BitmapImageType property but in gadgeteer project not in the windows net project. And i donā€™t need this property on the server side because i am trying to send byte array with picture information and it doesnā€™t matter what kind of format it is.
  2. I will try to use a sync socket and let you know
  3. Where i can find a code on enc28 wired network ?
  4. Can you give a direction where to find information about a PIC32ā€¦ i will try it also.
  5. How to check in the debug if the GC can be the case. Do i have an option to turn a GC off?

Thank you!

Hi Alex,

  1. Ok you have same option on .NET fwk, but Iā€™ve not in mind in this moment the correct syntax. For you itā€™s just a payload. Iā€™m only advising that on receiving side you have to work with a JPG not a BMP or a pure bitmap.
  2. Ok
  3. Oh I suggest to search on the forum and on codeshare, where you can find lot of example code. I canā€™t show it in this moment (Iā€™m mobile).
  4. PIC32 is from Microchip and I used Microchip XC32 compiler, a couple of years ago. Info on http://www.microchip.com. This are 32bit MIPS architecture chips, quite different from ARM Cortex.
  5. You can see GC advises on the ā€œOutput windowā€ in Visual Studio 20xx, the same window where you see Debug.Print() messages.

@ Alex Bilityuk - some more questions and ideas:
How did you prove that your socket server application on the PC works as expected?
Which Baudrate do you use for the Serial Connection to the WiFly module?
If the suggestions of Reinhard and dobova should not work, I would - as already suggested - try a smaller chunk-size (e.g. 512 Bytes) and try a longer thread.sleep (e.g. 300 ms instead of 50 ms) to eventually avoid buffer overflows of the WiFly module.

Edit: Just saw, you already tried.

  1. Ok thanks. if i send all bytes someday to PC i will try this approach
  2. The same thing with not async socketsā€¦

I did a step by step debug and see that every outputBuffer has a size as expected and the method sendbinary doesnā€™t throw an exception.

I think the baudrate is default 9600. I didnā€™t find a property to setup a baudrate for this module in codeā€¦ Do you know how to set up the baudrate? and maybe i need to update the firmware of this module? but donā€™t know how yetā€¦

@ Alex Bilityuk - you can easily update the firmware with this driver and utility:
first select the Join Network Option. If this is works start the update Firmware Option.

https://www.ghielectronics.com/community/codeshare/entry/927

@ RoSchmi - :smiley: great driverā€¦ I missed it!