Socket programming chaos

I’ve been struggling with getting the socket connection between a Spider and WinPhone7 client to work correctly but it just isn’t happening. I started out with Marco’s SocketServer class and the MSDN example socket client for using SocketAsyncEventArgs for WP7 (WP7 can’t use the regular non-async socket methods) and, since neither of them supports sending data from server to client I built out logic to do that. Total message length is determined by prepending the four-byte integer length to the message, once the connection is established the client receives over and over and whenever a full message has been received an event on the client class is raised. This works fine for occasionally sending small amounts of data but some really strange stuff is happening when I send a 57000 byte test pattern. If I do it as a single socket.Send(data) method call the client bombs out because it tries to read something that isn’t the first received packet as the first received packet (leading to thinking it’s got a huge message and exceeding the 80k message-processing buffer) and if I break it into 1024 byte chunks and send each individually only 42-43 of the 56 total chunks get processed.

Either way, the issue is that data is clearly being lost between the server send method and when the client is finished processing the message. I’m relatively sure the problem lies in the client’s receive handler, which is wired up to the SocketAsyncEventArgs Completed event, though being socket communication betweeen two very niche technologies it could be anything. Here’s the relevant code for both the main receive loop and the completed event handler method:

        void socketEventArg_Completed(object sender, SocketAsyncEventArgs e)
        {
            try
            {
                if (e.SocketError == SocketError.Success)
                {
                    if (!isReceiving)
                    {
                        // If this is the first part of the message, pull off the first four bytes as that's our length
                        byte[] len = new byte[4];
                        len[0] = e.Buffer[0];
                        len[1] = e.Buffer[1];
                        len[2] = e.Buffer[2];
                        len[3] = e.Buffer[3];

                        msgLen = BitConverter.ToInt32(len, 0);// maybe refactor this to get rid of len and just hit buffer?
                        isReceiving = true;
                        rcvIndex = 0;
                        chunks = 0;
                        buffer = new byte[MAX_MSG_SIZE];
                        partList.Clear();
                    }
                    // Copy this into the buffer at the appropriate spot
                    chunks++;

                    // Find any occurrences of 101 and read the next thing in sequence
                    for (int v = 0; v < e.Buffer.Length - 1; v++)
                    {
                        if (e.Buffer[v] == 101)
                            partList.Add(e.Buffer[v + 1]);
                    }
                    // NOTE: This only works properly if the total message length is always less than the buffer
                    //       which for this application is true--the max msg size is 80k but images are only around 60
                    e.Buffer.CopyTo(buffer, rcvIndex);
                    rcvIndex += e.BytesTransferred;
                    // Is this the end of the message?
                    if (rcvIndex >= msgLen)
                    {
                        count++;
                        // Done receiving, so do the callback
                        isReceiving = false;
                        OnMessageReceived(new SocketReceivedEventArgs(buffer, chunks));
                    }
                    // Call the loop again
                    ReceiveLoop();
                }
                else
                {
                    OnMessageReceived(new SocketReceivedEventArgs(Encoding.UTF8.GetBytes(e.SocketError.ToString()), 0));
                }
            }
            catch
            {
                partList.Add(-1);
                throw;
            }
        }

        private void ReceiveLoop()
        {
            // We are receiving over an established socket connection
            if (_socket != null)
            {
                // Make an asynchronous Receive request over the socket
                _socket.ReceiveAsync(socketEventArg);
            }
            else
            {
                OnMessageReceived(new SocketReceivedEventArgs(Encoding.UTF8.GetBytes("Socket is not initialized"), 0));
            }
        }

Any ideas what I’m doing wrong? My best guess is that the buffer is getting overwritten while socketEventArg_Completed is still processing, though that doesn’t make a lot of sense to me given that the next receive isn’t fired until the processing is done.

Is there a specific reason you need to use an Asynchronous socket method as opposed to running a thread that calls the blocking receive method? If not, then you can give this a try for your receive loop:


while (true)
            {
                //Accept the next in-line connection
                using (Socket incoming = socket.Accept())
                {
                    //Gather incoming data with our own time out to determine our own behavior
                    DateTime TimeOut = DateTime.Now.AddMilliseconds(ReceiveTimeOut);
                    int bytes_available = 0;
                    while (TimeOut > DateTime.Now)
                    {
                        if ((bytes_available = incoming.Available) > 0)
                        {
                            byte[] read = new byte[bytes_available];
                            incoming.Receive(read, bytes_available, SocketFlags.None);

                            ...

                            //Reset the time-out
                            TimeOut = DateTime.Now.AddMilliseconds(ReceiveTimeOut);

                        }

                        Thread.Sleep(10);
                    }
                }

                Thread.Sleep(10);
            }

Unfortunately yes, WP7 does not include the blocking synchronous methods–it’s SocketAsyncEventArgs or nothing. However, I think I’ve figured out what the problem is. Not only can a single Send() be split into many receives, if the client packet processing gets backed up multiple packets can be combined into a single logical receive. So what was happening was the end of one message and the beginning of the next were getting rolled together, the system read to the end of the current message but discarded the remainder and then on the next receive, it treated it as a fresh message when really the message had already been partially received and discarded.

I’ll get the fixed code off my laptop hopefully tomorrow and post it.

That’s because TCP is a “stream” protocol, and not a “datagram” protocol, like UDP.

Here’s the fixed code, weird things are happening when I send very large (>50000 bytes) messages so there’s still probably some problems with it but it handles what I need just fine now. Also pulled out the debugging code in case anyone needs this for something–it was made for WP7 but should work for other .NET platforms too.

        void socketEventArg_Completed(object sender, SocketAsyncEventArgs e)
        {
            try
            {
                if (e.SocketError == SocketError.Success)
                {
                    ProcessReceive(e.Buffer, e.BytesTransferred, e.Offset);
                }
                else
                {
                    OnMessageReceived(new SocketReceivedEventArgs(Encoding.UTF8.GetBytes(e.SocketError.ToString()), 0));
                    // Also close the connection--there's been some error
                    this.Close();
                }
            }
            catch
            {
                throw;
            }
        }

        private void ProcessReceive(byte[] rcvBuffer, int bytesTransferred, int offset)
        {
            // Retrieve the data from the buffer
            // Is this the first receive?
            if (!isReceiving)
            {
                // If this is the first part of the message, pull off the first four bytes as that's our length
                byte[] len = new byte[4];
                len[0] = rcvBuffer[offset];
                len[1] = rcvBuffer[offset+1];
                len[2] = rcvBuffer[offset+2];
                len[3] = rcvBuffer[offset+3];

                msgLen = BitConverter.ToInt32(len, 0) + 4;// maybe refactor this to get rid of len and just hit buffer?
                isReceiving = true;
                rcvIndex = 0;
                chunks = 0;
                buffer = new byte[MAX_MSG_SIZE];
            }
            // Copy this into the buffer at the appropriate spot
            chunks++;


            // NOTE: This only works properly if the total message length is always less than the buffer
            //       which for this application is true--the max msg size is 80k but images are only around 60
            try
            {
                Array.Copy(rcvBuffer, offset, buffer, rcvIndex, bytesTransferred);
                rcvIndex += bytesTransferred;
                // Is this the end of the message?
                if (rcvIndex >= msgLen)
                {
                    count++;
                    // Done receiving, so do the callback
                    isReceiving = false;
                    OnMessageReceived(new SocketReceivedEventArgs(buffer, chunks));

                    // It's possible that the sends will get stacked while receive is still processing
                    // and apparently .NET doesn't separate them by message so a single receive
                    // may actually include the end of one message and the start of another
                    // If this is the case, the index is greater than the message length and so
                    // we'll want to kick off another receive
                    if (rcvIndex > (msgLen + 4)) // the 4 is for the prepended msg length
                    {
                        // Figure out the offset of the new message
                        // basically, the received message length minus the difference between rcvIndex and msgLen
                        int newTransferred = rcvIndex - msgLen;
                        int newOffset = offset + (bytesTransferred - newTransferred);
                        
                        // Now call this thing again
                        ProcessReceive(rcvBuffer, newTransferred, newOffset);
                        return; // don't need to call ReceiveLoop twice
                    }
                }
            }
            catch(Exception ex)
            {
                throw ex;
            }

            // Call the loop again
            ReceiveLoop();

        }