Spider II and SocketException ErrorCode = 10053

Hi

I’am building for some time CCTV keyboard controller for IP PTZ camera with Spider II and ethernetJ11D (.NETMF 4.3 SDK 2015 R1). After sending few hundreds http requests each time it ends the same with SocketException ErrorCode = 10053.

Here is my code that makes trouble

 static void NewSendSocket(Message message, int count)
        {
            string serverIP = "192.168.100.10";
            int serverPort = 80;
            const Int32 c_microsecondsPerSecond = 100000;
            Socket serverSocket = ConnectSocket(serverIP, serverPort);

            try
            {
                if (serverSocket != null)
                {
                    // Create a socket connection to the specified server and port.
                    using (serverSocket)
                    {
                        // Send request to the server.
                        String request = "GET " + message.data + " HTTP/1.1\r\n" +
                            "Host:" + serverIP + "\r\n" +
                            "Content-Length:" + message.data.Length + "\r\n" +
                            "Connection:close\r\n" +
                            "Cache-Control:no-cache\r\n" +
                            "\r\n";

                        Byte[] bytesToSend = Encoding.UTF8.GetBytes(request);
                        serverSocket.Send(bytesToSend, bytesToSend.Length, 0);

                        Debug.Print("Socket " + serverSocket.LocalEndPoint.ToString() + " : " + message.command.ToString() + " Length: " + message.data.Length + "  Count: " + count.ToString() + " at " + DateTime.UtcNow.TimeOfDay.ToString() + " Thread :" + Thread.CurrentThread.ManagedThreadId.ToString());

                        if (message.parseResponse)
                        {
                            // Allocate a buffer that we'll keep reusing to receive HTML chunks
                            Byte[] buffer = new Byte[1024];

                            // 'page' refers to the HTML data as it is built up.
                            String page = String.Empty;

                            // Poll for data until 1 second time out - Returns true for data and connection closed

                            while (serverSocket.Poll((30 * c_microsecondsPerSecond), SelectMode.SelectRead))
                            {
                                // Zero all bytes in the re-usable buffer
                                Array.Clear(buffer, 0, buffer.Length);

                                // Read a buffer-sized HTML chunk
                                Int32 bytesRead = serverSocket.Receive(buffer);

                                // If 0 bytes in buffer, then connection is closed
                                if (bytesRead == 0)
                                    break;

                                // Append the chunk to the string
                                page = page + new String(Encoding.UTF8.GetChars(buffer));
                            }

                            if (page != "") preParseResponse(page);
                        }
                    }
                    Debug.Print("Socket relased at: " + DateTime.UtcNow.TimeOfDay.ToString());
                }
                else
                {
                    Debug.Print("Socket null");
                }
            }
            catch (Exception e)
            {
                
                Debug.Print("new Sender Excp");
                if (serverSocket != null) serverSocket.Close();
            }
        }


        private static Socket ConnectSocket(String server, Int32 port)
        {
            // Get server's IP address.
            //  IPHostEntry hostEntry = Dns.GetHostEntry(server);

            try
            {

                IPAddress hostEntry = IPAddress.Parse(server);
                IPEndPoint hostep = new IPEndPoint(hostEntry, port);

                // Create socket and connect to the server's IP address and port
                Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                socket.Connect(hostep);
                Debug.Print("Socket connected. LocalEndPoint " + socket.LocalEndPoint.ToString() + " RemoteEndPoint " + socket.RemoteEndPoint.ToString());
                return socket;
            }
            catch
            {
                Debug.Print("Socket connection failed: " + server);
            }

            return null;
            

        }

Below some info form Output window with proper application behaviour and then on 00:27:06.5090757 it jammed for 45 sek

Message. Queue cont: 2 at 00:27:04.7541167
************** SendEvent Set ************************
Socket connected. LocalEndPoint 192.168.100.14:52004 RemoteEndPoint 192.168.100.10:80
Socket 192.168.100.14:52004 : GET_PAN_POS Length: 106 Count: 1 at 00:27:05.4710115 Thread :8
Socket relased at: 00:27:05.5351009
Response Queue count:1
----- Sender queue count: 2
----- Sender overal count: 1513
Message. Queue cont: 2 at 00:27:05.7941196
************** SendEvent Set ************************
Azymut :83 00:27:05.7978129
Socket connected. LocalEndPoint 192.168.100.14:52005 RemoteEndPoint 192.168.100.10:80
Socket 192.168.100.14:52005 : SET_OSD Length: 163 Count: 2 at 00:27:06.4083988 Thread :8
Socket relased at: 00:27:06.4338229
----- Sender queue count: 2
----- Sender overal count: 1514
Message. Queue cont: 2 at 00:27:06.5090757
************** SendEvent Set ************************
Message. Queue cont: 3 at 00:27:07.2578566
************** SendEvent Set ************************
Message. Queue cont: 4 at 00:27:07.9710597
************** SendEvent Set ************************

Message. Queue cont: 64 at 00:27:52.0062181
************** SendEvent Set ************************
Socket connected. LocalEndPoint 232.202.159.160:0 RemoteEndPoint 248.231.159.160:0
#### Exception System.Net.Sockets.SocketException - CLR_E_FAIL (8) ####
#### Message:
#### Microsoft.SPOT.Net.SocketNative::send [IP: 0000] ####
#### System.Net.Sockets.Socket::Send [IP: 0018] ####
#### System.Net.Sockets.Socket::Send [IP: 0008] ####
#### VMS_ZERO_SERVER_SPIDER_II.MessageHandler::NewSendSocket [IP: 0077] ####
#### VMS_ZERO_SERVER_SPIDER_II.MessageHandler::Sender [IP: 0078] ####
#### SocketException ErrorCode = 10053
#### SocketException ErrorCode = 10053
A first chance exception of type ‘System.Net.Sockets.SocketException’ occurred in Microsoft.SPOT.Net.dll
#### SocketException ErrorCode = 10053
#### SocketException ErrorCode = 10053
new Sender Excp

What is wondering me much, why Socket EndPoints changed just before throwing exception:

Socket connected. LocalEndPoint 232.202.159.160:0 RemoteEndPoint 248.231.159.160:0

while it should be

Socket connected. LocalEndPoint 192.168.100.14:52005 RemoteEndPoint 192.168.100.10:80

Can anybody help here and advise what to fix and how.

PS
Sorry for my English - it is not my native language

Today I reached 12 996 reqests after 2,5 hrs work, sending one req every 700 ms.

Socket connected. LocalEndPoint 192.168.100.14:65009 RemoteEndPoint 192.168.100.10:80
Socket 192.168.100.14:65009 : SET_OSD Length: 167 Count: 1 at 04:01:38.2097120 Thread :8
Socket relased at: 04:01:38.2156793
----- Sender queue count: 1
----- Sender overal count: 12996

and again - strange Socket Local and Remote EndPoints change just before SocketException

Socket connected. LocalEndPoint 8.222.159.160:0 RemoteEndPoint 8.222.159.160:0
#### Exception System.Net.Sockets.SocketException - CLR_E_FAIL (8) ####
#### Message:
#### Microsoft.SPOT.Net.SocketNative::send [IP: 0000] ####
#### System.Net.Sockets.Socket::Send [IP: 0018] ####
#### System.Net.Sockets.Socket::Send [IP: 0008] ####
#### VMS_ZERO_SERVER_SPIDER_II.MessageHandler::NewSendSocket [IP: 0076] ####
#### VMS_ZERO_SERVER_SPIDER_II.MessageHandler::Sender [IP: 0078] ####
#### SocketException ErrorCode = 10053
#### SocketException ErrorCode = 10053
A first chance exception of type ‘System.Net.Sockets.SocketException’ occurred in Microsoft.SPOT.Net.dll
#### SocketException ErrorCode = 10053
#### SocketException ErrorCode = 10053
new Sender Excp

Have you tried the latest pre release?

Thank you for reply and suggestion. Now I’ve installed new SDK.
Also added two lines after socket.Connect

{0,0,0,0}); 
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.NoDelay, true);

what was suggested in this article [url]http://www.dotnetsolutions.co.uk/linger-error/[/url]
and raised my hopes to solve the problem :slight_smile:

Unfortunately now application behaves much less predictable then before. One time I reached over 20000 requests, once less then 500. But everytime it ends up with jammed socket. What is worse - now I have spontaneous and uncontrolled application restart instead of catching Exception. It looks like VS Debugger is losing its connection to process, since I have no Exception info in VS Output window

************** SendEvent Set ************************
----- Sender queue count: 1
----- Sender overal count: 810
Message. Queue cont: 1 at 00:10:27.5255763
************** SendEvent Set ************************
----- Sender queue count: 1
----- Sender overal count: 811
Message. Queue cont: 1 at 00:10:28.2563524
************** SendEvent Set ************************
----- Sender queue count: 1
----- Sender overal count: 812
Socket relased at: 00:10:28.3711997
Message. Queue cont: 1 at 00:10:29.0164666
************** SendEvent Set ************************
----- Sender queue count: 1
----- Sender overal count: 813
Message. Queue cont: 1 at 00:10:29.7454048
************** SendEvent Set ************************
----- Sender queue count: 1
----- Sender overal count: 814

So now it end up with no Exception, nor other Debug.Print info. After few seconds after last info in VS Output window application restarts itself.
(have no Watchdog feature setup).

After removing

socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.Linger, new Byte []{0,0,0,0});
socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.NoDelay, true);

application came back to its “normal” state - throwin an Exception instead of self-restart.

And still somehow Local nad RemoteEndPoints are chenged to strange value.

Are there any chance for help?

@ przemo - Have you tried a simple test application against other servers?

From the C# language reference:

In the problem code, you are not declaring or instantiating the socket within the using statement. When everything go well, the socket is not disposed and allowed to go out of scope, for later handling by GC. When there is an error, the socket is disposed by a close.

Try declaring and instantiating within the using statement.

@ John - not yet. But application talks to industrial grade CCTV camera Bosch MIC 7230 and I’m sure it is capable to serve all requests.
We use similar application (using windows socket and HTTP requests) runing on Embedded PC platform for 2 yers since now and there are no issues.
I tried also hard reset camera after exception occured but there was no result - socket was still jamed and only way to make it running again was Spider hard reset.

@ Mike - thank you for advice. I changed that line to



but no sucess.

I reached 50k request in 8 hr but ended with this same:
Strange IP addresses as Local and Remote EndPoints and ErrorCode = 10053

[b]Socket connected. LocalEndPoint 216.219.149.160:0 RemoteEndPoint 52.34.159.160:0[/b]
    #### Exception System.Net.Sockets.SocketException - CLR_E_FAIL (50896) ####
    #### Message: 
    #### Microsoft.SPOT.Net.SocketNative::send [IP: 0000] ####
    #### System.Net.Sockets.Socket::Send [IP: 0018] ####
    #### System.Net.Sockets.Socket::Send [IP: 0008] ####
    #### VMS_ZERO_SERVER_SPIDER_II.MessageHandler::NewSendSocket [IP: 0071] ####
    #### VMS_ZERO_SERVER_SPIDER_II.MessageHandler::Sender [IP: 005f] ####
    #### SocketException ErrorCode = 10053

Few minutes before crash GC had following results:

GC: 58msec 771036 bytes used, 6568632 bytes available
Type 0F (STRING              ):   3432 bytes
Type 11 (CLASS               ):  19416 bytes
Type 12 (VALUETYPE           ):   2688 bytes
Type 13 (SZARRAY             ):   9768 bytes
  Type 01 (BOOLEAN             ):     24 bytes
  Type 03 (U1                  ):   1380 bytes
  Type 04 (CHAR                ):   1032 bytes
  Type 07 (I4                  ):   1128 bytes
  Type 0F (STRING              ):   1296 bytes
  Type 11 (CLASS               ):   4740 bytes
  Type 12 (VALUETYPE           ):    168 bytes
Type 15 (FREEBLOCK           ): 6568632 bytes
Type 16 (CACHEDBLOCK         ): 101520 bytes
Type 17 (ASSEMBLY            ):  47484 bytes
Type 18 (WEAKCLASS           ):     96 bytes
Type 19 (REFLECTION          ):    168 bytes
Type 1B (DELEGATE_HEAD       ):   2196 bytes
Type 1C (DELEGATELIST_HEAD   ):    108 bytes
Type 1D (OBJECT_TO_EVENT     ):    504 bytes
Type 1E (BINARY_BLOB_HEAD    ): 570264 bytes
Type 1F (THREAD              ):   3456 bytes
Type 20 (SUBTHREAD           ):    336 bytes
Type 21 (STACK_FRAME         ):   3516 bytes
Type 22 (TIMER_HEAD          ):    432 bytes
Type 26 (WAIT_FOR_OBJECT_HEAD):    144 bytes
Type 27 (FINALIZER_HEAD      ):    144 bytes
Type 31 (IO_PORT             ):    108 bytes
Type 34 (APPDOMAIN_HEAD      ):     72 bytes
Type 36 (APPDOMAIN_ASSEMBLY  ):   5184 bytes

Please help !

@ przemo - Can you post as minimal of a complete program as possible that reproduces the issue for us to test here?

@ John
So here it is.
For our purpose important is to have sending speed 5 Hz minimum.
In this example I omit parsing response void but I think this has no influence on exception occurrence

.
using System;
using System.Net;
using System.Net.Sockets;
using System.Collections;
using System.Threading;
using System.Text;
using Microsoft.SPOT;
using Microsoft.SPOT.Presentation;
using Microsoft.SPOT.Presentation.Controls;
using Microsoft.SPOT.Presentation.Media;
using Microsoft.SPOT.Presentation.Shapes;
using Microsoft.SPOT.Touch;
using Microsoft.SPOT.Net.NetworkInformation;

using Gadgeteer.Networking;
using GT = Gadgeteer;
using GTM = Gadgeteer.Modules;
using Gadgeteer.Modules.GHIElectronics;

namespace VMS_Socket_Test
{
    public partial class Program
    {

        bool NetworkReady = false;
        string deviceIp = "192.168.100.14";
        string serverIp = "192.168.100.10";
        byte[] EthMac = new byte[] { 0x9E, 0x12, 0x81, 0x79, 0x3B, 0x04 };
        bool useDHCP = false;
 
        void ProgramStarted()
        {

            Debug.Print("Program Started");
                
            if (true)
            {
                var EthernetThread = new Thread(
                   () => EthernetInitialize());
                EthernetThread.Start();
            }

            Debug.Print("Startup suceeded!");
        }//Main

        
        #region Ethernet
        
        bool EthernetInitialize()
        {

            ethernetJ11D.NetworkInterface.PhysicalAddress = EthMac;
            Debug.Print("Opening interface...");
            ethernetJ11D.NetworkInterface.Open();
            if (useDHCP == false)
            {
                Debug.Print("Set static IP...");
                ethernetJ11D.NetworkInterface.EnableStaticIP(deviceIp, "255.255.255.0", "192.168.100.1");
                ethernetJ11D.NetworkInterface.EnableStaticDns(new string[] { "8.8.8.8", "192.168.100.1" });
            }
            else
            {
                Debug.Print("Get DHCP IP...");
                //// DHCP IP
                if (ethernetJ11D.NetworkInterface.IsDhcpEnabled == false)
                {
                    ethernetJ11D.NetworkInterface.EnableDhcp();
                    ethernetJ11D.NetworkInterface.EnableDynamicDns();
                }
            }

            ethernetJ11D.UseThisNetworkInterface();

            NetworkChange.NetworkAvailabilityChanged += new NetworkAvailabilityChangedEventHandler(NetworkChange_NetworkAvailabilityChanged);
            NetworkChange.NetworkAddressChanged += new NetworkAddressChangedEventHandler(NetworkChange_NetworkAddressChanged);

            int cnt = 0;
            while (NetworkReady == false)
            {
                Debug.Print("Waiting for network ready " + (cnt++));
                Thread.Sleep(1500);
            }
            NetworkReady = true;
            Thread.Sleep(3000);
            return NetworkReady;
        }

        void NetworkChange_NetworkAvailabilityChanged(object sender, NetworkAvailabilityEventArgs e)
        {
            Debug.Print("Network has changed! " + e.IsAvailable.ToString());
            NetworkReady = e.IsAvailable;
        }

        private void NetworkChange_NetworkAddressChanged(object sender, EventArgs e)
        {
            Debug.Print("New address for the Network Interface ");
            Debug.Print("------------------------------------------------------");
            if (ethernetJ11D.NetworkInterface.IPAddress != "0.0.0.0")
            {
                NetworkReady = true;
            }

            Thread.Sleep(3000);
            MessageHandler.Init();
            Thread.Sleep(3000);
            WorkerStart();
        }


        # endregion

        void WorkerStart()
        {
            Debug.Print("Worker started.");
            string command = "/rcp.xml?command=0x09A5&type=P_OCTET&direction=WRITE&num=1&payload=0x810006011201"; // HTTP request string

            while (true)
            {
                try
                {
                    MessageHandler.SendRCPCommand(serverIp, command, "GET_PAN_POS", true);              
                    Thread.Sleep(200);
                 }
                catch
                {
                    Debug.Print("!!!!!!!!!!!!!!!!!  Worker Exception");
                    Thread.Sleep(1000);
                    Debug.GC(true);
                }

            }
        }


    }

    public class Message
    {
        public string PtuIp = "";
        public string data = "";
        public string command = "";
        public DateTime born = DateTime.UtcNow;
        public bool parseResponse = false;
    }

    public static class MessageHandler 
    {
        private static Queue messagesToSend = new Queue();
        private static Thread sendThread = null;
        private static Thread parseThread = null;

        public static void Init()
        {

            try
            {

                sendThread = new Thread(new ThreadStart(Sender));
                sendThread.Priority = ThreadPriority.AboveNormal;
                sendThread.Start();

                Debug.Print("****** Message Handler Init Suceeded");
            }
            catch
            {
                Debug.Print("!!!!!!!!!! Message Handler Init Failed!");
            }
        }

        static void Sender()
        {
            Debug.Print("Starting Sender() thread.");
            Int32 senderCounter = 0;
            Int32 overallCounter = 0;

            while (true)
            {
                senderCounter = 0;
                try
                {
                 
                    while (messagesToSend.Count > 0)
                    {
                        try
                        {
                            senderCounter++;
                            overallCounter++;

                            Debug.Print("----- Sender queue count: " + messagesToSend.Count.ToString());
                            Debug.Print("----- Sender overal count: " + overallCounter.ToString());

                            Message message = null;

                            lock (messagesToSend)
                            {
                                message = messagesToSend.Dequeue() as Message;
                            }

                            if (message != null)
                            {
                                
                                SendSocket(message, senderCounter);

                            }
                            Thread.Sleep(70);

                            if (messagesToSend.Count > 55)
                            {
                                messagesToSend.Clear();
                                Debug.Print(" !!!!!!! Clear Queue");
                            }

                        }
                        catch
                        {
                            Debug.Print("ERROR Sender");
                        }
                    }
                }
                catch (Exception exception)
                {
                    Debug.Print("Error 90");
                }
            }
        }

        public static void SendRCPCommand(string IPAdr, string data, string command, bool ParseResponse)
        {
            Message message = new Message();
            message.PtuIp = IPAdr;
            message.data = data;
            message.command = command;
            message.parseResponse = ParseResponse;
            message.born = DateTime.UtcNow;
            EnqueueMessage(message);
        }

        private static void EnqueueMessage(Message message)
        {
            lock (messagesToSend)
            {
                messagesToSend.Enqueue(message);
            }
            Debug.Print("Message. Queue cont: " + messagesToSend.Count.ToString() + " at " + DateTime.UtcNow.TimeOfDay.ToString());
        }

        static void SendSocket(Message message, int count)
        {
            string serverIP = "192.168.100.10";
            int serverPort = 80;
            const Int32 c_microsecondsPerSecond = 100000;
            IPAddress hostEntry = IPAddress.Parse(serverIP);    // UDP
            IPEndPoint hostep = new IPEndPoint(hostEntry, serverPort);  // UDP

            try
            {
                 using (Socket serverSocket = ConnectSocket(serverIP, serverPort))
                {
                        String request = "GET " + message.data + " HTTP/1.1\r\n" +
                        "Host:" + serverIP + "\r\n" +
                        "Content-Length:" + message.data.Length + "\r\n" +
                        "Connection:close\r\n" +
                        "Cache-Control:no-cache\r\n" +
                        "\r\n";

                    Byte[] bytesToSend = Encoding.UTF8.GetBytes(request);
                    serverSocket.Send(bytesToSend, bytesToSend.Length, 0); // TCP
                    TimeSpan delay = (DateTime.Now - message.born);
                    Debug.Print("Socket sent " + serverSocket.LocalEndPoint.ToString() + " : " + message.command.ToString() + " Length: " + message.data.Length + "  Count: " + count.ToString() + " at " + DateTime.UtcNow.TimeOfDay.ToString() + " m created at: " + message.born.TimeOfDay.ToString() + " delay: " + delay.Milliseconds.ToString() + ". Thread :" + Thread.CurrentThread.ManagedThreadId.ToString());

                    if (message.parseResponse)
                    {
                        Byte[] buffer = new Byte[1024];
                        String page = String.Empty;

                        while (serverSocket.Poll((30 * c_microsecondsPerSecond), SelectMode.SelectRead))
                        {
                            Array.Clear(buffer, 0, buffer.Length);
                            Int32 bytesRead = serverSocket.Receive(buffer);
                            if (bytesRead == 0)
                                break;
                            page = page + new String(Encoding.UTF8.GetChars(buffer));
                        }

                        if (page != "")
                        {
                            Debug.Print("Page :" +  page);
                        }
                    }
                }
                Debug.Print("Socket relased at: " + DateTime.UtcNow.TimeOfDay.ToString());
            }
            catch (Exception e)
            {
                Debug.Print("new Sender Excp");
            }
        }
        
        private static Socket ConnectSocket(String server, Int32 port)
        {
            try
            {
                IPAddress hostEntry = IPAddress.Parse(server);
                IPEndPoint hostep = new IPEndPoint(hostEntry, port);
                Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                socket.SendTimeout = 500;
                socket.Connect(hostep);
                Debug.Print("Socket connected. LocalEndPoint " + socket.LocalEndPoint.ToString() + " RemoteEndPoint " + socket.RemoteEndPoint.ToString());
                return socket;
            }
            catch
            {
                Debug.Print("Socket connection failed: " + server);
            }
            return null;
        }
    }
}

@ przemo - What do we connect the program to? We need something to tests against here that fails for you.

@ John
I could be problem. It conects to CCTV IP camera. Do you have any avaliable in your office? If you could give me name and type I colud prepare proper HTTP request string. If not I will prepare server side code asap.

@ przemo - Unfortunately we do not

@ John
In a meantime we tried our app on Raptor + ENC28 and it works for few days with 1,2 M sent and parsed requests without issue.
We have to proceed our project so decided to change platform to G400 and I would like close this thread now.

But in my opinion this issue with Socket on SpiderS you should investigate further, since really smth is wrong.

You may test my code against any Http Server listening on port 80 and sending any response for request.

Thank you

@ przemo - We will take a look