HTTP Web Server - System IO Exception

Howdy! I’m doing some tests running an HTTP server and I keep running into a thrown exception.

Below is a method that opens the server and continuously responds to requests. Since the GetContext() method is blocking, the method runs on its own thread. The request/response process works 90% of the time without error, but every once in a while I will put the path into the web browser and it will not be able to load, spinning indefinitely. When I cancel the request from the browser or request something different, an uncaught IO exception will be thrown. This usually happens more frequently when I make a lot of requests quickly.

I was wondering if anyone could explain why this is happening, if there is a recommended change I should implement, or if this is just a bug. I am using an SC20260D SOM.

Here is the exception stack trace:

[(2025-01-06 09:59:56) Error   - ThreadManager_HttpServer: Exception was thrown: System.IO.IOException.
 -> System.Net.Sockets.NetworkStream::Write
 -> System.Net.OutputNetworkStreamWrapper::Write
 -> GHI_Framework.HttpController::ServerListenerTask
 -> GHI_Framework.ThreadManager::RunThreadAsync

Here is my code example:

///<summary> Asynchronous HTTP server listener task. </summary>
private static void ServerListenerTask()
{
    //Print a log message and enable the listener flag
    Logger.LogMessage(Logger.Level.Info, ControllerId, $"Opening new HTTP server (IP: {EthernetController.IpAddress}, Port: 80)...");

    //Create a new HTTP listener
    _httpListener = new HttpListener("http", 80); //http = 80, https = 443
    _httpListener.Start();

    //Continuously listen until the network connection is closed
    while (_httpListener != null && _httpListener.IsListening && EthernetController.ConnectionEstablished)
    {
        //Close any previous requests
        _httpRequest?.Request?.Reset();
        _httpRequest?.Response?.Close();
        _httpRequest?.Reset();
        _httpRequest = null;

        try
        {
            //Wait for a new request (blocking)
            Logger.LogMessage(Logger.Level.Debug, ControllerId, "Awaiting new HTTP request...");
            _httpRequest = _httpListener.GetContext();
        }
        catch (Exception ex)
        {
            Logger.LogMessage(Logger.Level.Warning, ControllerId, $"ServerListenerTask() failed - {ex.Message}.");
            continue;
        }

        //Print a log message and invoke the request received event
        string urlPath = _httpRequest.Request.RawUrl.TrimStart('/');
        Logger.LogMessage(Logger.Level.Debug, ControllerId, $"HTTP request received (IP: {_httpRequest.Request.RemoteEndPoint.Address}): {EthernetController.IpAddress}/{urlPath}");
        EthernetController.NetworkIsHealthy();
        OnHttpRequestReceived?.Invoke(_httpRequest);

        //If the request is for the favicon, return the icon bytes
        if (SdCardController.ExtractPathChild(urlPath, '/') == "favicon.ico")
        {
            Logger.LogMessage(Logger.Level.Debug, ControllerId, "Sending \"favicon.ico\"...");
            /*byte[] faviconBytes = Resources.GetBitmap(Resources.BitmapResources.favicon2).GetBitmap();
            _httpRequest.Response.OutputStream.Write(faviconBytes, 0, faviconBytes.Length);*/
            _httpRequest.Response.Close();
            Logger.LogMessage(Logger.Level.Debug, ControllerId, "Response sent successfully.");
        }
        //If the request is for the network config utility
        else if (SdCardController.ExtractPathChild(urlPath, '/') == NetworkUtilityName)
        {
            Logger.LogMessage(Logger.Level.Debug, ControllerId, $"Sending \"{NetworkUtilityName}\"...");
            byte[] utilityBytes = Encoding.UTF8.GetBytes(Resources.GetString(Resources.StringResources.NetworkConfigUtility));
            _httpRequest.Response.OutputStream.Write(utilityBytes, 0, utilityBytes.Length);
            _httpRequest.Response.Close();
            Logger.LogMessage(Logger.Level.Debug, ControllerId, "Response sent successfully.");
        }
        //Otherwise process the URL as a file path
        else
        {
            //Ensure the SD card is connected
            if (!SdCardController.SdCardIsConnected)
            {
                byte[] nullResponse = Encoding.UTF8.GetBytes("\r\nERROR: SD CARD IS NOT AVAILABLE.");
                _httpRequest.Response.OutputStream.Write(nullResponse, 0, nullResponse.Length);
                _httpRequest.Response.Close();
                continue;
            }

            //Extract the file path
            string filePath = new StringBuilder(urlPath).Replace('/', '\\').ToString();

            //Ensure the SD card contains the file
            if (!string.IsNullOrEmpty(filePath) && !SdCardController.Exists(filePath))
            {
                byte[] nullResponse = Encoding.UTF8.GetBytes($"\r\nERROR: FILE PATH \"{urlPath}\" DOES NOT EXIST.\r\nFORMAT EXAMPLE: {EthernetController.IpAddress}/directory_name/file_example.txt");
                _httpRequest.Response.OutputStream.Write(nullResponse, 0, nullResponse.Length);
                _httpRequest.Response.Close();
                continue;
            }

            //Determine the path attributes
            FileAttributes attributes = File.GetAttributes($@"{SdCardController.RootDirectory.Name}" + filePath.ToLower());

            //If the path is a directory, read the files in the directory
            if ((attributes & FileAttributes.Directory) == FileAttributes.Directory)
            {
                //Extract all the files and directories in the provided path
                DirectoryInfo[] allFolders = SdCardController.ReadAllSubDirectories(filePath);
                FileInfo[] allFiles = SdCardController.ReadAllDirectoryFiles(filePath);
                string[] allObjectLinks = new string[allFolders.Length + allFiles.Length + 1];
                string[] utility = Resources.GetString(Resources.StringResources.UtilityInterface).Split('$');

                //If the directory is empty, send an empty response
                if (allObjectLinks.Length == 0)
                {
                    byte[] emptyResponse = Encoding.UTF8.GetBytes(utility[0] + FormatHtmlListItem("N/A") + utility[1]);
                    _httpRequest.Response.OutputStream.Write(emptyResponse, 0, emptyResponse.Length);
                    _httpRequest.Response.Close();
                    continue;
                }

                //Add the list of folders
                int index = 0;
                foreach (DirectoryInfo folder in allFolders)
                {
                    allObjectLinks[index] = FormatHtmlLink(folder.Name + "/", $"{urlPath}/{folder.Name.TrimEnd('\\')}", false);
                    index++;
                }

                //Add the list of files
                foreach (FileInfo file in allFiles)
                {
                    allObjectLinks[index] = FormatHtmlLink($"{file.Name} ({(file.Length / 1000):N0}kB)", $"{urlPath}/{file.Name.TrimEnd('\\')}", true);
                    index++;
                }

                //If the directory is the root directory, add the network utility link
                if (string.IsNullOrEmpty(filePath))
                {
                    allObjectLinks[allObjectLinks.Length - 1] = FormatHtmlLink("Network Config Tool", $"{urlPath}/{NetworkUtilityName}", false);
                }

                //Format the link list
                StringBuilder formattedList = new StringBuilder();
                for (int i = 0; i < allObjectLinks.Length; i++)
                {
                    if (!string.IsNullOrEmpty(allObjectLinks[i]))
                    {
                        formattedList.Append(FormatHtmlListItem(allObjectLinks[i]));
                    }
                }

                //Send the response and close the stream
                Logger.LogMessage(Logger.Level.Debug, ControllerId, $"Sending \"{SdCardController.ExtractPathChild(filePath, '\\')}\"...");
                byte[] fullResponse = Encoding.UTF8.GetBytes(utility[0] + formattedList + utility[1]);
                _httpRequest.Response.OutputStream.Write(fullResponse, 0, fullResponse.Length);
                _httpRequest.Response.Close();
                Logger.LogMessage(Logger.Level.Debug, ControllerId, "Response sent successfully.");
            }
            //If the path is a file
            else
            {
                //Create a packet ready event subscription
                SdCardController.OnFileTransferPacketReady += (filePacket, numBytes, packetIndex, totalPackets) =>
                {
                    //Verify the packet is not empty
                    if (filePacket == null || filePacket.Length == 0)
                    {
                        return;
                    }

                    //Send the packet
                    try
                    {
                        _httpRequest.Response.OutputStream?.Write(filePacket, 0, numBytes);
                    }
                    catch (IOException ex)
                    {
                        SdCardController.UpdateFileTransferStatus(SdCardController.FileTransferStates.StopTransfer);
                        Logger.LogMessage(Logger.Level.Warning, ControllerId, "HTTP FILE TRANSFER EXCEPTION");
                        Logger.LogExceptionError(ControllerId, ex);
                        _httpRequest.Response.Close();
                    }
                };

                //Start an HTTP file transfer - synchronously
                SdCardController.StartFileTransfer(ControllerId, filePath, (uint)(Memory.IsExtendedHeap() ? 1024 * 1024 : 1024 * 64), false, false);
                _httpRequest.Response.Close();
            }
        }
    }
}

Have you attempted to see if you can figure out which Write is experiencing the IOException? There are multiple writes, and knowing if it is specific to one is an important clue.

Is there any additional information available with exception?

No that I know of :confused: This is a copy/paste of the exception from the debug console:

#### Exception System.IO.IOException - 0x00000000 (7) ####
#### Message: 
#### System.Net.Sockets.NetworkStream::Write [IP: 00a4] ####
#### System.Net.OutputNetworkStreamWrapper::Write [IP: 0017] ####
#### GHI_Framework.HttpController::SendHttpResponse [IP: 0075] ####
#### GHI_Framework.HttpController::ServerListenerTask [IP: 03eb] ####
#### GHI_Framework.ThreadManager::RunThreadAsync [IP: 0043] ####

There was no additional information in the exception object viewed with the debugger?

There’s a lot of .Write’s in there - do you know which one is throwing? Maybe I read too quickly, but it isn’t obvious to me which one is failing. Narrowing that down with catch’s would be a good first step. Is it always the same one that throws?

Good idea!

Oh! Sorry, I misinterpreted the question. The exception is primarily throwing at these two sections:

//Send the response and close the stream
Logger.LogMessage(Logger.Level.Debug, ControllerId, $"Sending \"{SdCardController.ExtractPathChild(filePath, '\\')}\"...");
byte[] fullResponse = Encoding.UTF8.GetBytes(utility[0] + formattedList + utility[1]);
_httpRequest.Response.OutputStream.Write(fullResponse, 0, fullResponse.Length);
_httpRequest.Response.Close();
Logger.LogMessage(Logger.Level.Debug, ControllerId, "Response sent successfully.");
//Send the packet
try
{
    _httpRequest.Response.OutputStream?.Write(filePacket, 0, numBytes);
}
catch (IOException ex)
{
    SdCardController.UpdateFileTransferStatus(SdCardController.FileTransferStates.StopTransfer);
    Logger.LogMessage(Logger.Level.Warning, ControllerId, "HTTP FILE TRANSFER EXCEPTION");
    Logger.LogExceptionError(ControllerId, ex);
    _httpRequest.Response.Close();
}

Although it’s probably important to note that these two sections are also the ones being called most often – they send either a directory path or a file. The other “writes” are more edge case.

Again, that the exception is thrown ~30% of the time when I rapidly send requests (I assume before the previous request is able to finish).

Try adding a Flush after the write and before the close.

if that does not work, try adding a short Sleep after the write.

1 Like

Alright…after some initial testing it looked like the .Flush() did help a little, the web server pages are a little snappier in the browser, but unfortunately I wasn’t able to shake the IOException :confused:. I tried a sleep, a flush, and both together and it continued to haunt me.

Even worse, while testing I found that every so often the request would hang in the browser like before but refreshing would not fix it, no IOException or anything. Instead, the GHI’s CPU usage maxes out at 100% and the browser just spins indefinitely. I’ve been able to repeat this a handful of times now which is really interesting :thinking:. Maybe there’s a while loop getting stuck in the HTTP nuget library?

If it helps at all, I’m testing using both Chrome and Firefox.

EDIT: Just tried calling the Reset() function AND tried removing the ethernet cable while it was stalling like that and neither of those fixed it.

You never responded to my question about additional IOException information available via the debugger.

Also, try an additional very simple response. Just a “Hello World”.

This could eliminate any issue with the contents of the responses. Locking of the browser is suspicious.

I am a bit confused. How could a loop in the nuget library cause the remote browser to go to 100% CPU?

Sorry, the GHI CPU usage goes to 100%, not the browser/PC (just made an edit).

As for the debug console, the only information I’m getting is from the exception message I posted before:

#### Exception System.IO.IOException - 0x00000000 (7) ####
#### Message: 
#### System.Net.Sockets.NetworkStream::Write [IP: 00a4] ####
#### System.Net.OutputNetworkStreamWrapper::Write [IP: 0017] ####
#### GHI_Framework.HttpController::SendHttpResponse [IP: 0075] ####
#### GHI_Framework.HttpController::ServerListenerTask [IP: 03eb] ####
#### GHI_Framework.ThreadManager::RunThreadAsync [IP: 0043] ####

Short responses like “Hello, world” seems to operate fine. If I extend the response length past about 1kB, I’m able to get the exception to trigger again.

I am not talking about the debug console.

I am talking about using the Debugger to examine the contents of the IOException. The exception could contain a Windows socket error code and/or a text explanation of the exception cause. It could also tell us nothing…

1 Like

Now this is interesting.

Can you prepare a simple project which displays this exception? My TinyClr environment is not active at the moment, but someone else could try to reproduce the problem.

If the problem can be reproduced, then you can submit a issue on Github.

Yeah I’ll work on it, might be a few days to package something up (working on a few other things too) but I’ll keep in touch.

If you suspect this, then you can de-select “Just My Code” in the debugger settings, debug your app, and when it hangs with 100% cpu hit the ‘break’ button (looks like a pause button) and it should break within whatever tight loop it’s in. At the very least, that will tell you whether it is caught looping in managed or unmanaged code, and if it’s a look in the nuget managed code, will tell you exactly where.

1 Like

I created a GIT hub issue with an example project. I was able to get the exception to throw in this project as well so I know it’s still there. Here is the link to the project/post: