HttpListener crashes

Hi all,

I’m experimenting for the first time with the httplistener class. I’m making a little test application based on the http server example of which the full code is below. I’m running this on a Cerbuino NET.

Two questions currently:
[ul]When I navigate to the ip address in my browser there seem to be 2 requests, this is the output:
[/ul]

Context counter: 1
Request handler counter: 1
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
Context counter: 2
Request handler counter: 2
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
The thread '<No Name>' (0x6) has exited with code 0 (0x0).
The thread '<No Name>' (0x7) has exited with code 0 (0x0).

[ul]When I keep refreshing the page in the browser quickly the application crashes with this message: “An unhandled exception of type ‘System.NullReferenceException’ occurred in System.Http.dll” and it tries to break in System.Net._OutputNetworkStreamWrapper.cs. This is the output[/ul]

Found debugger!

Create TS.

 Loading start at 8078af4, end 809ea80

   Assembly: mscorlib (4.3.1.0)     Assembly: Microsoft.SPOT.Native (4.3.1.0)     Assembly: Microsoft.SPOT.Hardware (4.3.1.0)  
   Assembly: Microsoft.SPOT.Graphics (4.3.1.0)     Assembly: Microsoft.SPOT.TinyCore (4.3.1.0)  
   Assembly: Microsoft.SPOT.IO (4.3.1.0)     Assembly: System.IO (4.3.1.0)     Assembly: Microsoft.SPOT.Hardware.Usb (4.3.1.0) 
    Assembly: Microsoft.SPOT.Hardware.SerialPort (4.3.1.0)     Assembly: Microsoft.SPOT.Hardware.PWM (4.3.1.0)  
Loading Deployment Assemblies.

Attaching deployed file.

   Assembly: GHI.Hardware (4.3.6.0)  Attaching deployed file.

   Assembly: GatewayServer (1.0.0.0)  Attaching deployed file.

   Assembly: System.Http (4.3.1.0)  Attaching deployed file.

   Assembly: System (4.3.1.0)  Attaching deployed file.

   Assembly: System.Net.Security (4.3.1.0)  Attaching deployed file.

   Assembly: GHI.Pins (4.3.6.0)  Attaching deployed file.

   Assembly: Microsoft.SPOT.Net (4.3.1.0)  Attaching deployed file.

   Assembly: Microsoft.SPOT.Net.Security (4.3.1.0)  Attaching deployed file.

   Assembly: GHI.Networking (4.3.6.0)  Resolving.

The debugging target runtime is loading the application assemblies and starting execution.
Ready.

'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\mscorlib.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Native.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Hardware.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Graphics.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.TinyCore.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.IO.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\System.IO.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Hardware.Usb.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Hardware.SerialPort.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Hardware.PWM.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\GHI Electronics\GHI NETMF v4.3 SDK\Libraries\le\GHI.Hardware.dll'
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Net.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\System.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\Microsoft.SPOT.Net.Security.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\System.Net.Security.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.3\Assemblies\le\System.Http.dll', Symbols loaded.
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\GHI Electronics\GHI NETMF v4.3 SDK\Libraries\le\GHI.Pins.dll'
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Program Files (x86)\GHI Electronics\GHI NETMF v4.3 SDK\Libraries\le\GHI.Networking.dll'
'Microsoft.SPOT.Debugger.CorDebug.12.dll' (Managed): Loaded 'C:\Data\Cerbuino\GatewayServer\GatewayServer\bin\Debug\le\GatewayServer.exe', Symbols loaded.
The thread '<No Name>' (0x2) has exited with code 0 (0x0).
Hello World!
Context counter: 1
Request handler counter: 1
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
Context counter: 2
Request handler counter: 2
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
The thread '<No Name>' (0x6) has exited with code 0 (0x0).
The thread '<No Name>' (0x7) has exited with code 0 (0x0).
Context counter: 3
Request handler counter: 3
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
Context counter: 4
Request handler counter: 4
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
Context counter: 5
Request handler counter: 5
The thread '<No Name>' (0x8) has exited with code 0 (0x0).
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in Microsoft.SPOT.Net.dll
The thread '<No Name>' (0x9) has exited with code 0 (0x0).
Context counter: 6
Request handler counter: 6
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
Context counter: 7
Request handler counter: 7
The thread '<No Name>' (0xb) has exited with code 0 (0x0).
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
Context counter: 8
Request handler counter: 8
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in Microsoft.SPOT.Net.dll
The thread '<No Name>' (0xc) has exited with code 0 (0x0).
Context counter: 9
Request handler counter: 9
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
The thread '<No Name>' (0xd) has exited with code 0 (0x0).
The thread '<No Name>' (0xe) has exited with code 0 (0x0).
Context counter: 10
Request handler counter: 10
A first chance exception of type 'System.Net.Sockets.SocketException' occurred in Microsoft.SPOT.Net.dll
Context counter: 11
Request handler counter: 11
The thread '<No Name>' (0xf) has exited with code 0 (0x0).
A first chance exception of type 'System.NullReferenceException' occurred in System.Http.dll
The thread '<No Name>' (0x10) has exited with code 0 (0x0).
An unhandled exception of type 'System.NullReferenceException' occurred in System.Http.dll


This is my code:

using System;
using Microsoft.SPOT;
using System.Threading;
using System.Net;

using GHI.Networking;
using GHI.Utilities;
using Microsoft.SPOT.Hardware;
using Microsoft.SPOT.Net.NetworkInformation;

using System.Collections;
using System.Net.Sockets;

namespace GatewayServer
{
	public class Program
	{
		static Queue m_responseQueue = new Queue();

		public static void Main()
		{
			Debug.Print(Resources.GetString(Resources.StringResources.String1));
			// make sure the date is set on the device
			if (Microsoft.SPOT.Hardware.SystemInfo.SystemID.SKU != 3)
			{
				Microsoft.SPOT.Hardware.Utility.SetLocalTime(new DateTime(2012, 8, 24));
			}

			EthernetENC28J60 netif = new EthernetENC28J60(SPI.SPI_module.SPI1, GHI.Pins.Generic.GetPin('A', 13), GHI.Pins.Generic.GetPin('A', 14), GHI.Pins.Generic.GetPin('B', 10));
			netif.Open();
			netif.EnableDhcp();
			netif.EnableDynamicDns();

			while (netif.IPAddress == "0.0.0.0")
			{
				Debug.Print("Waiting for DHCP");
				Thread.Sleep(250);
			}

			Thread serverThread = new Thread(StartServer);
			serverThread.Start();
			Thread.Sleep(Timeout.Infinite);
		}

		static void StartServer()
		{
			HttpListener listener = new HttpListener("http", -1);
			int contextCounter = 0;
			while (true)
			{
				try
				{
					if (!listener.IsListening)
					{
						listener.Start();
					}

					HttpListenerContext context = listener.GetContext();
					lock (m_responseQueue)
					{
						m_responseQueue.Enqueue(context);
					}
					contextCounter++;
					Debug.Print("Context counter: " + contextCounter.ToString());
					Thread th = new Thread(new ThreadStart(HandleRequestThread));
					th.Start();
				}
				catch (InvalidOperationException)
				{
					listener.Stop();
					Thread.Sleep(200);
				}
				catch (ObjectDisposedException)
				{
					listener.Start();
				}
				catch
				{
					Thread.Sleep(200);
				}
			}
		}

		private static int requestHandlerCounter = 0;
		private static void HandleRequestThread()
		{
			HttpListenerContext context = null;
			requestHandlerCounter++;
			Debug.Print("Request handler counter: " + requestHandlerCounter.ToString());
			try
			{
				lock (m_responseQueue)
				{
					context = (HttpListenerContext)m_responseQueue.Dequeue();
				}

				if (context != null)
				{
					HttpListenerRequest request = context.Request;
					switch (request.HttpMethod.ToUpper())
					{
						case "GET": ProcessClientGetRequest(context); break;
						//case "POST": ProcessClientPostRequest(context); break;
					}
				}
			}
			catch (SocketException)
			{
			}
			finally
			{
				if (context != null)
				{
					context.Close();
				}
			}
		}

		private static void ProcessClientGetRequest(HttpListenerContext context)
		{

			HttpListenerRequest request = context.Request;
			HttpListenerResponse response = context.Response;

			response.StatusCode = (int)HttpStatusCode.OK;
			// Find if file is present. If file exists - sends its context

			// Start HTML document
			string strResp = "<HTML><BODY>.Net Micro Framework Example HTTP Server<p>";

			// Print requested verb, URL and version.. Adds information from the request.
			strResp += "HTTP Method: " + request.HttpMethod + "<br> Requested URL: \"" + request.RawUrl +
				"<br> HTTP Version: " + request.ProtocolVersion + "\"<p>";

			// Information about the path that we access.
			strResp += "Raw URL: " + request.RawUrl + "<p>";

			// Closes HTML
			strResp += "</BODY></HTML>";

			// Sends it.
			byte[] messageBody = System.Text.Encoding.UTF8.GetBytes(strResp);
			response.ContentType = "text/html";
			response.OutputStream.Write(messageBody, 0, messageBody.Length);
			response.Close();
			context.Close();
		}
	}
}

Well, without digging into the rest of the problem, I can tell you that the two requests are probably because your browser silently tries to also retrieve the site’s icon whenever you first visit a site and no icon is cached locally. You can confirm this by running fiddler.

@ mcalsyn
Didn’t think of fiddler, I’m used to sockets, not to http, I will try it out. Thanks for the tip.

@ andre.m
I forgot to mention that the program seems to work correctly under some conditions. The first time always succeeds and at first sight it seems that the program keeps working if I refresh the page slowly, for example every five seconds. I didn’t do any endurance testing though. (I have some memory in the back of my head about a limited amount of sockets, I believe 128, and a linger time of 2 minutes and I don’t know how this would work out with an endurance test)

I am aware of those points but:

Sure, but I’m not the one using sockets, the httplistener is, shouldn’t the httplistener handle those issues, and if not, that class is completely useless

[quote]netmf always runs managed (makes things slow, not good for webthings)
debug.print makes things also slow[/quote]
Slow should not equal crash, I’m having an unhandled exception

I only added those to debug, the same issue exists without them. And again, slower ok, but crash not ok.

could this be the well known problem with the web server? I don’t believe it has been fixed in the latest SDK.

Do a search for the a null problem with the web server. you will find a solution.

I have been searching all over the place and have found lots of issues but nothing seems to be related. If this is a ‘famous’ problem it seems I’m searching with the wrong parameters, could you point me somewhere?

Potentially this : https://netmf.codeplex.com/workitem/2158
See also : https://www.ghielectronics.com/community/forum/topic?id=17222

You may need to apply the fix as described in this reply to that forum thread : https://www.ghielectronics.com/community/forum/topic?id=17222&page=1#msg171641

The bug is listed as fixed in 4.4.

In my test program I found that listener.GetContext() returns null sometimes when I reloaded the page a lot. It returns also null when the network cable is unplugged.

Exactly, which is why you need to apply the indicated fix.

I will give that a try. I my case the null pointer exception was caused when I did
HttpListenerRequest request = context.Request;
and not before.

How can we apply that fix? I’ve never modified the sourcecode.

The source code is here:

Is it possible just to add the two affected classes to the project or is it necessary to add the whole package?

Shouldn’t we use the code at codeplex? I believe the GHI SDK still uses 4.3?

You will need to build System.Http in its entirety. I can’t give much more guidance, because, although I have built the entire NETMF tree, I have never built just System.Http 4.4 and tried to use it with the 4.3 sdk, so I don’t know if there are other incompatibilities in there that will prevent the 4.4 source from working in 4.3. If there are, you will have to get the 4.3 source and make the fix there and then build the 4.3 version of the dll.

@ Jef Patat - You have two options : a) Try the 4.4 source, which has the fix and may just work, or it may fail due to other 4.4 changes or b) go to codeplex, make the fix to that code, and then build 4.3 (that’s probably the safer option).

Has anyone tries to add the 4.3 System.http package to the project instead of compiling the whole framework?

It would be interesting if that Codeplex download link actually worked, it times out on me everytime, is there another way of getting the code?

“git checkout c0d38eaddba9dfe80a82154e4f54abe1d827330b” will get you the initial github checkin, which theoretically will match the last codeplex checkin. Otherwise, try, try again, which is what I had to do to get my codeplex copy.

Going through bitbucket seems to be a better way. I also get the timeout. Seems to need a lot of retries…

Now I added the System.http package to my project, applied the fixes and now it is working fine.

I have seen that it is recommended to do:

output.Close();
response.Close();

When I do that, I get some exceptions and it looks like the close is done twice.