Discovery - GHIElectronics-NETMF-Net-HttpWebResponse is very particular about response timing

I have not solved this, but I believe I am much closer to the problem.

Basic problem: Using a Panda II and FEZ Ethernet board, I am unable to do HTTP GET requests to my own REST service. The problem is an exception:

#### Exception System.NullReferenceException - CLR_E_NULL_REFERENCE (4) ####
#### Message: 
#### GHIElectronics.NETMF.Net.HttpWebResponse::GetResponseHeader [IP: 0008] ####
#### GHIElectronics.NETMF.Net.HttpWebResponse::get_ContentType [IP: 0007] ####

I am using this code:
http://code.tinyclr.com/project/288/http-client-sample-code-for-fez-connect/

Which is simple. I have done code just like this on the Netduino Plus, against my same web service, and everything works fine.

When I use the GHIElectronics.NETMF.Net.HttpWebResponse against Microsoft “stock” web services, it also works fine. IOW: I have this code working.

However, there is something about my web service response that causes GHIElectronics.NETMF.Net.HttpWebResponse to blow up, but that does NOT cause the NETMF System.Net.HttpWebRequest to blow up. (Because on Netduino using System.Net.HttpWebRequest my stuff works fine.)

Since I do not have sources to GHIElectronics.NETMF.Net, I cannot debug into this (and, frankly, it is probably beyond me, as I am not an experienced coder).

I have carefully compared my web service response vs other responses. I have tried setting the content-type differently, etc. My web service returns simple text, not html.

There are clearly weaknesses in GHIElectronics.NETMF.Net.HttpWebResponse

Let me know how I can help.

Thank you!

Sam, if there is some code you can post that GHI can run against your web service to use to help debug the problem that will help.

While brushing my teeth last night, I put my finger on it.

I had two requests from the Panda. One was working fine, and one request blew up (e.g. the http client code in the panda blows up, this is the client:
http://code.tinyclr.com/project/288/http-client-sample-code-for-fez-connect/ )

I had first focused on headers, content-length, and actual content. (I have a lot of experience with this stuff, outside the microcontroller world). But I got the working response to mirror the problem response in ALL those ways… and still one worked, and one did not.

(I used fiddler to carefully compare the raw responses from the working and non working requests)

Brushing my teeth, I realized there is still a delta: Response time. The working responses were in the 100-250 msec response time range, but the nonworking responses were 300+ msec, usually 500msec to 2000 msec.

(This is material, because the panda is calling a web service that needs time to get its data and do some work… So my “good” calls from panda [that fetch data] fail, and my “bad” calls from panda [that result in a quick error message from the REST server] work.)

So this AM I worked on it (I normally only work on this in evening, as this is a hobby project, not my day job). And YES, response time from the server is THE issue.

The server Panda is hitting is a very simple, .NET REST server based on WCF, .NET v4.

Here is the way to see the issue:

>>>>>>>>>>>>>>>>>>>>
            // This is the block in the REST server that returns data to the client...
            if (ValidStringMask & Validparameter1Param & Validparameter2Param & Validparameter3Param)
            {
                // Params are valid, so Go Get Some Data!!
				
				// FAILS from Panda http client, because takes 500-2000 msec
                return 
                    new MemoryStream(Encoding.UTF8.GetBytes(myfuncs.BuildStringFromList(MyDataList)));
            }
            else
            {
                // GHI-Itis is this next line!
				// Remove it, and the http client on panda works fine, gets payload.
				// Put it in... and the http client on panda fails to handle response....
				System.Threading.Thread.Sleep(300);

                return new MemoryStream(Encoding.ASCII.GetBytes("Invalid parameters provided"));
				
            }
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

The main body of the rest server is this, includes the above:

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;
using System.IO;
using myfuncs = MyNewFuncs.myfunctionlib;
using MyNewFuncs;   // this seems redundant to above, because namespace is not well organized...

namespace REST1
{
    // NOTE: You can use the "Rename" command on the "Refactor" menu to change the class name "Service1" in code, svc and config file together.
    public class Service1 : IService1
    {
        public Stream GetDataFromWebSite(string returnstringmask, string parameter1, string parameter2, string parameter3)
        {

            // test that we received useful input params
            bool Validparameter1Param    = false;
            bool Validparameter2Param    = false;
            bool Validparameter3Param    = false;
            bool ValidStringMask         = false;
            Int16 TargetInt              = 0;

            if (returnstringmask.Length == 8) 
            {
                ValidStringMask = true;
            }
            
            Validparameter1Param = (Int16.TryParse(parameter1, out TargetInt));
            Validparameter2Param    = (Int16.TryParse(parameter2, out TargetInt));
            Validparameter3Param         = (Int16.TryParse(parameter3, out TargetInt));

			// Attempting to get the GHI http lib to work... this was a dead end, but is the right thing to do...
            WebOperationContext.Current.OutgoingResponse.ContentType = "text/html; charset=utf-8";

            if (ValidStringMask & Validparameter1Param & Validparameter2Param & Validparameter3Param)
            {
                // Params are valid, so Go Get Some Data!!
				
				// FAILS from Panda http client, because takes 500-2000 msec
                return 
                    new MemoryStream(Encoding.UTF8.GetBytes(myfuncs.BuildStringFromList(MyDataList)));
            }
            else
            {
                // GHI-Itis is this next line!
				// Remove it, and the http client on panda works fine, gets payload.
				// Put it in... and the http client on panda fails to handle response....
				System.Threading.Thread.Sleep(300);

                return new MemoryStream(Encoding.ASCII.GetBytes("Invalid parameters provided"));
				
            }

        }


    }

}
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<


Gus, to your question:

Sam, if there is some code you can post that GHI
can run against your web service to use to help debug the problem that will help.

My web service is not publicly accessible (yet), so I can’t get you there.

The code on the panda is this code:
http://code.tinyclr.com/project/288/http-client-sample-code-for-fez-connect/

With just a couple of mods:
– static IP on panda
– no dns
– hitting my url,

That is the only delta

You should be able to provide a simple example that runs with a public website that demonstrate the issue. This is the fastest way for us to to help.

I provided a simple example that demonstrates this exact behavior in: http://www.tinyclr.com/forum/2/3793/#/1/msg42161

The behavior of the WIZnet 5100 is not quite right. It appears to be very sensitive to the web site’s response timing. If the web site has not provided a response when the WIZnet 5100 is ready for it, you get the exception.

Looking at data sheet for the 5100, I would guess that timeout is not handled properly and when the code goes to read data, determines that there is no data and instead of waiting for the data to show up, just generates the exception. The same program in the emulator and with a CANxtra works correctly, but neither of these use WIZnet 5100 library.

If you know the maximum time it takes for the server to respond, you can add a sleep for that amount of time (or more) and then things work correctly. So, there is a work around, but it is not the most robust and does add some delay.

Gus,

I completely understand your perspective. In my day job, I support developers who consume code that we publish, and they report issues, and I ask them the same thing you are asking me.

If I had any idea how to provide what you ask for, I would, right away.

But I don’t. All my development is on a private network. I own both sides of the equation (the server code, and the panda code). You can reproduce the problem, creating a WCF web service, quite easily. (I made my WCF rest web service in one or two hours, with zero experience in the area, and only a lifetime total of 15 hours using visual studio… I just followed the MS tutorial video).

You have two GHI customers on this thread saying clearly:

  • There seems to be a problem in the GHI http implementation
  • We can reproduce it %100 consistently

Further, I can tell you that in NETMF on Netduino, (which obviously does not use the GHI http stack), that this problem does NOT exist.

I had my http GET running on Netduino directly from examples, in an hour of work.

On Panda, I have 10-20 hours into it so far, and now have to rewrite it on sockets, because the GHI http is too problematic.

How you choose to handle this is %100 your call.

Let me know how I can help.

Would making the http libraries open source help?

Having access to the source always helps :slight_smile:

Would making the http libraries open source help?

I don’t know if it would help me directly. I am not an experienced coder. My guesstimate is that tracing a problem like this would take more time and skill than I possess.

However, it may allow the community to track down the issue, and suggest a resolution.

I will pass this on to our developers to come up with a good solution.

The source code is open now:
[url]http://www.tinyclr.com/forum/2/5565/#/1/msg52735[/url]

The source code is open now:

Wow, nice!

Way beyond my ability to help with (sorry to say).