String handling Error?

Two Issues:
I am getting a string response from a weather web service. String length is about 471 characters.

Issue 1: On many occasions, my parsing fails because the whole string isn’t there. String.Length reports the correct length but Debug.WriteLine() doesn’t output the whole thing. I get an out of range exception. It’s like the string handling is buggy. You can see in the debug window that the string.length outputs are showing a full string. It fails because it can’t find the “feels_like” tag.

Why does string.length report a length of 470 but its clearly not that long? Or is? I can’t tell.

. . .Web Request code . . .        

using (Stream dataStream = response.GetResponseStream())
        {
            //create new buffer then length of the response plus a little
            byte[] buffer = Encoding.UTF8.GetBytes(new string(' ', (int)response.ContentLength + 10));

            dataStream.Read(buffer, 0, (int)response.ContentLength);

            System.Diagnostics.Debug.WriteLine("buffer length: " + buffer.Length.ToString());
            reply = UTF8Encoding.UTF8.GetString(buffer, 0, buffer.Length);
            System.Diagnostics.Debug.WriteLine("reply length: " + reply.Length.ToString());
            System.Diagnostics.Debug.WriteLine("reply: " + reply);
                                           
        }
        if(reply.Length > 0)
        {
            try
            {   //find the temperature
                System.Diagnostics.Debug.WriteLine("reply length: " + reply.Length.ToString());
                int start = reply.IndexOf("\"temp\":");
                int end = reply.IndexOf("\"feels_like\"");
                
                string str = reply.Substring(start, end - start );
                string[] elements = str.Split(':');
                string temp = elements[1].TrimEnd(',');

                //find the Description information
                System.Diagnostics.Debug.WriteLine("reply length: " + reply.Length.ToString());
                start = reply.IndexOf("\"description\":");
                end = reply.IndexOf("\"icon\":");
                System.Diagnostics.Debug.WriteLine($"Description pos: {start}, icon pos: {end}");
                str = reply.Substring(start, end - start);
                elements = str.Split(':');
                string desc = elements[1];
                desc = desc.Trim('"');
                desc = desc.TrimEnd(',', '"');

                string message = $"Temperature: {temp}F\nConditions: {desc}";

                TinyCLRApplication1.Program.UpdateText(message, true);

            }
            catch(Exception ex)
            {
                System.Diagnostics.Debug.WriteLine("deserialize: " + ex.Message);
            }
        }

Failure Debug Window Output:

buffer length: 470
reply length: 470
reply: {“coord”:{“lon”:-88.99,“lat”:42.42},“weather”:[{“id”:804,“main”:“Clouds”,“description”:“overcast clouds”,“icon”:“04n”}],“base”:“stations”,“main”:{“temp”:29.32,"
reply length: 470
Temp pos: 146, feels_like pos: -1
#### Exception System.ArgumentOutOfRangeException - CLR_E_OUT_OF_RANGE (4) ####
#### Message:
#### System.String::Substring [IP: 0000] ####
#### TinyCLRApplication1.NetworkWorker::StartWeather [IP: 0101] ####
#### TinyCLRApplication1.NetworkWorker::start [IP: 02ac] ####
Exception thrown: ‘System.ArgumentOutOfRangeException’ in mscorlib.dll

When it works:

buffer length: 470
reply length: 470
reply: {“coord”:{“lon”:-88.99,“lat”:42.42},“weather”:[{“id”:804,“main”:“Clouds”,“description”:“overcast clouds”,“icon”:“04n”}],“base”:“stations”,“main”:{“temp”:29.32,“feels_like”:22.26,“temp_min”:28,“temp_max”:30.99,“pressure”:1021,“humidity”:86},“visibility”:11265,“wind”:{“speed”:4.7,“deg”:190},“clouds”:{“all”:90},“dt”:1580436264,“sys”:{“type”:1,“id”:3947,“country”:“US”,“sunrise”:1580389929,“sunset”:1580425563},“timezone”:-21600,“id”:0,“name”:“Roscoe”,“cod”:200}
reply length: 470
Temp pos: 146, feels_like pos: 159
reply length: 470
Description pos: 73, icon pos: 105

Issue 2:
When debugging is there a string length limit for the locals window? It seems to truncate all strings to a max of 127 characters. It not an issue with Debug.WriteLine(). It’s just when code is stopped and looking at the variables in the local window.

Can you somehow give us the string to test it as example code that shows the issue, without internet?

I can send project via email if that helps.

1 Like

Can we have something we can test without relying on the internet?

I think I have narrowed it down to the dataStream.Read() function.

My first problem was in:
byte[] buffer = Encoding.UTF8.GetBytes(new string(' ', (int)response.ContentLength + 10));

It creates a buffer filled with spaces. So it will always return a length of response.ContentLength + 10.

I replaced:
dataStream.Read(buffer, 0, (int)response.ContentLength);

with:

int count = 0;
for (int i = 0; i < response.ContentLength; i++)
{
   byte b = (byte)dataStream.ReadByte();
   //System.Diagnostics.Debug.WriteLine($"{i} - [{b.ToString("X2")}],");
   buffer[i] = b;
   count++;
}

This so far hasn’t failed.

I am not sure how to get something for you to test since it is somewhere in the Stream returned from the HttpWebResponse.GetResponseStream() and the dataStream.Read(). At least that is what it appears to be.

I also found during this test that HttpWebRequest.HaveResponse Property never returns True. I believe it is supposed to return True when a response has been received.

Please email your project to Greg. @Dat_Tran does this make sense?