SSL Issues

NETMF & Gadgeteer
1

I’m trying to access a url over an SSL connection. Here’s my code:


                HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
                request.HttpsAuthentCerts = caCerts; 
                request.KeepAlive = true;
                byteData = UTF8Encoding.UTF8.GetBytes(requestText);
                request.Method = "POST";
                request.Timeout = 30000;
                request.ContentLength = byteData.Length;
                request.ContentType = "application/x-www-form-urlencoded";
                Stream reqStream = request.GetRequestStream();

I’m getting the following error:
#### Exception System.NotSupportedException - CLR_E_NOT_SUPPORTED (3) ####
#### Message:
#### Microsoft.SPOT.Net.Security.SslNative::SecureClientInit [IP: 0000] ####
#### Microsoft.SPOT.Net.Security.SslStream::Authenticate [IP: 0051] ####
#### Microsoft.SPOT.Net.Security.SslStream::AuthenticateAsClient [IP: 000c] ####
#### System.Net.HttpWebRequest::EstablishConnection [IP: 0246] ####
#### System.Net.HttpWebRequest::SubmitRequest [IP: 0013] ####
#### System.Net.HttpWebRequest::GetRequestStream [IP: 0008] ####
#### HemController.Program::GetHttpData [IP: 00fd] ####
#### HemController.Program::TalkToServer [IP: 0165] ####

I know the release notes say there is support for SSL, so what am I doing wrong?

2

Did you update the encryption seed using MFDeploy?

3

Thanks, Gus. That was the issue. I must have missed that in the docs.

Now I’m getting this error:
#### Exception System.Net.Sockets.SocketException - CLR_E_FAIL (3) ####
#### Message:
#### Microsoft.SPOT.Net.Security.SslNative::SecureConnect [IP: 0000] ####
#### Microsoft.SPOT.Net.Security.SslStream::Authenticate [IP: 0060] ####
#### Microsoft.SPOT.Net.Security.SslStream::AuthenticateAsClient [IP: 000c] ####
#### System.Net.HttpWebRequest::EstablishConnection [IP: 0246] ####
#### System.Net.HttpWebRequest::SubmitRequest [IP: 0013] ####
#### System.Net.HttpWebRequest::GetRequestStream [IP: 0008] ####
#### HemController.Program::GetHttpData [IP: 00fd] ####
#### HemController.Program::TalkToServer [IP: 0165] ####
#### SocketException ErrorCode = -2

I looked up the error, and it’s a handshake failure. Any clues why I would get this error? The certs are valid.

All the sample code I’ve found uses an SslStream, which doesn’t seem to be accessible to me in the Microsoft.SPOT.Net.Security.dll. I looked in the Object Browser, and it’s not there. I’m still digging around trying to find out why.

I feel like I’ve been spinning my wheels on this far longer than I should be, so I apologize if it seems like I’m trying to get everyone else to do the hard work for me. If anyone has sample code that they’d be willing to share showing how they handled an ssl connection, I’d be very grateful.

4

There is a sample in NETMF SDK with ssl and http, try that one.

5

I ran the sample in the NETMF SDK. It fails with the same error IF I pass in the cert to the HttpWebRequest object (HttpsAuthentCerts property). I commented out the 2nd line of code in my project:

request.HttpsAuthentCerts = caCerts;

and the code doesn’t throw the error.

What does this mean for the connection? I’m assuming it’s still a secure socket since that page should have redirected if it wasn’t secure. Does it mean the server certificate isn’t validated since there is no CA certificate against which it is validating?

6

You have secure SSL communications but the server is not getting authenticated!
This could a problem with some NETMF libraries. We are looking into this.

7

We have received a confirmation from Microsoft of the existence of the issue and now waiting on a fix. SSL is extremely complicated and tricky so this can take a bit of time.

8

Any update on this?

9

The situation is really complex and Microsoft is working on it. It can take some time.

10

Has this been resolved yet?

11

Unfortunately, not yet. See release notes http://www.tinyclr.com/release-notes/

12

From the release notes :

And I realize this is not a GHI specific issue, but does anyone know if it is possible to disable certificate verification ?

13

I believe you just set HttpsAuthentCerts to null and it should work.

14

Setting HttpsAuthentCerts to NULL does work.

it looks when you are performing an HTTPS request with HttpWebRequest you can only do one connection at time. If I spin up another HttpWebRequest to a HTTP or HTTPS URL it starts throwing exceptions like :

For a HTTP request ( no SSL ) :

 #### Exception System.Net.Sockets.SocketException - CLR_E_FAIL (5) ####
#### Message: 
#### Microsoft.SPOT.Net.SocketNative::recv [IP: 0000] ####
#### System.Net.Sockets.Socket::Receive [IP: 0018] ####
#### System.Net.Sockets.NetworkStream::Read [IP: 0062] ####
#### System.Net.InputNetworkStreamWrapper::ReadInternal [IP: 00d7] ####
#### System.Net.InputNetworkStreamWrapper::Read [IP: 000d] ####
#### SocketException ErrorCode = 10060
#### SocketException ErrorCode = 10060

A first chance exception of type ‘System.Net.Sockets.SocketException’ occurred in Microsoft.SPOT.Net.dll
#### SocketException ErrorCode = 10060
#### SocketException ErrorCode = 10060

and for an HTTPS request posting a file to the server :

#### Exception System.Net.Sockets.SocketException - CLR_E_FAIL (13) ####
#### Message: 
#### Microsoft.SPOT.Net.Security.SslNative::SecureWrite [IP: 0000] ####
#### Microsoft.SPOT.Net.Security.SslStream::Write [IP: 0040] ####
#### System.Net.InputNetworkStreamWrapper::Write [IP: 000a] ####
#### SocketException ErrorCode = -1
#### SocketException ErrorCode = -1

A first chance exception of type ‘System.Net.Sockets.SocketException’ occurred in Microsoft.SPOT.Net.Security.dll
#### SocketException ErrorCode = -1
#### SocketException ErrorCode = -1

Any further requests results in exceptions like the above, it looks like something has gone wrong in Microsoft.SPOT.NET. The device still responds to pings. I haven’t tried opening any raw tcp sockets while the device is in this condition but anything through HttpWebRequest fails.

15

Also if you write more than 1,427 bytes at a time to the request stream of an SSL request you will get an exception. Chunking data into 1,427 byte pieces works just fine though.

Also using SSL I see more exceptions like this when doing IO to the SD card:

Exception System.IO.IOException - CLR_E_INVALID_DRIVER (7)

#### Message: 
#### Microsoft.SPOT.IO.NativeIO::GetAttributes [IP: 0000] ####
#### System.IO.File::Delete [IP: 0014] ####

After remounting the SD card everything appears to work fine until the next exception.

In this case it was a delete operation but it can happen with file i/o. It doesn’t always happen on the next file i/o after an SSL http request but usually very soon after ( 2 or 3 file operations later ).

Could there be some sort of weird interplay going on under the covers in the Streams?

I’m working with multiple threads accessing the file system but only one thread a time.

16

This seems like a coincidence. The SD card is not related to other parts of the system. Maybe it wasn’t mounted or just the SD card not working. Maybe not enough power…
As for more than one connection, please make small complete example that shows the error. Don’t use multi threads or SD card…etc This will help eliminate issues and find the actual problem. In the example don’t use SSL if possible. This eliminates another layer of complications when narrowing the problem.

17

The SD card was indeed unrelated.

Here is some test code I wrote to demonstrate the issue.


using System;
using Microsoft.SPOT;
using System.Net;
using System.Collections;
using Sjtechs.NETMF;
using System.IO;
using System.Threading;

namespace DoorSnap
{
    class HttpTest
    {

        public void Run()
        {

            byte[] fileBytes1 = new byte[5 * 1024];
            byte[] fileBytes2 = new byte[5 * 1024];

            KeyValueStore alphaStore = new KeyValueStore();
            alphaStore.Set("DeviceId", "HttpTest1");

            HttpWebRequest alphaRequest = BuildServerRequest("https://10.100.1.42/upload.php", ref fileBytes1, "CameraImage", "image/jpeg", "http1.jpg", alphaStore, "test", "test");
            HttpWebResponse alphaResponse = null;
            Stream alphaResponseStream = null;
            StreamReader alphaResponseStreamReader = null;


            KeyValueStore bravoStore = new KeyValueStore();
            bravoStore.Set("DeviceId", "HttpTest2");


            HttpWebRequest bravoRequest = BuildServerRequest("https://10.100.1.42/upload.php", ref fileBytes2, "CameraImage", "image/jpeg", "http2.jpg", bravoStore, "test", "test");
            HttpWebResponse bravoResponse = null;
            Stream bravoResponseStream = null;
            StreamReader bravoResponseStreamReader = null;

            alphaResponse = (HttpWebResponse)alphaRequest.GetResponse();
            bravoResponse = (HttpWebResponse)bravoRequest.GetResponse();

            alphaResponseStream = alphaResponse.GetResponseStream();
            bravoResponseStream = bravoResponse.GetResponseStream();

            alphaResponseStreamReader = new StreamReader(alphaResponseStream);
            Debug.Print("ALPHA RESPONSE : " + alphaResponseStreamReader.ReadToEnd());

            bravoResponseStreamReader = new StreamReader(bravoResponseStream);
            Debug.Print("BRAVO RESPOSNE : " + bravoResponseStreamReader.ReadToEnd());
                        
            
        }

        public void RunThreads()
        {
            Thread thread1 = new Thread(HttpWorker1);
            Thread thread2 = new Thread(HttpWorker2);

            thread1.Start();
            thread2.Start();

            thread1.Join();
            thread2.Join();

        }


        private void HttpWorker1()
        {
            byte[] fileBytes1 = new byte[5 * 1024];

            KeyValueStore alphaStore = new KeyValueStore();
            alphaStore.Set("DeviceId", "HttpTest1");

            HttpWebRequest alphaRequest = BuildServerRequest("https://10.100.1.42/upload.php", ref fileBytes1, "CameraImage", "image/jpeg", "http1.jpg", alphaStore, "test", "test");
            HttpWebResponse alphaResponse = null;
            Stream alphaResponseStream = null;
            StreamReader alphaResponseStreamReader = null;


            alphaResponse = (HttpWebResponse)alphaRequest.GetResponse();

            alphaResponseStream = alphaResponse.GetResponseStream();

            alphaResponseStreamReader = new StreamReader(alphaResponseStream);
            Debug.Print("ALPHA RESPONSE : " + alphaResponseStreamReader.ReadToEnd());

        }


        private void HttpWorker2()
        {
            byte[] fileBytes2 = new byte[5 * 1024];

            KeyValueStore bravoStore = new KeyValueStore();
            bravoStore.Set("DeviceId", "HttpTest2");

            HttpWebRequest bravoRequest = BuildServerRequest("https://10.100.1.42/upload.php", ref fileBytes2, "CameraImage", "image/jpeg", "http2.jpg", bravoStore, "test", "test");
            HttpWebResponse bravoResponse = null;
            Stream bravoResponseStream = null;
            StreamReader bravoResponseStreamReader = null;

            bravoResponse = (HttpWebResponse)bravoRequest.GetResponse();

            bravoResponseStream = bravoResponse.GetResponseStream();

            bravoResponseStreamReader = new StreamReader(bravoResponseStream);
            Debug.Print("BRAVO RESPOSNE : " + bravoResponseStreamReader.ReadToEnd());
        }


        private HttpWebRequest BuildServerRequest(string url, ref byte[] fileBytes, string fileParamName, string fileContentType, string fileName
                                             , KeyValueStore formValues, string username, string password)
        {



            string boundary = "---------------------------" + DateTime.Now.Ticks.ToString();
            byte[] boundarybytes = System.Text.Encoding.UTF8.GetBytes("\r\n--" + boundary + "\r\n");

            string fileHeader = "";
            byte[] fileHeaderBytes = null;
            byte[] fileTrailer = null;

            ArrayList formItems = new ArrayList();
            int requestLen = 0;

            foreach (var kv in formValues.GetAll())
            {
                string itemString = "Content-Disposition: form-data; name=\"" + kv.Key.ToString() + "\"\r\n\r\n" + kv.Value.ToString() + "\r\n";
                var itemBytes = System.Text.Encoding.UTF8.GetBytes(itemString);

                requestLen += itemBytes.Length;
                requestLen += boundarybytes.Length;

                formItems.Add(itemBytes);
            }


            if (fileBytes != null && fileBytes.Length > 0)
            {
                fileHeader = "Content-Disposition: form-data; name=\"" + fileParamName + "\"; filename=\"" + fileName + "\"\r\nContent-Type: " + fileContentType + "\r\n\r\n";
                fileHeaderBytes = System.Text.Encoding.UTF8.GetBytes(fileHeader);
                fileTrailer = System.Text.Encoding.UTF8.GetBytes("\r\n--" + boundary + "--\r\n");

                requestLen += boundarybytes.Length + fileHeaderBytes.Length + fileTrailer.Length + fileBytes.Length;
            }

            HttpWebRequest request = null;
            Stream requestStream = null;

            request = HttpWebRequest.Create(url) as HttpWebRequest;
            request.HttpsAuthentCerts = null;
            request.ContentType = "multipart/form-data; boundary=" + boundary;
            request.Method = "POST";
            // request.KeepAlive = true;
            request.ContentLength = requestLen;
            request.Credentials = new NetworkCredential(username, password);
            request.Expect = string.Empty;
            request.Timeout = 1000 * 5;
            request.ReadWriteTimeout = 1000 * 5;


            try
            {
                requestStream = request.GetRequestStream();

                foreach (var item in formItems)
                {
                    requestStream.Write(boundarybytes, 0, boundarybytes.Length);
                    var bytes = item as byte[];
                    requestStream.Write(bytes, 0, bytes.Length);
                }

                if (fileBytes != null && fileBytes.Length > 0)
                {
                    requestStream.Write(boundarybytes, 0, boundarybytes.Length);
                    requestStream.Write(fileHeaderBytes, 0, fileHeaderBytes.Length);
                    int chunkSize = 1427; // any larger will cause an SSL request to fail

                    for (int i = 0; i < fileBytes.Length; i += chunkSize)
                    {
                        var toWrite = chunkSize;
                        if (i + 1 + chunkSize > fileBytes.Length)
                            toWrite = fileBytes.Length - i;
                        requestStream.Write(fileBytes, i, toWrite);
                    }

                    requestStream.Write(fileTrailer, 0, fileTrailer.Length);
                }

                requestStream.Close();
                requestStream.Dispose();
                requestStream = null;

            }
            catch (Exception ex)
            {
                if (request != null)
                    request.Dispose();

                request = null;
            }
            finally
            {
                if (requestStream != null)
                    requestStream.Dispose();
            }

            return request;
        }

    }
}

In this code, if I call the Run() method everything works fine using http or https.

However, if I call the RunThreads() method is called when using https it fails, works fine with http.

Trying to have multiple threads making SSL connections at the same time results in exceptions like below. I am working around this by synchronizing my threads so only one ssl request is made at a time.