I’d like to write sensor data from a Spider Mainboard to Azure Table Storage with https.
Anybody here who has code examples?
Kind regards
RoSchmi
@ RoSchmi - Does this help a bit ?
[url]Microsoft Learn: Build skills that open doors in your career
1 Like
@ .Peter. - Thanks, but I found this tutorial too. As I’m not sure if the kind of authentication and encryption is the same for blobs and tables, I tried to to translate the code from this tutorial:
[url]https://www.simple-talk.com/cloud/cloud-data/an-introduction-to-windows-azure-table-storage/[/url]
for PC (.NET 4.x) (worked nice on PC) back to NETMF. This turned out to be much work and I’m not sure if I’ll succeed.
Would be nice if anyone had a ready solution.
Take a look here for code that can read/write Azure blobs and tables : [url]https://github.com/PervasiveDigital/serialwifi/tree/master/src/Common/PervasiveDigital.Net.Azure.Storage.Shared[/url]
It uses the security routines here : [url]https://github.com/PervasiveDigital/serialwifi/tree/master/src/Common/PervasiveDigital.Security.ManagedProviders.Shared[/url]
The security routines may not be needed for all netmf programs - it depends whether your firmware includes the functionality. The managed providers will always work, but run very slowly. I needed the managed providers for Oxygen, which does not include the security firmware. GHI firmware may include unmanaged versions of these same routines - I’ve never looked into that though.
Either way, the code above should demonstrate how to construct http interactions (https not required) for writing to Azure. If you can open an https pathway, then I would recommend doing so.
I know this isn’t a complete step-by-step example, but if you run into trouble, I’m happy to answer any questions.
EDIT: Ooops - only the blob code is there. The table code is similar. I can help you work that up if needed.
1 Like
@ mcalsyn - thank you for the links, which I didn’t find before. I’ll have a look on it and will keep you informed. May be however, that I do not get the time before the next weekend.
@ mcalsyn - please can you give me an advice.
I’m trying to get your BlobClient working but I could not find out what I have to use for the first parameter of the constructor.
BlobClient myBlobClient = new BlobClient(?????????, myCloudStorageAccount);
.....
public BlobClient(INetworkAdapter adapter, CloudStorageAccount account)
What a coincidence… I’m looking at the sample code for @ mcalsyn awesome ESP8266 work (v 3.0.2) and there are a couple calls that want an INetworkAdapter, but have the Esp8266WifiDevice being passed in, which does not implement INetworkAdapter.
serialwifi/src/netmf43/Tests/SimpleHttpTest/Program.cs is what I’m looking at on github. lines 72, and 79. THe wifi object is the esp8266 device.
var sntp = new SntpClient(wifi, "time1.google.com");
...
var httpClient = new HttpClient(wifi);
So I’ll line up behind @ RoSchmi with the same Q. Is there some other object I should use, or should Esp8266WifiDevice implement INetworkAdapter?
Esp8266WifiDevice implements IWifiAdapter which inherits from INetworkAdapter, so yes, Esp8266WifiDevice does implement INetworkAdapter and can be used anywhere an INetworkAdapter is called for.
The nefarious plan here is that IWifiAdapter is a network adapter, but it will ultimately also implement additional wifi-specific methods, like scanning for access points, connect/disconnect, etc.
That wasn’t fully fleshed out in the serialwifi tree because this code was transplanted to another tree where I am working on ESP, SIMCOM GSM/GPRS, Wiznet, and a couple other chipsets all in one common codebase. In that newer tree, the interfaces are fully populated and have been more fully implemented. That tree will become public when the work is more complete and my sponsors decide it should be, as it currently contains some hints at future products and strategies that they are not ready to discuss yet,
But bottom line: through the magic of inheritance, Esp8266WifiDevice does implement INetworkAdapter
@ mcalsyn -
I’m actually using a spider mainboard in a Gadgeteer application with the EthernetJ11D Module.
The Module is declared automatically in “Program.generated.cs”
private Gadgeteer.Modules.GHIElectronics.EthernetJ11D ethernetJ11D;
...
this.ethernetJ11D = new GTM.GHIElectronics.EthernetJ11D(7);
and in ProgramStarted I use:
ethernetJ11D.UseThisNetworkInterface();
ethernetJ11D.NetworkSettings.EnableDhcp();
ethernetJ11D.NetworkSettings.EnableDynamicDns();
ethernetJ11D.NetworkUp += ethernetJ11D_NetworkUp;
ethernetJ11D.NetworkDown += ethernetJ11D_NetworkDown;
how can I use the EthenetJ11D Module in conjunction with your BlobClient Class?
You’ll have to modify the BlobClient class and change all the references to the INetworkAdapter and to instead refer to the ethernetJ11D adapter or some base class. There may also be subtle differences in the calls available on the ethernetJ11D class.
My use of the INetworkAdapter argument was designed to accomodate a) new network interfaces that are not built into the firmware (ESP, GSM, etc); and b) to provide non-static network interfaces that would allow for multi-interface uses (using more than one network interface at a time).
So, if you are using the ethernetJ11D, think of this source code as a generalized example of how to interop with Azure blobs, but you will have to make changes to the low-level networking calls in order to work against the ethernetJ11D. It’s not going to be plug and play.
@ mcalsyn - Thanks, now it becomes clear to me . I’ll have a closer look on the BlobClient to change the Code.
As I think, another solution would be, to implement a Class according to your Esp8266WifiDevice.cs which makes the corresponding calls to the ethernetJ11D .
By the way: Thank you very much for making your great treasure of code open source.
1 Like
@ mcalsyn - I‘ m back with some results. I managed to get your BlobClient working with the ethernetJ11D adapter and the example of Bejamin Perkins .NET Micro Framework - Create an IoT Device Using the Gadgeteer with Azure Blob Storage | Microsoft Learn as well using your method to create the Authorization header.
With your BlobClient I had some difficulties as Azure refused to accept the authentication of the request.
Finally I found the reason, which seems to be a little bug in your BlobClient code.
public bool PutBlockBlob(string containerName, string blobName, Byte[] ms)
{
try
{
string deploymentPath =
StringUtilities.Format("{0}/{1}/{2}", _account.UriEndpoints["Blob"], containerName,
blobName);
int contentLength = ms.Length;
HttpVerb = "PUT";
string canResource = StringUtilities.Format("/{0}/{1}/{2}", _account.AccountName, containerName, blobName);
string authHeader = CreateAuthorizationHeader(canResource, "\nx-ms-blob-type:BlockBlob", contentLength);
try
{
var blobTypeHeaders = new Hashtable();
blobTypeHeaders.Add("x-ms-blob-type", "BlockBlob");
var response = AzureStorageHttpHelper.SendWebRequest(_client, deploymentPath, authHeader, GetDateHeader(), VersionHeader, ms, contentLength, HttpVerb, true, blobTypeHeaders);
if (response.StatusCode != HttpStatusCode.Created)
{
Debug.Print("Deployment Path was " + deploymentPath);
Debug.Print("Auth Header was " + authHeader);
Debug.Print("Ms was " + ms.Length);
Debug.Print("Length was " + contentLength);
}
else
{
Debug.Print("Success");
Debug.Print("Auth Header was " + authHeader);
}
return response.StatusCode == HttpStatusCode.Created;
}
catch (WebException wex)
{
Debug.Print(wex.ToString());
return false;
}
}
catch (IOException ex)
{
Debug.Print(ex.ToString());
return false;
}
catch (Exception ex)
{
Debug.Print(ex.ToString());
return false;
}
}
protected string CreateAuthorizationHeader(string canResource, string options = "", int contentLength = 0)
{
string toSign = StringUtilities.Format("{0}\n\n\n{1}\n\n\n\n\n\n\n\n{5}\nx-ms-date:{2}\nx-ms-version:{3}\n{4}",
HttpVerb, contentLength, GetDateHeader(), VersionHeader, canResource, options);
var hmac = new HMACSHA256(Convert.FromBase64String(_account.AccountKey));
var hmacBytes = hmac.ComputeHash(Encoding.UTF8.GetBytes(toSign));
string signature = Convert.ToBase64String(hmacBytes).Replace("!", "+").Replace("*", "/"); ;
return "SharedKey " + _account.AccountName + ":" + signature;
}
protected string GetDateHeader()
{
return DateTime.Now.ToString("R");
}
In your code the method GetDateHeader() is for example called from the PutBlockBlob method directly to create the unsigned x-ms-date header and than in the sub method CreateAuthorizationHeader to create the signed Authorization header. Since GetDateHeader() method is called from different places in the code the method will return different values as the result. However the Authorization header has to be created with the same date value that is transmitted as the unsigned x-ms-date header. I think Azure needs the right unsigned x-ms-date header to test if the signed Authorization header is valid.
Another point is, that perhaps it may be better to use
return DateTime. UtcNow.ToString(“R”);
instead of
return DateTime.Now.ToString(“R”);
in GetDateHeader() method
to be independent from different time zones.
1 Like
Excellent find! Yes, those are two important bugs. Certainly, calling for a date stamp in two places is a Bad Thing, and I should be flogged with a soggy ethernet cable for using .Now instead of .UtcNow. Whether that matters or note would depend on whether the resulting string contains a TZ offset, but really, in protocols, UTC times are best practice. The doubled time query definitely would cause problems if the clock ticked over between queries.
Thanks for pointing those out!
The code has been repaired - you can find it here : [url]https://github.com/PervasiveDigital/serialwifi/blob/master/src/Common/PervasiveDigital.Net.Azure.Storage.Shared/BlobClient.cs[/url]
I made sure the timestamp is only acquired once.
I also changed to UtcNow, but it turns out that really wasn’t necessary, because DateTime.ToString(“R”) uses RFC1123 format, which is always in GMT. Using .UtcNow probably just saved some cycles doing unnecessary conversions.
Thanks again - I won’t be pushing new nuget packages right away, but these fixes will show up in the next cycle in April.
1 Like
@ mcalsyn - thanks, I’m happy if I could help.
1 Like
I send all of my telemetry data to TableStorage. I am interested in what use case you guys would use BlobStorage. Are you storing IOT documents or something?
Same here - I just included blob support for completeness.
@ mcalsyn - I didn’t see your TableStorage class, did I overlook it?
@ terrence - Probably not. It may not have gotten checked into the serialwifi project. I have been working on that code as part of a much larger lib for intelligent networking breakouts, so that there’s a unified Azure stack for those (SIMCOM, ESP, CC3100, W5500, etc). I will go back and make sure it shows up in the existing serialwifi project. Apologies for the oversight there.
@ terrence - Initially I wanted to use TableStorage as well and I’m still interested in NETMF code to upload TableStorage entities.
If you have code for TableStorage ready to show, I would be happy not needing to reinvent the wheel.
Until now I was struggling with the authentication which now works with the available Blob examples and struggling to port mcalsysn’s serialWiFi code to use the gadgeteer ethernet module.
While thinking about my goal to monitor the temperature in our house from everywhere,
I now thought that BlobStorage evtl. could be even better for this purpose than TableStorage.
I could upload an xml-File with one filenmame for each day for example every 15 min which holds all the measured values of this day.
Additionally I could once upload an xls Stylesheet which is referenced in the xml files.
So I could monitor the temperature from everywhere by using a simple internet browser.
For TableStorage I would certainly need a special App to see the data.