Do you mean “Extend deployment”? If so then yes. The application file works fine on multiple devices.
Instead of saving to IFU, save to SD, or calculate MD5, make sure what you get from network are correct.
Or make a simple project so we can test qucikly.
Is this a legit way to calculate MD5 while downloading?
static void DownloadFile()
{
var url = "http://my.url/myfile";
Debug.WriteLine("Start Download");
int read = 0, total = 0;
byte[] result = new byte[1024];
var qspiDrive = StorageController.FromName(SC20260.StorageController.QuadSpi);
var updater = new InFieldUpdate(qspiDrive);
var md5 = MD5.Create();
byte[] md5Hash = new byte[16];
try
{
updater.ResetChunks();
md5.Clear();
var appKey = new byte[] { 0x22, myappKey, 0xbd };
updater.LoadApplicationKey(appKey);
using (var req = HttpWebRequest.Create(url) as HttpWebRequest)
{
req.KeepAlive = true;
using (var res = req.GetResponse() as HttpWebResponse)
{
using (var stream = res.GetResponseStream())
{
do
{
read = stream.Read(result, 0, result.Length);
updater.LoadApplicationChunk(result, 0, read);
md5Hash = md5.ComputeHash(result, 0, read);
Array.Clear(result, 0, result.Length);
total += read;
}while (read != 0);
}
}
}
md5Hash = md5.Hash;
var Hash = "";
foreach(byte b in md5Hash)
{
Hash += b.ToString("x2");
}
Debug.WriteLine("Hash: " + Hash);
Debug.WriteLine("Total : " + total);
Debug.WriteLine("Start Verify");
var firmwareVersion = updater.VerifyApplication();
Debug.WriteLine("Verify Done: " + firmwareVersion.ToString());
}
catch (Exception ex)
{
Debug.WriteLine("Download Error: " + ex.Message);
}
_DownLoadFile = false;
}
The Hash that I get is different than what I get from the file that is on the webserver. (webserver file and dev pc MD5 match.) However, I am not sure I am calculating it correctly.
The byte count matches between the server and the total I am receiving. If the data is getting corrupted on download then I am not sure where to go next.
The MD5 and HttpWebRequest api is similar to Desktop. You can try on PC to compare.
Or, Crc16 or just download to SDCard… few ways to compare what you get by code above is correct or not.
Using a C# console app to calculate the MD5 and here are my results.
When doing the MD5.ComputeHash inline with the download I get the same I get on SitCore:
MD5 done inline: d41d8cd98f00b204e9800998ecf8427e
In the console app, I save the file to disk and calculate the MD5:
MD5 full file: 1a52799318f7b40d42412113b46bb51d
The full file hash matches the server and original .TCA file. So it appears that all the bytes are making it through the cellular download. But there is a difference between the inline compute and the full file compute. I would need to run the full hash on what gets saved to qspi flash. Is there a clever way to do that?
I don’t think you can do inline like that.
You should read them all, keep it memory (I see you use SC20260 with 32MB memory). Then do MD5 for all file one time. It is much more simple. Do same with PC.
Compare two values.
What we care about is the content of your download. You can download to SDCard and bring to PC to compare with orginal file.
You can make a program 1.8MB (add some rescources), extract TCA, and send project to us so we can run quickly. Make sure you can reproduce with that project first.
Any thing news so far?
We need to make sure IFU has no issue. Can you reproduce by a simple project and send to us?
I will be working on it again this evening. Out in the field visiting customers last few days.
I will get back to you.
Okay here is where I am at:
Downloading to SPI Flash does not work.
Downloading the file to SD card, closing, then reopening and verifying works.
I can make the SD card work but it would be simpler if SPI Flash would work.
Edit:
I am running updater in debug mode. I can see the “Writting to flash” messages. (yes, the spelling is wrong. 
)
Start Download
Erasing flash: 0x00321000
Writting to flash: 0x00321000, size 0x0000035a
. . . .after 6 to 10 minutes . . .
Writting to flash: 0x004efc92, size 0x00000030
Writting to flash: 0x004efcc2, size 0x0000033e
Writting to flash: 0x004f0000, size 0x00000000
The total file size is 1896448 (0x1cf000) bytes. (0x4f0000 - 0x321000 = 0x1cf000) So the bytes written seems to match.
Can you please buffer 1K and write 1K each time?
Loading Application in 1K chunks works!
I also noticed that every 4K there was an “Erase Flash” message, that wasn’t there with random chunk sizes.
Test Code:
using (var stream = res.GetResponseStream())
{
do
{
if (stream.Length >= 1024)
{
read = stream.Read(result, 0, result.Length);
updaterSPI.LoadApplicationChunk(result, 0, read);
md5Hash = md5.ComputeHash(result, 0, read);
Array.Clear(result, 0, result.Length);
total += read;
}
} while (stream.Length > 0); // (read != 0);
md5Hash = md5.Hash;
var Hash = "";
foreach (byte b in md5Hash)
{
Hash += b.ToString("x2");
}
Debug.WriteLine("Hash: " + Hash);
Debug.WriteLine("Total : " + total);
Debug.WriteLine("Start SPI Verify");
updaterSPI.LoadApplicationKey(appKey);
var firmwareVersion = updaterSPI.VerifyApplication();
Debug.WriteLine("Verify SPI Done: " + firmwareVersion.ToString());
}
}
Will the TCA files always be in 1K increments? The test code above doesn’t have a provision for less than 1K at the end. If the TCA file is always 1K increments then I won’t have to worry about it, or will I?
I am still calculating the MD5 hash as a test. The end result is that the MD5 hash calculated in line with the download doesn’t match the hash generated on the server or PC.
Frist:
- the buffer can be 512, 1K, 2K, 4K, 8K…
- every time write the size need to be same. You can’t write 1K then next 2K. If decided 1K then all must be 1K. If decided 2K then all must be 2K, except for last block.
- Tca is always multiply of 1K.
Second:
- I don’t think do MD5 inline will work. As your log, I see every time reading data from network, the size is different, so MD5 will give different result of couse.
- if tried 1K worked, then no worry, we know what your problem is. You can tried 2K, 4K for sure, but not 3K or 5K because sector size is 4K.
- If tried 2K, 4K… then need to take care last block. Example tca 9K = 4 + 4 + 1. If do 4 +4 +4 then exception :)).
@Dat_Tran Thanks for the help. I hope I didn’t miss something in the docs that could have saved everyone’s time. If I did I apologize.
I will stick with 1K buffer size. I don’t have 32MB ram and with MQTT connected to Azure my RAM is a bit tight.
I don’t think MD5 is important since the IFU verify will fail if the file is corrupt.
When VerifyApplication() fails are there different exceptions indicate if it was due to a bad Application Key or corrupted file?
Now the hard part, creating the back end to support over-the-air updates on a fleet of devices.
yes, MD5 is not important. Once we found out the problem, we recommend take MD5 checking off.
No, they are same exception.
Let us know if 1K totally working, thanks
Just an update. (Sorry for the long delay, got pulled to other projects.)
The OTA update of application and firmware on an SCM20100E with extended QSPI Flash, over cellular, worked. It is painful to test since it takes 15 to 20 minutes to get the application and firmware downloaded. I am using a CAT 1 modem and I have a very weak signal.
There is much to do to build out the device app and the back-end server but the update process does work.
Thanks to everyone for their help.
3 Likes
Hey skeller, while of no real help at this point with OTA, are you using any compression algorithms/engine.?
I would love to see compression with ability to .zip file/folder structures.
BTW:
When we IFU apps that are say, 1MB in size over cell CAT1 LTE to SD Card…it takes approx 5min…again no compression on the bytes and bits.
I am not using any compression. I suspect the reason for the slow download is due to the very weak signal that I have. I am inside a metal building. I think the signal is about as weak as possible but still able to connect. While sometimes a pain, it does make it a bit easier to test getting the modem and the network stack reconnected when the signal drops out.
Yeah, understand where you are coming from as far as cell testing.
We ended up building a tunable ‘poor mans’ anechoic chamber, which gives a pretty good ‘real-world - within-a-lab’ experience.
…of course cell has deeper inner-demons, only experienced when out in the field.
If anybody else is reading this thread…ideas for compression.?
This was helpful in debugging an HTTP update issue I was running into. I was trying to execute an IFU by downloading from an HTTP server and directly loading the chunks to the updater. Kept running into the generic system exception issue though. Turns out the HTTP class downloads in inconsistent chunks and were thus being loaded into the updater in various sizes causing the exception. Had to separately buffer the HTTP chunks into 1024 bytes before loading, and that seemed solve the issue. Wanted to share here for others having the same frustration. Might be worth adding an exception to the updater Load…Chunk() function that prevents abnormal byte array sizes?
1 Like