The Great Network Shootout

Archive
1

Cerberus vs Hydra vs Mountaineer vs Spider Wifi, who is the most reliable? Who has memory leaks? Place your bets.

The code is simple every 30 seconds for however long I leave them running.


 
        private void _timer_Tick(GT.Timer timer)
        {
            _memory = Debug.GC(false);
            _upTime = PowerState.Uptime;

            _iteration++;
            dataEntry.Fields[0] = _memory.ToString();
            dataEntry.Fields[1] = _iteration.ToString();
            dataEntry.Fields[2] = _status.ToString();
            dataEntry.Fields[3] = _resets.ToString();
            try
            {
                _status = _thingSpeak.Update("NOPE", dataEntry);
                dataEntry.Tweet = string.Empty;
                dataEntry.Twitter = string.Empty;
                dataEntry.Status = null;
            }
            catch (Exception ex)
            {
                Debug.Print(ex.Message);

                dataEntry.Status = "Last error: " + ex.Message;
                dataEntry.Tweet = "Mountaineer Error " +
                                  ((ex.Message.Length > 140) ? ex.Message.Substring(1, 140) : ex.Message);
                dataEntry.Twitter = "NOPE";

                _status = -100;
                _resets++;
                ethernetForMountaineerEth.UseDHCP();
            }
        }

The live results:

Cerberus Channels - ThingSpeak IoT
Hydra Channels - ThingSpeak IoT
Mountaineer Channels - ThingSpeak IoT
Spider Channels - ThingSpeak IoT

forgot the photo

11 Likes
2

hydra is behind by a couple due to a false start

3

Glad you are finally into finding out why there is so many flatliners on your office palms. Seriously looking forward to the results. :slight_smile:

4

Excellent

5

Excellent use of thingspeak! Canā€™t wait to see the results.

6

@ Duke Nukem - Can you adjust the scale on the x-axis on the public view, to encompass a few days maybe?

7

I believe I have updated all the public views to 6000 data points for each chart. Let me know if I missed any.

8

Perhaps time to shoot this and reload as it appears the only device still going is the Mountaineer and even it has lots of errors. NOTE when I say errors I mean something caused the upload to choke, why it choked is for investigation later as it could be a ton of things, but looking at the charts memory leaks donā€™t seem to be a problem with any of the boards as memory usage was consistent throughout the test.

Mountaineer (running 4.3) was still going (6450 entries recorded, and 114 errors)

Cerberus made 2570 iterations with 3 errors

Hydra made 2570 iterations with 3 errors as well, perhaps a ENC28 issue?

Spider with Wifi 1361 iterations, but no errors

I had hoped that the error messages would appear on ThingSpeak as I put them in the status, but perhaps there are text limitations or something that prevented this from working. Iā€™ll experiment with this some more as I really want those error messages and perhaps what Iā€™ll end up doing is a SD Card based log file or something.

9

Premium is awesome :-[ Just kidding, please let us know what you further find and we will try our best to make everything zero error.

10

Iā€™m with you Gus on this as I know we would both like this to be absolutely perfect. I think Iā€™ll add a Spider with a J11D module to the suite of test platforms. The other thing I think Iā€™ll do this time around is put all of these through the same router (wired went through one router and the wifi went through another) and record the syslog data from that router and see if there is any information in there that might help figure this out.

11

[quote=ā€œDuke Nukemā€]
The only device still going is the Mountaineer and even it has lots of errors.[/quote]
It is good to see the diligent work of Pierre-Yves on the lwIP integration code paying off, and it is consistent with the feedback from our commercial customers: no need to reboot a device, just to get networking up again.

Iā€™d definitely would like to learn more about the root causes of these errors (exceptions), and whether they are normal error situations as they may occur in IP networks, or not. Whether they depend on the router model, network saturation, etc. We had comparable tests, over longer periods of time, without such errors. To be able to reproduce them would be great.

Thanks for the testing effort, highly appreciated!

Cuno

12

Now that Iā€™ve seen some errors, time to figure out what they are, hence capturing the details and router logs etc. I think a lot of people falsely assume the internet just works all the time, when in fact there are so many potential points of failure that its enough to spin your head and your device. Time to find and line up all the dragons from biggest to smallest and start slaying them as networking is one of those key ā€˜gotta workā€™ features for devices.

Cuno I was impressed how the Mountaineer just kept plowing on so please pass my appreciation onto Pierre-Yves for his impressive work around networking.

13

@ Duke Nukem -
Any chance of trying mIP on the Cerberus with the ENC28?

I would love to use this as a benchmark for the CC3000 as well.

-Valkyrie-MT

14

What do I need to set it up? I should have another Cerberus and ENC28 around here that I could use for adding this to the test suite.

15

What does this mean? It just went to 1361 iterations and didnā€™t try to upload anymore but there was no error thrown? So, there was an error throwing errors??? :wink:

16

[quote=ā€œianlee74ā€]
What does this mean? It just went to 1361 iterations and didnā€™t try to upload anymore but there was no error thrown? So, there was an error throwing errors??? :wink:
[/quote]That would be the million dollar question, and maybe on the next round Iā€™ll have an answer.

17

Most likely your devices ran into the connect hangs forever bug. The Mountaineer is running 4.3, which has a fix for this or so I have heard. If you modify your code to run the internet connection part in a different thread and kill the thread after a timeout of several to 30 seconds, I am sure you will see it run for longer.

I had a device running 4.1 on a CANXtra that did over 150K iterations with two updates each iteration. It probably would have kept on going, but the windstorm took out power and internet.

18

@ Duke Nukem - is it 4.3?

19

The Mountaineer is running 4.3

I donā€™t do anything with threads in any of the devices as Iā€™m trying to keep it as generic as possible.

20

The log file from my latest Hydra test.

Interesting the error capture here all have to do with unexpected http response from ThingSpeak which messes up the parsing of the response.