Distributed Computing & Clustered Networking

Thanks Tom.

Your write up implies that all the nodes must be within on the same IP network, so UDP broadcast will be heard.

I have been doing some thinking about a world wide ad hoc network of NETMF devices, similar to what you described, but I have not
been able think of a good use for it.

I believe that PIP and Vault-Link are your projects?

Mike for that [em]specific mode[/em] you are correct. The mode we will be testing broadcasts the serverā€™s IP to my website (on a static IP) as well as over UDP. This way nodes outside of my LAN can grab the externally visible IP and connect. Obviously firewalls will come into play but Iā€™ll open mine up for the test.

My use for this will be quite specific. All of the devices in my house will network so I easily share file between all of them, update all my devices in a single command (it takes me about 2hrs to do it by hand at the moment) and reach into my devices remotely (i.e. from the office).

Other useful applications would be for swarming bots, clustered quad-copters, etc.

PIP & Vault-Link are personal projects, Iā€™ll probably open the source but theyā€™re not commercial apps. The names & interfaces are based off the Fallout series of games; my fiance and I both like to play them and the project is greatly inspired by creating devices for her. :slight_smile:

Again, thanks Tom.

Sounds like you have implemented what I was thinking about.

I am in for multiple devices, a Cobra and a Hydra (I have a fleet of USBizi on 4.1 if you want to try those, but I suspect not :slight_smile: ) At the moment all are running 4.1 so let me know if you only expect 4.2 devices and Iā€™ll update my SDK somewhere :slight_smile:

@ Brett - Vault-Link and SBASIC are both extremely small so USBizi should be able to run either or both just fine. I will be releasing 4.2 and 4.1 since I run both as well. :smiley:

@ Skewworks - Looks like you just missed out on the PIPā€¦

It was a similar project like that on Hack-A-Day that started this whole mess. :slight_smile:

I showed it to my fiance and jokingly said we should make one. She was so excited I had no choice but to actually go through with it.

If youā€™re targeting USBizi too then I can add 2x Pandaā€™s to the mix. 2x Panda, 1x Cobra, 1x Hydra, all on 4.1 so I donā€™t have to work on firmware versions :slight_smile:

You can get all of those connected to the web at the same time? Iā€™m impressed.

Logistically, yes I suspect I can do it. 10/100 hubs are cheap :slight_smile:

Iā€™m in GMT + 11 at the moment (summer time has kicked in). From a timing perspective, Saturday night your time is Sunday my time, Sunday is Monday; so Iā€™d be more likely to be able to help on the Sunday my time Saturday yours.

Iā€™m hoping that Iā€™ll be available and that I have some mainboards available (lots of projects), but I should be able to put up half a dozen or more including

ArgonR1
Mountaineer (might have to order a networked one to make this easier as currently I just have USB Mountaineer boards, but maybe this is my excuse to order some networked Mountaineer boards)
Spider
Hydra
Cerberus

Which should give you pretty much a clean sweep of the Gadgeteer boards. How long do you think youā€™ll need them online for? Of course firewalls etc might be a bit of a speed bump to consider for this project.

Hey guys, Iā€™ve got a test time that I think will work for everyone. It wasnā€™t easy trying to span nearly the entire globe (17hrs in total time difference) but if we can pull it off, the results should be pretty amazing.

Scheduled time would be Saturday December 1st at the following times for each zone
04:00 PM - Calgary, CA
05:00 PM - IL & TN, US
06:00 PM - VA & VT, US
11:00 PM - GB & FR

And Sunday December 2nd at 9AM for Sydney.

Prior to the test each device you plan to use would need to be programmed and configured for what it can do (ie FileIO, Screen Display, etc); this is extremely simple and there will be lines of code commented out and waiting for you to uncomment as needed.

Each device will then connect at the specified time (Iā€™ll probably be online about an hour early).

Devices will automatically create a GUID to identify them (you can also specify a name for each device or simply put your name for all devices; either way will help me see what locale is connecting)

About 5-10 min in as I see everything connected Iā€™ll blast a broadcast message to all devices, those with screens will display the message for you.

Following I will send PE files to those devices that support Distributed Computing and File IO. There will be 2 distributed programs.

Next I will trigger all available distributed computing devices to run an app that will split a list of PNGs between devices, which will then be decoded into black background bitmaps and uploaded for verification.

Once the devices report back, Iā€™ll trigger the second distributed app that will calculate prime numbers within a given range. These numbers will be reported back to the server.

During this time any device that cannot support distributed computing but DOES have a screen & USB keyboard can be used by you as you like to ping devices and run commands.

After the distributed computing tests are complete Iā€™ll force the server offline, which should trigger all the devices to locate a new server after a set timeout of not being pinged.

Tests should conclude in about 1 hr. After everything is complete Iā€™ll blast another message thanking everyone and then force all devices to disconnect.

Calendar entry added, therefore time booked :slight_smile:

1 Like

What prerequisites do I need on the boards (netmf version, etc), and what do you need for info from me, if anything?

Latest firmware 4.1 or 4.2, both will be supplied. You need minimum of Ethernet or WiFi. Any device that will participate in Distributed Computing test (not required) needs SD as well. If you attach an LCD and keyboard you can test sending any commands you like to any device you like (command set will be supplied). If I have time I will include a Windows app as well so you can join with your PC.

Will it work on a Panda-II + Connect shield? If so, I can hook up one of those + a Hydra.

Itā€™s on the calendar!

1 Like

I havenā€™t tested with anything other than built in Ethernet and Wifi. Iā€™m hoping to get some work done Thusrday morn and Friday. Maybe I can send something off to you this weekend for a quick pre-test?

Sure. If youā€™re not going to hand deliver it :frowning:

@ Skewworks -

Do you mean 11PM LOCAL TIME for France ? Note that thereā€™s 1 hour between GB and FRā€¦

Sorry I was just looking at a small world map of timezones when calculating. Start time is 5PM Central (central is UTC - 6)