ML on TinyCLR 2.0

Hi GHI Team,

In the era of AIoT (AI and IoT), I think you need to consider to add machine learning feature on TinyCLR 2.0. With the power of SITCore, relatively huge flash size and enough ram we will be able to execute lite version of ML (machine learning) model.

AI on the edge is really useful when we need to off-load some AI processing from cloud to device to improve processing speed, safe bandwidth, reduce costs, and we can still operate without internet connection.

I got some references:

  1. STM32 X-Cube-AI https://www.st.com/en/embedded-software/x-cube-ai.html - we can convert model from keras, caffe, tensorflowlite, convnetjs and run it on mcu
  2. Execute simple neural network on arduino - https://www.anscenter.com/Blogs/Blog/GetPost/636879318392972431
  3. Tensorflow Lite - https://www.tensorflow.org/lite/microcontrollers

Some scenarios that we can achieve with this feature:

  1. Anomaly detection from time series data from sensors
  2. Preventive and Predictive maintenance
  3. Smart camera with object detection
  4. Smart Kiosk with product recommendation
  5. Hand written / gesture recognition on touch screen

What do you think ?

4 Likes

I actually think this is a strategic decision, not a power/capability decision. (As you may know, I’m a former MSFT employee, and while I have no knowledge of their product futures, and had no involvement in their product lines previously, I’m familiar with them so use them as example only).

The strategy for TinyCLR OS should be clear - if it’s trying to be the best sensor/endpoint device , then that’s what it should be. If it’s trying to be a good all-round device, then something like this may make sense. But don’t do it and sacrifice the core vision.

My 2c - that capability is probably intended to be deployed in an “intelligent edge” device in Microsoft’s terms - a Windows IoT device deployed centrally for many edge devices that I see TinyCLR devices being.

I personally think there are huge machines that does machine learning very well, like azure. SITCore is meant to be a perfect sweet spot in between a tiny door sensor and full Linux edge system.

That said, we have a powerful micro and we should consider options after we are done with the correct plan.

1 Like

So you agree with me then Gus ! Get the foundation right, build the house with room to expand :slight_smile: Or perhaps even open the avenues for someone to bring a modular room to attach to the house (like RLP was)

1 Like

Agreed sir.

1 Like

Brett, do you mean GHI should make sure there is enough conduit in the foundation for future grow?

and reo. Can’t have too much reo. Well I’m sure you can, but who would…

REO? Real estate owned? In US means bank foreclosure properties?

no, lowercase reo - reinforcing mesh, https://www.bing.com/images/search?q=reo+concrete&qpvt=reo+concrete&FORM=IGRE

While software solution such as X-Cube-AI & TFLite-micro are appealing since they don’t require extra hardware, their performance is brutally slow. See a XCUBEAI & TFLiteMicro STM32F746 benchmark here (https://www.stupid-projects.com/machine-learning-on-embedded-part-3/). - A simple naïve neural network trained on MINST hand writing data set yield result as slow as half second for only 28x28 pixels input.

For this level of performance forget about audio / video analyzing. It’s much better to run machine learning logic on dedicated hardware such as Jetson Nano and XAM3500.

Yes, I agree, we have a room for community contributions. Maybe in the future we can add NPU/APU/other AI accelerator as an expansion to handle ML task