I’m looking at getting into .NET MF/Gadgeteer by building a vehicle that sends video to and can be controlled by a Windows Phone device. Motor control sounds pretty easy and I’ve seen several projects for using a WinPhone as a controller so I’m not worried about the input side of things, but the video side has me worried as the stuff I’ve read is pretty sparse.
So, questions:
Is this even possible? That is, is a FEZ board capable of sending out a video stream or reasonably responsive series of images capable of allowing me to drive the vehicle remotely?
Is the GHI camera module my best bet or should I be looking elsewhere? 8-10 fps is fine I think but GHI’s description suggests that you only get decent framerates at really low resolutions and a video I’ve seen of a guy running it at full rez was rather comically bad.
Do I need an RS21 module or is it possible to use a wifi bridge given that I’d need to use the phone’s onboard wifi system? I’m curious about other communication options too, because if I’m satisfied with the results I might look at applying it to an airplane.
The Cerberus mainboard seems like it would be a fantastic choice except for the <1MB RAM and (possibly) no RS21 capability, has GHI considered doing a “Cerberus Plus” that’s got maybe 2MB RAM and the premium-feature goodies while still retaining the lower cost, better power use, and all those robotics-friendly PWM sockets? Spider seems like mad overkill for a project like this and “it just happens to have the right socket” is kind of a lame justification.
RS21 is only on Spider. The non-serial camera only works on Spider too, double bonus if you can see what I would suggest
Cerb is really too small to consider for such an intense requirement - you’ll hit memory constraints very quickly.
Having said all that, video is hard on it’s own, let alone when then trying to stream it out a network. These devices are NOT video processors, not optimised for that kind of task, don’t have high-speed interconnects to wireless devices, and I hate to say it but they are probably not capable of doing what you want with any degree of “performance”. Most FPV systems transmit megs of data over dedicated hardware and bandwidth, that a Fez will struggle with. There’s an existing project that does most of what you want, Wi-Fi Gadgeteer Robot controlled by Windows Phone with image streaming | Integral Design that Marco might chime in with some details here about how far he might have tried to push the frame rate up to.
Yeah, that’s the project I’ve seen the videos for. Very promising, but running the camera at full 320x240 looked like major fail. I’m not really going for hi-rez FPV so much as enough resolution to avoid large obstacles like walls and people walking around–a lower rez like 160x120 or even 128x96 would be totally fine. So yeah, if Marco is around I’d love to hear his take on this.
I’ve thought about doing some basic image processing–adding estimated speed and whatnot in a HUD like overlay–but it’s not a priority and is more something I’d think about if I did the airplane thing sometime in the future when that sort of thing really matters.
saying things like that might not get the detail you’re looking for
“I personally thought, well done for being so brave as to try that, but I might have tried sending a lower resolution image back to the phone”
Spider may not have the most grunt when it comes to mhz, but it certainly has a richness of supported connections/devices, and the amount of memory at it’s disposal. So all in all, it’s not a bad platform - but we also know there are other new things not far over the horizon, so depends if you’re itching to get started now or later. You haven’t updated your profile so nobody knows where you are from; I keep saying this because if someone from Sydney AU appears here and is deciding what device to buy, I have a “loan” device or two they can try out before they cash in their readies for hardware… bueller?
The goal of my Wi-Fi robot with image streaming was to demonstrate the capability of the platform, so I haven’t worked much on optimizing the speed of image streaming.
After this project, I have done some other experiments. A first approach can be using direct TCP connection instead of WebEvent, because in this way there isn’t the overhead of HTTP transport protocol.
Another problem is that the Camera Module I have used returns image in bitmap format. This means that, with a resolution of 320x240, every image has a size of 232440 bytes. Using the new Serial Camera, that natively supports JPEG, image size (and so transfer time) will be greatly reduced.
Hey Marco, thanks for the update. So from what you’ve seen do you think the bottleneck is getting the images off the camera or transmitting between board and phone? I just figured the camera was the problem based on the product description’s vague “sometimes it does 20 fps”, but maybe I’m misinterpreting. And yeah, for the transmit I’m probably going to go with something like TCP for faster speed and less reliance on web-centric technology since I may trade RS21 for XBee at some point.
Anyone know what sort of framerate the new serial camera will be able to achieve?
Serial camera is slower than USB camera. You get about 0.3 frames per second.
Video is a beast that is not easy to handle by any embedded device. You need 500Mhz+ and 50MB f=or RAM to do anything useful. Devices you see around here can do amazing work in any area you can think of, except audio/video processing, which are very computational intensive apps. Even high end systems, have special hardware (or processor instructions) to speed this up, even your 3Ghz PC does!
That’s not too shabby for what I’ll be needing it for. Heck, 160x120 is half a Youtube video and I just need enough to avoid large obstacles. I’m definitely warming up to the idea of going through with this, thanks for the info guys.
I have tried to describe all the factors that must be taken in mind about image streaming, but the most important one is surely the speed of the camera itself. So, as Gus said, considering the frame rate of Serial Camera, it can’t be used for your purpose.
I too will try to make some tests with image streaming over TCP using the USB Camera with 160x120 resolution. I will keep you informed If I discover something interesting.
Well, the most powerful device that I personally own that can run Gadgeteer would be my PC, which is a Core i5-2500K. Yours may be something different.
More seriously, to do video streaming, you use dedicated hardware. Even in a PC, you often use dedicated hardware (your video card), or at least dedicated instructions on your CPU.
@ godefroi - Nah that would be too cost ineffective. I wonder if the Xmos chips (10 bucks a pop) could be of use. Given that the Cerb40 can do High Speed USB transmission, it may be possible to push the frame data to multiple xmos chips which in turn push out to a wiznet module, which then sends the data over some sort of high speed wifi module (like a wireless access point).
Look into the MG3500/MG2580 ICs by Maxim. They’re the sort of thing you’re looking for. They also include ARM cores that could run NETMF, if a port were produced. An FPGA with sufficient gates could certainly be configured to encode video, but that’s way beyond hobbyist level, there.
Now I if I could only find a price for those Maxim chips. This RedPine module says that it supports 64Mbps over WIFI n (Silicon Labs) with SPI connectivity.
