Main Site Documentation

Capture Screen and Email


#1

OK, I’ve been beating my brains out over this for several days, and I realized that perhaps if I offer up more info on the scenario I’m trying to enable, perhaps some of the sharp folks on the forum can point me to a smarter way to approach the problem.

Basically, what I’m trying to build is an application that will:

  1. Capture images from the camera module and display them on the screen, preferably streaming the images until an on-screen button (or could be a button module, not picky) is pressed.

  2. Overlay a pre-defined image as a sort of frame for the picture captured by the camera.

  3. When the button (on-screen or module) is pressed, capture the screen image, and convert it to a format that can be sent via email.

  4. Send an email to a specified email address with the captured image.

I have 75-80% of this working. I am able to stream the images from the camera to the screen. I am able to overlay a predefined image. I am able to capture that image as a bitmap, and I’m able to send a text email (without the image, so far).

Where I’m falling down is getting the image data in a format that’s conducive to sending via email. The road I’ve been going down is to use the ConvertBase64.ToBase64String method, but that was throwing an OutOfMemoryException (per this thread: http://www.tinyclr.com/forum/21/4688/), so I’m beginning to wonder whether perhaps I’m approaching this the wrong way.

I was able to get ToBase64String to run without exception by chunking up the Byte array using the following code:


        string WindowToBase64()
        {
            byte[] buffer = new byte[8192];
            int offsetval = 0;
            byte[] imageBytes = window.Graphics.GetBitmap().GetBitmap();
            int winlen = imageBytes.Length;
            String tempStr = "";

            while (offsetval < imageBytes.Length)
            {
                int length =
                   System.Math.Min(buffer.Length, imageBytes.Length - offsetval);

                Array.Copy(imageBytes, offsetval, buffer, 0, length);

                offsetval += length;

                tempStr += ConvertBase64.ToBase64String(buffer);

            } 
            
            return tempStr;
        }

The problem with that is that it’s extremely slow, and the resulting string doesn’t appear to be a valid base64 string (or so the desktop framework tells me when I try to convert it back into an image in another program).

Is what I’m trying to do unrealistic for the Spider kit?

Is there another way of doing this that would make more sense?

Thanks in advance for any tips or suggestions.


#2

Is it an option to build your own RLP that handles Base64 conversion? I am not sure why teh bult in methods needs much memory!

When you get out of memory, how much is it trying to allocate and how much RAM you have left. I find this hard to believe with 16MB FEZ Spider.


#3

Well, that’s the weird part, Gus. I don’t think the Spider is out of memory. At least, when I tried running Debug.GC(true) as you suggested, the results didn’t indicate anything close to out of memory.

My guess is that the ToBase64String method in System.Http.dll may have some internal limitations that I’m hitting when I try to convert the entire byte array at once.

But the image is only 307k, so I’m as puzzled as you as to why it should be so problematic.

That’s why I started wondering whether perhaps getting a fresh set of eyes on the concept as a whole might yield some suggestions for a different way to approach the problem.

As it is, I’m getting kind of frustrated, so I may end up putting this aside and just focusing on building out some simpler demos for the presentation with Pete on Saturday.

Was really hoping to show something more elaborate to make the Gadgeteer platform shine! :slight_smile:


#4

Use Reflector to see how ToBase64String is implemented. Move the code to your project and optimize it if needed.


#5

Maybe even a bug. Peek in the porting kit if you like or report this at netmf.codeplex.com


#6

Hi There,
have you tried to actually convert the captured image directly from Camera to base64 first before involving the extra frame…

This worked for me… kind of slow but it worked…



        void CameraPictureCaptured(Camera sender, GT.Picture picture)
        {
           // display.SimpleGraphics.DisplayImage(picture, 5, 5);
           
           string myPicStr = ConvertBase64.ToBase64String(picture.PictureData);
  //when i check myPicStr in the watch window i see the data encoded...
        }

maybe if you post the e-mail code we can try it and let you know.

Cheers,

Jay,


#7

Jay

I’ll give that a shot, but it does miss one of the requirements, which is to overlay another image on the picture captured from the camera.

Still, it’s worth giving it a shot to see if that works without the memory issues. That might also give me a clue as to what’s going on.

Thanks!


#8

Good news/bad news.

The good news is that I was able to successfully call ToBase64String in the manner suggested by Jay, without any exceptions. Yay!

The bad news is that the resulting base64 string appears not to be valid base64. Boo!

So I’m back to the drawing board and open to any additional suggestions.


#9

Sounds like you just need to find a working ToBase64String(). You might try dropping in something off the web or write your own and see if that solves all your issues. Then submit a fix to NETMF.


#10

May have found an existing solution that will work:

http://bansky.net/blog/2008/08/sending-e-mails-from-net-micro-framework/

Need to get some sleep before I check out the code, however. Stay tuned… :slight_smile:


#11

Looks like there are a couple gotchas that Pavil has discovered and worked around. I think you’re onto something!


#12

Well, maybe.

As I write this, I’m running the app using the Base64.cs from his project. Trouble is it is enormously slow, as in many, many minutes to encode the roughly 307k image we’re talking about here.

I tried tweaking the code a bit to use the rough StringBuilder implementation posted by Chris Walker over on the netduino forums, and that was considerably faster, but unfortunately, ended up throwing an OutOfMemoryException in mscorlib.dll. Argh!!!

So, I’m going to let this code run, however long it ends up taking, and at least check to see whether it produces valid base64. Then the question becomes whether it’s possible to optimize the performance short of rewriting it completely as RLP (which I would first have to learn :)).


#13

Half an hour later, it’s still only about 1/3 of the way through concatenating the base64-encoded char array into a string. There’s GOT to be a better way to do this!


#14

Ouchers.

Get on communicator and show me the code.


#15

How about this:

http://netmf.codeplex.com/workitem/332

maybe it’ll give you some insight…

Cheers.


#16

GHI libs include a function to convert the NETMF byte format to standard BMP.


#17

BTW here are some faster C and ASM BaseTo64 Encoders:

Enjoy :wink:


#18

Thanks Gus and Jay,

Alas, I let the current code run overnight, as I just couldn’t stay up any later, and my laptop went to sleep, but the code had not yet completed converting the char array to a string. Looks as though that particular operation is sufficiently slow that trying to do it three hundred thousand times in a loop is impractical at best.

I think the reason that the author of the base64 library I’m using is converting one character at a time is to be able to insert newlines every 74 characters to comply with the MIME specification for sending base64 attachments in email.

The downside is that assuming this works the same as desktop .NET, that means the code is destroying and recreating a string over three hundred thousand times…not surprising then that it would perform poorly.

Given that the event that I’m working on this demo for is tomorrow, I may have to table this for now and work on some simpler demos, but one way or another I am going to make this work!

One solution I’m noodling on is using the new string constructor that takes a char array, and index, and a length, and using that to write the string 74 characters at a time. If my assumptions are correct, that would drop the number of loops required down to around 4096, which should dramatically improve the execution time of the conversion.

Greatly appreciate the support and suggestions from the TinyCLR community!


#19

Gus,

What’s the name of the function for converting native NETMF to standard Bitmap?

Progress has been made. I now have the base64 conversion function running within a manageable amount of time (typically finishes in about 5-10 minutes…still slow, but tolerable for now).

Once the encoding is done, and the email sent (all of which works, yay!), the base64 string validates at the other end, but so far I’m not able to use it, I’m guessing because it’s in NETMF bitmap format.

Once I get the info on the conversion function, I will try to run that prior to encoding, and see if I end up with something usable at the other end.

Unfortunately, this only seems to work with the direct output from the camera (picture.PictureData), but not with the byte array received from a call to window.Graphics.GetBitmap().GetBitmap(), which always seems to cause an OutOfMemoryException. That may mean I’ll need to abandon my plan to do an image overlay, but I can still at least demo capturing the picture and converting it to base64, so it’s not a total loss.

Again, I appreciate all the help and advice!


#20

http://www.ghielectronics.com/downloads/NETMF/Library%20Documentation/html/aad3722f-e37f-762d-332c-86aeabc2d11a.htm