Sometimes EMX module doesn't boot - power supply rise time issue?

NETMF & Gadgeteer
1

I have a few EMX modules on production boards that sometimes don’t seen ti come out of boot mode on power up. Yes, running latest firmware.

Sometimes they just don’t start. Cycling power enough times and eventually the module starts.

One solution that consistently works on power up is to boot into serial terminal at 115200 baud, holding %, and then press R.

I wonder if this is related to the power supply rise time to the EMX module? What’s the spec on min/max power supply rise time for EMX module?

What else could be causing this where sometimes the EMX module doesn’t boot?

2

Are you using serial only? Where is MODE pin connected?

3

Are you sure that it is booting up in the bootloader mode, not in reset state? The module has a voltage supervision chip that puts the processor in reset state if the supply voltage is not enough or not stable.

Where are the up,down and selec pins connected to?

4

LMODE pin J2 is not connected to select COM1 for programming and debugging.
Up and Down pins are not connected, left open.
Should I use an active pull up on any of these rather than leave them open-circuit?

The symptom is that 30% of the time when power is applied, the EMX doesn’t boot up at all, the program doesn’t run, as determined from other expected activity.

What’s curious is that I can turn it on 10 times in a row and if holding down % in COM1, always see 100% of time:


Force Boot Loader detected
GHI loader commands are now active
BL
BL

And I can always successfully launch the application by pressing R at this point.

The problem is when I don’t hold down % and try to boot normally, as the customer would do, and find the EMX only boots properly 7 times out of 10.
showing this expected output:


EMX
Version: 4.1.8.0
Debug: COM1
LCD: 320x240
IP: 10.10.25.196
.NetMF v4.1.2821.0
EMX, Build Date:
        Dec 22 2011 16:27:20
ARM Compiler version 410561

TinyCLR (Build 4.1.2821.0)

Starting...
Created EE.
Started Hardware.
5

Connect COM and when it doesn’t boot what do you see on terminal and what do you see when it boots?

6

When EMX doesn’t boot 30% of time in subset of production units (not pressing %), nothing is displayed.

7

I started out asking GHI if there is a spec on the power supply rise time for the EMX module. Is there?

The rise time on the 3.3V supply on our boards from 0-3.3V is 2.4 milliseconds. It’s a clean rise without noise.

I mentioned that it is a subset of production units that demonstrate this “failure of EMX to boot” problem. There is no difference between the 3.3V rise time on the production units that boot OK 100% of the time and the units that occasionally don’t boot (but can be started with the %-R trick).

Is there a way to disable or bypass the EMX voltage supervisory circuit if we think that’s causing the problem?

8

There is a rise time but it is much longer than what you have. If you look at FEZ Cobra schematics, you can see how the simplest power supply can be used to power up EMX.

I think it is a floating pin causing the problem but I can’t be sure.

9

I know this is an old thread, but it has some useful information in it, so I’ll append rather than starting my own for now.
We seem to be having the same problem here, and I’ve used the information Balkanaut provided and attempted to characterise the problem further.

We’ve had several prototype and pre-production units out in test in Australian plants, and this week we were meant to send some off to plants in Canada and Brazil for evaluation, and hopefully start getting some orders in. However we are getting the same sort of failure to start problems, which is a real show-stopper.

Our system uses USB, so we are using COM1 for debugging, with the LPMODE link open. This seems to be an essential part of the problem. On power-up a system will usually boot correctly, but occasionally it won’t start. When this happens, neither repeated RESET or power-cycling will get it going. Whether we get a good boot or not seems a bit random, I don’t know if it is the 70% : 30% ratio Balkanaut reports, but it is certainly more likely than not to boot ok, but when it fails then it fails many times in a row.

Hopefully the problem has been resolved by now, if so I’d really like to hear about it. If not, here is what I have found.

Following on Balanaut’s good information, I’ve been running test after test while monitoring the COM1 port via TeraTerm.

When we get a good boot, we see:

Ready.
Estimated Network RAM usage: 129667 (bytes)
ip address from interface info: 192.168.1.200
mac addrress from interface info: 0.1a.f1.0.42.d
EMX
Version: 4.1.8.0
Debug: COM1
LCD: 320x240
IP: 192.168.1.200
MAC: 00.1A.F1.00.42.0D
Managed heap size: 13041664
Custom heap size: 1048576
.NetMF v4.1.2821.0
EMX, Build Date:
Dec 22 2011 16:27:20
ARM Compiler version 410561

TinyCLR (Build 4.1.2821.0)

Starting…
Created EE.
Started Hardware.

followed by some MSdbg info in binary or another baud rate, then info on loading of libraries and assemblies, including the application, followed by
Ready.
and the system starts. running.

When we get a bad boot, we get nothing at all. No diagnostics, no messages, nothing. Hit RESET, still nothing. Power off, batteries out. Power up again. Still nothing.

However, if I hold Up/Down/Select when I reset or power cycle, I get the GHI loader prompt, same if I enter the %%%%. And if I hold Up/Down, I get the TinyBooter.
From either of these, reset or power cycle still goes to the “locked up” state, nothing happening. But entering R from the GHI loader, or pressing Select from the TinyBooter, results in the application starting up!

From this I assume that the system is proceeding through the GHI Loader and TinyBooter, but no further. It isn’t going on to load the application, but it also isn’t giving the TinyBooter prompt. Something is causing it to lock in this state, something that isn’t cleared by reset or power-down.

That the application is still in a runnable state is shown by the fact that either GHI Loader or TinyBooter can run it when manually prompted. And after this is done, we are generally back to staring up normally each time, at least for a while.

Hopefully the above may be enough of a clue to diagnose the problem, if not let me know what more would help.

Gus, you mentioned you suspected a floating pin in Balkanaut’s case, which pin could be responsible for these symptoms?

Thanks for any help,

David

10

If you switch to COM debugging then COM1 RX pin can’t be left floating. If ti s floating, the system will lockup because of some garbage data coming on on the COM1 RX pins, which should be used for debugging, as this is what you selected.

Are you sure RX pin is connected properly?

11

I never leave an input open, so I wouldn’t leave that MODE pin, internal pullups are weak and therefore the pin can pickup noise. don’t say this is the issue here, but I’ve seen many systems lockup because of floating inputs.

12

Thanks Gus, that’s a biggie - looks like we missed the documentation on that one! So the module doesn’t provide for this, then…
What is the recommended value - 10k? Pull-down, or pull-up? I’ll get it patched and tested, and we can be on our way!
Beers all round!

13

10K is fine and I believe it is pull down (whichever the idle state for TTL UART happen to be)

14

Yes, I have noticed quite a bit of variation in the “weak pull-up” values between boards. They seem to work ok on the push-buttons. Do you know if there a specified range they can take?

15

Sorry Gus, it is after 2am here, and I’m a bit slow!
I went to check the state (idle is 3.3 V) and then I realized that all my tests were done with the com port attached, so the Rx line isn’t floating, it is attached to the Tx line at the PC, on the FTDI TTL-232R. I can add a 10k pull-up for when it is running standalone in the field, but that definitely isn’t the cause of what I’m seeing.

16

Hmm, I know this sounds silly, but where does the FTDI TTL-232R get it’s power from? The PC?

This can cause a problem where the EMX module doesn’t have power, but the comport is being powered via the RX line. This in turn can cause unpredictable behaviour in the EMX module.

I would say, connect the pullups, remove power from the FTDI TTL-232R and see if you still have the power-up problems…

17

I’ve now tested with Rx0 floating, pulled up to 3.3V (TTL-RS232 idle state) ,and pulled down to ground.
In each case we still have the fail-to-boot problem. With the 3.3V pull-up it still seems around the 3-in-7 starts, with the pull down it was less in the runs I’ve done (1 in 10) but due to the randomness of the event that may not be significant given the number of tests I’ve had time to do.

18

Gus, with the tests I’ve done now, hopefully there is enough information for you to locate the problem for us.
I get the failure to start no matter whether the COM1 port is floating, connected to the PC TTL-RS232 (and monitored with Teraterm) , with Rx tied high, or pulled low.
Whether it boots normally or not seems pretty random, but normal boot is much more likely than not.

Once it has gone into this fail-to-boot mode, I can repeatedly RESET the board, without it starting up. I can tie Rx0 to 3.3V, to 0V, leave it float, or connect to RS232, and press RESET, but it still has no effect.
I can even disconnect the power (remove the battery completely), connect it again and power up, but it still doesn’t start. So whatever the state it has gotten into is, it is retained through reset and power cycle. (The RTC battery is still connected)
[New test - removed RTC battery and main battery. Reconnect, power up. Still doesn’t start, so whatever is retained isn’t time or RTC-memory based.]

However, if I hold Up & Down and press reset, it then comes up in the TinyBooter, and if I then press Select it starts normally!

So, whatever is happening, it is locking the system up in the TinyBooter stage before it gets near loading and starting our application. Which lets our software guys off the hook, at least for now. Unless there is something in the software that could trip up the booting next time around? We use Extended Weak Reference and SD card storage, I can’t see that the SD could be at fault, and our EWR use is a one-time setup for system defaults (LED brightness, LCD bias and backlight level, etc) which doesn’t change betweens boots, so that seems unlikely too. Can you think of anything else the software could be doing that may cause this? With three people on software I don’t know all the ins and outs, anything specific I can check for would help.

Back to hardware, I can send a schematic if that would help. Could be immediately obvious to you - “Oh, you disconnected the LCD/Ethernet/USB host and didn’t tie line X low, thats your problem” would be nice to hear!

Thanks, David

19

[quote]However, if I hold Up & Down and press reset, it then comes up in the TinyBooter, and if I then press Select it starts normally!
[/quote]

Might also be interesting to add if you can then boot the EMX normally after this operation. I mean, does this operation temporary fix the problem or is this the only way to get it ever booting again.

20

Might also be interesting to add if you can then boot the EMX normally after this operation. I mean, does this operation temporary fix the problem or is this the only way to get it ever booting again.
[/quote]

Yes, it goes back to booting normally after this, at least until next time it fails to start again.

We’ve had a few of these operating out in the plant for a year or so now, and very few reports of problems, I think basically because in normal operation it won’t be turned off or reset. It hibernates pretty quickly when not actively used, so the Lithium Ion battery will hold out for a day’s work, then it is back on the charger. However we’re shipping one off to Canada for evaluation, and can’t have it powered up in transit. We needed to set it up for powering up & down for that and then this problem became much more apparent.