After working in an intermittent hang problem for several months I’m concluding that the only fix will be an external watchdog, but I’d like to see if anyone in the community has experienced a similar problem or has a suggestion.
The environmental details are Panda II, SDK 4.1.5.1, and Visual Studio 2012 and 2010. While the Panda II is discontinued the fix for the hang may also be of use for other processors.
The problem is that 999 out of 1000 times the code runs just fine with the system going to hibernate for one second and then waking up to take data if necessary. However, sometimes it hangs and I’ve isolated the hang to a hibernate and RTC alarm statement. I’m pretty certain of this because just before that statement I toggle a pair of leds and then toggle them back right after the statement. When the hang occurs the board led (bled) is on and the other led (led) is off. That is the only place in the data collection loop where those two leds are written to. The pertinent code segment is in this post.
I’ve tried using the internal watchdog to reset the processor, an NXP LPC2387, but it does not cause a reset. Even with the watchdog feed or reset right after the hibernate/alarm statement the watchdog does not reset the system probably because the LPC2387 RTC is corrupted somehow.
There is a note in the NXP errata for the LPC2387 that Vbatt can’t be left floating or it may corrupt the RTC which of course could affect the wake up alarm. However, tying Vbatt to the 3.3 volt supply as recommended did not help.
My guess is that the root cause of the hang is a periodic thermal shutdown of the LM1117 5 volt regulator when I stress the system by taking nonstop data cycles. Normally this isn’t a problem because the system spends most of its time in hibernation. With a 12 volt supply however and constantly drawing 120 ma there isn’t enought of a heat sink on the LM1117 according to the data sheet. If the shutdown kills power when the systems is in hibernation itprobably corrupts the RTC and no wakeup gets issued.
Finally, I should mention that timestamping for any data collected uses a seperate RTC, a very accurate DS3231 that is on the I2C bus, but the Panda’s internal RTC is still used for the alarm since that is the alarm needed to wake from hibernation. There is essentially no connection between the two clocks and the internal RTC does not use a backup battery since It is only used when the system is powered. The DS3231 keeps the reference time with its own backup battery.
As a workaround I’m planning on either using the DS3231 alarms as the wake up interrupt or I could also use the DS3231 as an external watchdog. Afew years ago EriSan500 (Eric) wrote some nice code for the 1307 RTC that I’ve modified a bit for the DS3231, but it didn’t address the alarm registers. I can add that functionality, but if anyone has any time saving suggestions I’m all ears (no Yoda pun intended)
Thanks in advance for any help.
GHIElectronics.NETMF.Hardware.RealTimeClock.SetAlarm(RealTimeClock.GetTime().AddSeconds(1));
led.Write(false);
bled.Write(true);
GHIElectronics.NETMF.Hardware.LowLevel.Power.Hibernate(Power.WakeUpInterrupt.RTCAlarm);
// When system hangs it never gets to the next line - led is false and bled is true
GHIElectronics.NETMF.Hardware.LowLevel.Watchdog.ResetCounter();
led.Write(true);
bled.Write(false);
adc(ai, ref k, ref i, ref val, ai_vals, out val1, out val2, out vmax);