From time to time, devices in the field get into a state that the I2C no longer functions.
- A hard RESET will not fix the problem.
- Power cycle will sometimes fix the problem.
- Sometimes Erase All will fix the problem
- Sometimes NOTHING will fix the problem !!
I currently have a SC13048 device that was working fine for a long time, but now the I2C within the firmware is corrupt and I cannot seem to reset it.
Here’s the code from the screen shot:
using GHIElectronics.TinyCLR.Devices.I2c;
using GHIElectronics.TinyCLR.Pins;
using System;
using System.Collections;
using System.Diagnostics;
using System.Text;
using System.Threading;
namespace Minimal_I2C
{
internal class Program
{
static void Main()
{
var settings = new I2cConnectionSettings(0x49, 100_000); //The slave's address and the bus speed.
var controller = I2cController.FromName(SC13048.I2cBus.I2c1);
var device = controller.GetDevice(settings);
device.Write(new byte[] { 0x02, 0x08 + 0x04 + 0x01 });
}
}
}
The Assemblies on the device:
And the Firmware is 2.2.1.2000
How do I Reset this I2C interface within GHI TinyCLR so that it starts working again ?
I’ve tried
- RESET
- Power On/Off
- Erase All and reload firmware
Help ! We Have to get this device up and running again.
We just double checked I2C on SC13, version 2.2.1.200, it works as expected.
Make sure you have slave device connected well.
Thanks for the quick reply Dat.
Slave devices are connected and working.
We can connect a different processor to the I2C bus and it works fine.
This seems to be a common problem with I2C on the SC13048. Happens with Pico’s and our embedded devices. Usually a power cycle will reset the firmware. But, sometimes not !
How do I clear all settings in the device and reset the I2C bus ???
We have tried Erase All and flashing new firmware.
No, there is nothing to do with Erase all or flashing new firmware.
From your code, I think SC13xxx is master mode.
In master mode, you can use “i2cDevice.Dispose()”, this will force I2C reset;
For hard reset, just reset the SC13xx module.
But I think problem is your slave device. Reset on slave device is different with Reset on master.
Usually slave device has reset pin, or enable pin… we need schematic or data sheet to see how reset the slave device.
When slave enter unexpected state, master send anything, slave doesn’t response, I guess. You need way to reset slave device, try disconnect, connect power from slave, it will help.
That also explain “sometime fixed the problem”.
- A hard RESET will not fix the problem.
- Power cycle will sometimes fix the problem.
- Sometimes Erase All will fix the problem
- Sometimes NOTHING will fix the problem !!
if your slave reset, it work, I guess.
Dat,
Here’s more information …
When the I2C bus gets into this “broken state”, the processor only sends out a single pulse on the SCL line.
Error thrown :
#### Exception System.InvalidOperationException - 0x00000000 (1) ####
#### Message:
#### GHIElectronics.TinyCLR.Devices.I2c.I2cDevice::WriteRead [IP: 0036] ####
#### GHIElectronics.TinyCLR.Devices.I2c.I2cDevice::Write [IP: 000c] ####
#### Minimal_I2C.Program::DoWrite [IP: 0023] ####
#### Minimal_I2C.Program::Main [IP: 003a] ####
Exception thrown: ‘System.InvalidOperationException’ in GHIElectronics.TinyCLR.Devices.I2c.dll
Disconnect slave device and try to write, there is exception of course, try catch keep writing
Also, is there pullup on i2c scl and sda?
what do you see?
Dat,
Unfortunately, on this board it is impossible to disconnect the slaves.
But, the slaves do respond if we poll them with a different CPU manually connected to the I2C bus.
And yes, there are pull-up resistors on the I2C lines:
Is there any way I can check the I2C registers directly via Marshall ?
Yes, I2C registers are opened for user to check. But I think we are wrong way if we think reset I2C by Erasing / flashing firmware does help.
There is SoftwareI2C that you can try.
Any example code for reading the I2C registers via Marshall?
I looked at the online docs. Not much there on accessing I2C via Marshall.
Thanks for your help.
I think here it is Marshal (ghielectronics.com)
You need to pick what register you want to check.
But there is SoftwareI2C you can try quick to know if I2C master’s problem or not.
All you need just replace:
var controllers = I2cController.FromName(SC20260.I2cBus.Software, sda, scl, true); // true if need internall pulse up. You already external pullup, I think itt needs to be false.
i2cDevice = controllers.GetDevice(setting);
The rest should be same.
Dat,
Yep ! The bit-banging version worked just fine.
Very cool way to diagnose the problem.
So … now we know for sure that the slaves are responding.
We just need to figure out how to reset the TinyCLR’s I2C handler !!
Don
What is the big-banging version? do you mean softwareI2c?
Yes, the softwareI2c interface spoke with the slave devices just fine. The hardware is working. We have proven the problem to be within the TinyCLR firmware. Once it corrupts the I2C interface, it is difficult for it to recover. And on some devices, it can never recover.
I think we need to escalate this issue. We have seen this problem manifest itself many times. We cannot go into production with this I2C bug potentially destroying thousands of devices in the field. We cannot move forward with this commonly occurring I2C driver corruption issue outstanding.
A LITTLE MORE INFORMATION: our engineers have not seen this I2C issue on SC20xxx devices, just the SC1348Q. We saw it many times during development with Pico’s. Now with final pre-production devices, the I2C bus gets corrupted three to four times a day.
What slave device are you using? If you can give us something that we can reproduce the issue here, then we are willing to help more.
What I can suggest now is, SoftwareI2C has slow frequency, under 10KHz I think.
The slave maybe somehow is not happy at ~100KHz, try slower the lock to 5-10KHz to see if the slave response.
This board has 4 I2C slave devices:
-
Texas Instruments BQ25628
Battery charger / power controller
-
Texas Instruments TCA9537
Used as an interrupt aggregator
-
ST Microelectronics IIS2MDC
Magnetometer
-
ST Microelectronics LIS2W12
Accelerometer
When the I2C bus stops working, it will not communicate with any of the four slaves. The I2C bus outputs a single pulse on the SCL line and then generates an error on any read or write. A power-cycle will usually reset the I2C bus and it will then start working again. Unfortunately, these boards will NEVER get a power cycle in the field. They are connected and running without any human intervention for years. A CPU RESET will never fix the issue. Sometimes a power-cycle will not fix the I2C bus.
Error thrown :
#### Exception System.InvalidOperationException - 0x00000000 (1) ####
#### Message:
#### GHIElectronics.TinyCLR.Devices.I2c.I2cDevice::WriteRead [IP: 0036] ####
#### GHIElectronics.TinyCLR.Devices.I2c.I2cDevice::Write [IP: 000c] ####
#### Minimal_I2C.Program::DoWrite [IP: 0023] ####
#### Minimal_I2C.Program::Main [IP: 003a] ####
Exception thrown: ‘System.InvalidOperationException’ in GHIElectronics.TinyCLR.Devices.I2c.dll
The very simple code I posted in the first post in this thread demonstrates the problem once the I2C bus is corrupt. But of course, this code snippet does not cause the problem. The problem only manifests itself when the device is busy doing a lot of tasks. We originally saw this I2C issue last year when doing prototype development on a Pico. We have tried v2.2.1.2000 and v2.2.0.7000. Both firmware versions have the I2C issue.
This unstable I2C bus and not being able to clear/reset the problem when it happens is a very serious issue for this project. We cannot move forward with production. Switching to the software I2C is not a feasible option.
What do you mean CPU reset? I mean how do you do it?
CPU RESET = press the RESET button on the board.
These a pre-production prototype boards. They have two buttons to assists in development / testing. The final production devices will not have these two buttons.
Is there any way we have one prototype to test the issue?
I2C is not hard to fix, if we see the issue, of course.
It is weird to me that after reset, your slaves still don’t response.
and what version it work before?
We can’t be sure that the slave is not the one locking up and not the master. Do you have an i2c analyzer?
1 Like
