Help Crash GPIO43

jossvall

Hello community

I hope you are all doing well.

I'm reaching out to you regarding the following issue:

I currently have approximately 500 devices running OpenWRT version 18. We are working on updating to version 23.05, and I've already completed this process, generating the image with the custom programs.

During testing, I've encountered very strange behavior. Pin 43 (GPIO) is used to activate the PMIC of a modem.

When the modem boots with the system, everything works (while the system LED is blinking). However, if the modem needs to be restarted after the system boots, the Omega 2SP suddenly shuts down. There is no serial output indicating a power-off; everything simply shuts down, as if the power were disconnected internally.

This doesn't happen with version 18, only with version 23. Can anyone help me with this?

NOTE:

Perform the test using echo 1 > /sys/class/gpio/gpio43/value

If I type 0 there is no problem, but if I type 1 the card shuts down abruptly. However, this only happens when the system has booted and the system LED is on.

jossvall

Update on this issue.

I see that the problem isn't with pin 43. I've already tested pins 36 and 14, and the same thing happens. When I try to enable the modem's voltage regulator, the Omega turns off. Ironically, this doesn't happen with version 18. What did they break on the way to version 23?

luz

This sounds very much like a edge case in power supply to me. Enabling a voltage regulator for sure creates some additional, maybe quite sudden load on the overall system.

The thing that certainly has changed between 18.x and 23.x is the exact power consumption timing down to the microsecond and microampere, due to a quite different kernel with many internal architectural changes. So I can well imagine powering up that modem plus what was going on in 18 at the same time put just a tiny tiny (but deterministic) bit less overall load onto the power supply than now with 23 - just such that it triggers the reset now, while it did not before.

At least, I would (having >>1k o2 based devices in the field myself) first try to oscilloscope the power supply, and/or add some additional bulk capacitors in a lab unit and see if this eliminates the effect.

If it does - well then you need to find a way to time that critical modem power-on differently (maybe you can temporarily switch off something else before starting the modem), unless it's feasible for you to add bulk caps in the field - I guess it's not, with 500 units…

Of course, it's only a guess - but using 23.05 for years now, I am not aware of any problem with gpios having side effects.

@jossvall said in Help Crash GPIO43:

However, this only happens when the system has booted and the system LED is on.

I realize I don't understand this statement - how else could it happen when the system is not up? Directly changing gpio registers from uboot?

jossvall

@luz Hi, thank you so much for your help.

I'll run tests with an oscilloscope to analyze the voltage drops.

The scenario in detail is this:

Normally, the modem starts automatically. I have a script in "/etc/init.d/modem" that runs when the system is loading and starting programs. If it does this at this point, there's no problem; the modem is powered on and then starts.

But if the startup occurs after the operating system has loaded everything, that is, after:

Fri Mar 22 22:11:21 2024 daemon.info procd: - init complete -

simply setting the regulator's power pin high shuts down the modem. I recently discovered that this only happens if the modem is connected; if the modem is disconnected, this sudden shutdown doesn't occur. Furthermore, the modem has a separate regulator, which makes it even stranger to me. I don't understand why it works with version 18 but not with version 23, even though it has the same PCB.

jossvall

Update on this case:

After verifying with the oscilloscope, I found a voltage drop of 250µs, which explains why the Omega shuts down, since the drop reaches 2V and the Onion regulator cannot raise the voltage.

This makes it clear that it is not a PIN error but a problem in the regulation stage.

Now another question arises: why does it restart with version 18, but with version 23 the board remains off? Is there a parameter that allows the Omega to restart in case of a voltage drop or to shut down?

Thanks to colleague @luz for your help