Skip to content

fix: skip the profile restore when the EC is wedged on resume - #26

Open
little-schlapps wants to merge 1 commit into
dahui:mainfrom
little-schlapps:fix/skip-restore-on-wedged-ec
Open

little-schlapps wants to merge 1 commit into
dahui:mainfrom
little-schlapps:fix/skip-restore-on-wedged-ec

Conversation

@little-schlapps

Copy link
Copy Markdown
Contributor

The problem

waitForEC (added in #20) returns true on timeout and lets the restore proceed anyway:

if time.Now().After(deadline) {
    slog.Warn("EC still unresponsive after resume; restoring anyway", "waited", timeout)
    return true
}

The comment explains why — a machine with no battery reports the same way as a wedged EC, and must still get its profile back. But that escape hatch is the remaining path to the hard lock the wait was added to prevent. applyCustomHW writes PPT through asus_wmi, the write blocks inside acpi_evaluate_object holding the ACPI global mutex, and every other ACPI consumer piles up behind it.

What I saw

A GZ302EA (BIOS 308, kernel 7.0.0) came back from a 2.5 hour s2idle suspend with the EC down:

atkbd serio0: Failed to enable keyboard on isa0060/serio0
ucsi_acpi USBC000:00: failed to re-enable notifications (-110)
cat: /sys/class/power_supply/BAT0/energy_now: No such device

The daemon waited, gave up, and went ahead:

14:00:05 z13ctl WARN EC still unresponsive after resume; restoring anyway waited=20s
14:00:58 z13ctl INFO resume: lighting restored
14:00:58 z13ctl INFO resume: restoring custom profile profile=custom   ← never returned

Then:

14:04:51 INFO: task z13ctl:11215 blocked for more than 120 seconds.
           → asus_wmi_evaluate_method3 → ppt_pl2_sppt_store
         INFO: task z13ctl:11215 blocked on a semaphore likely last held by tokio-rt-worker:4140
           → acpi_os_read_port  RDI: 0x257  RAX: 0xff      (asusd, holding the mutex)
14:07:48 watchdog: CPU2: Watchdog detected hard LOCKUP on cpu 2

Soft lockups spread to ~20 of 32 cores over four minutes. Hard reset required.

The fix

The two cases are separable. FindBatteryCapacityPath globs BAT*/capacity and falls back to a BAT0 path that will not exist, so:

  • no battery → glob misses → read fails ENOENT
  • wedged EC → glob hits, path exists → read fails ENODEV

ecResponds() bool becomes probeEC() ecStatus over ecReady / ecAbsent / ecWedged, and waitForECWith returns what the last probe actually saw. ecAbsent still restores; ecWedged does not.

Only applyCustomHW is gated, not the whole restore:

  • lighting goes over hidraw and never touches the EC — in the trace above it restored fine
  • restoreVolatileState is what clears the suspending flag, so skipping it wholesale would stand the reconcile watcher down until its staleness ceiling expired
  • the curve and TDP floor left behind are what that watcher already puts back once the EC recovers, so the cost is a delay rather than a setting lost for the session

classifyECRead is split out from probeEC so the ENOENT/ENODEV distinction is testable without redirecting sysfs, which internal/daemon cannot do because cli's path vars are unexported.

Tests

  • TestClassifyECReadSeparatesAbsentFromWedged — table over nil / ENOENT / wrapped ENOENT / ENODEV / ETIMEDOUT
  • TestWaitForECReportsAWedgedECOnTimeout — new, the regression above
  • TestWaitForECRestoresAnywayOnTimeoutTestWaitForECRestoresAnywayWhenThereIsNoBattery, narrowed to the batteryless case it was actually protecting
  • existing wait/cancellation coverage updated for the new signature

go test ./... and the api module pass; gofmt and go vet clean. I don't have golangci-lint to hand, so that check is unrun.

Possibly #23

#23 reports a hard lock on resume still happening on v1.3.2 — the release that carries the #20 wait. This escape hatch is the obvious candidate, but that issue has no logs attached, so I can't confirm it's the same path. Deliberately not using a closing keyword. The line that would settle it is EC still unresponsive after resume; restoring anyway in journalctl --user -u z13ctl from the resume that locked up; I'll ask there.

Note this also applies to feat/v2-refactorwaitForECWith is byte-identical on that branch, escape hatch included.

🤖 Generated with Claude Code

waitForEC returned true on timeout and let the restore proceed regardless,
because a machine with no battery reports the same way as an EC that is not
answering. That escape hatch is the remaining path to the hard lock the wait
was added to prevent: applyCustomHW writes PPT through asus_wmi, the write
blocks inside acpi_evaluate_object holding the ACPI global mutex, and every
other ACPI consumer piles up behind it until the NMI watchdog reports lockups
across every core.

Observed on a GZ302EA after a 2.5 hour s2idle suspend that came back with the
EC down (atkbd re-enable failed, ucsi_acpi re-enable timed out with -110,
BAT0/energy_now read ENODEV). The daemon logged "EC still unresponsive after
resume; restoring anyway" and hung in ppt_pl2_sppt_store; asusd was caught
holding the mutex spinning in acpi_os_read_port. Soft lockups spread to about
20 of 32 cores over the next four minutes and the machine needed a hard reset.

The two cases are separable. FindBatteryCapacityPath globs BAT*/capacity and
falls back to a BAT0 path that will not exist, so a machine with no battery
fails the read with ENOENT while a wedged EC fails an existing path with
ENODEV. Replace the bool probe with a three-state ecStatus and return what the
last probe saw instead of a blanket "go ahead": ecAbsent still restores,
ecWedged does not.

Gate only applyCustomHW rather than the whole restore. Lighting goes over
hidraw and never touches the EC, and restoreVolatileState is what clears the
suspending flag — skipping it wholesale would stand the reconcile watcher down
until its staleness ceiling expired. The fan curve and TDP floor the skip
leaves behind are what that watcher already puts back once the EC recovers, so
the cost is a delay rather than a setting lost for the session.

classifyECRead is split out from probeEC so the ENOENT/ENODEV distinction can
be tested without redirecting sysfs, which internal/daemon cannot do because
cli's path vars are unexported.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant