An in-flight TV restore raced a reconnect's takeover and resurrected sddm under the mask #246

2026-08-15T10:50:28Z

enricobuehler commented

2026-08-15 10:50:28 +00:00

Field incident (.41 VM, 2026-08-15): under keep_alive=off the restore debounce is 0 s, so the restore worker pops the deadline before a reconnect's cancel arrives — clearing PENDING_RESTORE then cancels nothing, and do_restore_tv_session runs concurrently with the new session's takeover. The takeover found the autologin unit inactive (the restore hadn't restarted it yet), so dm_plan correctly refused to stop the DM for a dead unit and went mask-only — and the in-flight restore then restarted sddm underneath that mask at 10:36:39.638. On Bazzite, sddm's helper execs the session script directly, so the mask stops only the final unit start, never the retry: 1,272 relogins in ~4 minutes, a fresh Steam launch each round, ~170k PIDs, load 7+, until a manual systemctl stop sddm.

The fix serializes the two with a RESTORE_FLIGHT mutex: the worker pops the deadline and runs the restore under it, and cancel_pending_restore (which create_managed_session now routes through instead of a bare PENDING_RESTORE clear) blocks until an in-flight restore completes. Either the cancel wins — no restore starts, warm reuse exactly as before — or the restore wins and the connect takes over a fully restored box, where the autologin unit is live again and the DM stop engages through the ordinary plan. restore_takeover_now (host shutdown) takes the same lock so the two restore entry points can't interleave either. Lock order: RESTORE_FLIGHT is outermost, taken only at those three entry points.

Regression test: reconnect_cancel_waits_out_an_in_flight_restore pins the property (a cancel must not race past a running restore, and must complete once it releases).

Gate: rebased onto current main (post-#245); xcheck linux check+clippy clean; pf-vdisplay 237/237 in the rust:1.96 container.

Field incident (.41 VM, 2026-08-15): under `keep_alive=off` the restore debounce is 0 s, so the restore worker pops the deadline before a reconnect's cancel arrives — clearing `PENDING_RESTORE` then cancels nothing, and `do_restore_tv_session` runs **concurrently** with the new session's takeover. The takeover found the autologin unit inactive (the restore hadn't restarted it yet), so `dm_plan` correctly refused to stop the DM for a dead unit and went mask-only — and the in-flight restore then restarted sddm **underneath that mask** at 10:36:39.638. On Bazzite, sddm's helper execs the session script directly, so the mask stops only the final unit start, never the retry: **1,272 relogins in ~4 minutes**, a fresh Steam launch each round, ~170k PIDs, load 7+, until a manual `systemctl stop sddm`. The fix serializes the two with a `RESTORE_FLIGHT` mutex: the worker pops the deadline **and** runs the restore under it, and `cancel_pending_restore` (which `create_managed_session` now routes through instead of a bare `PENDING_RESTORE` clear) blocks until an in-flight restore completes. Either the cancel wins — no restore starts, warm reuse exactly as before — or the restore wins and the connect takes over a fully restored box, where the autologin unit is live again and the DM stop engages through the ordinary plan. `restore_takeover_now` (host shutdown) takes the same lock so the two restore entry points can't interleave either. Lock order: `RESTORE_FLIGHT` is outermost, taken only at those three entry points. Regression test: `reconnect_cancel_waits_out_an_in_flight_restore` pins the property (a cancel must not race past a running restore, and must complete once it releases). Gate: rebased onto current main (post-#245); xcheck linux check+clippy clean; pf-vdisplay **237/237** in the rust:1.96 container.

enricobuehler added 1 commit 2026-08-15 10:50:30 +00:00

fix(vdisplay): an in-flight TV restore raced a reconnect's takeover and resurrected sddm under the mask

ci / docs-site (pull_request) Successful in 1m18s

Details

ci / bun-nix (pull_request) Successful in 1m31s

Details

ci / web (pull_request) Successful in 5m11s

Details

android / android (pull_request) Successful in 5m18s

Details

ci / rust-arm64 (pull_request) Successful in 7m34s

Details

ci / rust (pull_request) Failing after 18m40s

Details

054bf26393

Under keep_alive=off the restore debounce is 0 s, so the worker pops the
deadline before a reconnect's cancel arrives — clearing PENDING_RESTORE
then cancels nothing, and the restore runs concurrently with the new
takeover. Field trace (.41, 2026-08-15): the new takeover listed the
autologin unit inactive (the restore hadn't restarted it yet), so
dm_plan correctly refused to stop the DM for a dead unit and went
mask-only — and the in-flight restore then restarted sddm underneath
that mask at 10:36:39.638. SDDM's helper execs the session script
directly, so the mask stops only the final unit start, never the retry:
1272 relogins in ~4 minutes, a fresh Steam launch each round, load 7+,
until a manual 'systemctl stop sddm'.

RESTORE_FLIGHT now serializes the two: the worker pops the deadline and
runs do_restore_tv_session under it, and cancel_pending_restore (which
create_managed_session now routes through instead of clearing
PENDING_RESTORE bare) blocks until an in-flight restore completes. Either
the cancel wins — no restore starts, warm reuse as before — or the
restore wins and the connect takes over a fully restored box, where the
autologin unit is live again and the DM stop engages through the
ordinary plan. restore_takeover_now (host shutdown) takes the same lock
so the two restore entry points can't interleave either.

Gate: xcheck linux check+clippy clean; pf-vdisplay 237/237 in the
rust:1.96 container, including the new regression test.

enricobuehler merged commit 04b6f84fc3 into main

2026-08-15 10:57:05 +00:00

enricobuehler deleted branch worktree-dm-restore-reconnect-race

2026-08-15 10:57:07 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unom/punktfunk#246