The console waits for the host instead of failing its first enable #435

Merged
enricobuehler merged 1 commits from worktree-web-console-start-race into main 2026-08-28 21:46:35 +00:00
Owner

On a fresh install the web console's first start always failed at the systemd level:

punktfunk-web.service: Failed to load environment files: No such file or directory
punktfunk-web.service: Failed to spawn 'start' task: No such file or directory
Failed with result 'resources'.

punktfunk-web.service declares After=punktfunk-host.service, but that was never a readiness gate: the host is Type=simple, so systemd considers it started the instant it is spawned — seconds before it writes anything. The console's mandatory EnvironmentFile=%h/.config/punktfunk/mgmt-token then read a file that did not exist yet.

StartLimitIntervalSec=0 meant the restart two seconds later succeeded, so the console did come up. But the first enable printed a hard failure on a perfectly good install, and every caller that watches the exit status believed it.

Field report 2026-08-28 (Omarchy). punktfunk-omarchy setup enables the host and the console back to back, so it lost that race every single time and reported Failed to start punktfunk management web console on an install where pacman, the repo and all three packages were fine.

The fix

The gate now lives in punktfunk-web-init.service, which the console already orders after (After= + Wants=) and which is Type=oneshot — so blocking there is the readiness gate the ordering already claimed to be. No new unit, no new directive, no change to the host's startup path.

It waits for the console's own precondition rather than a proxy for it, and that distinction is load-bearing. The two files it needs are written far apart:

  • mgmt-token goes out early in serve (main.rs, before the listeners);
  • the identity cert comes last, inside mgmt::runidentity::load_or_adopt.

So a gate that waited on the token alone would only have moved the failure to the cert — which is the half Bun.serve needs to listen at all. The check mirrors web/nitro-entry/tls-paths.mjs exactly: a token, plus a non-empty cert/key pair from one directory, native or legacy (a host that never took the identity split writes no native-*.pem, and waiting for one would stall it forever).

Timing out is not fatal. It exits 0 so the console still starts and the existing Restart backstop takes over, but it says why — where the bare systemd error it replaces named a missing file and never the host that owed it.

Also here

ConditionPathExists=!%h/.config/punktfunk/web-password is dropped from web-init. That skipped the unit from the second boot onward, which is precisely when the wait must still run, and it tied a host-readiness gate to the presence of an unrelated password file. web-init.sh is idempotent instead, and in steady state — every start after the host's first run — it returns without sleeping.

Mirrored into the NixOS module, and module-check.nix now asserts the absence of the path condition it used to assert the presence of. All four Linux packages ship the same two files (deb, RPM, Arch, and Bazzite via the Fedora RPM), so the fix reaches every one of them from here.

Test

check-docs-drift.sh gate 8 runs the real script against a faked config dir:

  • waits when nothing is there;
  • still waits when only the token is there — the leg a token-only fix would have missed;
  • returns instantly for both the native and the legacy identity;
  • stops waiting as soon as the files land, rather than sleeping out its budget;
  • never exits non-zero, so it can never fail the unit itself.

Each case was proved non-vacuous by reverting the fix and watching it go red. The full suite passes under both sh and dash (what the CI container uses), and both edited shell scripts parse under dash.

Not done

Type=notify + sd_notify on the host is the textbook readiness protocol and would cover this class outright. It locks the unit file to the binary, and the documented install route copies punktfunk-host.service into ~/.config/systemd/user by hand — a new unit beside an older host would hang for TimeoutStartSec, then fail to start at all. Marked with a ponytail: comment at the poll naming that upgrade path.

Untouched: the kms_swrast fallback and the NVENC-only encode note from the same field report. Both real, both separate, neither is why setup reported a failure.

On a fresh install the web console's **first** start always failed at the systemd level: ``` punktfunk-web.service: Failed to load environment files: No such file or directory punktfunk-web.service: Failed to spawn 'start' task: No such file or directory Failed with result 'resources'. ``` `punktfunk-web.service` declares `After=punktfunk-host.service`, but that was never a readiness gate: the host is `Type=simple`, so systemd considers it started the instant it is **spawned** — seconds before it writes anything. The console's mandatory `EnvironmentFile=%h/.config/punktfunk/mgmt-token` then read a file that did not exist yet. `StartLimitIntervalSec=0` meant the restart two seconds later succeeded, so the console did come up. But the first enable printed a hard failure on a perfectly good install, and every caller that watches the exit status believed it. **Field report 2026-08-28 (Omarchy).** `punktfunk-omarchy setup` enables the host and the console back to back, so it lost that race every single time and reported `Failed to start punktfunk management web console` on an install where pacman, the repo and all three packages were fine. ## The fix The gate now lives in `punktfunk-web-init.service`, which the console already orders after (`After=` + `Wants=`) and which is `Type=oneshot` — so blocking there is the readiness gate the ordering already claimed to be. No new unit, no new directive, no change to the host's startup path. It waits for the console's **own** precondition rather than a proxy for it, and that distinction is load-bearing. The two files it needs are written far apart: * `mgmt-token` goes out early in `serve` (`main.rs`, before the listeners); * the identity cert comes **last**, inside `mgmt::run` → `identity::load_or_adopt`. So a gate that waited on the token alone would only have moved the failure to the cert — which is the half `Bun.serve` needs to listen at all. The check mirrors `web/nitro-entry/tls-paths.mjs` exactly: a token, plus a non-empty cert/key pair from one directory, native **or** legacy (a host that never took the identity split writes no `native-*.pem`, and waiting for one would stall it forever). Timing out is not fatal. It exits 0 so the console still starts and the existing `Restart` backstop takes over, but it says **why** — where the bare systemd error it replaces named a missing file and never the host that owed it. ## Also here `ConditionPathExists=!%h/.config/punktfunk/web-password` is dropped from web-init. That skipped the unit from the second boot onward, which is precisely when the wait must still run, and it tied a host-readiness gate to the presence of an unrelated password file. `web-init.sh` is idempotent instead, and in steady state — every start after the host's first run — it returns without sleeping. Mirrored into the NixOS module, and `module-check.nix` now asserts the *absence* of the path condition it used to assert the presence of. All four Linux packages ship the same two files (deb, RPM, Arch, and Bazzite via the Fedora RPM), so the fix reaches every one of them from here. ## Test `check-docs-drift.sh` gate 8 runs the real script against a faked config dir: * waits when nothing is there; * **still waits when only the token is there** — the leg a token-only fix would have missed; * returns instantly for both the native and the legacy identity; * stops waiting as soon as the files land, rather than sleeping out its budget; * never exits non-zero, so it can never fail the unit itself. Each case was proved non-vacuous by reverting the fix and watching it go red. The full suite passes under both `sh` and `dash` (what the CI container uses), and both edited shell scripts parse under `dash`. ## Not done `Type=notify` + `sd_notify` on the host is the textbook readiness protocol and would cover this class outright. It locks the unit file to the binary, and the documented install route copies `punktfunk-host.service` into `~/.config/systemd/user` **by hand** — a new unit beside an older host would hang for `TimeoutStartSec`, then fail to start at all. Marked with a `ponytail:` comment at the poll naming that upgrade path. Untouched: the `kms_swrast` fallback and the NVENC-only encode note from the same field report. Both real, both separate, neither is why setup reported a failure.
enricobuehler added 1 commit 2026-08-28 21:24:18 +00:00
The console waits for the host instead of failing its first enable
ci / bun-nix (pull_request) Successful in 30s
ci / docs-drift (pull_request) Successful in 34s
ci / docs-site (pull_request) Successful in 1m36s
ci / web (pull_request) Successful in 1m51s
apple / swift (pull_request) Successful in 2m9s
apple / distribute (pull_request) Skipped
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 2m44s
android / android (pull_request) Successful in 6m46s
ci / rust (pull_request) Successful in 7m24s
nix / flake (pull_request) Successful in 8m27s
9620c71b1f
On a fresh install the web console's FIRST start always failed at the systemd
level:

    punktfunk-web.service: Failed to load environment files: No such file or directory
    punktfunk-web.service: Failed to spawn 'start' task: No such file or directory
    Failed with result 'resources'.

`punktfunk-web.service` declares `After=punktfunk-host.service`, but that was
never a readiness gate: the host is `Type=simple`, so systemd considers it
started the instant it is SPAWNED — seconds before it writes anything. The
console's mandatory `EnvironmentFile=%h/.config/punktfunk/mgmt-token` then read
a file that did not exist yet. `StartLimitIntervalSec=0` meant the restart two
seconds later succeeded, so the console did come up — but the first enable
printed a hard failure on a perfectly good install, and every caller that
watches the exit status believed it.

Field report 2026-08-28 (Omarchy): `punktfunk-omarchy setup` enables the host
and the console back to back, so it lost that race every single time and
reported "Failed to start punktfunk management web console" on an install where
pacman, the repo and all three packages were fine.

The gate now lives in punktfunk-web-init.service, which the console already
orders after (`After=` + `Wants=`) and which is `Type=oneshot` — so blocking
there is the readiness gate the ordering already claimed to be. No new unit, no
new directive, no change to the host's startup path.

It waits for the console's OWN precondition rather than a proxy for it, which
matters because the two files it needs are written far apart: `mgmt-token` goes
out early in `serve` (main.rs, before the listeners) while the identity cert
comes LAST, inside `mgmt::run` -> `identity::load_or_adopt`. Waiting on the
token alone would only have moved the failure to the cert. The check mirrors
web/nitro-entry/tls-paths.mjs exactly: a token, plus a non-empty cert/key pair
from one directory — native, or legacy for a host that never took the identity
split.

Also drops `ConditionPathExists=!%h/.config/punktfunk/web-password` from
web-init. That skipped the unit from the second boot onward, which is precisely
when the wait must still run, and it tied a host-readiness gate to the presence
of an unrelated password file. web-init.sh is idempotent instead, and in steady
state (every start after the host's first run) it returns without sleeping.

Timing out is not fatal — it exits 0 so the console still starts and the
existing Restart backstop takes over, but says WHY, where the bare systemd error
it replaces named a missing file and never the host that owed it.

Mirrored into the NixOS module, and module-check.nix now asserts the absence of
the path condition it used to assert the presence of. All four Linux packages
(deb, RPM, Arch, and Bazzite via the RPM) ship the same two files, so the fix
reaches every one of them from here.

check-docs-drift.sh gate 8 runs the real script against a faked config dir:
it waits when nothing is there, still waits when only the token is there (the
leg a token-only fix would have missed), returns instantly for both the native
and the legacy identity, and stops waiting as soon as the files land. Each case
proved non-vacuous by reverting the fix and watching it go red.
enricobuehler merged commit e12f3a62de into main 2026-08-28 21:46:35 +00:00
enricobuehler deleted branch worktree-web-console-start-race 2026-08-28 21:46:47 +00:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unom/punktfunk#435