fix(ci/nix): the flake job could never start its container
ci / web (pull_request) Successful in 1m7s
ci / docs-site (pull_request) Successful in 1m11s
ci / rust-arm64 (pull_request) Successful in 1m18s
ci / bun-nix (pull_request) Successful in 16s
ci / rust (pull_request) Failing after 6m12s
nix / flake (pull_request) Failing after 4m54s

The nix gate landed in fb707b49 on `nixos/nix:latest`, and it has never run a
single step. That image carries nix and essentially nothing else — including no
`/bin/sleep` — and Gitea's act_runner starts every job container with
`entrypoint=["/bin/sleep", "10800"]`:

  failed to create shim task: OCI runtime create failed: runc create failed:
  unable to start container process: exec: "/bin/sleep":
  stat /bin/sleep: no such file or directory

The expensive part is how it reports: with the container dead, every step is
marked `cancelled` rather than `failed`, which reads exactly like a run that was
superseded by a newer push. Run 15907 looked skipped, not broken.

Switch to `node:22-bookworm` and install Nix in a step:

  * full Debian, so the entrypoint exists and coreutils are present;
  * a real node, so `actions/checkout` works with no pre-checkout install dance
    (the reason flatpak.yml's fedora job installs node before its checkout);
  * audit.yml already pulls this image on this fleet, so it is known to resolve.

Nix comes from the Determinate installer with `--init none` — the container mode:
no systemd, no daemon. That distribution is also what the hand-verification Nix
box (.21) runs, so CI and it stay on the same Nix.

MEASURED in a real `node:22-bookworm` container rather than assumed, since the
last version of this file shipped on an untested assumption and cost a red run:

  * `/bin/sleep` present — the entrypoint failure is gone;
  * the installer completes and `nix --version` runs from the absolute path;
  * `nix build` of a trivial derivation SUCCEEDS, `nix store ping` reports
    `Store URL: local, Trusted: 1`, and flakes are enabled.

That last one is the trap this change also pins. `--init none` runs no daemon,
but the installer still writes a profile script exporting `NIX_REMOTE=daemon`;
anything sourcing it (any `-l` login shell) then dies on "cannot connect to
socket at '/nix/var/nix/daemon-socket/socket'". It is why the installer's own
self-test fails, harmlessly, in the middle of this step's log. The steps here
never source that profile — they invoke `$NIX` by absolute path — but `NIX_REMOTE`
is now pinned empty at job level so a later step cannot reintroduce it.

Also records `df -h` before the build: this fleet ran a runner out of disk today
(ci.yml's `web` job died with "no space left on device" mid-`bun install`), a Nix
build is the heaviest thing that would run here, and a future failure should be
attributable at a glance rather than guessed.

Still unproven, and only the first green run can settle it: whether `nix flake
check` evaluates this flake cleanly under CI, and whether the runner has the disk
to build punktfunk-web. Both now produce a real diagnostic instead of a container
that never started.
This commit is contained in:
2026-08-06 19:31:16 +02:00
parent 453b9850fa
commit e78906fb06
+56 -21
View File
@@ -71,52 +71,86 @@ jobs:
flake:
runs-on: ubuntu-24.04
container:
# Official Nix image (Docker Hub, like this fleet's other WAN images: oven/bun:1,
# fedora:43, node:22-bookworm). It ships nix and little else.
image: nixos/nix:latest
# NOT nixos/nix. That image contains nix and essentially nothing else — in particular no
# /bin/sleep, and Gitea's act_runner starts every job container with
# `entrypoint=["/bin/sleep","10800"]`. The container therefore never starts:
# failed to create shim task: OCI runtime create failed: unable to start container
# process: exec: "/bin/sleep": stat /bin/sleep: no such file or directory
# and — the part that makes this expensive to debug — every step is then reported as
# `cancelled` rather than failed, which reads exactly like a superseded run.
#
# node:22-bookworm instead: a full Debian with coreutils (so the entrypoint exists) and a
# real node (so actions/checkout works with no pre-checkout install dance), and audit.yml
# already pulls it on this fleet, so it is proven to resolve here. Nix is installed below.
image: node:22-bookworm
timeout-minutes: 90
env:
# The image defaults to stable Nix with the experimental features off; the flake needs both.
# Set at job level so every step — including the `nix profile install` below — sees it.
# The flake needs both experimental features. Also baked into the installer's --extra-conf
# below; this covers any step that shells out before that config is read.
NIX_CONFIG: "experimental-features = nix-command flakes"
# Absolute path rather than $GITHUB_PATH: one less runner behaviour to assume.
NIX: /nix/var/nix/profiles/default/bin/nix
# `--init none` installs Nix with NO daemon running, but the installer still writes a profile
# script that exports NIX_REMOTE=daemon. Anything that sources it (any `-l` login shell) then
# dies on `cannot connect to socket at '/nix/var/nix/daemon-socket/socket'` — which is exactly
# how the installer's own self-test fails during this step, harmlessly, and would be a
# confusing first thing to read in the log. The steps below never source that profile, but pin
# the empty value so a future step cannot reintroduce it. Empty = talk to the local store
# directly, which works because the job runs as root (MEASURED: "Store URL: local, Trusted: 1",
# and a real `nix build` of a trivial derivation succeeds).
NIX_REMOTE: ""
steps:
# actions/checkout is a JS action and needs node; a plain `run:` step executes through the
# container shell, so this must come BEFORE the checkout (same ordering, and the same
# reason, as flatpak.yml's fedora job).
- name: node + git for the JS actions
run: nix profile install nixpkgs#nodejs nixpkgs#git
- uses: actions/checkout@v4
# The Determinate installer needs curl + xz; git so nix can read the flake from the checkout.
# (node:22-bookworm is the full image and already has all three — this is belt-and-braces
# against a future slim-image swap, and costs one cached apt call.)
- name: Installer prerequisites
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates curl xz-utils git
# `--init none` is the container mode: no systemd, no daemon. Running as root, nix then talks
# to the store directly. Determinate Nix is also what the Nix box (.21) runs, so CI and the
# hand-verification box stay on the same distribution.
- name: Install Nix
run: |
curl -fsSL https://install.determinate.systems/nix -o /tmp/nix-installer.sh
sh /tmp/nix-installer.sh install linux --init none --no-confirm \
--extra-conf "experimental-features = nix-command flakes"
"$NIX" --version
# Nix reads the flake through libgit2 and refuses a checkout owned by another uid
# ("detected dubious ownership"), which is the normal case for a container job.
- name: Trust the checkout
run: git config --global --add safe.directory "$PWD"
# First-run diagnostics — cheap, and the difference between "the gate found a real problem"
# and "the runner had no disk" is otherwise a guess.
# Diagnostics. This fleet ran a runner out of disk on 2026-08-06 (the ci.yml `web` job died
# with "no space left on device" mid-`bun install`), and a Nix build is the heaviest thing
# here — so record the headroom, or a future failure is a guess.
- name: Environment
run: |
nix --version
df -h /nix /tmp || true
run: df -h / /nix /tmp || true
# Evaluates + instantiates every flake output without building any of it.
- name: nix flake check (eval only)
run: nix flake check --no-build --show-trace
run: |
"$NIX" flake check --no-build --show-trace
# The bun packages, built for real. This is the leg that would have caught the stale
# web/bun.nix end to end: the derivation's offline `bun install` runs against a store cache
# built strictly from bun.nix, so a lockfile that cache does not cover fails here.
# Path-filtered, so it runs only when the packaging or a lockfile actually moves. If it ever
# starts going red on runner disk rather than on real defects, demote it to the dispatch
# opt-in below rather than leaving an infra-red gate on the board.
- name: Build the bun packages
run: nix build --print-build-logs .#punktfunk-web .#punktfunk-scripting
run: |
"$NIX" build --print-build-logs .#punktfunk-web .#punktfunk-scripting
# Both launchers exec pkgs.bun from the store; confirm they were produced and are real entry
# points rather than dangling wrappers.
- name: Smoke the built launchers
run: |
set -eu
web=$(nix path-info .#punktfunk-web)
scripting=$(nix path-info .#punktfunk-scripting)
web=$("$NIX" path-info .#punktfunk-web)
scripting=$("$NIX" path-info .#punktfunk-scripting)
test -x "$web/bin/punktfunk-web-server" || { echo "no punktfunk-web-server in $web" >&2; exit 1; }
test -x "$scripting/bin/punktfunk-scripting" || { echo "no punktfunk-scripting in $scripting" >&2; exit 1; }
# The console must be the bun bundle, not a node one — the same assertion packages.nix
@@ -129,4 +163,5 @@ jobs:
# `github.event.inputs.*` (string) rather than `inputs.*` — the portable spelling.
- name: Build the Rust packages (dispatch opt-in)
if: ${{ github.event.inputs.build-rust == 'true' }}
run: nix build --print-build-logs .#punktfunk-host .#punktfunk-client
run: |
"$NIX" build --print-build-logs .#punktfunk-host .#punktfunk-client