The bun OOM is a 7 GiB cgroup cap, not a big build — measure it in the log #341

Merged

enricobuehler merged 2 commits from nix-serialise-bun-builds into main 2026-08-19 18:18:30 +00:00

Conversation 0 Commits 2 Files Changed 1

+13 -1

2 Commits

Author	SHA1	Message	Date
enricobuehler	08c45b96eb	fix(nix): measure the cgroup cap instead of guessing at the bun OOM ... Reverts the --max-jobs 1 from the previous commit. The hypothesis it encoded — that punktfunk-web and punktfunk-scripting bundling concurrently exhausted the box — is DISPROVEN: run 19444 failed identically with it, and the Environment step it added shows why the premise was wrong: Mem: 125Gi total, 48Gi available Swap: 8.0Gi nproc: 16 overlay 172G, 108G avail Nothing was scarce. Serialising also would have cost real wall-clock on the publish tier, which builds six derivations including an hour of Rust, so it does not stay on an unproven basis. Also ruled out: bun is 1.3.13 in BOTH the old and new nixpkgs, so the lock bump did not change the bundler. What remains is what those numbers cannot see. Inside a container /proc/meminfo and free(1) report the HOST, so a cgroup memory cap is invisible to them — and a cap is the only explanation left for a SIGKILL with 48Gi apparently free. So read it directly: cgroup v2 memory.max, falling back to v1 memory.limit_in_bytes. No fix in this commit, deliberately. Two guesses have already cost a day of red runs; the next change should follow the number.	2026-08-19 17:06:24 +00:00
enricobuehler	a4f6e259e3	fix(nix): serialise the bun builds — punktfunk-web is OOM-killed beside scripting ... The nix job's recurring red is not a broken bundle, it is memory: punktfunk-web> error: script "build" was terminated by signal SIGKILL (Forced quit) Reason: builder failed with exit code 137 MEASURED 2026-08-19: five runs across main and two branches died exactly there, while other runs on the SAME commits passed — the signature of pressure, not of a defect. nix builds punktfunk-web and punktfunk-scripting concurrently, and each one runs a memory-hungry bundler; that pairing is what the runner cannot afford. `--max-jobs 1` caps how many DERIVATIONS build at once, not the cores inside one, so two bundles that each pin a core simply run in sequence. Applied to the publish tier too, which builds six derivations including those same two. This step sits BEFORE the publish tier, so every one of those failures also stopped the binary cache from filling — it has blocked more runs today than every real bug combined. Also prints memory in the Environment step. An exit 137 cannot be explained by the disk numbers that step used to gather on their own.	2026-08-19 16:30:41 +00:00