merge: bring current main into the audio-substrate branch
ci / bun-nix (pull_request) Successful in 47s
ci / web (pull_request) Successful in 1m19s
apple / swift (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m27s
ci / rust-arm64 (pull_request) Successful in 4m25s
android / android (pull_request) Successful in 5m59s
ci / rust (pull_request) Successful in 7m19s

Two conflicts, both unions of independent removals/fixes: main fixed the
same three install.rs SAFETY comments this branch fixed (main's phrasing
kept), and the runner provisioning drops BOTH env lines — main removed
PF_FFVK_VULKAN_INCLUDE (pf-ffvk is gone since the FFmpeg replacement),
this branch removed VBCABLE_DIR (the retirement).
This commit is contained in:
2026-08-07 17:49:50 +02:00
334 changed files with 127958 additions and 17022 deletions
+27 -10
View File
@@ -1,11 +1,11 @@
#!/usr/bin/env bash
# CI runner disk hygiene — invoked by docker-prune.service (every 30 min). Lives in a real script
# CI runner disk hygiene — invoked by docker-prune.service (every 2 min). Lives in a real script
# rather than inline ExecStart= lines because systemd does its OWN $-expansion on ExecStart and
# empties shell vars / $(...) before /bin/sh sees them (silently breaking the logic under `|| true`).
#
# See docker-prune.service for the full why. The headline: the act_runner cache server's blob store
# lives INSIDE the long-running runner container's writable layer, where `docker prune` can't reach
# it — left alone it grows to tens of GB and fills the disk on its own.
# See docker-prune.service for the full why. Sibling: docker-reclaim.sh (hourly) handles what
# act_runner *leaks* — per-job volumes, stale networks, old build cache. This one handles what
# CI legitimately *produces* and then abandons: per-SHA app tags and the layers they pin.
set -u
export PATH=/usr/bin:/bin:/usr/local/bin:$PATH
@@ -23,11 +23,26 @@ MIN_FREE_GB=${MIN_FREE_GB:-60} # ...or this little is left, whichever t
# 2026-07-29: zero burst clears fired in six hours
# while deb still died of ENOSPC between polls.
# 1) Routine: trim aged images / build cache / stopped containers. sha-<commit> tags aren't
# dangling, so -a is required. until=2h, not 6h: on a busy day every image is younger than six
# hours, so the filter matched nothing and a run reclaimed 0B while `docker system df` was
# reporting 20+ GB reclaimable. Two hours still protects a re-run of the push being worked on.
docker image prune -af --filter until=2h || true
# 1) Routine: retire aged per-SHA app tags, then sweep what untagging released.
# ⚠ NEVER `docker image prune -a` on this tick. `until=` filters on image CREATION time, so a
# CI *base* image (built days ago) that merely has no container this instant counts as "aged" —
# including one a job JUST PULLED whose container does not exist yet. Measured 2026-08-07:
# this tick ran 07:36:09:29 and a rust job's `docker create` failed at 07:36:29 with
# "No such image: …punktfunk-rust-ci:latest" — three sampled failures that morning, each
# coinciding with a prune run to the second — and every idle base image was re-pulled within
# minutes (47 GB each), churning the LAN registry for nothing.
# The only tag debris this host actually accretes is the per-SHA app tags (web/docs — their
# creation time IS the local build time, so a 2h age gate is exact), and a dangling-only prune
# cannot touch a tagged image, so neither step can race a starting job.
now=$(date +%s)
docker images --format '{{.Repository}}:{{.Tag}}' 2>/dev/null | grep ':sha-' | while read -r ref; do
created=$(docker image inspect -f '{{.Created}}' "$ref" 2>/dev/null) || continue
cts=$(date -d "$created" +%s 2>/dev/null) || continue
if [ $((now - cts)) -ge 7200 ]; then
docker rmi "$ref" >/dev/null 2>&1 || true
fi
done
docker image prune -f || true
docker builder prune -af --filter until=2h || true
docker buildx prune -af --filter until=2h || true
docker container prune -f --filter until=2h || true
@@ -44,7 +59,9 @@ docker network prune -f --filter until=2h || true
# what matters is absolute headroom for three concurrent target/ dirs, not a ratio — and the
# ratio moves whenever the disk is resized (it went 123 G -> 175 G on 2026-07-29) while the
# headroom three jobs need does not. In-use images are protected by the daemon, so a burst clear
# cannot pull the rug from a live job.
# cannot pull the rug from a live job — but the blanket `-a` prune below CAN race an image that
# is pulled-but-not-yet-created (the section 1 lesson). That narrow window is accepted HERE
# only: when the alternative is every concurrent job dying of ENOSPC, one job re-pulling loses.
PCT=$(df --output=pcent / | tr -dc '0-9')
FREE_GB=$(df --output=avail -BG / | tr -dc '0-9')
# Two flat tests into a flag rather than one multi-line `{ …; } || { …; }` condition: the brace-group
+20
View File
@@ -0,0 +1,20 @@
# Hourly reclaim of Docker resources act_runner LEAKS (per-job volumes, stale networks, old build
# cache). Sibling of docker-prune.service, which handles what CI legitimately produces and then
# abandons; the split matters because this one must stay conservative enough to run while jobs are
# live (dangling-only volumes, age-gated networks) — see docker-reclaim.sh for the full why.
#
# Install: see the header of docker-reclaim.sh (note the installed unit name is
# ci-docker-reclaim.service — existing fleet hosts already run it under that name).
[Unit]
Description=Reclaim disk leaked by Gitea act_runner (per-job volumes, networks, stale build cache)
Documentation=https://git.unom.io/unom/punktfunk
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/ci-docker-reclaim.sh
# Never let maintenance starve a running build.
Nice=10
IOSchedulingClass=idle
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Reclaim the disk that Gitea act_runner leaks on this host.
#
# Why this exists: act_runner creates a per-job network and a pair of named volumes, and leaks both
# when a job is killed or the runner restarts. By 2026-07-25 that had accumulated 252 unused volumes
# (11.7 GB) and 94 stale networks — some dating to task 5626 while current tasks were ~25233 — and
# concurrent builds then exhausted the disk, failing CI with "No space left on device" at both the
# cargo and the Docker/overlayfs layer. The stale networks are also what once broke the docs deploy
# by exhausting Docker's default address pool and swallowing the DMZ 192.168.50.0/24 range.
#
# This ran on home-runner-1 only, hand-installed; home-runner-2 went without it and by 2026-08-07
# had re-accumulated 176 leaked volumes (~60 GB) + 22 GB build cache and spent two days failing
# jobs at ENOSPC. Hence checked in: BOTH runner hosts install it, from here.
#
# Install on a runner host (root):
# install -m755 scripts/ci/docker-reclaim.sh /usr/local/sbin/ci-docker-reclaim.sh
# install -m644 scripts/ci/docker-reclaim.service /etc/systemd/system/ci-docker-reclaim.service
# install -m644 scripts/ci/docker-reclaim.timer /etc/systemd/system/ci-docker-reclaim.timer
# systemctl daemon-reload && systemctl enable --now ci-docker-reclaim.timer
#
# Deliberately NOT `docker volume prune -a`: that would also delete any intentional named volume
# that merely has no container attached at the moment the timer fires — e.g. the `docker-mirror`
# pull-through registry cache or the runner cache during a restart — silently destroying it. Only
# volumes act_runner named are removed here.
#
# Also deliberately NOT pruning images: on this host the per-SHA CI tags share all their layers with
# `:latest`, so removing them reclaims nothing while forcing re-pulls. `docker system df`'s
# "RECLAIMABLE" column counts shared layers once per image and overstates the win badly.
# (docker-prune.sh owns tag retirement — age-gated and never `image prune -a`, see its header.)
set -uo pipefail
log() { echo "ci-docker-reclaim: $*"; }
before_avail=$(df --output=avail -BM / | tail -1 | tr -dc '0-9')
# 1. Leaked per-job volumes — dangling AND named by act_runner. In-use volumes are never listed as
# dangling, so a running job's volumes cannot be hit.
mapfile -t stale_vols < <(docker volume ls -qf dangling=true 2>/dev/null | grep '^GITEA-ACTIONS-TASK-' || true)
if ((${#stale_vols[@]})); then
printf '%s\n' "${stale_vols[@]}" | xargs -r docker volume rm >/dev/null 2>&1
log "removed ${#stale_vols[@]} leaked act_runner volumes"
else
log "no leaked act_runner volumes"
fi
# 2. Unused networks older than 2h — never touches a live job's network (it is in use), and the age
# filter keeps a just-created one safe against a race with a starting job.
net_out=$(docker network prune -f --filter until=2h 2>&1 | grep -c '^GITEA-ACTIONS' || true)
log "removed ${net_out:-0} stale job networks"
# 3. Build cache older than 48h. Recent cache is what makes builds fast, so it is kept.
cache_freed=$(docker builder prune -f --filter until=48h 2>&1 | awk '/^Total:/ {print $2}')
log "build cache freed: ${cache_freed:-0B}"
after_avail=$(df --output=avail -BM / | tail -1 | tr -dc '0-9')
log "avail ${before_avail}M -> ${after_avail}M (reclaimed $((after_avail - before_avail))M)"
df -h / | tail -1 | sed 's/^/ci-docker-reclaim: /'
+16
View File
@@ -0,0 +1,16 @@
# Hourly is the right cadence for LEAKS: they only accrue when jobs die abnormally, and the
# per-tick docker-prune.timer (every 2 min) already carries the burst guard for genuine
# disk-pressure emergencies. Install: see the header of docker-reclaim.sh.
[Unit]
Description=Hourly reclaim of act_runner-leaked Docker disk
[Timer]
OnCalendar=hourly
# Catch up after a reboot rather than waiting for the next slot.
Persistent=true
# Spread it off the hour so it does not collide with scheduled CI.
RandomizedDelaySec=300
[Install]
WantedBy=timers.target
+1 -1
View File
@@ -1,5 +1,5 @@
# Idempotent pre-flight for punktfunk's Windows CI dependencies: WDK + cargo-wdk (driver builds),
# FFmpeg x64/ARM64 trees, Inno Setup, and the aarch64-pc-windows-msvc rustup target. Run at the
# the x64 FFmpeg tree (host amf-qsv only), Inno Setup, and the aarch64-pc-windows-msvc rustup target. Run at the
# start of every Windows CI job so ANY runner - freshly built from unom/infra's windows-runner/
# template, rebuilt, or a new one added later - self-provisions on first real use, instead of
# needing a human to remember to dispatch a separate provisioning workflow first (and instead of
@@ -1,5 +1,5 @@
# Layers punktfunk-specific tooling onto the shared unom Windows CI runner: per-arch FFmpeg
# (host + client native builds), Inno Setup (the host installer), and the aarch64-pc-windows-msvc
# Layers punktfunk-specific tooling onto the shared unom Windows CI runner: FFmpeg (the HOST's
# amf-qsv encode leg, x64 only), Inno Setup (the host installer), and the aarch64-pc-windows-msvc
# rustup target (windows-msix.yml's ARM64 leg). The runner itself - act_runner, Node, rustup,
# VS Build Tools/NASM/CMake/LLVM - is provisioned generically by unom/infra
# (windows-runner/windows-runner.pkr.hcl + proxmox/windows-runner's Terraform clone); this script
@@ -26,14 +26,19 @@ if (Test-Path $rustup) {
Write-Warning "rustup not found at $rustup - has unom/infra's setup-gitea-runner-base.ps1 run on this box yet?"
}
# --- FFmpeg shared trees for the host (amf-qsv encode) + clients (decode). BtbN **lgpl-shared**
# --- FFmpeg shared tree for the HOST's amf-qsv encode leg (windows-host.yml). BtbN **lgpl-shared**
# builds: the AMD/Intel AMF + Intel QSV encoders, swscale, and the HEVC decoder are all present in
# the LGPL build, and punktfunk never calls the GPL-only encoders (x264/x265 - software encode is
# the separate BSD-2 openh264 crate; NVENC is the direct NVIDIA SDK). lgpl-shared keeps the
# bundled DLLs LGPL-2.1+ (dynamic linking satisfies the relink duty) rather than GPL, so the
# shipped installer/MSIX stay consistent with punktfunk's MIT OR Apache-2.0 posture.
# MIGRATION: a runner previously provisioned with the old *gpl-shared* trees must be
# re-provisioned - delete C:\Users\Public\ffmpeg and C:\Users\Public\ffmpeg-arm64, then re-run.
# ⚠ The CLIENT no longer links FFmpeg at all (M10, design/client-native-decode.md §6): it decodes
# with pf-vkdecode / pf-dxvadec / openh264 + rav1d. windows.yml and windows-msix.yml set no
# FFMPEG_DIR and the MSIX bundles no libav* DLLs, so only the x64 tree is fetched now - the ARM64
# one existed solely for the ARM64 client leg. Delete a stale C:\Users\Public\ffmpeg-arm64 by
# hand; this script does not remove what it no longer installs.
# MIGRATION: a runner previously provisioned with the old *gpl-shared* tree must be
# re-provisioned - delete C:\Users\Public\ffmpeg, then re-run.
# These DLLs are bundled verbatim into the code-signed host installer/MSIX, so the download is
# SHA-256-pinned (like VB-CABLE below): BtbN's `latest` tag is a ROLLING release whose assets are
# re-uploaded over time, so an unverified fetch would let a hijacked/MITM'd upstream asset land
@@ -42,7 +47,7 @@ if (Test-Path $rustup) {
# that is intentional: re-download, re-verify the new archive, and update the two pins here.
# Refresh a pin: (Get-FileHash .\ffmpeg-<tag>.zip -Algorithm SHA256).Hash
function Get-BtbnFfmpeg {
param([string]$Dir, [string]$ZipTag, [string]$Sha) # ZipTag: 'win64' (x64) or 'winarm64' (ARM64 cross tree)
param([string]$Dir, [string]$ZipTag, [string]$Sha) # ZipTag: 'win64' (x64); BtbN also publishes 'winarm64'
if (Test-Path (Join-Path $Dir 'lib\avcodec.lib')) { info "FFmpeg ($ZipTag) already present at $Dir"; return }
info "fetching FFmpeg ($ZipTag, BtbN lgpl-shared, SHA-256 pinned)"
$url = "https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-n7.1-latest-$ZipTag-lgpl-shared-7.1.zip"
@@ -60,27 +65,13 @@ function Get-BtbnFfmpeg {
Move-Item -Path $inner.FullName -Destination $Dir
Remove-Item -Force $zip; Remove-Item -Recurse -Force $tmp -ErrorAction SilentlyContinue
}
Get-BtbnFfmpeg -Dir "C:\Users\Public\ffmpeg" -ZipTag 'win64' -Sha '89F3469706E5D53AEA5CF34AEE63E62CE746E6159D7AEE473D330B02A47558E6'
Get-BtbnFfmpeg -Dir "C:\Users\Public\ffmpeg-arm64" -ZipTag 'winarm64' -Sha 'D96B4CE08CEBDCC6AD0E3934A3F962915E440EEFB9D73831AFEA4D80E35129A5'
Get-BtbnFfmpeg -Dir "C:\Users\Public\ffmpeg" -ZipTag 'win64' -Sha '89F3469706E5D53AEA5CF34AEE63E62CE746E6159D7AEE473D330B02A47558E6'
# --- Vulkan-Headers (pf-ffvk's bindgen: libavutil/hwcontext_vulkan.h includes <vulkan/vulkan.h>,
# and Windows has no system copy). Headers only - the loader (vulkan-1.dll) is a GPU-driver
# component and is never linked at build time, so the full Vulkan SDK is deliberately NOT
# required. Pinned Khronos tag; bump deliberately alongside FFmpeg/driver expectations. ---
$vkHdrDir = "C:\Users\Public\vulkan-headers"
$vkHdrTag = "v1.4.309"
if (-not (Test-Path (Join-Path $vkHdrDir 'include\vulkan\vulkan.h'))) {
info "fetching Vulkan-Headers $vkHdrTag"
$url = "https://github.com/KhronosGroup/Vulkan-Headers/archive/refs/tags/$vkHdrTag.zip"
$zip = "$vkHdrDir.zip"; $tmp = "$vkHdrDir-extract"
Invoke-WebRequest -Uri $url -OutFile $zip -UseBasicParsing
if (Test-Path $tmp) { Remove-Item -Recurse -Force $tmp }
Expand-Archive -Path $zip -DestinationPath $tmp -Force # one top-level Vulkan-Headers-<ver> folder
$inner = Get-ChildItem $tmp -Directory | Select-Object -First 1
if (Test-Path $vkHdrDir) { Remove-Item -Recurse -Force $vkHdrDir }
Move-Item -Path $inner.FullName -Destination $vkHdrDir
Remove-Item -Force $zip; Remove-Item -Recurse -Force $tmp -ErrorAction SilentlyContinue
} else { info "Vulkan-Headers already present at $vkHdrDir" }
# --- No Vulkan-Headers here any more: they existed only for pf-ffvk's bindgen over
# libavutil/hwcontext_vulkan.h, and that crate is gone (M10). Nothing punktfunk builds on Windows
# needs Vulkan headers at compile time - ash generates its own bindings and dlopens vulkan-1.dll,
# which is a GPU-driver component. A stale C:\Users\Public\vulkan-headers is harmless; delete it
# by hand if you want the disk back. ---
# --- Inno Setup (ISCC.exe) for the host installer build (windows-host.yml). pack-host-installer.ps1
# locates it at its fixed Program Files path, so it need not be on PATH - just present. The .iss
@@ -101,13 +92,14 @@ if (-not (Test-Path $isccPath) -or ($innoVer -and [version]$innoVer -lt [version
# --- Drop punktfunk's env vars into the generic runner's daemon wrapper extension point (see
# unom/infra's scripts/setup-gitea-runner-base.ps1) so the act_runner daemon - and therefore every
# job it runs - sees FFMPEG_DIR without unom/infra needing to know punktfunk exists. ---
# job it runs - sees FFMPEG_DIR without unom/infra needing to know punktfunk exists.
# FFMPEG_DIR + the PATH prepend are the HOST's (windows-host.yml amf-qsv: import libs at link time,
# the DLLs at test time). The client workflows ignore both - they link no libav*. ---
$projectEnv = "C:\Users\Public\act-runner\project-env.ps1"
@'
$env:FFMPEG_DIR = "C:\Users\Public\ffmpeg"
$env:PF_FFVK_VULKAN_INCLUDE = "C:\Users\Public\vulkan-headers\include"
$env:PATH = "C:\Users\Public\ffmpeg\bin;" + $env:PATH
'@ | Set-Content -Encoding UTF8 $projectEnv
info "wrote $projectEnv (FFMPEG_DIR, PF_FFVK_VULKAN_INCLUDE) - restart the gitea-act-runner scheduled task to pick it up"
info "wrote $projectEnv (FFMPEG_DIR) - restart the gitea-act-runner scheduled task to pick it up"
info "punktfunk extras provisioned OK."