feat(ci): builder images move to the LAN registry, content-keyed; fan-out scoped by paths
The five builder images now live on home-ci-core's LAN registry (192.168.1.58:5010) under content keys — a hash of the ci/ tree (+ rust-toolchain.toml for the cross image). docker.yml builds one only when its key has no manifest yet, so a push that doesn't touch ci/ costs a curl per image instead of seven WAN pushes and a set of per-SHA tags that no plain prune could ever reclaim. Releases pin builders by copying the key manifest to a vX.Y.Z tag via the registry API — no rebuild, no bytes moved. Around that: deb/rpm/arch/android/apple/decky get path filters so docs-only pushes stop lighting up the whole fleet (branch pushes only — tag runs match tags:, as flatpak/windows-msix releases have proven for months); the report-only bench job moves to bench.yml (nightly + dispatch) and stops occupying a fleet slot per push; flatpak caches its Flathub runtimes and builder state instead of re-downloading multi-GB every run; rpm's cargo registry cache gets its own key namespace instead of sharing the Ubuntu jobs'; audit caches cargo bin+registry rather than the whole toolchain dir; docker-prune.sh loses the local act-cache cap/burst-clear (the cache is central now — deleting it under disk pressure was how runner-2 ended up cold-building everything) and gains a leaked-network prune. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
+11
-19
@@ -9,16 +9,12 @@
|
||||
set -u
|
||||
export PATH=/usr/bin:/bin:/usr/local/bin:$PATH
|
||||
|
||||
# The cache-server's blob store is a HOST directory: the fleet runs a standalone cache-server
|
||||
# service (compose.yml) that bind-mounts this path to /data, and every replica points at it over
|
||||
# HTTP (`external_server`). It used to live inside a runner container's writable layer, which is
|
||||
# why this reached in with `docker exec` — that is no longer where it is, and the container name it
|
||||
# looked for (`gitea-runner-runner`) does not exist either now that the replicas are named
|
||||
# `gitea-runner-fleet-runner-N-1`. Both halves silently did nothing: the filter matched zero
|
||||
# containers, so the cap and the burst-clear below were dead code. A plain host path needs neither.
|
||||
CACHE_DIR=${CACHE_DIR:-/home/runner/gitea-runner-fleet/cache}
|
||||
CAP_MB=${CAP_MB:-20000} # clear the cache store once it exceeds ~20 GB
|
||||
BURST_PCT=${BURST_PCT:-80} # full clear once the disk is this % full
|
||||
# The actions cache no longer lives on this box AT ALL: home-ci-core (192.168.1.58, see
|
||||
# unom/infra runners/ci-core/) serves every runner host, sized and GC'd there. The old local
|
||||
# store cap + burst-clear are gone with it — they were self-defeating anyway: under disk
|
||||
# pressure they deleted exactly the cache that made the next job smaller, which is how
|
||||
# runner-2 ended up cold-building every Rust job with an empty 28 KB cache dir.
|
||||
BURST_PCT=${BURST_PCT:-80} # burst-clear docker debris once the disk is this % full
|
||||
MIN_FREE_GB=${MIN_FREE_GB:-60} # ...or this little is left, whichever trips first.
|
||||
# 60, not 45: this has to fire BEFORE the disk is
|
||||
# actually tight, because the clear only reclaims idle
|
||||
@@ -36,14 +32,11 @@ docker builder prune -af --filter until=2h || true
|
||||
docker buildx prune -af --filter until=2h || true
|
||||
docker container prune -f --filter until=2h || true
|
||||
|
||||
# 2) Cap the cache-server store. Clearing the blobs is safe — act_runner repopulates it and cache
|
||||
# keys are content-hashed, so this only drops stale entries.
|
||||
if [ -d "$CACHE_DIR" ]; then
|
||||
SZ=$(du -sm "$CACHE_DIR" 2>/dev/null | cut -f1)
|
||||
if [ -n "${SZ:-}" ] && [ "$SZ" -ge "$CAP_MB" ]; then
|
||||
rm -rf "${CACHE_DIR:?}"/* && echo "cache-server store cleared (was ${SZ} MB)"
|
||||
fi
|
||||
fi
|
||||
# 2) Leaked job networks. act_runner leaks per-job GITEA-ACTIONS-TASK-* bridges when jobs are
|
||||
# killed; enough of them exhausted the docker address pool once (it then swallowed the DMZ
|
||||
# subnet — see unom/infra runners/ci-core/README.md) and each one is another interface for
|
||||
# the host dnsmasq to bind. until=2h protects the networks of live jobs.
|
||||
docker network prune -f --filter until=2h || true
|
||||
|
||||
# 3) Burst guard: a push-storm fills the disk WITHIN one interval — three concurrent Rust builds,
|
||||
# each with a multi-GB target/, on top of a ~40 GB containerd image baseline. Trigger on a free
|
||||
@@ -65,5 +58,4 @@ if [ "$BURST" = 1 ]; then
|
||||
docker image prune -af || true
|
||||
docker builder prune -af || true
|
||||
docker buildx prune -af || true
|
||||
[ -d "$CACHE_DIR" ] && rm -rf "${CACHE_DIR:?}"/* || true
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user