enricobuehler
c3cdee9bf5
docker image prune -af --filter until=2h keyed on image CREATION time, so a
base image built days ago that merely had no container at that instant was
"aged" — including one a job had just pulled and not yet created. Measured
2026-08-07: three job failures, each coinciding with a prune tick to the
second ("No such image: …punktfunk-rust-ci:latest", every step cancelled),
plus a 4-7 GB re-pull of every idle base image within minutes.
The routine tick now retires only what this host actually accretes — per-SHA
app tags older than 2h (their creation time IS the local build time) — then
sweeps dangling layers, which cannot touch a tagged image. The blanket -a
prune survives only in the near-ENOSPC burst guard, where one re-pull beats
every concurrent job dying.
docker-reclaim.{sh,service,timer} are the hourly leak reclaimer that so far
lived hand-installed on home-runner-1 only; home-runner-2 went without it and
re-accumulated 176 leaked volumes (~60 GB) until jobs died of ENOSPC on
2026-08-06/07. Checked in so both hosts install the same files from here.
21 lines
907 B
Desktop File
21 lines
907 B
Desktop File
# Hourly reclaim of Docker resources act_runner LEAKS (per-job volumes, stale networks, old build
|
|
# cache). Sibling of docker-prune.service, which handles what CI legitimately produces and then
|
|
# abandons; the split matters because this one must stay conservative enough to run while jobs are
|
|
# live (dangling-only volumes, age-gated networks) — see docker-reclaim.sh for the full why.
|
|
#
|
|
# Install: see the header of docker-reclaim.sh (note the installed unit name is
|
|
# ci-docker-reclaim.service — existing fleet hosts already run it under that name).
|
|
|
|
[Unit]
|
|
Description=Reclaim disk leaked by Gitea act_runner (per-job volumes, networks, stale build cache)
|
|
Documentation=https://git.unom.io/unom/punktfunk
|
|
After=docker.service
|
|
Requires=docker.service
|
|
|
|
[Service]
|
|
Type=oneshot
|
|
ExecStart=/usr/local/sbin/ci-docker-reclaim.sh
|
|
# Never let maintenance starve a running build.
|
|
Nice=10
|
|
IOSchedulingClass=idle
|