enricobuehler
c3cdee9bf5
docker image prune -af --filter until=2h keyed on image CREATION time, so a
base image built days ago that merely had no container at that instant was
"aged" — including one a job had just pulled and not yet created. Measured
2026-08-07: three job failures, each coinciding with a prune tick to the
second ("No such image: …punktfunk-rust-ci:latest", every step cancelled),
plus a 4-7 GB re-pull of every idle base image within minutes.
The routine tick now retires only what this host actually accretes — per-SHA
app tags older than 2h (their creation time IS the local build time) — then
sweeps dangling layers, which cannot touch a tagged image. The blanket -a
prune survives only in the near-ENOSPC burst guard, where one re-pull beats
every concurrent job dying.
docker-reclaim.{sh,service,timer} are the hourly leak reclaimer that so far
lived hand-installed on home-runner-1 only; home-runner-2 went without it and
re-accumulated 176 leaked volumes (~60 GB) until jobs died of ENOSPC on
2026-08-06/07. Checked in so both hosts install the same files from here.
17 lines
555 B
SYSTEMD
17 lines
555 B
SYSTEMD
# Hourly is the right cadence for LEAKS: they only accrue when jobs die abnormally, and the
|
|
# per-tick docker-prune.timer (every 2 min) already carries the burst guard for genuine
|
|
# disk-pressure emergencies. Install: see the header of docker-reclaim.sh.
|
|
|
|
[Unit]
|
|
Description=Hourly reclaim of act_runner-leaked Docker disk
|
|
|
|
[Timer]
|
|
OnCalendar=hourly
|
|
# Catch up after a reboot rather than waiting for the next slot.
|
|
Persistent=true
|
|
# Spread it off the hour so it does not collide with scheduled CI.
|
|
RandomizedDelaySec=300
|
|
|
|
[Install]
|
|
WantedBy=timers.target
|