Files
punktfunk/scripts/ci/docker-prune.service
T
enricobuehler 693eedd314 ci(runner): cap the act_runner cache + 30-min prune (fix recurring disk-full)
The hourly docker-prune could never reclaim the real disk filler: the act_runner
cache server's blob store (cache.dir:"" -> /root/.cache/actcache/cache) lives in
the long-running runner container's WRITABLE LAYER, which docker prune can't see.
It grew to ~66 GB and filled the 125 GB disk on its own.

- New docker-prune.sh holds the logic (inline ExecStart= broke under systemd's
  own $-expansion, which emptied $SZ/$(...) before sh ran them — silently no-oping
  the burst guard). The unit now just calls the script.
- Caps the actcache: clears the blobs once they exceed ~20 GB (act_runner
  repopulates; keys are content-hashed, so only stale entries drop).
- Burst guard lowered 85%->80% and now also clears the actcache.
- Timer hourly -> every 30 min; image/cache `until` 12h -> 6h.

Live: cleared 66 GB on home-runner-1 (93% -> 20%), deployed + verified.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 07:53:08 +00:00

33 lines
1.7 KiB
Desktop File

# Docker disk hygiene for the self-hosted Gitea Actions runner (home-runner-1, 192.168.1.52).
#
# Why this exists: every CI push builds and sha-<commit>-tags a Docker image per pipeline
# (rust-ci, web, docs, fedora-rpm, fedora44-rpm, ...). Those tags are never dangling, so a
# plain `docker image prune` SKIPS them and they accumulate. Host-level, not per-repo CI,
# because the runner is shared (punktfunk + other orgs all benefit).
#
# THE BIG ONE (2026-06-19): the act_runner CACHE SERVER store lives in the long-running runner
# container's WRITABLE LAYER (HOME/.cache/actcache/cache inside gitea-runner-runner-1,
# `cache.dir: ""` -> defaults under /root). `docker prune` can NEVER see it — only stopped
# containers + unused images/cache are prunable, not a 13-day-up container's layer. That store
# grew to ~66 GB and filled a 125 GB disk on its own. docker-prune.sh caps it by clearing the
# blobs in-place (act_runner repopulates; keys are content-hashed).
#
# The logic is in docker-prune.sh, NOT inline ExecStart=, because systemd does its own
# $-expansion on ExecStart and would empty the shell vars / $(...) before sh runs them.
#
# Install on the runner host (root):
# install -m755 scripts/ci/docker-prune.sh /usr/local/bin/ci-docker-prune.sh
# cp scripts/ci/docker-prune.{service,timer} /etc/systemd/system/
# systemctl daemon-reload && systemctl enable --now docker-prune.timer
# See also scripts/ci/setup-macos-runner.sh for the macOS runner.
[Unit]
Description=Prune aged Docker images/cache + cap the act_runner cache (CI runner disk hygiene)
Documentation=https://git.unom.io/unom/punktfunk
Wants=docker.service
After=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/ci-docker-prune.sh