enricobuehler
f1dc6c9f94
Three independent reasons Rust CI stayed slow despite sccache, fixed together because
they share the same measurement.
1. sccache only ever covered RUSTC. Every C/C++ dependency in the tree — aws-lc-sys,
openh264-sys2's vendored C++, the CMake-built libopus behind audiopus_sys — was
compiled from scratch on every job of every workflow. CMAKE_{C,CXX}_COMPILER_LAUNCHER
plus CC_/CXX_x86_64_unknown_linux_gnu route both build-script styles (cc-rs and
cmake-rs) through the same shared cache.
The CC_* vars are JOB-scoped in ci.yml and deb.yml, never workflow-scoped: the
arm64 cross image sets its own CC_x86_64_unknown_linux_gnu=pf-host-cc, the wrapper
that keeps ffmpeg-sys-next's host probe off the arm64 include dirs. Overwriting it
would surface as a header mismatch rather than as a CI config error.
2. Linking is cacheable by nothing, and these jobs relink the host, client, session,
cli, worker and tray on every run — twice per push for rpm (f43 + f44). The four
Linux builder images now install mold and carry a $CARGO_HOME/config.toml that uses
it for x86_64. aarch64 is deliberately left alone (cross driver, already-fast legs).
Each image asserts `mold --version` in its build, so an image can never ship the
flag without the linker: docker.yml goes red and :latest stays on the last good one.
3. THE EXPENSIVE ONE. ci.yml (debug) and deb.yml (release) named a byte-identical
target-cache key, under a comment claiming the release build reused ci.yml's
artifacts. It never could. actions/cache is first-saver-wins on an exact key and
ci.yml is the faster job, so the shared key always held a debug-only target/ — and,
worse, deb.yml could then never save its own, because the key was taken. Every
canary .deb has been a from-scratch release build for as long as both keys existed.
Same collision on the arm64 pair, and a third participant in
linux-client-screenshots.yml. Split into -debug-/-release- key families; that job
reads deb's tree via restore-keys but keeps its own exact key so it can never win
the save race and replace a full tree with its single-crate one.
Also: one scripts/ci/ensure-sccache.sh replaces ten copy-pasted bootstrap blocks that
had already drifted into two dialects (GNU tar --wildcards vs bsdtar), every Rust job
now ends with --show-stats so a cache regression is visible instead of just "CI got
slower", and deb.yml's web install joins every other CI install on --ignore-scripts.
No behaviour change to any artifact: same compilers, same flags, same outputs.
66 lines
2.9 KiB
Bash
66 lines
2.9 KiB
Bash
#!/bin/sh
|
|
# Ensure `sccache` is on PATH, self-healing on a runner/image that does not already carry it.
|
|
#
|
|
# WHY THIS EXISTS: this block was copy-pasted into ten jobs across six workflows (ci.yml x2,
|
|
# deb.yml x3, rpm.yml, bench.yml, apple.yml x2, the Apple release leg), in two dialects that had
|
|
# already drifted apart — the Linux copies pass `--wildcards` to GNU tar, the macOS copies must NOT
|
|
# (bsdtar globs by default and rejects the flag). One copy per platform, here, so a version bump or
|
|
# a mirror change is one edit rather than ten.
|
|
#
|
|
# The builder images (ci/*.Dockerfile) BAKE sccache, so on Linux this is a no-op in the normal case;
|
|
# it stays because a job runs against the image from the PREVIOUS push (docker.yml's bootstrap lag),
|
|
# and the macOS runner is a persistent host with no image at all.
|
|
#
|
|
# POSIX sh on purpose: Gitea's act_runner executes a step's `run:` under `sh -e` (dash) inside the
|
|
# Linux job containers — no bashisms, no process substitution (see the shader-gate note in ci.yml
|
|
# for what that cost the last time someone assumed bash).
|
|
#
|
|
# Usage: sh scripts/ci/ensure-sccache.sh
|
|
set -e
|
|
|
|
# Keep in step with the ARG SCCACHE_VERSION in ci/*.Dockerfile — the images bake this same version,
|
|
# and a job that heals to a DIFFERENT one would quietly split the shared cache's key universe in two
|
|
# (sccache's cache keys are not versioned across incompatible releases).
|
|
SCCACHE_VERSION="${SCCACHE_VERSION:-0.10.0}"
|
|
|
|
if command -v sccache >/dev/null 2>&1; then
|
|
sccache --version
|
|
exit 0
|
|
fi
|
|
|
|
BASE="https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}"
|
|
|
|
case "$(uname -s)" in
|
|
Darwin)
|
|
# The macOS runner is a LaunchAgent in the user's Aqua session, not root — install into the
|
|
# user prefix. ~/.local/bin is already on the runner daemon's PATH; GITHUB_PATH is
|
|
# belt-and-braces for the steps that follow in THIS job.
|
|
DEST="$HOME/.local/bin"
|
|
mkdir -p "$DEST"
|
|
case "$(uname -m)" in
|
|
arm64|aarch64) ARCH=aarch64-apple-darwin ;;
|
|
*) ARCH=x86_64-apple-darwin ;;
|
|
esac
|
|
# bsdtar globs by default and does not accept --wildcards.
|
|
curl -fsSL "$BASE/sccache-v${SCCACHE_VERSION}-${ARCH}.tar.gz" \
|
|
| tar -xz --strip-components=1 -C "$DEST" '*/sccache'
|
|
chmod 0755 "$DEST/sccache"
|
|
PATH="$DEST:$PATH"
|
|
export PATH
|
|
if [ -n "${GITHUB_PATH:-}" ]; then
|
|
echo "$DEST" >> "$GITHUB_PATH"
|
|
fi
|
|
;;
|
|
*)
|
|
# Linux job containers run as root; /usr/local/bin is on PATH already, so no GITHUB_PATH
|
|
# dance is needed. The musl build is static — one binary serves the Ubuntu, Fedora and Arch
|
|
# images alike.
|
|
DEST=/usr/local/bin
|
|
curl -fsSL "$BASE/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
|
| tar -xz --wildcards --strip-components=1 -C "$DEST" '*/sccache'
|
|
chmod 0755 "$DEST/sccache"
|
|
;;
|
|
esac
|
|
|
|
sccache --version
|