forked from unom/punktfunk
ci: cache the C/C++ half, link with mold, and split the debug/release target caches
Three independent reasons Rust CI stayed slow despite sccache, fixed together because
they share the same measurement.
1. sccache only ever covered RUSTC. Every C/C++ dependency in the tree — aws-lc-sys,
openh264-sys2's vendored C++, the CMake-built libopus behind audiopus_sys — was
compiled from scratch on every job of every workflow. CMAKE_{C,CXX}_COMPILER_LAUNCHER
plus CC_/CXX_x86_64_unknown_linux_gnu route both build-script styles (cc-rs and
cmake-rs) through the same shared cache.
The CC_* vars are JOB-scoped in ci.yml and deb.yml, never workflow-scoped: the
arm64 cross image sets its own CC_x86_64_unknown_linux_gnu=pf-host-cc, the wrapper
that keeps ffmpeg-sys-next's host probe off the arm64 include dirs. Overwriting it
would surface as a header mismatch rather than as a CI config error.
2. Linking is cacheable by nothing, and these jobs relink the host, client, session,
cli, worker and tray on every run — twice per push for rpm (f43 + f44). The four
Linux builder images now install mold and carry a $CARGO_HOME/config.toml that uses
it for x86_64. aarch64 is deliberately left alone (cross driver, already-fast legs).
Each image asserts `mold --version` in its build, so an image can never ship the
flag without the linker: docker.yml goes red and :latest stays on the last good one.
3. THE EXPENSIVE ONE. ci.yml (debug) and deb.yml (release) named a byte-identical
target-cache key, under a comment claiming the release build reused ci.yml's
artifacts. It never could. actions/cache is first-saver-wins on an exact key and
ci.yml is the faster job, so the shared key always held a debug-only target/ — and,
worse, deb.yml could then never save its own, because the key was taken. Every
canary .deb has been a from-scratch release build for as long as both keys existed.
Same collision on the arm64 pair, and a third participant in
linux-client-screenshots.yml. Split into -debug-/-release- key families; that job
reads deb's tree via restore-keys but keeps its own exact key so it can never win
the save race and replace a full tree with its single-crate one.
Also: one scripts/ci/ensure-sccache.sh replaces ten copy-pasted bootstrap blocks that
had already drifted into two dialects (GNU tar --wildcards vs bsdtar), every Rust job
now ends with --show-stats so a cache regression is visible instead of just "CI got
slower", and deb.yml's web install joins every other CI install on --ignore-scripts.
No behaviour change to any artifact: same compilers, same flags, same outputs.
This commit is contained in:
@@ -63,6 +63,15 @@ env:
|
||||
SCCACHE_REGION: home-central
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_SECRET_ACCESS_KEY }}
|
||||
# The C/C++ half of the cache. The one that pays here is the CMake-built vendored libopus
|
||||
# (audiopus_sys), which kit/build.gradle.kts drives through cargo-ndk once per ABI — three
|
||||
# from-scratch libopus builds per run until now. The per-ABI compilers come from the NDK via
|
||||
# cargo-ndk's own CC_<android-triple> vars, which this does not touch; CC_x86_64_unknown_linux_gnu
|
||||
# covers only the HOST build scripts and proc macros.
|
||||
CMAKE_C_COMPILER_LAUNCHER: sccache
|
||||
CMAKE_CXX_COMPILER_LAUNCHER: sccache
|
||||
CC_x86_64_unknown_linux_gnu: sccache cc
|
||||
CXX_x86_64_unknown_linux_gnu: sccache c++
|
||||
# sccache and incremental compilation are mutually exclusive; CI wants the shared
|
||||
# cache, dev boxes keep incremental.
|
||||
CARGO_INCREMENTAL: "0"
|
||||
|
||||
@@ -85,17 +85,10 @@ jobs:
|
||||
dirname "$RUSTUP" >> "$GITHUB_PATH"
|
||||
"$RUSTUP" target add aarch64-apple-darwin x86_64-apple-darwin
|
||||
|
||||
# Shared compile cache. ~/.local/bin is on the runner daemon's PATH; GITHUB_PATH is
|
||||
# belt-and-braces. bsdtar (macOS) globs by default — no --wildcards.
|
||||
# Shared compile cache. The script handles the macOS side (user-prefix install +
|
||||
# GITHUB_PATH, bsdtar globbing) — see scripts/ci/ensure-sccache.sh.
|
||||
- name: sccache (self-healing install)
|
||||
run: |
|
||||
if ! command -v sccache >/dev/null; then
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-aarch64-apple-darwin.tar.gz \
|
||||
| tar -xz --strip-components=1 -C "$HOME/.local/bin" '*/sccache'
|
||||
fi
|
||||
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
# `punktfunk-core` now decodes Opus in-core for the Apple client (surround), pulling
|
||||
# `audiopus_sys`, which builds a vendored static libopus via CMake when pkg-config can't find a
|
||||
@@ -155,17 +148,10 @@ jobs:
|
||||
"$RUSTUP" target add aarch64-apple-darwin x86_64-apple-darwin \
|
||||
aarch64-apple-ios aarch64-apple-ios-sim x86_64-apple-ios
|
||||
|
||||
# Shared compile cache. ~/.local/bin is on the runner daemon's PATH; GITHUB_PATH is
|
||||
# belt-and-braces. bsdtar (macOS) globs by default — no --wildcards.
|
||||
# Shared compile cache. The script handles the macOS side (user-prefix install +
|
||||
# GITHUB_PATH, bsdtar globbing) — see scripts/ci/ensure-sccache.sh.
|
||||
- name: sccache (self-healing install)
|
||||
run: |
|
||||
if ! command -v sccache >/dev/null; then
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-aarch64-apple-darwin.tar.gz \
|
||||
| tar -xz --strip-components=1 -C "$HOME/.local/bin" '*/sccache'
|
||||
fi
|
||||
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
# See the swift job: audiopus_sys (via the in-core Opus decode) builds vendored libopus with CMake.
|
||||
- name: CMake (for the vendored libopus audiopus_sys builds)
|
||||
|
||||
@@ -81,6 +81,14 @@ env:
|
||||
SCCACHE_REGION: home-central
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_SECRET_ACCESS_KEY }}
|
||||
# The C/C++ half of the cache (aws-lc-sys, vendored libopus, openh264's C++). Safe at workflow
|
||||
# level — no cross-compiling job here (see ci.yml's `rust` job for that trap). NOTE these ALSO
|
||||
# have to be named in the makepkg step's `sudo -u builder env …` list: sudo's env_reset drops
|
||||
# everything not listed, which is why the sccache vars are already spelled out there.
|
||||
CMAKE_C_COMPILER_LAUNCHER: sccache
|
||||
CMAKE_CXX_COMPILER_LAUNCHER: sccache
|
||||
CC_x86_64_unknown_linux_gnu: sccache cc
|
||||
CXX_x86_64_unknown_linux_gnu: sccache c++
|
||||
CARGO_INCREMENTAL: "0"
|
||||
|
||||
jobs:
|
||||
@@ -252,8 +260,19 @@ jobs:
|
||||
SCCACHE_BUCKET="$SCCACHE_BUCKET" SCCACHE_ENDPOINT="$SCCACHE_ENDPOINT" \
|
||||
SCCACHE_REGION="$SCCACHE_REGION" \
|
||||
AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID" AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY" \
|
||||
CMAKE_C_COMPILER_LAUNCHER="$CMAKE_C_COMPILER_LAUNCHER" \
|
||||
CMAKE_CXX_COMPILER_LAUNCHER="$CMAKE_CXX_COMPILER_LAUNCHER" \
|
||||
CC_x86_64_unknown_linux_gnu="$CC_x86_64_unknown_linux_gnu" \
|
||||
CXX_x86_64_unknown_linux_gnu="$CXX_x86_64_unknown_linux_gnu" \
|
||||
makepkg -f -d --holdver
|
||||
ls -lh "$GITHUB_WORKSPACE/dist"
|
||||
# Visibility only. The stats have to be read as the SAME user that ran the compiles —
|
||||
# sccache keeps its stats in a per-user server process, so a root `--show-stats` here
|
||||
# would report an idle server and zero everything.
|
||||
sudo -u builder env SCCACHE_BUCKET="$SCCACHE_BUCKET" SCCACHE_ENDPOINT="$SCCACHE_ENDPOINT" \
|
||||
SCCACHE_REGION="$SCCACHE_REGION" \
|
||||
AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID" AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY" \
|
||||
sccache --show-stats || true
|
||||
|
||||
# The host must ship a VERSIONED libav soname dep, and nothing else in this pipeline proves
|
||||
# it. packaging/arch/PKGBUILD lists bare `libavcodec.so` etc. and relies on makepkg rewriting
|
||||
|
||||
@@ -38,14 +38,9 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
- name: Prep
|
||||
run: |
|
||||
|
||||
+56
-18
@@ -23,6 +23,12 @@ on:
|
||||
# Shared compile cache: sccache -> RustFS S3 (storage.unom.io, LAN-pinned via ci-core's
|
||||
# unbound). Keys include compiler hash + target + flags, so cross-OS/arch entries can
|
||||
# never collide; every Rust job on every host feeds and reads one warm cache.
|
||||
#
|
||||
# RUSTC_WRAPPER covers RUST compilations and nothing else. The C/C++ half of this workspace —
|
||||
# aws-lc-sys, openh264-sys2's vendored C++, the CMake-built libopus behind audiopus_sys, pyrowave —
|
||||
# was paid in full on every run until the CMAKE_*_COMPILER_LAUNCHER / CC_* wiring below existed.
|
||||
# Linking is the third phase and is cacheable by nothing: that one is addressed in the builder
|
||||
# images with mold (ci/cargo-config-mold.toml).
|
||||
env:
|
||||
RUSTC_WRAPPER: sccache
|
||||
SCCACHE_BUCKET: unom-ci-sccache
|
||||
@@ -30,6 +36,11 @@ env:
|
||||
SCCACHE_REGION: home-central
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_SECRET_ACCESS_KEY }}
|
||||
# Route CMake-driven C/C++ through the same cache. Safe at workflow level: it names no
|
||||
# triple, and cmake-rs overrides it per-invocation with a `-D` flag when cc-rs reports a
|
||||
# wrapper, so the two can never double-wrap into `sccache sccache cc`.
|
||||
CMAKE_C_COMPILER_LAUNCHER: sccache
|
||||
CMAKE_CXX_COMPILER_LAUNCHER: sccache
|
||||
# sccache and incremental compilation are mutually exclusive; CI wants the shared
|
||||
# cache, dev boxes keep incremental.
|
||||
CARGO_INCREMENTAL: "0"
|
||||
@@ -40,18 +51,25 @@ jobs:
|
||||
container:
|
||||
image: 192.168.1.58:5010/punktfunk-rust-ci:latest
|
||||
timeout-minutes: 90
|
||||
env:
|
||||
# cc-rs recognises `sccache` as a compiler wrapper when it leads CC/CXX, and cmake-rs then
|
||||
# forwards it as -DCMAKE_C_COMPILER_LAUNCHER, so this covers both build-script styles.
|
||||
#
|
||||
# ⚠ JOB-scoped, NOT workflow-scoped, and it must stay that way: the `rust-arm64` job below
|
||||
# runs in the cross image, which sets CC_x86_64_unknown_linux_gnu=/usr/local/bin/pf-host-cc
|
||||
# (ci/rust-ci-arm64cross.Dockerfile) — a wrapper that strips arm64 include dirs off
|
||||
# HOST-targeted compiles so ffmpeg-sys-next's probe resolves against the amd64 headers.
|
||||
# Setting this at workflow level would silently overwrite that wrapper and break the cross
|
||||
# build in a way that looks like a header mismatch, not a CI config error.
|
||||
CC_x86_64_unknown_linux_gnu: sccache cc
|
||||
CXX_x86_64_unknown_linux_gnu: sccache c++
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
# punktfunk-client-linux link deps. Also baked into rust-ci.Dockerfile — but ci.yml
|
||||
# runs against the image from the PREVIOUS push (docker.yml bootstrap note), so this
|
||||
@@ -105,8 +123,17 @@ jobs:
|
||||
# out of disk mid-build and actions/cache saved a truncated target/ (a dep's .rmeta
|
||||
# went missing -> E0463 "can't find crate"). A suffix bump wouldn't help — restore-keys
|
||||
# would fall back to the poisoned prefix — so the prefix itself is versioned.
|
||||
key: cargo-target-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-v3-${{ env.rustc }}-
|
||||
#
|
||||
# `-debug-`: THIS JOB BUILDS DEBUG. deb.yml builds RELEASE and used to share this exact
|
||||
# key, with a comment claiming the release build "reuses ci.yml's clean artifacts" — it
|
||||
# never could. actions/cache is first-saver-wins on an exact key and this job is the
|
||||
# faster of the two, so what landed under the shared key was always a debug-only target/
|
||||
# (target/debug, no target/release). deb.yml restored a tree containing nothing it could
|
||||
# use and, because the key was already taken, never got to save its own — so every
|
||||
# release build re-linked from scratch, forever. Splitting the profiles into separate key
|
||||
# families is the fix; do not merge them again, however tempting the dedupe looks.
|
||||
key: cargo-target-debug-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-debug-v3-${{ env.rustc }}-
|
||||
|
||||
- name: Format
|
||||
run: cargo fmt --all --check
|
||||
@@ -220,14 +247,9 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
- name: Cache keys
|
||||
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
|
||||
@@ -242,8 +264,16 @@ jobs:
|
||||
with:
|
||||
path: target
|
||||
# Its OWN prefix: aarch64 artifacts must never share the amd64 jobs' target cache.
|
||||
key: cargo-target-arm64-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-arm64-v1-${{ env.rustc }}-
|
||||
#
|
||||
# `-debug-` in the name is load-bearing. This job builds DEBUG (clippy + a
|
||||
# `cargo build`), while deb.yml's arm64 leg builds RELEASE into the same
|
||||
# target/aarch64-unknown-linux-gnu tree. They used to share this exact key, and
|
||||
# actions/cache is first-saver-wins on an exact key: this job finishes in ~1.5 min and
|
||||
# saved a debug-only tree, so the .deb leg's release artifacts were NEVER persisted and
|
||||
# it re-linked everything from sccache on every run. Same disease as the amd64 pair —
|
||||
# see the note on deb.yml's `cargo-target-release-v1-` key.
|
||||
key: cargo-target-arm64-debug-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-arm64-debug-v1-${{ env.rustc }}-
|
||||
|
||||
- name: Clippy for aarch64 (deny warnings)
|
||||
run: |
|
||||
@@ -259,6 +289,14 @@ jobs:
|
||||
cargo build --release --target aarch64-unknown-linux-gnu --locked \
|
||||
-p punktfunk-client-session --no-default-features
|
||||
|
||||
# Visibility only — but the ONLY way to tell a warm cache from a cold one after the fact.
|
||||
# Every Rust job in this repo ends with this line for that reason; a hit rate that quietly
|
||||
# collapses (a toolchain bump, a flag change, an S3 outage) is otherwise invisible and just
|
||||
# looks like "CI got slower".
|
||||
- name: sccache stats (visibility only)
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
web:
|
||||
runs-on: ubuntu-24.04
|
||||
container:
|
||||
|
||||
+78
-32
@@ -70,6 +70,11 @@ env:
|
||||
SCCACHE_REGION: home-central
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_SECRET_ACCESS_KEY }}
|
||||
# CMake-driven C/C++ through the same cache (aws-lc-sys, the vendored libopus). Workflow level
|
||||
# is safe — it names no triple; the CC_*/CXX_* half is per-job below, because the arm64 leg's
|
||||
# image needs its own CC_x86_64 wrapper. See ci.yml's `rust` job for the full note.
|
||||
CMAKE_C_COMPILER_LAUNCHER: sccache
|
||||
CMAKE_CXX_COMPILER_LAUNCHER: sccache
|
||||
# sccache and incremental compilation are mutually exclusive; CI wants the shared
|
||||
# cache, dev boxes keep incremental.
|
||||
CARGO_INCREMENTAL: "0"
|
||||
@@ -80,18 +85,18 @@ jobs:
|
||||
container:
|
||||
image: 192.168.1.58:5010/punktfunk-rust-ci:latest
|
||||
timeout-minutes: 90
|
||||
env:
|
||||
# ⚠ Job-scoped, never workflow-scoped: build-publish-client-arm64 runs in the cross image,
|
||||
# whose own CC_x86_64_unknown_linux_gnu=pf-host-cc must survive. See ci.yml's `rust` job.
|
||||
CC_x86_64_unknown_linux_gnu: sccache cc
|
||||
CXX_x86_64_unknown_linux_gnu: sccache c++
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
- name: Version + channel
|
||||
# vX.Y.Z tag -> X.Y.Z, published to the `stable` apt distribution (a real release).
|
||||
@@ -129,7 +134,9 @@ jobs:
|
||||
apt-get install -y --no-install-recommends dpkg-dev python3 \
|
||||
libgtk-4-dev libadwaita-1-dev libsdl3-dev
|
||||
|
||||
# Share ci.yml's cache keys so the release build reuses its registry + target artifacts.
|
||||
# The cargo-home (registry/git) cache IS shared with ci.yml — those are pure downloads,
|
||||
# profile-independent, and deduping them across the fleet is the whole point. The target
|
||||
# cache is NOT; see below.
|
||||
- name: Cache keys
|
||||
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
|
||||
- uses: actions/cache@v4
|
||||
@@ -142,10 +149,20 @@ jobs:
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: target
|
||||
# -v3-: bypass a target cache poisoned by a disk-full build (see ci.yml). Shares the
|
||||
# key with ci.yml so the release build reuses its clean artifacts.
|
||||
key: cargo-target-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-v3-${{ env.rustc }}-
|
||||
# -v3-: bypass a target cache poisoned by a disk-full build (see ci.yml).
|
||||
#
|
||||
# `-release-`, and NOT ci.yml's key. This step used to read
|
||||
# `cargo-target-v3-<rustc>-<lock>` — byte-identical to ci.yml's — under a comment saying
|
||||
# the release build "reuses its clean artifacts". It never did, and could not: ci.yml
|
||||
# builds DEBUG, this job builds RELEASE, and actions/cache is first-saver-wins on an
|
||||
# exact key. ci.yml's `rust` job finishes in ~6 min against this job's ~7-8, so ci.yml
|
||||
# always won the save, this job always restored a target/ with an empty target/release,
|
||||
# and — the expensive half — its own release artifacts were then never persisted,
|
||||
# because the key it would have saved under was already taken. Every canary .deb has
|
||||
# therefore been a from-scratch release build (sccache-assisted, but every link and
|
||||
# every build script re-run) for as long as both keys have existed.
|
||||
key: cargo-target-release-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-release-v3-${{ env.rustc }}-
|
||||
|
||||
- name: Build release clients
|
||||
env:
|
||||
@@ -176,7 +193,12 @@ jobs:
|
||||
}
|
||||
export PATH="$HOME/.bun/bin:$PATH"
|
||||
cd web
|
||||
bun install --frozen-lockfile
|
||||
# --ignore-scripts, like every other web install in CI (ci.yml, web-screenshots.yml,
|
||||
# windows-host.yml, the SDK installs). This was the ONE site still running lifecycle
|
||||
# scripts, and web's `postinstall` is `bun2nix -o bun.nix` — a Nix codegen step this job
|
||||
# neither consumes nor commits, whose only effect here is to make the install depend on
|
||||
# bun2nix resolving. `build` re-runs its own `prebuild` codegen regardless.
|
||||
bun install --frozen-lockfile --ignore-scripts
|
||||
bun run build
|
||||
if ! grep -q 'Bun\.serve' .output/server/index.mjs; then
|
||||
echo "ERROR: web build is not a bun bundle — need the 'bun' preset + custom entry"; exit 1
|
||||
@@ -198,6 +220,13 @@ jobs:
|
||||
# The plugin/script runner (bun-bundled Effect SDK) — same vendored-bun mechanics.
|
||||
VERSION="$VERSION" BUN_BIN="$(command -v bun || true)" bash packaging/debian/build-scripting-deb.sh
|
||||
|
||||
# Visibility only. With the target cache now actually saving release artifacts (see the
|
||||
# cache key note above), this is how a regression in that arrangement becomes visible:
|
||||
# a run that suddenly reports thousands of misses is a cache that stopped restoring.
|
||||
- name: sccache stats (visibility only)
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
- name: Publish to the Gitea apt registry
|
||||
env:
|
||||
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
@@ -242,18 +271,17 @@ jobs:
|
||||
container:
|
||||
image: 192.168.1.58:5010/punktfunk-rust-ci-noble:latest
|
||||
timeout-minutes: 90
|
||||
env:
|
||||
# ⚠ Job-scoped — see build-publish above and ci.yml's `rust` job.
|
||||
CC_x86_64_unknown_linux_gnu: sccache cc
|
||||
CXX_x86_64_unknown_linux_gnu: sccache c++
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
- name: Version + channel
|
||||
run: |
|
||||
@@ -289,9 +317,12 @@ jobs:
|
||||
with:
|
||||
path: target
|
||||
# Own key: this target dir is built against 24.04's glibc/toolchain and must NOT share
|
||||
# ci.yml's 26.04 target cache (mixing would poison both).
|
||||
key: cargo-target-noble-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-noble-v1-${{ env.rustc }}-
|
||||
# ci.yml's 26.04 target cache (mixing would poison both). It is also the only job using
|
||||
# this prefix, so — unlike the amd64/arm64 pairs above — it has always been able to save
|
||||
# and restore its own release artifacts. `-release-` is spelled out anyway so the whole
|
||||
# file reads consistently and nobody "unifies" it back into a shared key later.
|
||||
key: cargo-target-noble-release-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-noble-release-v1-${{ env.rustc }}-
|
||||
|
||||
- name: Build release host
|
||||
env:
|
||||
@@ -326,6 +357,12 @@ jobs:
|
||||
run: |
|
||||
VERSION="$VERSION" BUNDLE_FFMPEG=1 bash packaging/debian/build-deb.sh
|
||||
|
||||
# Visibility only — placed here, right after the last compile, rather than at the end of the
|
||||
# job: the gamescope gate below must stay the final step (see its own note).
|
||||
- name: sccache stats (visibility only)
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
# Read the capability matrix out of the BUILT .deb before it is published. dpkg carries no
|
||||
# capability metadata — the postinst applies them — so this reads the postinst that will
|
||||
# actually run on a user's box, plus the payload. 0.26.0-1 granted the host cap_sys_nice=ep
|
||||
@@ -476,14 +513,14 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
#
|
||||
# NOTE this job deliberately sets no CC_x86_64_unknown_linux_gnu: the cross image already
|
||||
# points it at /usr/local/bin/pf-host-cc, which is what keeps ffmpeg-sys-next's HOST probe
|
||||
# from picking up arm64 include dirs. The target-side compiles go through
|
||||
# CC_aarch64_unknown_linux_gnu (also set by the image) and are not sccache-wrapped.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
# Byte-identical to build-publish's version step (pf-version.sh is deterministic per
|
||||
# commit), so the arm64 package always shares the amd64 version line.
|
||||
@@ -520,8 +557,13 @@ jobs:
|
||||
path: target
|
||||
# Its OWN key — these are aarch64 artifacts under target/aarch64-unknown-linux-gnu/
|
||||
# and must never share the amd64 jobs' target cache.
|
||||
key: cargo-target-arm64-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-arm64-v1-${{ env.rustc }}-
|
||||
#
|
||||
# `-release-`: this key was previously identical to ci.yml's `rust-arm64` key, which
|
||||
# builds DEBUG (clippy) and finishes in ~1.5 min against this job's ~5. Exactly the
|
||||
# amd64 collision described on the release key above — ci.yml won every save, this job
|
||||
# restored a tree with no release artifacts and could never persist its own.
|
||||
key: cargo-target-arm64-release-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-arm64-release-v1-${{ env.rustc }}-
|
||||
|
||||
- name: Build the arm64 client .deb
|
||||
env:
|
||||
@@ -534,6 +576,10 @@ jobs:
|
||||
readelf -h target/aarch64-unknown-linux-gnu/release/punktfunk-session \
|
||||
| grep -q AArch64 || { echo "ERROR: session binary is not AArch64"; exit 1; }
|
||||
|
||||
- name: sccache stats (visibility only)
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
- name: Publish to the Gitea apt registry
|
||||
env:
|
||||
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
|
||||
@@ -29,6 +29,12 @@ env:
|
||||
SCCACHE_REGION: home-central
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_SECRET_ACCESS_KEY }}
|
||||
# The C/C++ half of the cache — same wiring as ci.yml/deb.yml. Safe at workflow level: no
|
||||
# cross-compiling job here.
|
||||
CMAKE_C_COMPILER_LAUNCHER: sccache
|
||||
CMAKE_CXX_COMPILER_LAUNCHER: sccache
|
||||
CC_x86_64_unknown_linux_gnu: sccache cc
|
||||
CXX_x86_64_unknown_linux_gnu: sccache c++
|
||||
# sccache and incremental compilation are mutually exclusive; CI wants the shared
|
||||
# cache, dev boxes keep incremental.
|
||||
CARGO_INCREMENTAL: "0"
|
||||
@@ -45,14 +51,9 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
# Client link deps (baked into the image; kept here so the job is green across image
|
||||
# rebuilds — a no-op once present) PLUS the headless-render extras: a virtual X server,
|
||||
@@ -67,7 +68,8 @@ jobs:
|
||||
libgl1-mesa-dri mesa-vulkan-drivers \
|
||||
adwaita-icon-theme fonts-cantarell fonts-dejavu-core
|
||||
|
||||
# Reuse the workspace cargo caches (same keys as ci.yml/deb.yml).
|
||||
# Reuse the workspace cargo caches. The cargo-home (download) cache is shared verbatim —
|
||||
# it is profile-independent.
|
||||
- name: Cache keys
|
||||
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
|
||||
- uses: actions/cache@v4
|
||||
@@ -80,8 +82,20 @@ jobs:
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: target
|
||||
key: cargo-target-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: cargo-target-v3-${{ env.rustc }}-
|
||||
# This job builds RELEASE (see the build step) in the same image and target layout as
|
||||
# deb.yml's `build-publish`, so it wants THAT tree — it used to name ci.yml's key, which
|
||||
# holds a debug build and gave it nothing. (Third participant in the collision documented
|
||||
# on ci.yml's `cargo-target-debug-v3-` key.)
|
||||
#
|
||||
# Its OWN exact key with deb's prefix as a FALLBACK restore-key, deliberately: both
|
||||
# workflows run on a v* tag, and an exact-key match would make them race for the single
|
||||
# save slot — this job builds one crate, so if it won that race it would replace deb's
|
||||
# full release tree with a nearly empty one for the rest of the lockfile's life. This way
|
||||
# it always READS the warm tree and never blocks the job that fills it.
|
||||
key: cargo-target-shots-release-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
|
||||
restore-keys: |
|
||||
cargo-target-shots-release-v1-${{ env.rustc }}-
|
||||
cargo-target-release-v3-${{ env.rustc }}-
|
||||
|
||||
- name: Build client
|
||||
run: cargo build --release -p punktfunk-client-linux --locked
|
||||
|
||||
@@ -51,6 +51,14 @@ env:
|
||||
SCCACHE_REGION: home-central
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_SECRET_ACCESS_KEY }}
|
||||
# The C/C++ half of the cache (aws-lc-sys, the vendored libopus, openh264's C++). Safe at
|
||||
# workflow level here: unlike ci.yml/deb.yml this workflow has no cross-compiling job whose
|
||||
# image sets its own CC_x86_64_unknown_linux_gnu. See ci.yml's `rust` job for that trap.
|
||||
# This matters twice per push — the f43 and f44 legs are the two longest jobs in the fleet.
|
||||
CMAKE_C_COMPILER_LAUNCHER: sccache
|
||||
CMAKE_CXX_COMPILER_LAUNCHER: sccache
|
||||
CC_x86_64_unknown_linux_gnu: sccache cc
|
||||
CXX_x86_64_unknown_linux_gnu: sccache c++
|
||||
# sccache and incremental compilation are mutually exclusive; CI wants the shared
|
||||
# cache, dev boxes keep incremental.
|
||||
CARGO_INCREMENTAL: "0"
|
||||
@@ -80,14 +88,9 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Shared compile cache (sccache -> RustFS S3 over the LAN). Baked into the builder
|
||||
# images; this fetch keeps the job green while the running :latest predates the bake.
|
||||
# images; this heals the job while the running :latest predates the bake.
|
||||
- name: sccache (no-op once the image bakes it)
|
||||
run: |
|
||||
command -v sccache >/dev/null 2>&1 || {
|
||||
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache'
|
||||
}
|
||||
sccache --version
|
||||
run: sh scripts/ci/ensure-sccache.sh
|
||||
|
||||
# rpmbuild + git archive need the checkout trusted; cache the crates download.
|
||||
# The client link deps are also baked into the fedora-rpm image, but this job runs
|
||||
@@ -151,6 +154,12 @@ jobs:
|
||||
# Recommends both). Both need bun (ensured in Prep).
|
||||
run: PF_VERSION="$PF_VERSION" PF_RELEASE="$PF_RELEASE" PF_WITH_WEB=1 PF_WITH_SCRIPTING=1 bash packaging/rpm/build-rpm.sh
|
||||
|
||||
# Visibility only — the two RPM legs are the longest jobs in the fleet, so a cache
|
||||
# regression here is the most expensive one to leave undetected.
|
||||
- name: sccache stats (visibility only)
|
||||
if: always()
|
||||
run: sccache --show-stats
|
||||
|
||||
# Signs with packages@unom.io (org secret) and self-verifies before publish. On a v* tag a
|
||||
# missing key FAILS the build rather than publishing unsigned RPMs into a gpgcheck=1 repo.
|
||||
- name: Sign RPMs
|
||||
|
||||
@@ -48,6 +48,11 @@ RUN pacman -Syu --noconfirm --needed \
|
||||
hwdata luajit seatd sdl2-compat vulkan-icd-loader \
|
||||
xcb-util-errors xcb-util-wm xorg-xwayland \
|
||||
meson glm wayland-protocols benchmark libxcursor \
|
||||
# mold: link-phase accelerator (sccache cannot cache linking). makepkg links the release
|
||||
# host, client, worker and tray on every arch.yml run. Wired via cargo-config-mold.toml
|
||||
# below. It does NOT affect the gamescope companion leg — that is meson + its own linker,
|
||||
# and its `-static-libstdc++` link is untouched.
|
||||
mold \
|
||||
&& pacman -Scc --noconfirm
|
||||
|
||||
# bun builds the punktfunk-web console + the punktfunk-scripting runner AND is vendored
|
||||
@@ -64,3 +69,16 @@ ARG SCCACHE_VERSION=0.10.0
|
||||
RUN curl -fsSL "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache' \
|
||||
&& sccache --version
|
||||
|
||||
# CARGO_HOME is declared here only so this image agrees with what arch.yml already sets at job
|
||||
# level (and so `cargo` finds the config below when the image is used by hand). The workflow still
|
||||
# passes CARGO_HOME explicitly across the `sudo -u builder env …` boundary, which strips ambient
|
||||
# env — that is why the C/C++ sccache wiring has to be re-exported there by name while THIS file,
|
||||
# being a file, crosses the boundary for free.
|
||||
ENV CARGO_HOME=/usr/local/cargo
|
||||
RUN mkdir -p /usr/local/cargo && chmod -R a+w /usr/local/cargo
|
||||
|
||||
# Link x86_64 with mold — see cargo-config-mold.toml's header for the rustflags traps, and
|
||||
# rust-ci.Dockerfile for why the `mold --version` assertion sits next to the COPY.
|
||||
COPY cargo-config-mold.toml /usr/local/cargo/config.toml
|
||||
RUN mold --version && test -r /usr/local/cargo/config.toml
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# Installed as $CARGO_HOME/config.toml in every Linux CI builder image (ci/*.Dockerfile).
|
||||
#
|
||||
# WHAT: link the x86_64 Linux targets with mold instead of GNU ld. Linking is the one phase of a
|
||||
# Rust build that sccache CANNOT cache — every job relinks punktfunk-host, punktfunk-client-linux,
|
||||
# punktfunk-client-session, punktfunk-cli, pf-update and punktfunk-encode-worker from scratch on
|
||||
# every run, and the packaging legs (deb/rpm/arch) do it for release binaries with full debug info.
|
||||
# mold is the only lever that touches that phase.
|
||||
#
|
||||
# ⚠ THE TRAP THIS FILE HAS TO STAY CLEAR OF — read before editing, and before adding rustflags
|
||||
# anywhere else in this repo:
|
||||
#
|
||||
# 1. A `RUSTFLAGS` ENVIRONMENT VARIABLE OVERRIDES CONFIG RUSTFLAGS ENTIRELY. It does not merge
|
||||
# and it does not append. Any job that sets RUSTFLAGS silently loses mold here (it still
|
||||
# builds — just with the default linker), and, far worse, would lose the aarch64
|
||||
# `--cfg aes_armv8` / `--cfg polyval_armv8` flags from the workspace's own .cargo/config.toml,
|
||||
# which are worth a measured ~3x on the decrypt path. audit.yml's miri gf8 step is the one
|
||||
# place in the repo that sets RUSTFLAGS, and its comment already carries this warning; keep it
|
||||
# that way. Never "simplify" this file into a RUSTFLAGS export.
|
||||
#
|
||||
# 2. CONFIG FILES MERGE PER KEY, HIGHEST-PRECEDENCE FILE WINS — they do not concatenate. The
|
||||
# workspace's .cargo/config.toml outranks this one ($CARGO_HOME is the LOWEST precedence).
|
||||
# Today that is harmless because the two files touch DISJOINT keys: the workspace file defines
|
||||
# only `target.'cfg(target_arch = "aarch64")'.rustflags`, this one only
|
||||
# `target.x86_64-unknown-linux-gnu.rustflags`, and cargo JOINS a matching cfg-spec table with
|
||||
# the triple table rather than picking one. But the moment someone adds an x86_64 rustflags
|
||||
# entry to the workspace .cargo/config.toml, IT WINS and mold silently stops being used here.
|
||||
# If that ever happens, move the link-arg into that file instead of duplicating it.
|
||||
#
|
||||
# 3. aarch64 IS DELIBERATELY NOT WIRED. The cross image links with aarch64-linux-gnu-gcc against a
|
||||
# multiarch sysroot (ci/rust-ci-arm64cross.Dockerfile); pointing that driver at mold is a
|
||||
# separate thing to prove, and those legs are already the fast ones (~1.5 min of clippy, ~5 min
|
||||
# for the arm64 .deb). Add it only with a measurement, and in a commit of its own.
|
||||
#
|
||||
# Requires GCC >= 12.1 (or clang) for `-fuse-ld=mold`; every base here ships far newer. mold itself
|
||||
# is installed in the same Dockerfile layer that copies this file, so an image can never carry the
|
||||
# flag without the linker — see the `mold --version` assertion there.
|
||||
#
|
||||
# NOTE this affects the HOST-targeted compiles of build scripts and proc macros too (they are
|
||||
# x86_64-unknown-linux-gnu), which is exactly what we want: those link constantly and are pure
|
||||
# overhead.
|
||||
[target.x86_64-unknown-linux-gnu]
|
||||
rustflags = ["-C", "link-arg=-fuse-ld=mold"]
|
||||
@@ -22,6 +22,12 @@ RUN dnf -y install \
|
||||
rpm-build rpmdevtools systemd-rpm-macros git tar gzip nodejs unzip \
|
||||
# build toolchain + bindgen
|
||||
gcc gcc-c++ clang clang-devel cmake nasm pkgconf-pkg-config curl ca-certificates \
|
||||
# mold: link-phase accelerator (sccache cannot cache linking). This image links the release
|
||||
# host, client, worker and tray on every rpm.yml run, TWICE per push (f43 + f44). Wired via
|
||||
# cargo-config-mold.toml below. Note the linker DRIVER is unchanged — still gcc, so Fedora's
|
||||
# default `-Wl,--build-id` still reaches the link and rpmbuild's debuginfo extraction (which
|
||||
# hard-requires a build-id) behaves exactly as before; mold implements --build-id natively.
|
||||
mold \
|
||||
# ffmpeg (NVENC), capture/audio/display link deps
|
||||
ffmpeg-devel pipewire-devel wayland-devel libxkbcommon-devel opus-devel \
|
||||
mesa-libGL-devel mesa-libgbm-devel \
|
||||
@@ -76,3 +82,8 @@ ARG SCCACHE_VERSION=0.10.0
|
||||
RUN curl -fsSL "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache' \
|
||||
&& sccache --version
|
||||
|
||||
# Link x86_64 with mold — see cargo-config-mold.toml's header for the rustflags traps, and
|
||||
# rust-ci.Dockerfile for why the `mold --version` assertion sits next to the COPY.
|
||||
COPY cargo-config-mold.toml /usr/local/cargo/config.toml
|
||||
RUN mold --version && test -r /usr/local/cargo/config.toml
|
||||
|
||||
@@ -26,6 +26,9 @@ ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
# toolchain + bindgen; nodejs runs the JS actions (checkout/cache); unzip for the rustup installer's deps
|
||||
build-essential clang libclang-dev pkg-config cmake git curl ca-certificates nodejs unzip \
|
||||
# mold: link-phase accelerator (sccache cannot cache linking). This image links the release
|
||||
# host + encode worker on every deb.yml run. Wired via cargo-config-mold.toml below.
|
||||
mold \
|
||||
# .deb assembly: dpkg-shlibdeps/dpkg-deb; patchelf repoints the binary's rpath at the bundled FFmpeg
|
||||
dpkg-dev patchelf \
|
||||
# FFmpeg 8 build deps: nasm (asm), VAAPI (libva/libdrm) so the built libav* keep the AMD/Intel
|
||||
@@ -99,3 +102,10 @@ ARG SCCACHE_VERSION=0.10.0
|
||||
RUN curl -fsSL "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache' \
|
||||
&& sccache --version
|
||||
|
||||
# Link x86_64 with mold — see cargo-config-mold.toml's header for the rustflags traps, and
|
||||
# rust-ci.Dockerfile for why the `mold --version` assertion sits next to the COPY.
|
||||
# ⚠ This does NOT touch the from-source FFmpeg built above: that is a plain ./configure && make in
|
||||
# an earlier layer, linked by GNU ld exactly as before. Only cargo's links move to mold.
|
||||
COPY cargo-config-mold.toml /usr/local/cargo/config.toml
|
||||
RUN mold --version && test -r /usr/local/cargo/config.toml
|
||||
|
||||
@@ -13,6 +13,9 @@ ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
# toolchain + bindgen; nodejs runs the JS actions (checkout/cache); unzip is for the bun installer
|
||||
build-essential clang libclang-dev pkg-config cmake git curl ca-certificates nodejs unzip \
|
||||
# mold: the link-phase accelerator. Linking is the one thing sccache cannot cache, and this
|
||||
# image relinks the whole workspace on every job. Wired via cargo-config-mold.toml below.
|
||||
mold \
|
||||
# ffmpeg-next 9, built against whatever libav* 26.04 ships (FFmpeg 8 / libavcodec 62 today).
|
||||
# The crate major is a CEILING — ffmpeg-sys-next 9 spans libavcodec 56..63 — so this image does
|
||||
# not need to move in lockstep with Arch's FFmpeg 9; it just links what the distro has.
|
||||
@@ -61,3 +64,12 @@ ARG SCCACHE_VERSION=0.10.0
|
||||
RUN curl -fsSL "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
||||
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache' \
|
||||
&& sccache --version
|
||||
|
||||
# Link x86_64 with mold (see the file's own header for the rustflags-precedence traps).
|
||||
#
|
||||
# The assertion is the point: an image carrying the flag but NOT the linker would fail every cargo
|
||||
# invocation in every consuming job, which is a catastrophic way to find out that a base image
|
||||
# renamed the package. `mold --version` fails the docker build instead, so nothing is pushed and
|
||||
# `:latest` keeps pointing at the previous working image — consumers never see it.
|
||||
COPY cargo-config-mold.toml /usr/local/cargo/config.toml
|
||||
RUN mold --version && test -r /usr/local/cargo/config.toml
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
#!/bin/sh
|
||||
# Ensure `sccache` is on PATH, self-healing on a runner/image that does not already carry it.
|
||||
#
|
||||
# WHY THIS EXISTS: this block was copy-pasted into ten jobs across six workflows (ci.yml x2,
|
||||
# deb.yml x3, rpm.yml, bench.yml, apple.yml x2, the Apple release leg), in two dialects that had
|
||||
# already drifted apart — the Linux copies pass `--wildcards` to GNU tar, the macOS copies must NOT
|
||||
# (bsdtar globs by default and rejects the flag). One copy per platform, here, so a version bump or
|
||||
# a mirror change is one edit rather than ten.
|
||||
#
|
||||
# The builder images (ci/*.Dockerfile) BAKE sccache, so on Linux this is a no-op in the normal case;
|
||||
# it stays because a job runs against the image from the PREVIOUS push (docker.yml's bootstrap lag),
|
||||
# and the macOS runner is a persistent host with no image at all.
|
||||
#
|
||||
# POSIX sh on purpose: Gitea's act_runner executes a step's `run:` under `sh -e` (dash) inside the
|
||||
# Linux job containers — no bashisms, no process substitution (see the shader-gate note in ci.yml
|
||||
# for what that cost the last time someone assumed bash).
|
||||
#
|
||||
# Usage: sh scripts/ci/ensure-sccache.sh
|
||||
set -e
|
||||
|
||||
# Keep in step with the ARG SCCACHE_VERSION in ci/*.Dockerfile — the images bake this same version,
|
||||
# and a job that heals to a DIFFERENT one would quietly split the shared cache's key universe in two
|
||||
# (sccache's cache keys are not versioned across incompatible releases).
|
||||
SCCACHE_VERSION="${SCCACHE_VERSION:-0.10.0}"
|
||||
|
||||
if command -v sccache >/dev/null 2>&1; then
|
||||
sccache --version
|
||||
exit 0
|
||||
fi
|
||||
|
||||
BASE="https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}"
|
||||
|
||||
case "$(uname -s)" in
|
||||
Darwin)
|
||||
# The macOS runner is a LaunchAgent in the user's Aqua session, not root — install into the
|
||||
# user prefix. ~/.local/bin is already on the runner daemon's PATH; GITHUB_PATH is
|
||||
# belt-and-braces for the steps that follow in THIS job.
|
||||
DEST="$HOME/.local/bin"
|
||||
mkdir -p "$DEST"
|
||||
case "$(uname -m)" in
|
||||
arm64|aarch64) ARCH=aarch64-apple-darwin ;;
|
||||
*) ARCH=x86_64-apple-darwin ;;
|
||||
esac
|
||||
# bsdtar globs by default and does not accept --wildcards.
|
||||
curl -fsSL "$BASE/sccache-v${SCCACHE_VERSION}-${ARCH}.tar.gz" \
|
||||
| tar -xz --strip-components=1 -C "$DEST" '*/sccache'
|
||||
chmod 0755 "$DEST/sccache"
|
||||
PATH="$DEST:$PATH"
|
||||
export PATH
|
||||
if [ -n "${GITHUB_PATH:-}" ]; then
|
||||
echo "$DEST" >> "$GITHUB_PATH"
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
# Linux job containers run as root; /usr/local/bin is on PATH already, so no GITHUB_PATH
|
||||
# dance is needed. The musl build is static — one binary serves the Ubuntu, Fedora and Arch
|
||||
# images alike.
|
||||
DEST=/usr/local/bin
|
||||
curl -fsSL "$BASE/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
||||
| tar -xz --wildcards --strip-components=1 -C "$DEST" '*/sccache'
|
||||
chmod 0755 "$DEST/sccache"
|
||||
;;
|
||||
esac
|
||||
|
||||
sccache --version
|
||||
Reference in New Issue
Block a user