enricobuehler
f1dc6c9f94
Three independent reasons Rust CI stayed slow despite sccache, fixed together because
they share the same measurement.
1. sccache only ever covered RUSTC. Every C/C++ dependency in the tree — aws-lc-sys,
openh264-sys2's vendored C++, the CMake-built libopus behind audiopus_sys — was
compiled from scratch on every job of every workflow. CMAKE_{C,CXX}_COMPILER_LAUNCHER
plus CC_/CXX_x86_64_unknown_linux_gnu route both build-script styles (cc-rs and
cmake-rs) through the same shared cache.
The CC_* vars are JOB-scoped in ci.yml and deb.yml, never workflow-scoped: the
arm64 cross image sets its own CC_x86_64_unknown_linux_gnu=pf-host-cc, the wrapper
that keeps ffmpeg-sys-next's host probe off the arm64 include dirs. Overwriting it
would surface as a header mismatch rather than as a CI config error.
2. Linking is cacheable by nothing, and these jobs relink the host, client, session,
cli, worker and tray on every run — twice per push for rpm (f43 + f44). The four
Linux builder images now install mold and carry a $CARGO_HOME/config.toml that uses
it for x86_64. aarch64 is deliberately left alone (cross driver, already-fast legs).
Each image asserts `mold --version` in its build, so an image can never ship the
flag without the linker: docker.yml goes red and :latest stays on the last good one.
3. THE EXPENSIVE ONE. ci.yml (debug) and deb.yml (release) named a byte-identical
target-cache key, under a comment claiming the release build reused ci.yml's
artifacts. It never could. actions/cache is first-saver-wins on an exact key and
ci.yml is the faster job, so the shared key always held a debug-only target/ — and,
worse, deb.yml could then never save its own, because the key was taken. Every
canary .deb has been a from-scratch release build for as long as both keys existed.
Same collision on the arm64 pair, and a third participant in
linux-client-screenshots.yml. Split into -debug-/-release- key families; that job
reads deb's tree via restore-keys but keeps its own exact key so it can never win
the save race and replace a full tree with its single-crate one.
Also: one scripts/ci/ensure-sccache.sh replaces ten copy-pasted bootstrap blocks that
had already drifted into two dialects (GNU tar --wildcards vs bsdtar), every Rust job
now ends with --show-stats so a cache regression is visible instead of just "CI got
slower", and deb.yml's web install joins every other CI install on --ignore-scripts.
No behaviour change to any artifact: same compilers, same flags, same outputs.
76 lines
4.5 KiB
Docker
76 lines
4.5 KiB
Docker
# CI builder for the Rust workspace — Ubuntu 26.04 to match the dev/host boxes
|
|
# (FFmpeg 8 / libavcodec 62, PipeWire 1.6). Used by .gitea/workflows/ci.yml as the job
|
|
# container; rebuilt+pushed by .gitea/workflows/docker.yml.
|
|
#
|
|
# docker build -f ci/rust-ci.Dockerfile -t punktfunk-rust-ci ci
|
|
#
|
|
# The workspace links real system libs at build time (CLAUDE.md "Pinned crate facts"):
|
|
# FFmpeg, PipeWire, Opus, GL/EGL/GBM — and libcuda, which has no real driver here; the
|
|
# zerocopy path only needs the symbols at link time, so a driver userspace package plus a
|
|
# libcuda.so -> libcuda.so.1 symlink stands in for it (CI never executes the CUDA path).
|
|
FROM ubuntu:26.04
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
# toolchain + bindgen; nodejs runs the JS actions (checkout/cache); unzip is for the bun installer
|
|
build-essential clang libclang-dev pkg-config cmake git curl ca-certificates nodejs unzip \
|
|
# mold: the link-phase accelerator. Linking is the one thing sccache cannot cache, and this
|
|
# image relinks the whole workspace on every job. Wired via cargo-config-mold.toml below.
|
|
mold \
|
|
# ffmpeg-next 9, built against whatever libav* 26.04 ships (FFmpeg 8 / libavcodec 62 today).
|
|
# The crate major is a CEILING — ffmpeg-sys-next 9 spans libavcodec 56..63 — so this image does
|
|
# not need to move in lockstep with Arch's FFmpeg 9; it just links what the distro has.
|
|
libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libavfilter-dev \
|
|
libavdevice-dev \
|
|
# capture / audio / display stacks (+xkbcommon for the wlr input backend)
|
|
libpipewire-0.3-dev libopus-dev libwayland-dev libxkbcommon-dev \
|
|
# zerocopy link deps (GL via libglvnd, EGL, GBM)
|
|
libgl-dev libegl-dev libgbm-dev \
|
|
# punktfunk-client-linux (GTK4/libadwaita shell, SDL3 gamepads)
|
|
libgtk-4-dev libadwaita-1-dev libsdl3-dev \
|
|
# No libvulkan-dev: nothing in the workspace compiles or links against Vulkan (pyrowave-sys
|
|
# bindgens its own vendored headers, and both host and client reach Vulkan through ash, which
|
|
# dlopens the loader), so neither the build nor deb.yml's dpkg-shlibdeps ever asks for it.
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# bun — builds the punktfunk-web console in deb.yml (which runs the web build in THIS image).
|
|
# ci.yml's web/docs jobs use the oven/bun image instead, so this is only for the deb job.
|
|
RUN curl -fsSL https://bun.sh/install | bash \
|
|
&& install -m0755 /root/.bun/bin/bun /usr/local/bin/bun \
|
|
&& bun --version
|
|
|
|
# libcuda link stub: the NVIDIA userspace library (no kernel module needed) provides
|
|
# every cuXxx symbol. On 26.04 the package already ships the libcuda.so dev symlink;
|
|
# -sf keeps this idempotent if a future package drops it again.
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends libnvidia-compute-580-server \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& ln -sf libcuda.so.1 /usr/lib/x86_64-linux-gnu/libcuda.so \
|
|
&& test -e /usr/lib/x86_64-linux-gnu/libcuda.so.1
|
|
|
|
# Toolchain shared across CI users (jobs may run as different uids).
|
|
ENV RUSTUP_HOME=/usr/local/rustup \
|
|
CARGO_HOME=/usr/local/cargo \
|
|
PATH=/usr/local/cargo/bin:$PATH
|
|
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
|
|
| sh -s -- -y --no-modify-path --profile minimal \
|
|
--component rustfmt,clippy \
|
|
&& chmod -R a+w "$RUSTUP_HOME" "$CARGO_HOME" \
|
|
&& rustc --version && cargo clippy --version && cargo fmt --version
|
|
|
|
# Shared compile cache: jobs set RUSTC_WRAPPER=sccache (backend = RustFS S3 on the LAN,
|
|
# see .gitea/workflows — the env lives there so dev use of this image stays uncached).
|
|
# musl build: one static binary serves the Ubuntu and Fedora images alike.
|
|
ARG SCCACHE_VERSION=0.10.0
|
|
RUN curl -fsSL "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/sccache-v${SCCACHE_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
|
|
| tar -xz --wildcards --strip-components=1 -C /usr/local/bin '*/sccache' \
|
|
&& sccache --version
|
|
|
|
# Link x86_64 with mold (see the file's own header for the rustflags-precedence traps).
|
|
#
|
|
# The assertion is the point: an image carrying the flag but NOT the linker would fail every cargo
|
|
# invocation in every consuming job, which is a catastrophic way to find out that a base image
|
|
# renamed the package. `mold --version` fails the docker build instead, so nothing is pushed and
|
|
# `:latest` keeps pointing at the previous working image — consumers never see it.
|
|
COPY cargo-config-mold.toml /usr/local/cargo/config.toml
|
|
RUN mold --version && test -r /usr/local/cargo/config.toml
|