build(ci): cross-compile and publish an aarch64 Linux client

Adds punktfunk-rust-ci-arm64cross (the rust-ci toolchain + an Ubuntu ports arm64 multiarch sysroot) and a deb.yml job that cross-builds the client package on the ordinary amd64 runner — no arm64 runner in the fleet and none needed. Client only: the Linux host encodes with NVENC/QSV/AMF, all x86. The apt registry keys pool entries by the .deb's own arch, so an arm64 box picks the package up with no client-side configuration. Three things this had to get right, each of which cost a failed build: * rustup target add must run against the toolchain rust-toolchain.toml PINS, not the image's default `stable` — otherwise the pinned toolchain has no aarch64 std and the build dies ~230 crates in on "can't find crate for core". Copying the pin file in also pre-downloads the toolchain every CI job currently re-fetches on first use. * ffmpeg-sys-next compiles a probe it intends to RUN, so it forces .target(HOST) while still passing the TARGET's include paths; the arm64 dir then shadows the host's own libc headers and the x86 compiler dies in bits/math-vector.h on NEON/SVE types. Prepending the host dir does NOT help — GCC drops a -I that duplicates a system directory, keeping its original later position — so ci/pf-host-cc strips target include dirs from host compiles instead. * the job hard-fails on a non-AArch64 session binary, rather than shipping an amd64 payload under an arm64 package name. skia needs no special handling: the aarch64-linux-gnu textlayout+vulkan prebuilt resolves, so arm64 ships the FULL client, OSD included — unlike Windows ARM64, which still builds --no-default-features. The cross image builds in its own docker.yml job (needs: build-push) since it is FROM punktfunk-rust-ci:latest and must not race the entry that publishes that base. rpm/Arch/Flatpak stay x86_64 for now — their builders have no arm64 sysroot, which is its own piece of work. Plan: punktfunk-planning design/embedded-arm64-client.md Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
parent 915fc3ef9e
commit 000f8b85b0
6 changed files with 285 additions and 9 deletions
@@ -1,9 +1,12 @@
-# Root build context is used only by web/Dockerfile, which needs web/ and
-# api/openapi.json. Allowlist those; keep everything else (target/, .git, crates)
-# out of the context upload.
+# The root build context is used by web/Dockerfile (which needs web/ and
+# api/openapi.json) and by ci/rust-ci-arm64cross.Dockerfile (which needs the toolchain
+# pin). Allowlist those; keep everything else (target/, .git, crates) out of the
+# context upload.
 *
 !web
 !api/openapi.json
+!rust-toolchain.toml
+!ci/pf-host-cc
 web/node_modules
 web/.output
 web/dist
@@ -1,9 +1,13 @@
 # Build the punktfunk .debs and publish them to Gitea's Debian package registry, so Ubuntu
-# boxes get new builds via `apt update && apt upgrade`. Two jobs, both publishing to the same
+# boxes get new builds via `apt update && apt upgrade`. Three jobs, all publishing to the same
 # apt distribution/component:
 #
 #   build-publish       — client + web + scripting, on the Ubuntu 26.04 rust-ci image (the client
 #                         needs 24.04-absent libs: SDL3, GTK4 ≥ 4.20).
+#   build-publish-client-arm64
+#                       — the same client package for arm64, CROSS-compiled on the same amd64
+#                         runner in the rust-ci-arm64cross image. No host counterpart: the Linux
+#                         host's encode stack is x86 (NVENC/QSV/AMF).
 #   build-publish-host  — the HOST, on the Ubuntu 24.04 rust-ci-noble image with a from-source
 #                         FFmpeg 8 BUNDLED into the .deb. This lowers the host's glibc floor to 2.39
 #                         and removes the hard `Depends: libavcodec62`, so the ONE host .deb installs
@@ -269,3 +273,94 @@ jobs:
          for DEB in dist/*.deb; do
            upsert_asset "$RID" "$DEB"
          done
+
+  # ---------------------------------------------------------------------------------------------
+  # The aarch64 CLIENT .deb. Cross-compiled on the ordinary amd64 runner in the
+  # punktfunk-rust-ci-arm64cross image (the rust-ci toolchain + an arm64 multiarch sysroot — see
+  # ci/rust-ci-arm64cross.Dockerfile); there is no arm64 runner in the fleet and none is needed.
+  # Client only, by decision: the Linux host encodes with NVENC/QSV/AMF, all x86.
+  # Publishes to the same distribution/component as the amd64 jobs — the apt registry keys pool
+  # entries by arch, so `apt` on an arm64 box picks this one up with no client-side configuration.
+  build-publish-client-arm64:
+    runs-on: ubuntu-24.04
+    container:
+      image: git.unom.io/unom/punktfunk-rust-ci-arm64cross:latest
+    timeout-minutes: 90
+    steps:
+      - uses: actions/checkout@v4
+
+      # Byte-identical to build-publish's version step (pf-version.sh is deterministic per
+      # commit), so the arm64 package always shares the amd64 version line.
+      - name: Version + channel
+        run: |
+          eval "$(bash scripts/ci/pf-version.sh)"
+          SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
+          case "$GITHUB_REF" in
+            refs/tags/v*) V="${GITHUB_REF_NAME#v}"; DIST=stable ;;
+            *)            V="${PF_BASE}~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;;
+          esac
+          echo "VERSION=$V" >> "$GITHUB_ENV"
+          echo "DISTRIBUTION=$DIST" >> "$GITHUB_ENV"
+          echo "package version $V -> apt distribution '$DIST' (arm64)"
+
+      # dpkg-shlibdeps + dpkg-deb. The arm64 link deps themselves are the cross image's whole
+      # point and are already baked in; python3 is for scripts/ci/gitea-release.sh.
+      - name: dpkg-dev
+        run: |
+          apt-get update
+          apt-get install -y --no-install-recommends dpkg-dev python3
+
+      - name: Cache keys
+        run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
+      - uses: actions/cache@v4
+        with:
+          path: |
+            /usr/local/cargo/registry
+            /usr/local/cargo/git
+          key: cargo-home-${{ hashFiles('Cargo.lock') }}
+          restore-keys: cargo-home-
+      - uses: actions/cache@v4
+        with:
+          path: target
+          # Its OWN key — these are aarch64 artifacts under target/aarch64-unknown-linux-gnu/
+          # and must never share the amd64 jobs' target cache.
+          key: cargo-target-arm64-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
+          restore-keys: cargo-target-arm64-v1-${{ env.rustc }}-
+
+      - name: Build the arm64 client .deb
+        env:
+          PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }}   # stamped into the binaries (build.rs)
+        run: |
+          git config --global --add safe.directory "$PWD"
+          ARCH=arm64 TARGET=aarch64-unknown-linux-gnu \
+            bash packaging/debian/build-client-deb.sh
+          # Fail here rather than shipping an amd64 binary under an arm64 package name.
+          readelf -h target/aarch64-unknown-linux-gnu/release/punktfunk-session \
+            | grep -q AArch64 || { echo "ERROR: session binary is not AArch64"; exit 1; }
+
+      - name: Publish to the Gitea apt registry
+        env:
+          TOKEN: ${{ secrets.REGISTRY_TOKEN }}
+        run: |
+          for DEB in dist/*.deb; do
+            echo "uploading $DEB"
+            NAME=$(dpkg-deb -f "$DEB" Package)
+            VER=$(dpkg-deb -f "$DEB" Version)
+            ARCH=$(dpkg-deb -f "$DEB" Architecture)
+            curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
+              "https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/$NAME/$VER/$ARCH" || true
+            curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$DEB" \
+              "https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/upload"
+          done
+          echo "published arm64 client to $OWNER/debian $DISTRIBUTION/$COMPONENT"
+
+      - name: Attach the arm64 .deb to the Gitea release (stable tags only)
+        if: startsWith(gitea.ref, 'refs/tags/v')
+        env:
+          GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
+        run: |
+          . scripts/ci/gitea-release.sh
+          RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
+          for DEB in dist/*.deb; do
+            upsert_asset "$RID" "$DEB"
+          done
@@ -2,6 +2,7 @@
 #   punktfunk-web      — management console (web/Dockerfile, repo-root context)
 #   punktfunk-docs     — documentation site (docs-site/Dockerfile)
 #   punktfunk-rust-ci  — Rust CI builder image consumed by ci.yml
+#   punktfunk-rust-ci-arm64cross — the above + an arm64 sysroot, for the aarch64 client legs
 #   punktfunk-fedora-rpm — Fedora 43 builder image consumed by rpm.yml (Bazzite RPM)
 # Host and clients are intentionally NOT containerized (see CLAUDE.md "What's left").
 #
@@ -80,6 +81,41 @@ jobs:
          docker push "$REGISTRY/$OWNER/${{ matrix.image }}:latest"
          case "$GITHUB_REF" in refs/tags/v*) docker push "$REGISTRY/$OWNER/${{ matrix.image }}:${GITHUB_REF_NAME}" ;; esac

+  # The aarch64 CROSS builder — a SEPARATE job because it is `FROM punktfunk-rust-ci:latest`
+  # and so must not race the matrix entry that publishes that base. Consumed by the arm64
+  # client legs in deb.yml/rpm.yml/arch.yml. Root context: it needs rust-toolchain.toml to
+  # install the target against the toolchain the workspace actually pins.
+  build-push-arm64cross:
+    runs-on: ubuntu-24.04
+    needs: build-push
+    timeout-minutes: 45
+    env:
+      IMAGE: punktfunk-rust-ci-arm64cross
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Login to registry
+        run: |
+          echo "${{ secrets.REGISTRY_TOKEN }}" \
+            | docker login "$REGISTRY" -u enricobuehler --password-stdin
+
+      - name: Build
+        run: |
+          EXTRA=""
+          case "$GITHUB_REF" in refs/tags/v*) EXTRA="-t $REGISTRY/$OWNER/$IMAGE:${GITHUB_REF_NAME}" ;; esac
+          docker build --pull \
+            -f ci/rust-ci-arm64cross.Dockerfile \
+            -t "$REGISTRY/$OWNER/$IMAGE:latest" \
+            -t "$REGISTRY/$OWNER/$IMAGE:sha-${GITHUB_SHA::8}" \
+            $EXTRA \
+            .
+
+      - name: Push
+        run: |
+          docker push "$REGISTRY/$OWNER/$IMAGE:sha-${GITHUB_SHA::8}"
+          docker push "$REGISTRY/$OWNER/$IMAGE:latest"
+          case "$GITHUB_REF" in refs/tags/v*) docker push "$REGISTRY/$OWNER/$IMAGE:${GITHUB_REF_NAME}" ;; esac
+
  # Deploy the docs site to unom-1, the DMZ services VM website/cms also deploy to
  # (docs.punktfunk.unom.io via Caddy on home-reverse-proxy-1 -> :3220). Same secret set
  # as unom/website's deploy: DEPLOY_HOST/DEPLOY_USER/DEPLOY_PORT/DEPLOY_SSH_KEY (the
@@ -0,0 +1,45 @@
+#!/bin/bash
+# Host-side C compiler wrapper for the aarch64 cross image (ci/rust-ci-arm64cross.Dockerfile).
+#
+# Why this exists: ffmpeg-sys-next's build script compiles a probe it intends to RUN — it
+# executes the binary to read the libav* version macros — so it forces `.target(HOST)` with
+# the comment "don't cross-compile this", but still hands that host compile the TARGET's
+# pkg-config include paths. `-I/usr/include/aarch64-linux-gnu` then shadows the host's own
+# multiarch libc headers and the x86 compiler dies inside bits/math-vector.h on NEON/SVE
+# types it has never heard of.
+#
+# Prepending the host's multiarch dir does NOT fix it: GCC drops a `-I` that duplicates a
+# directory already on its system include path (keeping it in the original, later position),
+# so the arm64 dir stays in front. The reliable fix is to remove the target include dirs from
+# the host compile entirely — the probe only wants FFmpeg's version macros, and the amd64
+# libav*-dev headers are installed and on the default search path, at the same version (both
+# come from this Ubuntu release).
+#
+# Scope: only ever invoked as CC for the HOST triple (CC_x86_64_unknown_linux_gnu). Target
+# compiles go to aarch64-linux-gnu-gcc and never pass through here.
+set -euo pipefail
+
+declare -a out=()
+while (($#)); do
+  case "$1" in
+  # `-I dir` as two arguments — the form cc's Command building and ffmpeg-sys both emit.
+  -I)
+    if [[ ${2-} == *aarch64-linux-gnu* ]]; then
+      shift 2
+      continue
+    fi
+    out+=("$1" "${2-}")
+    shift 2
+    ;;
+  # `-Idir` glued into one argument.
+  -I*aarch64-linux-gnu*)
+    shift
+    ;;
+  *)
+    out+=("$1")
+    shift
+    ;;
+  esac
+done
+
+exec /usr/bin/cc "${out[@]}"
@@ -0,0 +1,81 @@
+# Cross-compiling CI builder: amd64 host toolchain + an arm64 multiarch sysroot, for the
+# aarch64 Linux CLIENT artifacts (punktfunk-client + punktfunk-session).
+#
+#   docker build -f ci/rust-ci-arm64cross.Dockerfile -t punktfunk-rust-ci-arm64cross .
+#
+# Derived from punktfunk-rust-ci so the Rust toolchain, clang, and CMake are byte-identical
+# to the amd64 legs — this image only adds the target side. Kept as a SEPARATE image rather
+# than folded into the base because the :arm64 dev libs are ~1 GB that every other CI job
+# would otherwise pull for nothing.
+#
+# Client only: the Linux HOST stays amd64 (its encode stack is NVENC/QSV/AMF), so none of the
+# host's CUDA/GBM link deps are mirrored here.
+#
+# Ubuntu splits archives by architecture: amd64 lives on archive.ubuntu.com, every port
+# (arm64 included) on ports.ubuntu.com. Both stanzas therefore have to be pinned with an
+# explicit `Architectures:` or apt tries to fetch arm64 from the amd64 mirror and 404s.
+#
+# Built from the REPO ROOT context (not ci/) — see the rust-toolchain.toml copy below.
+FROM git.unom.io/unom/punktfunk-rust-ci:latest
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# 1. Pin the stock sources to amd64, add ports.ubuntu.com for arm64.
+RUN sed -i 's|^Types: deb$|Types: deb\nArchitectures: amd64|' /etc/apt/sources.list.d/ubuntu.sources \
+    && . /etc/os-release \
+    && printf 'Types: deb\nArchitectures: arm64\nURIs: http://ports.ubuntu.com/ubuntu-ports/\nSuites: %s %s-updates %s-backports %s-security\nComponents: main universe restricted multiverse\nSigned-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg\n' \
+        "$VERSION_CODENAME" "$VERSION_CODENAME" "$VERSION_CODENAME" "$VERSION_CODENAME" \
+        > /etc/apt/sources.list.d/ubuntu-ports-arm64.sources \
+    && dpkg --add-architecture arm64
+
+# 2. The cross toolchain + every arm64 dev lib the client links. Mirrors the client half of
+#    rust-ci.Dockerfile's list (FFmpeg, PipeWire, Opus, SDL3, GTK4/libadwaita, xkbcommon,
+#    Vulkan headers for pf-ffvk's bindgen over hwcontext_vulkan.h).
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    crossbuild-essential-arm64 \
+    libavcodec-dev:arm64 libavformat-dev:arm64 libavutil-dev:arm64 libswscale-dev:arm64 \
+    libavfilter-dev:arm64 libavdevice-dev:arm64 \
+    libpipewire-0.3-dev:arm64 libopus-dev:arm64 \
+    libsdl3-dev:arm64 libgtk-4-dev:arm64 libadwaita-1-dev:arm64 \
+    libwayland-dev:arm64 libxkbcommon-dev:arm64 libvulkan-dev:arm64 \
+    && rm -rf /var/lib/apt/lists/*
+
+# 3. The Rust target — installed against the toolchain the WORKSPACE pins, not the image's
+#    default. The base image bakes whatever `stable` was at its build time, while every build
+#    in the repo switches to the exact channel in rust-toolchain.toml; adding the target to
+#    the default toolchain instead leaves the pinned one without an aarch64 std, and the build
+#    dies on `can't find crate for core` a few hundred crates in. Running rustup from a
+#    directory that contains the pin file resolves the right toolchain (and pre-downloads it,
+#    which every workspace job would otherwise pay for on first use).
+COPY rust-toolchain.toml /opt/pf-toolchain/
+WORKDIR /opt/pf-toolchain
+RUN rustup target add aarch64-unknown-linux-gnu && rustup show
+WORKDIR /
+
+# 4. Cross wiring. Everything in this image is a cross build, so the plain (un-suffixed)
+#    variables are safe and cover the crates that roll their own pkg-config/bindgen calls
+#    instead of going through the target-scoped lookups.
+#      * PKG_CONFIG uses Debian's multiarch wrapper, which resolves the arm64 .pc files and
+#        rewrites -I/-L into the sysroot without per-crate cooperation.
+#      * BINDGEN_EXTRA_CLANG_ARGS: clang defaults to the host triple, so bindgen would parse
+#        arm64 headers with amd64 type layouts (silently wrong, not a build error) — the
+#        explicit --target plus the multiarch include dir is what keeps the layouts honest.
+#      * CC_x86_64_unknown_linux_gnu routes HOST-targeted compiles through a wrapper that
+#        strips the arm64 include dirs — see ci/pf-host-cc for the ffmpeg-sys-next probe it
+#        exists for.
+COPY ci/pf-host-cc /usr/local/bin/pf-host-cc
+RUN chmod 0755 /usr/local/bin/pf-host-cc
+
+ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
+    CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
+    CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
+    AR_aarch64_unknown_linux_gnu=aarch64-linux-gnu-ar \
+    CC_x86_64_unknown_linux_gnu=/usr/local/bin/pf-host-cc \
+    PKG_CONFIG=aarch64-linux-gnu-pkg-config \
+    PKG_CONFIG_ALLOW_CROSS=1 \
+    BINDGEN_EXTRA_CLANG_ARGS="--target=aarch64-unknown-linux-gnu -I/usr/include/aarch64-linux-gnu"
+
+# Fail the BUILD, not some later CI job, if the wrapper or a sysroot .pc is missing.
+RUN command -v aarch64-linux-gnu-pkg-config \
+    && aarch64-linux-gnu-pkg-config --cflags libavcodec sdl3 gtk4 libpipewire-0.3 \
+    && aarch64-linux-gnu-gcc -dumpmachine | grep -q aarch64
@@ -7,24 +7,40 @@
 # the package names the target boxes ship. The client links no NVIDIA libs — no filter
 # needed.
 #
-# Usage: VERSION=0.0.1~ci42.gdeadbee [ARCH=amd64] bash packaging/debian/build-client-deb.sh
+# Usage: VERSION=0.0.1~ci42.gdeadbee [ARCH=amd64] [TARGET=<rust triple>] \
+#          bash packaging/debian/build-client-deb.sh
 # Output: dist/punktfunk-client_<version>_<arch>.deb
+#
+# TARGET cross-compiles (and moves the binaries under target/<triple>/release). Set it
+# together with ARCH — e.g. ARCH=arm64 TARGET=aarch64-unknown-linux-gnu inside the
+# ci/rust-ci-arm64cross.Dockerfile image, which carries the matching :arm64 sysroot.
 set -euo pipefail

 VERSION="${VERSION:?set VERSION (e.g. 0.0.1 or 0.0.1~ci42.gdeadbee)}"
 ARCH="${ARCH:-amd64}"
+TARGET="${TARGET:-}"
 PKG="punktfunk-client"
 CRATE="punktfunk-client-linux"
 ROOTDIR="$(cd "$(dirname "$0")/../.." && pwd)"
 cd "$ROOTDIR"

-BIN="target/release/$PKG"
+# Cargo drops a cross build under target/<triple>/release, a native one straight in
+# target/release.
+if [ -n "$TARGET" ]; then
+  OUTDIR="target/$TARGET/release"
+  CARGO_TARGET_ARGS=(--target "$TARGET")
+else
+  OUTDIR="target/release"
+  CARGO_TARGET_ARGS=()
+fi
+
+BIN="$OUTDIR/$PKG"
 # The Vulkan/Skia session streamer the shell execs for a connect — shipped alongside the shell
 # (the shell resolves it as its /usr/bin sibling), or desktop streaming breaks.
-SESSION_BIN="target/release/punktfunk-session"
+SESSION_BIN="$OUTDIR/punktfunk-session"
 if [ ! -x "$BIN" ] || [ ! -x "$SESSION_BIN" ]; then
-  echo "==> building $CRATE + punktfunk-client-session (release)"
-  cargo build --release --locked -p "$CRATE" -p punktfunk-client-session
+  echo "==> building $CRATE + punktfunk-client-session (release${TARGET:+ for $TARGET})"
+  cargo build --release --locked "${CARGO_TARGET_ARGS[@]}" -p "$CRATE" -p punktfunk-client-session
 fi

 STAGE="$(mktemp -d)"