Merge pull request 'Miri interprets the FFI-free leaf crates, one of them at MSVC layout' (#169) from worktree-miri-ci into main

.gitea/workflows/audit.yml

@@ -15,6 +15,12 @@
 #                   fails if any crate carries a license outside the allowlist — the regression
 #                   guard about.toml always promised. (The Android Gradle tree has no lockfile, so
 #                   nothing scans it — see the CRA roadmap.)
+#   * miri        → NON-BLOCKING interpretation of the few FFI-free leaf crates, one of them
+#                   cross-compiled to MSVC layout. Not a supply-chain scan; it lives here because
+#                   audit.yml already has exactly the shape it needs (weekly cron,
+#                   workflow_dispatch, the rust-ci container, the same cache pattern) and because
+#                   ci.yml runs on every push against a fleet where 37 of 46 jobs contend for
+#                   ubuntu-24.04. See the `miri:` job below for what it does and does not buy.
 # Triggers: weekly (catch newly-disclosed CVEs in pinned deps), on every lockfile/allowlist
 # change, and on demand.
 # To silence a known-unfixable Rust advisory, add it to `.cargo/audit.toml` ([advisories] ignore=[…]).
@@ -44,6 +50,13 @@ on:
       - 'about.toml'
       - '.gitea/workflows/audit.yml'
   workflow_dispatch:
+# NOTE on the `paths:` list above and the `miri:` job: `crates/pf-driver-proto/**` is deliberately
+# NOT listed, even though that crate is what the Miri job exists to watch. `paths:` is a
+# WORKFLOW-level filter — adding it would fire all six jobs (three bun trees, pnpm, cargo-audit,
+# the license gate) on every driver-proto edit, onto a fleet where 37 of 46 jobs contend for
+# ubuntu-24.04, to run one 2-minute job. Weekly cron + workflow_dispatch is the day-one cadence;
+# revisit once the job has a green history, and if you do, prefer moving miri to its own workflow
+# file over widening this filter.
 
 jobs:
   cargo-audit:
@@ -177,3 +190,177 @@ jobs:
           command -v cargo-about >/dev/null 2>&1 || cargo install --locked cargo-about --version 0.9.1 --features cli
           cargo about generate about.hbs --fail -o /dev/null
           cargo about generate -m packaging/windows/drivers/Cargo.toml -c about.toml about.hbs --fail -o /dev/null
+
+  # ── Miri ─────────────────────────────────────────────────────────────────────────────────────
+  # WHAT THIS BUYS, precisely — one thing, and it is worth having:
+  #   It interprets `pf-driver-proto` CROSS-COMPILED TO `x86_64-pc-windows-msvc`, on a Linux
+  #   runner, with no Windows box anywhere in the loop. That crate is `#![forbid(unsafe_code)]`
+  #   and is path-dep'd by BOTH the main workspace and the driver workspace, so it is the layout
+  #   oracle for every frame and IOCTL crossing that boundary — and drift there is silent
+  #   corruption, not a compile error. Nothing else in CI checks it at MSVC layout.
+  #   On the first run ever performed against this repo it found a real defect: a layout test
+  #   reading an align-8 struct out of an align-1 stack buffer, which had passed on every machine
+  #   and every CI leg since it was written because a stack `[u8; 40]` usually lands 8-aligned.
+  #
+  # WHAT IT DOES NOT BUY — do not let anyone report this as unsafe coverage, and do not publish a
+  #   "Miri coverage" percentage; it would be noise. Miri can execute on the order of 2% of the
+  #   host's unsafe. It cannot run ash, windows-rs, ffmpeg, CUDA or the WDK, and in those crates
+  #   the unsafe *is* the foreign call, so there is nothing for an interpreter to execute. This
+  #   job is a targeted instrument for three leaf surfaces, not a safety net.
+  #
+  # NON-BLOCKING, deliberately, and via a step-level `||` — NOT job-level `continue-on-error`,
+  # which act_runner does not reliably honor (same reasoning as docs-site-audit above; a red job
+  # here would take the whole run red). Flip to blocking only after several weeks of green
+  # establish the nightly-drift rate.
+  #
+  # Do NOT add crates here because they merely compile under Miri. Add them because they contain
+  # pure-Rust unsafe or a layout contract worth interpreting. Explicitly excluded:
+  #   * pf-bitstream    — its compile did not finish in 27 min at 2.1 GB RSS, and it is
+  #                       `forbid(unsafe_code)`, so there is nothing to find. Do not re-add it.
+  #   * pf-update-check — ring; every FFI crate — dies on the first foreign call. Structural.
+  #   * punktfunk-core in bulk — `-- fec packet crypto` selects 63 tests and was killed at a
+  #                       25-minute cap with not one test reported complete. Only the narrow
+  #                       `fec::gf8` selection below is affordable, and it was timed before it
+  #                       was committed. Do not widen this filter without timing the result.
+  #
+  # MEASURED, not estimated — 192.168.1.25 (Ubuntu, 8 cores), on the DATED toolchain this job
+  # actually installs, with a COLD target dir and a COLD sysroot cache (so each step's figure
+  # includes building the Miri sysroot it needs) and a warm cargo registry. Every step below has
+  # been run start to finish; nothing here is extrapolated:
+  #   step A   21 + 12 + 4 pass    43 s
+  #   step B          21 pass      26 s
+  #   step C           2 pass      63 s
+  #   TOTAL 132 s cold. Interpretation itself is ~10 s of that; the rest is compiling, plus ~38 s
+  #   of one-time sysroot builds (21 s host + 17 s MSVC) that the cache below then carries.
+  #   Warm, the three steps are ~6 s / ~3 s / ~10 s. `timeout-minutes: 30` is therefore vast
+  #   headroom, kept deliberately so a first fully-uncached run — which additionally downloads a
+  #   ~400 MB toolchain and the registry — cannot trip it.
+  # If you add a step, MEASURE IT FIRST. The estimate this job replaced said "under 15 s across
+  # all four steps" and was extrapolated from a partial run; the real punktfunk-core figure was
+  # >25 min. Extrapolation is exactly how that happened.
+  miri:
+    runs-on: ubuntu-24.04
+    container:
+      image: 192.168.1.58:5010/punktfunk-rust-ci:latest
+    timeout-minutes: 30
+    env:
+      # A DATED nightly, bumped deliberately — exactly like rust-toolchain.toml, and for the same
+      # reason. The cache keys below carry this value, so bumping it self-invalidates them.
+      # ⚠ `nightly-<date>` names the day rustup PUBLISHED the build, and that build is compiled
+      # from the PREVIOUS day's commit. This pin therefore resolves to
+      # `rustc 1.99.0-nightly (969b803cb 2026-08-09)` [verified by installing it], NOT the
+      # `12c36e253 2026-08-10` that the rust-safety programme doc's §7 table cites — that figure
+      # came from the ROLLING `nightly` channel and was mislabelled as the dated one. Harmless,
+      # but do not "fix" the date to chase that hash: all three steps below were re-run and are
+      # green on the dated toolchain this job actually installs.
+      MIRI_TOOLCHAIN: nightly-2026-08-10
+      # A GUARD, not a fix for a present problem: audit.yml sets no sccache — only ci.yml does, at
+      # workflow level (ci.yml:27). `cargo-miri` REPLACES rustc and cannot be wrapped; it prints
+      # "Ignoring `RUSTC_WRAPPER` environment variable, Miri does not support wrapping" and
+      # carries on [verified]. This keeps a future workflow-level sccache from becoming a puzzle.
+      RUSTC_WRAPPER: ""
+      # -Zmiri-disable-isolation: pf-gpu's tests mkdir, and Miri aborts them without it [verified].
+      # -Zmiri-symbolic-alignment-check: the whole point — it refuses to let an accidentally
+      #   favourable stack slot stand in for an alignment guarantee. This is the flag that caught
+      #   the pf-driver-proto defect.
+      # NOTE the absence of -Zmiri-ignore-leaks. Miri leak-checks by DEFAULT, and that is the one
+      #   leak-detection capability it offers here. None of the crates below leaks, so the job is
+      #   green. The tree does contain DELIBERATE leaks (pf-umdf-util/src/section.rs `ViewCell`,
+      #   gamepad_raii.rs leak-on-timeout) — when coverage ever reaches them, annotate those two
+      #   sites; do not blanket-disable the check.
+      MIRIFLAGS: -Zmiri-disable-isolation -Zmiri-symbolic-alignment-check
+    steps:
+      - uses: actions/checkout@v4
+
+      # Two caches, split on purpose so a Cargo.lock change does not re-download a ~400 MB
+      # toolchain. Both use their OWN `miri-` key prefix — never a shared one.
+      # The Miri sysroot is per-toolchain and per-target (two are built here: host + MSVC), so it
+      # belongs with the toolchain, not with the lockfile.
+      - name: cache the nightly toolchain + Miri sysroots
+        uses: actions/cache@v4
+        with:
+          path: |
+            /usr/local/rustup/toolchains/${{ env.MIRI_TOOLCHAIN }}-x86_64-unknown-linux-gnu
+            ~/.cache/miri
+          key: miri-toolchain-v1-${{ env.MIRI_TOOLCHAIN }}
+      - name: cache the cargo registry
+        uses: actions/cache@v4
+        with:
+          path: /usr/local/cargo/registry
+          key: miri-registry-v1-${{ hashFiles('Cargo.lock') }}
+          restore-keys: miri-registry-v1-
+
+      # The image needs no change for this: ci/rust-ci.Dockerfile:51-54 installs via rustup and
+      # `chmod -R a+w`s both RUSTUP_HOME and CARGO_HOME, so a job can add a toolchain at runtime.
+      # `rust-src` is required — cargo-miri builds its sysroot from source, per target.
+      #
+      # This does NOT disturb the 1.96.0 pin: `cargo +<toolchain>` overrides rust-toolchain.toml
+      # for that single invocation only, so `cargo fmt` / `clippy` keep resolving 1.96.0 and the
+      # fmt-parity contract in CLAUDE.md is untouched. The two echo lines below keep that claim
+      # honest in the log. They are deliberately NOT `rustup show active-toolchain`: that command
+      # RESOLVES the toolchain file and would install the whole 1.96.0 toolchain just to print a
+      # line, in a job where every cargo call is `+$MIRI_TOOLCHAIN` and 1.96.0 is never needed.
+      # Deliberately NOT `rustup override set` — that writes persistent per-directory state into
+      # the runner's rustup config, which leaks into unrelated later jobs on a self-hosted fleet.
+      # Deliberately NOT a second rust-toolchain.toml in a subdirectory — that would apply to
+      # every cargo invocation under that subtree including fmt, which is the drift the root pin
+      # exists to prevent.
+      - name: install the pinned nightly + miri
+        run: |
+          git config --global --add safe.directory "$PWD"
+          rustup toolchain install "$MIRI_TOOLCHAIN" \
+            --profile minimal \
+            --component miri,rust-src \
+            --target x86_64-pc-windows-msvc
+          echo "root pin, untouched by this job: $(grep -E '^channel' rust-toolchain.toml)"
+          cargo +"$MIRI_TOOLCHAIN" --version
+
+      # A run that reports `0 passed` is a selection that matched nothing, not a success — that
+      # exact mistake has already cost one round-trip here. So each step below checks a zero exit
+      # AND that at least one target reported a non-zero pass count, which is what catches a
+      # crate rename or a `--` filter that stops matching. (Each step legitimately prints several
+      # `0 passed` lines too — the empty bin/doctest targets — so the check is "at least one
+      # non-zero", not "no zeroes".) Expected counts at the time of writing: 21 + 12 + 4.
+      - name: miri — FFI-free leaf crates (native)
+        run: |
+          set -o pipefail
+          ok=1
+          cargo +"$MIRI_TOOLCHAIN" miri test \
+            -p pf-driver-proto -p pf-host-config -p pf-gpu 2>&1 | tee /tmp/miri-native.log || ok=0
+          grep -qE 'test result: ok\. [1-9][0-9]* passed' /tmp/miri-native.log || ok=0
+          [ "$ok" = 1 ] || echo "::warning::miri (FFI-free leaf crates, native) did not pass — non-blocking; see punktfunk-planning design/rust-safety-programme.md §7"
+
+      # THE step that justifies the job: pf-driver-proto at MSVC layout, on Linux, no Windows box.
+      # Expected: 21 passed. If this one ever goes red, treat it as a layout-contract break
+      # between the host and driver workspaces until proven otherwise.
+      - name: miri — pf-driver-proto at x86_64-pc-windows-msvc layout
+        run: |
+          set -o pipefail
+          ok=1
+          cargo +"$MIRI_TOOLCHAIN" miri test \
+            -p pf-driver-proto --target x86_64-pc-windows-msvc 2>&1 | tee /tmp/miri-msvc.log || ok=0
+          grep -qE 'test result: ok\. [1-9][0-9]* passed' /tmp/miri-msvc.log || ok=0
+          [ "$ok" = 1 ] || echo "::warning::miri (pf-driver-proto @ MSVC layout) did not pass — non-blocking, but this is the layout oracle for every frame and IOCTL; see design/rust-safety-programme.md §7"
+
+      # fec-rs dispatches its GF(2^8) multiply through RUNTIME `is_x86_feature_detected!`. Under
+      # Miri that detection reports the COMPILE-TIME target features, so WITHOUT these RUSTFLAGS
+      # the step silently interprets the scalar fallback and is worthless. Verified both ways on
+      # 192.168.1.25: bare, `avx2=false ssse3=false`; with the flags, `avx2=true ssse3=true` and
+      # `_mm256_shuffle_epi8` genuinely executes under the interpreter. GFNI stays false either
+      # way — Miri does not implement it — so the gfni branch is simply not covered here.
+      #
+      # ⚠ x86_64 ONLY, and it must stay that way. A RUSTFLAGS env var OVERRIDES config rustflags
+      # ENTIRELY (.cargo/config.toml:11-13 says so), and that config carries `--cfg aes_armv8` /
+      # `--cfg polyval_armv8` for aarch64 — worth a measured ~3x decrypt-throughput cliff if
+      # dropped. Harmless here because this job pins ubuntu-24.04/x86_64; fatal on mac-mini-1.
+      # Narrow selection is mandatory, not an optimisation: see the punktfunk-core note above.
+      - name: miri — punktfunk-core fec::gf8, taking the real AVX2/SSSE3 branches
+        env:
+          RUSTFLAGS: -C target-feature=+avx2,+ssse3
+        run: |
+          set -o pipefail
+          ok=1
+          cargo +"$MIRI_TOOLCHAIN" miri test \
+            -p punktfunk-core --lib -- fec::gf8 2>&1 | tee /tmp/miri-gf8.log || ok=0
+          grep -qE 'test result: ok\. [1-9][0-9]* passed' /tmp/miri-gf8.log || ok=0
+          [ "$ok" = 1 ] || echo "::warning::miri (punktfunk-core fec::gf8, AVX2/SSSE3) did not pass — non-blocking; see design/rust-safety-programme.md §7"