From 9ea2c1741918ea10de68b22f3bb35b1eb9429972 Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Fri, 26 Jun 2026 16:22:40 +0000 Subject: [PATCH] docs(windows): add design/windows-build-and-packaging.md + refresh packaging README A single repo-internal source of truth for the Windows build/packaging: what ships, the all-Rust driver workspace built FROM SOURCE in CI (+ the anti-stale rationale), the toolchain (clang 22 + bindgen 0.72, no LLVM pin), the Inno installer, the web console bundle, the CI workflows, signing, and the dev loop. (design/, not the docs-site.) packaging/windows/README.md: drop the deleted vendored-driver dir + its "Vendored driver" callout, add the build-* / install-gamepad / clear-force-integrity rows, point at the new design doc. Co-Authored-By: Claude Opus 4.8 (1M context) --- design/windows-build-and-packaging.md | 217 ++++++++++++++++++++++++++ packaging/windows/README.md | 31 ++-- 2 files changed, 235 insertions(+), 13 deletions(-) create mode 100644 design/windows-build-and-packaging.md diff --git a/design/windows-build-and-packaging.md b/design/windows-build-and-packaging.md new file mode 100644 index 0000000..0328eca --- /dev/null +++ b/design/windows-build-and-packaging.md @@ -0,0 +1,217 @@ +--- +title: "Windows build & packaging" +description: "How the punktfunk Windows host is built, signed, and packaged: the all-Rust driver workspace built from source in CI, the Inno Setup installer, the web console bundle, the CI workflows, and the dev-iteration helpers. Repo-internal source of truth - not part of the user-facing docs-site." +--- + +# Windows build & packaging + +Single source of truth for **how the Windows host ships**: what artifacts are built, the all-Rust +driver workspace and why we build it from source in CI, the Inno Setup installer, the web console +bundle, the CI workflows, signing, and the dev loop. Architecture lives in +[`windows-host-rewrite.md`](windows-host-rewrite.md); deployment/runtime in +[`windows-service.md`](windows-service.md). This doc is repo-internal (do **not** mirror into +`docs-site/`). + +> **x64-only by design.** The host is coupled to NVENC (`nvEncodeAPI64.dll`) and the pf-vdisplay IddCx +> driver, neither of which exists on Windows ARM64 (no ARM64 NVIDIA driver / IddCx path). The *client* +> ships x64 + ARM64 MSIX; the *host* does not. + +## 1. What ships + +The signed `punktfunk-host-setup-.exe` (Inno Setup) lays down, under `C:\Program Files\punktfunk\`: + +| Component | What it is | +|-----------|------------| +| `punktfunk-host.exe` | the host binary (`--features nvenc,amf-qsv` = NVIDIA + AMD/Intel in one build) | +| `pf-vdisplay` driver | all-Rust UMDF IddCx virtual display (per-session client-resolution output) | +| `pf-dualsense` driver | virtual DualSense / DualShock 4 (one type-aware HID minidriver) | +| `pf-xusb` driver | virtual Xbox 360 / XInput companion | +| `pf-vkhdr-layer` | Vulkan implicit layer that advertises HDR formats on the virtual display | +| web console | self-contained Nitro `.output` + a portable `bun` runtime (the `PunktfunkWeb` task) | +| FFmpeg DLLs | `avcodec`/`avutil`/`swscale`/... - the AMD/Intel (AMF/QSV) encode backend link-imports them | +| `nefconc.exe` | nefarius' nefcon (creates the `root\pf_vdisplay` device node; pnputil can't) | + +All three drivers and the HDR layer are **bundled, not external** - no ViGEmBus, no SudoVDA, no separate +driver download. The host installs a `LocalSystem` SCM service that `CreateProcessAsUserW`s into the +interactive session for secure-desktop capture (why MSIX is unusable - see +[`windows-service.md`](windows-service.md)). + +## 2. Component map (source -> artifact) + +| Source | Built by | Artifact | +|--------|----------|----------| +| `crates/punktfunk-host/` | `cargo build --release -p punktfunk-host --features nvenc,amf-qsv` | `punktfunk-host.exe` | +| `packaging/windows/drivers/pf-vdisplay/` | `build-pf-vdisplay.ps1` (workspace `cargo build` + sign) | `pf_vdisplay.{dll,inf,cat}` + `.cer` | +| `packaging/windows/drivers/pf-dualsense/` `pf-xusb/` | `build-gamepad-drivers.ps1` (sign the workspace build) | `pf_{dualsense,xusb}.{dll,inf,cat}` + shared `.cer` | +| `packaging/windows/pf-vkhdr-layer/` | `pack-host-installer.ps1` (`cargo build --release`) | `pf_vkhdr_layer.dll` + `.json` | +| `web/` | `scripts/windows/build-web.ps1` (`bun run build`) | self-contained `.output` | +| `packaging/windows/nvenc/nvenc.def` | `gen-nvenc-importlib.ps1` (llvm-dlltool) | `nvencodeapi.lib` (link import, no GPU/SDK) | + +## 3. The driver workspace - `packaging/windows/drivers/` + +A **separate cargo workspace** (its own `[workspace]` root) because driver crates are `cdylib`s built +with the WDK toolchain on Windows only. Members: + +- `pf-vdisplay` - the IddCx virtual display (the real driver). +- `pf-dualsense`, `pf-xusb` - the virtual gamepad HID/XUSB minidrivers. +- `wdk-iddcx` - hand-written IddCx DDI wrappers (the `iddcx` ApiSubset bindgen reuses `wdk_default`). +- `wdk-probe` - a toolchain/surface-assert probe crate. +- `vendor/wdk-sys` + `vendor/wdk-build` - **vendored** microsoft/windows-drivers-rs 0.5.1 (the published + crates) + an added `iddcx` ApiSubset. A `[patch.crates-io]` redirects every `wdk-sys`/`wdk-build` + reference (incl. `wdk` 0.4.1's transitive deps) to these copies, so the graph has exactly one + iddcx-capable `wdk-sys`. **Pinned - do not chase upstream.** + +Path-deps the owned ABI crate `crates/pf-driver-proto` (the host<->driver control protocol). `.cargo/ +config.toml` sets an explicit `--target x86_64-pc-windows-msvc` + `target-feature=+crt-static` (UMDF +needs the static CRT; the explicit target keeps `crt-static` off host build-scripts/proc-macros). +`[workspace.metadata.wdk.driver-model]` sets UMDF 2.31 once for all members. + +Driver-specific gotchas (handled by the build scripts): + +- **`/INTEGRITYCHECK` (FORCE_INTEGRITY).** `wdk-build` links `/INTEGRITYCHECK`, which a non-EV + (self-signed) cert can't satisfy, so the driver won't load. `clear-force-integrity.ps1` clears the PE + `DllCharacteristics` bit (offset `0x5e`) **before** signing. +- **Self-signed cert.** The drivers are signed with a self-signed CodeSigning cert; the installer trusts + the bundled `.cer` (machine `Root` + `TrustedPublisher`) at install time so PnP loads them silently. + Validated to load under Secure Boot on. (CI can use a stable `DRIVER_CERT_PFX_B64` secret instead.) +- **Device node via nefcon, never devgen.** The `root\pf_vdisplay` node is created with `nefconc` + (a clean `ROOT\DISPLAY` node). `devgen` leaves persistent `SWD\DEVGEN` phantoms that survive reboot + + registry deletion. The gamepad drivers create their per-session nodes from the host via + `SwDeviceCreate` (no install-time node). +- **Strictly-increasing `DriverVer`.** `9.9.MMdd.HHmm` (stampinf). pnputil silently keeps the old binary + on a non-increasing version; a later-minute redeploy always wins. + +## 4. Drivers are BUILT FROM SOURCE - the anti-stale decision + +The drivers used to ship as **checked-in prebuilt binaries** (`packaging/windows/pf-vdisplay/` + +`gamepad-drivers/`). That model went stale and shipped two field bugs on a fresh install: + +1. A repo-wide rename edited `pf_vdisplay.inf` (a comment) but never re-signed `pf_vdisplay.cat`. A + catalog hashes the INF+DLL byte-for-byte, so `pnputil /add-driver` failed + `SPAPI_E_FILE_HASH_NOT_IN_CATALOG` **on every box** - the driver never installed, every session died + "pf-vdisplay driver interface not found". +2. The frozen binary predated `IOCTL_SET_RENDER_ADAPTER`, which the host needs to pin the IddCx render + GPU on hybrid/Optimus boxes. + +Fix: **build from source every release.** `pack-host-installer.ps1` calls `build-pf-vdisplay.ps1` (which +`cargo build`s the *whole* workspace) then `build-gamepad-drivers.ps1 -SkipBuild` (sign the already-built +gamepad cdylibs), so `.dll`/`.inf`/`.cat` are always in lockstep and current driver features ship. The +checked-in binaries were deleted. Re-introducing a vendored binary is the bug; if you must, a catalog +guard (`Test-FileCatalog` hash-membership) belongs in the build script. + +The build scripts share the same shape (WDK env -> build -> clear FORCE_INTEGRITY -> sign DLL -> +stampinf -> Inf2Cat -> sign cat -> export `.cer`); `build-gamepad-drivers.ps1` loops over the two gamepad +drivers and signs both with one shared cert. (A `_driver-pack-common.ps1` helper to dedup the ~90% they +share is a known TODO - keep behavior identical and re-run `windows-host` if you do it.) + +## 5. Toolchain / build env + +The drivers build with **plain `cargo build`** against the vendored windows-drivers-rs - **no cargo-make, +no cargo-wdk for the build** (cargo-wdk is only provisioned + probed by `windows-drivers.yml`). The build +needs, on the runner: + +- **WDK 26100** - `Version_Number=10.0.26100.0` pins the SDK version `wdk-build` uses (it otherwise picks + `10.0.28000.0`, which has no `km`/`crt`, and bindgen fails). Provisioned by + `scripts/ci/provision-windows-wdk.ps1` (iddcx headers are the "WDK present" signal). +- **clang 22 + bindgen 0.72** - the vendored `bindgen` is `0.72.1`, which builds clean on the runner's + **default** LLVM (`C:\Program Files\LLVM`, currently clang 22). `LIBCLANG_PATH` is left unset (defaults + to the runner default). *History:* LLVM 21.1.2 was briefly pinned (`C:\llvm-21`) to dodge a + bindgen-0.71 layout-test overflow on clang 22; the 0.72 bump retired that pin, so there's now one + toolchain for both driver builds (the pack and `windows-drivers.yml`). +- NVENC import lib synthesised from a 2-export `.def` via `llvm-dlltool` (`gen-nvenc-importlib.ps1`) - + no GPU or NVIDIA SDK at build time. +- `FFMPEG_DIR` (the BtbN gpl-shared x64 tree) for the AMD/Intel AMF/QSV link; NASM + CMake + + `CMAKE_POLICY_VERSION_MINIMUM=3.5` for the CMake-from-source deps (aws-lc, opus). +- **Gotcha:** `CARGO_HOME` must be an ASCII path (a non-ASCII username breaks SDL3's MSVC precompiled + header). The runner uses `C:\Users\Public\.cargo`. +- **`CARGO_TARGET_DIR` for the driver build must be the DEFAULT (in-tree) dir.** `wdk-build`'s + `find_top_level_cargo_manifest()` walks up from `OUT_DIR` to the first ancestor with a `Cargo.lock`; a + relocated `C:\t` target dir hides the workspace lock and the build-script panics "a Cargo.lock file + should exist...". The driver deps have no deep CMake crates, so the in-tree target stays under MAX_PATH. + (The host/client builds *do* relocate to `C:\t` to dodge MAX_PATH - that's the opposite need.) + +## 6. The installer - Inno Setup + +`pack-host-installer.ps1` orchestrates, in order: resolve a code-signing cert -> sign `punktfunk-host.exe` +-> **build + sign the drivers from source** (`build-pf-vdisplay.ps1` + `build-gamepad-drivers.ps1`, +staged via `stage-pf-vdisplay.ps1` which also fetches/verifies pinned nefcon) -> stage FFmpeg DLLs + the +web console + a portable bun -> build + sign the HDR Vulkan layer -> run `ISCC` on `punktfunk-host.iss` +-> sign `setup.exe`. + +`punktfunk-host.iss` (Inno) lays down `{app}`, runs the install steps, and registers things. **Optional +tasks** (all default-checked): install the pf-vdisplay driver, install the gamepad drivers, install the +HDR Vulkan layer, start the service. Silent install: `/VERYSILENT` (omit a task with +`/MERGETASKS="!installdriver"`). + +Install-time work (currently `[Run]` -> `powershell.exe -File install-*.ps1` / `web-setup.ps1`; **being +moved into `punktfunk-host.exe` subcommands** so there are no locale-parsed PowerShell scripts on the +end-user box - the root fix for the recurring ANSI-codepage parse breakage, see +[`windows-service.md`](windows-service.md) for the `service install` precedent): + +- **Driver install:** trust the bundled `.cer` (Root + TrustedPublisher), create the `root\pf_vdisplay` + node if absent (nefconc, gated so a re-create can't spawn a phantom), `pnputil /add-driver /install`. + Best-effort - a driver hiccup never aborts the install (the host degrades to a physical display). +- **Web console:** write the ACL'd `web-password`, register the `PunktfunkWeb` task (boot, SYSTEM, + restart-on-failure -> `bun` on `:3000`), open TCP 3000, start it. Upgrade-safe: stop + reap any old + console (by the `:3000` owner, runtime-agnostic) before re-registering so the new one can bind. + +**Signing:** the exe/setup/HDR-layer use the **`MSIX_CERT_PFX_B64`/`MSIX_CERT_PASSWORD`** secrets +(`CN=unom`, shared with the client); the **drivers** use a separate cert (self-signed per build, or a +stable `DRIVER_CERT_PFX_B64`) and their own bundled `.cer` - the two never collide. Without the MSIX +secrets, an ephemeral self-signed cert is generated and its `.cer` published next to the installer. + +## 7. The web console bundle + +The console is a TanStack Start / Nitro SSR app (`web/`). `vite.config.ts` sets `noExternals: true`, so +`bun run build` emits a **self-contained `.output`** (~75 files, deps bundled + tree-shaken, no +`node_modules`/`.npmrc`). The installer ships that `.output` + a portable `bun.exe`; the `PunktfunkWeb` +task runs `bun .output/server/index.mjs` on `:3000`, auto-wired to the host's loopback mgmt API via +`web-run.cmd` (sources `%ProgramData%\punktfunk\mgmt-token` + `web-password`). No node, no node_modules +forest. (`build-web.ps1` is the dev-box rebuild-and-restart helper.) + +## 8. CI workflows (`.gitea/workflows/`) + +All run on the single self-hosted `windows-amd64` runner (`home-windows-1`), which **serializes** the +whole Windows fleet - a `Cargo.lock`/`packaging/windows/**` touch queues several builds back-to-back. + +| Workflow | Trigger | Does | +|----------|---------|------| +| `windows-host.yml` | `crates/punktfunk-host`, `packaging/windows`, `scripts/windows`, `web`, tags `v*` | build host + clippy + HDR layer + web smoke-boot -> pack + sign installer -> publish (canary/latest) | +| `windows-drivers.yml` | `packaging/windows/drivers`, `crates/pf-driver-proto` | probe the driver toolchain + build/test/clippy `pf-driver-proto` + `cargo build` the driver workspace + inspect FORCE_INTEGRITY (the fast driver-only gate; coverage the pack lacks) | +| `windows-drivers-provision.yml` | `provision-windows-wdk.ps1` | one-shot WDK + cargo-wdk provisioning onto the persistent runner | +| `windows.yml` / `windows-msix.yml` | client | build the Windows *client* + its signed MSIX (x64 + ARM64) | + +`windows-host.yml` also builds the drivers from source (in pack), so it overlaps `windows-drivers.yml` on +a `drivers/**` edit (two driver builds on the serialized runner). They're kept separate on purpose - +`windows-drivers.yml` is the fast pre-pack gate. **CI builds, never launches the exe** (no GPU on the +runner), so AMF/QSV + on-glass behavior are validated on a real box, not in CI. + +## 9. Dev iteration + +- **Host:** `scripts/windows/deploy-host.ps1` (build + redeploy the exe to a box), `build-web.ps1` + (rebuild + restart the console). +- **pf-vdisplay driver:** `packaging/windows/drivers/deploy-dev.ps1` (build -> clear FORCE_INTEGRITY -> + sign -> stampinf a strictly-increasing `DriverVer` -> Inf2Cat -> sign -> `-Install`); + `redeploy-pf-vdisplay.ps1` (one-shot: stop host -> install -> reload adapter -> start); + `reset-pf-vdisplay.ps1` (recover a wedged driver: reap ghost monitor nodes + cycle the adapter, no + reboot). Run elevated; default to the `PunktfunkHost` service. +- Drive any of these from Linux over SSH: + `ssh user@box 'powershell -ExecutionPolicy Bypass -File C:\...\reset-pf-vdisplay.ps1'`. +- The RTX/on-glass box is where NVENC encode + IDD-push frame flow are validated (CI can't). + +## 10. Release + +Push a `vX.Y.Z` tag (one tag releases every platform): `windows-host.yml` builds + signs +`punktfunk-host-setup-X.Y.Z.exe` + the public `.cer`, refreshes the `latest/` alias, and attaches them to +the unified Gitea Release. Main pushes publish rolling `0.3.` **canary** builds to `canary/`. +Download: `https://git.unom.io/api/packages/unom/generic/punktfunk-host-windows/{latest,canary}/punktfunk-host-setup.exe`. + +## 11. See also + +- [`windows-host-rewrite.md`](windows-host-rewrite.md) - host architecture (capture/encode/vdisplay + backends, IDD-push, the rewrite milestones). The architecture source of truth. +- [`windows-service.md`](windows-service.md) - the SYSTEM service + secure-desktop deployment model. +- [`windows-virtual-display-rust-port.md`](windows-virtual-display-rust-port.md) - history of the all-Rust + IddCx driver port (SUPERSEDED in its conclusion: IDD-push became the primary capture path). +- `packaging/windows/pf-vkhdr-layer/README.md` - the HDR Vulkan layer. +- `packaging/windows/README.md` - the file index for `packaging/windows/`. diff --git a/packaging/windows/README.md b/packaging/windows/README.md index ab462cc..695ffb4 100644 --- a/packaging/windows/README.md +++ b/packaging/windows/README.md @@ -3,6 +3,8 @@ A one-file, signed `setup.exe` for the punktfunk streaming **host** on Windows, published to Gitea's generic package registry (`punktfunk-host-windows`) by `.gitea/workflows/windows-host.yml`. +> Full picture (drivers-from-source, toolchain, CI, dev loop): **[`design/windows-build-and-packaging.md`](../../design/windows-build-and-packaging.md)**. This README is the `packaging/windows/` file index. + ## x64 only (no ARM64) Unlike the client (which ships x64 + ARM64 MSIX), the host is **x64-only by design**. It is coupled to @@ -66,28 +68,31 @@ read it from `%ProgramData%\punktfunk\web-password`. | File | Role | |------|------| | `punktfunk-host.iss` | Inno Setup script (the installer definition). | -| `pack-host-installer.ps1` | Orchestrator: cert + sign, stage the driver + FFmpeg + **web console** (`.output` + bun) bundles, run ISCC, sign setup.exe, emit registry paths. | -| `stage-pf-vdisplay.ps1` | Stage the **vendored** pf-vdisplay driver + fetch/verify the **pinned** nefcon release into the bundle. | +| `pack-host-installer.ps1` | Orchestrator: cert + sign exe, **build + sign the drivers from source**, stage them + FFmpeg + the **web console** (`.output` + bun) + the HDR layer, run ISCC, sign setup.exe. | +| `build-pf-vdisplay.ps1` | Build pf-vdisplay from source (the `drivers/` workspace) + clear FORCE_INTEGRITY + sign `.dll`/`.cat` + export `.cer`. | +| `build-gamepad-drivers.ps1` | Sign + catalog the gamepad drivers (`pf-dualsense` + `pf-xusb`) from the same workspace build (`-SkipBuild`), one shared cert. | +| `clear-force-integrity.ps1` | Clear the `/INTEGRITYCHECK` PE bit so a self-signed driver loads (reused by every driver build). | +| `stage-pf-vdisplay.ps1` | Stage the just-built pf-vdisplay bundle + fetch/verify the **pinned** nefcon release. | | `install-pf-vdisplay.ps1` | Runs at install time (elevated): trust cert → gated device-node create (nefconc) → `pnputil` install. | +| `install-gamepad-drivers.ps1` | Runs at install time (elevated): trust cert → `pnputil /add-driver` each gamepad `.inf` (per-session devnodes are SwDeviceCreate'd by the host). | | `../../scripts/windows/web-run.cmd` | The `PunktfunkWeb` task action: loads the mgmt token + login password env, runs the bundled `bun` on the Nitro server (`:3000`). | | `../../scripts/windows/web-setup.ps1` | Install-time (elevated): write the ACL'd console password, register the `PunktfunkWeb` task + firewall rule, start it. | -| `pf-vdisplay/` | **Vendored** signed pf-vdisplay driver: `pf_vdisplay.inf` / `pf_vdisplay.cat` / `pf_vdisplay.dll` / `punktfunk-driver.cer`. Built from `drivers/`. | | `drivers/` | The all-Rust IddCx **driver source** workspace: the `pf-vdisplay` crate on `wdk-sys` / windows-drivers-rs + the owned `pf-driver-proto` ABI + `wdk-iddcx` / `wdk-probe`, plus `deploy-dev.ps1` (build/sign/install for dev). | | `reset-pf-vdisplay.ps1` | **Dev:** recover a wedged driver — stop host → reap ghost monitor nodes → reload the adapter → start host (no reboot). See *Dev iteration* below. | | `redeploy-pf-vdisplay.ps1` | **Dev:** one-shot redeploy — (optional) build → stop host → `deploy-dev.ps1 -Install` → reload adapter → start host. | | `nvenc/nvenc.def`, `nvenc/gen-nvenc-importlib.ps1` | Synthesise `nvencodeapi.lib` for the `--features nvenc` link (llvm-dlltool / lib.exe). | | `pf-vkhdr-layer/` | **HDR Vulkan layer** (standalone `cdylib`): lets Vulkan games (Doom: The Dark Ages, etc.) enable HDR over the virtual display by advertising the HDR surface formats the NVIDIA/AMD ICDs hide on an indirect display. Built by the packer, laid into `{app}\vklayer`, registered under `HKLM64\…\Khronos\Vulkan\ImplicitLayers` (opt-out *Install the HDR Vulkan layer* task). Self-gated on the display's HDR state. See its README. | -> **Vendored driver:** pf-vdisplay is our **all-Rust IddCx** virtual display (UMDF2), built from -> `packaging/windows/drivers/`. It replaced the vendored SudoVDA C++ driver — full story in -> [`design/windows-virtual-display-rust-port.md`](../../design/windows-virtual-display-rust-port.md). The -> **signed** output (`pf_vdisplay.dll`/`.inf`/`.cat` + `punktfunk-driver.cer`; signer -> `punktfunk-ds-test` — the same cert the gamepad drivers ship, Class=Display, HWID `root\pf_vdisplay`) -> is checked in under `pf-vdisplay/`. To refresh it after a driver-source change, rebuild + re-sign with -> `drivers/deploy-dev.ps1` and copy the staged `pf_vdisplay.{dll,inf,cat}` over the vendored -> copies. nefcon (the device-node tool — the install creates the node with it, **never** `devgen`, which -> leaves persistent phantom devices) **is** fetched + SHA-256-verified from its pinned release in -> `stage-pf-vdisplay.ps1`. +> **Drivers are built from source, not vendored.** All three (pf-vdisplay + the gamepad pf-dualsense / +> pf-xusb) are members of the all-Rust `drivers/` workspace (windows-drivers-rs / IddCx) and are +> **rebuilt + signed every release** by `build-pf-vdisplay.ps1` + `build-gamepad-drivers.ps1` - the +> checked-in prebuilt binaries were deleted (a stale `.cat` once stopped covering its `.inf` → +> `SPAPI_E_FILE_HASH_NOT_IN_CATALOG` on every box, and a frozen binary predated a driver IOCTL the host +> needed). Building from source keeps `.dll`/`.inf`/`.cat` in lockstep. nefcon (the device-node tool - +> the install creates the `root\pf_vdisplay` node with it, **never** `devgen`, which leaves persistent +> phantom devices) is fetched + SHA-256-verified from its pinned release in `stage-pf-vdisplay.ps1`. See +> [`design/windows-build-and-packaging.md`](../../design/windows-build-and-packaging.md) for the toolchain +> + signing details. ## Dev iteration on the test box (driver)