punktfunk/design/windows-client-bootstrap.md

Windows native client — bootstrap handoff

Status: SHIPPED — clients/windows (binary punktfunk-client), WinUI 3 via windows-reactor; commits 0a3b92d..0cc36fa. Build + clippy + fmt green on x86_64-pc-windows-msvc and aarch64-pc-windows-msvc (ARM64 cross-compiled off the one x64 runner; signed MSIX for both arches via windows-msix.yml). This doc is trimmed to design rationale, the HDR reference, hard-won gotchas, and open items. The shipped source under clients/windows/src/ is the truth.

The native Windows punktfunk/1 client connects to a host (serve / punktfunk1-host), decodes HEVC, presents it low-latency on a SwapChainPanel, plays Opus audio, and captures local mouse/keyboard/gamepad to send back — the Windows analogue of the GTK4 Linux client (clients/linux), which was the architectural template. The locked decisions below are the durable "why"; the HDR section is the evergreen present reference.

Locked decisions (the "why")

• Pure Rust. windows-rs + Windows App SDK "Reactor" (WinUI 3 from Rust, merged windows-rs PR #4479). No C++/C#. Reactor + SwapChainPanel was the only novel/uncertain piece and was de-risked first; everything else is a known-good port of the Linux client.
  • Reactor is viable: windows-rs PR #4499 (merged 2026-06-01) added a SwapChainPanel widget to windows-reactor with set_swap_chain over CreateSwapChainForComposition, so a DXGI presenter can be hosted. (An earlier read that Reactor had no swapchain hatch was wrong/stale.) The UI is a declarative React-like tree (App::new().render(app), use_state/use_resource/use_effect hooks); the video page is swap_chain_panel().on_ready(|p| p.set_swap_chain(&sc)) driven by on_rendering.
• Links punktfunk-core directly (Cargo path dep, features = ["quic"]) — no C ABI, exactly like the GTK client, unlike the Apple path. NativeClient is already Sync (mutexed plane receivers), so it drops into a UI app cleanly. The C ABI (punktfunk_connect + next_au/ next_audio/next_rumble/next_hidout/send_input/send_rich_input) is the Apple path; the native Rust clients call crates/punktfunk-core/src/client.rs (NativeClient) methods directly.
• Video widget = WinUI 3 SwapChainPanel (built-in), fed a D3D11 swapchain via ISwapChainPanelNative::SetSwapChain. present.rs owns the D3D11 composition swapchain (WARP fallback, runtime shaders, Contain-fit) — the same renderer, bound to the panel instead of an HWND.
• Decode = FFmpeg-next + D3D11VA (HEVC; Main10 for 10-bit/HDR — see below).
• Audio playback = WASAPI render + Opus decode (opus crate, vendors libopus via cmake).
• Input capture→send: the client captures LOCAL input and sends it. Mouse (abs + relative) + keyboard via the inverse VK table (Windows VK is the native source, so simpler than Linux); gamepad via SDL3 (already a workspace dep, cross-platform) → NativeClient::send_input/ send_rich_input. (SendInput/ViGEm are HOST-side injection — not used by the client.)
  • Stream input is Win32 low-level hooks, not XAML: reactor exposes only keyboard accelerators
    • pointer button-state (no raw key-down/up, no pointer position, no wheel), insufficient for a game stream. input.rs installs WH_KEYBOARD_LL/WH_MOUSE_LL on the stream page (uninstalled on exit), maps the pointer through the window client rect, sends native VK + abs mouse + wheel, with a Ctrl+Alt+Shift+Q capture toggle. (A future alternative: generate Microsoft.UI.Xaml.UIElement bindings from the staged winmd and subscribe to KeyDown/ PointerMoved — scoped to the panel.)
• Discovery = mdns-sd (cross-platform, browses _punktfunk._udp).
• Trust = shared client identity + SPAKE2 PIN pairing + TOFU (trust.rs; same identity files/logic as the other native clients).

10-bit + HDR (the present reference)

The host negotiates and emits HEVC Main10 + BT.2020 PQ HDR10 when the captured desktop is HDR (and 10-bit SDR Main10 when negotiated). The Windows client mirrors the Apple present:

• Advertise caps in the Hello: video_caps = VIDEO_CAP_10BIT | VIDEO_CAP_HDR (crates/punktfunk-core/src/quic.rs). The host enables 10-bit only if the client advertised it.
• Detect HDR in-band from the HEVC VUI (transfer characteristics = SMPTE ST 2084 / PQ), exactly like the Apple client's VideoDecoder.isHDRFormat (clients/apple/Sources/PunktfunkKit/). This handles a mid-session HDR toggle without renegotiation. Welcome.bit_depth (8/10) is also available.
• Decode Main10 → P010 (10-bit) via D3D11VA.
• Present HDR: swapchain in DXGI_FORMAT_R10G10B10A2_UNORM (or R16G16B16A16_FLOAT), IDXGISwapChain3::SetColorSpace1(DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020) + SetHDRMetaData for HDR10; the host's stream is BT.2020 PQ, so present PQ. For SDR, the existing DXGI_FORMAT_B8G8R8A8_UNORM + BT.709 path. (The host-side HDR conversion math is in crates/punktfunk-host/src/capture/dxgi.rs HDR_PS/HdrConverter if you need the inverse.)

Build gotchas (hard-won)

• CARGO_HOME must be an ASCII path (C:\Users\Public\.cargo). SDL3's build-from-source PCH embeds the registry source path; the ü in the dev box's username makes MSVC fail (MSB8084 / C4828). Build under an ASCII path generally (the same ü triggers LNK1201 PDB-write failures under ~/Developer).
• CMAKE_POLICY_VERSION_MINIMUM=3.5 in the build env (CMake 4 rejects libopus's old minimum).
• Toolchain: winget install NASM.NASM Kitware.CMake LLVM.LLVM (NASM for aws-lc-rs on the quic path; libclang/LLVM for ffmpeg-sys).
• windows-reactor is unpublished (version 0.0.0) and fast-moving — depend on it as a git dep pinned to a commit (b4129fcc), and pin the windows crate to the same commit so the IDXGISwapChain1 you pass to set_swap_chain satisfies reactor's windows_core::Interface. Its build.rs downloads the Windows App SDK NuGets (Foundation/Interactive/Runtime), stages the bootstrap DLL + resources.pri next to the exe, and .unwrap()s CARGO_WORKSPACE_DIR — set it in the build env (CARGO_WORKSPACE_DIR=C:\Users\Public\punktfunk). It writes /temp + /winmd to the workspace root (gitignored). The App SDK runtime must be installed to run.
• Windows clippy is stricter than Linux CI and cfg(windows) code is excluded from Linux CI → run cargo clippy -p punktfunk-client-windows -- -D warnings ON the Windows box before committing.

Open items

• On-glass validation on the RTX box of D3D11VA hardware decode + 10-bit/HDR present + the WinUI GUI — the dev VM is headless / SSH Session 0 / WARP, so the WinUI window can't show there and there is no hardware decode. Validate over RDP or on the RTX box against a live HDR host.
• RAWINPUT relative-mouse pointer-lock for the stream view.
• Per-host speed-test widget in the UI.

Key references

• Full Windows plan + SudoVDA/host details: design/windows-host.md.
• Template ported from: clients/linux/src/*.
• Apple HDR present (the pattern mirrored): clients/apple/Sources/PunktfunkKit/{VideoDecoder, MetalVideoPresenter,Stage2Pipeline}.swift — in-band PQ detection, P010 decode, EDR present.
• Core client API: crates/punktfunk-core/src/client.rs (NativeClient).
• Protocol: crates/punktfunk-core/src/quic.rs (Hello.video_caps, Welcome.bit_depth, VIDEO_CAP_10BIT/VIDEO_CAP_HDR).
• Host HDR conversion (inverse math): crates/punktfunk-host/src/capture/dxgi.rs (HDR_PS, HdrConverter) + crates/punktfunk-host/src/encode/nvenc.rs (BT.2020/PQ VUI).