punktfunk/docs/windows-client-bootstrap.md

Windows native client — bootstrap handoff

A handoff for an agent picking up the native Windows punktfunk/1 client. The host side is done and live-validated on a real RTX 4090; the client is the remaining piece. This doc is the concrete starting point: the locked decisions, the reference code to port, the stack swaps, the dev loop, and the gotchas. Read it top to bottom, then start at Phase 1 (de-risk Reactor first).

Status — stage 1 landed (2026-06-15)

The client is implemented in crates/punktfunk-client-windows (binary punktfunk-client) and is build + clippy + fmt + test green on x86_64-pc-windows-msvc (built on the dev VM). Done: winit window + Direct3D11 flip-model swapchain present (WARP on the GPU-less box; runtime-compiled fullscreen-triangle shaders, Contain-fit letterbox), FFmpeg software HEVC decode, WASAPI render

• mic capture, keyboard/mouse/wheel capture (physical-KeyCode→VK, click-to-capture), SDL3 gamepads, mdns-sd discovery, and the full trust surface (identity + TOFU + SPAKE2 PIN over --connect/--discover/--pair/--headless).

• Reactor was evaluated and rejected (a research pass + the points below): windows-rs Reactor ships no SwapChainPanel and no ISwapChainPanelNative::SetSwapChain escape hatch, so it cannot host a DXGI presenter. The client uses the doc's sanctioned fallback — winit + a raw D3D11 swapchain on the HWND — which builds and runs against WARP on the GPU-less VM. A native WinUI look would need the SwapChainPanel hatch to land upstream first.
• Build gotcha (in addition to the ASCII output path below): CARGO_HOME itself must be on an ASCII path (e.g. C:\Users\Public\.cargo). SDL3's build-from-source compiles a precompiled header whose #include embeds the registry source path; the ü in the dev box's username makes MSVC's PCH/structured-output fail (MSB8084 / C4828). Set CARGO_HOME=C:\Users\Public\.cargo.
• Still pending: live host validation (the dev box has no GPU → glass-to-glass numbers defer to the RTX box), D3D11VA hardware decode + 10-bit/HDR present, a native host-list/settings GUI (CLI flags for now), and RAWINPUT relative-mouse pointer-lock. Phases 4–7 below are the map.

What we're building

A native Windows client that connects to a punktfunk/1 host (serve --native / m3-host), decodes HEVC, presents it low-latency, plays Opus audio, and captures local mouse/keyboard/gamepad to send back — i.e. the Windows analogue of the GTK4 Linux client (crates/punktfunk-client-linux), which is the architectural template. The Windows client is close to a 1:1 port of the Linux client with the platform layers swapped.

Locked decisions (from the Windows-host/client plan, docs/windows-host.md + project memory)

• Pure Rust. windows-rs + Windows App SDK "Reactor" (WinUI 3 from Rust, merged windows-rs PR #4479). No C++/C#. De-risk Reactor + SwapChainPanel FIRST — it's the only novel/uncertain piece; everything else is a known-good port.
• Links punktfunk-core directly (Cargo path dep, features = ["quic"]) — no C ABI, exactly like the GTK client. NativeClient is already Sync (mutexed plane receivers), so it drops into a UI app cleanly. The C ABI (punktfunk_connect + next_au/next_audio/next_rumble/next_hidout/ send_input/send_rich_input) is the Apple path; the native Rust clients call crates/punktfunk-core/src/client.rs (NativeClient) methods directly.
• Video widget = WinUI 3 SwapChainPanel (built-in), fed a D3D11 swapchain via ISwapChainPanelNative::SetSwapChain.
• Decode = FFmpeg-next + D3D11VA (HEVC; Main10 for 10-bit/HDR — see below).
• Audio playback = WASAPI render + Opus decode (opus crate, vendors libopus via cmake; set CMAKE_POLICY_VERSION_MINIMUM=3.5).
• Input capture→send: the client captures LOCAL input and sends it. Mouse (abs + relative) + keyboard via the inverse VK table (port keymap.rs); gamepad via SDL3 (already a workspace dep, cross-platform) → NativeClient::send_input/send_rich_input. (SendInput/ViGEm are HOST-side injection — not used by the client.)
• Discovery = mdns-sd (cross-platform, browses _punktfunk._udp).
• Trust = shared client identity + SPAKE2 PIN pairing + TOFU (port trust.rs; same identity files/logic as the other native clients).

The reference: crates/punktfunk-client-linux/src/

Port these files (near 1:1; only the platform layers change):

Linux file	Role	Windows swap
main.rs / app.rs	app shell, lifecycle	WinUI 3 App/Window via Reactor
ui_hosts.rs	host list / connect screen	WinUI 3 page
ui_settings.rs	settings	WinUI 3 page
ui_stream.rs	the streaming view	WinUI 3 page hosting SwapChainPanel
video.rs	FFmpeg decode + present	FFmpeg D3D11VA → D3D11 swapchain in SwapChainPanel
audio.rs	Opus decode + playback	WASAPI render (was PipeWire)
session.rs	NativeClient connect + plane pumps	reuse almost verbatim (core is cross-platform)
trust.rs	identity, PIN, TOFU	reuse almost verbatim
discovery.rs	mDNS browse	reuse verbatim (mdns-sd)
keymap.rs	inverse VK table	reuse; Windows VK is the native source so this is simpler
gamepad.rs	SDL3 pad capture + rumble/feedback	reuse almost verbatim (SDL3 is cross-platform)

session.rs, trust.rs, discovery.rs, keymap.rs, gamepad.rs are mostly platform-neutral (they touch punktfunk-core + SDL3 + mdns, all cross-platform) — expect to reuse them with minimal changes. The real work is video.rs (D3D11VA + swapchain), audio.rs (WASAPI), and the WinUI shell.

10-bit + HDR (NEW — landed this session, the client MUST handle it)

The host now negotiates and emits HEVC Main10 + BT.2020 PQ HDR10 when the captured desktop is HDR (and 10-bit SDR Main10 when negotiated). The Apple client already does the matching present; the Windows client should mirror it:

• Advertise caps in the Hello: video_caps = VIDEO_CAP_10BIT | VIDEO_CAP_HDR (crates/punktfunk-core/src/quic.rs). The host enables 10-bit only if the client advertised it. (The native-client connector in client.rs currently hardcodes video_caps: 0 with a TODO — thread the real caps through when you wire decode; or detect HDR purely in-band, see next.)
• Detect HDR in-band from the HEVC VUI (transfer characteristics = SMPTE ST 2084 / PQ), exactly like the Apple client's VideoDecoder.isHDRFormat (clients/apple/Sources/PunktfunkKit/). This handles a mid-session HDR toggle without renegotiation. Welcome.bit_depth (8/10) is also available.
• Decode Main10 → P010 (10-bit) via D3D11VA.
• Present HDR: swapchain in DXGI_FORMAT_R10G10B10A2_UNORM (or R16G16B16A16_FLOAT), IDXGISwapChain3::SetColorSpace1(DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020) + SetHDRMetaData for HDR10; the host's stream is BT.2020 PQ, so present PQ. For SDR, the existing DXGI_FORMAT_B8G8R8A8_UNORM + BT.709 path. (The host-side HDR conversion math is in crates/punktfunk-host/src/capture/dxgi.rs HDR_PS/HdrConverter if you need the inverse.)

Dev boxes

• No-GPU dev box (UI + connect + software decode): ssh "Enrico Bühler"@192.168.1.57 — Win11 Pro 25H2 (build 26200), QEMU Q35, 8 vCPU/12 GB, no working GPU (so no NVENC, no D3D11VA hardware decode — use FFmpeg software decode here; this box is for UI/connect/protocol work). Has Rust 1.96 MSVC, VS 2026 + VC tools + Win SDK, Win App Runtime 2.2, SudoVDA + Parsec VDD.
• Real-GPU box (HDR / hardware decode / end-to-end): ssh "Enrico Bühler"@192.168.1.174 — Win11, RTX 4090, runs the host. Use it to test the client against a live HDR host.

Dev-loop gotchas (both boxes)

• Build under an ASCII path (C:\Users\Public\…). The username "Enrico Bühler" has a ü → MSVC LNK1201 PDB-write failure under ~/Developer.
• Toolchain gaps: winget install NASM.NASM Kitware.CMake LLVM.LLVM (aws-lc-rs on the quic path, ffmpeg-sys needs libclang).
• CMAKE_POLICY_VERSION_MINIMUM=3.5 in the build env (CMake 4 rejects libopus's old minimum).
• File transfer = sftp (scp is broken under the PowerShell DefaultShell): printf 'put %s /C:/Users/Public/REL/PATH\n' LOCAL | sftp -b - "Enrico Bühler@192.168.1.57" — note the leading slash /C:/…. Let the VM regenerate its own Cargo.lock (don't transfer it).
• Windows clippy is stricter than Linux CI and cfg(windows) code is excluded from Linux CI → run cargo clippy -p punktfunk-client-windows -- -D warnings ON THE VM before committing.
• Work on main; fetch+merge origin/main before pushing.

Suggested phased plan

1. De-risk Reactor (do this first). A windows-rs Reactor (WinUI 3) hello-world that hosts a SwapChainPanel and presents a cleared D3D11 swapchain into it. Confirm the windows-rs Reactor version/API (PR #4479) and ISwapChainPanelNative::SetSwapChain interop. If Reactor proves too raw, the fallback is winit + a child HWND swapchain, but try Reactor first per the decision.
2. Crate scaffold. crates/punktfunk-client-windows, [target.'cfg(windows)'.dependencies]: punktfunk-core { path, features=["quic"] }, windows, the Reactor crate, ffmpeg-next, opus, sdl3, mdns-sd, anyhow, tracing. Mirror crates/punktfunk-client-linux/Cargo.toml.
3. Connect + control plane. Port session.rs + trust.rs; validate headless against the 4090 box (m3-host/serve --native) — handshake, PIN/TOFU, plane counters — before any UI/decode.
4. Decode + present. FFmpeg D3D11VA → SwapChainPanel. SDR (8-bit BGRA) first, then P010 + HDR colorspace (see the HDR section).
5. Audio. WASAPI render + Opus decode (port audio.rs).
6. Input. Mouse + keyboard capture→send (port keymap.rs), gamepad via SDL3 (port gamepad.rs), feedback from next_rumble/next_hidout.
7. Discovery + UI. Port discovery.rs + ui_hosts.rs + ui_settings.rs to WinUI pages.

Key references

• Template: crates/punktfunk-client-linux/src/* (the client to port).
• Apple HDR present (the pattern to mirror): clients/apple/Sources/PunktfunkKit/{VideoDecoder, MetalVideoPresenter,Stage2Pipeline}.swift — in-band PQ detection, P010 decode, EDR present.
• Core client API: crates/punktfunk-core/src/client.rs (NativeClient).
• Protocol: crates/punktfunk-core/src/quic.rs (Hello.video_caps, Welcome.bit_depth, VIDEO_CAP_10BIT/VIDEO_CAP_HDR).
• Full Windows plan + SudoVDA/host details: docs/windows-host.md.
• Host HDR conversion (for the inverse math): crates/punktfunk-host/src/capture/dxgi.rs (HDR_PS, HdrConverter) + crates/punktfunk-host/src/encode/nvenc.rs (BT.2020/PQ VUI).