punktfunk/docs/windows-client-bootstrap.md

Windows native client — bootstrap handoff

A handoff for an agent picking up the native Windows punktfunk/1 client. The host side is done and live-validated on a real RTX 4090; the client is the remaining piece. This doc is the concrete starting point: the locked decisions, the reference code to port, the stack swaps, the dev loop, and the gotchas. Read it top to bottom, then start at Phase 1 (de-risk Reactor first).

Status — WinUI 3 client landed (2026-06-15)

The client is implemented in clients/windows (binary punktfunk-client) and is build + clippy + fmt green on x86_64-pc-windows-msvc and aarch64-pc-windows-msvc (the ARM64 target cross-compiled off the one x64 runner — see windows.yml; signed MSIX for both arches via windows-msix.yml). It is the WinUI 3 client this doc planned: native chrome (host list, settings, in-app SPAKE2 PIN pairing) + the video on a SwapChainPanel, all in pure Rust.

• Reactor is viable after all — it is what we use. The locked decision held. windows-rs PR #4499 (merged 2026-06-01) added a SwapChainPanel widget to windows-reactor with set_swap_chain over CreateSwapChainForComposition — so a DXGI presenter can be hosted. (An earlier read that Reactor had no swapchain hatch was wrong/stale.) The UI is a declarative React-like tree (App::new().render(app), use_state/use_resource/use_effect hooks, list_view/text_box/ combo_box/content_dialog/button/ToggleSwitch); the video page is swap_chain_panel() .on_ready(|p| p.set_swap_chain(&sc)) driven by on_rendering. present.rs owns the D3D11 composition swapchain (WARP fallback, runtime shaders, Contain-fit) — the same renderer, bound to the panel instead of an HWND.
• windows-reactor is unpublished (version 0.0.0) and fast-moving — depend on it as a git dep pinned to a commit (b4129fcc), and pin the windows crate to the same commit so the IDXGISwapChain1 you pass to set_swap_chain satisfies reactor's windows_core::Interface. Its build.rs downloads the Windows App SDK NuGets (Foundation/Interactive/Runtime) and stages the bootstrap DLL + resources.pri next to the exe; it .unwrap()s CARGO_WORKSPACE_DIR, so set it in the build env (CARGO_WORKSPACE_DIR=C:\Users\Public\punktfunk). It writes /temp + /winmd to the workspace root (gitignored). The App SDK runtime must be installed to run.
• Stream input is Win32 low-level hooks, not XAML: reactor exposes only keyboard accelerators + pointer button-state (no raw key-down/up, no pointer position, no wheel), insufficient for a game stream. input.rs installs WH_KEYBOARD_LL/WH_MOUSE_LL on the stream page (uninstalled on exit), maps the pointer through the window client rect, sends native VK + abs mouse + wheel, with a Ctrl+Alt+Shift+Q capture toggle. (A future alternative: generate Microsoft.UI.Xaml.UIElement bindings from the staged winmd and subscribe to KeyDown/PointerMoved — scoped to the panel.)
• Build gotcha: CARGO_HOME must be on an ASCII path (C:\Users\Public\.cargo). SDL3's build-from-source PCH embeds the registry source path; the ü in the dev box's username makes MSVC fail (MSB8084 / C4828).
• Still pending: on-glass validation — the dev VM is headless / SSH Session 0, so the WinUI window can't show there; validate over RDP or on the RTX box. Then D3D11VA hardware decode + 10-bit/HDR present, RAWINPUT relative-mouse pointer-lock, and a per-host speed test in the UI.

What we're building

A native Windows client that connects to a punktfunk/1 host (serve --native / punktfunk1-host), decodes HEVC, presents it low-latency, plays Opus audio, and captures local mouse/keyboard/gamepad to send back — i.e. the Windows analogue of the GTK4 Linux client (clients/linux), which is the architectural template. The Windows client is close to a 1:1 port of the Linux client with the platform layers swapped.

Locked decisions (from the Windows-host/client plan, docs/windows-host.md + project memory)

• Pure Rust. windows-rs + Windows App SDK "Reactor" (WinUI 3 from Rust, merged windows-rs PR #4479). No C++/C#. De-risk Reactor + SwapChainPanel FIRST — it's the only novel/uncertain piece; everything else is a known-good port.
• Links punktfunk-core directly (Cargo path dep, features = ["quic"]) — no C ABI, exactly like the GTK client. NativeClient is already Sync (mutexed plane receivers), so it drops into a UI app cleanly. The C ABI (punktfunk_connect + next_au/next_audio/next_rumble/next_hidout/ send_input/send_rich_input) is the Apple path; the native Rust clients call crates/punktfunk-core/src/client.rs (NativeClient) methods directly.
• Video widget = WinUI 3 SwapChainPanel (built-in), fed a D3D11 swapchain via ISwapChainPanelNative::SetSwapChain.
• Decode = FFmpeg-next + D3D11VA (HEVC; Main10 for 10-bit/HDR — see below).
• Audio playback = WASAPI render + Opus decode (opus crate, vendors libopus via cmake; set CMAKE_POLICY_VERSION_MINIMUM=3.5).
• Input capture→send: the client captures LOCAL input and sends it. Mouse (abs + relative) + keyboard via the inverse VK table (port keymap.rs); gamepad via SDL3 (already a workspace dep, cross-platform) → NativeClient::send_input/send_rich_input. (SendInput/ViGEm are HOST-side injection — not used by the client.)
• Discovery = mdns-sd (cross-platform, browses _punktfunk._udp).
• Trust = shared client identity + SPAKE2 PIN pairing + TOFU (port trust.rs; same identity files/logic as the other native clients).

The reference: clients/linux/src/

Port these files (near 1:1; only the platform layers change):

Linux file	Role	Windows swap
main.rs / app.rs	app shell, lifecycle	WinUI 3 App/Window via Reactor
ui_hosts.rs	host list / connect screen	WinUI 3 page
ui_settings.rs	settings	WinUI 3 page
ui_stream.rs	the streaming view	WinUI 3 page hosting SwapChainPanel
video.rs	FFmpeg decode + present	FFmpeg D3D11VA → D3D11 swapchain in SwapChainPanel
audio.rs	Opus decode + playback	WASAPI render (was PipeWire)
session.rs	NativeClient connect + plane pumps	reuse almost verbatim (core is cross-platform)
trust.rs	identity, PIN, TOFU	reuse almost verbatim
discovery.rs	mDNS browse	reuse verbatim (mdns-sd)
keymap.rs	inverse VK table	reuse; Windows VK is the native source so this is simpler
gamepad.rs	SDL3 pad capture + rumble/feedback	reuse almost verbatim (SDL3 is cross-platform)

session.rs, trust.rs, discovery.rs, keymap.rs, gamepad.rs are mostly platform-neutral (they touch punktfunk-core + SDL3 + mdns, all cross-platform) — expect to reuse them with minimal changes. The real work is video.rs (D3D11VA + swapchain), audio.rs (WASAPI), and the WinUI shell.

10-bit + HDR (NEW — landed this session, the client MUST handle it)

The host now negotiates and emits HEVC Main10 + BT.2020 PQ HDR10 when the captured desktop is HDR (and 10-bit SDR Main10 when negotiated). The Apple client already does the matching present; the Windows client should mirror it:

• Advertise caps in the Hello: video_caps = VIDEO_CAP_10BIT | VIDEO_CAP_HDR (crates/punktfunk-core/src/quic.rs). The host enables 10-bit only if the client advertised it. (The native-client connector in client.rs currently hardcodes video_caps: 0 with a TODO — thread the real caps through when you wire decode; or detect HDR purely in-band, see next.)
• Detect HDR in-band from the HEVC VUI (transfer characteristics = SMPTE ST 2084 / PQ), exactly like the Apple client's VideoDecoder.isHDRFormat (clients/apple/Sources/PunktfunkKit/). This handles a mid-session HDR toggle without renegotiation. Welcome.bit_depth (8/10) is also available.
• Decode Main10 → P010 (10-bit) via D3D11VA.
• Present HDR: swapchain in DXGI_FORMAT_R10G10B10A2_UNORM (or R16G16B16A16_FLOAT), IDXGISwapChain3::SetColorSpace1(DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020) + SetHDRMetaData for HDR10; the host's stream is BT.2020 PQ, so present PQ. For SDR, the existing DXGI_FORMAT_B8G8R8A8_UNORM + BT.709 path. (The host-side HDR conversion math is in crates/punktfunk-host/src/capture/dxgi.rs HDR_PS/HdrConverter if you need the inverse.)

Dev boxes

• No-GPU dev box (UI + connect + software decode): ssh "Enrico Bühler"@192.168.1.57 — Win11 Pro 25H2 (build 26200), QEMU Q35, 8 vCPU/12 GB, no working GPU (so no NVENC, no D3D11VA hardware decode — use FFmpeg software decode here; this box is for UI/connect/protocol work). Has Rust 1.96 MSVC, VS 2026 + VC tools + Win SDK, Win App Runtime 2.2, SudoVDA + Parsec VDD.
• Real-GPU box (HDR / hardware decode / end-to-end): ssh "Enrico Bühler"@192.168.1.174 — Win11, RTX 4090, runs the host. Use it to test the client against a live HDR host.

Dev-loop gotchas (both boxes)

• Build under an ASCII path (C:\Users\Public\…). The username "Enrico Bühler" has a ü → MSVC LNK1201 PDB-write failure under ~/Developer.
• Toolchain gaps: winget install NASM.NASM Kitware.CMake LLVM.LLVM (aws-lc-rs on the quic path, ffmpeg-sys needs libclang).
• CMAKE_POLICY_VERSION_MINIMUM=3.5 in the build env (CMake 4 rejects libopus's old minimum).
• File transfer = sftp (scp is broken under the PowerShell DefaultShell): printf 'put %s /C:/Users/Public/REL/PATH\n' LOCAL | sftp -b - "Enrico Bühler@192.168.1.57" — note the leading slash /C:/…. Let the VM regenerate its own Cargo.lock (don't transfer it).
• Windows clippy is stricter than Linux CI and cfg(windows) code is excluded from Linux CI → run cargo clippy -p punktfunk-client-windows -- -D warnings ON THE VM before committing.
• Work on main; fetch+merge origin/main before pushing.

Suggested phased plan

1. De-risk Reactor (do this first). A windows-rs Reactor (WinUI 3) hello-world that hosts a SwapChainPanel and presents a cleared D3D11 swapchain into it. Confirm the windows-rs Reactor version/API (PR #4479) and ISwapChainPanelNative::SetSwapChain interop. If Reactor proves too raw, the fallback is winit + a child HWND swapchain, but try Reactor first per the decision.
2. Crate scaffold. clients/windows, [target.'cfg(windows)'.dependencies]: punktfunk-core { path, features=["quic"] }, windows, the Reactor crate, ffmpeg-next, opus, sdl3, mdns-sd, anyhow, tracing. Mirror clients/linux/Cargo.toml.
3. Connect + control plane. Port session.rs + trust.rs; validate headless against the 4090 box (punktfunk1-host/serve --native) — handshake, PIN/TOFU, plane counters — before any UI/decode.
4. Decode + present. FFmpeg D3D11VA → SwapChainPanel. SDR (8-bit BGRA) first, then P010 + HDR colorspace (see the HDR section).
5. Audio. WASAPI render + Opus decode (port audio.rs).
6. Input. Mouse + keyboard capture→send (port keymap.rs), gamepad via SDL3 (port gamepad.rs), feedback from next_rumble/next_hidout.
7. Discovery + UI. Port discovery.rs + ui_hosts.rs + ui_settings.rs to WinUI pages.

Key references

• Template: clients/linux/src/* (the client to port).
• Apple HDR present (the pattern to mirror): clients/apple/Sources/PunktfunkKit/{VideoDecoder, MetalVideoPresenter,Stage2Pipeline}.swift — in-band PQ detection, P010 decode, EDR present.
• Core client API: crates/punktfunk-core/src/client.rs (NativeClient).
• Protocol: crates/punktfunk-core/src/quic.rs (Hello.video_caps, Welcome.bit_depth, VIDEO_CAP_10BIT/VIDEO_CAP_HDR).
• Full Windows plan + SudoVDA/host details: docs/windows-host.md.
• Host HDR conversion (for the inverse math): crates/punktfunk-host/src/capture/dxgi.rs (HDR_PS, HdrConverter) + crates/punktfunk-host/src/encode/nvenc.rs (BT.2020/PQ VUI).