diff --git a/CLAUDE.md b/CLAUDE.md index b931bdb..2b03a56 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -113,20 +113,25 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc presenter (wp_presentation feedback, tearing-control, Vulkan Video on NVIDIA) — **wgpu/winit rejected** (no dmabuf import / presentation feedback / shortcuts-inhibit). **Windows stage 1 done 2026-06-15** (`crates/punktfunk-client-windows`, binary - `punktfunk-client`): pure-Rust **winit + Direct3D11 flip-model swapchain** present (WARP - fallback for the GPU-less dev box; runtime-compiled fullscreen-triangle shaders, Contain-fit - letterbox), **FFmpeg software HEVC decode** (D3D11VA hw decode is the follow-up), **WASAPI** - shared-mode render + mic capture, keyboard (physical-`KeyCode`→VK) + absolute mouse + wheel - capture (Moonlight-style click-to-capture, Ctrl+Alt+Shift+Q release), **SDL3** gamepads - (rumble/lightbar/DualSense, built from source), `mdns-sd` discovery, shared client identity - + TOFU + SPAKE2 PIN pairing (`--connect`/`--discover`/`--pair`/`--headless`). Builds + clippy - + fmt + tests green on `x86_64-pc-windows-msvc` (built on the dev VM). **UI = winit + raw - D3D11, NOT WinUI3/Reactor** — a research pass confirmed windows-rs Reactor ships no - `SwapChainPanel`/`SetSwapChain` escape hatch, so it can't host the presenter (the bootstrap - doc's sanctioned fallback). Gotcha: `CARGO_HOME` must be an ASCII path — the `ü` in the dev - box's username breaks SDL3's MSVC precompiled-header build. Next: live host validation - (no GPU on the dev box → glass-to-glass defers to the RTX box), D3D11VA hw decode + 10-bit/HDR - present, a native host-list/settings GUI, and RAWINPUT relative-mouse pointer-lock. + `punktfunk-client`): pure-Rust **WinUI 3** UI via **windows-reactor** (a declarative React-like + framework backed by WinUI; PR #4499 added the `SwapChainPanel` widget + `set_swap_chain`). The + video is a **`SwapChainPanel`** bound to a **D3D11 composition swapchain** (WARP fallback for + the GPU-less dev box; runtime-compiled fullscreen-triangle shaders, Contain-fit letterbox), + driven by reactor's per-frame `on_rendering`. **FFmpeg software HEVC decode** (D3D11VA hw decode + is the follow-up), **WASAPI** render + mic capture, **SDL3** gamepads (rumble/lightbar/DualSense), + `mdns-sd` discovery, and the full trust surface — all **in-app**: host list (live mDNS + saved + + manual), settings (resolution/refresh/mic), SPAKE2 PIN pairing screen, TOFU, pinned-fp-mismatch + re-pair. **Stream input** is Win32 low-level hooks (`WH_KEYBOARD_LL`/`WH_MOUSE_LL`) — reactor + exposes no raw key/pointer events; native Windows VK + absolute mouse (client-rect Contain-fit) + + wheel, Ctrl+Alt+Shift+Q capture toggle. `--headless`/`--discover` keep CLI paths. Builds + clippy + + fmt green on `x86_64-pc-windows-msvc` (on the dev VM). **windows-reactor is unpublished** (git + dep pinned to commit `b4129fcc`; `windows` pinned to the SAME commit so `IDXGISwapChain1` unifies + with `set_swap_chain`); its `build.rs` downloads the Win App SDK NuGets + needs `CARGO_WORKSPACE_DIR` + set (in the VM build env; `/temp`+`/winmd` gitignored). Gotcha: `CARGO_HOME` must be an ASCII path + — the `ü` in the dev box's username breaks SDL3's MSVC precompiled-header build. Next: **on-glass + validation** (the dev VM is headless/Session-0 → the WinUI window needs a display: RDP or the RTX + box), D3D11VA hw decode + 10-bit/HDR present, RAWINPUT relative-mouse pointer-lock, and a per-host + speed test in the UI. 2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~2–4 ms at high res). diff --git a/docs-site/content/docs/clients.md b/docs-site/content/docs/clients.md index 79a94e5..34c2c66 100644 --- a/docs-site/content/docs/clients.md +++ b/docs-site/content/docs/clients.md @@ -57,16 +57,17 @@ punktfunk-client --connect :9777 # skip the picker, start a session imme ## Windows desktop client (in development) `punktfunk-client` for Windows (`crates/punktfunk-client-windows`) is the native graphical client -for Windows — pure Rust, the same `punktfunk/1` core as the Apple and Linux apps, with a winit + -Direct3D11 present surface, WASAPI audio, FFmpeg decode, SDL3 controllers, network discovery, and -PIN pairing. It builds on `x86_64-pc-windows-msvc` and runs the connect/decode/present/input path; -hardware (D3D11VA) decode, 10-bit/HDR present, and a native host-list/settings window are in -progress, so it is not yet packaged. For now it is driven from the command line: +for Windows — pure Rust, the same `punktfunk/1` core as the Apple and Linux apps, with a **WinUI 3** +UI (host list, settings, PIN pairing) and the video on a `SwapChainPanel`, plus WASAPI audio, FFmpeg +decode, SDL3 controllers, network discovery, and PIN pairing. Launch it and pick a host from the +list, just like the Apple and Linux apps. It builds on `x86_64-pc-windows-msvc`; hardware (D3D11VA) +decode, 10-bit/HDR present, and packaging are in progress, so it is not yet shipped. A headless CLI +path exists for scripting/measurement: ```sh -punktfunk-client --discover # list hosts on the network -punktfunk-client --connect :9777 # stream (trust-on-first-use) -punktfunk-client --connect :9777 --pair 1234 # pair with the PIN the host shows +punktfunk-client # open the WinUI 3 window (host list / settings) +punktfunk-client --discover # list hosts on the network +punktfunk-client --headless --connect :9777 # no window: connect, count frames, print stats ``` Until it ships, **Moonlight** remains the recommended way to stream to Windows (see below). diff --git a/docs/windows-client-bootstrap.md b/docs/windows-client-bootstrap.md index 6192156..9074569 100644 --- a/docs/windows-client-bootstrap.md +++ b/docs/windows-client-bootstrap.md @@ -5,28 +5,42 @@ and live-validated on a real RTX 4090; the client is the remaining piece. This d starting point: the locked decisions, the reference code to port, the stack swaps, the dev loop, and the gotchas. Read it top to bottom, then start at **Phase 1** (de-risk Reactor first). -## Status — stage 1 landed (2026-06-15) +## Status — WinUI 3 client landed (2026-06-15) The client is implemented in `crates/punktfunk-client-windows` (binary `punktfunk-client`) and is -**build + clippy + fmt + test green on `x86_64-pc-windows-msvc`** (built on the dev VM). Done: winit -window + **Direct3D11 flip-model swapchain** present (WARP on the GPU-less box; runtime-compiled -fullscreen-triangle shaders, Contain-fit letterbox), **FFmpeg software HEVC decode**, **WASAPI** render -+ mic capture, keyboard/mouse/wheel capture (physical-`KeyCode`→VK, click-to-capture), **SDL3** -gamepads, `mdns-sd` discovery, and the full trust surface (identity + TOFU + SPAKE2 PIN over -`--connect`/`--discover`/`--pair`/`--headless`). +**build + clippy + fmt green on `x86_64-pc-windows-msvc`** (built on the dev VM). It is the **WinUI 3** +client this doc planned: native chrome (host list, settings, in-app SPAKE2 PIN pairing) + the video on +a **`SwapChainPanel`**, all in pure Rust. -- **Reactor was evaluated and rejected** (a research pass + the points below): windows-rs Reactor - ships **no `SwapChainPanel` and no `ISwapChainPanelNative::SetSwapChain` escape hatch**, so it - cannot host a DXGI presenter. The client uses the doc's sanctioned fallback — **winit + a raw - D3D11 swapchain on the HWND** — which builds and runs against WARP on the GPU-less VM. A native - WinUI look would need the `SwapChainPanel` hatch to land upstream first. -- **Build gotcha (in addition to the ASCII *output* path below):** `CARGO_HOME` itself must be on an - **ASCII path** (e.g. `C:\Users\Public\.cargo`). SDL3's `build-from-source` compiles a precompiled - header whose `#include` embeds the registry source path; the `ü` in the dev box's username makes - MSVC's PCH/structured-output fail (`MSB8084` / `C4828`). Set `CARGO_HOME=C:\Users\Public\.cargo`. -- **Still pending:** live host validation (the dev box has no GPU → glass-to-glass numbers defer to - the RTX box), **D3D11VA hardware decode** + **10-bit/HDR present**, a native host-list/settings - GUI (CLI flags for now), and RAWINPUT relative-mouse pointer-lock. Phases 4–7 below are the map. +- **Reactor is viable after all — it is what we use.** The locked decision held. windows-rs + [PR #4499](https://github.com/microsoft/windows-rs/pull/4499) (merged 2026-06-01) added a + `SwapChainPanel` widget to **`windows-reactor`** with `set_swap_chain` over + `CreateSwapChainForComposition` — so a DXGI presenter *can* be hosted. (An earlier read that Reactor + had no swapchain hatch was wrong/stale.) The UI is a declarative React-like tree + (`App::new().render(app)`, `use_state`/`use_resource`/`use_effect` hooks, `list_view`/`text_box`/ + `combo_box`/`content_dialog`/`button`/`ToggleSwitch`); the video page is `swap_chain_panel() + .on_ready(|p| p.set_swap_chain(&sc))` driven by `on_rendering`. **`present.rs`** owns the D3D11 + composition swapchain (WARP fallback, runtime shaders, Contain-fit) — the same renderer, bound to + the panel instead of an HWND. +- **windows-reactor is unpublished** (`version 0.0.0`) and fast-moving — depend on it as a **git dep + pinned to a commit** (`b4129fcc`), and pin the `windows` crate to the **same commit** so the + `IDXGISwapChain1` you pass to `set_swap_chain` satisfies reactor's `windows_core::Interface`. Its + `build.rs` downloads the Windows App SDK NuGets (Foundation/Interactive/Runtime) and stages the + bootstrap DLL + `resources.pri` next to the exe; it **`.unwrap()`s `CARGO_WORKSPACE_DIR`**, so set + it in the build env (`CARGO_WORKSPACE_DIR=C:\Users\Public\punktfunk`). It writes `/temp` + `/winmd` + to the workspace root (gitignored). The App SDK runtime must be installed to *run*. +- **Stream input is Win32 low-level hooks**, not XAML: reactor exposes only keyboard *accelerators* + + pointer *button-state* (no raw key-down/up, no pointer position, no wheel), insufficient for a game + stream. `input.rs` installs `WH_KEYBOARD_LL`/`WH_MOUSE_LL` on the stream page (uninstalled on exit), + maps the pointer through the window client rect, sends native VK + abs mouse + wheel, with a + Ctrl+Alt+Shift+Q capture toggle. (A future alternative: generate `Microsoft.UI.Xaml.UIElement` + bindings from the staged winmd and subscribe to `KeyDown`/`PointerMoved` — scoped to the panel.) +- **Build gotcha:** `CARGO_HOME` must be on an **ASCII path** (`C:\Users\Public\.cargo`). SDL3's + `build-from-source` PCH embeds the registry source path; the `ü` in the dev box's username makes + MSVC fail (`MSB8084` / `C4828`). +- **Still pending:** **on-glass validation** — the dev VM is headless / SSH Session 0, so the WinUI + window can't show there; validate over RDP or on the RTX box. Then **D3D11VA hardware decode** + + **10-bit/HDR present**, RAWINPUT relative-mouse pointer-lock, and a per-host speed test in the UI. ## What we're building