187 Commits

Author	SHA1	Message	Date
enricobuehler	4edfcd4b43	feat(host/windows): two-process mux test toggle + live-validate step 5 ... PUNKTFUNK_SECURE_TEST_PERIOD_MS=N drives a square-wave secure/normal toggle in virtual_stream_relay (instead of the real DesktopWatcher), to exercise the mid-session helper↔DDA mux without a live UAC/lock. Gated behind the env var, in the style of PUNKTFUNK_VIDEO_DROP / PUNKTFUNK_FEC_PCT. Live-validated on the RTX 4090 (host as SYSTEM): with a 4s toggle the mux switched secure(DDA)↔normal(WGC relay) cleanly 5× in one session and the client decoded 308 HEVC Main-10 frames continuously across every switch — the wait-for-IDR latch held with no decode break. The real Winlogon DDA capture is pre-proven by the single-process secure path (f4b4a6c); the toggle exercises the new surface (the mux). Doc updated with the validation + the SYSTEM-mode audio caveat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 08:13:16 +00:00
enricobuehler	8d6cbb81fe	fix(host/windows): merge host PUNKTFUNK_* env into the WGC helper's environment ... CreateProcessAsUserW gives the spawned helper the *user's* environment block, so the host's PUNKTFUNK_ENCODER=nvenc (and ZEROCOPY/PERF/…) were dropped and the helper fell back to the software (H.264-only) encoder — the client negotiated H265 → "WGC helper exited". `merged_env_block` now parses the user block, strips any PUNKTFUNK_* it carried, overlays this (host) process's PUNKTFUNK_* vars, and passes the merged UTF-16 block. Validated live on the RTX 4090 (host as SYSTEM): the helper spawns via CreateProcessAsUserW, runs WGC with no hang (HDR FP16 BT.2020 PQ), opens NVENC (D3D11 Main10), and relays AUs over the pipe — client-rs decoded 411 HEVC Main-10 frames over the LAN. Step 4 (spawn + relay) complete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 08:05:43 +00:00
enricobuehler	140209bbfc	feat(host/windows): two-process secure-desktop step 5 — DDA mux on Winlogon ... `virtual_stream_relay` now muxes the AU source by input desktop. A DesktopWatcher (SYSTEM-only Winlogon-name poll) drives it: the user-session WGC helper relay feeds the normal (Default) desktop; the host's OWN DDA capturer+encoder — opened lazily on the first secure transition, on the same SudoVDA target with a no-op keepalive (the host still holds the real isolation owner) — captures the secure (Winlogon: UAC/lock/login) desktop that WGC can't see. Every switch latches "wait for IDR" and forces the now-active source to emit a keyframe (the two encoders keep independent infinite-GOP state, so the client must resume on an IDR); returning to the helper also drains its stale buffered AUs first. Reconfigure drops the stale-target DDA; keyframe requests route to the live source. Send path (FEC/seal/paced-send) unchanged. Also: wgc_relay gains try_recv (drain on switch-back); open_dda takes dims as args (avoids a closure borrow of the reassigned cur_mode); the forward! macro returns bool with `break 'outer` at the call site (no in-macro label hygiene). cfg-gated windows-only. Live validation (UAC switch over a session) pending. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:55:29 +00:00
enricobuehler	9f50b3930d	feat(host/windows): two-process secure-desktop step 4 — spawn helper + relay AUs ... The SYSTEM host now sources the normal-desktop video from a user-session WGC helper instead of capturing in-process (WGC won't activate as SYSTEM). New `capture/wgc_relay.rs`: `HelperRelay::spawn` launches `m3-host wgc-helper` in the interactive user session via CreateProcessAsUserW (WTSQueryUserToken → DuplicateTokenEx(TokenPrimary) → lpDesktop="winsta0\\default", CREATE_NO_WINDOW) with three anonymous pipes — stdout (framed Annex-B AUs → parsed back to RelayAu), stdin (control: force-keyframe), stderr (helper logs → host tracing). The host holds the SudoVDA keepalive (sole isolation/topology owner); the helper captures by GDI name only. m3.rs: `virtual_stream` dispatches to the new `virtual_stream_relay` when `should_use_helper()` (running as SYSTEM, or PUNKTFUNK_FORCE_HELPER; disable with PUNKTFUNK_NO_HELPER). The relay loop feeds the existing send thread — same FEC/seal/paced-send path. Reconfigure rebuilds the output + re-spawns the helper; keyframe requests forward over the control pipe; helper pts_ns (same-machine monotonic clock) is used directly as capture_ns. Disconnect ends the stream (step 6 adds the relaunch watchdog). wgc_helper.rs: reads the stdin control byte to request an IDR; --bit-depth flag threaded through so SDR 10-bit (Main10) negotiation reaches the helper's encoder. cfg-gated windows-only; Linux/macOS build unaffected. Step 5 (DesktopWatcher mux to host DDA on the Winlogon secure desktop) is next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:43:20 +00:00
enricobuehler	a0f6cddc70	feat(host/windows): WGC helper subcommand (two-process secure-desktop, step 3) ... `m3-host wgc-helper --target-id N --gdi NAME --mode WxHxHz --bitrate K`: the USER-session half of the two-process secure-desktop design (docs/windows-secure-desktop.md). Opens WGC on the EXISTING SudoVDA output by GDI name only (never creates a virtual output — a second topology owner re-trips the ACCESS_LOST born-lost storm), encodes via NVENC, and ships framed Annex-B AUs on stdout for the SYSTEM host to relay onto the live QUIC session: `[u32 magic "PFAU"][u32 len][u64 pts_ns][u8 keyframe][data]`. tracing → stderr so stdout stays the pure AU stream. cfg-gated windows-only; Linux build unaffected. scripts/headless/win-build.cmd: the canonical box build script (sets PUNKTFUNK_BUILD_VERSION so build.rs stamps the version + the NVENC LIB path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:28:05 +00:00
enricobuehler	80e222d3b8	feat(host/windows): DesktopWatcher (secure-desktop detection) — step 1 of the two-process build ... Polls the input-desktop name (OpenInputDesktop + GetUserObjectInformationW(UOI_NAME)) on its own thread → Default/Winlogon atomic; the authoritative normal-vs-secure signal for the capture mux + input path (WTS notifications miss UAC). Not yet wired into the mux (needs the SYSTEM host + WGC helper, steps 3-5 in docs/windows-secure-desktop.md). NOTE: detecting the secure desktop requires the host to run as SYSTEM (a user-token process can't OpenInputDesktop the Winlogon desktop). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:13:33 +00:00
enricobuehler	5c2bcbc2a2	docs(windows): secure-desktop two-process design + WGC impersonation attempt (vestigial) ... Validated design for adding secure-desktop (UAC/lock/login) coverage on top of the shipped WGC animation fix. Key verified constraint: WGC won't activate under SYSTEM (0x80070424) even with thread-level ImpersonateLoggedOnUser, and DDA+SendInput on Winlogon need LOCAL_SYSTEM — so one process can't do both. Architecture: SYSTEM host (QUIC + SudoVDA + DDA-secure + SendInput + AU mux) + a USER-session WGC helper (CreateProcessAsUser) that relays encoded Annex-B AUs over a named pipe; the host muxes helper-AUs (normal desktop) vs its own DDA encoder (secure desktop), switched by a desktop-name watcher. No shared GPU texture (rejected — MIC/keyed-mutex pain); just AU bytes. docs/windows-secure-desktop.md has the ordered, box-testable steps. The impersonate_active_user() in wgc.rs is kept as a harmless no-op (under a user-token process WTSQueryUserToken fails → no impersonation → WGC works natively); it does NOT make WGC work under SYSTEM (the two-process design uses a real user process for WGC instead). + Win32_System_RemoteDesktop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:08:50 +00:00
enricobuehler	28ab448a29	feat(host/windows): WGC capture backend (overlay/HDR-correct) with watchdog'd DDA fallback ... The capture-architecture reset from the research: add a Windows.Graphics.Capture (WGC) backend that captures the COMPOSED desktop — including the overlay/independent-flip/MPO planes DXGI Desktop Duplication misses — which structurally fixes the frozen HDR animations + video (proven live: a WGC frame decodes to the real 5120x1440 HDR content DDA freezes on). It reuses the whole pipeline unchanged: the WGC frame's GPU texture → same scRGB→BT.2020-PQ shader → NVENC zero-copy; the OS composites the cursor (IsCursorCaptureEnabled) so no manual cursor pass. crates/punktfunk-host/src/ capture/wgc.rs; find_output/make_device/HdrConverter/nudge_cursor_onto made pub(crate) for reuse. Reliability findings + mitigations (live on the RTX 4090): - WGC can't activate under the SYSTEM account (0x80070424) — it needs the interactive user token. The host must run as the user for WGC (run.cmd: drop PsExec -s). DDA still needs SYSTEM for the secure desktop — that token reconciliation (impersonation) is the remaining task. - WGC's Direct3D11CaptureFramePool::CreateFreeThreaded intermittently HANGS on the headless SudoVDA (IddCx) display, correlated with accumulated SudoVDA churn (failed REMOVEs leaving lingering displays); clean-state opens reliably. Since it's a blocking hang, capture_virtual_output runs WGC open on a watchdog thread with a 5s timeout and falls back to DDA on hang/error — the session is NEVER left black: WGC when it opens (fixed animations), DDA otherwise. First-frame nudge added (WGC fires FrameArrived on change; a static desktop otherwise never delivers the first frame). - Default WGC; PUNKTFUNK_CAPTURE=dda forces DDA. DDA path unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 06:32:54 +00:00
enricobuehler	84e17fbb49	feat(windows-client): polish the WinUI 3 UI — Mica, cards, typography ... The first cut was a flat stack of buttons. Reworked the chrome to match the windows-reactor gallery's look: - Mica backdrop on the window. - A centred, scrollable, max-width column (`page()` helper) instead of full-width sprawl. - Card surfaces (`border` + `ThemeRef::CardBackground`/`CardStroke`, rounded, padded) grouping content, with all-caps section labels. - Host rows are clickable cards: name (semibold) + address + a PIN/Open/Paired badge + chevron, laid out with a grid so the badge/chevron sit right; tap to connect. - Header row with title + Settings button; a ProgressRing while searching / connecting; settings as grouped "Stream" / "Audio" cards; the pairing screen is a centred card. Pure styling/layout — no logic change. Build + clippy + fmt green on x86_64-pc-windows-msvc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 05:28:15 +00:00
enricobuehler	5029fa727e	feat(windows-client): stream input — Win32 low-level keyboard/mouse hooks ... windows-reactor exposes no raw key-down/up or pointer-position/wheel events (only keyboard accelerators + pointer button-state), so the WinUI 3 stream page captures input below XAML via WH_KEYBOARD_LL / WH_MOUSE_LL, installed on the UI thread when the stream page mounts and removed on unmount (held keys/buttons flushed). The SwapChainPanel fills the window, so the pointer maps through the client rect (Contain-fit into the negotiated mode); keys carry the native Windows VK directly (the wire contract — no table needed). While captured, events inside the video area are swallowed so Alt+Tab/Win reach the host; Ctrl+Alt+Shift+Q toggles capture; clicks on the title bar (outside the client rect) pass through. Mouse buttons (L/M/R/X1/X2), vertical + horizontal wheel, and absolute motion all forwarded. Build + clippy + fmt green on x86_64-pc-windows-msvc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 23:04:17 +00:00
enricobuehler	4994f7f4ba	feat(windows-client): WinUI 3 (windows-reactor) UI — host list, settings, pairing, SwapChainPanel present ... Replaces the winit + raw-HWND-D3D11 shell with a native WinUI 3 UI via windows-reactor (a declarative React-like framework backed by WinUI). The earlier "Reactor can't host a swapchain" read was wrong — PR #4499 (merged 2026-06-01) added a SwapChainPanel widget with `set_swap_chain` over `CreateSwapChainForComposition`. Builds + clippy + fmt green on x86_64-pc-windows-msvc. - Cargo: drop winit/raw-window-handle; add windows-reactor + the `windows` crate, both pinned to the SAME windows-rs commit (b4129fcc) so the `IDXGISwapChain1` handed to `set_swap_chain` satisfies reactor's `windows_core::Interface`. Reactor's build.rs downloads the Windows App SDK NuGets + stages the bootstrap DLL/resources.pri — it requires `CARGO_WORKSPACE_DIR` set (now in the VM build env); /temp + /winmd gitignored. - present.rs: composition swapchain (B8G8R8A8 FLIP_SEQUENTIAL premultiplied) bound to the SwapChainPanel; WARP fallback, runtime D3DCompile shaders, dynamic RGBA texture, Contain-fit letterbox; driven by reactor's per-frame `on_rendering`. - app.rs: the WinUI 3 shell — host list (live mDNS + saved + manual), settings (resolution/ refresh/mic combos+toggle), in-app SPAKE2 PIN pairing screen, and the stream page. Trust gate mirrors the GTK client (pinned → silent, pair=optional → TOFU, else PIN); a pinned-fp mismatch routes to re-pair. The session pump + decoded-frame handoff cross to the UI thread via a Mutex side-channel + thread-locals (the SwapChainPanel sample's pattern). - gamepad: `ctl` sender now `Arc<Mutex<…>>` so GamepadService is `Sync` (shared across the UI and session-pump threads). main.rs: windowed = in-app UI; `--headless`/`--discover` keep the CLI paths. Not yet wired: raw stream keyboard/mouse input (next commit — reactor exposes no raw key/ pointer events, so it needs Win32 low-level hooks or Microsoft.UI.Xaml bindings). On-glass validation pending a display (the dev VM is headless/GPU-less). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 22:58:48 +00:00
enricobuehler	296b976b8f	feat(windows-client): SDL3 gamepads + docs — full stage-1 parity, MSVC-green ... Adds the SDL3 gamepad service (near-verbatim port of the GTK client's — SDL3 is cross-platform) and wires it into the winit app: per-session capture (buttons/axes, DualSense touchpad + motion 0xCC), feedback (rumble, lightbar, raw DualSense effects), single-pad-forwarded model with auto pad-type from the physical controller. Built from source on Windows (no system SDL3). - gamepad.rs: GamepadService (app-lifetime SDL thread) attach/detach on session connect/end; auto_pref resolves "Automatic" to the attached pad's type. - app.rs: hold the service, attach on Connected, detach on Ended/Failed/close. Also simplify the keydown path (drop the identical if/else arms). - main.rs: start the service for the windowed path, resolve GamepadPref from settings + the physical pad. Build gotcha documented + fixed in the dev loop: SDL3's build-from-source MSVC precompiled-header chokes on the `ü` in the dev box's username embedded in the cargo registry path (MSB8084/C4828) — CARGO_HOME must be an ASCII path (C:\Users\Public\.cargo). Unrelated to our code. Docs: CLAUDE.md M4 + docs/windows-client-bootstrap.md status banner (winit-not-Reactor rationale, CARGO_HOME gotcha, what's pending) + docs-site clients.md "Windows desktop client (in development)". Crate is build + clippy + fmt + test green on x86_64-pc-windows-msvc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 22:11:35 +00:00
enricobuehler	e4bdec97bd	feat(windows-client): winit + D3D11 present, WASAPI render, input — builds live on MSVC ... Builds on the prior headless scaffold (which was committed but never VM-built — its audio.rs had two non-compiling wasapi calls). This makes the whole crate build + clippy + fmt + test green on x86_64-pc-windows-msvc and adds the windowed client. - Fix audio.rs: `DeviceEnumerator::new()?.get_default_device(...)` (the free fn doesn't exist) and the 3-arg `write_to_device` (wasapi 0.23). WASAPI shared-mode event-driven render + mic capture now compile and link. - present.rs: D3D11 renderer with WARP fallback (GPU-less dev box), runtime-compiled fullscreen-triangle shaders, dynamic RGBA video-texture upload, Contain-fit letterbox draw, and a flip-model swapchain on the window HWND. - app.rs: winit 0.30 ApplicationHandler — present loop + Moonlight-style click-to-capture input (keyboard via the physical-KeyCode→VK keymap, absolute mouse, wheel, F11), held state flushed on release/focus-loss. - keymap.rs: winit physical KeyCode → Windows VK (layout-independent positional mapping, the analogue of the Linux client's evdev table). - main.rs: windowed default + `--headless` counting mode, `--discover` (mDNS list), `--pair PIN` (SPAKE2 ceremony), `--pin HEX`/known-host/TOFU trust, settings-backed CLI defaults. UI decision: winit + raw D3D11 (the bootstrap doc's sanctioned fallback), confirmed by a research pass — windows-rs "Reactor" ships no SwapChainPanel / SetSwapChain escape hatch, so it can't host the presenter; winit+WARP validates on the GPU-less VM. Native-chrome host-list/settings GUI + D3D11VA hardware decode + 10-bit/HDR present are follow-ups. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 21:59:40 +00:00
enricobuehler	b9f4cf1f3e	fix(host/windows): don't 2-way-split-encode Main10 — it's SLOWER on Ada (fixes broken HDR animations) ... The "broken animations in HDR" was an encode-throughput cliff, not the ACCESS_LOST churn. Measured at 5120x1440@240 HEVC Main10 on the RTX 4090: forced 2-way split-encode = 7.6 ms/frame (~131 fps, well over the 4.17 ms/240fps budget → choppy), while SINGLE engine = 2.8-3.9 ms/frame (~256-357 fps, fits 240). The split/merge overhead dominates for 10-bit; a single Ada NVENC engine already handles 5K@240 Main10 comfortably. So the split decision now forces DISABLE for Main10 (bit_depth >= 10), keeping the existing forced-2 only for 8-bit above 1 Gpix/s. PUNKTFUNK_SPLIT_ENCODE still overrides. Added a split-mode log line. Validated live on the 4090: encode_us_p50 7.6 ms → 3.9 ms at 5K240 HDR with no env override. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 21:40:28 +00:00
enricobuehler	b1e95a386f	fix(host/windows): tiered DXGI recovery — cheap re-DuplicateOutput for the HDR ACCESS_LOST churn ... The HDR path produced a constant ACCESS_LOST churn during real desktop activity (window resize / Start menu / DWM transitions): the duplication keeps getting invalidated but the OUTPUT stays valid (probe passes — 0 born-lost over 72 rebuilds). The old recovery did a FULL rebuild (new device + factory) on every loss, which re-inits NVENC + seeds black + was throttled to 4x/s → mostly-frozen, re-init churn = "broken animations". Now recovery is tiered (mirrors Sunshine): try_reduplicate() does a fresh DuplicateOutput on the EXISTING device+output — no new device, so NO encoder re-init, NO black seed, gpu_copy/HDR textures/last_present kept → frames resume immediately. Only a genuine output loss (secure-desktop switch) or a dead device (DEVICE_REMOVED/RESET) falls back to the full, throttled recreate_dupl. Both paths probe the new duplication and reject a born-lost one. Validated synthetically (1080p60 + 5120x1440@240 HDR): pipeline stable, 0 churn, frames flow. The real-desktop churn needs live validation (can't synthesize DWM animations). Secure-desktop "UI never appears in-session" is a separate issue (output gone in-session; only a fresh monitor re-add works) — still open. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 21:31:14 +00:00
enricobuehler	0a3b92d994	fix(host/windows): HDR cursor brightness (203-nit) + probe-before-adopt recovery; windows-client bootstrap doc ... - HDR cursor: sRGB→linear decode + scale to HDR graphics white (PUNKTFUNK_HDR_CURSOR_NITS, default 203 per BT.2408) in the FP16 cursor composite, so it's no longer ~2.5x too dim. SDR path unchanged; the masked-color (I-beam) inversion blend left unscaled. Cursor cbuffer widened 16→32 + bound to PS. (Validated live: cursor now correct brightness in HDR.) - Secure-desktop recovery: recreate_dupl now PROBES the rebuilt duplication with a 50ms AcquireNextFrame and only adopts it when live (Ok/WAIT_TIMEOUT); a born-lost one (immediate ACCESS_LOST) is dropped so the caller repeats the last frame + retries. Plus reassert_isolation() re-detaches physical displays on every recovery (re-routing the secure/HDR desktop to the virtual output, the delta a fresh reconnect has). NOTE: the born-lost ACCESS_LOST storm in HDR is NOT yet resolved by these — still under investigation (animations/secure-UI/cursor-trail in HDR remain). - docs/windows-client-bootstrap.md: handoff for the native Windows Rust client (windows-rs Reactor + WinUI 3 SwapChainPanel, D3D11VA decode, WASAPI audio, SDL3 input; ports crates/punktfunk-client-linux; 10-bit/HDR present; dev boxes + gotchas). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 21:20:42 +00:00
enricobuehler	e99a1aea43	fix(apple): resolve QoS priority inversions + two Swift concurrency warnings ... Priority inversions (Thread Performance Checker): the Apple client drains every plane on .userInteractive threads (video pump, audio, gamepad feedback) and connects on a .userInitiated Task, but the connector's producer threads ran at the default QoS — so a high-QoS consumer parked waiting on a lower-QoS producer. Pin the connector's producers (outer worker thread, all tokio runtime threads via on_thread_start, and the data-plane spawn_blocking pump) to .userInteractive on Apple so they match the consumers. #[cfg(target_vendor = "apple")] helper using the existing libc dep; no-op off Apple, no Swift-side change (no latency regression). GamepadFeedback.swift: the init's MainActor hop captured self implicitly-strong while the inner $active sink captured it weakly — capture [weak self] in the hop too (the sink stays weak to avoid the retain cycle). StreamPump.swift: the @Sendable pump-thread closure captured the non-Sendable AVSampleBufferDisplayLayer. enqueue/flush are documented thread-safe and only the pump thread drives it after start(), so assert that with nonisolated(unsafe). cargo build/test/clippy/fmt green (core + host); xcframework rebuilt; swift build + iOS/tvOS targets clean with both warnings gone. Runtime confirmation of the inversion warnings needs a GUI run under Xcode's Thread Performance Checker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 22:48:10 +02:00
enricobuehler	bbabc04bca	feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery ... Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an 8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the capture losing the stream across a secure-desktop transition. Protocol (punktfunk-core/quic.rs): - Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`, both as optional trailing bytes (back-compat). client-rs advertises 10-bit via PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives the native clients). Regenerated punktfunk_core.h. Windows host: - 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10, pixelBitDepthMinus8). - HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the BGRA path. Reacts dynamically to the HDR toggle. - Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s, repeating the last good frame between attempts (format-tagged last_present). During a secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the send loop for seconds so the client timed out and disconnected — now the session stays fed (frozen) until the desktop returns. Also seeds a black frame on recovery. Apple client (PunktfunkKit): - Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010, and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged. Switches automatically on a mid-session HDR toggle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 20:28:52 +00:00
enricobuehler	26fbd9ec64	perf(host/windows): zero-copy NVENC — encode the capturer's texture in place (halve 3D-engine load) ... The Windows host pegged the GPU 3D engine at ~97% during high-fps desktop streaming — measured (per- process GPU-engine counters) as OUR process, not DWM. Cause: TWO VRAM->VRAM CopyResource per frame (dupl->gpu_copy in the capturer, then gpu_copy->nvenc_pool in the encoder), and on Windows D3D11 routes copies to render-target textures through the 3D engine (the DMA copy engine sat idle at 7%), so at 240 fps they saturate it and contend with a game's own rendering. Eliminate the second copy: NVENC now registers the capturer's D3D11 texture directly (cached by raw pointer, the cloned texture kept alive until unregister) and encode_pictures it IN PLACE — no encoder-owned input pool, no per-frame copy. Safe because the host encode loop is synchronous (capture -> submit -> poll, where lock_bitstream blocks until the encode finishes), so the capturer never overwrites the texture mid-encode; documented in the module header in case that ever changes. 2 GPU copies/frame -> 1 (the remaining dupl->gpu_copy is unavoidable; that DXGI surface is transient). Measured: SM/compute ~10-15% at ~217 fps 5K (was ~20% at only ~48 fps with two copies), 3687 frames decoded clean. Windows-only; Linux/macOS unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 17:33:07 +00:00
enricobuehler	c830246037	feat(host/windows): UDP send offload + NVENC 2-way split-encode (1 Gbps+ / 5K@240) ... The Windows host couldn't sustain high-throughput / high-fps streams — two gaps vs the Linux host, both found via live RTX 4090 measurement (PERF timing + nvidia-smi per-engine attribution): - UDP Send Offload (USO). punktfunk-core's UdpTransport sent one packet per `send` syscall on Windows (send_batch/send_gso were Linux-only), capping throughput at high packet rates. Add a Windows `send_gso` override using `WSASendMsg` + `UDP_SEND_MSG_SIZE` (the Windows analogue of Linux UDP GSO) via windows-sys — one syscall segments a coalesced <=512-segment super-buffer to the connected peer. On by default with auto-fallback (PUNKTFUNK_GSO=0 disables, error latches off); plugs into the existing paced send path. SO_SNDBUF (32MB) was already cross-platform. - NVENC 2-way split-frame encoding. A single Ada NVENC session tops out ~0.8 Gpix/s, so 5K@240 (1.77 Gpix/s) took ~8 ms/frame -> a ~125 fps ceiling at high motion (the in-game stutter). Set NV_ENC_INITIALIZE_PARAMS.splitEncodeMode = TWO_FORCED above ~1 Gpix/s (matching the Linux libavcodec split_encode_mode path) to use both 4090 encoders — measured ~8 ms -> ~4 ms/frame at throughput. Env override PUNKTFUNK_SPLIT_ENCODE; init-failure fallback disables it (e.g. H264). Windows-only paths; Linux/macOS unaffected. Builds clean on x86_64-pc-windows-msvc. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 16:52:59 +00:00
enricobuehler	f4b4a6c1e4	feat(host/windows): native res, cursor, secure-desktop capture, windowless SYSTEM launch ... Live-validated Mac <-> RTX 4090 at the display's native 5120x1440@240: - Resolution: set_active_mode enumerates the IDD's advertised modes and sets the requested resolution at the best supported refresh (keeps 5120x1440@240; no more silent fallback to the 1080p OS default when an exact mode is briefly unavailable). - Bitrate auto-cap: NVENC init probes and steps the average bitrate down to the GPU's codec-level max so a high client bitrate connects (matches the Linux host; we do not split NVENC sessions). - Mouse cursor: DXGI duplication excludes the HW cursor; capture the pointer shape/position (GetFramePointerShape) and GPU-composite it before NVENC. Color cursors alpha-blend; masked-color (the text I-beam) uses an INV_DEST_COLOR inversion blend so the caret inverts the screen and shows on any background (no black box); monochrome handled too. - Secure desktop (lock / login / UAC): run as SYSTEM in the interactive session, follow the input desktop via SetThreadDesktop, and on the WinSta switch recreate the D3D11 device and re-resolve the virtual output's GDI name from the stable SudoVDA target id (the name changes across the topology rebuild; the old failure hunted the stale \\.\DISPLAYn and dropped). ACCESS_LOST / INVALID_CALL / device-removed are recoverable, and a mid-stream resolution change is followed (capturer + NVENC re-init at the new size). isolate_displays detaches other monitors so Winlogon renders to the virtual output. One real session recovered 1012 desktop switches and completed cleanly. Windows-only backends; Linux/macOS unaffected. Builds clean on x86_64-pc-windows-msvc. Deployment (windowless SYSTEM launch via PsExec + hidden VBScript) documented in docs/windows-host.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 15:46:34 +00:00
enricobuehler	ecd7d4a7e3	feat(android): mic uplink + connect-screen redesign ... Microphone uplink (client → host's virtual mic, 0xCB) and a cleaner connect screen. Mic (Rust-heavy, mirrors the audio playback path in reverse): - crates/punktfunk-android/src/mic.rs: AAudio LowLatency **input** → realtime callback hands captured f32 to a channel → a worker thread Opus-encodes 20 ms stereo frames (48 kHz, VOIP, 64 kbps) and calls NativeClient::send_mic. MicCapture owns the stream + encode thread (RAII stop). - session.rs: SessionHandle gains a `mic` slot; nativeStartMic/nativeStopMic JNI (mirror of audio); stopped in Drop. NativeBridge: the two externs. - Settings: a `micEnabled` flag + a Microphone toggle in SettingsScreen that requests RECORD_AUDIO (denied → stays off). StreamScreen starts the mic only if enabled AND the permission is held. Connect-screen redesign: - One scrollable Column (was a fixed centered layout that could clip with the new tab bar); host rows render via forEach (no nested LazyColumn). Colored section labels ("Saved hosts", "Discovered on the network", "Connect manually"), full-width host cards / fields / Connect button, a header + subtitle, and a muted footer. Verified live (emulator pf_phone -> home-worker-2): toggling mic requests RECORD_AUDIO; with it granted, a session sends mic frames (client "mic: sent=250 … peak=0.439" — real audio) and the host logs "client datagram stream ended … mic=276". Redesigned screen confirmed via screenshots. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 17:05:25 +02:00
enricobuehler	3bcc36c801	feat(android): native display resolution + Settings screen ... The connect mode was hardcoded to 720p60 — violating the "native client resolution, no scaling" invariant. Derive the device's real display mode (landscape, long edge = width) and add a Settings screen to tune the stream, mirroring the Linux/Apple clients. - crates/punktfunk-android: nativeConnect gains bitrateKbps + compositorPref + gamepadPref (CompositorPref/GamepadPref wire bytes via from_u8); these were hardcoded Auto/Auto/0. - app/Settings.kt: Settings (width/height/hz/bitrate/compositor/gamepad; 0 = native/auto) + a SharedPreferences store + nativeDisplayMode (Display.mode, landscape-swapped) + effectiveMode + the UI option tables. - app/SettingsScreen.kt: dropdowns for resolution / refresh / bitrate / compositor / controller. - MainActivity: App owns the settings + a Settings screen; ConnectScreen resolves the effective mode (Native = the display), shows it on the Connect button, and threads the prefs through nativeConnect. Mic + codec selection deferred (mic uplink isn't wired yet; the decoder is HEVC-only). Verified live (emulator pf_phone -> home-worker-2): default -> host mode=2400x1080@60 (the emulator's native display, was 720p); Settings 1920x1080 + 20 Mbps + DualSense -> host mode=1920x1080, requested_kbps=20000, gamepad=dualsense (host created a UHID DualSense). Settings persist across screens; pinned reconnect stays silent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 16:14:01 +02:00
enricobuehler	8ab262f8f8	feat(trust): host-gated trust-on-first-use — PIN pairing mandatory by default ... TOFU let anyone who could reach the host click "Trust" and stream, which defeats the point on a LAN. Make SPAKE2 PIN pairing the default and only way to trust a NEW host; TOFU survives as an explicit HOST opt-in (for fully trusted networks), advertised over mDNS so clients render their trust UI from the host's policy rather than offering trust on faith. Contract: - Host advertises pair=required (default) or pair=optional. pair=required rejects unpaired clients at the handshake; pair=optional accepts them (TOFU). - Clients: a pinned host whose fingerprint matches connects silently; a pinned host whose fingerprint CHANGED forces re-pairing via PIN (no re-trust shortcut); a NEW host is offered TOFU only if it advertised pair=optional, otherwise PIN pairing is mandatory; a manually-typed or unknown-policy host is always PIN. Host (crates/punktfunk-host/src/main.rs): - m3-host now REQUIRES pairing by default (was open by default). New --allow-tofu opts into accepting unpaired clients + advertising pair=optional; pairing is always armed (PIN logged at startup). serve --native was already secure-by-default (serve --open). The mDNS advert and the accept loop already mapped require_pairing -> pair=required + reject; only the m3-host CLI default + help text changed. Clients honor the advertised policy: - Android (MainActivity.kt): TOFU only for a discovered pair=optional host; manual/unknown -> PIN; fp-change -> re-pair only (dropped the "Forget & re-TOFU" shortcut). - Apple (HostDiscovery/SessionModel/ContentView/HostCards/HostStore): new allowsTofu (pair==optional, distinct from unknown); connect() gates .awaitingTrust on it; unpinned non-optional hosts route to the PIN sheet; "Forget Identity" re-pairs rather than re-TOFUs. - Linux (app.rs/ui_hosts.rs/session.rs): ConnectRequest.pair_required -> pair_optional; initiate_connect routes pinned/fp-changed/optional/else; manual + --connect unknown -> PIN; a pinned connect rejected on trust grounds re-pairs. Docs (CLAUDE.md, README.md, docs-site/content/docs/pairing.md): describe the gated model — PIN is the default, TOFU an explicit opt-in with an impostor warning. Verified: host cargo check/clippy/fmt clean; Android built + live (emulator -> home-worker-2): a manual connect now opens the PIN dialog (no Trust button) and the PIN ceremony streams; Apple swift build clean; Linux clippy -D warnings + fmt clean on the Linux box. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 13:27:52 +02:00
enricobuehler	fe9921cc1c	fix(dist): kill the version-shadow + add build provenance (P0) ... The stale code a default install/upgrade got was a TAG LEAK: deb.yml/rpm.yml shared `tags: ['v*']` with the Apple-client release.yml, so the v0.1.0/v0.1.1 tags cut to ship the macOS app ALSO published host packages versioned 0.1.1 — which outranks every rolling 0.0.1~ciN / 0.0.1-0.ciN build in both registries (dpkg/rpm version compares confirm), so `apt install`/`rpm-ostree install` silently fetched ~99-commits-stale code while the READMEs claimed auto-tracking. Two fixes: - Decouple host publishing from Apple `v*` tags: deb.yml/rpm.yml now trigger on `host-v*` only, so a client tag can never poison the host channel again. - Bump the rolling base 0.0.1 -> 0.2.0 (deb `0.2.0~ciN`, rpm `0.2.0-0.ciN`): sits ABOVE the stray 0.1.1 yet BELOW a future 0.2.0 tag, and still climbs monotonically by run number — so `apt upgrade`/`rpm-ostree upgrade` genuinely move forward. Spec default + build scripts + PKGBUILD pkgver bumped to match. Build provenance (so a stale/shadowed host is detectable): build.rs stamps PUNKTFUNK_BUILD_VERSION (set by CI = the full package version, e.g. 0.2.0~ci120.g802e98d; falls back to the crate version for a plain `cargo build`) into the binary via rustc-env. Surfaced in `punktfunk-host --version`, the startup log, and the mgmt /health + /host `version` field (was a hardcoded CARGO_PKG_VERSION). Deliberately env-driven, not git-derived — the RPM builds from a git-archive tarball with no .git. Version computed BEFORE the build in deb.yml; the spec %build exports it from %{version}-%{release} (and gains --locked for reproducibility parity with the .deb path). Validated: plain build reports 0.0.1, env-stamped build reports 0.2.0~ci999.gdeadbee. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 10:30:21 +00:00
enricobuehler	b0df291ffe	feat(android): pairing/identity — persistent identity, TOFU pinning, SPAKE2 PIN ceremony ... M4 Android stage 1 (trust). The client now presents a persistent self-signed identity on every connect, pins host certs trust-on-first-use, and runs the SPAKE2 PIN pairing ceremony — parity with the Apple/Linux clients. The Rust connector already exposed this; this wires it through the JNI + a Keystore-backed Kotlin store + the connect UI. - crates/punktfunk-android: nativeGenerateIdentity (mint), nativeConnect gains certPem/keyPem/pinHex (identity + TOFU/pinned), nativeHostFingerprint, nativePair (SPAKE2). hex32/parse_hex32 helpers. - kit/security: IdentityStore (AndroidKeyStore AES-256-GCM-wrapped PEM blob; StrongBox with TEE fallback; four-state load so a decrypt failure never shadow-mints), PinStore (host-id -> fp-hex in SharedPreferences). obtainIdentity mints once on genuine first run. - app: ConnectScreen loads/mints the identity, looks up the stored pin, and gates connect on a trust decision — TOFU prompt (first connect), fingerprint-changed warning, PIN dialog. - AndroidManifest: allowBackup=false (Keystore keys don't restore; a restored device re-mints rather than carrying a dead blob). Verified live (emulator -> home-worker-2, synthetic m3-host): - identity: host logs the presented client fingerprint; stable across an app restart. - TOFU: first-connect prompt -> Trust -> pins the observed host fp -> pinned reconnect skips the prompt. - SPAKE2: PIN ceremony -> "pairing complete — client trusted" -> auto-connect under --require-pairing; wrong PIN / host down -> "Pairing failed". Known follow-up: trust is keyed by mDNS instance id for discovered hosts but by "host:port" for manually-typed ones, so pairing via one path isn't recognized by the other. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 12:29:04 +02:00
enricobuehler	7654b20b2a	fix(host/windows): NVENC capture on real GPU + HOME-less config dir ... Validated live on an RTX 4090 (Windows 11) host streaming to the Rust reference client over the LAN: SudoVDA virtual display → DXGI Desktop Duplication (D3D11 zero-copy) → NVENC HEVC → punktfunk/1. 720p60 and 1080p60 both clean (181 / 177 frames, 0 mismatched, p50 1.6 / 3.45 ms cross-machine), coexisting with Apollo. Two real-hardware bugs the GPU-less VM couldn't surface: - DXGI capturer: the SudoVDA virtual monitor's DXGI output is enumerated under the GPU that *renders* it (the 4090, LUID 0x15df6), NOT under the SudoVDA "adapter" LUID SudoVDA reports (0x23276). Restricting the output search to that LUID found nothing → "adapter has no output named \\.\DISPLAYn". Now search ALL adapters for the GDI name, bind the D3D11 device to whichever adapter exposes it (NVENC then shares that device), with a settle-retry (the output appears a beat after display creation) and topology logging. - native_pairing / apps: keyed config paths off raw $HOME, which a Windows service/scheduled-task context doesn't set → "HOME unset" hard-fail at m3-host startup. Route both through gamestream::config_dir(), which falls back to %APPDATA% on Windows (cert/paired/apps now under AppData\Roaming). clippy -D warnings + build green on x86_64-pc-windows-msvc (default and --features nvenc) and Linux (78/78 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 09:18:15 +00:00
enricobuehler	b2e5878711	feat(host/mgmt): HTTPS + token auth by default (no loopback no-auth fallback) ... The mgmt API already always serves HTTPS (the host identity cert), but on a loopback bind with no token it ran unauthenticated — any local process could drive it. Make auth required ALWAYS: - new mgmt_token::load_or_generate(): token precedence is --mgmt-token > env PUNKTFUNK_MGMT_TOKEN > persisted ~/.config/punktfunk/mgmt-token > freshly generated 32-byte hex, persisted 0600 in KEY=VALUE form (so the bundled web console can source it directly as a systemd EnvironmentFile — one source of truth). config_dir() made pub(crate). - parse_serve() resolves the token via load_or_generate() when unset, so a bare `serve` Just Works with auth on and no operator step. - mgmt::run() drops the loopback no-token exemption and requires a token; require_auth()'s unauthenticated fallback now returns 401. The paired-cert (mTLS) branch is unchanged — Apple client + library auth unaffected. - web /api proxy: 503 (legible) instead of forwarding an empty bearer. - tests: test_app/test_app_native default a token, send() auto-attaches the bearer; blank-token test asserts the new "no token" refusal. 80 pass. - docs: mgmt module doc + host.env.example reflect always-on auth + auto-gen. Compiles, clippy/fmt clean, openapi no drift. Part B (bundle the web console into apt, auto-wired to this token) follows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 08:42:28 +00:00
enricobuehler	104639bcc1	feat(android): DualSense host->client feedback — rumble + lightbar/LEDs/triggers ... M4 Android stage 1 (DualSense feedback, host->client). Two Kotlin poll threads drain the connector's rumble (0xCA) + HID-output (0xCD) planes via blocking native pulls and render in Kotlin (Option B — no JNI upcalls, Android APIs stay in Kotlin). - crates/punktfunk-android: feedback.rs — nativeNextRumble (returns (low<<16)|high, or -1) + nativeNextHidout (writes [kind][fields] into a caller's direct ByteBuffer). Ungated; no new Cargo deps (next_rumble/next_hidout are on the quic feature already). - clients/android: GamepadFeedback.kt — rumble -> VibratorManager (two-motor amplitude), HID Led -> lightbar + PlayerLeds -> player LED via LightsManager (API 33+), adaptive triggers parsed + logged (no public Android API); resolves the connected pad, emulator -> logged no-op. Started/stopped in the StreamScreen lifecycle (stop + join before nativeClose). Verified live (emulator -> synthetic host, PUNKTFUNK_TEST_FEEDBACK=1): client received + decoded the full burst -- rumble low=16384 high=32768, Led r=10 g=20 b=30, PlayerLeds bits=4 player=1, Trigger which=1 mode=0x21 -- matching the host hook exactly. Rendering is a logged no-op on the emulator (no controller); real haptics/lightbar/player-LED need a physical pad. Deferred (need a physical DualSense + device enumeration): client->host rich input (touchpad/motion send_rich_input) and DualSense controller-type negotiation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 10:30:32 +02:00
enricobuehler	1e871854cd	feat(android): gamepad forwarding — buttons + sticks/triggers/dpad → send_input ... M4 Android stage 1 (gamepad). One controller forwarded as pad 0; mirrors the Linux/Apple gamepad mapping (byte-identical GamepadButton/GamepadAxis events). - crates/punktfunk-android: 2 JNI fns (nativeSendGamepadButton/Axis) building the GamepadButton/GamepadAxis InputEvents (flags = pad index 0). - clients/android: Gamepad.kt — BTN_*/AXIS_* wire constants, KEYCODE_*->BTN_* map, and an AxisMapper (joystick MotionEvent -> sticks +-32767 +y-up / triggers 0..255 / HAT->BTN_DPAD_* with on-change gating + release-all reset). MainActivity routes gamepad-source KeyEvents in dispatchKeyEvent (DPAD only when from a gamepad, so keyboard arrows still map to VK) and adds dispatchGenericMotionEvent for joystick axes. Verified live (emulator -> gamescope host, `adb input gamepad keyevent`): host created the virtual X-Box 360 uinput pad (index=0) and received the gamepad datagrams (input=22). Axes can't be adb-injected (joystick MotionEvents) -- build/clippy + code-review this increment; live stick/trigger test deferred to a physical controller. Deferred: device enumeration/selection, controller-type negotiation, DualSense rich input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 10:06:57 +02:00
enricobuehler	2bca89c555	feat(host/windows): Steam library auto-discovery on Windows ... The Steam `LibraryProvider` keyed off `$HOME` + Linux paths, so the game library was empty on Windows. Add Windows discovery: the default Steam install dirs under Program Files (`ProgramFiles(x86)`/`ProgramFiles`/ `ProgramW6432`), with games on other drives picked up via each root's `libraryfolders.vdf` — whose Windows values are backslash-escaped, so unescape `\\` → `\`. The existing root-scan/dedup logic is shared via a new `steam_roots_existing` helper. The custom store (mgmt JSON CRUD) was already cross-platform; only Steam auto-discovery was Linux-only. Not yet covered: a non-default Steam install dir (the registry `Valve\Steam\InstallPath`). Degrades gracefully — no Steam → empty list. clippy -D warnings + library tests green on Windows and Linux. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:59:21 +00:00
enricobuehler	66f579461f	feat(host/windows): GameStream (Moonlight) audio on Windows — stereo ... `serve` gave Moonlight clients no audio on Windows: the GameStream audio stream thread was Linux-only (a non-Linux stub errored). Widen the stereo path to Windows — the encode/RTP/AES-CBC/hand-rolled-RS(4,2)-FEC logic is platform-neutral and already live-validated byte-identical on Linux, and it now runs over the WASAPI capturer + the (already cross-platform) `opus` crate. The cfg gates go from `linux` to `any(linux, windows)`; only the surround path stays Linux-only because its libopus *multistream* encoder needs `audiopus_sys` (a Linux dep) — on Windows a surround request fails cleanly with a "use stereo" error. Linux stays byte-identical (the `SessionEncoder::Surround` variant and its match arm keep `#[cfg(linux)]`, so Linux compiles exactly as before). Verified: clippy -D warnings + host test suite green on both x86_64-pc-windows-msvc (73/73) and Linux (78/78). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:56:11 +00:00
enricobuehler	ff1cc6c6d9	feat(android): input forwarding — keyboard + touch trackpad → send_input ... M4 Android stage 1 (input). Kotlin captures input and forwards it over JNI to NativeClient::send_input (the connector is linked as a Rust crate). - crates/punktfunk-android: 4 JNI send fns (pointer move / button / scroll / key) building InputEvent with the GameStream wire codes — ungated, &self on the Sync connector (safe from the UI thread). - clients/android: Keymap.kt (Android KEYCODE_* -> Windows VK, the host's wire contract, mirroring the Linux/Apple tables); Activity-level dispatchKeyEvent forwards hardware keys to the active session (above the Compose focus system, so it's reliable); a Compose touch-trackpad overlay -- 1-finger drag -> relative move, tap -> left click, 2-finger drag -> scroll. Verified live (emulator -> gamescope host on the LAN box, synthetic `adb input`): host received 31 input datagrams (input=31) and libei injected KeyDown/KeyUp, MouseButtonDown/Up and MouseMove all emitted=true. Physical-mouse pointer capture + gamepad are next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 09:49:27 +02:00
enricobuehler	e3de19b52e	test(host): make two host tests portable to Windows ... The Windows host test suite hit two pre-existing portability failures (the autonomous Windows bring-up never ran `cargo test` on the VM): - `vdisplay::detect_active_session_*` asserted a non-empty XDG runtime dir — a Linux concept with no Windows equivalent. Gate just that assertion to Linux (keep the call so the fn stays used → no dead_code). - `mgmt::openapi_document_is_complete_and_checked_in` did a byte compare against the checked-in spec, which git may check out CRLF on Windows while serde_json emits LF. Compare content with `\r` stripped. Host suite now 73/73 on x86_64-pc-windows-msvc; Linux unchanged (78 ok). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:49:01 +00:00
enricobuehler	9c61b03101	feat(host/windows): ViGEm rumble back-channel + Windows clippy clean ... Wire the host→client rumble path on Windows, the analogue of the Linux uinput EV_FF read loop: a game's force-feedback on the virtual Xbox 360 pad is delivered by ViGEm's notification API (`request_notification` → `spawn_thread`, gated by the crate's `unstable_xtarget_notification` feature). A per-pad background thread stores the latest motor levels; `pump_rumble` relays changes to the client on the universal 0xCA plane (motors scaled 0..255 → 0..65535). Dropping the target aborts the notification, so the thread exits with the session. Live verification still needs a physical pad. Also fix the Windows backends' clippy debt — these modules are cfg- excluded from Linux CI, so `clippy -D warnings` never saw them, and the VM's rustc 1.96 clippy is stricter on shared code than the CI image: - dxgi: manual checked division → checked_div().map_or - sendinput: `x = x | y` → `x |= y` - sudovda: `.then(|| ptr)` → `.then_some(ptr)` - m3 pick_compositor: drop the needless early return (match form) - m3 resolve_compositor: Windows arm is a tail expr, not `return` All Windows backends now build + clippy clean (default and --features nvenc); Linux unaffected (fmt/clippy/check green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:43:40 +00:00
enricobuehler	7d5dbd47b7	fix(host/dualsense): heartbeat virtual DualSense so it isn't dropped when idle ... "Controller disconnected every few seconds" (Forza Horizon, held steady): the virtual UHID DualSense emitted HID report 0x01 ONLY on state change, but a real DualSense streams it continuously (~250 Hz). When the player holds the wheel/throttle steady the client sends no wire events, so the host wrote nothing and /dev/uhid went silent for seconds — the kernel hid-playstation driver / Proton / SDL treat that as an unplugged controller. (The uinput X-Box pad is immune: evdev holds last-known state with no periodic-report requirement.) Add DualSenseManager::heartbeat(max_gap): re-emit each live pad's CURRENT report when it's been silent for max_gap (idempotent — a stale-but-correct frame, never a phantom input; write_state bumps seq+timestamp). write() resets the per-pad timer, so an actively-used pad emits no extra reports — the heartbeat only fills genuine silence. PadBackend::heartbeat() drives it at an 8 ms gap (~125 Hz) for DualSense (no-op for X-Box), called every input-thread tick (the loop already runs ≤4 ms). GET_REPORT feature replies + the pad lifecycle were ruled out by the investigation (pad is created once, never torn down mid-session). Compiles, clippy/fmt clean, 78 host tests pass. Verify on the box: held-idle DualSense stays present in evtest / no SDL CONTROLLERDEVICEREMOVED; Forza no longer toasts "controller disconnected". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:37:49 +00:00
enricobuehler	b8a1b7e469	feat(host/windows): host→client Opus audio — vendored libopus on MSVC ... The `m3` audio_thread (desktop capture → Opus 48 kHz stereo 5 ms CBR → AUDIO_MAGIC datagrams) now runs on Windows, fed by the WASAPI loopback capturer. The `opus` crate vendors libopus via `audiopus_sys` + cmake (no system lib / vcpkg), so it builds on MSVC — moved into a `cfg(any(linux, windows))` deps table and widened the audio_thread cfg to match (the stub now only covers other targets, e.g. macOS). Build note: CMake 4 rejects libopus's old `cmake_minimum_required`; set `CMAKE_POLICY_VERSION_MINIMUM=3.5` when building the host on Windows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:30:00 +00:00
enricobuehler	8c8d576e52	feat(android): host→client audio — Opus → AAudio (LowLatency) ... M4 Android stage 1 (audio). An audio thread pulls Opus packets from the connector (next_audio), decodes to interleaved f32 stereo, and feeds AAudio via its realtime data callback through a jitter ring ported from the Linux client (prime ~3 quanta, drop-oldest cap, re-prime on drain). All in Rust on native threads — symmetric with the video decode path. - crates/punktfunk-android: audio.rs (Opus decode + jitter ring + AAudio callback); SessionHandle gains an audio slot; nativeStartAudio/nativeStopAudio JNI; Drop stops it. Android-only deps: opus 0.3 (libopus via cmake, static) + ndk "audio" (AAudio) — pure C/NDK, no libc++_shared to bundle. - clients/android: NativeBridge start/stop audio, called in the SurfaceView lifecycle. - kit/build.gradle.kts: cargo-ndk env for the libopus cmake build (NDK root, Ninja, LIBOPUS_STATIC/NO_PKG) + --platform 31 (libaaudio is API 26+). Verified live (emulator -> gamescope host on the LAN box): AAudio opened 48k/stereo/f32; a 440 Hz tone played into the host capture sink reached the client decoded -- opus ~200/s, pcm_frames climbing in lockstep, peak=0.089 (real content, not silence), with video streaming concurrently. Some underruns under emulator jitter (verify on hardware). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 09:25:24 +02:00
enricobuehler	336357643c	feat(host): KWin virtual output primary + settle portal env on switch ... Two parked follow-ups from the session-aware host work: #3 — KWin/Mutter virtual output not set primary. The auto-detected desktop path *is* "stream this desktop", but the per-session virtual output wasn't promoted to primary, so KDE/GNOME panels + windows stayed on an unstreamed real output and the streamed screen showed only wallpaper. apply_session_env now defaults PUNKTFUNK_KWIN_VIRTUAL_PRIMARY / PUNKTFUNK_MUTTER_VIRTUAL_PRIMARY on for the auto path (explicit config still wins), so the streamed output becomes the sole desktop. #2 — input flaky after a mid-stream Gaming->Desktop switch. The xdg portal (D-Bus-activated) and the systemd --user env still pointed at the old session, so the host's RemoteDesktop portal opened against a half-stale env: it accepted events but they didn't reach the compositor until a reconnect. New vdisplay::settle_desktop_portal() pushes the live session env into the systemd/D-Bus activation environment and (for KWin) restarts the portal so it re-reads it, mirroring a fresh desktop login (and the existing wlroots portal restart). Called from the mid-stream switch rebuild slot before the injector reopens. GNOME uses Mutter's direct EIS, so it only gets the env push. Compiles, clippy/fmt clean, 78 host tests pass. Live validation on the Bazzite box next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 06:49:53 +00:00
enricobuehler	2448a33698	style(host/windows): rustfmt the Windows backends ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:50:16 +00:00
enricobuehler	8cba886c17	feat(host/windows): ViGEm virtual gamepad backend ... Windows GamepadManager via vigem-client (ViGEmBus) — the uinput-xpad analogue: one virtual Xbox 360 controller per client pad index, created lazily on first State. GameStream/Moonlight already uses the XInput conventions (low-16 button bits, sticks -32768..32767 +Y up, triggers 0..255), so the GamepadFrame->XGamepad mapping is 1:1. Replaces the non-Linux GamepadManager stub (same new/handle/pump_rumble API the m3 PadBackend drives, so no m3 change). Graceful when ViGEmBus is absent (gamepad disabled, session continues). Compiles clean on Windows + Linux; live-test needs the ViGEmBus driver + a physical pad. Rumble back-channel is a TODO (ViGEm notification API). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:46:51 +00:00
enricobuehler	69ba6ec45d	feat(host/windows): NVENC D3D11 hardware encoder (--features nvenc) ... Zero-copy capture->encode on the GPU via the raw NVENC API (nvidia_video_codec_sdk sys + ENCODE_API; the safe wrapper is CUDA-only). Opens an NV_ENC_DEVICE_TYPE_DIRECTX session on the SAME ID3D11Device as the DXGI capturer (carried on the new FramePayload::D3d11), registers a pool of BGRA textures once, CopyResources each captured texture in and encode_picture; CBR/ULL, infinite GOP, P-only, forced-IDR for RFI. The DXGI capturer gains a D3D11 zero-copy output (selected, like the encoder, by PUNKTFUNK_ENCODER=nvenc) so capture+encode share textures. OFF by default (the nvenc feature pulls the NVENC SDK + cudarc): the default Windows host links without it (openh264 path). cudarc builds toolkit-less via the SDK ci-check feature (dynamic-loading). At link time --features nvenc needs nvencodeapi.lib (NVENC SDK, or an import lib generated from the driver's nvEncodeAPI64.dll) on PUNKTFUNK_NVENC_LIB_DIR. Both default and --features nvenc builds validated to compile+link GPU-less on the VM (import lib generated from the driver DLL). Runtime needs a real NVIDIA GPU. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:39:46 +00:00
enricobuehler	04b76ebfc7	feat(host/windows): run serve/m3-host on Windows (config paths + compositor) ... The punktfunk/1 control plane already compiled on Windows; these wire the last gaps so the host actually runs: config_dir falls back to %APPDATA% (HOME\.config when set), paired_path uses it, hostname from COMPUTERNAME, and resolve_compositor short-circuits the Linux session-detection on Windows (SudoVDA is the single backend; vdisplay::open ignores the compositor arg). Validated live on the VM: m3-host creates its identity, binds the QUIC endpoint (fingerprint logged), advertises mDNS (_punktfunk._udp, host from COMPUTERNAME), and accepts sessions. GPU-less validations green: m0 synthetic->openh264->core FEC loopback (120/120, 0 mismatches) and the m3 c_abi_connection_roundtrip control-plane test. Full session capture (SudoVDA->DXGI) + NVENC remain GPU-gated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:15:51 +00:00
enricobuehler	9c2499fd45	feat(host/windows): DXGI Desktop Duplication capture backend ... Windows Capturer via DXGI Desktop Duplication: create a D3D11 device on the SudoVDA adapter (by LUID), find the matching output (by GDI name), DuplicateOutput, and per AcquireNextFrame copy the desktop into a CPU-readable staging texture -> tightly-packed BGRA (FramePayload::Cpu, feeds the openh264 software encoder GPU-lessly). Handles WAIT_TIMEOUT (reuse last frame) and ACCESS_LOST (re-duplicate). Adds FramePayload::D3d11(D3d11Frame) for the future NVENC zero-copy path, and a VirtualOutput.win_capture identity (adapter LUID + GDI name) carried out of the SudoVDA backend. Pure helpers (pack_luid/gdi_name_matches/depad_bgra) unit-tested on the VM; the live duplication path needs a real GPU + an activated SudoVDA monitor. Compiles clean on Windows + Linux. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:06:21 +00:00
enricobuehler	45e5157091	feat(host/windows): WASAPI loopback audio capture ... Windows AudioCapturer via the wasapi crate (0.23): loopback the default render endpoint (Render device + Direction::Capture + shared mode => STREAMFLAGS_LOOPBACK) at 48 kHz stereo f32 with autoconvert, feeding the existing Opus path with no resampling. Dedicated COM-MTA thread owns the !Send WASAPI objects; interleaved f32 chunks leave over a bounded lossy channel; RAII Drop stops + joins. Bring-up handshake reports a missing endpoint as Err so a session continues without audio. open_audio_capture Windows factory arm + module. Init chain validated live on the VM (open succeeds; next_chunk waits on a silent system). Virtual mic deferred (no Windows virtual-audio endpoint). m3 audio_thread wiring + opus hoist land with the integration task. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 00:57:19 +00:00
enricobuehler	cbbeaa5c29	feat(host/windows): openh264 software H.264 encoder (GPU-less path) ... Windows Encoder impl via the openh264 crate (statically-bundled, BSD-2): low-latency screen-content config (Baseline/no-B-frames, bitrate RC, BT.709 limited, near-infinite GOP + forced-IDR recovery via request_keyframe), packed CPU pixels (BGRx/BGRA/RGB/RGBA/RGBx/BGR) -> I420 -> AnnexB with in-band SPS/PPS each IDR. Synchronous: submit encodes immediately, poll hands back the one AU, flush is a no-op. Windows open_video factory selects it (PUNKTFUNK_ENCODER=software|nvenc|auto; NVENC arm lands later), H.264-only with a clear error otherwise, SW bitrate ceiling. Unit-tested live on the VM: synthetic BGRx -> AnnexB IDR + SPS NAL. Unblocks the GPU-less capture->encode->FEC->send pipeline. Compiles clean on Windows + Linux. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 00:43:19 +00:00
enricobuehler	cce2eb60f6	feat(host/windows): SendInput input-injection backend ... Windows InputInjector via SendInput (Win32 KeyboardAndMouse), mirroring the wlroots backend: absolute mouse (MOUSEEVENTF_VIRTUALDESK normalized to the virtual desktop), relative mouse, scancode keyboard (MapVirtualKeyExW + extended-key flagging), scroll (no sign flip — Windows wheel matches GameStream), buttons. Client already sends Windows VK codes (no keycode table). Reattaches the thread to the input desktop (OpenInputDesktop/SetThreadDesktop) to survive UAC/lock switches. New Backend::SendInput, the Windows auto-default in default_backend(), open() arm, windows-crate features. Compiles clean on Windows + Linux. Live injection validates with the in-session host run (SendInput is desktop-isolated from an SSH network logon). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 00:34:40 +00:00
enricobuehler	2264474c68	Merge remote-tracking branch 'origin/main'	2026-06-15 00:05:58 +00:00
enricobuehler	26741feada	feat(host/windows): SudoVDA virtual-display backend (control path) ... Windows VirtualDisplay backend driving SudoVDA (the Apollo IDD) via its DeviceIoControl IOCTL protocol: open by interface GUID, ADD at the client's exact WxH@Hz (mode baked into the IOCTL, no EDID seeding), mandatory watchdog ping thread, QueryDisplayConfig name resolution, RAII Drop -> REMOVE. Wired behind the existing VirtualDisplay trait (open()/probe() Windows arms). Validated live on the GPU-less VM (standalone + via the trait, env-gated test): version 0.2.1, ADD 1920x1080@60 -> target, watchdog hold, REMOVE. Monitor activation into a WDDM path (-> capturable \\.\DisplayN) needs a real GPU and is deferred with capture/NVENC. docs/windows-host.md updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 00:05:40 +00:00
enricobuehler	de7b8ac282	feat(android): video decode pipeline — NDK AMediaCodec → SurfaceView ... M4 Android stage 1 (video). Pull HEVC access units from the connector and render them to the SurfaceView entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI, honoring the native-thread hot-path invariant. - crates/punktfunk-android: decode.rs (one-in/one-out AMediaCodec loop; in-band VPS/SPS/PPS so no out-of-band csd; dims from NativeClient::mode). SessionHandle now holds an Arc<NativeClient> + the decode thread; nativeStartVideo/nativeStopVideo. - clients/android: connect screen (host/port) + full-screen SurfaceView stream screen — surfaceCreated -> nativeStartVideo, leaving -> stop + close. Verified live (Android emulator -> m3-host on the LAN box, ABI v2): QUIC handshake, 8-round clock-skew sync, HEVC decoder configured at 1280x720, and the data plane delivered + fed all 299 access units (the punktfunk/1 NAT hole-punch worked through the emulator's SLIRP). Real-pixel render is pending a non-synthetic source: `m3-host --source synthetic` emits dummy transport payloads (not HEVC), so the decoder correctly produces nothing; `--source virtual` (a compositor on the host) is needed to verify decode-to-screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 02:03:48 +02:00