14 Commits

Author	SHA1	Message	Date
enricobuehler	d11f2bf800	fix(host/windows): stop the DDA freeze — kill the HDR format-change storm + throttle ACCESS_LOST recovery ... Two freeze drivers found live on the RTX box (DDA-only, 5K@240 HDR SudoVDA): Step 1 — the per-frame format-change check (995db69) mis-fired EVERY frame in HDR (827+/session): self.hdr_fp16 is derived from the duplication ModeDesc (FP16 scanout mode), but legacy DuplicateOutput always hands back 8-bit BGRA, so the acquired-texture format never equals hdr_fp16 → a rebuild storm (each rebuild re-inits device+NVENC → freeze). Make the acquire check SIZE-only; a real HDR<->SDR toggle still arrives as ACCESS_LOST → recreate_dupl re-detects it. Step 3 — ACCESS_LOST (0x887A0026) churn: HDR overlay/MPO flips invalidate the duplication continuously and the recovery loop had no rate limit (the 250ms throttle guarded only the full rebuild, not the cheap try_reduplicate), so it spun DuplicateOutput + up-to-16ms Acquire and starved the encode thread. Add a last_recover throttle capping ALL recovery attempts to ~one per 5ms; between attempts return None so the caller repeats the last frame, paced at the frame interval (no busy-spin, encode thread keeps running). Real FP16 HDR capture (DuplicateOutput1) + per-loss desktop-reisolation cleanup are the next steps; validate this in SDR first. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 11:54:23 +00:00
enricobuehler	995db69387	fix(host/windows): detect format/size change on the DDA acquire path ... DDA only re-read the duplication format/size on rebuild (recreate_dupl) and initial open. A mid-stream HDR<->SDR flip (FP16<->BGRA — e.g. the SudoVDA output dropping out of HDR for the secure desktop) or a resolution change that does NOT raise ACCESS_LOST left hdr_fp16/width/height stale, so present_acquired copied into a mismatched-format/size target — the secure-desktop "works once, then HDR breaks" symptom. Re-read the acquired texture's desc every frame (as Apollo does) and rebuild on a real change instead of presenting a mismatched frame; throttled like the ACCESS_LOST path so a flapping toggle can't hammer DuplicateOutput. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 11:18:53 +00:00
enricobuehler	6d7301ccf5	fix(windows): two-pass cursor compositing (alpha + XOR) in DXGI capture ... A single DXGI cursor shape can need BOTH an alpha-blended layer AND a screen-inverting (XOR) layer at once — a masked-color text I-beam (opaque hot-spot + inverting bar) or a monochrome cursor mixing opaque and invert pixels. The old path produced ONE BGRA image per shape and picked ONE blend (cursor_invert) for the whole shape, so such mixed cursors rendered wrong (masked-color opaque pixels forced through the invert blend; monochrome (AND=1,XOR=1) invert pixels approximated as solid black). Port Apollo/Sunshine's decomposition: convert_pointer_shape now returns a CursorShape with optional alpha/xor layers; CursorCompositor holds tex_alpha + tex_xor and draw_layer renders each with its own blend (alpha = src-over, HDR-scaled; XOR = inversion, unscaled — it operates on the framebuffer reference). The CPU software path blends both layers too. Empty layers are never uploaded or drawn. Removes the single cursor_invert flag. Fixes #13 in docs/apollo-comparison.md. Independently reviewed (ship); Windows-only code — compile verified by CI / dev VM. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 09:48:34 +00:00
enricobuehler	7bf2899301	fix(host/windows): secure-desktop black screen — capture the real frame, don't seed black ... Root cause (confirmed live: "black until I pressed a key, then the image came back"): the secure desktop (lock/login/UAC) is STATIC, and DXGI Desktop Duplication only emits a frame on CHANGE. On the normal→secure switch the duplication is rebuilt (recreate_dupl / try_reduplicate), and we then SEEDED A BLACK frame as last_present — which the static secure desktop never replaced (no change-frame) until the user pressed a key. So we streamed black. Fix: after rebuilding the duplication, CAPTURE the current desktop frame instead of seeding black. A freshly-created duplication's first AcquireNextFrame returns the full current desktop; grab it and present it. New `present_acquired` factors the frame-processing out of `acquire`; both recovery paths now call it: - recreate_dupl: after adopting the new duplication, acquire+present the real frame (born-lost ACCESS_LOST / no-initial-frame → seed black as fallback and let the 250ms-throttled caller retry — a brief flash, then real content). - try_reduplicate: adopt-first, then capture its probe frame (was discarded). Also (independently-correct safe fixes, per the adversarial review): - DesktopWatcher computes the current desktop synchronously in start() before returning, so a session that begins on the secure desktop (reconnect to a locked box) doesn't relay one stale normal-desktop frame (the "flash"). - DuplCapturer::open reasserts SudoVDA isolation at open time (mirrors recreate_dupl) — forces the secure desktop back onto the virtual output if a lock/UAC re-attached a physical monitor. - Instrumentation: dbg_black_seeds counter + a throttled warn when black is seeded, and an info when a real secure-desktop frame is captured on recovery. Pending: the user's real-lock smoke test on the 4090 (a headless PsExec LockWorkStation runs as SYSTEM and can't lock an interactive session, so this must be validated with an actual lock). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 09:15:33 +00:00
enricobuehler	28ab448a29	feat(host/windows): WGC capture backend (overlay/HDR-correct) with watchdog'd DDA fallback ... The capture-architecture reset from the research: add a Windows.Graphics.Capture (WGC) backend that captures the COMPOSED desktop — including the overlay/independent-flip/MPO planes DXGI Desktop Duplication misses — which structurally fixes the frozen HDR animations + video (proven live: a WGC frame decodes to the real 5120x1440 HDR content DDA freezes on). It reuses the whole pipeline unchanged: the WGC frame's GPU texture → same scRGB→BT.2020-PQ shader → NVENC zero-copy; the OS composites the cursor (IsCursorCaptureEnabled) so no manual cursor pass. crates/punktfunk-host/src/ capture/wgc.rs; find_output/make_device/HdrConverter/nudge_cursor_onto made pub(crate) for reuse. Reliability findings + mitigations (live on the RTX 4090): - WGC can't activate under the SYSTEM account (0x80070424) — it needs the interactive user token. The host must run as the user for WGC (run.cmd: drop PsExec -s). DDA still needs SYSTEM for the secure desktop — that token reconciliation (impersonation) is the remaining task. - WGC's Direct3D11CaptureFramePool::CreateFreeThreaded intermittently HANGS on the headless SudoVDA (IddCx) display, correlated with accumulated SudoVDA churn (failed REMOVEs leaving lingering displays); clean-state opens reliably. Since it's a blocking hang, capture_virtual_output runs WGC open on a watchdog thread with a 5s timeout and falls back to DDA on hang/error — the session is NEVER left black: WGC when it opens (fixed animations), DDA otherwise. First-frame nudge added (WGC fires FrameArrived on change; a static desktop otherwise never delivers the first frame). - Default WGC; PUNKTFUNK_CAPTURE=dda forces DDA. DDA path unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 06:32:54 +00:00
enricobuehler	b1e95a386f	fix(host/windows): tiered DXGI recovery — cheap re-DuplicateOutput for the HDR ACCESS_LOST churn ... The HDR path produced a constant ACCESS_LOST churn during real desktop activity (window resize / Start menu / DWM transitions): the duplication keeps getting invalidated but the OUTPUT stays valid (probe passes — 0 born-lost over 72 rebuilds). The old recovery did a FULL rebuild (new device + factory) on every loss, which re-inits NVENC + seeds black + was throttled to 4x/s → mostly-frozen, re-init churn = "broken animations". Now recovery is tiered (mirrors Sunshine): try_reduplicate() does a fresh DuplicateOutput on the EXISTING device+output — no new device, so NO encoder re-init, NO black seed, gpu_copy/HDR textures/last_present kept → frames resume immediately. Only a genuine output loss (secure-desktop switch) or a dead device (DEVICE_REMOVED/RESET) falls back to the full, throttled recreate_dupl. Both paths probe the new duplication and reject a born-lost one. Validated synthetically (1080p60 + 5120x1440@240 HDR): pipeline stable, 0 churn, frames flow. The real-desktop churn needs live validation (can't synthesize DWM animations). Secure-desktop "UI never appears in-session" is a separate issue (output gone in-session; only a fresh monitor re-add works) — still open. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 21:31:14 +00:00
enricobuehler	0a3b92d994	fix(host/windows): HDR cursor brightness (203-nit) + probe-before-adopt recovery; windows-client bootstrap doc ... - HDR cursor: sRGB→linear decode + scale to HDR graphics white (PUNKTFUNK_HDR_CURSOR_NITS, default 203 per BT.2408) in the FP16 cursor composite, so it's no longer ~2.5x too dim. SDR path unchanged; the masked-color (I-beam) inversion blend left unscaled. Cursor cbuffer widened 16→32 + bound to PS. (Validated live: cursor now correct brightness in HDR.) - Secure-desktop recovery: recreate_dupl now PROBES the rebuilt duplication with a 50ms AcquireNextFrame and only adopts it when live (Ok/WAIT_TIMEOUT); a born-lost one (immediate ACCESS_LOST) is dropped so the caller repeats the last frame + retries. Plus reassert_isolation() re-detaches physical displays on every recovery (re-routing the secure/HDR desktop to the virtual output, the delta a fresh reconnect has). NOTE: the born-lost ACCESS_LOST storm in HDR is NOT yet resolved by these — still under investigation (animations/secure-UI/cursor-trail in HDR remain). - docs/windows-client-bootstrap.md: handoff for the native Windows Rust client (windows-rs Reactor + WinUI 3 SwapChainPanel, D3D11VA decode, WASAPI audio, SDL3 input; ports crates/punktfunk-client-linux; 10-bit/HDR present; dev boxes + gotchas). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 21:20:42 +00:00
enricobuehler	bbabc04bca	feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery ... Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an 8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the capture losing the stream across a secure-desktop transition. Protocol (punktfunk-core/quic.rs): - Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`, both as optional trailing bytes (back-compat). client-rs advertises 10-bit via PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives the native clients). Regenerated punktfunk_core.h. Windows host: - 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10, pixelBitDepthMinus8). - HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the BGRA path. Reacts dynamically to the HDR toggle. - Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s, repeating the last good frame between attempts (format-tagged last_present). During a secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the send loop for seconds so the client timed out and disconnected — now the session stays fed (frozen) until the desktop returns. Also seeds a black frame on recovery. Apple client (PunktfunkKit): - Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010, and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged. Switches automatically on a mid-session HDR toggle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 20:28:52 +00:00
enricobuehler	f4b4a6c1e4	feat(host/windows): native res, cursor, secure-desktop capture, windowless SYSTEM launch ... Live-validated Mac <-> RTX 4090 at the display's native 5120x1440@240: - Resolution: set_active_mode enumerates the IDD's advertised modes and sets the requested resolution at the best supported refresh (keeps 5120x1440@240; no more silent fallback to the 1080p OS default when an exact mode is briefly unavailable). - Bitrate auto-cap: NVENC init probes and steps the average bitrate down to the GPU's codec-level max so a high client bitrate connects (matches the Linux host; we do not split NVENC sessions). - Mouse cursor: DXGI duplication excludes the HW cursor; capture the pointer shape/position (GetFramePointerShape) and GPU-composite it before NVENC. Color cursors alpha-blend; masked-color (the text I-beam) uses an INV_DEST_COLOR inversion blend so the caret inverts the screen and shows on any background (no black box); monochrome handled too. - Secure desktop (lock / login / UAC): run as SYSTEM in the interactive session, follow the input desktop via SetThreadDesktop, and on the WinSta switch recreate the D3D11 device and re-resolve the virtual output's GDI name from the stable SudoVDA target id (the name changes across the topology rebuild; the old failure hunted the stale \\.\DISPLAYn and dropped). ACCESS_LOST / INVALID_CALL / device-removed are recoverable, and a mid-stream resolution change is followed (capturer + NVENC re-init at the new size). isolate_displays detaches other monitors so Winlogon renders to the virtual output. One real session recovered 1012 desktop switches and completed cleanly. Windows-only backends; Linux/macOS unaffected. Builds clean on x86_64-pc-windows-msvc. Deployment (windowless SYSTEM launch via PsExec + hidden VBScript) documented in docs/windows-host.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 15:46:34 +00:00
enricobuehler	7654b20b2a	fix(host/windows): NVENC capture on real GPU + HOME-less config dir ... Validated live on an RTX 4090 (Windows 11) host streaming to the Rust reference client over the LAN: SudoVDA virtual display → DXGI Desktop Duplication (D3D11 zero-copy) → NVENC HEVC → punktfunk/1. 720p60 and 1080p60 both clean (181 / 177 frames, 0 mismatched, p50 1.6 / 3.45 ms cross-machine), coexisting with Apollo. Two real-hardware bugs the GPU-less VM couldn't surface: - DXGI capturer: the SudoVDA virtual monitor's DXGI output is enumerated under the GPU that *renders* it (the 4090, LUID 0x15df6), NOT under the SudoVDA "adapter" LUID SudoVDA reports (0x23276). Restricting the output search to that LUID found nothing → "adapter has no output named \\.\DISPLAYn". Now search ALL adapters for the GDI name, bind the D3D11 device to whichever adapter exposes it (NVENC then shares that device), with a settle-retry (the output appears a beat after display creation) and topology logging. - native_pairing / apps: keyed config paths off raw $HOME, which a Windows service/scheduled-task context doesn't set → "HOME unset" hard-fail at m3-host startup. Route both through gamestream::config_dir(), which falls back to %APPDATA% on Windows (cert/paired/apps now under AppData\Roaming). clippy -D warnings + build green on x86_64-pc-windows-msvc (default and --features nvenc) and Linux (78/78 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 09:18:15 +00:00
enricobuehler	9c61b03101	feat(host/windows): ViGEm rumble back-channel + Windows clippy clean ... Wire the host→client rumble path on Windows, the analogue of the Linux uinput EV_FF read loop: a game's force-feedback on the virtual Xbox 360 pad is delivered by ViGEm's notification API (`request_notification` → `spawn_thread`, gated by the crate's `unstable_xtarget_notification` feature). A per-pad background thread stores the latest motor levels; `pump_rumble` relays changes to the client on the universal 0xCA plane (motors scaled 0..255 → 0..65535). Dropping the target aborts the notification, so the thread exits with the session. Live verification still needs a physical pad. Also fix the Windows backends' clippy debt — these modules are cfg- excluded from Linux CI, so `clippy -D warnings` never saw them, and the VM's rustc 1.96 clippy is stricter on shared code than the CI image: - dxgi: manual checked division → checked_div().map_or - sendinput: `x = x | y` → `x |= y` - sudovda: `.then(|| ptr)` → `.then_some(ptr)` - m3 pick_compositor: drop the needless early return (match form) - m3 resolve_compositor: Windows arm is a tail expr, not `return` All Windows backends now build + clippy clean (default and --features nvenc); Linux unaffected (fmt/clippy/check green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 07:43:40 +00:00
enricobuehler	2448a33698	style(host/windows): rustfmt the Windows backends ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:50:16 +00:00
enricobuehler	69ba6ec45d	feat(host/windows): NVENC D3D11 hardware encoder (--features nvenc) ... Zero-copy capture->encode on the GPU via the raw NVENC API (nvidia_video_codec_sdk sys + ENCODE_API; the safe wrapper is CUDA-only). Opens an NV_ENC_DEVICE_TYPE_DIRECTX session on the SAME ID3D11Device as the DXGI capturer (carried on the new FramePayload::D3d11), registers a pool of BGRA textures once, CopyResources each captured texture in and encode_picture; CBR/ULL, infinite GOP, P-only, forced-IDR for RFI. The DXGI capturer gains a D3D11 zero-copy output (selected, like the encoder, by PUNKTFUNK_ENCODER=nvenc) so capture+encode share textures. OFF by default (the nvenc feature pulls the NVENC SDK + cudarc): the default Windows host links without it (openh264 path). cudarc builds toolkit-less via the SDK ci-check feature (dynamic-loading). At link time --features nvenc needs nvencodeapi.lib (NVENC SDK, or an import lib generated from the driver's nvEncodeAPI64.dll) on PUNKTFUNK_NVENC_LIB_DIR. Both default and --features nvenc builds validated to compile+link GPU-less on the VM (import lib generated from the driver DLL). Runtime needs a real NVIDIA GPU. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:39:46 +00:00
enricobuehler	9c2499fd45	feat(host/windows): DXGI Desktop Duplication capture backend ... Windows Capturer via DXGI Desktop Duplication: create a D3D11 device on the SudoVDA adapter (by LUID), find the matching output (by GDI name), DuplicateOutput, and per AcquireNextFrame copy the desktop into a CPU-readable staging texture -> tightly-packed BGRA (FramePayload::Cpu, feeds the openh264 software encoder GPU-lessly). Handles WAIT_TIMEOUT (reuse last frame) and ACCESS_LOST (re-duplicate). Adds FramePayload::D3d11(D3d11Frame) for the future NVENC zero-copy path, and a VirtualOutput.win_capture identity (adapter LUID + GDI name) carried out of the SudoVDA backend. Pure helpers (pack_luid/gdi_name_matches/depad_bgra) unit-tested on the VM; the live duplication path needs a real GPU + an activated SudoVDA monitor. Compiles clean on Windows + Linux. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:06:21 +00:00