28 Commits

Author	SHA1	Message	Date
enricobuehler	fdda7144ed	fix(encode): harden loss-recovery correctness across host encoders (F1–F7) ... Phases 1–4 of design/encoder-recovery-hardening.md — make the shipped RFI/ freeze-until-reanchor recovery honest and rebuild-safe across every backend. F1 — frame-index domain desync: the encode loop now owns a session-lifetime `au_seq`; `Encoder::submit_indexed(au_seq + inflight)` pins NVENC inputTimeStamp and AMF LTR slots to the WIRE frame index, so `invalidate_ref_frames` compares client frame numbers in the same domain and survives adaptive-bitrate rebuilds (an internal counter desynced on the first rebuild → RFI silently dead / an AMF force-ref onto a never-decoded frame). `FrameMsg.frame_index` → `Session::seal_frame_at`; GameStream gets the same via `VideoPacketizer:: packetize(.., Some(idx))`. F2 — Windows NVENC left the client frozen ~1s per loss: NVENC RFI was transparent (no anchor tag) while the session glue armed the 750ms IDR cooldown, so the freeze only lifted on the ~1s keyframe re-ask. NVENC now mirrors AMF — `pending_anchor` tags the first post-invalidate AU (the clean re-anchor P-frame) `recovery_anchor`, incl. the covering-range dedupe re-arm; the client lifts at ~RTT. F3 — speed-test probe filler burned video frame indexes: moved to its own index space (`Packetizer::alloc_probe_index` + `Session::submit_probe_frame`) with a second client reassembly window routed on FLAG_PROBE, gated on the new VIDEO_CAP_PROBE_SEQ Hello bit (mid-session probes declined for older clients). F4 — RFI range sanity cap: forward gaps wider than `packet::RFI_MAX_RANGE` (256) resync via keyframe instead of an out-of-range RFI, host- and client-side (client huge-gap → keyframe in `RfiRecovery::observe` + the pf-client-core pump). F5 — reset() parity: Windows NVENC (teardown + lazy re-init), Linux VAAPI (drop-inner), Linux NVENC (reopen from stored OpenArgs) now give the stall watchdog a heal lever instead of ending the session. F6 — sw.rs `pending: VecDeque` (was `Option`), killing the silent AU drop at capturer pipeline depth > 1. F7 — doc sweep on the RFI/anchor comments. Verified: punktfunk-core lib tests (macOS + Linux), full punktfunk-host suite on Linux (RTX 5070 Ti), Windows compile. Owed: the on-glass client matrix (F2 freeze A/B, AMF LTR spike across a bitrate rebuild). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:17:19 +02:00
enricobuehler	2271f67202	style: rustfmt the recovery + resize changes (Windows CI fmt check) ... The `cargo fmt --check` step on the x86_64-pc-windows-msvc job was failing: the mid-stream loss-recovery and resize-overlay commits landed with unformatted wraps across pf-presenter, pf-client-core, punktfunk-core, pf-console-ui, and a few host files. Applied `cargo fmt`, and hand-relocated two trailing comments in session.rs (a decoded-frame note and the wrap-counter note) to their own lines so rustfmt no longer column-aligns the following comment block to a deep indent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 23:25:06 +02:00
enricobuehler	e55ff1bb28	feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI ... Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike. Client (pf-client-core): after a reference loss the hardware decoder conceals the missing-reference deltas (on RADV, a gray plate with new motion painted over) and returns Ok. The pump now freezes on the last good picture until a clean re-anchor instead of showing the concealment — lifting on a real IDR, an intra-refresh recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The frame_index gap is the early, precise loss signal and drives an RFI request. Host recovery signals (inert unless the backend supports them): - USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR). - USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame. AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the only clean-recovery lever is user LTR: mark frames as long-term references, and on loss force the next frame to re-reference the newest known-good one — a clean P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR before the loss; a range older than the live slots falls back to a keyframe. Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi(). Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success anchors the keyframe cooldown so the client's frames_dropped echo of the same loss is coalesced away rather than emitting a redundant IDR. Spike: synthetic NV12 GPU source for headless AMF encoder testing. Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests (incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux; punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90 forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:31:17 +02:00
enricobuehler	35d97ae6ac	feat(windows): parallel virtual displays — proto v3 ring binding, manager slot map, group topology (W0–W3) ... design/windows-parallel-virtual-displays.md (display-management Stage 7 / §6.6): N simultaneously-live pf-vdisplay monitors, one sealed ring each, every idd-push-security invariant preserved per-ring. - proto v3: SharedHeader._pad → target_id — the ring NAMES its monitor, host-stamped before the magic; the driver publisher refuses a cross-bound ring via the shared, unit-tested frame::check_attach (new DRV_STATUS_BIND_FAIL — the gamepad pad_index validation applied to frames, invariant #10); the host's wait_for_attach surfaces the refusal loudly and self-checks its own stamp. - manager: the one-monitor MgrState becomes a slot map keyed by the client's identity slot (0 = anonymous/GameStream); per-slot reconnect + dead-WUDFHost preempts, slot-scoped begin_idd_setup (a different identity is an admission question, never a preempt), ONE device-level watchdog pinger, per-slot /display/state + /display/release. - group topology: isolate_displays_ccd takes the managed target SET (a sibling slot is never deactivated); SavedConfig + the DDC/PnP axes move to the group record (first-in captures, last-out restores); desktop layout via CCD source origins from the pure layout::arrange (auto-row default, manual pins win), re-applied on create + reconfigure. - admission: the Windows separate→reject override now sits behind the PUNKTFUNK_WIN_SEPARATE=1 validation hatch (the wedge it guarded is structurally gone — a second identity gets its own monitor + ring; default flips in W5 after soak); max_displays and NVENC session-unit budgets decline an unaffordable display AT admission; kick_dwm_compose is process-globally throttled and per-display — cursor jump + 35 ms dwell (a sub-tick jump composes nothing; DWM reads dirties from current state at the next vsync tick). On-glass on the RTX box: V1/V2/V4/V5/V6/V9 green — two paired clients on two monitors streaming ~60 fps each with zero mismatches and zero bind failures, churn-hammer clean (no 0x80070490), per-ring mode-change recreate leaves the sibling untouched, typed budget rejection, fault-injected cross-bind refused loudly with the sibling undisturbed. V7: WUDFHost-kill shared fate is clean; in-process device recovery is a known follow-up (the retired-never-closed control handles block the adapter cycle — reset-pf-vdisplay.ps1 recovers). DWM composes two IDD monitors concurrently at 60 fps — the plan's load-bearing unknown, answered yes. Also carries the client-HDR EDID forwarding that shared this working tree (Hello::display_hdr → AddRequest luminance tail → the monitor's CTA-861.3 HDR block, PUNKTFUNK_CLIENT_PEAK_NITS hatch) and the Deck client fixes (40 ms rumble keep-alive with 1-LSB jitter, HDR self-diagnosing presenter warn, flatpak HDR env). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-11 01:06:44 +02:00
enricobuehler	0dacb37088	feat(linux): zero-copy 4:4:4 — the EGL worker converts to planar YUV444 on the GPU ... A 4:4:4 session no longer falls to the CPU path (SHM capture + swscale RGB→YUV444P + re-upload — the fps-ceiling triple tax). The zero-copy worker grows a Yuv444Blit: three full-res R8 GL passes (the proven BT.709 coefficients; studio or full range per PUNKTFUNK_444_FULLRANGE, read by both processes so pixels and VUI flip together) into ONE stacked 3-plane pitched CUDA allocation — which keeps the worker↔host wire and IPC single-plane. The encoder copies the planes into ffmpeg's yuv444p CUDA surface and hevc_nvenc emits Range-Extensions 4:4:4 natively. ImportKind::Tiled444 is APPENDED to the worker protocol (a worker outliving a replaced host binary must keep the old tags stable; an old worker just errors the import, which the fail machinery already handles). A 4:4:4 session on a LINEAR/gamescope capture — no convert wired there — fails with a clear message instead of letting hevc_nvenc silently subsample. caps().chroma_444 now keys off the session (it missed the GPU path when keyed off the swscale's existence). Live-verified on the CachyOS VM (RTX 5070 Ti): per-frame "imported to CUDA yuv444=true", stream Rext/yuv444p/bt709 in both tv and pc range, no CPU-path warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 18:09:28 +02:00
enricobuehler	d55cde61d3	style: cargo fmt — settle the CSC/tvOS changes' layout (CI Format gate) ... Formatting only, no code change: the signaled-CSC and tvOS commits (1fcf9e11, 3ba19f28) left six files unformatted and the rust job's Format step rejects main. cargo fmt --all --check is clean after this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 17:11:27 +02:00
enricobuehler	1fcf9e11ec	fix(video): honor the signaled CSC matrix end-to-end + tvOS HDR presentation ... Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded 709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer renders with a constant hue error. Host-side signaling made honest across NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth. Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on both cores. Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now splits on the display's live EDR headroom - PQ passthrough when the per-session AVDisplayManager mode switch landed (a real HDR10 output tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer config. The 10-bit stream keeps its full decode depth either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehler	968a9c66c1	fix(host): NVENC AV1 sessions actually open — stop stamping HEVC config onto the AV1 union ... Every AV1 session died at initialize_encoder with NV_ENC_ERR_INVALID_PARAM ("rejected even at the floor bitrate") while negotiation and the session plan looked healthy. Live-bisected on the RTX 4090 box against a 10-bit macOS AV1 session; three independent poisons, all HEVC-isms applied across the codec union: - Unconditional `hevcConfig.tier = 1`: NVENC's AV1 encoder is Main-tier only, and the same write scribbled HEVC offsets into h264Config too. Tier/level are now per-codec; AV1 deliberately writes NEITHER — its preset defaults are the only accepted values, and `level = 0` is NOT autoselect on AV1 (that's Level 2.0 in the AV1 enum — it pins the stream to the smallest level and rejects any real mode). Explicit `idrPeriod = INFINITE` is likewise rejected; with PTD enabled the driver follows the infinite gopLength, matching the proven HEVC pattern of never touching idrPeriod. - The 10-bit branch was gated on bit depth only, never codec: it stamped NV_ENC_HEVC_PROFILE_MAIN10_GUID + hevcConfig bitfields onto AV1 (the killer for HDR/10-bit clients). AV1's Main profile covers 10-bit natively; it now sets output/input depth on its own config. - The split-encode fallback only retried FORCED modes, so an AUTO-mode rejection masqueraded as a bitrate cap and failed the whole clamp search. Any non-disabled mode now retries once with split disabled. (Also corrects the split-mode log legend: NV_ENC_SPLIT_DISABLE_MODE is 15, not 0.) The AV1 RFI DPB write (maxNumRefFramesInDPB = 5) survives the bisect and stays. Verified end to end: 10-bit AV1, macOS client → RTX 4090 host. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 23:02:37 +02:00
enricobuehler	efed8d5a20	fix(host): AMF encoder reconnect reliability on Windows AMD ... A second connection to a Windows AMD host came up black with nothing in the logs. The native AMF encoder's teardown never Flushed before Terminate, so a reconnect whose teardown overlapped the new session (a client may not signal an explicit exit, so session 1 tears down late — on the reconnect preempt grace or the QUIC idle timeout) left AMD's limited VCN encode-session slot occupied. The new session's Init then opened onto a wedged session that returns AMF_OK but never emits an AU. NVENC has no equivalent per-session cap, so NVIDIA never showed it. Recovery couldn't help either: the stall watchdog re-Init'd the SAME context, which can't clear a context/VCN-level fault, so it looped a dead context until MAX_ENCODER_RESETS ended the session. Reliability: - Component::drop now Flushes before Terminate (mirrors reset() and the design doc), releasing the VCN session cleanly so the next session's Init gets a free slot. - reset() escalates to a FULL context teardown once an in-place re-Init has run without producing an AU (resets_without_output >= 2), so a wedged reconnect self-heals via a fresh CreateContext+InitDX11 within the reset budget instead of re-initing a dead context in a loop. Logging (the failure was silent): - Per-context bring-up sequence number (context #N) — distinguishes a first connection from a reconnect's fresh context. - A one-shot "AMF produced its first AU on this context" line; its absence after a context #N bring-up is the smoking gun for a silent VCN wedge. - Terminate result logged on drop for both the component and the context. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 11:09:59 +02:00
enricobuehler	4b88aa63c2	style: rustfmt drift from the native AMF SDK encoder ... The native AMF encoder (6f47aba) landed unformatted, failing CI's Format step (and short-circuiting Clippy/Build/Test). Reformatted amf.rs with the pinned rustfmt 1.96.0 — no functional change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-06 17:39:40 +00:00
enricobuehler	6f47abab8c	feat(host): native AMF SDK encoder for Windows AMD — drop libavcodec ... Direct-SDK AMF encoder (encode/windows/amf.rs), the AMD analogue of the direct-NVENC path, replacing the libavcodec *_amf dispatch. C-vtable FFI pinned to AMF headers v1.4.36, runtime-loaded from the driver's amfrt64.dll (no build feature, no new dependency) exactly as NVENC loads its DLL. - AVC/HEVC (SDR NV12 + 10-bit HDR P010) and AV1 (RDNA3+, probed); a bounded poll retires the libavcodec ~2-frame output hold; native in-place reset(). - Intra-refresh wave (PUNKTFUNK_INTRA_REFRESH), in-band HDR mastering/CLL metadata (*InHDRMetadata -> HEVC SEI / AV1 OBU), and a native codec probe feeding the GameStream advertisement (windows_backend_is_ffmpeg -> windows_backend_is_probed). - AMD dispatch / advertisement / 4:4:4 are native-only; the libavcodec AMF fallback and the PUNKTFUNK_AMF_FFMPEG hatch are removed. FFmpeg serves QSV only (its AMF path retained solely as the latency A/B comparator). - Overload back-pressure: submit bounds in-flight surfaces below the input ring, draining finished AUs (buffered for poll, FIFO-preserved) to free a slot and retry on AMF_INPUT_FULL instead of tearing the encoder down and forcing an IDR; this also closes a latent ring-overwrite corruption seen under load on-glass. Validated on the lab Ryzen iGPU (AMF runtime 1.4.37): HEVC/AVC across a native reset, HEVC Main10 mastering+CLL SEIs byte-verified, intra-refresh accepted, a backpressure burst FIFO-clean, and end-to-end via the macOS client. Measured §5.2 latency A/B: native encode_us p50 ~5 ms (0.31 frame periods) vs libavcodec ~17 ms (1.01). 4:4:4 stays unsupported (VCN hardware limit). Live-gated tests skip cleanly on non-AMD boxes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-06 17:33:23 +02:00
enricobuehler	76bc7fee97	feat(host): default the Windows AMF encode input to zero-copy D3D11 ... On-glass A/B on the Ryzen 7000 iGPU (1080p120 HDR P010, hevc_amf, PUNKTFUNK_PERF stage split): the system-memory readback costs the encode thread 2.7-2.9 ms p50 (6.6 ms p99) per frame in submit; the zero-copy D3D11 pool path does the same work in 0.26 ms p50 (0.5 ms p99) — and on an iGPU the readback also burns the shared memory bandwidth the game needs. The docs-site already promised "on by default ... D3D11 on Windows" since the Linux flip (9814368 was Linux-only); the Windows code now delivers it. PUNKTFUNK_ZEROCOPY becomes a tri-state override: unset defers to a per-vendor default in zerocopy_enabled(vendor) — ON for AMF (validated above; open failures still fall back to system-memory readback), OFF for QSV until it is validated on Intel glass (the fallback only catches *setup* errors; a QSV derive that opens but maps wrong would corrupt silently, so probe-never-assume applies). Explicit values force either way: 0|false|off|no = readback, anything else = zero-copy, so the old presence-style =1 keeps working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 14:36:43 +02:00
enricobuehler	167d590349	fix(host): encode-stall watchdog — heal the silent AMF/QSV freeze in place ... Field reports: Windows AMD/Intel streams freeze after ~3-5 min regardless of desktop activity. Root cause: the libavcodec AMF/QSV poll is non-blocking (EAGAIN -> Ok(None)), and the encode loop's drain treated None as benign without popping `inflight` — a wedged driver (QueryOutput stops producing) meant frames kept being submitted, inflight grew unboundedly, no AU ever reached the send thread, and nothing logged: a silent permanent freeze. The input-side twin: once libavcodec's one-frame buffer fills, avcodec_send_frame EAGAINs and the submit `?` killed the whole session. Add `Encoder::reset()` (in-place encoder rebuild; implemented for AMF/QSV by dropping the wedged libavcodec encoder so the next submit re-opens it on the current device, forced IDR) and an encode-stall watchdog in the stream loop: trip on a poll error, on no AU within max(2 s, 8 frame intervals) while frames are owed, or on an owed backlog worth more than the window's frames (the slow-leak latency-runaway form). Recovery is a bounded (5 consecutive, cleared by any delivered AU) in-place rebuild + forced IDR — a logged ~one-second hiccup instead of a dead stream; exhaustion or a reset-less backend still fails the session with a clear error. Submit failures route through the same bounded recovery. The three existing pipeline-rebuild paths (session switch, mode switch, capture loss) now also clear the stale in-flight records that pointed at the dropped encoder. Backends whose poll blocks (direct NVENC sync, software) can't false-trip: they never return Ok(None) mid-stream and drain inflight below depth each tick. Validated: clippy -D warnings (nvenc,amf-qsv), 191 host tests, synthetic E2E 300/300 frames, and an on-glass AMD iGPU session (1080p120 HDR hevc_amf). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 14:36:29 +02:00
enricobuehler	550b02c8ff	perf(encode/windows): AMF quality=speed + bf=0; drop the useless poll spin ... On-box A/B on the .173 Ryzen 7000 iGPU (720p60, real composition via input injection — an idle virtual desktop composes ~1 fps and gives meaningless encode timings): the encode-time-first `quality=speed` preset + explicit `bf=0` cut host-side encode_us from ~36 ms to ~19.5 ms. The blocking-poll idea from the prior commit was WRONG and is reverted to a single non-blocking receive (default PUNKTFUNK_FFWIN_POLL_MS=0): libavcodec's hevc_amf holds ~2 frames before releasing the oldest (needs frame N+2 to flush N), so a spin between submits provably never yields the owed AU — verified with a 150 ms cap pegging at exactly 150 ms across every usage preset and pipeline depth. That ~2-frame buffer is inherent to the libavcodec wrapper, not host scheduling; the real latency lever is a direct AMF SDK encoder (the AMF analogue of the direct-NVENC path), tracked as the next AMD work item. The env knob is retained for a future VCN/driver where a bounded spin can help. Also measured and rejected: PUNKTFUNK_ZEROCOPY=1 on AMF is ~2x WORSE (68 ms vs 36 ms) — the D3D11 import path adds sync overhead beyond the readback it saves, so the system-memory default stays. GPU-priority elevation is already process-wide (dxgi.rs), so it covers the iGPU encode session with no change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 14:57:39 +00:00
enricobuehler	23b7871648	chore(encode/windows): AMF forensics knobs — PUNKTFUNK_AMF_USAGE + PUNKTFUNK_FFWIN_POLL_MS ... The blocking poll landed but wait_us pegs at exactly the 2-frame-period cap: AMF holds the AU ~2 frame periods regardless of retrieval. Field knobs to bisect on-box (usage preset × poll cap) without rebuild cycles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 14:39:36 +00:00
enricobuehler	8e8b536dae	fix(encode/windows): AMF latency — honor the loop's blocking-poll contract + preset polish ... The session loop's pipeline deferral was designed around direct NVENC, whose poll() BLOCKS in lock_bitstream; libavcodec's AMF wrapper is truly async (EAGAIN until the ASIC finishes), so a single non-blocking receive quantized AU retrieval to the submit cadence: +1–2 frame periods flat (~43 ms p50 at 720p60 on the Ryzen iGPU vs ~3.5 ms of actual encode). FfmpegWinEncoder now tracks in-flight frames and, while an AU is owed, spin-polls with short sleeps bounded to ~2 frame periods (an overloaded encoder degrades to next-tick pickup instead of stalling capture). Also: quality=speed (latency-first, iGPU-class VCN), explicit bf=0 (h264_amf defaults >0 on RDNA3+), AMF low-latency submission mode (FFmpeg ≥6.1, ignored on older). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 14:32:41 +00:00
enricobuehler	4482c97bd3	fix(encode/windows): resolve NVENC at runtime — AMD/Intel hosts no longer crash at start ... The nvenc build linked nvEncodeAPI64.dll's entry points at load time, so a --features nvenc binary hard-crashed on any box without the NVIDIA driver (AMD/Intel). Entry points now come from a runtime LoadLibrary table (encode/windows/nvenc.rs load_api); a missing DLL just falls through the encoder auto-detect to AMF/QSV/software. The generated import lib and all its plumbing (gen-nvenc-importlib.ps1, nvenc.def, PUNKTFUNK_NVENC_LIB_DIR, setup-build-env wiring) are gone. Live-validated on the RTX 4090 box (NVENC session, 7000+ frames). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 12:09:18 +00:00
enricobuehler	42629a746a	fix(host/windows): clippy while_let_loop in the async poll drain ... The rebase onto main picked up the pre-fix loop{match} variant of the async retrieve drain — the exact shape the Windows clippy gate rejects (run 6722 failed on it; the while-let form passed run 6724 on the CI branch). Restore the gated form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 17:31:45 +00:00
enricobuehler	75c113f331	feat(host/windows): two-thread async NVENC retrieve (PUNKTFUNK_NVENC_ASYNC, opt-in) ... The gpu-contention plan's §5.B lever: today submit and the blocking lock_bitstream share one thread, so under a GPU-saturating game the pipeline serializes on the WDDM scheduling wait (1000/17ms ≈ 59fps — the depth-1 collapse; the old 'deeper pipeline just stacks latency' result was a same-thread implementation, not a disproof). Async mode opens the session enableEncodeAsync=1, registers an auto-reset completion event per pool bitstream, and moves the wait+lock+copy+ unlock onto an internal retrieve thread feeding poll() through a channel — the exact split the NVENC guide mandates. Register/map/unmap stay on the encode thread; teardown drops the job channel, joins the thread, THEN destroys the session. In-flight depth is bounded by PUNKTFUNK_NVENC_ASYNC_DEPTH (default 4, hard cap POOL-1) — both for output-buffer reuse and because NVENC encodes the capture ring's textures in place. Idle latency cost ≈ 0 (same-tick pickup); under contention completed frames queue instead of stalling capture. CI-compile validated only — on-glass A/B under game load on the RTX box still pending (box offline). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:33:56 +00:00
enricobuehler	dacc3b5209	feat(host,web): multi-GPU selection — GPU inventory + preference API, web-console GPU card ... - new crate::gpu (compiled on all platforms so the OpenAPI doc stays platform-independent): DXGI / sysfs GPU inventory with reboot-stable ids (PCI vendor:device + occurrence — LUIDs are per-boot), persisted auto/manual preference (<config>/gpu-settings.json, atomic temp+rename with in-memory rollback), one selection with precedence console preference > PUNKTFUNK_RENDER_ADAPTER > max VRAM and graceful fallback when the preferred GPU is absent, plus a live "in use" record (RAII session guard wrapped around every encoder open_video returns) - fix: windows_gpu_vendor derived the encoder backend from DXGI adapter 0 instead of the selected render adapter — on a hybrid box (e.g. Intel iGPU at index 0 + NVIDIA dGPU) the backend could disagree with the GPU the capture ring / IddCx render pin sit on. The NVENC 4:4:4 probe now also runs on the selected adapter (was: OS default), the codec/4:4:4 probe caches are keyed per selected GPU (were process-lifetime OnceLocks), and an explicit PUNKTFUNK_ENCODER conflicting with the selected GPU's vendor warns up front - mgmt API: GET /api/v1/gpus (inventory + mode + preferred + next-session selection with reason + in-use GPU/backend/session-count) and PUT /api/v1/gpus/preference (validates mode/gpu_id before writing); openapi.json regenerated; the vdisplay render pin now also engages for a console preference (not just the env pin) - web console: GPU card on the Host page — list with vendor + VRAM, Automatic / Prefer controls, Preferred / Next session / "In use · backend" badges, missing-preferred-GPU warning and env-pin note; en + de messages - Linux: a matched manual preference picks the VAAPI render node and the NVENC-vs-VAAPI auto choice; auto mode is exactly the previous behavior Validated live on the hybrid laptop (RTX 3500 Ada + Intel Arc Pro, which enumerates twice — the occurrence ids disambiguate): enumerate, prefer, bad-id 400, restart persistence, auto-restore keeping the stored pick. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 13:57:26 +02:00
enricobuehler	28e04e5e62	feat(protocol,host): negotiate video codec + add a GPU-less software (openh264) encode path ... Phase 1 of codec negotiation, and the Linux software H.264 encode path it unblocks. **Codec negotiation (core `quic`):** - `Hello.video_codecs` (bitfield: CODEC_H264/HEVC/AV1) — the client advertises what it can decode; appended as a trailing byte (older client → 0 = HEVC-only, back-compat). - `Welcome.codec` — the single codec the host resolved and will emit; trailing byte (older host → HEVC). - `resolve_codec(client, host_capable)` picks the shared codec (precedence HEVC > AV1 > H.264) or `None` → the host refuses honestly rather than sending an undecodable stream. - Roundtrip + back-compat tests; cbindgen exports the CODEC_* constants. **Software encoder (host):** - The openh264 `OpenH264Encoder` (was Windows-only) is now built on Linux too — it's platform-agnostic (consumes CPU RGB `CapturedFrame`s, statically-bundled openh264). `openh264` moved to the shared linux+windows Cargo target. - `PUNKTFUNK_ENCODER=software` selects it: `open_video` gains a `software` branch (H.264 only), and `session_plan::resolve_encoder` / `capture::gpu_encode` resolve `EncoderBackend::Software` → `output_format().gpu = false`, so the portal capturer delivers CPU RGB. Explicit-only (auto never picks it — a box with a dead driver still has /dev/nvidiactl and would mis-resolve NVENC). **Host codec resolution (`punktfunk1`):** - The native path no longer hardcodes HEVC: it resolves the codec from the client's advertised set ∩ the host's capability (`Codec::host_wire_caps`: software→H.264, else HEVC), threads it through `SessionPlan.codec`, and opens the encoder + validates reconfigures at that codec. A software host + HEVC-only client is refused with a clear error. - 4:4:4 is gated on HEVC (it's HEVC-only). **Probe:** advertises H264|HEVC|AV1 and logs the resolved codec. Validated on the GPU-less dev box: negotiation is live end-to-end (probe advertises 0x07 → host resolves H.264 → Welcome reports it → plan = Software/H264), and the openh264 unit test (CPU RGB → AnnexB IDR) now runs on Linux. Full capture→encode still needs a GPU on this box — every compositor screencast path (KWin GL, gamescope VK_EXT_physical_device_drm, wlroots EGL) requires one; software render (llvmpipe/pixman) can't be captured — so this box exercises negotiation + encoder, not live capture. The software path unblocks GPU-less-*encode* boxes that still have a display GPU. Phase 2 (clients advertising real codecs + decoding per Welcome.codec) is a follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-01 23:13:39 +00:00
enricobuehler	070658d611	fix(host): clear clippy doc_lazy_continuation in the 4:4:4 docs ... A line-wrap put `+`/`*`-style markers at the start of two doc lines, which clippy (Windows host job, rust 1.96) reads as markdown list items whose unindented follow-on lines trip `doc_lazy_continuation` under `-D warnings`: - encode/windows/nvenc.rs `chroma_444` field doc (the failing Windows-host clippy job): "+ chromaFormatIDC = 3" → "and chromaFormatIDC = 3". - encode/linux/vaapi.rs `probe_can_encode_444` doc: "+ validate" → "and validate" (last line, didn't fire yet, but fragile — fixed pre-emptively). Pure doc rewording, no behaviour change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:38:07 +00:00
enricobuehler	7612238a59	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	33328ee417	docs(host): prove the last 3 files + crate-root deny (unsafe-proof program 4/N, final) ... Completes the unsafe-proof program now that the parallel WIP has landed: - idd_push.rs (25 sites), nvenc.rs (7), punktfunk1.rs (21): a SAFETY proof on every unsafe block — D3D11/DXGI COM (same-device textures, immediate-context single-thread, keyed-mutex-held convert), the NVENC SDK table (versioned POD, register/map/lock-bitstream pairing), cross-process shm reads (atomic magic/generation handshake), and the C-ABI harness (each call cross-checked against its abi.rs `# Safety` doc). No SUSPECT (UB) blocks. - capture.rs / encode.rs: the parent-module deny is restored (their WIP children are now proven), and main.rs gains a crate-root #![deny(clippy::undocumented_unsafe_blocks)] — the permanent catch-all gate so no future unsafe block anywhere in the crate can land without a proof. - Fixed 4 blocks the agents missed: unsafe blocks nested inside `assert_eq!(...)` macro args (the comment-above-statement didn't associate) — hoisted to a `let`. - rustfmt-canonicalized the Windows files (the agents' SAFETY comments + some pre-existing 1.9.0 drift) so `cargo fmt --all --check` is clean. Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings AND cargo fmt -p punktfunk-host --check both green with the crate-root deny active. Windows cfg(windows) re-verified on the box next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:57:00 +00:00
enricobuehler	28cc6acb50	feat(windows-host): IDD-push encodes native NV12/P010 (skip NVENC's SM-side CSC) ... GPU-contention work (host-latency plan §5.A): the IDD-push output ring now hands NVENC native YUV instead of RGB, so NVENC skips its internal RGB→YUV colour conversion on the SM/3D engine the running game saturates. - idd_push.rs: out_ring is now NV12 (SDR, BT.709 limited) via a D3D11 VIDEO-engine BGRA→NV12 VideoConverter (keeps the CSC off the contended 3D/compute engine), or P010 (HDR, BT.2020 PQ limited) via the FP16→P010 shader (NVIDIA's VideoProcessor can't do RGB→P010). The ring drops its per-slot RTV (textures only), matching the WGC YUV ring; converters rebuild on a size/HDR flip. - nvenc.rs: NV12 input forces bit_depth=8 so an HDR→SDR toggle (or a 10-bit- negotiated client on an SDR display) re-inits the session at the matching depth — NV12 can't feed a 10-bit session (register_resource rejects it). - punktfunk1.rs: per-stage latency instrumentation under PUNKTFUNK_PERF (cap=try_latest, submit=encode_picture, wait=lock_bitstream µs p50/p99/max) to pinpoint where capture→encoded latency goes under GPU saturation.	2026-06-26 09:35:23 +00:00
enricobuehler	79c8641c3f	docs(host): prove unsafe blocks in the Windows + cross-platform files + gate them (unsafe-proof program 3/N) ... Continues the unsafe-proof program across the Windows/cross-platform host files (~75 blocks, 21 files), each with a SAFETY proof of the real invariant and a per-file #![deny(clippy::undocumented_unsafe_blocks)] gate: capture/windows: dxgi.rs, wgc_relay.rs, wgc.rs, desktop_watch.rs, composed_flip.rs (windows-rs COM: interface validity, same-D3D11-device textures, immediate-context single-thread, borrowed args outlive the call) windows: service.rs (SCM/token/CreateProcessAsUserW/event handles — OwnedHandle liveness, no double-close/signal race), win_display, wgc_helper, interactive vdisplay/windows: manager.rs, pf_vdisplay.rs (SwDeviceCreate/IddCx/ioctl handle liveness via the OnceLock VDM singleton + OwnedHandle) encode/windows: ffmpeg_win.rs (full AVBufferRef refcount audit — balanced, NO leaks, unlike the vaapi sibling), sw.rs cross-platform: gamestream/audio.rs (libopus), gamestream/stream.rs (sendmmsg), inject/windows/sendinput.rs, audio/windows/wasapi_mic.rs, session_tuning.rs, vdisplay.rs Two findings (handled separately): - wgc_relay.rs `unsafe impl Sync for HelperRelay` is UNSOUND (its mpsc Receiver is !Sync) though not live-exploited — marked SUSPECT inline; fix pending box check (it touches the in-flight punktfunk1.rs). - capture.rs / encode.rs (PARENT modules of the WIP idd_push.rs / nvenc.rs) do NOT get the file deny yet — it would propagate the lint into the undocumented WIP children. The deny lands there once those are documented (after the WIP commits). Linux-visible parts verified green (cargo clippy -p punktfunk-host --all-targets -- -D warnings). The cfg(windows) deny gates are box-verified next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:23:25 +00:00
enricobuehler	85973cae43	feat(windows-host): EncoderCaps — query RFI/HDR-SEI caps (Goal-1 stage 5, tightening 3) ... The last §2.3 seam-trait tightening: give `Encoder` a `caps() -> EncoderCaps` so the session glue routes by *query* instead of relying on the no-op/`false` defaults of `invalidate_ref_frames`/`set_hdr_meta`. `EncoderCaps { supports_rfi, supports_hdr_metadata }` is a cheap `Copy` struct. The trait gains a default `caps()` returning `EncoderCaps::default()` (all false) — correct for every SDR/libavcodec backend (Linux NVENC, VAAPI, AMF/QSV, software openh264), so they need no change. Only the Windows direct-NVENC path (`NvencD3d11Encoder`) overrides it, reporting the real `rfi_supported` (probed once at open via `nvEncGetEncodeCaps`) and `hdr` (HDR-SEI on keyframes). Consumer: the GameStream encode loop (`gamestream/stream.rs`) hoists `supports_rfi` once before the loop and gates the loss-recovery path on it — `!(supports_rfi && enc.invalidate_ref_frames(..))` forces a keyframe directly on non-RFI encoders instead of making an always-`false` call every loss event. Behaviour-preserving (same keyframe/RFI outcome), one fewer no-op call, intent explicit. The native host (punktfunk1) uses FEC+keyframes, no RFI consumer. Linux `cargo clippy -p punktfunk-host --all-targets -D warnings` clean; the three edited files are rustfmt-clean. The NVENC override is Windows-only (1:1 with the existing impl style) → CI/on-glass gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 21:27:20 +00:00
enricobuehler	fced221684	refactor(windows-host): confine platform code under windows/ + linux/ folders (Goal-1 stage 6) ... Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the shared HID codecs into `inject/proto/`): capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/ audio/{windows,linux}/ vdisplay/{windows,linux}/ src/windows/ (service, wgc_helper, win_adapter, win_display) src/linux/ (dmabuf_fence, drm_sync, zerocopy/) Done with `#[path]`, NOT a module rename: every file moves into its folder while the `crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::` reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in behaviour with ZERO reference churn, far lower risk than collapsing to a single `crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so the path churn didn't fight them. Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3 remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all 36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file (the inline `mod X {` blocks are self-contained). Box build to follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 18:53:45 +00:00