11 Commits

Author	SHA1	Message	Date
enricobuehler	5249d31dfa	feat(host/linux): cursor-as-metadata — pointer in gamescope streams, no perf hit ... gamescope draws its pointer on a hardware DRM cursor plane that never enters the framebuffer feeding its PipeWire capture node, so captured frames arrive cursorless. Rather than force the producer's Embedded full-frame composite, request the pointer as PipeWire SPA_META_Cursor and composite it ourselves — a ≤256×256 blit into the encoder-OWNED surface, never the compositor's read-only dmabuf. Capture (capture/linux/mod.rs, capture.rs): - choose_cursor_mode() gates on available_cursor_modes(): Metadata > Embedded > Hidden (defaults Embedded on query error — never silently lose the cursor). Applied on both the plain and remote-desktop portal paths. - build_cursor_meta_param() adds a SPA_PARAM_Meta pod requesting SPA_META_Cursor (bitmap up to 256x256) to the connect params on every path. - CursorState parses spa_meta_cursor (id 0 = hidden; position - hotspot; bitmap re-read only when bitmap_offset != 0), normalizing RGBA/BGRA/ARGB/ABGR. Updated in .process before the corrupted/size-0 skip so cursor-only Mutter buffers still track movement. - CapturedFrame gains cursor: Option<CursorOverlay> (Arc rgba + serial) riding the GPU (Dmabuf/Cuda) payloads; the CPU de-pad path composites inline. GPU composite into each zero-copy backend's owned surface: - Vulkan Video + PyroWave: folded into the shared rgb2yuv.comp CSC shader — cursor sampled and alpha-mixed over RGB before the YUV convert (correct chroma, no extra pass). binding 3 (combined image sampler) + 16B push constant, per-slot cursor image uploaded only on serial change. spv regenerated. - CUDA/NVENC: real on-GPU kernel (cursor_blend.cu -> cursor_blend.ptx, compute_75 Turing baseline, JIT-forward) with blend_argb/blend_yuv444/ blend_nv12 (BT.709 limited, matching the shader). Loaded via the hand-rolled libcuda fn-table; blended into the ring InputSurface after copy, degrading to no-cursor on any failure — never drops a frame. VAAPI (AMD/Intel fallback) deferred: Vulkan Video already covers those GPUs; blind libva struct-layout FFI shouldn't ship unverified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 17:19:55 +02:00
enricobuehler	705a8baddf	feat(core,host,client): PyroWave datagram-aligned packets + partial-frame delivery (Phase 4, §4.4) ... PyroWave AUs now packetize on the negotiated shard payload, so a lost datagram costs a few wavelet blocks of localized blur rather than a whole frame — and the client can render an aged-out lossy frame instead of freezing until the next one. Host (opt-in, PyroWave only): - The encoder packetizes at the shard payload behind a 4-byte window prefix (used-len u16 + kind u16). Whole packets pack into WIN_PACKED windows; a packet too large for one shard (PyroWave 32x32 blocks are atomic and can exceed a shard) rides a WIN_FRAG_FIRST/CONT/LAST chain. `set_wire_chunking()` joins the Encoder trait (forwarded through TrackedEncoder — the silent-no-op trap); EncodedFrame.chunk_aligned marks the AU. - virtual_stream tags the AU with USER_FLAG_CHUNK_ALIGNED and re-applies chunking after every encoder (re)build, the adaptive-bitrate rebuild included. Core: - USER_FLAG_CHUNK_ALIGNED (0x40) wire bit. Reassembler opt-in (set_deliver_partial): a chunk-aligned frame that ages out with holes is handed over as Frame{complete:false} — received shards at their exact offsets, missing ranges zero-filled — instead of being dropped. Partials age out on a tight 30ms fuse (PARTIAL_WINDOW_NS) instead of the 120ms loss window: each frame is independently decodable, so an ancient partial has no value in a live stream. Newest-wins. A partial still counts as dropped for loss reporting. Client (PyroWave decode): - The session opts in when codec == PyroWave. The decoder walks the AU window-by-window, skipping zero (missing) windows and reassembling FRAG chains, then decodes whatever survived. A newest-decoded-index guard drops partials the pump has already moved past (no time-travel present). Also fixes a redundant-closure clippy nit in the PyroWave planar-present path. Validated on an RTX 5070 Ti under 2% netem loss with FEC pinned off: 60fps sustained entirely via partials, e2e 43ms p50 (146ms before the fuse) vs 23ms lossless, no keyframe-recovery chatter. Tests green: core 149, host 310 + the GPU-gated encoder smoke (framed-window walk + FRAG reassembly + upstream round-trip), client 26; clippy clean on the pyrowave feature combos. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 11:14:24 +02:00
enricobuehler	49ba1cd11b	fix(host): cover Codec::PyroWave in the Windows NVENC/AMF match arms ... The nine non-exhaustive matches windows-host CI tripped on (run 9917) — all inside encoder objects a PyroWave session can never open (the open_video dispatch routes PyroWave to its own backend on Linux and bails on Windows), so the arms are dispatch-guarded unreachable!(). Verified: cargo check -p punktfunk-host --features nvenc,amf-qsv --release green on the windows-amd64 runner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 01:20:56 +02:00
enricobuehler	a1af916e38	feat(host): in-place encoder rate reconfigure — ABR steps no longer cost an IDR ... Every adaptive-bitrate step used to tear the encoder down and rebuild it, opening on a full IDR (a 20-40x frame-size spike, in-flight AU forfeit and an IDR-cooldown anchor) — exactly when the Automatic controller is climbing. Encoder::reconfigure_bitrate(bps) retargets the LIVE encoder instead (default false, so libavcodec/software paths keep the rebuild fallback, which also still owns the bitrate clamping): - Linux + Windows direct NVENC: nvEncReconfigureEncoder (added to the hand-rolled runtime EncodeApi tables) with resetEncoder=0 / forceIDR=0; the same init/config is re-authored via the new shared build_config/ build_init_params with only avg/max bitrate + VBV (PUNKTFUNK_VBV_FRAMES) moved. On-hardware test: 20→60→10 Mbps in place, zero IDRs (RTX 5070 Ti). - Native AMF: TargetBitrate/PeakBitrate/VBVBufferSize are dynamic properties — SetProperty on the live component, no Terminate/re-Init. - Vulkan Video (HEVC + AV1): stage the rate and emit an ENCODE_RATE_CONTROL control command on the next recorded frame (begin keeps declaring the session's current state, as the spec requires). The session glue tries the in-place retarget first and skips the rebuild/ inflight-clear/IDR-cooldown bookkeeping when it succeeds — the reference chain and the wire-index prediction survive, so RFI keeps working across rate steps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 19:53:18 +02:00
enricobuehler	644c035a34	feat(encode/amf): accept AMF runtime >=1.4.34 (graceful degrade) + log loaded amfrt64.dll identity ... The Windows AMF encoder hard-rejected any runtime <1.4.36 — a Jan-2025 (Adrenalin 25.1.1) driver floor. Every AMD host on an older driver failed the session with "update the AMD driver" after 8 retries, notably Boot Camp Macs whose bundled amfrt64.dll lags far behind. Split the single pin: - AMF_MIN_VERSION (1.4.34): the ABI floor accepted at load. Every vtable slot the FFI mirrors is a base-interface slot stable since well before 1.4.34; the 1.4.35/1.4.36-only features are string-keyed encoder properties already applied via set_prop(required=false), which log-and-continue — so an older driver degrades those features individually instead of failing. - AMF_HEADER_VERSION (1.4.36): the header the mirror targets, now passed to AMFInit capped at min(header, runtime) so claiming a version newer than the runtime can't make AMFInit reject an otherwise-usable older driver. No functionality removed: a >=1.4.36 runtime behaves exactly as before. Also logs, once per process, the AMF runtime version AND the loaded amfrt64.dll's full path + file-version resource (via GetModuleFileNameW + VerQueryValueW). This surfaces the Boot Camp failure mode where the display driver reads 25.x but the System32 amfrt64.dll is a stale build reporting an old AMF version; the too-old decline now names the DLL path/build and points at reboot + DDU reinstall. Not compile-verified: amf.rs is Windows-only and this Linux box can't cross-build it (a dependency's C build fails for the msvc target). Needs cargo check/clippy on the Windows build box / CI. rustfmt-clean; the windows-crate FFI signatures were verified against the on-disk 0.62.2 bindings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 01:32:51 +02:00
enricobuehler	fdda7144ed	fix(encode): harden loss-recovery correctness across host encoders (F1–F7) ... Phases 1–4 of design/encoder-recovery-hardening.md — make the shipped RFI/ freeze-until-reanchor recovery honest and rebuild-safe across every backend. F1 — frame-index domain desync: the encode loop now owns a session-lifetime `au_seq`; `Encoder::submit_indexed(au_seq + inflight)` pins NVENC inputTimeStamp and AMF LTR slots to the WIRE frame index, so `invalidate_ref_frames` compares client frame numbers in the same domain and survives adaptive-bitrate rebuilds (an internal counter desynced on the first rebuild → RFI silently dead / an AMF force-ref onto a never-decoded frame). `FrameMsg.frame_index` → `Session::seal_frame_at`; GameStream gets the same via `VideoPacketizer:: packetize(.., Some(idx))`. F2 — Windows NVENC left the client frozen ~1s per loss: NVENC RFI was transparent (no anchor tag) while the session glue armed the 750ms IDR cooldown, so the freeze only lifted on the ~1s keyframe re-ask. NVENC now mirrors AMF — `pending_anchor` tags the first post-invalidate AU (the clean re-anchor P-frame) `recovery_anchor`, incl. the covering-range dedupe re-arm; the client lifts at ~RTT. F3 — speed-test probe filler burned video frame indexes: moved to its own index space (`Packetizer::alloc_probe_index` + `Session::submit_probe_frame`) with a second client reassembly window routed on FLAG_PROBE, gated on the new VIDEO_CAP_PROBE_SEQ Hello bit (mid-session probes declined for older clients). F4 — RFI range sanity cap: forward gaps wider than `packet::RFI_MAX_RANGE` (256) resync via keyframe instead of an out-of-range RFI, host- and client-side (client huge-gap → keyframe in `RfiRecovery::observe` + the pf-client-core pump). F5 — reset() parity: Windows NVENC (teardown + lazy re-init), Linux VAAPI (drop-inner), Linux NVENC (reopen from stored OpenArgs) now give the stall watchdog a heal lever instead of ending the session. F6 — sw.rs `pending: VecDeque` (was `Option`), killing the silent AU drop at capturer pipeline depth > 1. F7 — doc sweep on the RFI/anchor comments. Verified: punktfunk-core lib tests (macOS + Linux), full punktfunk-host suite on Linux (RTX 5070 Ti), Windows compile. Owed: the on-glass client matrix (F2 freeze A/B, AMF LTR spike across a bitrate rebuild). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:17:19 +02:00
enricobuehler	2271f67202	style: rustfmt the recovery + resize changes (Windows CI fmt check) ... The `cargo fmt --check` step on the x86_64-pc-windows-msvc job was failing: the mid-stream loss-recovery and resize-overlay commits landed with unformatted wraps across pf-presenter, pf-client-core, punktfunk-core, pf-console-ui, and a few host files. Applied `cargo fmt`, and hand-relocated two trailing comments in session.rs (a decoded-frame note and the wrap-counter note) to their own lines so rustfmt no longer column-aligns the following comment block to a deep indent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 23:25:06 +02:00
enricobuehler	e55ff1bb28	feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI ... Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike. Client (pf-client-core): after a reference loss the hardware decoder conceals the missing-reference deltas (on RADV, a gray plate with new motion painted over) and returns Ok. The pump now freezes on the last good picture until a clean re-anchor instead of showing the concealment — lifting on a real IDR, an intra-refresh recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The frame_index gap is the early, precise loss signal and drives an RFI request. Host recovery signals (inert unless the backend supports them): - USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR). - USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame. AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the only clean-recovery lever is user LTR: mark frames as long-term references, and on loss force the next frame to re-reference the newest known-good one — a clean P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR before the loss; a range older than the live slots falls back to a keyframe. Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi(). Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success anchors the keyframe cooldown so the client's frames_dropped echo of the same loss is coalesced away rather than emitting a redundant IDR. Spike: synthetic NV12 GPU source for headless AMF encoder testing. Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests (incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux; punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90 forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:31:17 +02:00
enricobuehler	efed8d5a20	fix(host): AMF encoder reconnect reliability on Windows AMD ... A second connection to a Windows AMD host came up black with nothing in the logs. The native AMF encoder's teardown never Flushed before Terminate, so a reconnect whose teardown overlapped the new session (a client may not signal an explicit exit, so session 1 tears down late — on the reconnect preempt grace or the QUIC idle timeout) left AMD's limited VCN encode-session slot occupied. The new session's Init then opened onto a wedged session that returns AMF_OK but never emits an AU. NVENC has no equivalent per-session cap, so NVIDIA never showed it. Recovery couldn't help either: the stall watchdog re-Init'd the SAME context, which can't clear a context/VCN-level fault, so it looped a dead context until MAX_ENCODER_RESETS ended the session. Reliability: - Component::drop now Flushes before Terminate (mirrors reset() and the design doc), releasing the VCN session cleanly so the next session's Init gets a free slot. - reset() escalates to a FULL context teardown once an in-place re-Init has run without producing an AU (resets_without_output >= 2), so a wedged reconnect self-heals via a fresh CreateContext+InitDX11 within the reset budget instead of re-initing a dead context in a loop. Logging (the failure was silent): - Per-context bring-up sequence number (context #N) — distinguishes a first connection from a reconnect's fresh context. - A one-shot "AMF produced its first AU on this context" line; its absence after a context #N bring-up is the smoking gun for a silent VCN wedge. - Terminate result logged on drop for both the component and the context. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 11:09:59 +02:00
enricobuehler	4b88aa63c2	style: rustfmt drift from the native AMF SDK encoder ... The native AMF encoder (6f47aba) landed unformatted, failing CI's Format step (and short-circuiting Clippy/Build/Test). Reformatted amf.rs with the pinned rustfmt 1.96.0 — no functional change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-06 17:39:40 +00:00
enricobuehler	6f47abab8c	feat(host): native AMF SDK encoder for Windows AMD — drop libavcodec ... Direct-SDK AMF encoder (encode/windows/amf.rs), the AMD analogue of the direct-NVENC path, replacing the libavcodec *_amf dispatch. C-vtable FFI pinned to AMF headers v1.4.36, runtime-loaded from the driver's amfrt64.dll (no build feature, no new dependency) exactly as NVENC loads its DLL. - AVC/HEVC (SDR NV12 + 10-bit HDR P010) and AV1 (RDNA3+, probed); a bounded poll retires the libavcodec ~2-frame output hold; native in-place reset(). - Intra-refresh wave (PUNKTFUNK_INTRA_REFRESH), in-band HDR mastering/CLL metadata (*InHDRMetadata -> HEVC SEI / AV1 OBU), and a native codec probe feeding the GameStream advertisement (windows_backend_is_ffmpeg -> windows_backend_is_probed). - AMD dispatch / advertisement / 4:4:4 are native-only; the libavcodec AMF fallback and the PUNKTFUNK_AMF_FFMPEG hatch are removed. FFmpeg serves QSV only (its AMF path retained solely as the latency A/B comparator). - Overload back-pressure: submit bounds in-flight surfaces below the input ring, draining finished AUs (buffered for poll, FIFO-preserved) to free a slot and retry on AMF_INPUT_FULL instead of tearing the encoder down and forcing an IDR; this also closes a latent ring-overwrite corruption seen under load on-glass. Validated on the lab Ryzen iGPU (AMF runtime 1.4.37): HEVC/AVC across a native reset, HEVC Main10 mastering+CLL SEIs byte-verified, intra-refresh accepted, a backpressure burst FIFO-clean, and end-to-end via the macOS client. Measured §5.2 latency A/B: native encode_us p50 ~5 ms (0.31 frame periods) vs libavcodec ~17 ms (1.01). 4:4:4 stays unsupported (VCN hardware limit). Live-gated tests skip cleanly on non-AMD boxes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-06 17:33:23 +02:00