12 Commits

Author	SHA1	Message	Date
enricobuehler	fdda7144ed	fix(encode): harden loss-recovery correctness across host encoders (F1–F7) ... Phases 1–4 of design/encoder-recovery-hardening.md — make the shipped RFI/ freeze-until-reanchor recovery honest and rebuild-safe across every backend. F1 — frame-index domain desync: the encode loop now owns a session-lifetime `au_seq`; `Encoder::submit_indexed(au_seq + inflight)` pins NVENC inputTimeStamp and AMF LTR slots to the WIRE frame index, so `invalidate_ref_frames` compares client frame numbers in the same domain and survives adaptive-bitrate rebuilds (an internal counter desynced on the first rebuild → RFI silently dead / an AMF force-ref onto a never-decoded frame). `FrameMsg.frame_index` → `Session::seal_frame_at`; GameStream gets the same via `VideoPacketizer:: packetize(.., Some(idx))`. F2 — Windows NVENC left the client frozen ~1s per loss: NVENC RFI was transparent (no anchor tag) while the session glue armed the 750ms IDR cooldown, so the freeze only lifted on the ~1s keyframe re-ask. NVENC now mirrors AMF — `pending_anchor` tags the first post-invalidate AU (the clean re-anchor P-frame) `recovery_anchor`, incl. the covering-range dedupe re-arm; the client lifts at ~RTT. F3 — speed-test probe filler burned video frame indexes: moved to its own index space (`Packetizer::alloc_probe_index` + `Session::submit_probe_frame`) with a second client reassembly window routed on FLAG_PROBE, gated on the new VIDEO_CAP_PROBE_SEQ Hello bit (mid-session probes declined for older clients). F4 — RFI range sanity cap: forward gaps wider than `packet::RFI_MAX_RANGE` (256) resync via keyframe instead of an out-of-range RFI, host- and client-side (client huge-gap → keyframe in `RfiRecovery::observe` + the pf-client-core pump). F5 — reset() parity: Windows NVENC (teardown + lazy re-init), Linux VAAPI (drop-inner), Linux NVENC (reopen from stored OpenArgs) now give the stall watchdog a heal lever instead of ending the session. F6 — sw.rs `pending: VecDeque` (was `Option`), killing the silent AU drop at capturer pipeline depth > 1. F7 — doc sweep on the RFI/anchor comments. Verified: punktfunk-core lib tests (macOS + Linux), full punktfunk-host suite on Linux (RTX 5070 Ti), Windows compile. Owed: the on-glass client matrix (F2 freeze A/B, AMF LTR spike across a bitrate rebuild). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:17:19 +02:00
enricobuehler	e55ff1bb28	feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI ... Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike. Client (pf-client-core): after a reference loss the hardware decoder conceals the missing-reference deltas (on RADV, a gray plate with new motion painted over) and returns Ok. The pump now freezes on the last good picture until a clean re-anchor instead of showing the concealment — lifting on a real IDR, an intra-refresh recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The frame_index gap is the early, precise loss signal and drives an RFI request. Host recovery signals (inert unless the backend supports them): - USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR). - USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame. AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the only clean-recovery lever is user LTR: mark frames as long-term references, and on loss force the next frame to re-reference the newest known-good one — a clean P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR before the loss; a range older than the live slots falls back to a keyframe. Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi(). Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success anchors the keyframe cooldown so the client's frames_dropped echo of the same loss is coalesced away rather than emitting a redundant IDR. Spike: synthetic NV12 GPU source for headless AMF encoder testing. Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests (incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux; punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90 forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:31:17 +02:00
enricobuehler	0dacb37088	feat(linux): zero-copy 4:4:4 — the EGL worker converts to planar YUV444 on the GPU ... A 4:4:4 session no longer falls to the CPU path (SHM capture + swscale RGB→YUV444P + re-upload — the fps-ceiling triple tax). The zero-copy worker grows a Yuv444Blit: three full-res R8 GL passes (the proven BT.709 coefficients; studio or full range per PUNKTFUNK_444_FULLRANGE, read by both processes so pixels and VUI flip together) into ONE stacked 3-plane pitched CUDA allocation — which keeps the worker↔host wire and IPC single-plane. The encoder copies the planes into ffmpeg's yuv444p CUDA surface and hevc_nvenc emits Range-Extensions 4:4:4 natively. ImportKind::Tiled444 is APPENDED to the worker protocol (a worker outliving a replaced host binary must keep the old tags stable; an old worker just errors the import, which the fail machinery already handles). A 4:4:4 session on a LINEAR/gamescope capture — no convert wired there — fails with a clear message instead of letting hevc_nvenc silently subsample. caps().chroma_444 now keys off the session (it missed the GPU path when keyed off the swscale's existence). Live-verified on the CachyOS VM (RTX 5070 Ti): per-frame "imported to CUDA yuv444=true", stream Rext/yuv444p/bt709 in both tv and pc range, no CPU-path warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 18:09:28 +02:00
enricobuehler	1fcf9e11ec	fix(video): honor the signaled CSC matrix end-to-end + tvOS HDR presentation ... Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded 709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer renders with a constant hue error. Host-side signaling made honest across NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth. Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on both cores. Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now splits on the display's live EDR headroom - PQ passthrough when the per-session AVDisplayManager mode switch landed (a real HDR10 output tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer config. The 10-bit stream keeps its full decode depth either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehler	d7e01ad92e	feat(host/vaapi): submit-split instrumentation + async_depth knob (depth 1 stays default) ... Chasing the 8ms submit at 1440p on the 780M: the sampled PUNKTFUNK_PERF split (push/pull/send) shows desc+buffersrc at ~5us, hwmap-import+VPP CSC at ~0.2-0.5ms, and avcodec_send_frame owning the rest — so neither a VA-surface import cache nor CSC overlap would help. Two facts landed: (1) async_depth>=2 in libavcodec's vaapi_encode is a structural +1-frame latency (frame N's packet only materializes when N+1 queues; measured 18ms vs 8.3ms p50 at depth 1) — depth 1 stays the default, PUNKTFUNK_VAAPI_ASYNC_DEPTH exists for pixel rates beyond the ASIC's serial budget, and poll() now does a bounded in-flight wait so a deeper depth still ships the AU as soon as the ASIC finishes. (2) The residual send_frame block tracks GPU CLOCKS, not the ASIC: ~8ms/frame at a 60fps duty cycle vs ~4.4ms at 120fps pacing vs 3.5ms back-to-back (270fps CLI benchmark, even at -async_depth 1) — the clock-sag fix lands in gpuclocks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:33:56 +00:00
enricobuehler	7ac71159ff	feat(host/vaapi): fall back to the low-power (VDEnc) entrypoint — unblocks modern Intel ... Gen12+/Arc iHD exposes ONLY EncSliceLP, so the default open fails with 'no usable encoding entrypoint'. Try full-feature first (AMD unchanged, validated on the 780M), retry low_power=1, cache the mode per codec; PUNKTFUNK_VAAPI_LOW_POWER pins it. Probes inherit the ladder. Docs note the Intel HuC firmware requirement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:33:56 +00:00
enricobuehler	dacc3b5209	feat(host,web): multi-GPU selection — GPU inventory + preference API, web-console GPU card ... - new crate::gpu (compiled on all platforms so the OpenAPI doc stays platform-independent): DXGI / sysfs GPU inventory with reboot-stable ids (PCI vendor:device + occurrence — LUIDs are per-boot), persisted auto/manual preference (<config>/gpu-settings.json, atomic temp+rename with in-memory rollback), one selection with precedence console preference > PUNKTFUNK_RENDER_ADAPTER > max VRAM and graceful fallback when the preferred GPU is absent, plus a live "in use" record (RAII session guard wrapped around every encoder open_video returns) - fix: windows_gpu_vendor derived the encoder backend from DXGI adapter 0 instead of the selected render adapter — on a hybrid box (e.g. Intel iGPU at index 0 + NVIDIA dGPU) the backend could disagree with the GPU the capture ring / IddCx render pin sit on. The NVENC 4:4:4 probe now also runs on the selected adapter (was: OS default), the codec/4:4:4 probe caches are keyed per selected GPU (were process-lifetime OnceLocks), and an explicit PUNKTFUNK_ENCODER conflicting with the selected GPU's vendor warns up front - mgmt API: GET /api/v1/gpus (inventory + mode + preferred + next-session selection with reason + in-use GPU/backend/session-count) and PUT /api/v1/gpus/preference (validates mode/gpu_id before writing); openapi.json regenerated; the vdisplay render pin now also engages for a console preference (not just the env pin) - web console: GPU card on the Host page — list with vendor + VRAM, Automatic / Prefer controls, Preferred / Next session / "In use · backend" badges, missing-preferred-GPU warning and env-pin note; en + de messages - Linux: a matched manual preference picks the VAAPI render node and the NVENC-vs-VAAPI auto choice; auto mode is exactly the previous behavior Validated live on the hybrid laptop (RTX 3500 Ada + Intel Arc Pro, which enumerates twice — the occurrence ids disambiguate): enumerate, prefer, bad-id 400, restart persistence, auto-restore keeping the stored pick. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 13:57:26 +02:00
enricobuehler	070658d611	fix(host): clear clippy doc_lazy_continuation in the 4:4:4 docs ... A line-wrap put `+`/`*`-style markers at the start of two doc lines, which clippy (Windows host job, rust 1.96) reads as markdown list items whose unindented follow-on lines trip `doc_lazy_continuation` under `-D warnings`: - encode/windows/nvenc.rs `chroma_444` field doc (the failing Windows-host clippy job): "+ chromaFormatIDC = 3" → "and chromaFormatIDC = 3". - encode/linux/vaapi.rs `probe_can_encode_444` doc: "+ validate" → "and validate" (last line, didn't fire yet, but fragile — fixed pre-emptively). Pure doc rewording, no behaviour change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:38:07 +00:00
enricobuehler	7612238a59	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	f70832b59d	fix(host/vaapi): plug two AVBufferRef leaks in DmabufInner::open ... Surfaced while writing the unsafe-soundness proofs (2/N): both are refcount leaks (sound — never dangling/double-free — so the SAFETY proofs held, but real bugs on the persistent punktfunk1-host listener that opens a fresh encoder per session). 1. Per-session leak: `par->hw_frames_ctx = av_buffer_ref(drm_frames)` created a second owned ref. `av_buffersrc_parameters_set` takes its OWN ref of `par->hw_frames_ctx`, and `av_free(par)` frees only the struct, not the ref — so the extra ref leaked every session, pinning the DRM frames ctx + device. Fix: assign `drm_frames` borrowed (the standard ffmpeg pattern); our single owned ref lives in DmabufInner and is unref'd in Drop. 2. Error-path leak: the final `open_vaapi_encoder(...)?` returned without the unref ladder every other error path runs, leaking graph/drm_frames/ vaapi_device/drm_device on encoder-open failure. Fix: match + clean up before returning (nv12_ctx is borrowed from the sink → freed by graph teardown). cargo clippy -p punktfunk-host --all-targets -- -D warnings clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:02:54 +00:00
enricobuehler	b54695739d	docs(host): prove every unsafe block in the Linux FFI files + gate them (unsafe-proof program 2/N) ... Continues the structural unsafe-proof program (every unsafe carries a documented proof of soundness; the file gains #![deny(clippy::undocumented_unsafe_blocks)] so it stays proven). This batch covers all 10 remaining pure-Linux files (104 blocks), each proof stating the REAL invariant — not boilerplate: zerocopy/cuda.rs (26) leaked process-lifetime libcuda fn-ptr table; opaque CUcontext never dereferenced; free-exactly-once via the Arc<Mutex<PoolInner>> ownership graph; dmabuf fd take/close split zerocopy/egl.rs (18) eglGetProcAddress'd procs with the GL context current; EGLImage liveness; the two-call modifier-query bounds zerocopy/vulkan.rs (4) copy-bounds arithmetic (src_size>=span); Send = thread confinement to the punktfunk-pipewire thread dmabuf_fence.rs (4) poll/ioctl/close fd liveness + ownership capture/linux/mod.rs (16) spa_data repr(transparent) cast; null-checked spa derefs; single-loop-thread buffer ownership until requeue inject/linux/gamepad.rs (10) uinput ioctl request-number ↔ struct-size match (static-asserted); InputEventRaw no-padding for the byte cast encode/linux/vaapi.rs (15) + encode/linux/mod.rs (9) ffmpeg object ownership/ free ladders; VAAPI/DRM graph; Send = single-thread transfer inject/linux/wlr.rs (2), vdisplay/linux/kwin.rs (1) No memory-unsafety SUSPECT blocks were found — the unsafe is sound. The vaapi agent did flag two real AVBufferRef *leaks* (not UB) in DmabufInner::open; marked inline with NOTE(leak) and addressed in a follow-up. Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings is clean (each file's deny gate hard-errors on any undocumented block). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:00:30 +00:00
enricobuehler	fced221684	refactor(windows-host): confine platform code under windows/ + linux/ folders (Goal-1 stage 6) ... Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the shared HID codecs into `inject/proto/`): capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/ audio/{windows,linux}/ vdisplay/{windows,linux}/ src/windows/ (service, wgc_helper, win_adapter, win_display) src/linux/ (dmabuf_fence, drm_sync, zerocopy/) Done with `#[path]`, NOT a module rename: every file moves into its folder while the `crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::` reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in behaviour with ZERO reference churn, far lower risk than collapsing to a single `crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so the path churn didn't fight them. Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3 remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all 36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file (the inline `mod X {` blocks are self-contained). Box build to follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 18:53:45 +00:00