14 Commits

Author	SHA1	Message	Date
enricobuehler	7099266594	refactor(host): hoist the shared low-latency RC contract into encode/libav.rs ... The three libavcodec backends each set the identical low-latency rate-control block on the not-yet-opened encoder context: fixed time_base/frame_rate, CBR (bit_rate == max_bit_rate), B-frames off, and a tight ~1-frame VBV/HRD buffer written through the raw rc_buffer_size field. Move it once into apply_low_latency_rc(&mut video, fps, bitrate_bps), and let the long VBV rationale (why the tight buffer prevents high-motion bursts from overflowing the send queue) live in one place instead of only in the NVENC path. Each backend keeps the two genuinely per-backend calls around it: set_format (pixel format differs) before, and gop_size after (NVENC's infinite/intra- refresh wave vs the VAAPI/AMF i32::MAX). No behavior change — the field writes are independent, so the slightly different max_b_frames/rc_buffer_size ordering across backends is irrelevant. Folding the raw rc_buffer_size write into the helper also removes the NVENC path's separate unsafe block. Drops the now-unused ffmpeg::Rational import from all three. Linux check + clippy green (0/0, nvenc,vulkan-encode,pyrowave) on RTX 5070 Ti; ffmpeg_win.rs is Windows-cfg, pending .173 compile. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 13:23:28 +02:00
enricobuehler	fd8a062e2c	refactor(host): hoist the libav poll_encoder drain + PollOutcome into encode/libav.rs ... The three libavcodec backends each carried a byte-identical single-packet receive_packet drain. Move it once into the shared Tier-2 glue as poll_encoder -> PollOutcome (the richest form: Packet / Again / Eof), and have the callers narrow it: - Linux NVENC (encode/linux/mod.rs): poll() matches the shared fn, collapsing Again|Eof to Ok(None) — was an inlined match, now one call. - VAAPI (encode/linux/vaapi.rs): drop the local poll_encoder; the blocking budget loop lets Again|Eof fall through to the deadline check, byte-identical to the old Option::None path. - Windows AMF/QSV (encode/windows/ffmpeg_win.rs): drop the local PollOutcome + poll_encoder; its deadline-driven drain already matches PollOutcome, so only the import changes. No behavior change on any backend. Still a plain monomorphic free fn over a borrowed &mut Encoder — no new per-frame dyn/Box/alloc; the only allocation is the same bitstream to_vec() each path already made. Drops the now-unused ffmpeg::Packet import from all three. Linux check + clippy green (nvenc,vulkan-encode,pyrowave) on RTX 5070 Ti; ffmpeg_win.rs is Windows-cfg, pending .173 compile. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 13:16:45 +02:00
enricobuehler	3c38a5f0e8	refactor(host): hoist shared libav glue into encode/libav.rs (pixel_to_av + swscale consts) ... First step of the W2 libav de-dup (plan §2.2, the missing Tier-2 mid-layer). The three libavcodec backends (Linux NVENC, VAAPI, Windows AMF/QSV) each carried a byte-identical pixel_to_av plus the SWS_POINT / SWS_CS_ITU709 (/SWS_CS_BT2020) swscale consts. Hoist them into a new encode/libav.rs and import from super::libav. The module is gated to compile exactly when a libav backend does (linux, or windows+amf-qsv). Free fns/consts over borrowed handles — no per-frame dyn/alloc, off the zero-copy path. Verified: Linux cargo check green (linux/mod.rs + vaapi.rs compile against it); ffmpeg_win.rs is Windows-cfg — same mechanical swap, covered by Windows CI on push. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 13:04:52 +02:00
enricobuehler	abecb5226c	refactor(host): route the three libav backends' VBV re-parse through vbv_frames_env ... The libavcodec paths (Linux NVENC, VAAPI, Windows QSV) each re-parsed PUNKTFUNK_VBV_FRAMES locally in f32, duplicating and diverging in precision from the f64 vbv_frames_env() helper the direct-NVENC/AMF paths already use. Now that the helper lives in encode/codec.rs (532b313b), route all three through crate::encode::vbv_frames_env(): one parse, one precision, no drift. Behaviour-identical (same filter finite && > 0, same 1.0 default), f64 not f32. Verified: Linux cargo check green (linux/mod.rs + vaapi.rs compile); ffmpeg_win.rs is Windows-cfg and mirrors the amf.rs/nvenc.rs sites already using the helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-16 12:48:26 +02:00
enricobuehler	705a8baddf	feat(core,host,client): PyroWave datagram-aligned packets + partial-frame delivery (Phase 4, §4.4) ... PyroWave AUs now packetize on the negotiated shard payload, so a lost datagram costs a few wavelet blocks of localized blur rather than a whole frame — and the client can render an aged-out lossy frame instead of freezing until the next one. Host (opt-in, PyroWave only): - The encoder packetizes at the shard payload behind a 4-byte window prefix (used-len u16 + kind u16). Whole packets pack into WIN_PACKED windows; a packet too large for one shard (PyroWave 32x32 blocks are atomic and can exceed a shard) rides a WIN_FRAG_FIRST/CONT/LAST chain. `set_wire_chunking()` joins the Encoder trait (forwarded through TrackedEncoder — the silent-no-op trap); EncodedFrame.chunk_aligned marks the AU. - virtual_stream tags the AU with USER_FLAG_CHUNK_ALIGNED and re-applies chunking after every encoder (re)build, the adaptive-bitrate rebuild included. Core: - USER_FLAG_CHUNK_ALIGNED (0x40) wire bit. Reassembler opt-in (set_deliver_partial): a chunk-aligned frame that ages out with holes is handed over as Frame{complete:false} — received shards at their exact offsets, missing ranges zero-filled — instead of being dropped. Partials age out on a tight 30ms fuse (PARTIAL_WINDOW_NS) instead of the 120ms loss window: each frame is independently decodable, so an ancient partial has no value in a live stream. Newest-wins. A partial still counts as dropped for loss reporting. Client (PyroWave decode): - The session opts in when codec == PyroWave. The decoder walks the AU window-by-window, skipping zero (missing) windows and reassembling FRAG chains, then decodes whatever survived. A newest-decoded-index guard drops partials the pump has already moved past (no time-travel present). Also fixes a redundant-closure clippy nit in the PyroWave planar-present path. Validated on an RTX 5070 Ti under 2% netem loss with FEC pinned off: 60fps sustained entirely via partials, e2e 43ms p50 (146ms before the fuse) vs 23ms lossless, no keyframe-recovery chatter. Tests green: core 149, host 310 + the GPU-gated encoder smoke (framed-window walk + FRAG reassembly + upstream round-trip), client 26; clippy clean on the pyrowave feature combos. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 11:14:24 +02:00
enricobuehler	fdda7144ed	fix(encode): harden loss-recovery correctness across host encoders (F1–F7) ... Phases 1–4 of design/encoder-recovery-hardening.md — make the shipped RFI/ freeze-until-reanchor recovery honest and rebuild-safe across every backend. F1 — frame-index domain desync: the encode loop now owns a session-lifetime `au_seq`; `Encoder::submit_indexed(au_seq + inflight)` pins NVENC inputTimeStamp and AMF LTR slots to the WIRE frame index, so `invalidate_ref_frames` compares client frame numbers in the same domain and survives adaptive-bitrate rebuilds (an internal counter desynced on the first rebuild → RFI silently dead / an AMF force-ref onto a never-decoded frame). `FrameMsg.frame_index` → `Session::seal_frame_at`; GameStream gets the same via `VideoPacketizer:: packetize(.., Some(idx))`. F2 — Windows NVENC left the client frozen ~1s per loss: NVENC RFI was transparent (no anchor tag) while the session glue armed the 750ms IDR cooldown, so the freeze only lifted on the ~1s keyframe re-ask. NVENC now mirrors AMF — `pending_anchor` tags the first post-invalidate AU (the clean re-anchor P-frame) `recovery_anchor`, incl. the covering-range dedupe re-arm; the client lifts at ~RTT. F3 — speed-test probe filler burned video frame indexes: moved to its own index space (`Packetizer::alloc_probe_index` + `Session::submit_probe_frame`) with a second client reassembly window routed on FLAG_PROBE, gated on the new VIDEO_CAP_PROBE_SEQ Hello bit (mid-session probes declined for older clients). F4 — RFI range sanity cap: forward gaps wider than `packet::RFI_MAX_RANGE` (256) resync via keyframe instead of an out-of-range RFI, host- and client-side (client huge-gap → keyframe in `RfiRecovery::observe` + the pf-client-core pump). F5 — reset() parity: Windows NVENC (teardown + lazy re-init), Linux VAAPI (drop-inner), Linux NVENC (reopen from stored OpenArgs) now give the stall watchdog a heal lever instead of ending the session. F6 — sw.rs `pending: VecDeque` (was `Option`), killing the silent AU drop at capturer pipeline depth > 1. F7 — doc sweep on the RFI/anchor comments. Verified: punktfunk-core lib tests (macOS + Linux), full punktfunk-host suite on Linux (RTX 5070 Ti), Windows compile. Owed: the on-glass client matrix (F2 freeze A/B, AMF LTR spike across a bitrate rebuild). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:17:19 +02:00
enricobuehler	e55ff1bb28	feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI ... Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike. Client (pf-client-core): after a reference loss the hardware decoder conceals the missing-reference deltas (on RADV, a gray plate with new motion painted over) and returns Ok. The pump now freezes on the last good picture until a clean re-anchor instead of showing the concealment — lifting on a real IDR, an intra-refresh recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The frame_index gap is the early, precise loss signal and drives an RFI request. Host recovery signals (inert unless the backend supports them): - USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR). - USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame. AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the only clean-recovery lever is user LTR: mark frames as long-term references, and on loss force the next frame to re-reference the newest known-good one — a clean P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR before the loss; a range older than the live slots falls back to a keyframe. Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi(). Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success anchors the keyframe cooldown so the client's frames_dropped echo of the same loss is coalesced away rather than emitting a redundant IDR. Spike: synthetic NV12 GPU source for headless AMF encoder testing. Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests (incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux; punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90 forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:31:17 +02:00
enricobuehler	0dacb37088	feat(linux): zero-copy 4:4:4 — the EGL worker converts to planar YUV444 on the GPU ... A 4:4:4 session no longer falls to the CPU path (SHM capture + swscale RGB→YUV444P + re-upload — the fps-ceiling triple tax). The zero-copy worker grows a Yuv444Blit: three full-res R8 GL passes (the proven BT.709 coefficients; studio or full range per PUNKTFUNK_444_FULLRANGE, read by both processes so pixels and VUI flip together) into ONE stacked 3-plane pitched CUDA allocation — which keeps the worker↔host wire and IPC single-plane. The encoder copies the planes into ffmpeg's yuv444p CUDA surface and hevc_nvenc emits Range-Extensions 4:4:4 natively. ImportKind::Tiled444 is APPENDED to the worker protocol (a worker outliving a replaced host binary must keep the old tags stable; an old worker just errors the import, which the fail machinery already handles). A 4:4:4 session on a LINEAR/gamescope capture — no convert wired there — fails with a clear message instead of letting hevc_nvenc silently subsample. caps().chroma_444 now keys off the session (it missed the GPU path when keyed off the swscale's existence). Live-verified on the CachyOS VM (RTX 5070 Ti): per-frame "imported to CUDA yuv444=true", stream Rext/yuv444p/bt709 in both tv and pc range, no CPU-path warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 18:09:28 +02:00
enricobuehler	d55cde61d3	style: cargo fmt — settle the CSC/tvOS changes' layout (CI Format gate) ... Formatting only, no code change: the signaled-CSC and tvOS commits (1fcf9e11, 3ba19f28) left six files unformatted and the rust job's Format step rejects main. cargo fmt --all --check is clean after this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 17:11:27 +02:00
enricobuehler	1fcf9e11ec	fix(video): honor the signaled CSC matrix end-to-end + tvOS HDR presentation ... Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded 709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer renders with a constant hue error. Host-side signaling made honest across NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth. Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on both cores. Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now splits on the display's live EDR headroom - PQ passthrough when the per-session AVDisplayManager mode switch landed (a real HDR10 output tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer config. The 10-bit stream keeps its full decode depth either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehler	6da3e2fe6d	feat(host): intra-refresh loss recovery + delivery-anchored latency instrumentation ... Intra-refresh (opt-in PUNKTFUNK_INTRA_REFRESH=1 until on-glass validated): NVENC runs a moving intra band + recovery-point SEI (gop_size becomes the wave period, ffmpeg forces the real GOP infinite; default fps/2, PUNKTFUNK_IR_PERIOD_FRAMES overrides; ENOSYS latches a fallback to IDR-only). Clients request a keyframe on every FEC-unrecoverable frame, so under intra-refresh the session glue serves the first request instantly and suppresses the rest for a 2s window — the wave heals loss without the 20-40x IDR spike cascade. VAAPI/software keep IDR recovery. Instrumentation: the wire pts now anchors at the PipeWire delivery stamp (client-measured latency covers delivery + queue age, not just submit->glass; repeats/synthetic stamps fall back to now), encode_us keeps its submit->AU meaning via a separate inflight stamp, and a new 'queue' stage (delivery->submit age of fresh frames) rides PUNKTFUNK_PERF and the web-console stats samples. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:33:56 +00:00
enricobuehler	7612238a59	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	b54695739d	docs(host): prove every unsafe block in the Linux FFI files + gate them (unsafe-proof program 2/N) ... Continues the structural unsafe-proof program (every unsafe carries a documented proof of soundness; the file gains #![deny(clippy::undocumented_unsafe_blocks)] so it stays proven). This batch covers all 10 remaining pure-Linux files (104 blocks), each proof stating the REAL invariant — not boilerplate: zerocopy/cuda.rs (26) leaked process-lifetime libcuda fn-ptr table; opaque CUcontext never dereferenced; free-exactly-once via the Arc<Mutex<PoolInner>> ownership graph; dmabuf fd take/close split zerocopy/egl.rs (18) eglGetProcAddress'd procs with the GL context current; EGLImage liveness; the two-call modifier-query bounds zerocopy/vulkan.rs (4) copy-bounds arithmetic (src_size>=span); Send = thread confinement to the punktfunk-pipewire thread dmabuf_fence.rs (4) poll/ioctl/close fd liveness + ownership capture/linux/mod.rs (16) spa_data repr(transparent) cast; null-checked spa derefs; single-loop-thread buffer ownership until requeue inject/linux/gamepad.rs (10) uinput ioctl request-number ↔ struct-size match (static-asserted); InputEventRaw no-padding for the byte cast encode/linux/vaapi.rs (15) + encode/linux/mod.rs (9) ffmpeg object ownership/ free ladders; VAAPI/DRM graph; Send = single-thread transfer inject/linux/wlr.rs (2), vdisplay/linux/kwin.rs (1) No memory-unsafety SUSPECT blocks were found — the unsafe is sound. The vaapi agent did flag two real AVBufferRef *leaks* (not UB) in DmabufInner::open; marked inline with NOTE(leak) and addressed in a follow-up. Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings is clean (each file's deny gate hard-errors on any undocumented block). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:00:30 +00:00
enricobuehler	fced221684	refactor(windows-host): confine platform code under windows/ + linux/ folders (Goal-1 stage 6) ... Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the shared HID codecs into `inject/proto/`): capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/ audio/{windows,linux}/ vdisplay/{windows,linux}/ src/windows/ (service, wgc_helper, win_adapter, win_display) src/linux/ (dmabuf_fence, drm_sync, zerocopy/) Done with `#[path]`, NOT a module rename: every file moves into its folder while the `crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::` reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in behaviour with ZERO reference churn, far lower risk than collapsing to a single `crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so the path churn didn't fight them. Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3 remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all 36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file (the inline `mod X {` blocks are self-contained). Box build to follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 18:53:45 +00:00