33 Commits

Author	SHA1	Message	Date
enricobuehler	5249d31dfa	feat(host/linux): cursor-as-metadata — pointer in gamescope streams, no perf hit ... gamescope draws its pointer on a hardware DRM cursor plane that never enters the framebuffer feeding its PipeWire capture node, so captured frames arrive cursorless. Rather than force the producer's Embedded full-frame composite, request the pointer as PipeWire SPA_META_Cursor and composite it ourselves — a ≤256×256 blit into the encoder-OWNED surface, never the compositor's read-only dmabuf. Capture (capture/linux/mod.rs, capture.rs): - choose_cursor_mode() gates on available_cursor_modes(): Metadata > Embedded > Hidden (defaults Embedded on query error — never silently lose the cursor). Applied on both the plain and remote-desktop portal paths. - build_cursor_meta_param() adds a SPA_PARAM_Meta pod requesting SPA_META_Cursor (bitmap up to 256x256) to the connect params on every path. - CursorState parses spa_meta_cursor (id 0 = hidden; position - hotspot; bitmap re-read only when bitmap_offset != 0), normalizing RGBA/BGRA/ARGB/ABGR. Updated in .process before the corrupted/size-0 skip so cursor-only Mutter buffers still track movement. - CapturedFrame gains cursor: Option<CursorOverlay> (Arc rgba + serial) riding the GPU (Dmabuf/Cuda) payloads; the CPU de-pad path composites inline. GPU composite into each zero-copy backend's owned surface: - Vulkan Video + PyroWave: folded into the shared rgb2yuv.comp CSC shader — cursor sampled and alpha-mixed over RGB before the YUV convert (correct chroma, no extra pass). binding 3 (combined image sampler) + 16B push constant, per-slot cursor image uploaded only on serial change. spv regenerated. - CUDA/NVENC: real on-GPU kernel (cursor_blend.cu -> cursor_blend.ptx, compute_75 Turing baseline, JIT-forward) with blend_argb/blend_yuv444/ blend_nv12 (BT.709 limited, matching the shader). Loaded via the hand-rolled libcuda fn-table; blended into the ring InputSurface after copy, degrading to no-cursor on any failure — never drops a frame. VAAPI (AMD/Intel fallback) deferred: Vulkan Video already covers those GPUs; blind libva struct-layout FFI shouldn't ship unverified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 17:19:55 +02:00
enricobuehler	739a5f76bf	feat(apple): PyroWave Phase 5 — native Metal decode on Mac / Apple TV / iPad (§4.7) ... The Apple client now decodes PyroWave natively on the presenter's own MTLDevice — no MoltenVK, no upstream C++ in the app. Completes and wires up the decoder whose early working-tree snapshot rode along in 9127c346: - MetalWaveletShaders.swift: wavelet_dequant + idwt hand-ported from the vendored GLSL (STORAGE_MODE 0 only; subgroup scans → 32-wide simdgroups; DCShift spec constant → function constant; precision-1 split: fp16 levels 0-1 / fp32 2-4). - MetalWaveletDecoder.swift: Swift reimplementation of push_packet/decode_packet incl. the Phase-4 chunk-aligned window walk (FRAG chains, zeroed missing shards, the >half-blocks partial rule), init_block_meta's block-index space, and the 42-dequant + 13-idwt dispatch structure with encoder-boundary barriers. SOF-dims changes rebuild the size-dependent resources, which is also the mid-stream resize path. Ring of 4 output plane sets on the presenter's queue. - Presenter: pf_frag_planar (3xR8, the planar_csc.frag twin) + renderPlanar with a shared present tail; ReadyFrame carries an image enum (.video | .planar). - Stage2Pipeline: a dedicated PyroWave pump — no VideoToolbox machinery, no keyframe/re-anchor recovery (all-intra; partials render as localized blur by design), newest-frame-index staleness guard for late partials. - Opt-in: "PyroWave (wired LAN)" codec entry (probe-gated, ≈A13 floor via a real kernel-compile probe), selecting it advertises + prefers the codec and forces the session SDR (HDR/10-bit/4:4:4 caps dropped, plan contract). - Core ABI: punktfunk_connection_shard_payload() — the Welcome's negotiated shard payload, needed by native decoders to walk chunk-aligned AUs. - Validation: golden fixtures generated by the host encoder + upstream's own decoder (pyrowave_dump_golden, RTX 5070 Ti); the Metal decode PSNR-matches at 77-88 dB across all planes for dense AND chunk-aligned AUs, and a hole-punched partial still decodes. Parser unit tests cover the window walk, FRAG chains, broken chains, the half-blocks gate, and the block-index layout. Tests: apple 134 green (mac; iOS/tvOS build), host 312 w/ pyrowave on .21, core 148 w/ quic; clippy/fmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 12:16:43 +02:00
enricobuehler	9127c3465f	feat(client,host): PyroWave Apple Metal decoder + per-mode bitrate pin ... - clients/apple: native Metal wavelet decoder + compute shaders (Phase 5), decoding PyroWave without embedding MoltenVK. - pf-client-core: plumb user_flags/completeness through Decoder::decode_frame so the PyroWave backend parses chunk-aligned + partial AUs; gate the param's unused-warning to exactly the non-pyrowave builds (fixes -D warnings on the featureless Linux client build). - punktfunk-host: on a mid-stream mode switch, re-resolve the "Automatic" PyroWave bitrate for the new mode's ~1.6 bpp operating point (explicit rates and H.26x ABR stay put); reject sub-128px PyroWave modes before the encoder rebuild instead of after the ack. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 11:47:42 +02:00
enricobuehler	705a8baddf	feat(core,host,client): PyroWave datagram-aligned packets + partial-frame delivery (Phase 4, §4.4) ... PyroWave AUs now packetize on the negotiated shard payload, so a lost datagram costs a few wavelet blocks of localized blur rather than a whole frame — and the client can render an aged-out lossy frame instead of freezing until the next one. Host (opt-in, PyroWave only): - The encoder packetizes at the shard payload behind a 4-byte window prefix (used-len u16 + kind u16). Whole packets pack into WIN_PACKED windows; a packet too large for one shard (PyroWave 32x32 blocks are atomic and can exceed a shard) rides a WIN_FRAG_FIRST/CONT/LAST chain. `set_wire_chunking()` joins the Encoder trait (forwarded through TrackedEncoder — the silent-no-op trap); EncodedFrame.chunk_aligned marks the AU. - virtual_stream tags the AU with USER_FLAG_CHUNK_ALIGNED and re-applies chunking after every encoder (re)build, the adaptive-bitrate rebuild included. Core: - USER_FLAG_CHUNK_ALIGNED (0x40) wire bit. Reassembler opt-in (set_deliver_partial): a chunk-aligned frame that ages out with holes is handed over as Frame{complete:false} — received shards at their exact offsets, missing ranges zero-filled — instead of being dropped. Partials age out on a tight 30ms fuse (PARTIAL_WINDOW_NS) instead of the 120ms loss window: each frame is independently decodable, so an ancient partial has no value in a live stream. Newest-wins. A partial still counts as dropped for loss reporting. Client (PyroWave decode): - The session opts in when codec == PyroWave. The decoder walks the AU window-by-window, skipping zero (missing) windows and reassembling FRAG chains, then decodes whatever survived. A newest-decoded-index guard drops partials the pump has already moved past (no time-travel present). Also fixes a redundant-closure clippy nit in the PyroWave planar-present path. Validated on an RTX 5070 Ti under 2% netem loss with FEC pinned off: 60fps sustained entirely via partials, e2e 43ms p50 (146ms before the fuse) vs 23ms lossless, no keyframe-recovery chatter. Tests green: core 149, host 310 + the GPU-gated encoder smoke (framed-window walk + FRAG reassembly + upstream round-trip), client 26; clippy clean on the pyrowave feature combos. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 11:14:24 +02:00
enricobuehler	8dc5d672e2	feat(host): PyroWave capture advertises the Vulkan device's dmabuf modifiers ... The pyrowave passthrough rode VAAPI's LINEAR-only modifier policy, which starves it on Mutter+NVIDIA (tiled-only allocations → the compositor declines the offer → CPU capture fallback). The encoder imports through VK_EXT_image_drm_format_modifier, not libva, so the capture now extends the advertisement with every single-memory-plane modifier the PyroWave device samples from (probed via DrmFormatModifierPropertiesListEXT with the same device selection as the encoder). Live on .21 (Mutter+NVIDIA, RTX 5070 Ti): 7 modifiers advertised, the compositor negotiated block-linear (216172782120099861), no CPU downgrade, and the encoder's per-buffer import cache populated exactly as designed (8 PipeWire pool buffers imported once, silent reuse after). Zero-copy session numbers: static 60 fps, e2e 2.9-3.0 ms p50 (p95 3.4), host stage 1.6 ms; full-window motion 60 fps at ~80 Mb/s all-intra, decode ~1 ms. Also neutralizes the VAAPI-specific wording in the passthrough hand-off log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 09:30:59 +02:00
enricobuehler	719b1ef403	fix(core): let CODEC_PYROWAVE survive the Welcome decode whitelist ... Found by the first live session on .21: the host negotiated PyroWave and put codec=8 on the wire, but Welcome::decode's codec whitelist (H264/AV1, else HEVC — the corrupt-byte guard) folded it to HEVC, so the client opened an FFmpeg HEVC decoder against wavelet AUs. Roundtrip test now pins the pyrowave byte (and that a genuinely unknown future bit still folds to the HEVC default). With the fix the Phase-2 exit session runs END TO END on .21 (host + session client on one box, host capturing the GNOME virtual display, client presenting into a headless weston): negotiated codec=PyroWave (adv 0x0f) → PyroWave encoder (CPU-capture path — this box's Mutter+NVIDIA rejects the LINEAR-dmabuf offer) → wire → PyroWave decoder on the presenter's device → planar CSC. Static desktop: stable 60 fps, e2e 2.1-4.1 ms p50 (p95 <= 6 ms), decode 0.2-0.6 ms, vs HEVC/NVENC-direct baseline 2.1 ms — parity at idle. Full-window motion: 60 fps at ~80 Mb/s all-intra (HEVC ~7), decode still sub-ms, zero decode errors or keyframe-request chatter across every run. Deeper motion/loss characterization needs a dmabuf-accepting host box (this one is capped by the CPU capture path). Also retires the stale "no shipping client decodes this" wording in the host encoder/dispatch logs — the negotiation exists now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 09:09:39 +02:00
enricobuehler	f77eec1299	feat(client): PyroWave planar present path + Linux NVENC match-arm fix (Phase 2b, part 2) ... The arch package job (--features nvenc) tripped the same class of Codec::PyroWave non-exhaustive matches as windows-host had, in nvenc_cuda.rs (6 sites) — dispatch-guarded unreachable!() arms, plus the vk_util-extraction leftover unused imports in vulkan_video.rs. All Linux host feature combos (none / pyrowave / nvenc,vulkan-encode / all three) now compile clean on .21. Presenter: planar_csc.frag (+ committed .spv) — the 3-plane variant of nv12_csc.frag (separate Cb/Cr R8 planes, same push-constant CSC-row contract, siting correction self-disables at full-res chroma). CscPass grows a shared builder + new_planar()/bind_planes_planar() (GENERAL-layout descriptors — pyrowave planes stay GENERAL); the Vk presenter builds the planar pass when the device passed the pyrowave probe, FrameInput::PyroWave rides present_frame (no acquire barrier needed: the decoder fence-completed and barriered the planes on the same queue), and run.rs presents it with no demote rung (only device loss ends the session). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 01:42:15 +02:00
enricobuehler	e71cb9b7bd	feat(core,host): CODEC_PYROWAVE negotiation — opt-in only, host dispatch wired ... Phase 2a+2c of design/pyrowave-codec-plan.md. Core: CODEC_PYROWAVE = 0x08 on Hello::video_codecs/Welcome::codec. Deliberately absent from resolve_codec's precedence ladder (plan §3 — a 100-400 Mbps codec must never win a negotiation by mere mutual support): reachable exclusively through the client's explicit preferred_codec. Invariant tests cover never-auto-selected (even as the only shared codec), preferred-path selection, and graceful fallback. ABI mirror PUNKTFUNK_CODEC_PYROWAVE + lockstep assert for the Apple/Android embedders. Host: Codec::PyroWave variant threaded through the wire mappings; a negotiated PyroWave session routes straight to the backend ahead of the PUNKTFUNK_ENCODER pref dispatch (which stays a lab override). The advertisement bit rides host_wire_caps only when the capture side would actually deliver ingestible frames — linux_zero_copy_is_vaapi(), i.e. AMD/Intel auto or an explicit operator pref on NVIDIA; per-session raw-dmabuf OutputFormat plumbing is recorded as the Phase-3 item. The libavcodec name helpers are dispatch-guarded unreachable; the web console gains ApiCodec::PyroWave (api/openapi.json regenerated). Validated on .21: 308 host tests green with and without the feature, 145 core tests green with quic, clippy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 01:14:04 +02:00
enricobuehler	767f028bdf	feat(host): PyroWave encoder — Phase 1 of the LAN low-latency codec plan ... PyroWaveEncoder behind --features pyrowave + an explicit PUNKTFUNK_ENCODER=pyrowave (loud EXPERIMENTAL warning: no client can decode the stream until CODEC_PYROWAVE negotiation lands, plan Phase 2). Design (plan §4.3): a private ash Vulkan-1.3 device shared with pyrowave via pyrowave_create_device — DeviceHold pins the instance/device create-infos the 0.4.0 API requires alive for the device's lifetime. Capture dmabufs pass straight through on ANY vendor (linux_zero_copy_is_vaapi → true for pyrowave; NVIDIA dmabuf→Vulkan import validated by upstream's interop test on .21) with the same per-buffer import cache as the Vulkan Video backend; the shared rgb2yuv.comp BT.709-limited CSC writes R8+RG8 images pyrowave samples directly (R/G view swizzles synthesize Cb/Cr — no NV12 copy). Encode records into OUR command buffer (pyrowave_device_set_command_buffer), so ingest + CSC + encode are one submission with a sub-ms fence wait; the AU is exactly one pyrowave packet, keyframe=true on every frame. reconfigure_bitrate is a free in-place budget change (Phase 3 pins the session rate); reset() recreates only the pyrowave encoder object. Shared ash leaf helpers (dmabuf import, image/memory utils) extracted from vulkan_video.rs into encode/linux/vk_util.rs — vulkan-encode builds unchanged. Validated on .21 (RTX 5070 Ti): pyrowave_smoke green — encodes CPU fills through the full open→CSC→GPU-encode→packetize path, decodes every AU with upstream's own decoder, checks BT.709 plane means ±3; rate retarget + rebuild covered. clippy clean, 308 host tests green with the feature on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 00:58:02 +02:00
enricobuehler	a1af916e38	feat(host): in-place encoder rate reconfigure — ABR steps no longer cost an IDR ... Every adaptive-bitrate step used to tear the encoder down and rebuild it, opening on a full IDR (a 20-40x frame-size spike, in-flight AU forfeit and an IDR-cooldown anchor) — exactly when the Automatic controller is climbing. Encoder::reconfigure_bitrate(bps) retargets the LIVE encoder instead (default false, so libavcodec/software paths keep the rebuild fallback, which also still owns the bitrate clamping): - Linux + Windows direct NVENC: nvEncReconfigureEncoder (added to the hand-rolled runtime EncodeApi tables) with resetEncoder=0 / forceIDR=0; the same init/config is re-authored via the new shared build_config/ build_init_params with only avg/max bitrate + VBV (PUNKTFUNK_VBV_FRAMES) moved. On-hardware test: 20→60→10 Mbps in place, zero IDRs (RTX 5070 Ti). - Native AMF: TargetBitrate/PeakBitrate/VBVBufferSize are dynamic properties — SetProperty on the live component, no Terminate/re-Init. - Vulkan Video (HEVC + AV1): stage the rate and emit an ENCODE_RATE_CONTROL control command on the next recorded frame (begin keeps declaring the session's current state, as the spec requires). The session glue tries the in-place retarget first and skips the rebuild/ inflight-clear/IDR-cooldown bookkeeping when it succeeds — the reference chain and the wire-index prediction survive, so RFI keeps working across rate steps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 19:53:18 +02:00
enricobuehler	9b7fc127ef	feat(core): Automatic bitrate scales to measured link capacity — probe ceiling + slow start ... The ABR ceiling was the negotiated start rate, so an 'Automatic' session was permanently boxed at the 20 Mbps default no matter the link — the most user-visible cap left after the transport work lifted the client receive ceiling to ~4.8 Gbps wire. - Startup link-capacity probe: ~2 s into an Automatic session the pump fires one speed-test burst (2 Gbps target, 800 ms) over the existing ProbeRequest machinery; delivered wire throughput x0.7 (FEC + variance headroom) becomes the controller's climb ceiling via set_ceiling(). Old hosts decline (all-zero reply) or never answer (a 6 s timeout clears the stuck probe state so LossReports resume) — the ceiling then stays negotiated, exactly the old behavior. PUNKTFUNK_ABR_PROBE=0 opts out. - Slow start: until the first congestion signal, every cooled clean window DOUBLES the rate toward the ceiling (20 Mbps -> 640 Mbps in ~10 s) instead of +6% per ~10 s (which would have taken ~10 minutes). Any congestion signal ends it for good; classic AIMD takes over. - Faster, severity-aware AIMD: a SEVERE window (unrecoverable frame, jump-to-live flush, or >=6% loss) backs off x0.7 immediately instead of waiting two windows; ordinary congestion (2-6% loss, OWD rise) keeps the two-window fuse. Additive climbs need 6 clean windows (~4.5 s, was ~10 s); the change cooldown drops 3 s -> 1.5 s. - PUNKTFUNK_VBV_FRAMES now also scales the direct-NVENC VBV (Windows + Linux, previously hardwired to 1 frame) — parity with AMF/VAAPI/QSV. Each accepted step still costs an encoder rebuild + IDR on the host; in-place rate reconfigure (NvEncReconfigureEncoder / AMF dynamic properties / Vulkan per-frame RC) is the planned follow-up that makes stepping free. Controller tests rewritten to the new policy (severity classes, slow-start climb, ceiling semantics; 144 green). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 19:28:11 +02:00
enricobuehler	05868ef634	fix(encode): Vulkan-HEVC full-RPS reference retention + AV1 feature gate (RFI review) ... 2026-07-12 review of the host encoders / client decoders / RFI plane. NVENC (both), AMF-LTR, the session glue, and the client RfiTracker came out clean; every fix lands in the Vulkan Video backend + dispatch: 1. HEVC: author each P-frame's short-term RPS to retain ALL resident DPB pictures (minus the setup slot), not just its one reference. HEVC 8.3.2 evicts unlisted pictures, and clients keep FEEDING the decoder while frozen — so with the old single-pic RPS, a conforming parser (FFmpeg = the Linux VAAPI/Vulkan and Windows D3D11VA clients) had already discarded the picture an RFI recovery anchor references whenever a fed post-loss frame preceded it: generate_missing_ref, and the "clean" anchor plus everything chained after it decodes as garbage. Pure builder (`build_h265_rps_s0`) + unit tests; AV1 needs nothing (slot-based retention). The smoke test now encodes a fed post-loss frame between loss@4 and anchor@6 so an ffmpeg decode of the dropped dump exercises exactly this (expect ONE POC-4 complaint, never POC 3) — revalidate on the AMD box; this NVIDIA dev box fails the backend earlier at HEVC header retrieval (pre-existing). 2. AV1: chain PhysicalDeviceVideoEncodeAV1FeaturesKHR (videoEncodeAV1 = TRUE, stype 1000513004) into device creation — spec-required for the ENCODE_AV1 codec op; RADV tolerated the omission, validation layers and stricter drivers do not. 3. RFI decline no longer self-arms force_kf — that bypassed the session glue's 750 ms IDR cooldown, turning a storm of hopeless RFI requests into one full IDR each. Decline like NVENC/AMF and let the caller's coalesced keyframe path own the fallback; add the missing first>last guard for parity. 4. open_video_backend now returns the label of the branch that ACTUALLY opened, so the mgmt API / web console reports "vulkan" instead of "vaapi" for the default-on Vulkan sessions (the old dispatch-mirror resolved_backend_label went stale when the backend gained its VAAPI fallback; deleted). Structure: the ~230-line inline HEVC coding block moves to record_coding_h265 (symmetric with record_coding_av1) and the duplicated pre-encode barriers dedupe into begin_encode_cmd. Follow-up plan (separate, punktfunk-planning): bring the post-loss freeze + RECOVERY_ANCHOR/POINT lift to the Android/Windows/Apple clients via a shared ReanchorGate (design/client-reanchor-freeze-parity.md). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-12 23:12:12 +02:00
enricobuehler	2d37835545	feat(encode): AV1 on the Linux Vulkan Video encoder (real RFI) ... Extend the raw Vulkan Video backend to AV1 (`VK_KHR_video_encode_av1`) alongside HEVC, so AMD/Intel Linux hosts get the same clean-P-frame loss recovery for AV1 that HEVC already has — no full IDR on packet loss. ash 0.38.0+1.3.281 predates the AV1-encode extension (finalized in Vulkan 1.3.290) and bumping ash breaks the SDL/Vulkan client (it drops the lifetime on AllocationCallbacks, which sdl3-sys still generates). So the AV1-encode structs/flags/enums are vendored host-only in `encode/linux/vk_av1_encode.rs`, copied verbatim from ash-master's generated code and chained into ash's generic video-encode calls via raw p_next — the common StdVideoAV1* types (from AV1 decode) are reused from ash 1.3.281. `vulkan_video.rs` gains a parallel AV1 path: AV1 Main profile/caps/session (+ max-level session-create), a bit-packed sequence-header OBU + per-frame temporal-delimiter framing (Vulkan AV1 encode, unlike H26x, emits only the frame OBU), and per-frame StdVideoEncodeAV1PictureInfo with the RFI reference model — a normal P inherits CDF context from its reference for compression, while an IDR or recovery anchor sets primary_ref_frame=NONE + error_resilient_mode so it decodes independent of the (possibly lost) frames since its reference. HEVC recording is unchanged; the shared CSC / ring / DPB-barrier pipeline is reused as-is. Codec routing in `open_video_backend` extends the HEVC arm to HEVC|AV1. The seq header enables only order-hint (+128px superblocks per caps), matching FFmpeg's proven Vulkan AV1 config — enabling CDEF/restoration made VCN emit frame-header sections our seq header didn't match, desyncing every inter frame. Headless-validated on real RADV (780M): open + 6-frame encode (I P P P P P) decodes 0-error on both dav1d and ffmpeg/cbs; the RFI recovery anchor at frame 4 is a clean P (not IDR), and dropping the "lost" frame 3 still decodes clean (re-anchored to frame 2). HEVC smoke unchanged (no regression). `cargo clippy --features vulkan-encode -- -D warnings` and the no-feature build both green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-12 22:06:24 +02:00
enricobuehler	84329205eb	feat(encode): default-on the Linux Vulkan Video HEVC backend ... On-glass validated 2026-07-12 on an AMD RADV 780M with a real Deck-class client: the pipelined raw-Vulkan HEVC encoder ran a rock-solid 1080p@240 session and healed loss with clean P-frame recovery anchors (real RFI the libav VAAPI path can't express). Ship it on by default, mirroring the NVENC default-on. - vulkan_encode_enabled() defaults ON; PUNKTFUNK_VULKAN_ENCODE=0 is the libav VAAPI escape hatch. A failed open still falls back to VAAPI, so a device without h265 Vulkan encode (or an untested Intel/ANV that misbehaves at open) degrades gracefully instead of breaking the stream. - Ring depth defaults to 2 (one frame of overlap, lowest added latency — the on-glass-validated real-time setting); PUNKTFUNK_VULKAN_INFLIGHT still tunes it. - Compile --features punktfunk-host/vulkan-encode into the arch/deb/rpm host builds (pure-Rust ash, no new lib / no link-time dep), alongside nvenc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-12 20:20:31 +02:00
enricobuehler	e9d8f2bc04	perf(encode): pipeline the Vulkan Video encoder (frames in flight) ... The Linux raw-Vulkan HEVC backend blocked on two GPU fences per frame, so CPU readback and the next capture could not overlap the GPU encode. Refactor into a small ring of per-in-flight-frame resources (own command buffers, CSC descriptor set + Y/UV/NV12 scratch, bitstream buffer, feedback query and sync objects) so submit() records into a free slot and returns without waiting, and poll() reads back the oldest slot once its fence signals. The pump's non-blocking poll then overlaps a frame's CSC+encode with the next capture — the throughput win — with no capturer/pump change (VAAPI untouched). - New `Frame` struct + `make_frame`; encoder holds `frames`/`ring`/`in_flight`. - `record_submit` (non-blocking) + `read_slot` (fence-gated readback) replace the synchronous `encode_frame`; `enqueue` applies backpressure by draining the oldest slot when the ring is full. - DPB self-barrier between consecutive encodes: orders frame N's reconstruct write before N+1's reference read now that they can be in flight together. - flush() drains all in-flight slots in order; reset() waits idle + discards. - Ring depth defaults to 3, overridable via PUNKTFUNK_VULKAN_INFLIGHT (2..=6). - Smoke test drains via poll-loop + flush (async breaks one-AU-per-submit). Headless-validated on real RADV 780M: cargo check (feature on/off) + clippy -D warnings + rustfmt clean; smoke test at ring depth 2/3/6 all ffmpeg-decode clean (I P P P P P) and drop-heal (I P P P P) with 0 errors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-12 19:51:21 +02:00
enricobuehler	bbbb7f5723	fix+perf(encode): clamp Vulkan CSC to source edge + cache dmabuf imports ... Two refinements after the initial on-glass validation on RADV (780M): - Green padding bar at non-16-aligned heights (e.g. 1080 → coded 1088): the CSC compute shader read past the edge of the shorter source dmabuf for the 8 alignment-padding rows, producing undefined/green garbage that showed on a client rendering the coded frame. Clamp every source fetch to `textureSize-1` so padding rows duplicate the last real row (invisible, and the SPS conformance window still crops it for a compliant decoder). BT.709 conversion is byte-identical for in-bounds pixels. 5120x1440 (exactly aligned) was never affected. - Per-frame dmabuf import churn: the backend created + imported + destroyed a VkImage every frame (allocation jitter → stutter). PipeWire cycles a small fixed pool, so import each underlying buffer ONCE (keyed by st_dev/st_ino — each frame's fd is a fresh dup of the same buffer) and reuse it, matching the CUDA-path VkBridge. First import acquires from the foreign producer; cached re-reads keep queue ownership and use a plain visibility barrier. On-glass: ~3-6 imports per session then silent (was ~one per frame at 240 Hz), stutter gone at resolutions with headroom. Also adds a PF_SMOKE_W/H override to the headless smoke test to exercise the conformance-window crop path (ffprobe confirms coded 1088 → displayed 1080). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-12 19:24:12 +02:00
enricobuehler	5ab6daa694	feat(encode): raw Vulkan Video HEVC backend on Linux (AMD/Intel) with real RFI ... Add `VulkanVideoEncoder` (`VK_KHR_video_encode_h265` via ash) — the open-stack twin of the direct-NVENC RFI path, giving AMD/Intel Linux hosts real reference-frame invalidation loss recovery: a clean P-frame recovery anchor that re-references a known-good older frame instead of a full IDR. The app owns the DPB, so recovery = pointing the P-frame's single L0 reference at a resident slot strictly older than the loss (never a concealed frame). The backend owns its own ash instance/device with encode + compute queues, authors VPS/SPS/PPS (Main, conformance-window crop for non-16-aligned heights like 1080->1088), runs a DPB-ring reference-slot state machine with monotonic POC and CBR rate control, and does an on-GPU RGB->NV12 BT.709 compute CSC (embedded rgb2yuv.spv) since capture delivers packed-RGB dmabufs — importing each frame's dmabuf (explicit DRM modifier) or uploading a CPU-RGB fallback, CSC on the compute queue, then encode on the encode queue, ordered by a semaphore. Wired into `open_video_backend`: an AMD/Intel HEVC session opens this instead of libav VAAPI when `PUNKTFUNK_VULKAN_ENCODE=1` (VAAPI fallback on any open error, so it can only improve recovery, never break a stream); `PUNKTFUNK_ ENCODER=vulkan` forces it. Gated behind the new `vulkan-encode` Cargo feature, which pulls no new dependency (reuses the `ash` bindings already carried for the dmabuf zero-copy bridge). Opt-in until on-glass validated, mirroring how the direct-NVENC path shipped. Headless-validated on real RADV (RDNA3 780M, Mesa 26): open + multi-frame encode + `invalidate_ref_frames` all run through the real struct and ffmpeg decodes the output `I P P P P P` with 0 errors; the recovery frame is a clean P-frame (not an IDR); and dropping the "lost" AU still decodes cleanly because the recovery re-anchored to an older frame — the RFI heal, proven on real hardware. `cargo check`/`clippy -D warnings` green with the feature on and off. Design: design/linux-vulkan-video-encode.md. Harness: design/vkenc-probe-harness/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-12 17:08:15 +02:00
enricobuehler	ad532b08a0	style(encode): rustfmt the direct-SDK NVENC Linux backend ... `cargo fmt` was not run on the Phase 5.2 additions (nvenc_cuda.rs + the encode.rs dispatcher fork), failing the ci.yml rust fmt job. Whitespace/wrapping only — no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:35:07 +02:00
enricobuehler	93093f3cf9	feat(encode): direct-SDK NVENC on Linux (CUDA input) with real RFI ... Phase 5.2 of design/encoder-recovery-hardening.md (design/linux-direct-nvenc.md). The Linux NVIDIA host encodes through libavcodec `hevc_nvenc`, which structurally cannot express `nvEncInvalidateRefFrames` — so every FEC-unrecoverable loss is a full IDR and, since the client freezes-until-reanchor, a per-loss freeze for RTT+IDR-encode. This ports the Windows raw-NVENC backend to NV_ENC_DEVICE_TYPE_CUDA over the shared CUcontext so Linux NVIDIA gets the same real RFI + F2 recovery-anchor + reset() stall lever + HDR-SEI/Main10 plumbing. New `encode/linux/nvenc_cuda.rs` (`NvencCudaEncoder`): - runtime-loaded entry table via `dlopen libnvidia-encode.so.1` (never link-time, mirroring the zerocopy::cuda libcuda loader) — one binary still starts on AMD/Intel Linux boxes and falls through to VAAPI/software; - session on the shared CUcontext (zerocopy::cuda::context()); - an encoder-owned ring of registered CUDADEVICEPTR input surfaces (zerocopy::cuda::InputSurface + a contiguous-NV12 allocator), each captured DeviceBuffer device→device copied in via the existing copy_* helpers — mirrors the libav recycled-hwframe-pool copy, so zero regression vs today; - config/RFI/anchor/reset ported from the Windows backend; sync-only (NVENC async is Windows-only, so that whole subsystem is dropped); - Main10/HDR-SEI wired but inert until a Linux P010 capture path (Phase 5.1). Wired behind PUNKTFUNK_NVENC_DIRECT (default OFF) in open_nvenc_probed; the Windows path is untouched (no shared extraction in v1). Two on-hardware `#[ignore]` smokes added. Validated on .21 (RTX 5070 Ti, driver 610.43.03): builds on Linux under ci-check, clippy-clean, full host suite 272/0, NV12 smoke (8 AUs, real invalidate_ref_frames + recovery_anchor on a P-frame) and YUV444 FREXT smoke (6 AUs, chroma_444) green; Windows compile unaffected. Owed: the client-in-the-loop matrix (RFI-survives-ABR, reset() heal, A/B vs libav) and the default flip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:18:21 +02:00
enricobuehler	fdda7144ed	fix(encode): harden loss-recovery correctness across host encoders (F1–F7) ... Phases 1–4 of design/encoder-recovery-hardening.md — make the shipped RFI/ freeze-until-reanchor recovery honest and rebuild-safe across every backend. F1 — frame-index domain desync: the encode loop now owns a session-lifetime `au_seq`; `Encoder::submit_indexed(au_seq + inflight)` pins NVENC inputTimeStamp and AMF LTR slots to the WIRE frame index, so `invalidate_ref_frames` compares client frame numbers in the same domain and survives adaptive-bitrate rebuilds (an internal counter desynced on the first rebuild → RFI silently dead / an AMF force-ref onto a never-decoded frame). `FrameMsg.frame_index` → `Session::seal_frame_at`; GameStream gets the same via `VideoPacketizer:: packetize(.., Some(idx))`. F2 — Windows NVENC left the client frozen ~1s per loss: NVENC RFI was transparent (no anchor tag) while the session glue armed the 750ms IDR cooldown, so the freeze only lifted on the ~1s keyframe re-ask. NVENC now mirrors AMF — `pending_anchor` tags the first post-invalidate AU (the clean re-anchor P-frame) `recovery_anchor`, incl. the covering-range dedupe re-arm; the client lifts at ~RTT. F3 — speed-test probe filler burned video frame indexes: moved to its own index space (`Packetizer::alloc_probe_index` + `Session::submit_probe_frame`) with a second client reassembly window routed on FLAG_PROBE, gated on the new VIDEO_CAP_PROBE_SEQ Hello bit (mid-session probes declined for older clients). F4 — RFI range sanity cap: forward gaps wider than `packet::RFI_MAX_RANGE` (256) resync via keyframe instead of an out-of-range RFI, host- and client-side (client huge-gap → keyframe in `RfiRecovery::observe` + the pf-client-core pump). F5 — reset() parity: Windows NVENC (teardown + lazy re-init), Linux VAAPI (drop-inner), Linux NVENC (reopen from stored OpenArgs) now give the stall watchdog a heal lever instead of ending the session. F6 — sw.rs `pending: VecDeque` (was `Option`), killing the silent AU drop at capturer pipeline depth > 1. F7 — doc sweep on the RFI/anchor comments. Verified: punktfunk-core lib tests (macOS + Linux), full punktfunk-host suite on Linux (RTX 5070 Ti), Windows compile. Owed: the on-glass client matrix (F2 freeze A/B, AMF LTR spike across a bitrate rebuild). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 11:17:19 +02:00
enricobuehler	e55ff1bb28	feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI ... Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike. Client (pf-client-core): after a reference loss the hardware decoder conceals the missing-reference deltas (on RADV, a gray plate with new motion painted over) and returns Ok. The pump now freezes on the last good picture until a clean re-anchor instead of showing the concealment — lifting on a real IDR, an intra-refresh recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The frame_index gap is the early, precise loss signal and drives an RFI request. Host recovery signals (inert unless the backend supports them): - USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR). - USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame. AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the only clean-recovery lever is user LTR: mark frames as long-term references, and on loss force the next frame to re-reference the newest known-good one — a clean P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR before the loss; a range older than the live slots falls back to a keyframe. Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi(). Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success anchors the keyframe cooldown so the client's frames_dropped echo of the same loss is coalesced away rather than emitting a redundant IDR. Spike: synthetic NV12 GPU source for headless AMF encoder testing. Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests (incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux; punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90 forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:31:17 +02:00
enricobuehler	0dacb37088	feat(linux): zero-copy 4:4:4 — the EGL worker converts to planar YUV444 on the GPU ... A 4:4:4 session no longer falls to the CPU path (SHM capture + swscale RGB→YUV444P + re-upload — the fps-ceiling triple tax). The zero-copy worker grows a Yuv444Blit: three full-res R8 GL passes (the proven BT.709 coefficients; studio or full range per PUNKTFUNK_444_FULLRANGE, read by both processes so pixels and VUI flip together) into ONE stacked 3-plane pitched CUDA allocation — which keeps the worker↔host wire and IPC single-plane. The encoder copies the planes into ffmpeg's yuv444p CUDA surface and hevc_nvenc emits Range-Extensions 4:4:4 natively. ImportKind::Tiled444 is APPENDED to the worker protocol (a worker outliving a replaced host binary must keep the old tags stable; an old worker just errors the import, which the fail machinery already handles). A 4:4:4 session on a LINEAR/gamescope capture — no convert wired there — fails with a clear message instead of letting hevc_nvenc silently subsample. caps().chroma_444 now keys off the session (it missed the GPU path when keyed off the swscale's existence). Live-verified on the CachyOS VM (RTX 5070 Ti): per-frame "imported to CUDA yuv444=true", stream Rext/yuv444p/bt709 in both tv and pc range, no CPU-path warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 18:09:28 +02:00
enricobuehler	d55cde61d3	style: cargo fmt — settle the CSC/tvOS changes' layout (CI Format gate) ... Formatting only, no code change: the signaled-CSC and tvOS commits (1fcf9e11, 3ba19f28) left six files unformatted and the rust job's Format step rejects main. cargo fmt --all --check is clean after this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 17:11:27 +02:00
enricobuehler	1fcf9e11ec	fix(video): honor the signaled CSC matrix end-to-end + tvOS HDR presentation ... Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded 709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer renders with a constant hue error. Host-side signaling made honest across NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth. Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on both cores. Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now splits on the display's live EDR headroom - PQ passthrough when the per-session AVDisplayManager mode switch landed (a real HDR10 output tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer config. The 10-bit stream keeps its full decode depth either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehler	d7e01ad92e	feat(host/vaapi): submit-split instrumentation + async_depth knob (depth 1 stays default) ... Chasing the 8ms submit at 1440p on the 780M: the sampled PUNKTFUNK_PERF split (push/pull/send) shows desc+buffersrc at ~5us, hwmap-import+VPP CSC at ~0.2-0.5ms, and avcodec_send_frame owning the rest — so neither a VA-surface import cache nor CSC overlap would help. Two facts landed: (1) async_depth>=2 in libavcodec's vaapi_encode is a structural +1-frame latency (frame N's packet only materializes when N+1 queues; measured 18ms vs 8.3ms p50 at depth 1) — depth 1 stays the default, PUNKTFUNK_VAAPI_ASYNC_DEPTH exists for pixel rates beyond the ASIC's serial budget, and poll() now does a bounded in-flight wait so a deeper depth still ships the AU as soon as the ASIC finishes. (2) The residual send_frame block tracks GPU CLOCKS, not the ASIC: ~8ms/frame at a 60fps duty cycle vs ~4.4ms at 120fps pacing vs 3.5ms back-to-back (270fps CLI benchmark, even at -async_depth 1) — the clock-sag fix lands in gpuclocks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:33:56 +00:00
enricobuehler	6da3e2fe6d	feat(host): intra-refresh loss recovery + delivery-anchored latency instrumentation ... Intra-refresh (opt-in PUNKTFUNK_INTRA_REFRESH=1 until on-glass validated): NVENC runs a moving intra band + recovery-point SEI (gop_size becomes the wave period, ffmpeg forces the real GOP infinite; default fps/2, PUNKTFUNK_IR_PERIOD_FRAMES overrides; ENOSYS latches a fallback to IDR-only). Clients request a keyframe on every FEC-unrecoverable frame, so under intra-refresh the session glue serves the first request instantly and suppresses the rest for a 2s window — the wave heals loss without the 20-40x IDR spike cascade. VAAPI/software keep IDR recovery. Instrumentation: the wire pts now anchors at the PipeWire delivery stamp (client-measured latency covers delivery + queue age, not just submit->glass; repeats/synthetic stamps fall back to now), encode_us keeps its submit->AU meaning via a separate inflight stamp, and a new 'queue' stage (delivery->submit age of fresh frames) rides PUNKTFUNK_PERF and the web-console stats samples. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:33:56 +00:00
enricobuehler	7ac71159ff	feat(host/vaapi): fall back to the low-power (VDEnc) entrypoint — unblocks modern Intel ... Gen12+/Arc iHD exposes ONLY EncSliceLP, so the default open fails with 'no usable encoding entrypoint'. Try full-feature first (AMD unchanged, validated on the 780M), retry low_power=1, cache the mode per codec; PUNKTFUNK_VAAPI_LOW_POWER pins it. Probes inherit the ladder. Docs note the Intel HuC firmware requirement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:33:56 +00:00
enricobuehler	dacc3b5209	feat(host,web): multi-GPU selection — GPU inventory + preference API, web-console GPU card ... - new crate::gpu (compiled on all platforms so the OpenAPI doc stays platform-independent): DXGI / sysfs GPU inventory with reboot-stable ids (PCI vendor:device + occurrence — LUIDs are per-boot), persisted auto/manual preference (<config>/gpu-settings.json, atomic temp+rename with in-memory rollback), one selection with precedence console preference > PUNKTFUNK_RENDER_ADAPTER > max VRAM and graceful fallback when the preferred GPU is absent, plus a live "in use" record (RAII session guard wrapped around every encoder open_video returns) - fix: windows_gpu_vendor derived the encoder backend from DXGI adapter 0 instead of the selected render adapter — on a hybrid box (e.g. Intel iGPU at index 0 + NVIDIA dGPU) the backend could disagree with the GPU the capture ring / IddCx render pin sit on. The NVENC 4:4:4 probe now also runs on the selected adapter (was: OS default), the codec/4:4:4 probe caches are keyed per selected GPU (were process-lifetime OnceLocks), and an explicit PUNKTFUNK_ENCODER conflicting with the selected GPU's vendor warns up front - mgmt API: GET /api/v1/gpus (inventory + mode + preferred + next-session selection with reason + in-use GPU/backend/session-count) and PUT /api/v1/gpus/preference (validates mode/gpu_id before writing); openapi.json regenerated; the vdisplay render pin now also engages for a console preference (not just the env pin) - web console: GPU card on the Host page — list with vendor + VRAM, Automatic / Prefer controls, Preferred / Next session / "In use · backend" badges, missing-preferred-GPU warning and env-pin note; en + de messages - Linux: a matched manual preference picks the VAAPI render node and the NVENC-vs-VAAPI auto choice; auto mode is exactly the previous behavior Validated live on the hybrid laptop (RTX 3500 Ada + Intel Arc Pro, which enumerates twice — the occurrence ids disambiguate): enumerate, prefer, bad-id 400, restart persistence, auto-restore keeping the stored pick. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 13:57:26 +02:00
enricobuehler	070658d611	fix(host): clear clippy doc_lazy_continuation in the 4:4:4 docs ... A line-wrap put `+`/`*`-style markers at the start of two doc lines, which clippy (Windows host job, rust 1.96) reads as markdown list items whose unindented follow-on lines trip `doc_lazy_continuation` under `-D warnings`: - encode/windows/nvenc.rs `chroma_444` field doc (the failing Windows-host clippy job): "+ chromaFormatIDC = 3" → "and chromaFormatIDC = 3". - encode/linux/vaapi.rs `probe_can_encode_444` doc: "+ validate" → "and validate" (last line, didn't fire yet, but fragile — fixed pre-emptively). Pure doc rewording, no behaviour change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:38:07 +00:00
enricobuehler	7612238a59	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	f70832b59d	fix(host/vaapi): plug two AVBufferRef leaks in DmabufInner::open ... Surfaced while writing the unsafe-soundness proofs (2/N): both are refcount leaks (sound — never dangling/double-free — so the SAFETY proofs held, but real bugs on the persistent punktfunk1-host listener that opens a fresh encoder per session). 1. Per-session leak: `par->hw_frames_ctx = av_buffer_ref(drm_frames)` created a second owned ref. `av_buffersrc_parameters_set` takes its OWN ref of `par->hw_frames_ctx`, and `av_free(par)` frees only the struct, not the ref — so the extra ref leaked every session, pinning the DRM frames ctx + device. Fix: assign `drm_frames` borrowed (the standard ffmpeg pattern); our single owned ref lives in DmabufInner and is unref'd in Drop. 2. Error-path leak: the final `open_vaapi_encoder(...)?` returned without the unref ladder every other error path runs, leaking graph/drm_frames/ vaapi_device/drm_device on encoder-open failure. Fix: match + clean up before returning (nv12_ctx is borrowed from the sink → freed by graph teardown). cargo clippy -p punktfunk-host --all-targets -- -D warnings clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:02:54 +00:00
enricobuehler	b54695739d	docs(host): prove every unsafe block in the Linux FFI files + gate them (unsafe-proof program 2/N) ... Continues the structural unsafe-proof program (every unsafe carries a documented proof of soundness; the file gains #![deny(clippy::undocumented_unsafe_blocks)] so it stays proven). This batch covers all 10 remaining pure-Linux files (104 blocks), each proof stating the REAL invariant — not boilerplate: zerocopy/cuda.rs (26) leaked process-lifetime libcuda fn-ptr table; opaque CUcontext never dereferenced; free-exactly-once via the Arc<Mutex<PoolInner>> ownership graph; dmabuf fd take/close split zerocopy/egl.rs (18) eglGetProcAddress'd procs with the GL context current; EGLImage liveness; the two-call modifier-query bounds zerocopy/vulkan.rs (4) copy-bounds arithmetic (src_size>=span); Send = thread confinement to the punktfunk-pipewire thread dmabuf_fence.rs (4) poll/ioctl/close fd liveness + ownership capture/linux/mod.rs (16) spa_data repr(transparent) cast; null-checked spa derefs; single-loop-thread buffer ownership until requeue inject/linux/gamepad.rs (10) uinput ioctl request-number ↔ struct-size match (static-asserted); InputEventRaw no-padding for the byte cast encode/linux/vaapi.rs (15) + encode/linux/mod.rs (9) ffmpeg object ownership/ free ladders; VAAPI/DRM graph; Send = single-thread transfer inject/linux/wlr.rs (2), vdisplay/linux/kwin.rs (1) No memory-unsafety SUSPECT blocks were found — the unsafe is sound. The vaapi agent did flag two real AVBufferRef *leaks* (not UB) in DmabufInner::open; marked inline with NOTE(leak) and addressed in a follow-up. Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings is clean (each file's deny gate hard-errors on any undocumented block). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:00:30 +00:00
enricobuehler	fced221684	refactor(windows-host): confine platform code under windows/ + linux/ folders (Goal-1 stage 6) ... Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the shared HID codecs into `inject/proto/`): capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/ audio/{windows,linux}/ vdisplay/{windows,linux}/ src/windows/ (service, wgc_helper, win_adapter, win_display) src/linux/ (dmabuf_fence, drm_sync, zerocopy/) Done with `#[path]`, NOT a module rename: every file moves into its folder while the `crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::` reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in behaviour with ZERO reference churn, far lower risk than collapsing to a single `crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so the path churn didn't fight them. Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3 remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all 36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file (the inline `mod X {` blocks are self-contained). Box build to follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 18:53:45 +00:00