32 Commits

Author	SHA1	Message	Date
enricobuehler	78dba293a8	feat(encode): Vulkan Video B0 — vendored VK_VALVE_video_encode_rgb_conversion + RGB-direct probe telemetry ... First phase of design/vulkan-rgb-direct-encode.md (punktfunk-planning): make the captured BGRx dmabuf the direct encode source with the VCN EFC front-end doing the 709-narrow CSC — deleting the per-frame compute CSC, both plane copies, the semaphore hop and one queue submit. B0 changes nothing about the encode path. It vendors the extension surface (vk_valve_rgb.rs — ash 0.38 predates it; same rationale and style as the vk_av1_encode module) and probes at open whether this host qualifies: extension present (Mesa >= 26.0 + EFC hardware) → feature bit → conversion caps cover the compute shader's exact math (709 / narrow / midpoint both axes) → encode-src format set offers B8G8R8A8 with DRM-modifier tiling. The verdict is one INFO line (rgb_direct=available | first missing requirement) — the field telemetry that decides where B1 can default on. Verified: cargo check + clippy clean + unit tests green (Linux box); the probe short-circuits to no-ext on non-RADV drivers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-20 19:29:22 +02:00
enricobuehler	cfdec2729d	perf(encode): Vulkan Video — explicit VCN preset, persistent bitstream map, CSC split timing ... Field follow-up to 6dc195f9 (the AMD host whose H265 encode sits at ~5 ms): the remaining time is the VCN ASIC preset, and we never chose one. 1. Explicit encode quality level. RADV only emits a VCN preset op when the app issues ENCODE_QUALITY_LEVEL — we never did, so every session ran the firmware's default preset. Install the resolved level on the first frame's RESET control (chained ahead of the CBR state, which satisfies the spec's quality-change-carries-rate-control rule) and bake the same level into the session-parameters object (the spec requires the match). Default 0 = the fastest tier the driver exposes (RADV: SPEED, except H265 on pre-RDNA4 which the driver pins to BALANCE); clamped to the profile's maxQualityLevels. PUNKTFUNK_VULKAN_QUALITY=0..3 overrides for quality-biased setups. Logged at open so field logs show the tier. 2. Persistent bitstream mapping. read_slot vkMapMemory/vkUnmapMemory'd the host-visible bitstream buffer every frame; map each ring slot once at build instead (coherent memory; vkFreeMemory implicitly unmaps). 3. PUNKTFUNK_PERF CSC split. GPU timestamps bracket the compute batch (import barriers + cursor prep + CSC dispatch + plane copies) and a sampled csc_us line logs every ~2 s — separating shader/copy time from the ASIC encode inside the pump's wait_us, the VAAPI submit-split analog for this backend. Off (and unrecorded) unless PUNKTFUNK_PERF. Verified: cargo check + clippy + DPB/RPS unit tests green (Linux box). The GPU smoke tests still need a RADV host (NVIDIA driver fails session open on the build box, pre-existing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-20 19:17:34 +02:00
enricobuehler	6dc195f982	fix(encode): Vulkan Video poll() must block per the depth-1 pump contract ... The session pump's depth-1 loop (capture → submit → poll) treats a poll() None as "the backend holds the frame internally, re-poll next tick" — true only of the libav AMF/QSV wrappers, whose codecs genuinely hold ~2 frames. The Vulkan Video backend probed its slot fence with get_fence_status instead, so the AU of every frame — finished on the ASIC in ~5 ms — sat unharvested until the tick AFTER the next frame was submitted: encode_us read as one full frame period (~17 ms at 60 Hz vs VAAPI's ~5.3 ms on the same VCN, the AMD field report) and every frame shipped a frame period late — one frame of avoidable glass-to-glass latency on the path that exists to beat libav VAAPI. Wait the oldest in-flight slot's fence in poll() instead, matching the sync NVENC backend's blocking lock_bitstream and the documented Capturer::pipeline_depth contract ("capture → submit → poll-blocks"). Bounded by ENCODE_FENCE_TIMEOUT_NS like enqueue()'s backpressure wait, for the same reason: poll runs on the thread the stall recovery runs on, so a wedged GPU must surface as an error, not park it. None now only means "nothing submitted". Covers H265 and AV1 (shared poll). Verified: cargo check + DPB/RPS unit tests green (home-worker-5); the GPU smoke tests fail at session open on that box's NVIDIA driver identically on unmodified origin/main — pre-existing, needs the RADV box for on-glass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-20 18:53:21 +02:00
enricobuehler	1e8b267d93	Merge branch 'fix/vulkan-open-leak' into land/sweep-all	2026-07-20 00:43:19 +02:00
enricobuehler	28491acf2a	fix(encode): unwind Vulkan Video open failure instead of leaking every prior object ... VulkanVideoEncoder::open_inner creates ~20 Vulkan objects across ~15 fallible steps, but all cleanup lived in the encoder's Drop — which only runs once the value exists at the final Ok(Self). Any earlier ?/bail! leaked everything built so far (a VkDevice + GPU memory per retried open, and this backend is the default encode path on AMD/Intel Linux hosts where open can fail transiently). Factor the entire teardown sequence — unchanged — into a VkTeardown guard whose Drop destroys any prefix of the build (vkDestroy*/vkFree* are defined no-ops on VK_NULL_HANDLE): open_inner mirrors each object into the guard as it is created and disarms it only at Ok(Self); the encoder's own Drop rebuilds one from its fields, so both paths share one sequence and cannot drift. make_frame now builds in place into a guard-parked null Frame so a mid-build failure unwinds its partial handles too, and make_video_image / vk_util::make_plain_image (also used by the PyroWave backend) / build_parameters_h265 no longer leak their own partially-created objects on failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-19 22:15:44 +02:00
enricobuehler	b0fbb80fd5	fix(encode): key the PyroWave plane-import cache on the capturer's ring generation ... Completes the partial fix from the previous commit. The Windows PyroWave backend caches its imported plane images keyed on the D3D11 texture's COM address, and holds no reference on that texture — so once the capturer recreates its ring, those addresses can be handed straight back out by the allocator and a pointer-keyed cache hit returns an image bound to a texture that no longer exists. Adding the extent to the key ruled out same-address-different-size aliasing, but a recycle at identical dimensions still aliased. The capturer already tracks exactly the value needed: `generation`, bumped on every ring recreate. Plumbed it onto `PyroFrameShare` and the encoder now flushes every cached import when it changes, which makes cache identity independent of allocator behaviour rather than a bet against pointer reuse. Validated on the RTX box: `pyrowave_win_smoke` (forced with `--ignored`, the only test that actually exercises this path on real hardware) passes all ten configurations — 1024²/720p/1080p/1440p across SDR/HDR and 4:2:0/4:4:4 — with correct decoded chroma means, so the steady-state cache-hit path still works. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 21:45:14 +02:00
enricobuehler	6f52397342	fix(encode): bound NVENC async pipelining by the capturer's texture ring ... The Windows direct-NVENC backend registers and encodes the capturer's textures IN PLACE (no CopyResource), so how deep it may pipeline is a property of the CAPTURER, not of the encoder. It was bounded only by `async_inflight_cap()` — `PUNKTFUNK_NVENC_ASYNC_DEPTH`, default 4, clamped to the output-bitstream pool — which consults nothing about the capturer, while the comment at the backpressure loop claimed it "keep[s] in-flight depth within the capturer's texture ring". It never did. The IDD-push capturer rotates `OUT_RING = 3` per delivered frame with no regard for encode completion (its own invariant note says OUT_RING(3) > max pipeline_depth(2)). With the default async depth of 4 the encoder can therefore still be reading a texture the capturer has already handed out again and overwritten: torn or mixed frames. It is visual corruption rather than UB, so it fails silently and intermittently — the worst shape to diagnose from a field report. Adds `Encoder::set_input_ring_depth`, reported from `Capturer::pipeline_depth`, and bounds the async backpressure loop by `min(async_inflight_cap(), depth)`. For IDD-push that yields 2, matching the capturer's stated contract; backends that copy their input, or are synchronous, ignore it. Wired at ALL THREE encoder-creation sites (initial open, stall/resize rebuild, ABR rebuild) and forwarded through `TrackedEncoder` — this crate has a documented trap where an unforwarded defaulted trait method silently no-ops through that wrapper, which has already bitten the direct-NVENC work once and the wire-chunking probe once. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 21:42:53 +02:00
enricobuehler	a38adad943	fix(encode): bound GPU waits, validate encode status, repair command-buffer and cache invariants ... Five of the nine medium findings from the pf-encode sweep. The remaining four need cross-crate plumbing or an unwind refactor and are deliberately left out. - vulkan_video `enqueue` waited on the backpressure fence with `u64::MAX`. That wait runs ON the host encode thread — the same thread the stall watchdog's `reset()` would run on — so a wedged GPU parked the one thread that could recover the session: no error, no reset, and teardown blocking on the join. This is the DEFAULT encode path for AMD/Intel Linux hosts (both shipped build recipes enable `vulkan-encode` and `vulkan_encode_enabled()` defaults true). Now bounded by ENCODE_FENCE_TIMEOUT_NS with expiry surfaced as an error. - vulkan_video `import_cached` evicted a cached dmabuf import and destroyed its image/view/memory with no fence wait, while up to `ring_depth - 1` submitted frames may still reference it — a GPU-side use-after-free. `Drop` and `reset` both idle first; this was the one unguarded destroy. Now idles before the eviction loop, guarded on the length test so the steady state pays nothing. - vulkan_video `read_slot` never asked for the encode's operation status, so a FAILED encode was indistinguishable from a successful one and its feedback was read as if it described real bitstream. Now requests WITH_STATUS_KHR and refuses anything that is not COMPLETE. - linux/pyrowave `encode_frame` opens its recording window early and has six fallible steps inside it; every one returned with `cmd` still RECORDING, and nothing repaired it (one `begin_command_buffer` in the file, and neither `reset()` nor `Drop` touches `cmd`), so the next frame called `begin` on a recording buffer — invalid usage. `submit` now resets the buffer on error; legal on all these paths since the pool carries RESET_COMMAND_BUFFER and the buffer is not pending. - windows/pyrowave `encode_frame` ignored `frame.width`/`height` and imported planes at the encoder's configured extent, so a ring recreate at a new mode (the IDD capturer does this autonomously on a confirmed descriptor change) read the planes under a stale VkImageCreateInfo. Added the size guard its QSV and AMF siblings already carry, and keyed the plane cache on (address, width, height) so a recycled COM address cannot resurrect an import of a different size. NOTE: a recycle at the SAME size is still theoretically possible; the complete fix keys on the capturer's ring generation and needs that plumbed onto `PyroFrameShare`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 21:34:16 +02:00
enricobuehler	b22d0da75b	fix(encode): port the RFI taint sweep to Vulkan Video, close the QSV sweep hole, bounds-check encode feedback ... Three defects from the pf-encode sweep, each adjudicated against source. - Vulkan Video never received fecbec2d's taint sweep (it was carved out one commit later). `pick_recovery_slot` accepts any resident slot whose wire is below the CURRENT loss start, but "resident and older than this loss" is not "the client decoded it": after an earlier loss [a,b] recovered at wire r, everything in [a, r-1] is undecodable at the client — the lost frames plus every frame that predicted through the gap — and those wires stay eligible until the 8-slot ring rolls them out. A later loss could therefore anchor on one and ship it tagged `recovery_anchor`, which is the client's definitive re-anchor signal (punktfunk-core/src/reanchor.rs): the host lifts the client's post-loss freeze onto a picture built from a reference it never had. Swept before anchor selection, matching AMF/QSV. `slot_wire` is blanked and `slot_poc` deliberately is NOT: `slot_poc` feeds `build_h265_rps_s0`, which must keep naming every physically-resident DPB picture or a conforming decoder evicts them and the anchor then references a picture the client already dropped. - QSV's sweep was incomplete, and in its MODAL case. `ltr_slots` mirrors the hardware DPB, but nulling an entry issues no VPL call — the frame stays marked long-term until that LongTermIdx is re-marked or an IDR flushes it (amf.rs states this verbatim). The rejection loop iterates the post-sweep mirror and only rejects `Some` slots, so it silently skipped the single entry the sweep exists to distrust, leaving the recovery frame free to predict from it. With NUM_LTR_SLOTS=2 the "exactly one slot swept" case is the common one, and the two existing tests cover only the both-survive and both-swept cases. Taint is now recorded in `ltr_tainted` with the FrameOrder left in place, so anchor selection and the queued-force guard skip it while the rejection list still names it. - `read_slot` built a slice from the driver-reported (offset, bytes-written) encode feedback with no validation against `bs_size`, so a driver reporting a range outside the bitstream buffer produced an out-of-bounds read shipped straight onto the wire. Checked in u64 (so the add cannot wrap) before `map_memory`, so the error path has no unmap to unwind. Adds `taint_sweep_excludes_slots_from_an_earlier_loss` covering the two-loss case the existing single-loss test does not reach. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 21:18:17 +02:00
enricobuehler	f9668b16a1	fix(encode): NVENC partial-init session leak + three backend-parity gaps ... All four found in the pf-encode quality sweep and verified against source. - NVENC partial-init leak (BOTH platforms, high): `init_session` publishes `self.encoder` — and on Windows charges LIVE_SESSION_UNITS — *before* its remaining fallible steps (bitstream buffers; on Linux also the input-surface alloc and `register_resource`). A failure there left a live session with `inited == false`, and every guard on the re-init path keys off `inited`, so the next submit skipped teardown and overwrote `self.encoder`: the session leaked permanently toward the driver's per-process cap, and its budget units never returned, progressively starving parallel-display admission. `teardown` already keys off `encoder.is_null()` rather than `inited`, so it cleans up exactly this half-built state — it just was never called. Now invoked on the `init_session` error path on both platforms. - `can_encode_10bit` asked the wrong backend (medium): it resolved via `linux_auto_is_vaapi`, which ignores `encoder_pref`, while `can_encode_444` and `open_video` honour it. On a host that forces a backend (e.g. `encoder_pref = "vaapi"` on an NVIDIA box) the probe answered for NVENC while the session opened VAAPI, so the negotiated bit depth — and the HDR/SDR colour label derived from it — described a backend that never ran. Now uses the same `linux_zero_copy_is_vaapi` mirror, and `linux_auto_is_vaapi` carries a warning that it resolves the `auto` case only and is not a dispatch mirror. - Linux software arm ignored SW_BITRATE_CEIL (low): the Windows arm clamped openh264 to 100 Mbps, the Linux arm passed the full negotiated rate. The constant is now module-scope so both arms share one value. - QSV/AMF env-parity (low): `PUNKTFUNK_IR_PERIOD_FRAMES` was a no-op on QSV despite the comment claiming parity with AMF, and `PUNKTFUNK_NO_QSV_LTR` / `PUNKTFUNK_INTRA_REFRESH` had dropped AMF's `trim()` and `yes`/`on` spellings, so a value with stray whitespace silently did nothing on Intel while the same value worked on AMD. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 20:47:55 +02:00
enricobuehler	04e4394ee0	fix(encode): close the two teardown memory-safety holes in the reset paths ... Both surfaced in a post-refactor quality sweep of pf-encode and were then verified against the source (and, for pyrowave, against the C side). - PyroWave (BOTH platforms): `reset()` destroyed the encoder and, when the rebuild failed, returned `false` leaving `pw_enc` pointing at the freed object — `Drop` then destroyed it a second time. `pyrowave_encoder_destroy` is a plain `delete` (pyrowave_c.cpp:1184, which also reads `encoder->device` afterwards) with no null check, so this is a real double free. The failure branch is not vacuous: the rebuild fails when the device is lost/OOM, which is exactly the state that makes the stall watchdog call `reset()` in the first place, so the host corrupts its heap on the path that runs when things are already going wrong. Now nulls `pw_enc` before the fallible create, publishes only on success, and null-guards both `Drop` and `encode_frame` (the Windows `Drop` already guarded `sync` this way). - QSV: `reset()` dropped `pending` — each entry owning the `Box<BsBuf>` the runtime writes into asynchronously — BEFORE `MFXVideoENCODE_Close` aborted those operations, so the VPL runtime could write into freed heap. The preceding drain is best-effort and bails on the first `Err`, i.e. precisely the wedged-encoder case that triggers the reset. Fixed by ordering: Close, then clear. The full-teardown path was already correct (`Inner` declares `session` before `pending`, and fields drop in declaration order). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 20:07:57 +02:00
enricobuehler	fecbec2daf	fix(encode): make LTR-RFI loss recovery sound under sustained loss ... Field report (lid-closed Intel laptop, ~6-19% sustained loss): the stream never healed — permanent macroblock soup. Three stacked bugs: - QSV answered RFI with PreferredRefList only, a reorder HINT per the VPL spec — the recovery frame could keep predicting from tainted short-term refs. Now rejects every other DPB candidate (RejectedRefList) and caps L0 at one active entry (AVC/HEVC), matching AMF's hard ForceLTRReferenceBitfield / NVENC invalidation semantics. - Neither QSV nor AMF taint-swept LTR slots across losses: a slot marked inside the client's corrupt window became the "known-good" anchor of the NEXT loss, propagating corruption through every recovery. Both now drop slots at-or-after the loss start before picking an anchor, and guard a queued force whose slot the sweep emptied (no false recovery_anchor tag). - The native plane re-anchored the FULL IDR cooldown on every successful RFI, so under sustained loss the client's escalating keyframe requests were coalesced away indefinitely (field log: dozens swallowed, one IDR per ~8 s). RFI now anchors a 300 ms echo window with a 2-swallow budget per loss episode; a client still asking past that gets its IDR. Live-validated on Arc (qsv feature): 6/6 including the new qsv_live_ltr_rfi_taint_sweep_declines (a loss covering every live mark declines the RFI and falls back to IDR recovery). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 17:53:48 +02:00
enricobuehler	232b41e88b	fix(pf-encode/windows): cover the Linux HDR pixel formats in the win swscale match ... The GNOME 50 HDR work added PixelFormat::X2Rgb10 / X2Bgr10 but only taught the Linux encoders about them. `sws_src` in the Windows-gated ffmpeg_win.rs matches PixelFormat exhaustively, so the Windows host stopped compiling: error[E0004]: non-exhaustive patterns: `X2Rgb10` and `X2Bgr10` not covered --> crates\pf-encode\src\enc\windows\ffmpeg_win.rs:132:14 Linux CI never caught it — the file is cfg(windows), so `cargo clippy --workspace` on the Linux runner never compiles it. Both are Linux-only screencast formats (the Windows HDR path stays Rgb10a2/P010, per the PixelFormat docs), so they join the existing bail arm. Spelled out rather than folded into a `_` catch-all so the next PixelFormat addition breaks this match again on purpose. Verified: cargo check --workspace --all-targets --features nvenc,amf-qsv on the Windows box (192.168.1.173). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 10:30:33 +02:00
enricobuehler	00f759ec72	style(pf-encode): clear the clippy gate on the HDR/PyroWave additions ... `cargo clippy --workspace --all-targets --locked -- -D warnings` was red on main — three lints landed with the GNOME 50 HDR + PyroWave 4:4:4 work: * pyrowave_wire.rs: `aw / 2 >> level` tripped clippy::precedence. Rust already binds `/` tighter than `>>`, so this always parsed as `(aw / 2) >> level` (subband dim at half res, then one halving per DWT level) — the parens are purely explicit, no change in behaviour. * linux/mod.rs: `probe_can_encode_10bit` sat after `mod hdr_tests` (clippy::items_after_test_module) — moved above the test module, unchanged. Lint-only; no functional change. fmt/clippy/test all green afterwards. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-19 10:23:47 +02:00
enricobuehler	a83acf3ee1	style(fmt): rustfmt the GNOME 50 HDR files under the pinned toolchain ... `feat(hdr): GNOME 50 HDR screencast capture` (0e977817) landed with rustfmt drift — six files were not clean under the pinned 1.96.0 toolchain, so `cargo fmt --all --check` (ci.yml "Format") is red on main. Pure whitespace/ wrapping from `cargo fmt --all`; no semantic change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-19 10:09:34 +02:00
enricobuehler	0e977817f9	feat(hdr): GNOME 50 HDR screencast capture + Linux Main10 encode ... GNOME 50 (Mutter MR 4928, PipeWire >= 1.6) added HDR screen sharing for monitor streams: 10-bit PQ formats (xRGB_210LE/xBGR_210LE) with MANDATORY BT.2020 + SMPTE-2084 colorimetry props, advertised while the mirrored monitor is in BT.2100 colour mode. Wire the Linux host into it end-to-end on the GameStream desktop-mirror path (PUNKTFUNK_VIDEO_SOURCE=portal): * pf-frame: PixelFormat::X2Rgb10/X2Bgr10 (DRM XR30/XB30; X2Bgr10 is the Windows Rgb10a2 layout) + fourccs. * pf-capture: want_hdr portal offer — HDR-only LINEAR-dmabuf pods with MANDATORY PQ/BT.2020 props (SHM excluded: Mutter's SHM record path paints 8-bit ARGB32 regardless of format; tiled excluded: the EGL de-tile blit is 8-bit RGBA8), negotiated-colorimetry parse, generic HDR10 hdr_meta(), packed-10-bit CPU cursor blend, a process-wide SDR downgrade latch on negotiation timeout, and a DisplayConfig BT.2100 colour-mode probe (gnome_hdr_monitor_active). * pf-encode: libav NVENC X2RGB10->P010 swscale (BT.2020 limited) -> HEVC Main10 / 10-bit AV1 with PQ VUI; VAAPI 10-bit on both paths (CPU P010 upload + dmabuf XR30 scale_vaapi p010/bt2020); can_encode_10bit now probes for real on Linux; 10-bit sessions route around the 8-bit-only Vulkan-video/direct-NVENC backends. * GameStream: host_hdr_capable() Linux arm, live monitor-HDR check at RTSP honor time, capturer-pool reuse keyed on HDR-ness, gs_bit_depth covers the new formats. New `punktfunk-host hdr-probe` diagnostic and a PUNKTFUNK_SPIKE_HDR spike lever. * Native plane stays honestly 8-bit via capturer_supports_hdr(): Mutter RecordVirtual streams are SDR-only upstream (GNOME 50 and 51-dev), so virtual-display sources cannot deliver HDR yet. Validated on the RTX 5070 Ti (GNOME 50.3 / PipeWire 1.6.8): the Main10 probes pass and the ignored nvenc_hdr10_smoke GPU test emits an IDR that ffprobe reads as Main 10 / yuv420p10le / bt2020nc / smpte2084 / limited. Live HDR capture negotiation still needs an HDR monitor on glass; VAAPI 10-bit needs the AMD box. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-19 09:30:55 +02:00
enricobuehler	75474dcc90	fix(pyrowave): guard 4:4:4 modes that overflow the rate controller's block index ... The vendored rate controller packs its wavelet block index into 16 bits (RDOperation.block_offset_saving), so a mode whose 32x32-block count exceeds u16::MAX wraps inside the controller and corrupts the bitstream — ~8K-class 4:4:4 territory. Compute the exact count (`block_count_32x32`, the counting walk of upstream init_block_meta, pinned against the validated Apple WaveletLayout) and expose `pyrowave_mode_fits_rdo`; the negotiator downgrades such a session to 4:2:0 before the Welcome (the honest-downgrade channel), and both encoders refuse outright if one slips through rather than emit a wrapped stream. Vendor patches: 0002-rdo-saving-clamp (analyze_rate_control.comp clamps the saving accumulation to the target, same overrun class as 0001; slangmosh.hpp regenerated), 0003-devel-encode-16bit-read (devel tool y4m 16-bit plane reads; tool-only, kept so the vendored source stays honest). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-19 01:16:51 +02:00
enricobuehler	ac0e73321c	perf(pyrowave): elevated GPU scheduling + global-priority encode queue ... PyroWave's wavelet encode runs on the GPU's compute/shader cores, so a GPU-bound game starves it: submit spikes from ~2 ms to ~15 ms under a 95%+ game load and the stream fps collapses. NVENC is immune (separate encoder ASIC). Two levers to let the encode get scheduled ahead of the game's rendering: - Windows process GPU scheduling: D3DKMTSetProcessSchedulingPriorityClass, env PUNKTFUNK_GPU_PRIORITY = off|above-normal|high (default)|realtime. Best-effort, once per process, non-fatal on refusal (enc/windows/pyrowave.rs). - Global-priority Vulkan encode queue (Granite patch 0005): request a VK_KHR_global_priority queue (PYROWAVE_QUEUE_PRIORITY = off|high|realtime, default realtime), downgrading REALTIME→HIGH→none on NOT_PERMITTED so a refused class never regresses the encoder to HEVC. HONEST STATUS: on an RTX 4090 / Windows / WDDM neither moved the ~15 ms spikes — the graphics-vs-compute preemption granularity is the wall, not the priority level. Kept because both are correct, harmless (graceful fallback), and may help other GPUs/drivers. For a GPU-saturated game the working levers are reducing the encode's GPU cost (4:2:0/8-bit) or H.265; PyroWave holds full rate on the desktop and in games that leave the GPU headroom. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-19 01:02:01 +02:00
enricobuehler	9fe9c451dc	perf(pyrowave): pool encoder scratch buffers + fix client parser O(n²) — lift the 2.5 Gbps wall ... Real-world PyroWave streaming maxed ~2.5 Gbps with sagging fps while raw transport does 4.8. Root-caused to serial per-frame paths at BOTH ends (the transport was never the limit); this fixes the two dominant ones. Host (vendored shim, patch 0004): pyrowave_encoder_encode_gpu_synchronous allocated four Vulkan buffers (meta + bitstream, Device + CachedHost) on EVERY frame. At 240 fps with MB-scale bitstreams that per-frame allocator churn stalled the encode itself. Pool them on the encoder and reuse across frames (recreate only on a size grow); the sizes are session-fixed, so it is pure reuse after frame 1. On an RTX 4090 the 5120x1440 submit+fence-wait drops ~15 ms -> ~1 ms, i.e. the host serial ceiling goes 64 -> 1025 fps (444+HDR 44 -> 614). Safe under the synchronous encode model; re-validated by pyrowave_win_smoke (Windows) and pyrowave_smoke/_444 (Linux). Applies to both host encoder paths (they share the shim). Client (Apple Metal decoder): WaveletBitstream.parse reserved the payload buffer per packet (reserveCapacity(count + words), an exact realloc each of ~3000 packets/frame => O(n²)) and copied word-by-word. Reserve once up front and memcpy each packet's coefficients in one shot (all Apple platforms are little-endian, so the wire's LE u32s land verbatim; memcpy is alignment-free). 5.44 ms -> 0.055 ms per 1.44 MB frame (25x); byte-identical (parser unit tests + golden-frame PSNR unchanged). Also: - native.rs: PUNKTFUNK_PYROWAVE_MAX_MBPS caps PyroWave's open-loop Automatic bitrate pin for hosts on a constrained link (unset => no cap; an explicit client rate bypasses it). The pin is all-intra + ABR-off, so at a high pixel rate it can outrun the fabric (4:4:4+HDR 5120x1440@240 pins ~5.3 Gbps, over a 5 GbE link) and the overshoot just becomes loss. - pf-encode caps(): report the real opened chroma instead of a hardcoded 4:2:0 default, so a genuine 4:4:4 session no longer trips the spurious "encoder chroma disagrees with the negotiated Welcome" warn. Also fix a latent Windows reset() that rebuilt at 4:2:0 for a 4:4:4 session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-18 19:09:23 +02:00
enricobuehler	188edde2b3	feat(pyrowave): Windows host HDR + 4:4:4, Rust client HDR present ... Phase 3 of design/pyrowave-444-hdr.md. A PyroWave session now negotiates HDR (10-bit) and 4:4:4 on a Windows host exactly like HEVC/AV1, and the Linux client presents it through the real HDR10 path. Host (Windows): BgraToYuvPlanes becomes mode-aware — SDR/BGRA and HDR/scRGB variants at half- or full-res chroma. The HDR passes reuse HdrP010Converter's exact colour math (scRGB -> PQ BT.2020 limited studio codes, verified by hdr_p010_selftest) but write P010-style MSB-packed codes into two separate shareable R16_UNORM/R16G16_UNORM textures; chroma keeps the pyrowave family's centre-sited 2x2 box. idd_push pins the composition to the NEGOTIATED depth (SDR sessions force advanced color off as before; 10-bit sessions enable it and ride the FP16 ring), and the descriptor poller re-asserts that state instead of following display flips the fixed-format encoder can't. The encoder imports 8/16-bit planes per session and stamps the sequence header's BT.2020/PQ/matrix bits on HDR (stamp_color_bits, extending 574e3e4e's range stamp); supports_10bit/can_encode_10bit/can_encode_444 gates open (HDR Windows-only — Linux capture has no HDR source). Client: the plane ring becomes R16_UNORM for 10-bit sessions (with a STORAGE_IMAGE format probe), the planar CSC pass joins the HDR10 swapchain rebuild (set_hdr_mode previously destroyed it without rebuilding — latent), st.hdr follows frame.color.is_pq(), and the planar push constants carry depth-10 MSB-packed rows + the PQ tonemap mode, identical to the NV12 arm. Verified: .173 (RTX 4090) deploy-config clippy + fmt + wire tests + the extended pyrowave_win_smoke (10-case {SDR,HDR}x{420,444} matrix incl. R16 imports and header stamps); .21 (RTX 5070 Ti) clippy across 4 crates, host 186 tests, client/presenter/encode tests, both Linux GPU smokes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-18 13:21:06 +02:00
enricobuehler	4861824e7d	feat(pyrowave): Linux 4:4:4 encode — per-pixel CSC, full-res chroma, gate open ... Phase 2 of design/pyrowave-444-hdr.md. can_encode_444(PyroWave) now returns true on Linux, so a client that advertises VIDEO_CAP_444 (the 4:4:4 setting) and prefers PyroWave negotiates a full-chroma wavelet session end to end (the client decoder side landed in 5eb930e7). - rgb2yuv444.comp: the 4:4:4 twin of rgb2yuv.comp — one invocation per pixel, full-res interleaved RG8 CbCr, no box filter/siting, byte-identical BT.709 limited coefficients; compiled .spv committed (glslangValidator -V, matches the existing shader's toolchain). - Encoder: chroma-conditional pyrowave create (open + reset), full-res chroma plane + views, per-pixel dispatch, 4:2:0-only even-dims check. - Tests: decode oracle grows a 4:4:4 mode (YUV444P CPU readback); pyrowave_smoke_444 round-trips plane means AND drives the busy test card at the ~2.6 bpp operating point asserting in-budget + run-to-run deterministic AU sizes — the exact regime that silently corrupted before the vendored payload_data fix (patches/0001), so this doubles as its regression test. Verified on .21 (RTX 5070 Ti): clippy -D warnings, host tests, and both GPU smokes (pyrowave_smoke + pyrowave_smoke_444) green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-18 12:43:34 +02:00
enricobuehler	5eb930e71d	feat(pyrowave): negotiation plumbing for 4:4:4 + HDR — thread chroma/depth/ColorInfo end to end ... Phase 1 of design/pyrowave-444-hdr.md. No behavior change yet: the handshake's 4:4:4 gate now admits PyroWave (probe = can_encode_444(codec), capture gate inherently satisfied — the wavelet path always ingests an RGB source and does its own CSC), but can_encode_444 stays false for PyroWave until the per-OS full-res-chroma CSC variants land (Phase 2 Linux, Phase 3 Windows), so every session still resolves 4:2:0/8-bit. - Both host encoders take the negotiated ChromaFormat (bail on 444 for now); the PUNKTFUNK_ENCODER=pyrowave lab override pins 4:2:0. - Bitrate: the automatic ~1.6 bpp pin resolves AFTER depth+chroma and scales x1.625 for 4:4:4 / x1.15 for 10-bit (factors from the Phase-0 fixture matrix); the mid-stream mode-switch re-resolve threads the session's values. - Client: PyroWaveDecoder builds its plane ring (full-res chroma when 444) and creates the upstream decoder from the negotiated chroma, keeps chroma fixed across mid-stream resizes, drops the even-dims requirement for 444, and returns the negotiated Welcome ColorInfo as the frame colour contract instead of hardcoded BT.709 (the wavelet bitstream has no VUI). Verified on .21 (RTX 5070 Ti): clippy -D warnings (host+client+encode), host 186 tests, client + pf-encode tests, fmt, and the pyrowave_smoke GPU round-trip through the patched vendored lib (97cf15e3). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-18 12:37:12 +02:00
enricobuehler	574e3e4e3f	fix(pyrowave): signal ycbcr_range=LIMITED in the sequence header ... pyrowave's encoder fills the BitstreamSequenceHeader with `= {}` and its C API offers no way to set colour/range, so it signals ycbcr_range=0=FULL — but both host CSCs (rgb2yuv.comp on Linux, BgraToYuvPlanes on Windows) always emit BT.709 LIMITED Y'CbCr (black = Y'16). A client that honours the VUI (the Apple wavelet decoder reads bit 30 of word1) then skips the limited→full expansion and shows washed-out, raised blacks — reported on both Linux and Windows hosts. Patch the range bit HONEST (mark_limited_range in the shared pyrowave_wire, called by both encoders after packetize). Clients that hardcode limited (the Vulkan video_pyrowave path) are unaffected, and pyrowave's own decode ignores the flag (raw Y'CbCr reconstruction). No client rebuild needed. Unit-tested + asserted in pyrowave_win_smoke; the smoke decode still round-trips 100/180/60. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-18 11:39:44 +02:00
enricobuehler	ebd9967547	feat(pyrowave): Windows host encoder — separate-plane zero-copy D3D11→Vulkan ... Wire PyroWave into the Windows host (design/pyrowave-windows-host-zerocopy.md). Before this a macOS client + Windows host that both selected PyroWave silently ran HEVC: the host never advertised CODEC_PYROWAVE and open_video_backend bailed. Approach (zero-copy, no GPU→CPU→GPU): pyrowave owns its own Vulkan device (create_device_by_compat, by render-GPU vendor/device-id — NOT LUID, invalid in Session 0). The capturer runs a BGRA→YUV BT.709-limited CSC (matching rgb2yuv.comp) into TWO SEPARATE shareable plane textures — full-res R8 Y + half-res R8G8 CbCr — which the encoder imports into pyrowave's device. Separate single/two-component textures import reliably on NVIDIA at any size; a single planar NV12 import does NOT (the vendored interop test: "only very specific resource sizes" — confirmed on-glass: 1024² fine, 720p/1080p/1440p garbage). A shared D3D11 fence, signalled after the CSC, is imported as a Vulkan timeline semaphore so the wavelet read is ordered after it. - pf-encode: enc/windows/pyrowave.rs (Encoder impl, two-plane import + Linux-style plane views); host_wire_caps advertises CODEC_PYROWAVE on Windows when the backend isn't Software; open_video_backend routes a negotiated PyroWave session first; pyrowave-sys on the Windows target; interop confirmed at open → clean HEVC fallback. - pf-encode: shared, unit-tested enc/pyrowave_wire.rs (single source of truth for the client-facing AU framing); Linux encoder uses it too. - pf-capture: dxgi.rs BgraToYuvPlanes CSC; idd_push.rs pyrowave mode — forces the virtual display SDR (the VideoProcessor can't ingest the FP16 HDR ring), a two-plane shareable out-ring, a shared fence passed every frame (so a rebuilt encoder re-imports it). Threaded via OutputFormat::pyrowave. - pf-frame: D3d11Frame::pyro carries the CbCr plane + fence; OutputFormat::pyrowave. Verified on .173 (RTX 4090): full-host build + clippy -D warnings (nvenc,amf-qsv) + fmt --all --check; pyrowave_wire unit tests; pyrowave_win_smoke GPU test round-trips distinct Y/Cb/Cr (100/180/60) exactly at 1024²/720p/1080p/1440p; Stage-0 interop validated in the real Session-0 service context on-glass. Deployed to the box. Owed: final on-glass picture/latency confirmation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-18 11:39:44 +02:00
enricobuehler	1bc156aab0	perf(latency): T2.2 Linux NVENC two-thread retrieve + T2.3 REALTIME auto-gate ... design/latency-reduction-2026-07.md tier 2, the two code-side halves: - T2.2: the Linux direct-NVENC backend gains the two-thread retrieve (PUNKTFUNK_NVENC_ASYNC, the same opt-in knob as Windows): the session stays sync-mode (async events are Windows-only) but the blocking lock_bitstream moves to a dedicated pf-nvenc-out thread — the NVENC guide's sanctioned submit-thread/output-thread split. poll() drains completions non-blocking, submit() backpressures at PUNKTFUNK_NVENC_ASYNC_DEPTH (default 4) in-flight; map/unmap and every other session call stay on the encode thread; teardown joins the thread before destroying the session. Under a GPU-saturating game completed frames queue instead of serializing capture on the encode wait. - T2.3: PUNKTFUNK_GPU_PRIORITY_CLASS gains 'auto' AND IT IS THE NEW DEFAULT (gpu-contention §5.C): HIGH immediately, then REALTIME where the documented NVIDIA+HAGS+near-full-VRAM NVENC hang cannot bite — HAGS probed once via D3DKMT WDDM_2_7_CAPS (off => REALTIME outright); HAGS on => a pf-gpu-prio monitor flips REALTIME<->HIGH on LOCAL-segment VRAM headroom (downgrade >92% of budget, restore <=85% for 3x2s polls). 'high' restores the old static default; 'realtime' pins it (operator owns the hazard). Validated: .21 clippy -D warnings (punktfunk-host --features nvenc) against the QSV-merged main; .133 Windows cargo check of pf-frame + punktfunk-host. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 19:57:38 +02:00
enricobuehler	df70bf00f9	test(qsv): live on-glass matrix — HEVC, Main10+HDR, AV1 10-bit, LTR-RFI, no-IDR retarget ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 19:55:43 +02:00
enricobuehler	9e6ab9b94d	fix(qsv): enable D3D11 multithread protection before SetHandle (on-glass -16 on Arc) ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 19:52:44 +02:00
enricobuehler	0a617e7779	fix(qsv): clippy — per-block SAFETY comments, div_ceil, clamp, vec_box allow ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 19:44:16 +02:00
enricobuehler	75d1322b8a	fix(qsv): end the inner borrow before the retarget param rebuild ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 19:39:08 +02:00
enricobuehler	55e7f3fca9	feat(encode): native QSV backend — libvpl-sys + qsv.rs (Phases 0-3 of design/native-qsv-encoder.md) ... Vendored MIT VPL dispatcher (static, trimmed tree, pin 674d015b/v2.17.0) built via cmake+bindgen behind new feature 'qsv' (pf-encode + punktfunk-host forward). qsv.rs: dispatcher session on the capture adapter (LUID-matched), SetHandle D3D11, AsyncDepth=1/GopRefDist=1/VDEnc/CBR + HRD-off low-latency config, GetSurfaceForEncode + GPU CopySubresourceRegion input (zero-copy, no readback path), bounded sync-point poll, in-place reset with teardown escalation, no-IDR bitrate retarget (Reset + StartNewSequence=OFF), 10-bit P010 HEVC-Main10/AV1, HDR mastering/CLL SEI-OBU at IDR + BT.2020/PQ VSI, LTR-RFI via mfxExtRefListCtrl (AMF slot policy port, Query-gated per codec, wire-index FrameOrder pinning). Dispatch: native-first with ffmpeg fallback + PUNKTFUNK_QSV_FFMPEG hatch; probes (can_encode_10bit / windows_codec_support / windows_backend_is_probed) now answer natively for QSV. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 19:34:18 +02:00
enricobuehler	22b352c1da	chore(release): bump workspace version to 0.13.0 ... The eight W6 leaf crates hardcoded 0.12.0 instead of inheriting the workspace version — switched to version.workspace = true so the next bump is one line again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 16:32:35 +02:00
enricobuehler	9a36ea2132	refactor(host/W6.2): extract the video encode backends into the pf-encode crate ... encode.rs + encode/* (NVENC, VAAPI, native AMF, AMF/QSV ffmpeg, direct-SDK NVENC/CUDA, raw Vulkan-Video, PyroWave, openh264) move into crates/pf-encode behind one Encoder trait + open_video selector (plan §W6). The crate speaks the shared frame vocabulary (pf-frame: CapturedFrame/PixelFormat + the DXGI identity D3d11Frame/make_device) and pf-zerocopy (CUDA context/buffers), and NEVER pf-capture — the capture→encode edge is one-way (ZeroCopyPolicy, prior commit). Dep moves: the heavy encoder deps (ffmpeg-next, the NVENC SDK, openh264, pyrowave-sys) move from the host to pf-encode; the host's nvenc/amf-qsv/vulkan-encode/pyrowave features now FORWARD to pf-encode/*. The host keeps a mod-encode shim (pub use pf_encode) so every crate::encode::* path (negotiator + GameStream/native/mgmt planes) is unchanged. resolve_render_adapter_luid moves from the host's windows/win_adapter.rs into pf-gpu (both pf-encode and pf-capture need it as a peer of GPU selection); its 5 call sites (encode amf/nvenc, capture idd_push/synthetic_nv12, vdisplay manager) rewire to pf_gpu::resolve_render_adapter_luid and win_adapter.rs is deleted. pf-frame's make_device gains a # Safety section (public-unsafe-fn lint, latent since the pf-frame carve — a full-workspace -D warnings clippy catches it). Verified: Linux clippy -D warnings (pf-encode + host nvenc,vulkan-encode,pyrowave --all-targets) + 13/13 pf-encode + 299/299 host tests; Windows clippy -D warnings (pf-encode nvenc,amf-qsv --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-17 10:42:51 +02:00