forked from unom/punktfunk
enricobuehlerandClaude Fable 5
46935bf0a5
Second attempt at WP6.3; the first (ce543668) was withdrawn after its tight CBR window measured as a 36x bandwidth regression (97% filler NALs). The correction that unlocked this one: that measurement's BASELINE row was an 8-frame artifact. Under the shipped 1000ms/500ms CBR window a calm stream overflows the CPB once the initial fill drains (~30 frames at 10Mbps/60fps) and RADV then pads every frame to the exact rate share, forever. Measured on the 780M (1280x720@60, 10Mbps, calm content): 64 frames = 97.5% filler; 300 frames = 5.63MB at 98.5% filler where this commit ships 83KB at 0%. AV1: 99.6% -> 0%. The status quo was ~the full target bitrate of zeros on every idle AMD/Intel Vulkan-encode desktop and Steam Deck — and stuffing to exactly the target permanently satisfied the ABR calm brake (actual >= 3/4 * current), the ratchet WP6.3's withdrawal feared from the tight window, live in the shipped code all along. The fix reads VkVideoEncodeCapabilitiesKHR::rateControlModes (previously ignored — rateControlMode was hardcoded CBR with no capability check) and installs VBR with average == max plus the house ~1-frame window (vbv_window_ms, PUNKTFUNK_VBV_FRAMES-scaled) when the driver advertises VBR. VBR permits underspend, the exact missing degree of freedom: Vulkan exposes no filler-suppression control (AMF's filler_data=false / NVENC's default-off have no VK equivalent), so the MODE is the only lever. CBR-only drivers keep the loose window untouched — tightening it under CBR just starts the stuffing 30 frames earlier. Drivers advertising neither mode (ANV per current Mesa) keep the pre-existing CBR install, now WARN-logged. No pacing claim, deliberately: burst A/B on the 780M is byte-identical between 1000ms CBR and 17ms VBR (max AU 1.19MB in both) — this firmware ignores the window for QP decisions entirely. The payload is filler elimination. PUNKTFUNK_VULKAN_RC=cbr|vbr is the field escape hatch and the on-box A/B control (two withdrawn attempts bought that insurance). Also on the same caps struct: maxBitrate is now read and clamps open + retarget (RADV reports 1 Gbps — within 5% of the 4K120 ABR targets), and applied_bitrate_bps() reports the encoder-side truth (pending-first, so the session loop's read right after reconfigure_bitrate sees the clamp) — without it a binding clamp would feed the ABR a phantom base, the trap the trait doc names. And the one-frame VUID-vkCmdBeginVideoCodingKHR-pBeginInfo-08254 violation found in the withdrawal review: record_submit promoted a pending retarget into self.bitrate BEFORE recording whenever first_frame was set, so after a mid-stream reset() (which preserves the pending rate and rc_installed) the begin-coding declaration named a rate the session had not installed — and the two triggers, ABR retarget and the stall watchdog, correlate. Now the declaration always names the session's current rate and the RESET install carries the pending one via its own struct; promotion stays in post_submit_bookkeeping. The extended validation-layer test reproduces the retarget-then-reset coincidence: exactly one 08254 on the pre-fix build, zero on this one (RADV PHOENIX, on glass). Gates: docker amd64 legs green; Windows .173 seven legs (34 passed); .25 full-suite parity vs origin/main (identical CUDA-only failures) + all 9 vulkan on-glass tests + validation layers clean. WP6.3, plus WP7.1's ms-form half (vbv_window_ms). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>