fix+perf(encode): clamp Vulkan CSC to source edge + cache dmabuf imports

Two refinements after the initial on-glass validation on RADV (780M):

- Green padding bar at non-16-aligned heights (e.g. 1080 → coded 1088): the CSC
  compute shader read past the edge of the shorter source dmabuf for the 8
  alignment-padding rows, producing undefined/green garbage that showed on a
  client rendering the coded frame. Clamp every source fetch to `textureSize-1`
  so padding rows duplicate the last real row (invisible, and the SPS
  conformance window still crops it for a compliant decoder). BT.709 conversion
  is byte-identical for in-bounds pixels. 5120x1440 (exactly aligned) was never
  affected.

- Per-frame dmabuf import churn: the backend created + imported + destroyed a
  VkImage every frame (allocation jitter → stutter). PipeWire cycles a small
  fixed pool, so import each underlying buffer ONCE (keyed by st_dev/st_ino —
  each frame's fd is a fresh dup of the same buffer) and reuse it, matching the
  CUDA-path VkBridge. First import acquires from the foreign producer; cached
  re-reads keep queue ownership and use a plain visibility barrier. On-glass:
  ~3-6 imports per session then silent (was ~one per frame at 240 Hz), stutter
  gone at resolutions with headroom.

Also adds a PF_SMOKE_W/H override to the headless smoke test to exercise the
conformance-window crop path (ffprobe confirms coded 1088 → displayed 1080).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-12 19:24:12 +02:00
parent 5ab6daa694
commit bbbb7f5723
3 changed files with 82 additions and 19 deletions
@@ -7,15 +7,20 @@ layout(binding = 2, rg8) uniform writeonly image2D uvImg; // half-res UV (inte
float lumaY(vec3 c) { return 16.0/255.0 + 0.1826*c.r + 0.6142*c.g + 0.0620*c.b; }
// Source may be SMALLER than the coded (16-aligned) Y plane — e.g. 1080 source vs 1088 coded. Clamp
// every fetch to the source edge so the alignment-padding rows duplicate the last real row instead
// of reading out of bounds (undefined → green garbage that shows if a client ignores the SPS
// conformance-window crop). `textureSize` gives the bound source's real extent.
void main() {
ivec2 sz = imageSize(yImg);
ivec2 rmax = textureSize(rgb, 0) - 1;
ivec2 uvc = ivec2(gl_GlobalInvocationID.xy);
ivec2 p = uvc * 2;
if (p.x >= sz.x || p.y >= sz.y) return;
vec3 c00 = texelFetch(rgb, p, 0).rgb;
vec3 c10 = texelFetch(rgb, p + ivec2(1, 0), 0).rgb;
vec3 c01 = texelFetch(rgb, p + ivec2(0, 1), 0).rgb;
vec3 c11 = texelFetch(rgb, p + ivec2(1, 1), 0).rgb;
vec3 c00 = texelFetch(rgb, min(p, rmax), 0).rgb;
vec3 c10 = texelFetch(rgb, min(p + ivec2(1, 0), rmax), 0).rgb;
vec3 c01 = texelFetch(rgb, min(p + ivec2(0, 1), rmax), 0).rgb;
vec3 c11 = texelFetch(rgb, min(p + ivec2(1, 1), rmax), 0).rgb;
imageStore(yImg, p, vec4(lumaY(c00), 0, 0, 1));
imageStore(yImg, p + ivec2(1, 0), vec4(lumaY(c10), 0, 0, 1));
imageStore(yImg, p + ivec2(0, 1), vec4(lumaY(c01), 0, 0, 1));