fix(client): the bottom rows stop smearing — the decode pool is taller than the picture
ci / web (push) Successful in 1m49s
ci / rust-arm64 (push) Successful in 2m19s
ci / docs-site (push) Successful in 2m29s
ci / rust (push) Successful in 7m3s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 19s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 8s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
deb / build-publish-client-arm64 (push) Successful in 2m28s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 21s
deb / build-publish (push) Successful in 4m17s
deb / build-publish-host (push) Successful in 4m51s
docker / builders-arm64cross (push) Successful in 6s
docker / deploy-docs (push) Successful in 36s
android / android (push) Successful in 8m24s
arch / build-publish (push) Successful in 8m28s
flatpak / build-publish (push) Successful in 6m8s
apple / swift (push) Successful in 5m54s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 13m5s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 3m10s
apple / screenshots (push) Canceled after 7m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 16m25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 15m57s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 5m3s

A user's 1080p stream repeated its last row of pixels over the final few rows, so the
image looked stretched at the bottom.

The Vulkan-Video CSC pass sampled the decoded planes with the fullscreen triangle's
normalized 0..1 UVs, but its render target is built at the CROPPED frame size. Those are
not the same rectangle: FFmpeg sizes the decode pool from `avctx->coded_*`, and H.264
codes `16 * mb_height` — so a 1080-row picture decodes into a 1088-row pool. Destination
row 1079 sampled source row ~1087.5, dragging the 8 alignment rows into view and squashing
the picture 0.7%. Encoders fill that padding by replicating the last picture line, which
is why it reads as a smeared bottom row rather than garbage.

Confirmed on glass (.173, RTX, H.264 1080p, vulkan-video):
  Vulkan Video first frame width=1920 height=1080 pool_w=1920 pool_h=1088

`VkVideoFrame` now carries the pool extent and `record_csc` takes a `uv_scale`, written to
the shader's `params.zw` — which the CSC shader already reserved for a use like this. The
chroma cositing offset is unchanged and stays correct: `textureSize` reports the pool
width, which is the space the scaled UV is already in.

Only the Vulkan-Video path passes a scale below 1.0. D3D11VA already clamps this in its
VideoProcessor blit (the same bug, seen as a green bar there because DXVA padding is
uninitialized rather than replicated); dmabuf imports its planes at the crop over the real
stride; PyroWave allocates its ring at exact stream dims. Apple and Android crop at the OS
layer. Every other call site passes [1.0, 1.0], so the change is inert there.

Also adds a one-time first-frame layout log mirroring the D3D11VA one, so the frame-vs-pool
gap is visible in the field instead of having to be re-derived from FFmpeg internals.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 23:40:33 +02:00
co-authored by Claude Opus 5
parent 9f72a3b6ad
commit 6af067da2d
5 changed files with 89 additions and 4 deletions
+15
View File
@@ -110,6 +110,21 @@ pub struct VkVideoFrame {
pub decode_done_value: u64,
pub width: u32,
pub height: u32,
/// The decode POOL's allocated extent (`AVHWFramesContext.width`/`.height`) — the
/// CODED picture size (rounded up to the codec's macroblock alignment, then to the
/// driver's Vulkan picture-access granularity), so it is `>=` `width`/`height`. At
/// 1080p the pool is 1088 rows tall: 1080 is not a multiple of 16.
///
/// The presenter samples this image with NORMALIZED coordinates, so it needs both
/// numbers — `width`/`height` is what to display, `coded_*` is what the texture
/// actually spans. Sampling `0..1` without the ratio stretches the alignment padding
/// into view; because encoders fill those rows by replicating the picture's last
/// line, that reads as the bottom row smeared over the final few rows of the image
/// (field report 2026-07-31). Same class as the D3D11VA source-rect clamp in
/// `crate::video_d3d11`, which shows as a green bar there only because DXVA padding
/// is left uninitialized rather than replicated.
pub coded_width: u32,
pub coded_height: u32,
pub color: ColorDesc,
/// Intra keyframe (IDR/I): the stream's re-anchor point. The pump resumes display on
/// one after suppressing the concealed frames a reference loss leaves in its wake (on
+38
View File
@@ -382,6 +382,13 @@ impl VulkanDecoder {
// sem_value was last written by the decode submission on THIS thread.
let timeline_sem = (*vkf).sem[0] as u64;
let decode_done_value = (*vkf).sem_value[0];
log_layout_once(
(*self.frame).width,
(*self.frame).height,
(*fc).width,
(*fc).height,
sw,
);
Ok(VkVideoFrame {
vkframe: vkf as usize,
frames_ctx: fc as usize,
@@ -392,6 +399,13 @@ impl VulkanDecoder {
decode_done_value,
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
// The pool extent, not the frame's: `avcodec_get_hw_frames_parameters`
// sizes it from `coded_width`/`coded_height` and FFmpeg's Vulkan layer
// rounds that up again to the driver's picture-access granularity. The
// `max` is defensive — a pool SMALLER than the frame would mean sampling
// past the surface, so degrade to "no crop" rather than trust it.
coded_width: ((*fc).width.max((*self.frame).width)) as u32,
coded_height: ((*fc).height.max((*self.frame).height)) as u32,
color: ColorDesc::from_raw(self.frame),
keyframe: frame_is_keyframe(self.frame),
guard: DrmFrameGuard(clone),
@@ -400,6 +414,30 @@ impl VulkanDecoder {
}
}
/// One-time dump of the first decoded frame's layout — the forensics for a new GPU/driver.
/// `pool_*` is the allocated decode surface (`>=` the frame); the gap is the alignment
/// padding the presenter's UV scale excludes. The D3D11VA path logs the same pair.
fn log_layout_once(
width: i32,
height: i32,
pool_w: i32,
pool_h: i32,
sw: ffmpeg::ffi::AVPixelFormat,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
tracing::info!(
width,
height,
pool_w,
pool_h,
?sw,
"Vulkan Video first frame"
);
}
}
impl Drop for VulkanDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
+16 -3
View File
@@ -15,6 +15,14 @@
// reference white, BT.2020→709 primaries, a soft maxRGB rolloff for highlights
// (BT.2390-flavored simplicity, not libplacebo), then sRGB encode.
// params.y = tonemap peak (display-relative, ~= peak_nits / 203).
// params.zw = the crop→surface UV scale (frame size / decode-pool size). A Vulkan-Video
// pool image is the CODED surface, taller than the picture whenever the height is
// not a multiple of the driver's alignment (1080 → 1088); sampling the full 0..1
// would drag those padding rows into view — and since encoders fill them by
// replicating the last picture line, that reads as the bottom row smeared over the
// final few rows. 1.0/1.0 for every path whose image is already crop-sized (dmabuf
// imports the planes at the crop over the real stride; D3D11VA clamps in its
// VideoProcessor blit).
//
// Regenerate: shaders/build.sh (committed .spv, no build-time toolchain).
#version 450
@@ -29,7 +37,7 @@ layout(push_constant) uniform Csc {
vec4 r0;
vec4 r1;
vec4 r2;
vec4 params; // x: mode, y: tonemap peak, z/w: reserved
vec4 params; // x: mode, y: tonemap peak, zw: crop/pool UV scale
} pc;
// SMPTE ST.2084 (PQ) EOTF: code value → display-referred linear, normalized to 1.0 =
@@ -62,17 +70,22 @@ vec3 srgb_oetf(vec3 c) {
}
void main() {
// Crop to the visible picture: the triangle spans the whole render target, so its 0..1
// maps onto the pool surface only after this scale (see params.zw above).
vec2 uv = v_uv * pc.params.zw;
// 4:2:0 chroma is left-cosited (H.273 type 0 — the default inference when unsignaled, and
// what the hosts produce), but sampling the half-res plane at the luma UV assumes CENTER
// siting — a ~0.5-luma-px rightward chroma shift on hard colored edges. Offset +0.25 chroma
// texels to re-align (the same correction the Apple/Windows clients apply). Self-disables
// when the plane widths match (a full-size 4:4:4 chroma plane needs no correction).
vec2 cuv = v_uv;
// textureSize is the POOL's chroma width, which is the space `uv` is already in — so the
// offset stays a true quarter-texel whatever the crop.
vec2 cuv = uv;
int cw = textureSize(u_c, 0).x;
if (cw < textureSize(u_y, 0).x) {
cuv.x += 0.25 / float(cw);
}
vec3 yuv = vec3(texture(u_y, v_uv).r, texture(u_c, cuv).rg);
vec3 yuv = vec3(texture(u_y, uv).r, texture(u_c, cuv).rg);
vec3 rgb = vec3(
dot(pc.r0.xyz, yuv) + pc.r0.w,
dot(pc.r1.xyz, yuv) + pc.r1.w,
Binary file not shown.
+20 -1
View File
@@ -248,9 +248,12 @@ impl Presenter {
height: v.height,
};
let ten_bit = f.is_p010();
// No crop: `dmabuf::import` already creates the plane images at the frame
// size over the surface's real stride, so 0..1 spans exactly the picture.
self.record_csc(
v.framebuffer,
extent,
[1.0, 1.0],
f.color,
if ten_bit { 10 } else { 8 },
ten_bit,
@@ -322,9 +325,17 @@ impl Presenter {
};
let ten_bit =
f.vk_format == vk::Format::G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16.as_raw();
// The one path that samples a surface BIGGER than the picture: FFmpeg's
// pool is the coded size (1080 → 1088 rows). Scale the UVs to the visible
// crop or the alignment padding — the last picture row, replicated by the
// encoder — is stretched into the bottom of the image.
self.record_csc(
v.framebuffer,
extent,
[
f.width as f32 / f.coded_width as f32,
f.height as f32 / f.coded_height as f32,
],
f.color,
if ten_bit { 10 } else { 8 },
ten_bit,
@@ -625,7 +636,11 @@ impl Presenter {
/// Record the NV12→RGBA CSC pass into the video image (framebuffer): fullscreen
/// triangle, CICP-driven push-constant rows. Shared by the dmabuf and Vulkan-Video
/// paths — only the plane views bound beforehand differ.
/// paths — only the plane views bound beforehand, and `uv_scale`, differ.
///
/// `extent` is the picture (the framebuffer's own size); `uv_scale` is picture/surface
/// per axis, i.e. `[1.0, 1.0]` unless the bound planes are a decode pool allocated
/// larger than the picture. See the shader's `params.zw` for why that happens.
///
/// # Safety
/// `self.cmd_buf` must be in the recording state; the CSC descriptor set must point
@@ -634,6 +649,7 @@ impl Presenter {
&self,
framebuffer: vk::Framebuffer,
extent: vk::Extent2D,
uv_scale: [f32; 2],
color: pf_client_core::video::ColorDesc,
depth: u8,
msb_packed: bool,
@@ -702,6 +718,9 @@ impl Presenter {
pc[..12].copy_from_slice(bytemuck_rows(&rows));
pc[12] = mode;
pc[13] = peak;
// Crop: 1.0 unless the source image is a decode pool bigger than the picture.
pc[14] = uv_scale[0];
pc[15] = uv_scale[1];
let bytes = std::slice::from_raw_parts(pc.as_ptr().cast::<u8>(), 64);
self.device.cmd_push_constants(
self.cmd_buf,