fix(encode/vaapi): tell libva the dmabuf's real size, not zero

The zero-copy DRM-PRIME descriptor declared `objects[0].size = 0`, on
the belief that ffmpeg would work the real size out. It does not: both
of its import paths pass the value straight through to libva —
`prime_desc.objects[i].size` on the PRIME_2 path, `buffer_desc.data_size`
on the legacy fallback it tries next — so every VA driver we have ever
handed a dmabuf to was told the backing object was empty and left to
derive the size itself.

The drivers this path has run on (radeonsi, modern Intel iHD) derive it
correctly, which is why nobody noticed. A Gen9 Intel host does not get
that far: `vaCreateSurfaces` answers VA_STATUS_ERROR_ALLOCATION_FAILED
on the first frame and every frame after it, `av_buffersink_get_frame`
returns EIO, and five in-place encoder rebuilds later the video session
is over. That host cannot stream at all with zero-copy on.

`lseek(SEEK_END)` is the standard dma-buf size query and the same one
this tree's Vulkan bridge already performs on these very fds. A kernel
that refuses it leaves the old 0 rather than costing a frame that might
still have encoded.

Whether this alone fixes that host is unconfirmed — the descriptor was
wrong either way, and it is the first thing the driver reads.
This commit is contained in:
2026-07-25 15:13:11 +02:00
parent 7c82a72ecd
commit f32207a6ba
+13 -2
View File
@@ -949,7 +949,8 @@ impl DmabufInner {
// `Box` puts it on the heap with a unique owner. // `Box` puts it on the heap with a unique owner.
// * `dmabuf.fd.as_raw_fd()` is the fd of the caller's `&DmabufFrame`, which owns it for the // * `dmabuf.fd.as_raw_fd()` is the fd of the caller's `&DmabufFrame`, which owns it for the
// whole synchronous `submit`; we describe one object/layer/plane from its // whole synchronous `submit`; we describe one object/layer/plane from its
// fourcc/modifier/offset/stride and pass `object.size = 0` (ffmpeg queries the real size). // fourcc/modifier/offset/stride and its `lseek`-queried size. `libc::lseek` on that live
// fd only reads the description's size and returns it (or -1); it touches no Rust memory.
// * `av_frame_alloc` → `drm` (null-checked); we set its scalar fields and // * `av_frame_alloc` → `drm` (null-checked); we set its scalar fields and
// `hw_frames_ctx = av_buffer_ref(self.drm_frames)` (new ref of the live owned ctx). // `hw_frames_ctx = av_buffer_ref(self.drm_frames)` (new ref of the live owned ctx).
// * `data[0] = Box::into_raw(desc)` transfers the box into the frame; `buf[0] = // * `data[0] = Box::into_raw(desc)` transfers the box into the frame; `buf[0] =
@@ -965,7 +966,17 @@ impl DmabufInner {
let mut desc: Box<ffi::AVDRMFrameDescriptor> = Box::new(std::mem::zeroed()); let mut desc: Box<ffi::AVDRMFrameDescriptor> = Box::new(std::mem::zeroed());
desc.nb_objects = 1; desc.nb_objects = 1;
desc.objects[0].fd = dmabuf.fd.as_raw_fd(); desc.objects[0].fd = dmabuf.fd.as_raw_fd();
desc.objects[0].size = 0; // The object's REAL size, not 0. libav does not query it for us — both of its import
// paths hand this value straight to libva, as `prime_desc.objects[i].size` on the
// PRIME_2 path and `buffer_desc.data_size` on the legacy fallback — so a 0 told every
// VA driver the backing object was empty and left it to work the real size out itself.
// The drivers this has run on (radeonsi, modern Intel iHD) do; a Gen9 Intel host
// answered `vaCreateSurfaces` with VA_STATUS_ERROR_ALLOCATION_FAILED on every single
// frame. `lseek(SEEK_END)` is the standard dma-buf size query — the same one the
// Vulkan bridge already uses on these fds (`pf_zerocopy::imp::vulkan`). If a kernel
// refuses it, keep the old 0 rather than drop a frame we could still have encoded.
let obj_size = libc::lseek(dmabuf.fd.as_raw_fd(), 0, libc::SEEK_END);
desc.objects[0].size = if obj_size > 0 { obj_size as _ } else { 0 };
desc.objects[0].format_modifier = dmabuf.modifier; desc.objects[0].format_modifier = dmabuf.modifier;
desc.nb_layers = 1; desc.nb_layers = 1;
desc.layers[0].format = self.fourcc; desc.layers[0].format = self.fourcc;