Phase 2 of AMD/Intel support: the VAAPI encoder now takes the capture dmabuf
directly and does the RGB->NV12 colour conversion on the GPU's video engine,
eliminating the host-side de-pad + swscale CSC + upload the CPU path pays.
- capture: a vendor-neutral FramePayload::Dmabuf (dup'd fd + fourcc/modifier/
layout). When zero-copy is on, the EGL->CUDA importer is unavailable (any
non-NVIDIA host), and the backend is VAAPI, the capturer advertises LINEAR
dmabuf and hands the raw buffer to the encoder instead of CPU-copying it.
- encode/vaapi: the encoder self-configures from the first frame's payload (no
open_video signature change). The dmabuf arm wraps the buffer as an
AV_PIX_FMT_DRM_PRIME frame and pushes it through a filter graph
buffer(drm_prime) -> hwmap(vaapi) -> scale_vaapi=nv12 -> buffersink; the
encoder takes NV12 surfaces straight from the sink. The Phase 1 CPU-upload
path is kept as the other arm (used when capture produces CPU frames).
Live-validated on a Radeon 780M (real Sway/xdpw desktop capture): correct,
pixel-perfect HEVC, and ~10x less host CPU at 1440p (4.2s -> 0.4s of CPU for
300 frames) -- the de-pad/CSC/upload moves to the GPU. NVIDIA unchanged
(zero-copy still imports to CUDA; the passthrough path only engages on
non-NVIDIA hosts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
enricobuehler
07c9244eaf