unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 3 Wiki Activity

fix(client-linux): VAAPI green screen on AMD — flatten NV12 planes across DRM layers

Browse Source 
First AMD test (Steam Deck, Mesa radeonsi) showed a mostly-green image
with red whites — the classic fingerprint of NV12 chroma read as 0.

Root cause (confirmed against FFmpeg/GTK/mpv source): FFmpeg's VAAPI
export uses VA_EXPORT_SURFACE_SEPARATE_LAYERS unconditionally, so an
NV12 surface comes back as TWO single-plane layers — layers[0]=R8
(luma), layers[1]=GR88 (chroma) — sharing one object/fd, the UV plane
reached via offset. map_dmabuf took layers[0] only and used its format
(R8) as the GTK fourcc, so GdkDmabufTexture got a luma-only texture
with the chroma plane dropped → chroma defaults to 0 → green field,
red highlights.

Fix (matches mpv's dmabuf_interop_gl flatten pattern):
- Derive the combined fourcc from the decoder's sw_format
  (AVHWFramesContext.sw_format → NV12 → DRM_FORMAT_NV12), NOT from the
  per-plane component formats. The frame format is absent from the
  separate-layer descriptor and must be deduced from sw_format.
- Flatten every plane across every layer in declared order (Y then UV),
  each with its own fd (objects[plane.object_index].fd), offset, pitch.
- One-time descriptor dump (objects/layers/formats/modifier) so a new
  driver's real layout is visible in the logs.
- Unit test locks the DRM FourCC magic numbers (NV12=0x3231564e).

Software decode (swscale, reads colorspace from the VUI) was always
correct, which isolated the bug to this path. PUNKTFUNK_DECODER=software
is the immediate workaround on an un-rebuilt binary. Awaiting Steam Deck
reconfirm (no AMD VAAPI on the NVIDIA dev box to live-verify).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-06-17 07:24:27 +00:00
parent ad0cb1b582
commit 64b167946f
2 changed files with 118 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+8 -2
View File
@@ -108,8 +108,14 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
default, saved-hosts list, .deb + RPM-subpackage CI (deb.yml/rpm.yml). **VAAPI decode
→ DRM-PRIME dmabuf → `GdkDmabufTexture`** (BT.709 color state; Tier-1 zero-copy on
Intel/AMD, `PUNKTFUNK_DECODER=software|vaapi` override) with a proven fallback ladder —
no VAAPI device (NVIDIA) or mid-session VAAPI error → software decode; needs an
Intel/AMD client box to live-verify the hw path. Next: the stage-2 raw-Wayland
no VAAPI device (NVIDIA) or mid-session VAAPI error → software decode. **First AMD test
(Steam Deck) hit a green-screen bug, fixed:** FFmpeg's VAAPI export uses
`SEPARATE_LAYERS`, so NV12 arrives as two single-plane layers (R8 luma + GR88 chroma,
one shared fd); the mapper took `layers[0]` only → GTK got a luma-only R8 texture, chroma
read as 0 → green field / red whites. Fix derives the combined fourcc from the decoder
`sw_format` (→ `DRM_FORMAT_NV12`) and flattens all planes across all layers (mpv's
pattern); a first-frame descriptor dump logs the real layout. Awaiting Steam Deck
reconfirm. Next: the stage-2 raw-Wayland
presenter (wp_presentation feedback, tearing-control, Vulkan Video on NVIDIA) —
**wgpu/winit rejected** (no dmabuf import / presentation feedback / shortcuts-inhibit).
**Windows stage 1 done 2026-06-15** (`crates/punktfunk-client-windows`, binary