feat(video): Vulkan Video decode on the presenter's device (NVIDIA hw decode)

FFmpeg's Vulkan Video decoder now runs on the PRESENTER's own VkDevice —
the decoded VkImage feeds the existing CICP CSC pass directly: zero
copy, no interop, and NVIDIA gets hardware decode for the first time
(its VAAPI is unusable by design). One decode architecture for every
vendor going forward; VAAPI-dmabuf and software remain the fallbacks
(auto: vulkan → vaapi → software; PUNKTFUNK_DECODER=vulkan pins it).

Presenter: instance 1.3; probes VK_KHR_video_queue/decode_queue + codec
extensions, a VIDEO_DECODE queue family (+ its codec caps via
QueueFamilyVideoPropertiesKHR), and the samplerYcbcrConversion/
timelineSemaphore/synchronization2 features — all enabled at device
creation when present, exported as a VulkanDecodeDevice handle bundle.

Decoder: AVVulkanDeviceContext built over those handles via pf-ffvk
(features chain, extension lists, deprecated queue indices + the qf[]
map); get_format supplies OUR frames context with MUTABLE_FORMAT so the
presenter's per-plane views are legal; output is DecodedImage::VkFrame
carrying AVVkFrame/frames-ctx pointers plus the lock fns.

Present: R8+R8G8 plane views over the multiplanar image, the live sync
state read under the AVVulkanFramesContext lock, a timeline-semaphore
wait(sem_value)/signal(sem_value+1) folded into the submit, layout/
queue-family/sem_value written back per FFmpeg's contract, and the frame
guard parked in the retire queue until the fence. CSC pass + video
framebuffer are now unconditional (NVIDIA has no dmabuf-import path).

Verified on the RTX 5070 Ti: device creates with decode_qf=3,
caps=DECODE_H264|H265|AV1|VP9; swapchain unaffected. Live stream
validation next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-07 21:46:34 +02:00
parent 58ccd530fc
commit c78ddc40cb
10 changed files with 1087 additions and 191 deletions
+7
View File
@@ -50,6 +50,13 @@ fn main() {
.allowlist_type("AVVkFrame.*")
.allowlist_function("av_vk_frame_alloc")
.allowlist_function("av_vkfmt_from_pixfmt")
// The feature structs chained into AVVulkanDeviceContext.device_features (plain
// vulkan.h types; generating them here keeps the chain in one type system).
.allowlist_type("VkPhysicalDeviceVulkan11Features")
.allowlist_type("VkPhysicalDeviceVulkan12Features")
.allowlist_type("VkPhysicalDeviceVulkan13Features")
// AVVulkanFramesContext.img_flags values (plane views need MUTABLE_FORMAT).
.allowlist_type("VkImageCreateFlagBits")
// …plus nothing else of FFmpeg: the core types these structs reference only
// ever appear behind pointers here, so keep them opaque instead of duplicating
// ffmpeg-sys-next's definitions (callers cast pointers between the crates).