fix(clients): shared-VkQueue race + swapchain recreate destroy-in-use — the intermittent device-lost stream killer
apple / swift (push) Successful in 1m7s
ci / web (push) Successful in 1m9s
ci / docs-site (push) Successful in 1m18s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m41s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m43s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m11s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m34s
ci / bench (push) Successful in 6m12s
apple / screenshots (push) Successful in 5m18s
docker / deploy-docs (push) Successful in 27s
flatpak / build-publish (push) Failing after 5m25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m50s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m35s
arch / build-publish (push) Successful in 19m55s
deb / build-publish (push) Successful in 18m59s
ci / rust (push) Successful in 20m9s
android / android (push) Successful in 20m11s

Live-diagnosed on the RTX box during the adaptive-bitrate A/B: 2 of 3 streams died
with VK_ERROR_DEVICE_LOST at stream start or at a mid-stream encoder rebuild, then
zombied — the demote-to-software path rebuilt the decoder against the same dead
device, FFmpeg wedged inside the rebuild, and the pump flushed a never-draining
backlog every 2 s forever. No OS TDR: the client's own Vulkan misuse. Two causes:

1. The presenter creates ONE graphics-family queue and hands FFmpeg's
   AVVulkanDeviceContext the same family (nb_graphics_queues=1 ⇒ queue 0) for its
   transfer/compute prep — so the pump thread and the presenter thread submitted to
   the SAME VkQueue with no shared lock, violating vkQueueSubmit's external-sync
   rule exactly when FFmpeg puts work on the graphics queue (decoder open /
   frames-context rebuild = stream start and every ABR encoder re-target). New
   guard-less QueueLock (FFmpeg's lock_queue/unlock_queue callbacks are a raw pair)
   shared by all four queue users: FFmpeg (callbacks installed via user_opaque),
   the presenter's submit/present/wait-idle, and the Skia overlay's flushes.

2. Swapchain recreation destroyed the old swapchain + render semaphores after ONE
   fence cycle — the fence proves our submit, not the presentation engine's
   semaphore consumption (VUID-vkDestroySemaphore-05149 +
   VUID-vkDestroySwapchainKHR-01282 on every recreate). Recreate now drains the
   queue (vkQueueWaitIdle under the shared lock — safe now that FFmpeg honours it)
   and destroys immediately; the deferred DisplayGarbage machinery is gone.

Resilience: VK_ERROR_DEVICE_LOST anywhere in a present error chain is now fatal —
the run loop fails the session loudly instead of demoting to software on a dead
device (the zombie path).

Verified on the RTX box (RTX 4090 → host .21) under VK_LAYER_KHRONOS_validation:
3/3 stream start/stop cycles clean, then 8 mid-stream encoder rebuilds in one
session (4 ABR down-steps under 10% induced loss + 4 clean-link recovery up-steps
— the exact scenarios that previously killed the device): 0 device losses, 0
wedges, both recreate VUIDs gone (previously fired on every path). Remaining
validation messages are FFmpeg's own video-session VUIDs, untouched by this
change. Linux: clippy -D warnings + tests green (home-worker-2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-09 12:42:11 +02:00
parent 4839c0e6f6
commit f4c3a5d0c3
5 changed files with 265 additions and 90 deletions
+28 -1
View File
@@ -235,6 +235,18 @@ impl StreamState {
}
}
/// Whether a present error is `VK_ERROR_DEVICE_LOST` anywhere in its chain. A lost
/// device is unrecoverable by spec — every object on it (decoder frames, swapchain,
/// the Skia context) is dead, and the demote-to-software path would rebuild the
/// decoder against that same dead device (observed live 2026-07-09: FFmpeg wedges
/// inside the rebuild, the decode thread never returns, and the client zombies with
/// the pump flushing a never-draining backlog every 2 s). The only correct response
/// is to fail the session loudly and let the shell relaunch.
fn device_lost(e: &anyhow::Error) -> bool {
e.chain()
.any(|c| c.downcast_ref::<ash::vk::Result>() == Some(&ash::vk::Result::ERROR_DEVICE_LOST))
}
fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>> {
// Before any window exists: unpackaged runs adopt the shell's AppUserModelID so the
// shell⇄session windows group as one taskbar app (win32.rs; MSIX identity wins).
@@ -789,8 +801,13 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// Import/CSC failure is survivable (the stream continues on
// the next frame) — but a streak means this box can't do the
// hw path: demote the decoder to software, same contract as
// the GTK presenter's GL-converter failures.
// the GTK presenter's GL-converter failures. A lost DEVICE
// is not survivable and must not demote — see [`device_lost`].
Err(e) => {
if device_lost(&e) {
return Err(e)
.context("GPU device lost — the session cannot continue");
}
st.hw_fails += 1;
tracing::warn!(error = %format!("{e:#}"), fails = st.hw_fails,
"hardware present failed");
@@ -832,6 +849,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
p
}
Err(e) => {
// Lost device ⇒ unrecoverable, never demote ([`device_lost`]).
if device_lost(&e) {
return Err(e)
.context("GPU device lost — the session cannot continue");
}
st.hw_fails += 1;
tracing::warn!(error = %format!("{e:#}"), fails = st.hw_fails,
"hardware present failed");
@@ -873,6 +895,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
p
}
Err(e) => {
// Lost device ⇒ unrecoverable, never demote ([`device_lost`]).
if device_lost(&e) {
return Err(e)
.context("GPU device lost — the session cannot continue");
}
st.hw_fails += 1;
tracing::warn!(error = %format!("{e:#}"), fails = st.hw_fails,
"vulkan-video present failed");