fix(host): encode-stall watchdog — heal the silent AMF/QSV freeze in place

Field reports: Windows AMD/Intel streams freeze after ~3-5 min regardless of
desktop activity. Root cause: the libavcodec AMF/QSV poll is non-blocking
(EAGAIN -> Ok(None)), and the encode loop's drain treated None as benign
without popping `inflight` — a wedged driver (QueryOutput stops producing)
meant frames kept being submitted, inflight grew unboundedly, no AU ever
reached the send thread, and nothing logged: a silent permanent freeze. The
input-side twin: once libavcodec's one-frame buffer fills, avcodec_send_frame
EAGAINs and the submit `?` killed the whole session.

Add `Encoder::reset()` (in-place encoder rebuild; implemented for AMF/QSV by
dropping the wedged libavcodec encoder so the next submit re-opens it on the
current device, forced IDR) and an encode-stall watchdog in the stream loop:
trip on a poll error, on no AU within max(2 s, 8 frame intervals) while frames
are owed, or on an owed backlog worth more than the window's frames (the
slow-leak latency-runaway form). Recovery is a bounded (5 consecutive, cleared
by any delivered AU) in-place rebuild + forced IDR — a logged ~one-second
hiccup instead of a dead stream; exhaustion or a reset-less backend still
fails the session with a clear error. Submit failures route through the same
bounded recovery. The three existing pipeline-rebuild paths (session switch,
mode switch, capture loss) now also clear the stale in-flight records that
pointed at the dropped encoder.

Backends whose poll blocks (direct NVENC sync, software) can't false-trip:
they never return Ok(None) mid-stream and drain inflight below depth each
tick. Validated: clippy -D warnings (nvenc,amf-qsv), 191 host tests, synthetic
E2E 300/300 frames, and an on-glass AMD iGPU session (1080p120 HDR hevc_amf).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-06 14:36:29 +02:00
parent 622c8bf701
commit 167d590349
3 changed files with 141 additions and 4 deletions
+12
View File
@@ -194,6 +194,15 @@ pub trait Encoder: Send {
}
/// Pull the next encoded AU if one is ready.
fn poll(&mut self) -> Result<Option<EncodedFrame>>;
/// Tear the underlying hardware encoder down and rebuild it in place, keeping the session's
/// negotiated parameters — the encode-stall watchdog's recovery lever (a wedged AMF/QSV
/// driver stops emitting AUs or accepting frames without ever returning an error). Returns
/// `true` when the encoder was rebuilt: every submitted-but-unpolled frame is forfeited and
/// the next submitted frame starts a fresh stream (IDR). Default `false`: the backend has no
/// in-place rebuild and the caller must treat the stall as fatal instead.
fn reset(&mut self) -> bool {
false
}
/// Signal end-of-stream. After this, drain the remaining AUs with [`poll`](Self::poll)
/// until it returns `None` — NVENC buffers frames internally even at `delay=0`.
fn flush(&mut self) -> Result<()>;
@@ -370,6 +379,9 @@ impl Encoder for TrackedEncoder {
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
self.inner.poll()
}
fn reset(&mut self) -> bool {
self.inner.reset()
}
fn flush(&mut self) -> Result<()> {
self.inner.flush()
}