fix(encode): make LTR-RFI loss recovery sound under sustained loss
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m17s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m40s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m6s
apple / screenshots (push) Successful in 6m12s
deb / build-publish (push) Successful in 9m5s
docker / deploy-docs (push) Successful in 24s
deb / build-publish-host (push) Successful in 13m49s
windows-host / package (push) Successful in 16m17s
arch / build-publish (push) Successful in 17m24s
android / android (push) Successful in 12m53s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m31s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m5s
ci / rust (push) Successful in 25m35s

Field report (lid-closed Intel laptop, ~6-19% sustained loss): the stream
never healed — permanent macroblock soup. Three stacked bugs:

- QSV answered RFI with PreferredRefList only, a reorder HINT per the VPL
  spec — the recovery frame could keep predicting from tainted short-term
  refs. Now rejects every other DPB candidate (RejectedRefList) and caps
  L0 at one active entry (AVC/HEVC), matching AMF's hard
  ForceLTRReferenceBitfield / NVENC invalidation semantics.
- Neither QSV nor AMF taint-swept LTR slots across losses: a slot marked
  inside the client's corrupt window became the "known-good" anchor of the
  NEXT loss, propagating corruption through every recovery. Both now drop
  slots at-or-after the loss start before picking an anchor, and guard a
  queued force whose slot the sweep emptied (no false recovery_anchor tag).
- The native plane re-anchored the FULL IDR cooldown on every successful
  RFI, so under sustained loss the client's escalating keyframe requests
  were coalesced away indefinitely (field log: dozens swallowed, one IDR
  per ~8 s). RFI now anchors a 300 ms echo window with a 2-swallow budget
  per loss episode; a client still asking past that gets its IDR.

Live-validated on Arc (qsv feature): 6/6 including the new
qsv_live_ltr_rfi_taint_sweep_declines (a loss covering every live mark
declines the RFI and falls back to IDR recovery).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-19 17:53:48 +02:00
parent ba1caf0281
commit fecbec2daf
3 changed files with 176 additions and 32 deletions
+104 -22
View File
@@ -1017,7 +1017,7 @@ impl Encoder for QsvEncoder {
self.frame_idx += 1;
// --- LTR-RFI per-frame decisions (the AMF policy verbatim; see that module's doc) ---
let mut mark_slot: Option<usize> = None;
let mut force_slot: Option<usize> = None;
let mut force_ltr: Option<(usize, i64)> = None;
let mut recovery_anchor = false;
if self.ltr_active {
if forced {
@@ -1035,10 +1035,16 @@ impl Encoder for QsvEncoder {
);
}
if let Some(slot) = self.pending_force.take() {
force_slot = Some(slot);
recovery_anchor = true;
// Resolve the anchor NOW: a taint sweep in `invalidate_ref_frames` may have
// emptied the slot since the force was queued. An empty slot means there is
// nothing clean to re-reference — the frame must ship as a plain P WITHOUT the
// `recovery_anchor` tag (the client lifts its post-loss freeze on that tag).
if let Some(idx) = self.ltr_slots[slot] {
force_ltr = Some((slot, idx));
recovery_anchor = true;
}
}
if force_slot.is_none() && (forced || cur_idx % self.ltr_mark_interval == 0) {
if force_ltr.is_none() && (forced || cur_idx % self.ltr_mark_interval == 0) {
let slot = self.next_ltr_slot;
self.ltr_slots[slot] = Some(cur_idx);
self.next_ltr_slot = (self.next_ltr_slot + 1) % NUM_LTR_SLOTS;
@@ -1046,6 +1052,7 @@ impl Encoder for QsvEncoder {
}
}
let ltr_slots = self.ltr_slots;
let reject_ok = self.codec != Codec::Av1;
let inner = self.inner.as_mut().expect("ensure_inner succeeded");
// Bound the in-flight window BEFORE submitting: drain finished AUs (buffered for
// `poll`) instead of letting the queue grow under overload.
@@ -1130,7 +1137,7 @@ impl Encoder for QsvEncoder {
(*surf).Data.TimeStamp = captured.pts_ns.wrapping_mul(9) / 100_000; // 90 kHz
// Per-frame control: forced IDR and/or the LTR reflist.
let mut ctrl: Option<Box<FrameCtrl>> = None;
if forced || mark_slot.is_some() || force_slot.is_some() {
if forced || mark_slot.is_some() || force_ltr.is_some() {
let mut c = FrameCtrl::new();
if forced {
c.ctrl.FrameType = (vpl::MFX_FRAMETYPE_IDR
@@ -1148,24 +1155,55 @@ impl Encoder for QsvEncoder {
c.reflist.ApplyLongTermIdx = 1;
use_reflist = true;
}
if let Some(slot) = force_slot {
if let Some(ltr_frame) = ltr_slots[slot] {
// Force THIS frame to predict only from the known-good LTR — the
// clean re-anchor. LongTermIdx stays 0 inside PreferredRefList
// (the AV1 runtime rejects nonzero there; AVC/HEVC key on
// FrameOrder).
c.reflist.PreferredRefList[0].FrameOrder = ltr_frame as u32;
c.reflist.PreferredRefList[0].PicStruct =
vpl::MFX_PICSTRUCT_PROGRESSIVE as u16;
use_reflist = true;
tracing::info!(
slot,
ltr_frame,
frame = cur_idx,
"QSV LTR-RFI: re-referencing known-good LTR (clean recovery, \
no IDR)"
);
if let Some((slot, ltr_frame)) = force_ltr {
// Force THIS frame to predict only from the known-good LTR — the
// clean re-anchor. LongTermIdx stays 0 inside PreferredRefList
// (the AV1 runtime rejects nonzero there; AVC/HEVC key on
// FrameOrder).
c.reflist.PreferredRefList[0].FrameOrder = ltr_frame as u32;
c.reflist.PreferredRefList[0].PicStruct =
vpl::MFX_PICSTRUCT_PROGRESSIVE as u16;
// A preference alone is a reorder HINT (VPL spec) — the encoder may
// still predict from the tainted short-term refs alongside it. AMF's
// ForceLTRReferenceBitfield and NVENC's invalidation are hard
// exclusions; emulate that here by rejecting every other DPB
// candidate — the short-term sliding window (the 2 most recent
// frames) and the other LTR slot — and capping L0 at one active
// entry. Without this the "clean recovery" frame can carry the
// corruption forward, which the client cannot detect (the field
// failure: permanent macroblock soup under sustained loss).
// AVC/HEVC only: the AV1 runtime's universal-reflist rejection path
// is unvalidated, and an unhonored hint there still converges via
// the host's IDR escalation.
if reject_ok {
let mut rej = 0;
let mut reject = |idx: i64| {
if idx >= 0 && idx != ltr_frame {
c.reflist.RejectedRefList[rej].FrameOrder = idx as u32;
c.reflist.RejectedRefList[rej].PicStruct =
vpl::MFX_PICSTRUCT_PROGRESSIVE as u16;
rej += 1;
}
};
reject(cur_idx - 1);
reject(cur_idx - 2);
for (s, marked) in ltr_slots.iter().enumerate() {
if s != slot {
if let Some(idx) = *marked {
reject(idx);
}
}
}
c.reflist.NumRefIdxL0Active = 1;
}
use_reflist = true;
tracing::info!(
slot,
ltr_frame,
frame = cur_idx,
"QSV LTR-RFI: re-referencing known-good LTR (clean recovery, \
no IDR)"
);
}
if use_reflist {
c.attach_reflist();
@@ -1265,6 +1303,17 @@ impl Encoder for QsvEncoder {
if !self.ltr_active || first < 0 || first > last {
return false;
}
// Taint sweep BEFORE picking the anchor: an LTR marked at-or-after the loss start was
// encoded inside the client's corrupt window — the client either never received it or
// decoded it against a broken reference chain. Serving it as "known-good" on a LATER
// loss ships corruption as the recovery anchor, and every subsequent mark re-samples
// the soup — the sustained-loss field failure where the picture never healed. Dropped
// slots stay dropped; the cadence re-marks a clean frame within ~1/4 s.
for marked in self.ltr_slots.iter_mut() {
if marked.is_some_and(|idx| idx >= first) {
*marked = None;
}
}
let mut best: Option<(usize, i64)> = None;
for (slot, marked) in self.ltr_slots.iter().enumerate() {
if let Some(idx) = *marked {
@@ -1287,6 +1336,9 @@ impl Encoder for QsvEncoder {
true
}
None => {
// The sweep may have emptied the slot an earlier (un-consumed) force pointed
// at — clear it so the next submit can't half-apply a stale recovery.
self.pending_force = None;
tracing::info!(
first,
last,
@@ -1845,6 +1897,36 @@ mod tests {
);
}
/// Taint sweep: a loss that predates every live LTR mark leaves NO clean anchor — every
/// slot was marked inside the client's corrupt window. The invalidate must decline (the
/// caller then serves the IDR) and no recovery_anchor AU may ship; before the sweep this
/// force-referenced a tainted mark and shipped corruption tagged as a clean recovery.
#[test]
fn qsv_live_ltr_rfi_taint_sweep_declines() {
let mut rfi_answered = None;
let Some(aus) = drive_live(Codec::H264, false, 60, |enc, i| {
if i == 30 && enc.caps().supports_rfi {
// Frame 0 lost: the IDR itself — every mark (0, 15, ...) is at-or-after it.
rfi_answered = Some(enc.invalidate_ref_frames(0, 2));
}
}) else {
return;
};
assert_stream_shape(&aus, 60, true);
let Some(answered) = rfi_answered else {
eprintln!("note: driver declined LTR (supports_rfi=false) — sweep not exercised");
return;
};
assert!(
!answered,
"a loss covering every live LTR mark must fall back to IDR recovery"
);
assert!(
!aus.iter().any(|a| a.recovery_anchor),
"no recovery_anchor AU may ship when the sweep left no clean LTR"
);
}
/// No-IDR bitrate retarget — Phase 3 on-glass: `reconfigure_bitrate` mid-stream must be
/// accepted (HRD off + StartNewSequence=OFF) and must not emit a keyframe.
#[test]