feat(client): request a recovery keyframe on unrecoverable loss

Under infinite GOP the punktfunk/1 plane has no periodic IDR — the only recovery
keyframe is one the client requests. But the reassembler drops unrecoverable AUs
silently (frames_dropped) and hands the decoder reference-missing delta frames
that libavcodec conceals and returns Ok for, so keying recovery off a decode
error mostly never fires under real loss → a long/permanent freeze.

Surface the data-plane pump's Session.frames_dropped to NativeClient via a shared
atomic (NativeClient::frames_dropped()), updated every pump iteration so it stays
current through a total-loss drought. The Linux and Windows client video loops
watch it and call request_keyframe() when it climbs, throttled to 100 ms (the
decode stays wedged for several frames until the IDR lands). macOS already does
this; client-rs doesn't decode.

Resolves reliability backlog #2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-16 11:04:42 +00:00
parent 5f57be9012
commit 93fb745731
3 changed files with 62 additions and 1 deletions
@@ -157,6 +157,9 @@ fn pump(
let mut decode_us_sum = 0u64;
let mut lat_us: Vec<u64> = Vec::with_capacity(256);
let mut pcm = vec![0f32; 5760 * 2]; // decode scratch: max Opus frame (120 ms stereo)
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
let end: Option<String> = loop {
if stop.load(Ordering::SeqCst) {
@@ -197,6 +200,22 @@ fn pump(
Err(e) => break Some(format!("session: {e:?}")),
}
// Loss recovery: under infinite GOP the only recovery keyframe is one we request. The
// reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the
// reference-missing delta frames that follow and returns Ok, so keying off a decode error
// rarely fires. Request an IDR when the drop count climbs, throttled — the decode stays
// wedged for several frames until the IDR lands, so requesting every frame would flood.
let dropped = connector.frames_dropped();
if dropped > last_dropped {
last_dropped = dropped;
let now = Instant::now();
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
last_kf_req = Some(now);
let _ = connector.request_keyframe();
tracing::debug!(dropped, "requested keyframe (loss recovery)");
}
}
// Drain audio between frames (packets land every 5 ms; the queue holds 320 ms).
while let Ok(pkt) = connector.next_audio(Duration::ZERO) {
if let (Some(player), Some(dec)) = (&player, opus_dec.as_mut()) {