feat(client): freeze-until-reanchor loss recovery on Android + Apple via shared core gate
After unrecoverable loss the host keeps sending delta frames that reference a picture the client never received; hardware decoders conceal these as gray/ garbage with a success status. Linux already withheld them and held the last good frame until a proven clean re-anchor — this brings that behavior to the Android and Apple clients. Extract the Linux pump's freeze state machine into a shared `ReanchorGate` in punktfunk-core (reanchor.rs, 18 tests) exposed over the C ABI (ABI v6, additive — no wire change) for the Swift clients. Migrate the Linux/Deck pump (pf-client-core) onto it as the parity proof (no-op refactor). Then wire: - Android (decode.rs, both sync + async loops): arm on the frame-index gap, a pts-keyed flag map carries the wire flags to the output-buffer release, fold the gate per drained output, gate.poll replaces the dropped-climb block. - Apple Stage2Pipeline (default): arm on a gap (new noteFrameIndexGap), withhold at the ring-submit seam (CAMetalLayer holds its last drawable), poll framesDropped, fold VT decode errors through the no-output streak. - Apple StreamPump (stage-1): fold at enqueue, withhold via kCMSampleAttachmentKey_DoNotDisplay so the layer keeps decoding (reference chain intact) but holds the last displayed frame. - Apple VideoDecoder: thread the AU's wire flags to the async decode callback via a retained FrameContext refcon (replaces the receivedNs bit-pattern scalar). Lifts only on a proven re-anchor (IDR / RFI anchor / 2nd recovery mark) with a 500 ms backstop so a lost re-anchor can never freeze forever. Apple: swift build clean, 123/123 tests pass (incl. VideoToolboxRoundTripTests). On-glass loss-injection validation still owed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -10,6 +10,7 @@ use crate::audio;
|
||||
use crate::video::{DecodedFrame, DecodedImage, Decoder};
|
||||
use punktfunk_core::client::NativeClient;
|
||||
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
|
||||
use punktfunk_core::reanchor::{index_gap, GateVerdict, ReanchorGate};
|
||||
use punktfunk_core::PunktfunkError;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
@@ -99,86 +100,6 @@ pub struct Stats {
|
||||
pub decoder: &'static str,
|
||||
}
|
||||
|
||||
/// Consecutive no-output AUs that force a keyframe request. ~50 ms at 60 Hz — long
|
||||
/// enough not to fire on a one-frame decoder hiccup, short enough that a lost initial
|
||||
/// IDR (or a mid-GOP join) unfreezes almost immediately instead of never.
|
||||
const NO_OUTPUT_KEYFRAME_STREAK: u32 = 3;
|
||||
|
||||
/// Longest the pump holds the last good frame waiting for a post-loss re-anchor keyframe before it
|
||||
/// gives up and resumes display. After a reference loss the hardware decoder does not error — it
|
||||
/// conceals the reference-missing deltas (on RADV, the DPB-and-output-COINCIDE path renders them as
|
||||
/// a gray plate with the new frame's motion painted over it) and returns Ok, so displaying them is
|
||||
/// the "gray frames mid-stream" artifact. We instead freeze on the last good picture until a fresh
|
||||
/// IDR re-anchors decode — the behaviour NVIDIA already shows (its DISTINCT output image + different
|
||||
/// concealment reads as a brief freeze, not gray). This cap only bounds the freeze when recovery
|
||||
/// genuinely stalls (host ignores the request, or an RFI recovery that never emits a keyframe), so a
|
||||
/// glitch can never become a permanent freeze. A recovery IDR round-trips well under this on any
|
||||
/// live link.
|
||||
const REANCHOR_FREEZE_MAX: Duration = Duration::from_millis(500);
|
||||
|
||||
/// How many host intra-refresh recovery marks ([`USER_FLAG_RECOVERY_POINT`]) must arrive since the
|
||||
/// latest frame gap before the pump lifts its freeze on an IDR-free stream. TWO, not one: with a
|
||||
/// continuous rolling wave the host marks phase-fixed wave boundaries, so the FIRST boundary after a
|
||||
/// loss is only partially healed — stripes swept BEFORE the loss still reference the lost frame — and
|
||||
/// lifting there would flash a partially-stale picture. The SECOND boundary guarantees a full wave
|
||||
/// swept entirely after the loss, so the picture is clean. This stays correct under repeated loss
|
||||
/// because every new gap resets the count. The cost is up to ~2 wave periods of holding the last good
|
||||
/// frame — the deliberate "hold longer, never show garbage" trade.
|
||||
///
|
||||
/// [`USER_FLAG_RECOVERY_POINT`]: punktfunk_core::packet::USER_FLAG_RECOVERY_POINT
|
||||
const REANCHOR_MARKS_TO_LIFT: u32 = 2;
|
||||
|
||||
/// Backstop patience while a host intra-refresh heal is visibly in progress. Each recovery mark
|
||||
/// pushes the freeze deadline out by this much, so a live mark stream (the host actively healing via
|
||||
/// its wave) keeps the client patiently holding the last good frame instead of tripping the IDR
|
||||
/// floor mid-heal. Must exceed the inter-mark interval (one wave period, ~0.5 s) with margin; if the
|
||||
/// marks STOP (heal stalled, or the host isn't running intra-refresh) the deadline lapses and the
|
||||
/// normal recovery-IDR floor fires, so a real stall still recovers.
|
||||
const RECOVERY_MARK_PATIENCE: Duration = Duration::from_millis(1500);
|
||||
|
||||
/// Frames skipped when `got` arrives while `expected` was the next index, or `None` if `got` is
|
||||
/// contiguous (`== expected`) or a straggler we have already passed. Frame indices are u32 counters
|
||||
/// that wrap, so the "ahead" test is a wrapping subtraction split at the half-space: a small
|
||||
/// positive delta is a forward gap (missing frames whose dependents will decode against absent
|
||||
/// references); a delta in the top half is an index behind us.
|
||||
fn index_gap(expected: u32, got: u32) -> Option<u32> {
|
||||
let ahead = got.wrapping_sub(expected);
|
||||
(ahead != 0 && ahead < u32::MAX / 2).then_some(ahead)
|
||||
}
|
||||
|
||||
/// Fold one decoded frame into the re-anchor state and decide whether it lifts the post-loss freeze.
|
||||
///
|
||||
/// `is_keyframe` — a real IDR (always a clean re-anchor). `has_anchor` — this AU carried
|
||||
/// [`USER_FLAG_RECOVERY_ANCHOR`](punktfunk_core::packet::USER_FLAG_RECOVERY_ANCHOR), the host's
|
||||
/// definitive single-frame re-anchor from an LTR-RFI recovery (a clean P-frame coded against a
|
||||
/// known-good reference), so it lifts on the FIRST occurrence exactly like an IDR — no two-mark wait.
|
||||
/// `has_mark` — this AU carried [`USER_FLAG_RECOVERY_POINT`](punktfunk_core::packet::USER_FLAG_RECOVERY_POINT),
|
||||
/// a host-signalled intra-refresh wave boundary (only *half* a re-anchor). `marks` — recovery marks
|
||||
/// seen since the latest gap.
|
||||
///
|
||||
/// Returns `(lift, new_marks)`: `lift` clears the freeze; `new_marks` is the running count (reset to 0
|
||||
/// on a lift). The two-mark rule ([`REANCHOR_MARKS_TO_LIFT`]) lives here so it is unit-tested
|
||||
/// independent of the pump's channel/decoder plumbing — the first wave boundary after a loss is only
|
||||
/// partially healed, so a single mark must NOT lift. An anchor (or IDR) is a *whole* re-anchor and
|
||||
/// lifts immediately.
|
||||
fn reanchor_after_frame(
|
||||
is_keyframe: bool,
|
||||
has_anchor: bool,
|
||||
has_mark: bool,
|
||||
marks: u32,
|
||||
) -> (bool, u32) {
|
||||
let marks = if has_mark {
|
||||
marks.saturating_add(1)
|
||||
} else {
|
||||
marks
|
||||
};
|
||||
if is_keyframe || has_anchor || marks >= REANCHOR_MARKS_TO_LIFT {
|
||||
(true, 0)
|
||||
} else {
|
||||
(false, marks)
|
||||
}
|
||||
}
|
||||
|
||||
/// Frames the pump keeps waiting for their 0xCF host timing (pts → capture→received µs).
|
||||
/// ~2 s at 120 Hz — a timing arrives within a frame or two of its AU, and against an old
|
||||
/// host (no 0xCF at all) this just caps the dead-weight ring.
|
||||
@@ -382,27 +303,17 @@ fn pump(
|
||||
// What actually decoded the last frame — a VAAPI failure demotes mid-session, so
|
||||
// this is read off each frame's image variant rather than fixed at startup.
|
||||
let mut dec_path: &'static str = "";
|
||||
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
|
||||
let mut last_dropped = connector.frames_dropped();
|
||||
// The stats window keeps its own drop cursor — the OSD shows the per-window delta.
|
||||
let mut window_dropped = last_dropped;
|
||||
let mut window_dropped = connector.frames_dropped();
|
||||
let mut last_kf_req: Option<Instant> = None;
|
||||
// Consecutive received AUs that produced NO decoded frame (decode error, or the
|
||||
// decoder swallowed a reference-missing delta and returned nothing). Distinct from
|
||||
// `frames_dropped`, which counts reassembler drops: when the initial IDR is lost (or
|
||||
// we join mid-GOP) the reassembler delivers complete-but-undecodable deltas — it
|
||||
// never drops, so the drop-count trigger below stays silent and the stream freezes
|
||||
// on the last good frame. A short streak forces a fresh IDR to re-anchor.
|
||||
let mut no_output_streak = 0u32;
|
||||
// Freeze-until-reanchor: armed the moment we request a recovery keyframe (loss, decode error, or
|
||||
// a no-output streak), it withholds the decoder's concealed frames from the presenter — which
|
||||
// then redraws the last good picture — until a fresh keyframe re-anchors decode. See
|
||||
// [`REANCHOR_FREEZE_MAX`] for why this exists and its backstop deadline.
|
||||
let mut awaiting_reanchor = false;
|
||||
let mut reanchor_deadline: Option<Instant> = None;
|
||||
// Host intra-refresh recovery marks seen since the latest gap (see [`REANCHOR_MARKS_TO_LIFT`]).
|
||||
// Reset to 0 whenever the freeze is (re-)armed, so a fresh loss always waits out two fresh marks.
|
||||
let mut recovery_marks: u32 = 0;
|
||||
// Freeze-until-reanchor: the shared post-loss gate ([`punktfunk_core::reanchor::ReanchorGate`]).
|
||||
// Armed on any loss signal (frame-index gap, dropped-count climb, decoder wedge/demotion), it
|
||||
// withholds the decoder's concealed frames from the presenter — which then redraws the last good
|
||||
// picture — until a proven clean re-anchor (IDR / RFI anchor / second recovery mark) lifts it. It
|
||||
// also owns the no-output streak and the overdue-freeze backstop; the client keeps its own
|
||||
// `last_kf_req` request throttle and routes the gate's keyframe intents through it. Seeded with the
|
||||
// current drop count so the first `poll` doesn't read the baseline as a loss.
|
||||
let mut gate = ReanchorGate::new(connector.frames_dropped());
|
||||
// The frame_index we expect next (the host numbers frames consecutively). A jump means a frame
|
||||
// went missing — the earliest, most reliable signal that the decoder is about to conceal, ~120 ms
|
||||
// ahead of `frames_dropped` (the reassembler only declares a straggler lost once it ages out of
|
||||
@@ -447,9 +358,7 @@ fn pump(
|
||||
Some(exp) => {
|
||||
if let Some(gap) = index_gap(exp, frame.frame_index) {
|
||||
let now = Instant::now();
|
||||
awaiting_reanchor = true;
|
||||
recovery_marks = 0;
|
||||
reanchor_deadline = Some(now + REANCHOR_FREEZE_MAX);
|
||||
gate.arm(now);
|
||||
next_expected_index = Some(frame.frame_index.wrapping_add(1));
|
||||
// The gap carries the PRECISE lost range — [first missing, newest
|
||||
// received - 1] — so this is the one recovery signal that can drive true
|
||||
@@ -488,38 +397,14 @@ fn pump(
|
||||
}
|
||||
match decoder.decode(&frame.data) {
|
||||
Ok(Some(image)) => {
|
||||
// A decoded frame — the anchor holds.
|
||||
no_output_streak = 0;
|
||||
// Host-signalled intra-refresh recovery mark: on an IDR-free intra-refresh
|
||||
// stream this wave-boundary flag is the only clean point the client can honor
|
||||
// (the decoder never flags the re-anchor — the coded frame stays `P`). A live
|
||||
// mark stream also means the host is actively healing, so push the backstop out
|
||||
// rather than trip a mid-heal IDR (see `RECOVERY_MARK_PATIENCE`).
|
||||
let has_mark =
|
||||
frame.flags & punktfunk_core::packet::USER_FLAG_RECOVERY_POINT != 0;
|
||||
// The host's definitive single-frame re-anchor: an LTR-RFI recovery frame (a
|
||||
// clean P-frame off a known-good reference), the AMD twin of an IDR re-anchor
|
||||
// but without the spike. It lifts on the FIRST occurrence.
|
||||
let has_anchor =
|
||||
frame.flags & punktfunk_core::packet::USER_FLAG_RECOVERY_ANCHOR != 0;
|
||||
if has_mark && awaiting_reanchor {
|
||||
reanchor_deadline = Some(Instant::now() + RECOVERY_MARK_PATIENCE);
|
||||
}
|
||||
// A fresh clean re-anchor lifts the freeze and shows this frame: a real intra
|
||||
// keyframe (IDR, always clean), an LTR-RFI recovery anchor (also whole), OR the
|
||||
// second recovery mark since the gap (the first wave boundary is only
|
||||
// half-healed — see `reanchor_after_frame`).
|
||||
let (lift, marks) = reanchor_after_frame(
|
||||
image.is_keyframe(),
|
||||
has_anchor,
|
||||
has_mark,
|
||||
recovery_marks,
|
||||
);
|
||||
recovery_marks = marks;
|
||||
if lift {
|
||||
awaiting_reanchor = false;
|
||||
reanchor_deadline = None;
|
||||
}
|
||||
// Fold this decoded frame through the shared freeze gate: it reads the AU's
|
||||
// re-anchor wire flags (FLAG_SOF IDR marker / RECOVERY_ANCHOR / RECOVERY_POINT),
|
||||
// takes `image.is_keyframe()` as the ffmpeg keyframe belt, applies the two-mark
|
||||
// rule + the mark-patience backstop, clears the no-output streak, and returns
|
||||
// whether to present this frame or withhold it as a post-loss concealment.
|
||||
let present = gate
|
||||
.on_decoded(frame.flags, image.is_keyframe(), Instant::now())
|
||||
== GateVerdict::Present;
|
||||
total_frames += 1;
|
||||
dec_path = match &image {
|
||||
DecodedImage::Cpu(_) => "software",
|
||||
@@ -574,19 +459,19 @@ fn pump(
|
||||
DecodedImage::VkFrame(v) => Some((v.timeline_sem, v.decode_done_value)),
|
||||
_ => None,
|
||||
};
|
||||
if awaiting_reanchor {
|
||||
// Post-loss concealment: withhold this frame (it references a lost/gray
|
||||
// reference) so the presenter keeps redrawing the last good picture
|
||||
// rather than flashing the decoder's gray plate. Dropped here — the
|
||||
// hw-decode stat below still samples via `hw_fence` (raw handle + value,
|
||||
// valid past the guard). Cleared by the next keyframe or the backstop.
|
||||
tracing::trace!("holding last frame — awaiting post-loss re-anchor");
|
||||
} else {
|
||||
if present {
|
||||
let _ = frame_tx.force_send(DecodedFrame {
|
||||
pts_ns: frame.pts_ns,
|
||||
decoded_ns,
|
||||
image,
|
||||
});
|
||||
} else {
|
||||
// Post-loss concealment: withhold this frame (it references a lost/gray
|
||||
// reference) so the presenter keeps redrawing the last good picture rather
|
||||
// than flashing the decoder's gray plate. Dropped here — the hw-decode stat
|
||||
// below still samples via `hw_fence` (raw handle + value, valid past the
|
||||
// guard). The gate lifts the freeze on the next clean re-anchor / backstop.
|
||||
tracing::trace!("holding last frame — awaiting post-loss re-anchor");
|
||||
}
|
||||
// `decode` stage: received→decode COMPLETE, single clock.
|
||||
match hw_fence {
|
||||
@@ -602,36 +487,35 @@ fn pump(
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(None) => no_output_streak += 1,
|
||||
// The decoder produced nothing — under zero-reorder LOW_DELAY (one-in/one-out) that
|
||||
// means it's wedged on missing references with no reassembler drop to trigger
|
||||
// recovery. The gate counts the streak and, once it trips, arms the freeze and tells
|
||||
// us to (throttled) request a fresh IDR to re-anchor. Both the empty-output and the
|
||||
// survivable-decode-error arms feed it; a decoded frame resets the streak in
|
||||
// `on_decoded`.
|
||||
Ok(None) => {
|
||||
let now = Instant::now();
|
||||
if gate.on_no_output(now)
|
||||
&& last_kf_req
|
||||
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
|
||||
{
|
||||
last_kf_req = Some(now);
|
||||
let _ = connector.request_keyframe();
|
||||
tracing::debug!("requested keyframe (decoder produced no output)");
|
||||
}
|
||||
}
|
||||
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
|
||||
Err(e) => {
|
||||
no_output_streak += 1;
|
||||
tracing::debug!(error = %e, "decode error (recovering)");
|
||||
}
|
||||
}
|
||||
// The decoder has produced nothing for a short run — under zero-reorder
|
||||
// LOW_DELAY (one-in/one-out) that means it's wedged on missing references
|
||||
// with no reassembler drop to trigger recovery below. Ask for a fresh IDR
|
||||
// (throttled), then re-arm the streak so we wait out the request→IDR round
|
||||
// trip before asking again instead of flooding.
|
||||
if no_output_streak >= NO_OUTPUT_KEYFRAME_STREAK {
|
||||
let now = Instant::now();
|
||||
// Wedged on missing references: hold the last good frame until re-anchor
|
||||
// (armed even when the IDR request itself is throttled — the stream is broken
|
||||
// regardless of whether we ask again this iteration).
|
||||
awaiting_reanchor = true;
|
||||
recovery_marks = 0;
|
||||
reanchor_deadline = Some(now + REANCHOR_FREEZE_MAX);
|
||||
if last_kf_req
|
||||
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
|
||||
{
|
||||
last_kf_req = Some(now);
|
||||
let _ = connector.request_keyframe();
|
||||
tracing::debug!(
|
||||
streak = no_output_streak,
|
||||
"requested keyframe (decoder produced no output)"
|
||||
);
|
||||
no_output_streak = 0;
|
||||
let now = Instant::now();
|
||||
if gate.on_no_output(now)
|
||||
&& last_kf_req
|
||||
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
|
||||
{
|
||||
last_kf_req = Some(now);
|
||||
let _ = connector.request_keyframe();
|
||||
tracing::debug!("requested keyframe (decode error recovery)");
|
||||
}
|
||||
}
|
||||
}
|
||||
// The presenter's verdict: hardware frames can't be displayed (GL converter
|
||||
@@ -649,9 +533,7 @@ fn pump(
|
||||
// through the same throttle as loss recovery below.
|
||||
if decoder.take_keyframe_request() {
|
||||
let now = Instant::now();
|
||||
awaiting_reanchor = true;
|
||||
recovery_marks = 0;
|
||||
reanchor_deadline = Some(now + REANCHOR_FREEZE_MAX);
|
||||
gate.arm(now);
|
||||
if last_kf_req
|
||||
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
|
||||
{
|
||||
@@ -679,41 +561,23 @@ fn pump(
|
||||
}
|
||||
}
|
||||
|
||||
// Loss recovery: under infinite GOP the only recovery keyframe is one we request. The
|
||||
// reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the
|
||||
// reference-missing delta frames that follow and returns Ok, so keying off a decode error
|
||||
// rarely fires. Request an IDR when the drop count climbs, throttled — the decode stays
|
||||
// wedged for several frames until the IDR lands, so requesting every frame would flood.
|
||||
// Loss recovery + overdue backstop, folded through the shared gate. A climb in the
|
||||
// reassembler's unrecoverable-drop count (`frames_dropped`) means the AUs after the lost one
|
||||
// reference a picture we never decoded — the decoder conceals them (gray on RADV) and returns
|
||||
// Ok, so a decode-error trigger rarely fires; the gate arms the freeze on the climb instead. An
|
||||
// overdue freeze (held a full REANCHOR_FREEZE_MAX with no clean re-anchor — a lost recovery IDR,
|
||||
// or a benign reorder that produced no `frames_dropped`) re-asks while it keeps holding: NEVER
|
||||
// resume to gray — a genuinely dead stream is the QUIC idle-timeout watchdog's job. Both route
|
||||
// the gate's keyframe intent through the shared 100 ms throttle; under infinite GOP the only
|
||||
// recovery keyframe is one we request.
|
||||
let dropped = connector.frames_dropped();
|
||||
if dropped > last_dropped {
|
||||
last_dropped = dropped;
|
||||
let now = Instant::now();
|
||||
// A dropped AU means the frames after it reference a picture we never decoded — the
|
||||
// decoder will conceal them (gray on RADV). Freeze on the last good frame until a fresh
|
||||
// IDR re-anchors, so the concealment never reaches the screen.
|
||||
awaiting_reanchor = true;
|
||||
recovery_marks = 0;
|
||||
reanchor_deadline = Some(now + REANCHOR_FREEZE_MAX);
|
||||
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
|
||||
last_kf_req = Some(now);
|
||||
let _ = connector.request_keyframe();
|
||||
tracing::debug!(dropped, "requested keyframe (loss recovery)");
|
||||
}
|
||||
}
|
||||
// Re-anchor overdue: the freeze has held the whole window with no keyframe — a lost recovery
|
||||
// IDR, or a benign reorder that produced no `frames_dropped` and so requested none. Do NOT
|
||||
// resume to gray (the one thing worse than a freeze): keep holding the last good frame and
|
||||
// (re-)request a keyframe, throttled + host-coalesced, so a CLEAN re-anchor is what un-freezes
|
||||
// us. A genuinely dead stream — host gone, link collapsed — is caught by the QUIC idle-timeout
|
||||
// watchdog (returns to the menu), never by painting the decoder's concealment.
|
||||
if awaiting_reanchor && reanchor_deadline.is_some_and(|d| Instant::now() >= d) {
|
||||
let now = Instant::now();
|
||||
reanchor_deadline = Some(now + REANCHOR_FREEZE_MAX);
|
||||
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
|
||||
last_kf_req = Some(now);
|
||||
let _ = connector.request_keyframe();
|
||||
tracing::debug!("re-anchor overdue — still holding, re-requesting keyframe");
|
||||
}
|
||||
let now = Instant::now();
|
||||
if gate.poll(dropped, now)
|
||||
&& last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
|
||||
{
|
||||
last_kf_req = Some(now);
|
||||
let _ = connector.request_keyframe();
|
||||
tracing::debug!(dropped, "requested keyframe (loss recovery / overdue re-anchor)");
|
||||
}
|
||||
|
||||
if window_start.elapsed() >= Duration::from_secs(1) {
|
||||
@@ -836,111 +700,3 @@ fn spawn_audio(
|
||||
.map_err(|e| tracing::warn!(error = %e, "audio thread failed to start — audio disabled"))
|
||||
.ok()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{index_gap, reanchor_after_frame, REANCHOR_MARKS_TO_LIFT};
|
||||
|
||||
// Simulate the pump's re-anchor state across a sequence of decoded frames: each `(is_keyframe,
|
||||
// has_mark)` pair is folded through `reanchor_after_frame`, returning the frame index (0-based)
|
||||
// at which the freeze first lifts, or `None` if it never does. `gap_before` reset points model a
|
||||
// fresh loss re-arming the freeze (the pump zeroes the count at every gap/arm site).
|
||||
fn lift_at(frames: &[(bool, bool)]) -> Option<usize> {
|
||||
let mut marks = 0u32;
|
||||
for (i, &(is_kf, has_mark)) in frames.iter().enumerate() {
|
||||
// The intra-refresh-mark model never carries an LTR-RFI anchor (that path is exercised
|
||||
// by `an_rfi_anchor_lifts_immediately`), so `has_anchor` is always false here.
|
||||
let (lift, m) = reanchor_after_frame(is_kf, false, has_mark, marks);
|
||||
marks = m;
|
||||
if lift {
|
||||
return Some(i);
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_single_recovery_mark_does_not_lift() {
|
||||
// The first wave boundary after a loss is only half-healed — one mark must hold the freeze.
|
||||
assert_eq!(REANCHOR_MARKS_TO_LIFT, 2);
|
||||
assert_eq!(lift_at(&[(false, true)]), None);
|
||||
assert_eq!(
|
||||
lift_at(&[(false, false), (false, true), (false, false)]),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_second_recovery_mark_lifts() {
|
||||
// Two marks = a full wave swept after the loss → clean re-anchor.
|
||||
assert_eq!(lift_at(&[(false, true), (false, true)]), Some(1));
|
||||
assert_eq!(
|
||||
lift_at(&[(false, false), (false, true), (false, false), (false, true)]),
|
||||
Some(3)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_real_keyframe_lifts_immediately() {
|
||||
// An IDR is always a clean anchor — no marks needed.
|
||||
assert_eq!(lift_at(&[(true, false)]), Some(0));
|
||||
assert_eq!(lift_at(&[(false, true), (true, false)]), Some(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fresh_gap_resets_the_mark_count() {
|
||||
// The pump zeroes `recovery_marks` at each arm site, so one mark before a new gap plus one
|
||||
// after must NOT lift — the model resets the running count to imitate that.
|
||||
let mut marks = 0u32;
|
||||
let (_, m) = reanchor_after_frame(false, false, true, marks); // mark #1 (pre-gap)
|
||||
marks = m;
|
||||
assert_eq!(marks, 1);
|
||||
marks = 0; // a new gap re-arms the freeze → count reset
|
||||
let (lift, m) = reanchor_after_frame(false, false, true, marks); // first mark of the new wave
|
||||
assert!(!lift, "a single post-gap mark must not lift");
|
||||
assert_eq!(m, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_rfi_anchor_lifts_immediately() {
|
||||
// An LTR-RFI recovery anchor is a WHOLE re-anchor (a clean P-frame off a known-good
|
||||
// reference), so — like an IDR — it lifts on the FIRST occurrence, no two-mark wait.
|
||||
let (lift, marks) = reanchor_after_frame(false, true, false, 0);
|
||||
assert!(lift, "an RFI anchor must lift the freeze immediately");
|
||||
assert_eq!(marks, 0, "a lift resets the running mark count");
|
||||
// Even with zero prior marks and no keyframe, the anchor alone is sufficient.
|
||||
let (lift, _) = reanchor_after_frame(false, true, true, 1);
|
||||
assert!(lift, "an anchor lifts regardless of the pending mark count");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn contiguous_indices_are_not_a_gap() {
|
||||
assert_eq!(index_gap(5, 5), None);
|
||||
assert_eq!(index_gap(0, 0), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_forward_jump_reports_the_skip_count() {
|
||||
assert_eq!(index_gap(5, 6), Some(1)); // one frame missing
|
||||
assert_eq!(index_gap(5, 9), Some(4));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_straggler_behind_us_is_not_a_gap() {
|
||||
// The reassembler emitted a newer frame first; the late one must not re-arm.
|
||||
assert_eq!(index_gap(9, 5), None);
|
||||
assert_eq!(index_gap(1, 0), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_index_counter_wraps_cleanly() {
|
||||
// last frame = u32::MAX, so the next expected wraps to 0.
|
||||
// Contiguous across the wrap.
|
||||
assert_eq!(index_gap(0, 0), None);
|
||||
// waiting on u32::MAX, frame 0 arrived → MAX was skipped.
|
||||
assert_eq!(index_gap(u32::MAX, 0), Some(1));
|
||||
assert_eq!(index_gap(u32::MAX, 2), Some(3));
|
||||
// an old frame arriving just after the wrap is still a straggler.
|
||||
assert_eq!(index_gap(0, u32::MAX), None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
|
||||
use crate::error::PunktfunkStatus;
|
||||
use crate::input::InputEvent;
|
||||
use crate::reanchor::{GateVerdict, ReanchorGate};
|
||||
use crate::session::Session;
|
||||
use crate::stats::Stats;
|
||||
use crate::transport::{loopback_pair, Transport, UdpTransport};
|
||||
@@ -2620,3 +2621,142 @@ pub unsafe extern "C" fn punktfunk_connection_close(c: *mut PunktfunkConnection)
|
||||
drop(unsafe { Box::from_raw(c) });
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Post-loss re-anchor freeze gate ----
|
||||
//
|
||||
// The shared [`ReanchorGate`](crate::reanchor::ReanchorGate) exposed for the Swift client (Rust
|
||||
// embedders — Android/Windows/Linux — use the struct directly). After an unrecoverable reference
|
||||
// loss the decoder silently conceals the missing-reference deltas (gray/garbage picture, no error);
|
||||
// the client freezes on the last good frame and lifts only on a proven clean re-anchor. The gate
|
||||
// takes time internally (`Instant::now`) so no timestamps cross the boundary. Drive it per session:
|
||||
// `arm` on a loss (frame-index gap from `punktfunk_connection_note_frame_index`, a decoder
|
||||
// wedge/demotion), `on_decoded` per decoded frame to gate presentation, `on_no_output` per AU that
|
||||
// produced nothing, and `poll` each iteration for the dropped-count climb + overdue backstop. Route
|
||||
// the returned keyframe intents through the client's existing request throttle.
|
||||
|
||||
/// Create a re-anchor gate seeded with the session's current `frames_dropped` (so the first
|
||||
/// [`punktfunk_reanchor_gate_poll`] doesn't read the baseline as a loss). Free with
|
||||
/// [`punktfunk_reanchor_gate_free`]. Never returns NULL.
|
||||
#[no_mangle]
|
||||
pub extern "C" fn punktfunk_reanchor_gate_new(frames_dropped: u64) -> *mut ReanchorGate {
|
||||
Box::into_raw(Box::new(ReanchorGate::new(frames_dropped)))
|
||||
}
|
||||
|
||||
/// Free a gate created by [`punktfunk_reanchor_gate_new`]. NULL is a no-op.
|
||||
///
|
||||
/// # Safety
|
||||
/// `g` was returned by [`punktfunk_reanchor_gate_new`] and is not used after this call.
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_reanchor_gate_free(g: *mut ReanchorGate) {
|
||||
if !g.is_null() {
|
||||
drop(unsafe { Box::from_raw(g) });
|
||||
}
|
||||
}
|
||||
|
||||
/// Arm the freeze: a loss was detected (a frame-index gap, or a decoder wedge/demotion). Zeroes the
|
||||
/// recovery-mark count and (re-)sets the backstop deadline. NULL is a no-op.
|
||||
///
|
||||
/// # Safety
|
||||
/// `g` is a valid gate handle.
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_reanchor_gate_arm(g: *mut ReanchorGate) {
|
||||
if let Some(g) = unsafe { g.as_mut() } {
|
||||
g.arm(std::time::Instant::now());
|
||||
}
|
||||
}
|
||||
|
||||
/// Fold one decoded frame and write to `out_present` whether to display it (`true`) or withhold it as
|
||||
/// a post-loss concealment (`false`). `flags` is the AU's `user_flags` word ([`PunktfunkFrame::flags`]):
|
||||
/// the gate reads `FLAG_SOF` (the host's IDR marker), `USER_FLAG_RECOVERY_ANCHOR` and
|
||||
/// `USER_FLAG_RECOVERY_POINT`. Pass `decoder_keyframe = false` where the platform decoder doesn't flag
|
||||
/// IDRs (VideoToolbox/MediaCodec) — the wire `FLAG_SOF` covers it.
|
||||
///
|
||||
/// # Safety
|
||||
/// `g` is a valid gate handle; `out_present` is writable or NULL.
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_reanchor_gate_on_decoded(
|
||||
g: *mut ReanchorGate,
|
||||
flags: u32,
|
||||
decoder_keyframe: bool,
|
||||
out_present: *mut bool,
|
||||
) -> PunktfunkStatus {
|
||||
guard(|| {
|
||||
let g = match unsafe { g.as_mut() } {
|
||||
Some(g) => g,
|
||||
None => return PunktfunkStatus::NullPointer,
|
||||
};
|
||||
let present = g.on_decoded(flags, decoder_keyframe, std::time::Instant::now()) == GateVerdict::Present;
|
||||
if !out_present.is_null() {
|
||||
unsafe { *out_present = present };
|
||||
}
|
||||
PunktfunkStatus::Ok
|
||||
})
|
||||
}
|
||||
|
||||
/// A received AU produced no decoded frame. Writes to `out_request_kf` whether the no-output streak has
|
||||
/// tripped and the client should (throttled) request a keyframe — the gate arms the freeze at the same
|
||||
/// time.
|
||||
///
|
||||
/// # Safety
|
||||
/// `g` is a valid gate handle; `out_request_kf` is writable or NULL.
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_reanchor_gate_on_no_output(
|
||||
g: *mut ReanchorGate,
|
||||
out_request_kf: *mut bool,
|
||||
) -> PunktfunkStatus {
|
||||
guard(|| {
|
||||
let g = match unsafe { g.as_mut() } {
|
||||
Some(g) => g,
|
||||
None => return PunktfunkStatus::NullPointer,
|
||||
};
|
||||
let request = g.on_no_output(std::time::Instant::now());
|
||||
if !out_request_kf.is_null() {
|
||||
unsafe { *out_request_kf = request };
|
||||
}
|
||||
PunktfunkStatus::Ok
|
||||
})
|
||||
}
|
||||
|
||||
/// Periodic fold of the session's `frames_dropped` counter plus the overdue backstop. Writes to
|
||||
/// `out_request_kf` whether the client should (throttled) request a keyframe (a drop-count climb armed
|
||||
/// a fresh freeze, or the freeze is overdue and re-asks while it keeps holding).
|
||||
///
|
||||
/// # Safety
|
||||
/// `g` is a valid gate handle; `out_request_kf` is writable or NULL.
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_reanchor_gate_poll(
|
||||
g: *mut ReanchorGate,
|
||||
frames_dropped: u64,
|
||||
out_request_kf: *mut bool,
|
||||
) -> PunktfunkStatus {
|
||||
guard(|| {
|
||||
let g = match unsafe { g.as_mut() } {
|
||||
Some(g) => g,
|
||||
None => return PunktfunkStatus::NullPointer,
|
||||
};
|
||||
let request = g.poll(frames_dropped, std::time::Instant::now());
|
||||
if !out_request_kf.is_null() {
|
||||
unsafe { *out_request_kf = request };
|
||||
}
|
||||
PunktfunkStatus::Ok
|
||||
})
|
||||
}
|
||||
|
||||
/// Whether the gate is currently withholding concealed frames (frozen on the last good picture).
|
||||
/// Writes `false` on a NULL gate.
|
||||
///
|
||||
/// # Safety
|
||||
/// `g` is a valid gate handle; `out_holding` is writable or NULL.
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_reanchor_gate_is_holding(
|
||||
g: *const ReanchorGate,
|
||||
out_holding: *mut bool,
|
||||
) -> PunktfunkStatus {
|
||||
guard(|| {
|
||||
let holding = unsafe { g.as_ref() }.is_some_and(ReanchorGate::is_holding);
|
||||
if !out_holding.is_null() {
|
||||
unsafe { *out_holding = holding };
|
||||
}
|
||||
PunktfunkStatus::Ok
|
||||
})
|
||||
}
|
||||
|
||||
@@ -38,6 +38,7 @@ pub mod input;
|
||||
pub mod packet;
|
||||
#[cfg(feature = "quic")]
|
||||
pub mod quic;
|
||||
pub mod reanchor;
|
||||
pub mod session;
|
||||
pub mod stats;
|
||||
pub mod transport;
|
||||
@@ -61,7 +62,10 @@ pub use stats::Stats;
|
||||
/// TTL of a v2 envelope; `punktfunk_connection_next_rumble` is unchanged and drops it). Additive —
|
||||
/// the wire is backward-compatible (the envelope is a length-tolerant tail on 0xCA), so
|
||||
/// [`WIRE_VERSION`] is unchanged.
|
||||
pub const ABI_VERSION: u32 = 5;
|
||||
/// v6: added the `punktfunk_reanchor_gate_*` surface (post-loss freeze-until-reanchor gate for the
|
||||
/// Swift client; Rust embedders use [`reanchor::ReanchorGate`] directly). Additive, client-local —
|
||||
/// no wire change, so [`WIRE_VERSION`] is unchanged.
|
||||
pub const ABI_VERSION: u32 = 6;
|
||||
|
||||
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
|
||||
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
|
||||
|
||||
@@ -0,0 +1,456 @@
|
||||
//! Post-loss display freeze — the shared "freeze-until-reanchor" gate.
|
||||
//!
|
||||
//! After an unrecoverable reference loss the hardware decoder does **not** error: it *conceals* the
|
||||
//! reference-missing delta frames (on RADV, the DPB-and-output-COINCIDE path paints a gray plate with
|
||||
//! the new frame's motion on top) and returns Ok. Displaying that is the "gray frames mid-stream"
|
||||
//! artifact. Instead every client freezes on the last good picture — withholds the concealed frames
|
||||
//! from its presenter, which keeps redrawing the held frame — and lifts the freeze ONLY on a proven
|
||||
//! clean re-anchor: a real IDR, an LTR-RFI recovery anchor ([`USER_FLAG_RECOVERY_ANCHOR`]), or the
|
||||
//! second intra-refresh recovery mark ([`USER_FLAG_RECOVERY_POINT`]) since the loss.
|
||||
//!
|
||||
//! This module owns that decision so every embedder shares ONE implementation instead of re-deriving
|
||||
//! it (the Linux/Deck pump in `pf-client-core`, the Windows in-process pump, the Android decode loops,
|
||||
//! and — over the C ABI — the Apple client). The state machine is time-driven but takes `now` as a
|
||||
//! parameter so it is unit-testable without a clock; the C ABI wrappers supply `Instant::now()`.
|
||||
//!
|
||||
//! [`USER_FLAG_RECOVERY_POINT`]: crate::packet::USER_FLAG_RECOVERY_POINT
|
||||
//! [`USER_FLAG_RECOVERY_ANCHOR`]: crate::packet::USER_FLAG_RECOVERY_ANCHOR
|
||||
|
||||
use crate::packet::{FLAG_SOF, USER_FLAG_RECOVERY_ANCHOR, USER_FLAG_RECOVERY_POINT};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
/// Consecutive no-output AUs that force a keyframe request. ~50 ms at 60 Hz — long enough not to fire
|
||||
/// on a one-frame decoder hiccup, short enough that a lost initial IDR (or a mid-GOP join) unfreezes
|
||||
/// almost immediately instead of never.
|
||||
pub const NO_OUTPUT_KEYFRAME_STREAK: u32 = 3;
|
||||
|
||||
/// Longest the gate holds the last good frame waiting for a post-loss re-anchor keyframe before it
|
||||
/// re-asks. After a reference loss the hardware decoder does not error — it conceals the
|
||||
/// reference-missing deltas (on RADV, the DPB-and-output-COINCIDE path renders them as a gray plate
|
||||
/// with the new frame's motion painted over it) and returns Ok, so displaying them is the "gray frames
|
||||
/// mid-stream" artifact. We instead freeze on the last good picture until a fresh IDR re-anchors decode
|
||||
/// — the behaviour NVIDIA already shows (its DISTINCT output image + different concealment reads as a
|
||||
/// brief freeze, not gray). This cap only bounds the freeze when recovery genuinely stalls (host
|
||||
/// ignores the request, or an RFI recovery that never emits a keyframe): the freeze is NEVER lifted to
|
||||
/// the concealed picture — the deadline re-asks for a keyframe and keeps holding, so a glitch can never
|
||||
/// become a permanent freeze while a clean re-anchor is what un-freezes. A recovery IDR round-trips well
|
||||
/// under this on any live link.
|
||||
pub const REANCHOR_FREEZE_MAX: Duration = Duration::from_millis(500);
|
||||
|
||||
/// How many host intra-refresh recovery marks ([`USER_FLAG_RECOVERY_POINT`]) must arrive since the
|
||||
/// latest loss before the gate lifts its freeze on an IDR-free stream. TWO, not one: with a continuous
|
||||
/// rolling wave the host marks phase-fixed wave boundaries, so the FIRST boundary after a loss is only
|
||||
/// partially healed — stripes swept BEFORE the loss still reference the lost frame — and lifting there
|
||||
/// would flash a partially-stale picture. The SECOND boundary guarantees a full wave swept entirely
|
||||
/// after the loss, so the picture is clean. This stays correct under repeated loss because every fresh
|
||||
/// arm resets the count. The cost is up to ~2 wave periods of holding the last good frame — the
|
||||
/// deliberate "hold longer, never show garbage" trade.
|
||||
///
|
||||
/// [`USER_FLAG_RECOVERY_POINT`]: crate::packet::USER_FLAG_RECOVERY_POINT
|
||||
pub const REANCHOR_MARKS_TO_LIFT: u32 = 2;
|
||||
|
||||
/// Backstop patience while a host intra-refresh heal is visibly in progress. Each recovery mark pushes
|
||||
/// the freeze deadline out by this much, so a live mark stream (the host actively healing via its wave)
|
||||
/// keeps the gate patiently holding the last good frame instead of tripping the IDR floor mid-heal.
|
||||
/// Must exceed the inter-mark interval (one wave period, ~0.5 s) with margin; if the marks STOP (heal
|
||||
/// stalled, or the host isn't running intra-refresh) the deadline lapses and the normal recovery-IDR
|
||||
/// floor fires, so a real stall still recovers.
|
||||
pub const RECOVERY_MARK_PATIENCE: Duration = Duration::from_millis(1500);
|
||||
|
||||
/// Frames skipped when `got` arrives while `expected` was the next index, or `None` if `got` is
|
||||
/// contiguous (`== expected`) or a straggler we have already passed. Frame indices are u32 counters
|
||||
/// that wrap, so the "ahead" test is a wrapping subtraction split at the half-space: a small positive
|
||||
/// delta is a forward gap (missing frames whose dependents will decode against absent references); a
|
||||
/// delta in the top half is an index behind us.
|
||||
pub fn index_gap(expected: u32, got: u32) -> Option<u32> {
|
||||
let ahead = got.wrapping_sub(expected);
|
||||
(ahead != 0 && ahead < u32::MAX / 2).then_some(ahead)
|
||||
}
|
||||
|
||||
/// Fold one decoded frame into the re-anchor state and decide whether it lifts the post-loss freeze.
|
||||
///
|
||||
/// `is_keyframe` — a real IDR (always a clean re-anchor). `has_anchor` — this AU carried
|
||||
/// [`USER_FLAG_RECOVERY_ANCHOR`](crate::packet::USER_FLAG_RECOVERY_ANCHOR), the host's definitive
|
||||
/// single-frame re-anchor from an LTR-RFI recovery (a clean P-frame coded against a known-good
|
||||
/// reference), so it lifts on the FIRST occurrence exactly like an IDR — no two-mark wait. `has_mark` —
|
||||
/// this AU carried [`USER_FLAG_RECOVERY_POINT`](crate::packet::USER_FLAG_RECOVERY_POINT), a
|
||||
/// host-signalled intra-refresh wave boundary (only *half* a re-anchor). `marks` — recovery marks seen
|
||||
/// since the latest loss.
|
||||
///
|
||||
/// Returns `(lift, new_marks)`: `lift` clears the freeze; `new_marks` is the running count (reset to 0
|
||||
/// on a lift). The two-mark rule ([`REANCHOR_MARKS_TO_LIFT`]) lives here so it is unit-tested
|
||||
/// independent of the pump's channel/decoder plumbing — the first wave boundary after a loss is only
|
||||
/// partially healed, so a single mark must NOT lift. An anchor (or IDR) is a *whole* re-anchor and
|
||||
/// lifts immediately.
|
||||
fn reanchor_after_frame(is_keyframe: bool, has_anchor: bool, has_mark: bool, marks: u32) -> (bool, u32) {
|
||||
let marks = if has_mark {
|
||||
marks.saturating_add(1)
|
||||
} else {
|
||||
marks
|
||||
};
|
||||
if is_keyframe || has_anchor || marks >= REANCHOR_MARKS_TO_LIFT {
|
||||
(true, 0)
|
||||
} else {
|
||||
(false, marks)
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a decoded frame should be shown or withheld while the gate is (or isn't) frozen.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum GateVerdict {
|
||||
/// Present this frame — the gate is not frozen, or this frame is the clean re-anchor that lifts it.
|
||||
Present,
|
||||
/// Withhold this frame — it is a post-loss concealment; the presenter keeps the last good picture.
|
||||
Hold,
|
||||
}
|
||||
|
||||
/// The shared post-loss freeze state machine. A client feeds it three kinds of event — an *arm* (a
|
||||
/// loss was detected: a frame-index gap, a dropped-count climb, or a decoder wedge/demotion), each
|
||||
/// *decoded frame* ([`on_decoded`](Self::on_decoded), which decides present-vs-hold and interprets the
|
||||
/// re-anchor wire flags), and each *no-output* AU ([`on_no_output`](Self::on_no_output)) — plus a
|
||||
/// periodic [`poll`](Self::poll) that folds the dropped counter and fires the overdue backstop.
|
||||
///
|
||||
/// The gate emits *intents* only: [`on_no_output`](Self::on_no_output) and [`poll`](Self::poll) return
|
||||
/// `true` when the client should ask the host for a keyframe. The client routes that through its own
|
||||
/// ~100 ms request throttle (and the precise RFI-vs-keyframe range decision stays in the loss-range
|
||||
/// tracker behind [`crate::client::NativeClient::note_frame_index`]) — the gate never touches the wire.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ReanchorGate {
|
||||
/// Frozen on the last good frame, withholding the decoder's concealed output until a clean
|
||||
/// re-anchor. Armed by any loss signal; cleared only by [`on_decoded`](Self::on_decoded) lifting.
|
||||
awaiting: bool,
|
||||
/// Host intra-refresh recovery marks seen since the latest arm (see [`REANCHOR_MARKS_TO_LIFT`]).
|
||||
/// Reset to 0 whenever the freeze is (re-)armed, so a fresh loss always waits out two fresh marks.
|
||||
marks: u32,
|
||||
/// When the freeze becomes overdue and [`poll`](Self::poll) re-asks for a keyframe (holding, never
|
||||
/// resuming to the concealed picture). `None` when not frozen.
|
||||
deadline: Option<Instant>,
|
||||
/// Consecutive received AUs that produced no decoded frame — a decoder wedged on missing references
|
||||
/// with no reassembler drop to trigger recovery. A short streak forces a fresh IDR.
|
||||
no_output_streak: u32,
|
||||
/// The last `frames_dropped` value [`poll`](Self::poll) observed; a climb means the reassembler
|
||||
/// declared an AU unrecoverable and the following deltas will conceal, so arm.
|
||||
last_dropped: u64,
|
||||
}
|
||||
|
||||
impl ReanchorGate {
|
||||
/// Seed the gate with the session's current `frames_dropped` so the first [`poll`](Self::poll)
|
||||
/// doesn't read the baseline as a loss.
|
||||
pub fn new(frames_dropped: u64) -> Self {
|
||||
ReanchorGate {
|
||||
awaiting: false,
|
||||
marks: 0,
|
||||
deadline: None,
|
||||
no_output_streak: 0,
|
||||
last_dropped: frames_dropped,
|
||||
}
|
||||
}
|
||||
|
||||
/// Arm the freeze: a loss was detected (a frame-index gap, a dropped-count climb, or a decoder
|
||||
/// wedge/demotion). Zeroes the mark count so a fresh loss waits out two fresh recovery marks, and
|
||||
/// (re-)sets the backstop deadline. Idempotent while already frozen (re-arming just re-zeroes the
|
||||
/// marks and pushes the deadline — the correct behaviour when a second loss lands mid-freeze).
|
||||
pub fn arm(&mut self, now: Instant) {
|
||||
self.awaiting = true;
|
||||
self.marks = 0;
|
||||
self.deadline = Some(now + REANCHOR_FREEZE_MAX);
|
||||
}
|
||||
|
||||
/// Fold one decoded frame and decide whether to present or withhold it.
|
||||
///
|
||||
/// `wire_flags` is the AU's `user_flags` word ([`crate::session::Frame::flags`] /
|
||||
/// `PunktfunkFrame.flags`); the gate reads [`FLAG_SOF`](crate::packet::FLAG_SOF) (the host sets it
|
||||
/// only on IDR AUs — the codec-agnostic keyframe signal the platform decoders don't expose),
|
||||
/// [`USER_FLAG_RECOVERY_ANCHOR`] and [`USER_FLAG_RECOVERY_POINT`]. `decoder_keyframe` is an optional
|
||||
/// belt from decoders that flag IDRs themselves (libavcodec's `AV_FRAME_FLAG_KEY` on Linux/Windows);
|
||||
/// pass `false` where the decoder doesn't (Android MediaCodec, Apple VideoToolbox) and rely on the
|
||||
/// wire `FLAG_SOF`.
|
||||
///
|
||||
/// A decoded frame always clears the no-output streak. When frozen, a live mark stream pushes the
|
||||
/// backstop out ([`RECOVERY_MARK_PATIENCE`]) so a healing wave isn't pre-empted by a mid-heal IDR.
|
||||
///
|
||||
/// [`USER_FLAG_RECOVERY_ANCHOR`]: crate::packet::USER_FLAG_RECOVERY_ANCHOR
|
||||
/// [`USER_FLAG_RECOVERY_POINT`]: crate::packet::USER_FLAG_RECOVERY_POINT
|
||||
pub fn on_decoded(&mut self, wire_flags: u32, decoder_keyframe: bool, now: Instant) -> GateVerdict {
|
||||
self.no_output_streak = 0;
|
||||
let is_keyframe = decoder_keyframe || (wire_flags & FLAG_SOF as u32 != 0);
|
||||
let has_anchor = wire_flags & USER_FLAG_RECOVERY_ANCHOR != 0;
|
||||
let has_mark = wire_flags & USER_FLAG_RECOVERY_POINT != 0;
|
||||
if has_mark && self.awaiting {
|
||||
self.deadline = Some(now + RECOVERY_MARK_PATIENCE);
|
||||
}
|
||||
let (lift, marks) = reanchor_after_frame(is_keyframe, has_anchor, has_mark, self.marks);
|
||||
self.marks = marks;
|
||||
if lift {
|
||||
self.awaiting = false;
|
||||
self.deadline = None;
|
||||
}
|
||||
if self.awaiting {
|
||||
GateVerdict::Hold
|
||||
} else {
|
||||
GateVerdict::Present
|
||||
}
|
||||
}
|
||||
|
||||
/// A received AU produced no decoded frame (decode error, or the decoder swallowed a
|
||||
/// reference-missing delta). Returns `true` when the streak has tripped and the client should
|
||||
/// (throttled) request a keyframe — arming the freeze at the same time, since the stream is broken
|
||||
/// regardless of whether the throttle lets the request through this iteration.
|
||||
pub fn on_no_output(&mut self, now: Instant) -> bool {
|
||||
self.no_output_streak += 1;
|
||||
if self.no_output_streak >= NO_OUTPUT_KEYFRAME_STREAK {
|
||||
self.arm(now);
|
||||
self.no_output_streak = 0;
|
||||
true
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/// Periodic fold of the session's `frames_dropped` counter plus the overdue backstop. Returns
|
||||
/// `true` when the client should (throttled) request a keyframe: either the drop count climbed (a
|
||||
/// fresh unrecoverable loss — arm the freeze) or the freeze has held a full [`REANCHOR_FREEZE_MAX`]
|
||||
/// window with no re-anchor (re-ask and keep holding — NEVER resume to the concealed picture; a
|
||||
/// genuinely dead stream is the QUIC idle-timeout watchdog's job, not the gate's).
|
||||
pub fn poll(&mut self, frames_dropped: u64, now: Instant) -> bool {
|
||||
let mut want_keyframe = false;
|
||||
if frames_dropped > self.last_dropped {
|
||||
self.last_dropped = frames_dropped;
|
||||
self.arm(now);
|
||||
want_keyframe = true;
|
||||
}
|
||||
if self.awaiting && self.deadline.is_some_and(|d| now >= d) {
|
||||
self.deadline = Some(now + REANCHOR_FREEZE_MAX);
|
||||
want_keyframe = true;
|
||||
}
|
||||
want_keyframe
|
||||
}
|
||||
|
||||
/// Whether the gate is currently withholding concealed frames (frozen on the last good picture).
|
||||
pub fn is_holding(&self) -> bool {
|
||||
self.awaiting
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
// Simulate the gate's re-anchor state across a sequence of decoded frames: each `(is_keyframe,
|
||||
// has_mark)` pair is folded through `reanchor_after_frame`, returning the frame index (0-based) at
|
||||
// which the freeze first lifts, or `None` if it never does. A reset to 0 models a fresh loss
|
||||
// re-arming the freeze (the gate zeroes the count at every arm site).
|
||||
fn lift_at(frames: &[(bool, bool)]) -> Option<usize> {
|
||||
let mut marks = 0u32;
|
||||
for (i, &(is_kf, has_mark)) in frames.iter().enumerate() {
|
||||
// The intra-refresh-mark model never carries an LTR-RFI anchor (that path is exercised by
|
||||
// `an_rfi_anchor_lifts_immediately`), so `has_anchor` is always false here.
|
||||
let (lift, m) = reanchor_after_frame(is_kf, false, has_mark, marks);
|
||||
marks = m;
|
||||
if lift {
|
||||
return Some(i);
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_single_recovery_mark_does_not_lift() {
|
||||
// The first wave boundary after a loss is only half-healed — one mark must hold the freeze.
|
||||
assert_eq!(REANCHOR_MARKS_TO_LIFT, 2);
|
||||
assert_eq!(lift_at(&[(false, true)]), None);
|
||||
assert_eq!(lift_at(&[(false, false), (false, true), (false, false)]), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_second_recovery_mark_lifts() {
|
||||
// Two marks = a full wave swept after the loss → clean re-anchor.
|
||||
assert_eq!(lift_at(&[(false, true), (false, true)]), Some(1));
|
||||
assert_eq!(
|
||||
lift_at(&[(false, false), (false, true), (false, false), (false, true)]),
|
||||
Some(3)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_real_keyframe_lifts_immediately() {
|
||||
// An IDR is always a clean anchor — no marks needed.
|
||||
assert_eq!(lift_at(&[(true, false)]), Some(0));
|
||||
assert_eq!(lift_at(&[(false, true), (true, false)]), Some(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fresh_gap_resets_the_mark_count() {
|
||||
// The gate zeroes `marks` at each arm site, so one mark before a new gap plus one after must
|
||||
// NOT lift — the model resets the running count to imitate that.
|
||||
let mut marks = 0u32;
|
||||
let (_, m) = reanchor_after_frame(false, false, true, marks); // mark #1 (pre-gap)
|
||||
marks = m;
|
||||
assert_eq!(marks, 1);
|
||||
marks = 0; // a new gap re-arms the freeze → count reset
|
||||
let (lift, m) = reanchor_after_frame(false, false, true, marks); // first mark of the new wave
|
||||
assert!(!lift, "a single post-gap mark must not lift");
|
||||
assert_eq!(m, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_rfi_anchor_lifts_immediately() {
|
||||
// An LTR-RFI recovery anchor is a WHOLE re-anchor (a clean P-frame off a known-good reference),
|
||||
// so — like an IDR — it lifts on the FIRST occurrence, no two-mark wait.
|
||||
let (lift, marks) = reanchor_after_frame(false, true, false, 0);
|
||||
assert!(lift, "an RFI anchor must lift the freeze immediately");
|
||||
assert_eq!(marks, 0, "a lift resets the running mark count");
|
||||
// Even with zero prior marks and no keyframe, the anchor alone is sufficient.
|
||||
let (lift, _) = reanchor_after_frame(false, true, true, 1);
|
||||
assert!(lift, "an anchor lifts regardless of the pending mark count");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn contiguous_indices_are_not_a_gap() {
|
||||
assert_eq!(index_gap(5, 5), None);
|
||||
assert_eq!(index_gap(0, 0), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_forward_jump_reports_the_skip_count() {
|
||||
assert_eq!(index_gap(5, 6), Some(1)); // one frame missing
|
||||
assert_eq!(index_gap(5, 9), Some(4));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_straggler_behind_us_is_not_a_gap() {
|
||||
// The reassembler emitted a newer frame first; the late one must not re-arm.
|
||||
assert_eq!(index_gap(9, 5), None);
|
||||
assert_eq!(index_gap(1, 0), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_index_counter_wraps_cleanly() {
|
||||
// last frame = u32::MAX, so the next expected wraps to 0.
|
||||
assert_eq!(index_gap(0, 0), None);
|
||||
// waiting on u32::MAX, frame 0 arrived → MAX was skipped.
|
||||
assert_eq!(index_gap(u32::MAX, 0), Some(1));
|
||||
assert_eq!(index_gap(u32::MAX, 2), Some(3));
|
||||
// an old frame arriving just after the wrap is still a straggler.
|
||||
assert_eq!(index_gap(0, u32::MAX), None);
|
||||
}
|
||||
|
||||
// ---- gate-level sequence tests (the whole behavioural contract) ----
|
||||
|
||||
const SOF: u32 = FLAG_SOF as u32; // IDR wire flag
|
||||
const ANCHOR: u32 = USER_FLAG_RECOVERY_ANCHOR;
|
||||
const POINT: u32 = USER_FLAG_RECOVERY_POINT;
|
||||
|
||||
fn t0() -> Instant {
|
||||
Instant::now()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_clean_link_never_holds() {
|
||||
// Disarmed gate presents every frame, keyframe or not, and never asks for anything.
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let now = t0();
|
||||
assert_eq!(g.on_decoded(0, false, now), GateVerdict::Present);
|
||||
assert_eq!(g.on_decoded(SOF, true, now), GateVerdict::Present);
|
||||
assert!(!g.is_holding());
|
||||
assert!(!g.poll(0, now));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_gap_holds_until_the_wire_keyframe_lifts() {
|
||||
// Android/Apple path: no decoder keyframe flag, lift comes from the wire FLAG_SOF alone.
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let now = t0();
|
||||
g.arm(now); // frame-index gap
|
||||
assert!(g.is_holding());
|
||||
assert_eq!(g.on_decoded(0, false, now), GateVerdict::Hold); // concealed delta withheld
|
||||
assert_eq!(g.on_decoded(0, false, now), GateVerdict::Hold);
|
||||
assert_eq!(g.on_decoded(SOF, false, now), GateVerdict::Present); // IDR re-anchors
|
||||
assert!(!g.is_holding());
|
||||
assert_eq!(g.on_decoded(0, false, now), GateVerdict::Present); // stays presenting
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_gap_lifts_on_the_first_rfi_anchor() {
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let now = t0();
|
||||
g.arm(now);
|
||||
assert_eq!(g.on_decoded(0, false, now), GateVerdict::Hold);
|
||||
assert_eq!(g.on_decoded(ANCHOR, false, now), GateVerdict::Present);
|
||||
assert!(!g.is_holding());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_gap_lifts_on_the_second_recovery_mark() {
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let now = t0();
|
||||
g.arm(now);
|
||||
assert_eq!(g.on_decoded(POINT, false, now), GateVerdict::Hold); // first boundary: half-healed
|
||||
assert_eq!(g.on_decoded(0, false, now), GateVerdict::Hold);
|
||||
assert_eq!(g.on_decoded(POINT, false, now), GateVerdict::Present); // second: clean
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_second_gap_mid_freeze_resets_the_marks() {
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let now = t0();
|
||||
g.arm(now);
|
||||
assert_eq!(g.on_decoded(POINT, false, now), GateVerdict::Hold); // mark #1
|
||||
g.arm(now); // a fresh loss re-arms → mark count zeroed
|
||||
assert_eq!(g.on_decoded(POINT, false, now), GateVerdict::Hold); // this is mark #1 of the new wave
|
||||
assert_eq!(g.on_decoded(POINT, false, now), GateVerdict::Present); // #2 lifts
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_dropped_climb_arms_and_asks() {
|
||||
let mut g = ReanchorGate::new(5);
|
||||
let now = t0();
|
||||
assert!(!g.poll(5, now), "no climb → no ask"); // baseline
|
||||
assert!(g.poll(6, now), "a climb asks for a keyframe");
|
||||
assert!(g.is_holding(), "and arms the freeze");
|
||||
assert!(!g.poll(6, now), "same value → no repeat ask from the drop path");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_no_output_streak_trips_at_three() {
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let now = t0();
|
||||
assert!(!g.on_no_output(now));
|
||||
assert!(!g.on_no_output(now));
|
||||
assert!(g.on_no_output(now), "third no-output trips the streak");
|
||||
assert!(g.is_holding());
|
||||
// A decoded frame resets the streak.
|
||||
g.on_decoded(SOF, false, now); // lifts + resets streak
|
||||
assert!(!g.on_no_output(now));
|
||||
assert!(!g.on_no_output(now));
|
||||
assert!(g.on_no_output(now));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_overdue_freeze_re_asks_but_keeps_holding() {
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let start = t0();
|
||||
g.arm(start);
|
||||
// Before the deadline: holding, no re-ask.
|
||||
assert!(!g.poll(0, start));
|
||||
assert!(g.is_holding());
|
||||
// Past REANCHOR_FREEZE_MAX with no re-anchor: re-ask, still holding.
|
||||
let later = start + REANCHOR_FREEZE_MAX + Duration::from_millis(1);
|
||||
assert!(g.poll(0, later), "overdue freeze re-asks for a keyframe");
|
||||
assert!(g.is_holding(), "but never resumes to the concealed picture");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_live_mark_stream_pushes_the_deadline_out() {
|
||||
// A healing wave (marks arriving) must not be pre-empted by the overdue IDR floor.
|
||||
let mut g = ReanchorGate::new(0);
|
||||
let start = t0();
|
||||
g.arm(start);
|
||||
// A mark past the original freeze deadline pushes it out by RECOVERY_MARK_PATIENCE.
|
||||
let t = start + REANCHOR_FREEZE_MAX + Duration::from_millis(10);
|
||||
assert_eq!(g.on_decoded(POINT, false, t), GateVerdict::Hold); // mark #1, deadline pushed
|
||||
// At a time that WOULD have been overdue on the original deadline, poll does not re-ask.
|
||||
assert!(!g.poll(0, t + Duration::from_millis(1)));
|
||||
assert!(g.is_holding());
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user