feat(client): freeze-until-reanchor loss recovery on Android + Apple via shared core gate

After unrecoverable loss the host keeps sending delta frames that reference a
picture the client never received; hardware decoders conceal these as gray/
garbage with a success status. Linux already withheld them and held the last
good frame until a proven clean re-anchor — this brings that behavior to the
Android and Apple clients.

Extract the Linux pump's freeze state machine into a shared `ReanchorGate` in
punktfunk-core (reanchor.rs, 18 tests) exposed over the C ABI (ABI v6, additive —
no wire change) for the Swift clients. Migrate the Linux/Deck pump
(pf-client-core) onto it as the parity proof (no-op refactor). Then wire:

- Android (decode.rs, both sync + async loops): arm on the frame-index gap, a
  pts-keyed flag map carries the wire flags to the output-buffer release, fold
  the gate per drained output, gate.poll replaces the dropped-climb block.
- Apple Stage2Pipeline (default): arm on a gap (new noteFrameIndexGap), withhold
  at the ring-submit seam (CAMetalLayer holds its last drawable), poll
  framesDropped, fold VT decode errors through the no-output streak.
- Apple StreamPump (stage-1): fold at enqueue, withhold via
  kCMSampleAttachmentKey_DoNotDisplay so the layer keeps decoding (reference
  chain intact) but holds the last displayed frame.
- Apple VideoDecoder: thread the AU's wire flags to the async decode callback via
  a retained FrameContext refcon (replaces the receivedNs bit-pattern scalar).

Lifts only on a proven re-anchor (IDR / RFI anchor / 2nd recovery mark) with a
500 ms backstop so a lost re-anchor can never freeze forever. Apple: swift build
clean, 123/123 tests pass (incl. VideoToolboxRoundTripTests). On-glass
loss-injection validation still owed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-13 01:21:25 +02:00
parent cd701a9594
commit 8a18e130a2
11 changed files with 1104 additions and 380 deletions
+140
View File
@@ -13,6 +13,7 @@
use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use crate::error::PunktfunkStatus;
use crate::input::InputEvent;
use crate::reanchor::{GateVerdict, ReanchorGate};
use crate::session::Session;
use crate::stats::Stats;
use crate::transport::{loopback_pair, Transport, UdpTransport};
@@ -2620,3 +2621,142 @@ pub unsafe extern "C" fn punktfunk_connection_close(c: *mut PunktfunkConnection)
drop(unsafe { Box::from_raw(c) });
}
}
// ---- Post-loss re-anchor freeze gate ----
//
// The shared [`ReanchorGate`](crate::reanchor::ReanchorGate) exposed for the Swift client (Rust
// embedders — Android/Windows/Linux — use the struct directly). After an unrecoverable reference
// loss the decoder silently conceals the missing-reference deltas (gray/garbage picture, no error);
// the client freezes on the last good frame and lifts only on a proven clean re-anchor. The gate
// takes time internally (`Instant::now`) so no timestamps cross the boundary. Drive it per session:
// `arm` on a loss (frame-index gap from `punktfunk_connection_note_frame_index`, a decoder
// wedge/demotion), `on_decoded` per decoded frame to gate presentation, `on_no_output` per AU that
// produced nothing, and `poll` each iteration for the dropped-count climb + overdue backstop. Route
// the returned keyframe intents through the client's existing request throttle.
/// Create a re-anchor gate seeded with the session's current `frames_dropped` (so the first
/// [`punktfunk_reanchor_gate_poll`] doesn't read the baseline as a loss). Free with
/// [`punktfunk_reanchor_gate_free`]. Never returns NULL.
#[no_mangle]
pub extern "C" fn punktfunk_reanchor_gate_new(frames_dropped: u64) -> *mut ReanchorGate {
Box::into_raw(Box::new(ReanchorGate::new(frames_dropped)))
}
/// Free a gate created by [`punktfunk_reanchor_gate_new`]. NULL is a no-op.
///
/// # Safety
/// `g` was returned by [`punktfunk_reanchor_gate_new`] and is not used after this call.
#[no_mangle]
pub unsafe extern "C" fn punktfunk_reanchor_gate_free(g: *mut ReanchorGate) {
if !g.is_null() {
drop(unsafe { Box::from_raw(g) });
}
}
/// Arm the freeze: a loss was detected (a frame-index gap, or a decoder wedge/demotion). Zeroes the
/// recovery-mark count and (re-)sets the backstop deadline. NULL is a no-op.
///
/// # Safety
/// `g` is a valid gate handle.
#[no_mangle]
pub unsafe extern "C" fn punktfunk_reanchor_gate_arm(g: *mut ReanchorGate) {
if let Some(g) = unsafe { g.as_mut() } {
g.arm(std::time::Instant::now());
}
}
/// Fold one decoded frame and write to `out_present` whether to display it (`true`) or withhold it as
/// a post-loss concealment (`false`). `flags` is the AU's `user_flags` word ([`PunktfunkFrame::flags`]):
/// the gate reads `FLAG_SOF` (the host's IDR marker), `USER_FLAG_RECOVERY_ANCHOR` and
/// `USER_FLAG_RECOVERY_POINT`. Pass `decoder_keyframe = false` where the platform decoder doesn't flag
/// IDRs (VideoToolbox/MediaCodec) — the wire `FLAG_SOF` covers it.
///
/// # Safety
/// `g` is a valid gate handle; `out_present` is writable or NULL.
#[no_mangle]
pub unsafe extern "C" fn punktfunk_reanchor_gate_on_decoded(
g: *mut ReanchorGate,
flags: u32,
decoder_keyframe: bool,
out_present: *mut bool,
) -> PunktfunkStatus {
guard(|| {
let g = match unsafe { g.as_mut() } {
Some(g) => g,
None => return PunktfunkStatus::NullPointer,
};
let present = g.on_decoded(flags, decoder_keyframe, std::time::Instant::now()) == GateVerdict::Present;
if !out_present.is_null() {
unsafe { *out_present = present };
}
PunktfunkStatus::Ok
})
}
/// A received AU produced no decoded frame. Writes to `out_request_kf` whether the no-output streak has
/// tripped and the client should (throttled) request a keyframe — the gate arms the freeze at the same
/// time.
///
/// # Safety
/// `g` is a valid gate handle; `out_request_kf` is writable or NULL.
#[no_mangle]
pub unsafe extern "C" fn punktfunk_reanchor_gate_on_no_output(
g: *mut ReanchorGate,
out_request_kf: *mut bool,
) -> PunktfunkStatus {
guard(|| {
let g = match unsafe { g.as_mut() } {
Some(g) => g,
None => return PunktfunkStatus::NullPointer,
};
let request = g.on_no_output(std::time::Instant::now());
if !out_request_kf.is_null() {
unsafe { *out_request_kf = request };
}
PunktfunkStatus::Ok
})
}
/// Periodic fold of the session's `frames_dropped` counter plus the overdue backstop. Writes to
/// `out_request_kf` whether the client should (throttled) request a keyframe (a drop-count climb armed
/// a fresh freeze, or the freeze is overdue and re-asks while it keeps holding).
///
/// # Safety
/// `g` is a valid gate handle; `out_request_kf` is writable or NULL.
#[no_mangle]
pub unsafe extern "C" fn punktfunk_reanchor_gate_poll(
g: *mut ReanchorGate,
frames_dropped: u64,
out_request_kf: *mut bool,
) -> PunktfunkStatus {
guard(|| {
let g = match unsafe { g.as_mut() } {
Some(g) => g,
None => return PunktfunkStatus::NullPointer,
};
let request = g.poll(frames_dropped, std::time::Instant::now());
if !out_request_kf.is_null() {
unsafe { *out_request_kf = request };
}
PunktfunkStatus::Ok
})
}
/// Whether the gate is currently withholding concealed frames (frozen on the last good picture).
/// Writes `false` on a NULL gate.
///
/// # Safety
/// `g` is a valid gate handle; `out_holding` is writable or NULL.
#[no_mangle]
pub unsafe extern "C" fn punktfunk_reanchor_gate_is_holding(
g: *const ReanchorGate,
out_holding: *mut bool,
) -> PunktfunkStatus {
guard(|| {
let holding = unsafe { g.as_ref() }.is_some_and(ReanchorGate::is_holding);
if !out_holding.is_null() {
unsafe { *out_holding = holding };
}
PunktfunkStatus::Ok
})
}