fix(encode): harden loss-recovery correctness across host encoders (F1–F7)

Phases 1–4 of design/encoder-recovery-hardening.md — make the shipped RFI/
freeze-until-reanchor recovery honest and rebuild-safe across every backend.

F1 — frame-index domain desync: the encode loop now owns a session-lifetime
`au_seq`; `Encoder::submit_indexed(au_seq + inflight)` pins NVENC inputTimeStamp
and AMF LTR slots to the WIRE frame index, so `invalidate_ref_frames` compares
client frame numbers in the same domain and survives adaptive-bitrate rebuilds
(an internal counter desynced on the first rebuild → RFI silently dead / an AMF
force-ref onto a never-decoded frame). `FrameMsg.frame_index` →
`Session::seal_frame_at`; GameStream gets the same via `VideoPacketizer::
packetize(.., Some(idx))`.

F2 — Windows NVENC left the client frozen ~1s per loss: NVENC RFI was
transparent (no anchor tag) while the session glue armed the 750ms IDR cooldown,
so the freeze only lifted on the ~1s keyframe re-ask. NVENC now mirrors AMF —
`pending_anchor` tags the first post-invalidate AU (the clean re-anchor
P-frame) `recovery_anchor`, incl. the covering-range dedupe re-arm; the client
lifts at ~RTT.

F3 — speed-test probe filler burned video frame indexes: moved to its own index
space (`Packetizer::alloc_probe_index` + `Session::submit_probe_frame`) with a
second client reassembly window routed on FLAG_PROBE, gated on the new
VIDEO_CAP_PROBE_SEQ Hello bit (mid-session probes declined for older clients).

F4 — RFI range sanity cap: forward gaps wider than `packet::RFI_MAX_RANGE` (256)
resync via keyframe instead of an out-of-range RFI, host- and client-side
(client huge-gap → keyframe in `RfiRecovery::observe` + the pf-client-core pump).

F5 — reset() parity: Windows NVENC (teardown + lazy re-init), Linux VAAPI
(drop-inner), Linux NVENC (reopen from stored OpenArgs) now give the stall
watchdog a heal lever instead of ending the session.

F6 — sw.rs `pending: VecDeque` (was `Option`), killing the silent AU drop at
capturer pipeline depth > 1. F7 — doc sweep on the RFI/anchor comments.

Verified: punktfunk-core lib tests (macOS + Linux), full punktfunk-host suite on
Linux (RTX 5070 Ti), Windows compile. Owed: the on-glass client matrix (F2
freeze A/B, AMF LTR spike across a bitrate rebuild).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-12 11:17:19 +02:00
parent 0dc414f197
commit fdda7144ed
15 changed files with 801 additions and 152 deletions
+11 -2
View File
@@ -462,12 +462,21 @@ fn pump(
// paths (one recovery ask per 100 ms) so a burst of gaps — a full-screen
// pan shedding shards — can't storm the control stream. This fires ~120 ms
// before frames_dropped would, so recovery also starts sooner.
//
// A gap wider than RFI_MAX_RANGE is beyond any encoder's reference
// history (a seconds-long outage — or a phantom index jump, e.g. the
// first real AU after an old host's speed-test burst consumed video
// indexes): RFI is hopeless there, so ask for the IDR resync directly.
if last_kf_req
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ =
connector.request_rfi(exp, frame.frame_index.wrapping_sub(1));
if gap > punktfunk_core::packet::RFI_MAX_RANGE {
let _ = connector.request_keyframe();
} else {
let _ = connector
.request_rfi(exp, frame.frame_index.wrapping_sub(1));
}
}
tracing::trace!(
gap,
+88 -30
View File
@@ -375,14 +375,29 @@ struct RfiRecovery {
last_req: Option<Instant>,
}
/// What a forward gap should ask the host for: a precise RFI for a recoverable range, a plain
/// keyframe for a range wider than any encoder's reference history
/// ([`crate::packet::RFI_MAX_RANGE`] — a seconds-long outage, or a phantom index jump such as an
/// old host's speed-test burst consuming video indexes), or nothing (contiguous / straggler /
/// throttled).
#[derive(Debug, PartialEq, Eq)]
enum RecoveryAsk {
None,
Rfi(u32, u32),
Keyframe,
}
impl RfiRecovery {
/// Pure decision behind [`NativeClient::note_frame_index`]: fold one received `frame_index` (in
/// receive order) observed at `now`, advancing the expectation and returning
/// `(gap, rfi_range)`. `gap` is whether this frame revealed a forward gap; `rfi_range` is
/// `Some((first_missing, last_missing))` when a (throttled) RFI should fire for the lost span, or
/// `None` when contiguous, a straggler, or throttled. Split out from the connection so the wrapping
/// arithmetic + [`RFI_THROTTLE`] are unit-testable without a live session (see the tests below).
fn observe(&mut self, frame_index: u32, now: Instant) -> (bool, Option<(u32, u32)>) {
/// receive order) observed at `now`, advancing the expectation and returning `(gap, ask)`.
/// `gap` is whether this frame revealed a forward gap (the embedder arms its post-loss display
/// freeze on it); `ask` is the (throttled) recovery request to fire — an RFI naming the exact
/// lost span, or a keyframe when the span exceeds [`crate::packet::RFI_MAX_RANGE`] (RFI is
/// hopeless there: no encoder holds references that old, and a huge jump is more likely a
/// resync — e.g. the first real AU after an old host's speed test — than a real loss). Split
/// out from the connection so the wrapping arithmetic + [`RFI_THROTTLE`] are unit-testable
/// without a live session (see the tests below).
fn observe(&mut self, frame_index: u32, now: Instant) -> (bool, RecoveryAsk) {
match self.next_expected {
Some(exp) => {
// Wrapping split at the half-space: a small positive delta is a forward gap
@@ -390,10 +405,10 @@ impl RfiRecovery {
let ahead = frame_index.wrapping_sub(exp);
if ahead == 0 {
self.next_expected = Some(frame_index.wrapping_add(1)); // contiguous
(false, None)
(false, RecoveryAsk::None)
} else if ahead < u32::MAX / 2 {
// Forward gap: [exp, frame_index-1] lost. Advance past this frame so the same
// gap isn't re-detected, then fire a throttled RFI for the lost range.
// gap isn't re-detected, then fire a throttled recovery ask for the lost range.
self.next_expected = Some(frame_index.wrapping_add(1));
let send = self
.last_req
@@ -401,15 +416,22 @@ impl RfiRecovery {
if send {
self.last_req = Some(now);
}
let range = send.then(|| (exp, frame_index.wrapping_sub(1)));
(true, range)
let ask = if !send {
RecoveryAsk::None
} else if ahead > crate::packet::RFI_MAX_RANGE {
RecoveryAsk::Keyframe
} else {
RecoveryAsk::Rfi(exp, frame_index.wrapping_sub(1))
};
(true, ask)
} else {
(false, None) // straggler behind the delivery point — leave the expectation
// Straggler behind the delivery point — leave the expectation.
(false, RecoveryAsk::None)
}
}
None => {
self.next_expected = Some(frame_index.wrapping_add(1));
(false, None)
(false, RecoveryAsk::None)
}
}
}
@@ -970,13 +992,21 @@ impl NativeClient {
/// signal and shares one throttle across RFI + keyframe requests.)
pub fn note_frame_index(&self, frame_index: u32) -> bool {
// Decide (and update state) under the lock; fire the request after releasing it.
let (gap, rfi_range) = self
let (gap, ask) = self
.rfi
.lock()
.unwrap()
.observe(frame_index, Instant::now());
if let Some((first, last)) = rfi_range {
let _ = self.request_rfi(first, last);
match ask {
RecoveryAsk::Rfi(first, last) => {
let _ = self.request_rfi(first, last);
}
// A gap wider than any encoder's reference history (RFI_MAX_RANGE) — a seconds-long
// outage or a phantom index jump: RFI can't repair it, resync on a keyframe instead.
RecoveryAsk::Keyframe => {
let _ = self.request_keyframe();
}
RecoveryAsk::None => {}
}
gap
}
@@ -1410,7 +1440,12 @@ async fn worker_main(args: WorkerArgs) {
// when the matching bit is set, so `0` stays an 8-bit BT.709 stream. HOST_TIMING is
// OR'd in unconditionally: every NativeClient build demuxes the 0xCF plane, and the
// bit only asks the host for observability datagrams (never changes the encode).
video_caps: video_caps | crate::quic::VIDEO_CAP_HOST_TIMING,
// PROBE_SEQ likewise: the shared reassembler keeps probe filler in its own window
// (every embedder inherits it), so the host may burst speed tests without consuming
// video frame indexes.
video_caps: video_caps
| crate::quic::VIDEO_CAP_HOST_TIMING
| crate::quic::VIDEO_CAP_PROBE_SEQ,
// Requested surround channel count; the host echoes the resolved value in Welcome.
audio_channels,
// The codecs this client can decode + its soft preference (0 = auto). The host
@@ -2071,7 +2106,7 @@ mod rfi_recovery_tests {
//! The client-side loss-range detector shared by every embedder (Android, the C-ABI Apple
//! client, the Windows shell pump). `observe` is pure over `(frame_index, now)`, so the wrapping
//! frame arithmetic and the RFI throttle are exercised here without a live session.
use super::{RfiRecovery, RFI_THROTTLE};
use super::{RecoveryAsk, RfiRecovery, RFI_THROTTLE};
use std::time::{Duration, Instant};
// A fixed base instant; offsets model the throttle window deterministically (no sleeping).
@@ -2083,7 +2118,7 @@ mod rfi_recovery_tests {
fn first_frame_arms_without_a_gap() {
let mut r = RfiRecovery::default();
// The opening frame only seeds the expectation — there is no prior frame to be missing.
assert_eq!(r.observe(100, base()), (false, None));
assert_eq!(r.observe(100, base()), (false, RecoveryAsk::None));
assert_eq!(r.next_expected, Some(101));
}
@@ -2092,9 +2127,9 @@ mod rfi_recovery_tests {
let mut r = RfiRecovery::default();
let t = base();
r.observe(100, t);
assert_eq!(r.observe(101, t), (false, None));
assert_eq!(r.observe(102, t), (false, None));
assert_eq!(r.observe(103, t), (false, None));
assert_eq!(r.observe(101, t), (false, RecoveryAsk::None));
assert_eq!(r.observe(102, t), (false, RecoveryAsk::None));
assert_eq!(r.observe(103, t), (false, RecoveryAsk::None));
assert_eq!(r.next_expected, Some(104));
}
@@ -2104,7 +2139,7 @@ mod rfi_recovery_tests {
let t = base();
r.observe(100, t); // expecting 101 next
// 101..=104 were lost; 105 arrived. The RFI must name exactly the missing span.
assert_eq!(r.observe(105, t), (true, Some((101, 104))));
assert_eq!(r.observe(105, t), (true, RecoveryAsk::Rfi(101, 104)));
// The expectation advances past the delivered frame so the same gap can't re-fire.
assert_eq!(r.next_expected, Some(106));
}
@@ -2115,7 +2150,7 @@ mod rfi_recovery_tests {
let t = base();
r.observe(100, t);
// Exactly one frame (101) lost → range is the single index [101, 101].
assert_eq!(r.observe(102, t), (true, Some((101, 101))));
assert_eq!(r.observe(102, t), (true, RecoveryAsk::Rfi(101, 101)));
}
#[test]
@@ -2124,13 +2159,16 @@ mod rfi_recovery_tests {
let t0 = base();
r.observe(100, t0);
// First gap fires the request and stamps the throttle.
assert_eq!(r.observe(105, t0), (true, Some((101, 104))));
assert_eq!(r.observe(105, t0), (true, RecoveryAsk::Rfi(101, 104)));
// A second gap 50 ms later is still a gap, but the request is throttled away.
assert_eq!(r.observe(110, t0 + Duration::from_millis(50)), (true, None));
assert_eq!(
r.observe(110, t0 + Duration::from_millis(50)),
(true, RecoveryAsk::None)
);
// Past the window, the request re-opens for the still-accurate lost span.
assert_eq!(
r.observe(120, t0 + RFI_THROTTLE + Duration::from_millis(1)),
(true, Some((111, 119)))
(true, RecoveryAsk::Rfi(111, 119))
);
}
@@ -2142,7 +2180,7 @@ mod rfi_recovery_tests {
r.observe(105, t); // expecting 106 next
// A reordered late arrival (103, well behind 106) is neither a gap nor a request, and it
// must not rewind the expectation — otherwise the next in-order frame would false-gap.
assert_eq!(r.observe(103, t), (false, None));
assert_eq!(r.observe(103, t), (false, RecoveryAsk::None));
assert_eq!(r.next_expected, Some(106));
}
@@ -2151,9 +2189,9 @@ mod rfi_recovery_tests {
let mut r = RfiRecovery::default();
let t = base();
r.observe(u32::MAX - 1, t); // expecting u32::MAX next
assert_eq!(r.observe(u32::MAX, t), (false, None)); // contiguous, expectation wraps to 0
assert_eq!(r.observe(u32::MAX, t), (false, RecoveryAsk::None)); // contiguous, wraps to 0
assert_eq!(r.next_expected, Some(0));
assert_eq!(r.observe(0, t), (false, None)); // still contiguous across the wrap
assert_eq!(r.observe(0, t), (false, RecoveryAsk::None)); // still contiguous across the wrap
assert_eq!(r.next_expected, Some(1));
}
@@ -2163,9 +2201,29 @@ mod rfi_recovery_tests {
let t = base();
r.observe(u32::MAX - 1, t); // expecting u32::MAX next
// u32::MAX was lost and 1 arrived → the lost span wraps: [u32::MAX, 0].
assert_eq!(r.observe(1, t), (true, Some((u32::MAX, 0))));
assert_eq!(r.observe(1, t), (true, RecoveryAsk::Rfi(u32::MAX, 0)));
assert_eq!(r.next_expected, Some(2));
}
#[test]
fn huge_gap_resyncs_via_keyframe_not_rfi() {
let mut r = RfiRecovery::default();
let t = base();
r.observe(100, t); // expecting 101 next
// A jump wider than any encoder's reference history (RFI_MAX_RANGE): no valid
// reference exists for an RFI, and the jump may be a phantom (an old host's
// speed-test burst consuming video indexes) — ask for the IDR resync instead.
let jump = 100 + crate::packet::RFI_MAX_RANGE + 2;
assert_eq!(r.observe(jump, t), (true, RecoveryAsk::Keyframe));
// The expectation still advances past the delivered frame (no re-fire on the next one).
assert_eq!(r.next_expected, Some(jump + 1));
assert_eq!(r.observe(jump + 1, t), (false, RecoveryAsk::None));
// A huge gap consumes the shared throttle too — an immediate follow-up gap stays quiet.
assert_eq!(
r.observe(jump + 10, t + Duration::from_millis(1)),
(true, RecoveryAsk::None)
);
}
}
#[cfg(test)]
+221 -33
View File
@@ -54,6 +54,15 @@ pub const USER_FLAG_RECOVERY_POINT: u32 = 0x10;
/// `AV_FRAME_FLAG_KEY` — this host flag is the only signal.
pub const USER_FLAG_RECOVERY_ANCHOR: u32 = 0x20;
/// Widest lost-frame range (frames, wrapping `last - first`) a reference-frame-invalidation
/// recovery may be asked to repair; anything wider goes straight to the keyframe path on BOTH
/// ends. RFI can only re-reference history the encoder still holds — NVENC keeps a 5-frame DPB,
/// AMD LTR ~1 s of marks — and a genuine loss this wide (>1 s even at 240 fps) has no valid
/// reference anywhere, so an RFI request for it is either hopeless or (worse) a phantom range
/// from a desynced counter. Shared by the host's RFI dispatch (range → keyframe fallback) and the
/// client-side gap detectors (huge gap → resync + keyframe request, no RFI).
pub const RFI_MAX_RANGE: u32 = 256;
/// Crypto framing overhead [`Session`](crate::session::Session) adds when encrypting:
/// an 8-byte sequence prefix plus the GCM tag.
pub const CRYPTO_OVERHEAD: usize = 8 + crate::crypto::TAG_LEN;
@@ -117,8 +126,18 @@ const _: () = assert!(HEADER_LEN == 40, "PacketHeader must be 40 bytes / unpadde
// ---------------------------------------------------------------------------
/// Splits encoded access units into FEC-protected shard packets. Host-side only.
///
/// Frame numbering: a caller can pass an **explicit** `frame_index` to
/// [`packetize_each`](Self::packetize_each) (the punktfunk/1 encode loop owns the video numbering
/// so the encoder's reference-frame-invalidation bookkeeping stays 1:1 with the wire across
/// encoder rebuilds/resets), or pass `None` to draw from the internal counter (the legacy path —
/// synthetic/spike/ABI sessions where no encoder cares). Speed-test probe filler draws from a
/// **separate** index space ([`alloc_probe_index`](Self::alloc_probe_index)) so a burst never
/// consumes video indexes — see [`crate::quic::VIDEO_CAP_PROBE_SEQ`].
pub struct Packetizer {
next_frame_index: u32,
/// Probe-space frame counter (see [`alloc_probe_index`](Self::alloc_probe_index)).
next_probe_index: u32,
next_seq: u32,
shard_payload: usize,
fec: crate::config::FecConfig,
@@ -134,6 +153,7 @@ impl Packetizer {
pub fn new(config: &Config) -> Self {
Packetizer {
next_frame_index: 0,
next_probe_index: 0,
next_seq: 0,
shard_payload: config.shard_payload,
fec: config.fec,
@@ -142,6 +162,17 @@ impl Packetizer {
}
}
/// Allocate the next **probe-space** frame index (speed-test filler). A separate counter from
/// the video `frame_index`es so a multi-thousand-AU probe burst never advances the video
/// numbering — the client routes [`FLAG_PROBE`]-flagged shards into its own reassembly window
/// (see [`Reassembler`]), so the two spaces never collide. Only used against clients that
/// advertise [`crate::quic::VIDEO_CAP_PROBE_SEQ`].
pub fn alloc_probe_index(&mut self) -> u32 {
let i = self.next_probe_index;
self.next_probe_index = i.wrapping_add(1);
i
}
/// Live-adjust the FEC recovery percentage (adaptive FEC). Takes effect on the next
/// [`packetize`](Self::packetize); the wire is self-describing (each packet carries its block's
/// data/recovery counts), so the receiver needs no notification. Clamped to ≤ 90.
@@ -165,7 +196,7 @@ impl Packetizer {
coder: &dyn ErasureCoder,
) -> Result<Vec<Vec<u8>>> {
let mut packets = Vec::new();
self.packetize_each(frame, pts_ns, user_flags, coder, |hdr, body| {
self.packetize_each(frame, pts_ns, user_flags, None, coder, |hdr, body| {
let mut pkt = Vec::with_capacity(HEADER_LEN + body.len());
pkt.extend_from_slice(hdr.as_bytes());
pkt.extend_from_slice(body);
@@ -181,17 +212,27 @@ impl Packetizer {
/// shard straight into a pooled wire buffer and seal in place
/// ([`Session::seal_frame`](crate::session::Session::seal_frame)). An `emit` error aborts
/// the frame mid-way (packet numbering has already advanced — callers treat it as fatal).
///
/// `frame_index`: `Some(i)` stamps the AU with the caller's index — the punktfunk/1 encode
/// loop numbers video AUs itself so the encoder's RFI bookkeeping (LTR marks, DPB timestamps)
/// is 1:1 with what the client sees, surviving encoder rebuilds/resets that restart internal
/// counters. `None` draws from the internal counter (the legacy/self-numbering path). A
/// session must not mix the two styles for the same index space.
pub fn packetize_each(
&mut self,
frame: &[u8],
pts_ns: u64,
user_flags: u32,
frame_index: Option<u32>,
coder: &dyn ErasureCoder,
mut emit: impl FnMut(&PacketHeader, &[u8]) -> Result<()>,
) -> Result<()> {
let payload = self.shard_payload;
let frame_index = self.next_frame_index;
self.next_frame_index = self.next_frame_index.wrapping_add(1);
let frame_index = frame_index.unwrap_or_else(|| {
let i = self.next_frame_index;
self.next_frame_index = i.wrapping_add(1);
i
});
// At least one (zero-padded) data shard even for an empty frame.
let total_data = frame.len().div_ceil(payload).max(1);
@@ -343,10 +384,13 @@ impl ReassemblerLimits {
}
}
/// Buffers incoming shards, recovers lost ones via FEC, and emits whole access units.
/// Client-side only.
pub struct Reassembler {
limits: ReassemblerLimits,
/// One frame-index space's reassembly state: the in-flight frames, the recently-emitted memory,
/// and the loss-window anchor. The [`Reassembler`] keeps two — video and speed-test probe filler —
/// because the two ride **separate index counters** on a [`VIDEO_CAP_PROBE_SEQ`]-aware host
/// (a probe burst must neither advance the video loss window nor be dropped as "stale" against
/// it). [`VIDEO_CAP_PROBE_SEQ`]: crate::quic::VIDEO_CAP_PROBE_SEQ
#[derive(Default)]
struct ReassemblyWindow {
frames: HashMap<u32, FrameBuf>,
/// Recently-emitted frames, so stray/late shards can't resurrect them. Pruned to
/// the reorder window alongside `frames`.
@@ -357,13 +401,27 @@ pub struct Reassembler {
newest_frame: Option<(u32, u64)>,
}
/// Buffers incoming shards, recovers lost ones via FEC, and emits whole access units.
/// Client-side only.
pub struct Reassembler {
limits: ReassemblerLimits,
/// The video stream's window — its aged-out incomplete frames count into `frames_dropped`
/// (the client's loss-recovery trigger).
video: ReassemblyWindow,
/// Speed-test probe filler ([`FLAG_PROBE`] in `user_flags`). Routed by the flag, so it also
/// captures an OLD host's probe frames (which still carry video-space indexes — they complete
/// fine here, and keeping them out of the video window means a burst can no longer advance the
/// video loss anchor). Aged-out probe frames are NOT `frames_dropped` — probe loss is measured
/// bytes-wise by the probe accumulator and must not fire video recovery.
probe: ReassemblyWindow,
}
impl Reassembler {
pub fn new(limits: ReassemblerLimits) -> Self {
Reassembler {
limits,
frames: HashMap::new(),
completed: HashSet::new(),
newest_frame: None,
video: ReassemblyWindow::default(),
probe: ReassemblyWindow::default(),
}
}
@@ -424,18 +482,28 @@ impl Reassembler {
}
let payload = pkt[HEADER_LEN..HEADER_LEN + shard_bytes].to_vec();
self.advance_window(hdr.frame_index, hdr.pts_ns, stats);
// Route by index space: speed-test probe filler (FLAG_PROBE in user_flags) reassembles in
// its own window so its indexes never interact with the video loss window — a probe burst
// can neither advance the video anchor nor be dropped as stale against it (and its aged-out
// frames never count as `frames_dropped`, which would fire video loss recovery).
let is_probe = hdr.user_flags & (FLAG_PROBE as u32) != 0;
let win = if is_probe {
&mut self.probe
} else {
&mut self.video
};
win.advance_window(hdr.frame_index, hdr.pts_ns, stats, !is_probe);
// Drop shards for frames we've already emitted (e.g. the recovery shards of a
// frame that completed early via the all-originals-present fast path) or that
// have fallen out of the loss window.
if self.completed.contains(&hdr.frame_index) || self.is_stale(hdr.frame_index, hdr.pts_ns) {
if win.completed.contains(&hdr.frame_index) || win.is_stale(hdr.frame_index, hdr.pts_ns) {
drop(stats);
return Ok(None);
}
// First packet of a frame establishes its geometry; later packets must agree.
let frame = self
let frame = win
.frames
.entry(hdr.frame_index)
.or_insert_with(|| FrameBuf {
@@ -521,8 +589,8 @@ impl Reassembler {
// Whole frame ready?
if frame.block_data.len() == frame.block_count {
let frame = self.frames.remove(&hdr.frame_index).unwrap();
self.completed.insert(hdr.frame_index);
let frame = win.frames.remove(&hdr.frame_index).unwrap();
win.completed.insert(hdr.frame_index);
// Reserve based on the bytes we actually hold, not the (already-bounded but
// still caller-supplied) frame_bytes, so a small frame can't over-reserve.
let actual: usize = frame.block_data.values().map(|b| b.len()).sum();
@@ -541,11 +609,30 @@ impl Reassembler {
Ok(None)
}
/// Drop all in-flight state — every partially-assembled frame and the completed/abandoned
/// index memory, in both index spaces — as if the session just started. Used by the client's
/// backlog flush ([`Session::flush_backlog`](crate::session::Session::flush_backlog)): after
/// the socket backlog is discarded wholesale, the partial frames here can never complete
/// (their remaining shards were just thrown away) and the window anchors (`newest_frame`)
/// point into the discarded past.
pub fn reset(&mut self) {
self.video = ReassemblyWindow::default();
self.probe = ReassemblyWindow::default();
}
}
impl ReassemblyWindow {
/// Track the newest frame, declare incomplete frames that fell out of the loss window
/// ([`LOSS_WINDOW_NS`] behind the newest pts, or [`HARD_LOSS_WINDOW`] indices) lost — counting
/// them dropped, which is what drives the client's recovery-keyframe request — and prune the
/// completed-index memory to [`REORDER_WINDOW`].
fn advance_window(&mut self, frame_index: u32, pts_ns: u64, stats: &StatsCounters) {
/// ([`LOSS_WINDOW_NS`] behind the newest pts, or [`HARD_LOSS_WINDOW`] indices) lost — for the
/// video window (`count_drops`) counting them dropped, which is what drives the client's
/// recovery-keyframe request — and prune the completed-index memory to [`REORDER_WINDOW`].
fn advance_window(
&mut self,
frame_index: u32,
pts_ns: u64,
stats: &StatsCounters,
count_drops: bool,
) {
let (newest, newest_pts) = match self.newest_frame {
// `frame_index` is newer iff it's within the forward half of the index space.
Some((n, p)) if frame_index.wrapping_sub(n) > u32::MAX / 2 => (n, p),
@@ -567,29 +654,17 @@ impl Reassembler {
keep
});
let pruned = before - self.frames.len();
if pruned > 0 {
if pruned > 0 && count_drops {
StatsCounters::add(&stats.frames_dropped, pruned as u64);
}
self.completed
.retain(|&idx| newest.wrapping_sub(idx) <= REORDER_WINDOW);
}
/// Drop all in-flight state — every partially-assembled frame and the completed/abandoned
/// index memory — as if the session just started. Used by the client's backlog flush
/// ([`Session::flush_backlog`](crate::session::Session::flush_backlog)): after the socket
/// backlog is discarded wholesale, the partial frames here can never complete (their remaining
/// shards were just thrown away) and the window anchor (`newest_frame`) points into the
/// discarded past.
pub fn reset(&mut self) {
self.frames.clear();
self.completed.clear();
self.newest_frame = None;
}
/// True if this packet's frame lies outside the loss window (behind the newest frame by more
/// than [`LOSS_WINDOW_NS`] of capture time or [`HARD_LOSS_WINDOW`] indices) — its shards
/// arrive too late to be useful, and accepting one would only create a frame buffer the next
/// [`advance_window`] immediately declares lost.
/// [`advance_window`](Self::advance_window) immediately declares lost.
fn is_stale(&self, frame_index: u32, pts_ns: u64) -> bool {
match self.newest_frame {
Some((n, newest_pts)) => {
@@ -769,6 +844,119 @@ mod tests {
assert_eq!(stats.snapshot().frames_dropped, 1, "no double-count");
}
/// The explicit-index path stamps the caller's `frame_index` and leaves the internal video
/// counter untouched — the punktfunk/1 encode loop owns the numbering, and mixing must not
/// perturb the legacy self-numbering path (tests/ABI/synthetic).
#[test]
fn explicit_frame_index_is_stamped_and_internal_counter_untouched() {
use crate::config::{FecConfig, FecScheme, ProtocolPhase, Role};
let cfg = Config {
role: Role::Host,
phase: ProtocolPhase::P2Punktfunk,
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 0,
max_data_per_block: 8,
},
shard_payload: 16,
max_frame_bytes: 4096,
encrypt: false,
key: [0u8; 16],
salt: [0u8; 4],
loopback_drop_period: 0,
};
let coder = coder_for(FecScheme::Gf16);
let mut pk = Packetizer::new(&cfg);
let mut seen = Vec::new();
pk.packetize_each(&[1u8; 16], 0, 0, Some(4242), coder.as_ref(), |hdr, _| {
seen.push(hdr.frame_index);
Ok(())
})
.unwrap();
assert_eq!(seen, vec![4242]);
// The legacy wrapper still numbers from the untouched internal counter.
let pkts = pk.packetize(&[1u8; 16], 0, 0, coder.as_ref()).unwrap();
let hdr = PacketHeader::read_from_bytes(&pkts[0][..HEADER_LEN]).unwrap();
assert_eq!(hdr.frame_index, 0);
// The probe space is a third, independent counter.
assert_eq!(pk.alloc_probe_index(), 0);
assert_eq!(pk.alloc_probe_index(), 1);
}
/// Probe filler (FLAG_PROBE in user_flags) reassembles in its OWN window: a probe frame whose
/// index is far behind the video stream's completes anyway (an old client's single window
/// would drop it as stale), and video frames complete undisturbed around it.
#[test]
fn probe_frames_reassemble_in_their_own_window() {
let mut r = Reassembler::new(limits());
let coder = coder_for(FecScheme::Gf8);
let stats = StatsCounters::default();
// Establish a video stream far into its index space.
let mut v = base_header();
v.frame_index = 100_000;
v.pts_ns = 1_000_000_000;
assert!(r
.push(&packet(v), coder.as_ref(), &stats)
.unwrap()
.is_some());
// A probe frame at index 0 — 100k "behind" the video window — must still complete.
let mut p = base_header();
p.frame_index = 0;
p.pts_ns = 1_000_000_100;
p.user_flags = FLAG_PROBE as u32;
let got = r.push(&packet(p), coder.as_ref(), &stats).unwrap();
assert!(got.is_some(), "probe frame must complete in its own window");
assert_eq!(got.unwrap().flags & FLAG_PROBE as u32, FLAG_PROBE as u32);
// The probe burst must not have advanced the VIDEO window: the next video frame is
// contiguous and completes, with nothing counted dropped.
let mut v2 = base_header();
v2.frame_index = 100_001;
v2.pts_ns = 1_000_000_200;
assert!(r
.push(&packet(v2), coder.as_ref(), &stats)
.unwrap()
.is_some());
assert_eq!(stats.snapshot().frames_dropped, 0);
}
/// An incomplete probe frame aging out of the probe window is NOT a video `frames_dropped`
/// (which would fire the client's loss recovery) — probe loss is measured bytes-wise by the
/// probe accumulator.
#[test]
fn aged_out_probe_frames_do_not_count_as_dropped() {
let mut r = Reassembler::new(limits());
let coder = coder_for(FecScheme::Gf8);
let stats = StatsCounters::default();
// Probe frame 0: one of two shards — incomplete.
let mut p = base_header();
p.user_flags = FLAG_PROBE as u32;
p.data_shards = 2;
p.frame_bytes = 32;
assert!(r
.push(&packet(p), coder.as_ref(), &stats)
.unwrap()
.is_none());
// A much newer probe frame ages it out of the probe window.
let mut p2 = base_header();
p2.user_flags = FLAG_PROBE as u32;
p2.frame_index = 1;
p2.pts_ns = LOSS_WINDOW_NS + 1;
assert!(r
.push(&packet(p2), coder.as_ref(), &stats)
.unwrap()
.is_some());
assert_eq!(
stats.snapshot().frames_dropped,
0,
"probe-window drops must not fire video loss recovery"
);
}
#[test]
fn rejects_wrong_shard_bytes_and_oversized_frame() {
let coder = coder_for(FecScheme::Gf8);
+12
View File
@@ -107,6 +107,18 @@ pub const VIDEO_CAP_444: u8 = 0x04;
/// host ignores it and simply never sends any); a client that doesn't set it keeps the combined
/// stage. Purely observability — never changes what the host encodes.
pub const VIDEO_CAP_HOST_TIMING: u8 = 0x08;
/// [`Hello::video_caps`] bit: the client's reassembler keeps **speed-test probe filler in its own
/// frame-index space** (a second reassembly window keyed on the [`crate::packet::FLAG_PROBE`]
/// user-flag), so probe bursts no longer consume video `frame_index`es. Without this, a mid-session
/// speed test burns thousands of video indexes that are invisible to every client-side gap detector
/// (probe frames are filtered before the pump sees them) — the first real AU afterwards reads as a
/// phantom multi-thousand-frame loss (spurious freeze + a nonsense RFI). It also lets the host's
/// encode loop own the video numbering outright (the wire-index contract
/// [`crate::packet::Packetizer::packetize_each`] documents), which reference-frame invalidation
/// depends on. The host runs mid-session probe bursts ONLY against clients that set this bit — an
/// older client gets a declined (zeroed) [`ProbeResult`] instead of a measurement its single-window
/// reassembler would silently drop as stale.
pub const VIDEO_CAP_PROBE_SEQ: u8 = 0x10;
/// QUIC application error code a punktfunk/1 client closes the control connection with on a
/// **deliberate quit** (a user "stop", not a network drop). The host reads it off the connection's
+69 -26
View File
@@ -161,6 +161,31 @@ impl Session {
data: &[u8],
pts_ns: u64,
user_flags: u32,
) -> Result<Vec<Vec<u8>>> {
self.seal_frame_inner(data, pts_ns, user_flags, None)
}
/// [`seal_frame`](Self::seal_frame) with the caller's **explicit** `frame_index` instead of the
/// packetizer's internal counter. The punktfunk/1 encode loop owns the video numbering (one
/// session-lifetime counter, stamped per AU) so the encoder's reference-frame-invalidation
/// bookkeeping stays 1:1 with the wire across encoder rebuilds/resets — see
/// [`Packetizer::packetize_each`]. A session must use ONE numbering style per index space.
pub fn seal_frame_at(
&mut self,
data: &[u8],
pts_ns: u64,
user_flags: u32,
frame_index: u32,
) -> Result<Vec<Vec<u8>>> {
self.seal_frame_inner(data, pts_ns, user_flags, Some(frame_index))
}
fn seal_frame_inner(
&mut self,
data: &[u8],
pts_ns: u64,
user_flags: u32,
frame_index: Option<u32>,
) -> Result<Vec<Vec<u8>>> {
if self.config.role != Role::Host {
return Err(PunktfunkError::InvalidArg(
@@ -184,35 +209,36 @@ impl Session {
} = self;
let mut wires = std::mem::take(wire_pool);
let mut used = 0usize;
let result = packetizer.packetize_each(data, pts_ns, user_flags, coder.as_ref(), {
let wires = &mut wires;
let used = &mut used;
move |hdr, body| {
if *used == wires.len() {
wires.push(Vec::new());
}
let wire = &mut wires[*used];
*used += 1;
let seq = *next_seq;
*next_seq = next_seq.wrapping_add(1);
wire.clear();
match crypto {
Some(c) => {
// seq(8) ‖ header(40) ‖ shard ‖ tag scratch(16), sealed over [8..].
wire.extend_from_slice(&seq.to_be_bytes());
wire.extend_from_slice(hdr.as_bytes());
wire.extend_from_slice(body);
wire.resize(wire.len() + crate::crypto::TAG_LEN, 0);
c.seal_in_place(seq, &mut wire[8..])?;
let result =
packetizer.packetize_each(data, pts_ns, user_flags, frame_index, coder.as_ref(), {
let wires = &mut wires;
let used = &mut used;
move |hdr, body| {
if *used == wires.len() {
wires.push(Vec::new());
}
None => {
wire.extend_from_slice(hdr.as_bytes());
wire.extend_from_slice(body);
let wire = &mut wires[*used];
*used += 1;
let seq = *next_seq;
*next_seq = next_seq.wrapping_add(1);
wire.clear();
match crypto {
Some(c) => {
// seq(8) ‖ header(40) ‖ shard ‖ tag scratch(16), sealed over [8..].
wire.extend_from_slice(&seq.to_be_bytes());
wire.extend_from_slice(hdr.as_bytes());
wire.extend_from_slice(body);
wire.resize(wire.len() + crate::crypto::TAG_LEN, 0);
c.seal_in_place(seq, &mut wire[8..])?;
}
None => {
wire.extend_from_slice(hdr.as_bytes());
wire.extend_from_slice(body);
}
}
Ok(())
}
Ok(())
}
});
});
result?;
// A smaller frame uses fewer buffers than the pool holds: drop the unused tail, same
// as the previous `resize_with(packets.len(), ..)` did.
@@ -258,6 +284,23 @@ impl Session {
r.map(|_| ())
}
/// Host: seal + send one **speed-test probe filler** access unit in the probe index space
/// (its own frame counter + the [`crate::packet::FLAG_PROBE`] user-flag) so a burst never
/// consumes video `frame_index`es — the client reassembles probe frames in a separate window
/// and its gap detectors never see them. Only call this against a client that advertised
/// [`crate::quic::VIDEO_CAP_PROBE_SEQ`]; an older client's single-window reassembler would
/// drop probe-space indexes as stale against the video stream.
pub fn submit_probe_frame(&mut self, data: &[u8], pts_ns: u64) -> Result<()> {
let idx = self.packetizer.alloc_probe_index();
let wires =
self.seal_frame_inner(data, pts_ns, crate::packet::FLAG_PROBE as u32, Some(idx))?;
let refs: Vec<&[u8]> = wires.iter().map(|w| w.as_slice()).collect();
let r = self.send_sealed(&refs);
drop(refs);
self.reclaim_wires(wires);
r.map(|_| ())
}
/// Host: live-adjust the FEC recovery percentage (adaptive FEC). Affects the next
/// [`submit_frame`](Self::submit_frame)/[`seal_frame`](Self::seal_frame); the receiver needs no
/// notification (each packet's header carries its block's data/recovery shard counts).
+37 -13
View File
@@ -21,10 +21,13 @@ pub struct EncodedFrame {
pub keyframe: bool,
/// True when this AU is a **reference-frame-invalidation recovery frame** — a clean P-frame the
/// encoder coded against a known-good reference in response to
/// [`invalidate_ref_frames`](Encoder::invalidate_ref_frames) (AMD LTR force-reference). The pump
/// tags it [`punktfunk_core::packet::USER_FLAG_RECOVERY_ANCHOR`] so the client lifts its post-loss
/// freeze on it without an IDR. Only the native-AMF LTR path sets it; every other backend leaves
/// it `false` (their RFI, when present, re-references transparently with no distinct clean-point AU).
/// [`invalidate_ref_frames`](Encoder::invalidate_ref_frames). The pump tags it
/// [`punktfunk_core::packet::USER_FLAG_RECOVERY_ANCHOR`] so the client lifts its post-loss
/// freeze on it without an IDR. Set by BOTH RFI backends: native AMF (the LTR force-reference
/// frame) and Windows direct-NVENC (the first frame encoded after `nvEncInvalidateRefFrames` —
/// the invalidation applies at the next `encode_picture`, so that AU is by construction the
/// clean re-anchor). Without it the client's freeze can only lift on an IDR — which the host
/// suppresses after a successful RFI (the cooldown), a ~1 s frozen stall per loss event.
pub recovery_anchor: bool,
}
@@ -201,8 +204,9 @@ pub struct EncoderCaps {
/// The encoder can perform real reference-frame invalidation — i.e.
/// [`invalidate_ref_frames`](Encoder::invalidate_ref_frames) can return `true`. When `false`
/// the caller skips that always-`false` call and forces a keyframe directly on loss recovery.
/// Only the Windows direct-NVENC path implements RFI; libavcodec (Linux NVENC), VAAPI and
/// AMF/QSV always keyframe.
/// Two backends implement RFI: Windows direct-NVENC (`nvEncInvalidateRefFrames`) and native
/// AMF (user-LTR force-reference, when the driver accepted the LTR slots at open). The
/// libavcodec paths (Linux NVENC, VAAPI, QSV) can't express it and always keyframe.
pub supports_rfi: bool,
/// The encoder emits in-band HDR mastering/CLL SEI from [`set_hdr_meta`](Encoder::set_hdr_meta).
/// When `false`, `set_hdr_meta` is a no-op and no in-band grade reaches the client. Only the
@@ -242,6 +246,22 @@ pub struct EncoderCaps {
/// A hardware encoder. One per session; runs on the encode thread.
pub trait Encoder: Send {
fn submit(&mut self, frame: &CapturedFrame) -> Result<()>;
/// [`submit`](Self::submit) with the **wire frame index** this frame's AU will carry — the
/// number the packetizer stamps on it and the client's loss reports/RFI requests name. The
/// session glue predicts it exactly as `AUs sent so far + frames in flight` (AUs are emitted
/// FIFO, one per submission; anything that would break the prediction — an in-place reset, a
/// device-change teardown, an encoder rebuild — forfeits the in-flight frames on BOTH sides
/// and clears the encoder's reference state, so stale predictions die with it). The RFI
/// backends pin their frame numbering (LTR marks, DPB timestamps) to this so
/// [`invalidate_ref_frames`](Self::invalidate_ref_frames) compares client frame numbers
/// against the same domain — an encoder-internal counter desyncs from the wire on the first
/// mid-stream rebuild (adaptive bitrate steps do this under congestion, exactly when losses
/// happen). Default: ignore the index and delegate to `submit` (backends without per-frame
/// reference bookkeeping don't care).
fn submit_indexed(&mut self, frame: &CapturedFrame, wire_index: u32) -> Result<()> {
let _ = wire_index;
self.submit(frame)
}
/// This encoder's static [capabilities](EncoderCaps) (RFI, HDR SEI), so the session glue can
/// route by query rather than rely on the no-op/`false` defaults of
/// [`invalidate_ref_frames`](Self::invalidate_ref_frames) / [`set_hdr_meta`](Self::set_hdr_meta).
@@ -259,13 +279,14 @@ pub trait Encoder: Send {
/// Default: no-op (SDR encoders / libavcodec paths that don't attach it yet). Cheap to call
/// every frame; only the direct-NVENC path consumes it.
fn set_hdr_meta(&mut self, _meta: Option<punktfunk_core::quic::HdrMeta>) {}
/// Invalidate a contiguous range of previously-encoded reference frames (client frame numbers,
/// as reported in a loss-recovery request) so the encoder re-references an older still-valid
/// frame instead of emitting a full IDR. Returns `true` if a real reference invalidation was
/// performed; `false` means the encoder couldn't (range older than the DPB, or the backend has
/// no RFI) and the caller should fall back to [`request_keyframe`](Self::request_keyframe).
/// Default: `false` — only the Windows direct-NVENC path implements true RFI; libavcodec
/// (Linux NVENC) and VAAPI can't express `nvEncInvalidateRefFrames`, so they keyframe.
/// Invalidate a contiguous range of previously-encoded reference frames (client frame numbers
/// — WIRE frame indexes, the domain [`submit_indexed`](Self::submit_indexed) pins the encoder's
/// bookkeeping to) so the encoder re-references an older still-valid frame instead of emitting
/// a full IDR. Returns `true` if a real reference invalidation was performed; `false` means the
/// encoder couldn't (range older than the DPB/LTR history, or the backend has no RFI) and the
/// caller should fall back to [`request_keyframe`](Self::request_keyframe). Default: `false` —
/// the Windows direct-NVENC path (`nvEncInvalidateRefFrames`) and native AMF (LTR
/// force-reference) implement true RFI; the libavcodec paths can't express it, so they keyframe.
fn invalidate_ref_frames(&mut self, _first_frame: i64, _last_frame: i64) -> bool {
false
}
@@ -441,6 +462,9 @@ impl Encoder for TrackedEncoder {
fn submit(&mut self, frame: &CapturedFrame) -> Result<()> {
self.inner.submit(frame)
}
fn submit_indexed(&mut self, frame: &CapturedFrame, wire_index: u32) -> Result<()> {
self.inner.submit_indexed(frame, wire_index)
}
fn caps(&self) -> EncoderCaps {
self.inner.caps()
}
@@ -149,6 +149,22 @@ fn nvenc_input(format: PixelFormat) -> (Pixel, bool) {
}
}
/// The [`NvencEncoder::open`] arguments, kept on the encoder so [`Encoder::reset`] can rebuild it
/// in place with the session's negotiated parameters — the encode-stall watchdog's recovery lever
/// (drop the wedged libavcodec encoder, reopen fresh, forfeit the owed AUs, restart at an IDR).
#[derive(Clone, Copy)]
struct OpenArgs {
codec: Codec,
format: PixelFormat,
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
}
pub struct NvencEncoder {
enc: encoder::video::Encoder,
/// Reusable 4-bpp CPU input frame (CPU path only; `None` for the zero-copy/CUDA path).
@@ -181,6 +197,8 @@ pub struct NvencEncoder {
/// open so the pump's per-AU `caps()` doesn't re-read `PUNKTFUNK_IR_PERIOD_FRAMES`; the pump marks
/// every Nth AU with `USER_FLAG_RECOVERY_POINT` for the client's clean re-anchor.
intra_refresh_period: u32,
/// The open arguments, for the in-place [`reset`](Encoder::reset) rebuild.
args: OpenArgs,
}
// `CudaHw` holds raw `AVBufferRef`s and `sws_444` a raw `SwsContext`; the encoder lives on a single
@@ -534,6 +552,17 @@ impl NvencEncoder {
} else {
0
},
args: OpenArgs {
codec,
format,
width,
height,
fps,
bitrate_bps,
cuda,
bit_depth,
chroma,
},
})
}
}
@@ -582,6 +611,35 @@ impl Encoder for NvencEncoder {
self.force_kf = true;
}
/// Encode-stall recovery: drop the wedged libavcodec encoder and reopen it fresh with the
/// session's negotiated parameters (the stored [`OpenArgs`]) — the drop-and-reopen lever the
/// QSV/VAAPI paths use, so the encode-stall watchdog can heal a wedged NVENC/driver instead of
/// ending the session. Owed AUs are forfeited; the fresh encoder opens on an IDR.
fn reset(&mut self) -> bool {
let a = self.args;
match Self::open(
a.codec,
a.format,
a.width,
a.height,
a.fps,
a.bitrate_bps,
a.cuda,
a.bit_depth,
a.chroma,
) {
Ok(mut fresh) => {
fresh.force_kf = true;
*self = fresh; // drops the wedged encoder (frees its contexts) in the same step
true
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), "NVENC in-place reopen failed");
false
}
}
}
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
let mut pkt = Packet::empty();
match self.enc.receive_packet(&mut pkt) {
@@ -1126,6 +1126,19 @@ impl Encoder for VaapiEncoder {
self.force_kf = true;
}
/// Encode-stall recovery: drop the wedged libavcodec encoder (its `Drop` releases the VA
/// surfaces/filter graph/devices) and let the next `submit` rebuild it lazily from the first
/// frame's payload, exactly like first-frame bring-up — the same drop-and-reopen lever the
/// Windows QSV path has. The owed AUs are forfeited (`in_flight` zeroed) and the rebuilt
/// encoder's first frame is forced IDR so the client resyncs immediately. Without this the
/// encode-stall watchdog had no lever on Linux AMD/Intel and a wedged driver ended the session.
fn reset(&mut self) -> bool {
self.inner = None;
self.in_flight = 0;
self.force_kf = true;
true
}
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
// With `async_depth > 1`, `submit` no longer waits for the ASIC — the AU for the frame we
// just sent lands ~one hardware-encode-time later. Wait for it (bounded) so it still ships
+9 -5
View File
@@ -20,6 +20,7 @@ use openh264::encoder::{
};
use openh264::formats::YUVSlices;
use openh264::OpenH264API;
use std::collections::VecDeque;
pub struct OpenH264Encoder {
enc: Oh264,
@@ -34,8 +35,11 @@ pub struct OpenH264Encoder {
v_plane: Vec<u8>,
frame_idx: i64,
force_kf: bool,
/// At most one AU per submit (no lookahead), handed back by the next `poll`.
pending: Option<EncodedFrame>,
/// One AU per submit (no lookahead), handed back FIFO by `poll`. A queue, not an `Option`:
/// the session loop pipelines up to `capturer.pipeline_depth()` submits before polling, and a
/// single-slot pending would silently overwrite (lose) the older AUs — including the opening
/// IDR — and permanently skew the loop's FIFO pts pairing.
pending: VecDeque<EncodedFrame>,
}
// openh264's Encoder holds a raw C handle (not auto-Send); it lives on the single encode thread.
@@ -88,7 +92,7 @@ impl OpenH264Encoder {
v_plane: vec![0; (w / 2) * (h / 2)],
frame_idx: 0,
force_kf: false,
pending: None,
pending: VecDeque::new(),
})
}
@@ -207,7 +211,7 @@ impl Encoder for OpenH264Encoder {
if !data.is_empty() {
let keyframe = matches!(bs.frame_type(), FrameType::IDR | FrameType::I);
let pts_ns = self.frame_idx as u64 * 1_000_000_000 / self.fps.max(1) as u64;
self.pending = Some(EncodedFrame {
self.pending.push_back(EncodedFrame {
data,
pts_ns,
keyframe,
@@ -223,7 +227,7 @@ impl Encoder for OpenH264Encoder {
}
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
Ok(self.pending.take())
Ok(self.pending.pop_front())
}
fn flush(&mut self) -> Result<()> {
@@ -1840,7 +1840,12 @@ impl Encoder for AmfEncoder {
);
self.ensure_inner(&frame.device)?;
let cur_idx = self.frame_idx;
let forced = std::mem::take(&mut self.force_kf) || self.frame_idx == 0;
// A component's FIRST submission must be a forced IDR (stream-start contract: in-band
// headers + LTR re-anchor). Detected via the fresh ring counter, NOT `frame_idx == 0`:
// `submit_indexed` pins frame_idx to the wire index, which is non-zero when a mid-session
// rebuild (bitrate step / reset escalation) brings a new component up.
let opening = self.inner.as_ref().is_none_or(|i| i.next == 0);
let forced = std::mem::take(&mut self.force_kf) || opening;
let pts_100ns = self.frame_idx * 10_000_000 / self.fps.max(1) as i64;
self.frame_idx += 1;
// --- LTR-RFI per-frame decisions (design: the AMD twin of NVENC intra-refresh recovery) ---
@@ -2118,6 +2123,21 @@ impl Encoder for AmfEncoder {
Ok(())
}
/// Pin this submission's frame number to the wire frame index its AU will carry (see the
/// trait doc): the LTR slots then store WIRE indexes, so [`invalidate_ref_frames`]'s
/// pre-loss check (`slot < first`, both in client frame numbers) stays correct across every
/// encoder rebuild/reset — an internal counter desyncs on the first adaptive-bitrate rebuild,
/// making the check vacuously true and risking a force-reference to an LTR marked INSIDE the
/// lost range (a corrupted frame shipped as a clean recovery anchor). `frame_idx` also feeds
/// the AMF SetPts; a re-pin only ever moves it backward across a reset (fresh component, so a
/// pts restart is harmless) and forward on a rebuild (monotonic within any one component).
///
/// [`invalidate_ref_frames`]: Encoder::invalidate_ref_frames
fn submit_indexed(&mut self, frame: &CapturedFrame, wire_index: u32) -> Result<()> {
self.frame_idx = wire_index as i64;
self.submit(frame)
}
fn request_keyframe(&mut self) {
self.force_kf = true;
}
@@ -2145,8 +2165,10 @@ impl Encoder for AmfEncoder {
}
// Pick the newest LTR strictly OLDER than the loss: the most recent known-good reference the
// client still holds, so re-referencing it costs the least (smallest recovery-frame residual).
// Frame numbers are 1:1 with the client's (both count submissions in order — see the NVENC
// path), so `ltr_slots` (which store `frame_idx`) compare directly against `first`.
// `ltr_slots` store the WIRE frame index of the marked frame (`submit_indexed` pins
// `frame_idx` to it per submission), so they compare directly against the client's `first`
// — and stay comparable across encoder rebuilds/resets, where an internal counter would
// make this check vacuous and risk force-referencing an LTR marked INSIDE the lost range.
let mut best: Option<(usize, i64)> = None;
for (slot, marked) in self.ltr_slots.iter().enumerate() {
if let Some(idx) = *marked {
@@ -429,10 +429,25 @@ pub struct NvencD3d11Encoder {
async_rt: Option<AsyncRetrieve>,
/// `NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT` from the caps probe — gates the async retrieve mode.
async_supported: bool,
/// (bitstream, mapped input resource to unmap after retrieval, pts_ns) per in-flight encode.
pending: VecDeque<(nv::NV_ENC_OUTPUT_PTR, nv::NV_ENC_INPUT_PTR, u64)>,
/// (bitstream, mapped input resource to unmap after retrieval, pts_ns, recovery-anchor) per
/// in-flight encode. The fourth field tags the first frame encoded after a successful
/// [`invalidate_ref_frames`](Encoder::invalidate_ref_frames) — the clean re-anchor P-frame the
/// client lifts its post-loss freeze on (see [`EncodedFrame::recovery_anchor`]).
pending: VecDeque<(nv::NV_ENC_OUTPUT_PTR, nv::NV_ENC_INPUT_PTR, u64, bool)>,
/// The frame number of the NEXT submission (also its `inputTimeStamp`). Pinned per frame by
/// [`Encoder::submit_indexed`] to the WIRE frame index the AU will carry, so the DPB timestamps
/// `invalidate_ref_frames` compares client frame numbers against stay 1:1 with the wire across
/// encoder rebuilds/resets (an internal counter desyncs on the first adaptive-bitrate rebuild —
/// RFI then never matches again). Self-increments as a fallback for un-indexed callers (tests).
frame_idx: i64,
force_kf: bool,
/// A successful [`invalidate_ref_frames`](Encoder::invalidate_ref_frames) arms this; the next
/// `submit` consumes it into `pending` so that AU ships as the recovery anchor. NVENC applies
/// the invalidation at the next `encode_picture`, so that frame is by construction the first
/// one coded against only-valid references — without tagging it the client's freeze can only
/// lift on an IDR, which the session glue suppresses after an RFI success (the cooldown):
/// a ~1 s frozen stall per loss event on NVIDIA hosts.
pending_anchor: bool,
inited: bool,
/// GPU capabilities probed once via `nvEncGetEncodeCaps` before configuring (Apollo's
/// `get_encoder_cap`): gates 10-bit/custom-VBV/RFI on what this card actually supports instead
@@ -507,6 +522,7 @@ impl NvencD3d11Encoder {
pending: VecDeque::new(),
frame_idx: 0,
force_kf: false,
pending_anchor: false,
inited: false,
rfi_supported: false,
custom_vbv: false,
@@ -536,7 +552,7 @@ impl NvencD3d11Encoder {
while rt.done_rx.try_recv().is_ok() {}
}
// Unmap any in-flight inputs, then unregister every cached texture and destroy the bitstreams.
for (_, map, _) in &self.pending {
for (_, map, _, _) in &self.pending {
if !map.is_null() {
let _ = (api().unmap_input_resource)(self.encoder, *map);
}
@@ -569,8 +585,10 @@ impl NvencD3d11Encoder {
self.inited = false;
self.next = 0;
// The new session starts with an empty DPB (its first frame is an IDR), so any prior
// invalidation range is meaningless against it.
// invalidation range is meaningless against it — and the IDR is itself the re-anchor,
// so a pending anchor tag from a pre-teardown RFI is stale too.
self.last_rfi_range = None;
self.pending_anchor = false;
}
/// Query one `NV_ENC_CAPS` value for this codec on an open session; 0 on any error (treat an
@@ -1133,7 +1151,7 @@ impl NvencD3d11Encoder {
/// error surfaces AFTER the unmap (the resource is retired either way) so the session glue's
/// rebuild path starts from clean state.
fn absorb_done(&mut self, done: RetrieveDone) -> Result<()> {
let Some((bs, map, pts_ns)) = self.pending.pop_front() else {
let Some((bs, map, pts_ns, anchor)) = self.pending.pop_front() else {
bail!("NVENC async: completion with no in-flight frame (pairing bug)");
};
if bs as usize != done.bs {
@@ -1157,7 +1175,7 @@ impl NvencD3d11Encoder {
data,
pts_ns,
keyframe,
recovery_anchor: false,
recovery_anchor: anchor,
});
Ok(())
}
@@ -1249,6 +1267,11 @@ impl Encoder for NvencD3d11Encoder {
self.init_session(&device)?;
self.init_device = dev_raw;
}
// The session's opening frame — NVENC emits it as an IDR regardless of pic flags, so the
// in-band HDR SEI must ride it too. Detected via the still-empty output slot counter
// (`teardown` zeroes it), NOT via `pts == 0`: `submit_indexed` pins pts to the wire frame
// index, which is non-zero on a mid-session encoder rebuild's first frame.
let opening = self.next == 0;
// Async backpressure: never hand NVENC an output bitstream that is still in flight, and
// keep in-flight depth within the capturer's texture ring (see `async_inflight_cap`). At
// the cap, block on the OLDEST completion (the retrieve thread is already waiting on its
@@ -1323,6 +1346,10 @@ impl Encoder for NvencD3d11Encoder {
} else {
0
};
// Recovery anchor (armed by a successful invalidate_ref_frames): THIS frame is the
// first one encoded after the invalidation — the clean re-anchor. A simultaneous
// forced IDR is itself the re-anchor, so the tag is dropped in that case.
let anchor = std::mem::take(&mut self.pending_anchor) && flags == 0;
let mut pic = nv::NV_ENC_PIC_PARAMS {
version: nv::NV_ENC_PIC_PARAMS_VER,
inputWidth: self.width,
@@ -1349,7 +1376,7 @@ impl Encoder for NvencD3d11Encoder {
// built from the source display's metadata. Any decoder — incl. stock Moonlight — then
// tone-maps from the real grade. HEVC/H.264 carry SEI; AV1 uses metadata OBUs (follow-up).
// The scratch buffers must outlive `encode_picture`, so they live in this scope.
let is_idr = flags != 0 || pts == 0;
let is_idr = flags != 0 || opening;
let mastering_sei = self
.hdr_meta
.map(|m| crate::hdr::hevc_mastering_display_sei(&m));
@@ -1391,8 +1418,12 @@ impl Encoder for NvencD3d11Encoder {
(api().encode_picture)(self.encoder, &mut pic)
.nv_ok()
.map_err(|e| anyhow!("encode_picture: {e:?}"))?;
self.pending
.push_back((self.bitstreams[slot], mp.mappedResource, captured.pts_ns));
self.pending.push_back((
self.bitstreams[slot],
mp.mappedResource,
captured.pts_ns,
anchor,
));
// Async: hand the in-flight encode to the retrieve thread (channel capacity = POOL ≥
// in-flight, so this send never blocks). The pending entry above pairs with its
// completion FIFO in `absorb_done`.
@@ -1409,6 +1440,16 @@ impl Encoder for NvencD3d11Encoder {
Ok(())
}
/// Pin this submission's frame number (= its `inputTimeStamp`) to the wire frame index the AU
/// will carry, so the DPB timestamps `invalidate_ref_frames` matches client frame numbers
/// against are the wire's — 1:1 across every rebuild/reset (see the trait doc). Within a
/// session the loop's prediction is nondecreasing; a repeat after a reset lands on a fresh
/// session (teardown cleared the DPB and `last_rfi_range`), so re-pinning is always sound.
fn submit_indexed(&mut self, frame: &CapturedFrame, wire_index: u32) -> Result<()> {
self.frame_idx = wire_index as i64;
self.submit(frame)
}
fn request_keyframe(&mut self) {
self.force_kf = true;
}
@@ -1442,9 +1483,13 @@ impl Encoder for NvencD3d11Encoder {
if self.encoder.is_null() || !self.rfi_supported || first < 0 || first > last {
return false;
}
// Already invalidated a covering range for this loss event — nothing more to do, no IDR.
// Already invalidated a covering range for this loss event — no new driver calls needed,
// no IDR. RE-ARM the anchor though: the client re-asking means the previous recovery
// anchor AU may itself have been lost, and the next frame is just as clean a re-anchor
// (it too references only valid frames).
if let Some((pf, pl)) = self.last_rfi_range {
if first >= pf && last <= pl {
self.pending_anchor = true;
return true;
}
}
@@ -1460,9 +1505,11 @@ impl Encoder for NvencD3d11Encoder {
if first > last {
return false;
}
// We tag each input with `inputTimeStamp = frame_idx` (0,1,2,…), which is also the client's
// frame number (the packetizer numbers frames in submit order), so the client's lost-frame
// range maps 1:1 onto the timestamps NVENC invalidates here.
// Each input's `inputTimeStamp` is `frame_idx`, which `submit_indexed` pins to the WIRE
// frame index the AU carries — so the client's lost-frame range maps 1:1 onto the
// timestamps NVENC invalidates here, and stays 1:1 across encoder rebuilds/resets (an
// internal counter would desync on the first adaptive-bitrate rebuild and RFI would then
// clamp every range into first > last, silently degrading to IDR-only forever).
// SAFETY: `invalidate_ref_frames` is a function pointer from the runtime-loaded `EncodeApi` table.
// `self.encoder` was checked non-null at the top of this fn and is the live session; this runs
// on the encode thread (like submit/poll), so there is no concurrent NVENC use. Each `ts` was
@@ -1480,6 +1527,11 @@ impl Encoder for NvencD3d11Encoder {
}
}
self.last_rfi_range = Some((first, last));
// The next submitted frame is the first one encoded after the invalidation — the clean
// re-anchor P-frame. Arm the tag so its AU ships with `recovery_anchor` and the client
// lifts its post-loss freeze on it (instead of waiting ~1 s for the cooldown-suppressed
// IDR fallback).
self.pending_anchor = true;
true
}
@@ -1504,7 +1556,7 @@ impl Encoder for NvencD3d11Encoder {
.ready
.pop_front());
}
let Some((bs, map, pts_ns)) = self.pending.pop_front() else {
let Some((bs, map, pts_ns, anchor)) = self.pending.pop_front() else {
return Ok(None);
};
// SAFETY: a non-empty `pending` implies `submit` ran, so `self.encoder` is the live session
@@ -1545,11 +1597,28 @@ impl Encoder for NvencD3d11Encoder {
data,
pts_ns,
keyframe,
recovery_anchor: false,
recovery_anchor: anchor,
}))
}
}
/// Encode-stall recovery: tear the whole session down (the same teardown a capture-device
/// change uses) and let the next `submit` rebuild it lazily on the current device — the owed
/// AUs are forfeited and the fresh session opens on an IDR. Gives the encode-stall watchdog a
/// healing lever on NVENC instead of ending the session. Caveat: the SYNC retrieve mode blocks
/// inside `lock_bitstream`, so a driver wedge that hangs the lock never returns to the loop
/// for the watchdog to fire — this lever fully protects the async retrieve mode (5 s event
/// timeouts surface as poll errors) and the submit-side failure paths.
fn reset(&mut self) -> bool {
// SAFETY: `teardown` (an `unsafe fn`) requires the encode thread with no NVENC call in
// flight and a session whose cached resources belong to `self.encoder` — all hold here
// (reset is called from the session loop between submit/poll, like every other method),
// and it early-returns on an already-null session.
unsafe { self.teardown() };
self.force_kf = true;
true
}
fn flush(&mut self) -> Result<()> {
Ok(()) // P1/ULL + frameIntervalP=1: each submit yields its AU; no internal queue to drain.
}
+39 -8
View File
@@ -436,7 +436,10 @@ fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
/// behind encode (measured ~3 ms/frame at 4K, which capped GameStream's frame rate well below what
/// the encoder alone can sustain).
struct RawFrame {
aus: Vec<(Vec<u8>, FrameType)>,
/// `(bitstream, type, wire frameIndex)` per AU. The stream loop assigns the index (it owns
/// the numbering — see its `au_seq`), so the encoder's RFI bookkeeping stays 1:1 with what
/// Moonlight sees across mid-stream encoder rebuilds.
aus: Vec<(Vec<u8>, FrameType, u32)>,
ts: u32,
}
@@ -460,8 +463,8 @@ fn spawn_packetizer(
crate::punktfunk1::boost_thread_priority(false);
while let Ok(frame) = rx.recv() {
let mut batch: PacketBatch = Vec::new();
for (au, ft) in frame.aus {
batch.extend(pk.packetize(&au, ft, frame.ts));
for (au, ft, idx) in frame.aus {
batch.extend(pk.packetize(&au, ft, frame.ts, Some(idx)));
}
if batch.is_empty() {
continue;
@@ -660,6 +663,16 @@ fn stream_body(
// routed through the same coalesce gate as client IDR requests so a burst of drops (congestion)
// can't become an IDR storm.
let mut recover_after_drop = false;
// The stream's wire frameIndex numbering, owned HERE (the index of the next AU handed to the
// packetizer thread; a dropped-at-the-queue frame consumes none). A submission's future index
// is `au_seq + enc_inflight` (AUs are emitted FIFO, one per submission); passing it to
// `Encoder::submit_indexed` keeps the encoder's RFI bookkeeping 1:1 with Moonlight's frame
// numbers across the in-place encoder rebuild above (an internal counter would desync there).
// A pipeline-head drop desyncs the prediction by the dropped AU count for the frames already
// in flight — bounded and self-healing: the drop arms `recover_after_drop`, whose forced IDR
// resets the encoder's reference state (stale LTR/DPB bookkeeping dies with it).
let mut au_seq: u32 = 0;
let mut enc_inflight: u32 = 0;
while running.load(Ordering::SeqCst) {
let tick = Instant::now();
@@ -728,6 +741,10 @@ fn stream_body(
enc.request_keyframe();
last_keyframe = Some(Instant::now());
next_frame = Instant::now();
// The old encoder died with its in-flight submissions — their AUs will never
// arrive, so the numbering prediction restarts at `au_seq` (the fresh encoder's
// reference state is empty, so the reused predictions meet no stale bookkeeping).
enc_inflight = 0;
tracing::info!("gamestream: source rebuilt — stream continues");
continue;
}
@@ -742,7 +759,13 @@ fn stream_body(
if let Some((first, last)) = rfi_range.lock().unwrap().take() {
// Prefer reference-frame invalidation when the encoder supports it (no costly IDR
// spike); otherwise — or if the range is too old to invalidate — fall back to a keyframe.
if !(supports_rfi && enc.invalidate_ref_frames(first, last)) {
// Sanity-cap the range first: wider than RFI_MAX_RANGE exceeds any encoder's reference
// history (or is a phantom range from a desynced counter) — keyframe, never a
// force-reference that could ship corruption as a clean frame.
let width = (last as u32).wrapping_sub(first as u32);
if width > punktfunk_core::packet::RFI_MAX_RANGE
|| !(supports_rfi && enc.invalidate_ref_frames(first, last))
{
want_keyframe = true;
}
}
@@ -766,21 +789,27 @@ fn stream_body(
tracing::debug!("video: keyframe request coalesced (IDR still in flight)");
}
}
enc.submit(&frame).context("encoder submit")?;
enc.submit_indexed(&frame, au_seq.wrapping_add(enc_inflight))
.context("encoder submit")?;
enc_inflight = enc_inflight.wrapping_add(1);
let t_enc = tick.elapsed();
// 90 kHz RTP timestamp from wall-clock, so a variable capture rate stays correct.
let ts = (stream_start.elapsed().as_secs_f64() * 90_000.0) as u32;
// Drain the encoder's access units (owned buffers) — FEC/packetization runs on the
// packetizer thread, off this loop, so it never serializes behind encode.
let mut aus: Vec<(Vec<u8>, FrameType)> = Vec::new();
// packetizer thread, off this loop, so it never serializes behind encode. Each AU is
// stamped with its wire frameIndex here (`au_seq + position`); the numbering only
// ADVANCES if the batch is actually enqueued below (a dropped batch consumes none).
let mut aus: Vec<(Vec<u8>, FrameType, u32)> = Vec::new();
while let Some(au) = enc.poll().context("encoder poll")? {
let ft = if au.keyframe {
FrameType::Idr
} else {
FrameType::P
};
aus.push((au.data, ft));
let idx = au_seq.wrapping_add(aus.len() as u32);
aus.push((au.data, ft, idx));
enc_inflight = enc_inflight.saturating_sub(1);
}
let t_pkt = tick.elapsed();
@@ -788,9 +817,11 @@ fn stream_body(
// (packetizer, or the paced sender behind it) is behind — drop this frame (FEC/RFI covers the
// client) and keep encoding, so a downstream stall can never cap the encode rate.
if !aus.is_empty() {
let batch_len = aus.len() as u32;
match raw_tx.try_send(RawFrame { aus, ts }) {
Ok(()) => {
sent_batches += 1;
au_seq = au_seq.wrapping_add(batch_len);
}
Err(std::sync::mpsc::TrySendError::Full(_)) => {
dropped_batches += 1;
+15 -7
View File
@@ -69,14 +69,22 @@ impl VideoPacketizer {
}
/// Packetize one encoded AU into wire datagrams (data shards + Cauchy RS parity shards).
///
/// `frame_index`: `Some(i)` stamps the caller's index (the stream loop owns the numbering so
/// the encoder's RFI bookkeeping stays 1:1 with the wire across mid-stream encoder rebuilds —
/// see `Encoder::submit_indexed`); `None` draws from the internal counter (tests/harnesses).
pub fn packetize(
&mut self,
au: &[u8],
frame_type: FrameType,
timestamp_90k: u32,
frame_index: Option<u32>,
) -> Vec<Vec<u8>> {
let frame_index = self.frame_index;
self.frame_index = self.frame_index.wrapping_add(1);
let frame_index = frame_index.unwrap_or_else(|| {
let i = self.frame_index;
self.frame_index = i.wrapping_add(1);
i
});
let pps = self.payload_per_shard;
let blocksize = SHARD_HEADER + pps; // = packet_size + 16
let pct = self.fec_percentage;
@@ -235,7 +243,7 @@ mod tests {
let mut pk = VideoPacketizer::new(1392, 0, 0); // data-only; pps = 1392+16-32 = 1376
assert_eq!(pk.payload_per_shard, 1376);
let au = vec![0xABu8; 4000]; // 8+4000 = 4008 → ceil(4008/1376) = 3 data shards
let pkts = pk.packetize(&au, FrameType::Idr, 90_000);
let pkts = pk.packetize(&au, FrameType::Idr, 90_000, None);
assert_eq!(pkts.len(), 3);
for p in &pkts {
assert_eq!(p.len(), SHARD_HEADER + 1376);
@@ -266,7 +274,7 @@ mod tests {
for ps in [0usize, 15, 16, 17, 32] {
let mut pk = VideoPacketizer::new(ps, 20, 2);
assert!(pk.payload_per_shard >= 1, "pps must never be 0 (ps={ps})");
let _ = pk.packetize(&[0xCDu8; 200], FrameType::Idr, 0); // must not panic
let _ = pk.packetize(&[0xCDu8; 200], FrameType::Idr, 0, None); // must not panic
}
}
@@ -274,7 +282,7 @@ mod tests {
fn multi_block_split() {
let mut pk = VideoPacketizer::new(1392, 0, 0); // data-only
let au = vec![0u8; 600_000];
let pkts = pk.packetize(&au, FrameType::P, 0);
let pkts = pk.packetize(&au, FrameType::P, 0, None);
let total = (8 + au.len()).div_ceil(1376);
assert_eq!(pkts.len(), total);
let n_blocks = total.div_ceil(255).clamp(1, 4);
@@ -286,7 +294,7 @@ mod tests {
fn emits_parity_shards() {
let mut pk = VideoPacketizer::new(1392, 20, 0); // pps = 1376, 20% FEC
let au = vec![0xABu8; 4000]; // 8+4000 = 4008 → 3 data shards (k=3)
let pkts = pk.packetize(&au, FrameType::Idr, 0);
let pkts = pk.packetize(&au, FrameType::Idr, 0, None);
// m = ceil(3*20/100) = 1 parity shard → 4 packets; wire_pct = 100*1/3 = 33.
assert_eq!(pkts.len(), 4);
for p in &pkts {
@@ -313,7 +321,7 @@ mod tests {
fn parity_recovers_full_datagram_incl_flags() {
let mut pk = VideoPacketizer::new(1392, 50, 0); // high pct → plenty of parity
let au = vec![0x5Au8; 4000]; // k = 3
let pkts = pk.packetize(&au, FrameType::Idr, 0);
let pkts = pk.packetize(&au, FrameType::Idr, 0, None);
let k = 3usize;
let m = pkts.len() - k;
assert!(m >= 1);
+96 -10
View File
@@ -1540,6 +1540,10 @@ async fn serve_session(
// and gets no extra datagrams.
let timing_conn =
(hello.video_caps & punktfunk_core::quic::VIDEO_CAP_HOST_TIMING != 0).then(|| conn.clone());
// Probe-sequence capability: the client reassembles speed-test filler in its own index window,
// so mid-session bursts don't consume video frame indexes. An older client (bit clear) gets
// mid-session probes declined instead — see `run_probe_burst`.
let probe_seq = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ != 0;
let stats_dp = stats; // data-plane handle to the shared stats recorder
// Short label for web-console stats captures: the client's cert-fingerprint prefix, else its
// peer IP (no fingerprint = anonymous TOFU/--open client).
@@ -1595,6 +1599,7 @@ async fn serve_session(
&probe_result_tx,
&fec_target_dp,
timing_conn.as_ref(),
probe_seq,
),
Punktfunk1Source::Virtual => {
let compositor = compositor
@@ -1620,6 +1625,7 @@ async fn serve_session(
fec_target: fec_target_dp,
conn: conn_stream,
timing_conn,
probe_seq,
stats: stats_dp,
client_label,
launch: launch_for_dp,
@@ -2437,6 +2443,7 @@ fn mark_recovery_boundary(ir_wave_pos: &mut u32, is_keyframe: bool, period: u32)
}
}
#[allow(clippy::too_many_arguments)]
fn synthetic_stream(
session: &mut Session,
frames: u32,
@@ -2445,6 +2452,7 @@ fn synthetic_stream(
probe_result_tx: &tokio::sync::mpsc::UnboundedSender<ProbeResult>,
fec_target: &AtomicU8,
timing_conn: Option<&quinn::Connection>,
probe_seq: bool,
) -> Result<()> {
let interval = std::time::Duration::from_millis(1000 / 60);
for idx in 0..frames {
@@ -2453,7 +2461,7 @@ fn synthetic_stream(
}
apply_fec_target(session, fec_target);
// Service speed-test probes between synthetic frames (loopback bandwidth tests).
service_probes(session, stop, probe_rx, probe_result_tx);
service_probes(session, stop, probe_rx, probe_result_tx, probe_seq);
let data = test_frame(idx, 64 * 1024);
let pts_ns = now_ns();
session
@@ -2795,9 +2803,34 @@ const MAX_PROBE_MS: u32 = 5_000;
/// was actually offered so the client can compute delivery ratio (`received / bytes_sent`) and
/// throughput. Video is paused for the duration (the caller's loop is blocked here) — a speed test
/// is a deliberate, short interruption the client initiates.
fn run_probe_burst(session: &mut Session, req: ProbeRequest, stop: &AtomicBool) -> ProbeResult {
fn run_probe_burst(
session: &mut Session,
req: ProbeRequest,
stop: &AtomicBool,
probe_seq: bool,
) -> ProbeResult {
let target_kbps = req.target_kbps.min(MAX_PROBE_KBPS);
let duration_ms = req.duration_ms.min(MAX_PROBE_MS);
// Probe filler is sealed in the PROBE index space (its own frame counter — video indexes are
// owned by the encode loop and must stay 1:1 with the encoder's RFI bookkeeping). A client
// that didn't advertise VIDEO_CAP_PROBE_SEQ reassembles everything in one window and would
// drop probe-space frames as stale against the video stream — measuring garbage — so its
// mid-session probe is DECLINED (zeroed result) instead. Old sealing (probe filler consuming
// video indexes) is not an option anymore: those indexes are invisible to every client gap
// detector and read as a phantom multi-thousand-frame loss after the burst.
if !probe_seq {
tracing::info!(
"declining speed-test probe: client predates VIDEO_CAP_PROBE_SEQ (its reassembler \
cannot window probe-space frames)"
);
return ProbeResult {
bytes_sent: 0,
packets_sent: 0,
duration_ms: 0,
wire_packets_sent: 0,
send_dropped: 0,
};
}
if target_kbps == 0 || duration_ms == 0 {
return ProbeResult {
bytes_sent: 0,
@@ -2831,8 +2864,9 @@ fn run_probe_burst(session: &mut Session, req: ProbeRequest, stop: &AtomicBool)
let allowed = (start.elapsed().as_secs_f64() * bytes_per_sec as f64) as u64;
if bytes_sent < allowed {
// A full send buffer drops on WouldBlock/ENOBUFS (UdpTransport returns Ok) — that loss is
// part of what the probe measures (it surfaces as send_dropped), so keep going.
let _ = session.submit_frame(&filler, now_ns(), FLAG_PROBE as u32);
// part of what the probe measures (it surfaces as send_dropped), so keep going. Sealed
// in the probe index space (FLAG_PROBE + its own counter) — never a video frame_index.
let _ = session.submit_probe_frame(&filler, now_ns());
bytes_sent += chunk as u64;
packets_sent += 1;
} else {
@@ -2863,15 +2897,17 @@ fn run_probe_burst(session: &mut Session, req: ProbeRequest, stop: &AtomicBool)
}
/// Drain any pending speed-test requests and run each burst, replying with its [`ProbeResult`].
/// Called once per data-plane loop iteration so a probe runs between frames.
/// Called once per data-plane loop iteration so a probe runs between frames. `probe_seq` = the
/// client advertised [`punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ`] (see [`run_probe_burst`]).
fn service_probes(
session: &mut Session,
stop: &AtomicBool,
probe_rx: &std::sync::mpsc::Receiver<ProbeRequest>,
probe_result_tx: &tokio::sync::mpsc::UnboundedSender<ProbeResult>,
probe_seq: bool,
) {
while let Ok(req) = probe_rx.try_recv() {
let result = run_probe_burst(session, req, stop);
let result = run_probe_burst(session, req, stop, probe_seq);
let _ = probe_result_tx.send(result);
}
}
@@ -2886,16 +2922,18 @@ fn service_probes(
/// buffer → EAGAIN drop → under infinite GOP, a freeze until the next keyframe). With no slack
/// (encode ≈ interval) the budget collapses to 0 and even the overflow goes out immediately, so
/// this is never slower than unpaced.
#[allow(clippy::too_many_arguments)]
fn paced_submit(
session: &mut Session,
data: &[u8],
pts_ns: u64,
flags: u32,
frame_index: u32,
deadline: std::time::Instant,
burst_cap: usize,
) -> Result<PaceStat> {
let wires = session
.seal_frame(data, pts_ns, flags)
.seal_frame_at(data, pts_ns, flags, frame_index)
.map_err(|e| anyhow!("seal_frame: {e:?}"))?;
let mut refs: Vec<&[u8]> = wires.iter().map(|w| w.as_slice()).collect();
// FEC/recovery test knob (PUNKTFUNK_VIDEO_DROP) — same knob the GameStream plane honors.
@@ -2925,6 +2963,12 @@ struct FrameMsg {
data: Vec<u8>,
capture_ns: u64,
flags: u32,
/// The wire `frame_index` this AU is sealed with. Assigned by the encode loop's
/// session-lifetime counter (`au_seq`) — the loop owns the video numbering so the index it
/// PREDICTED at submit time (`au_seq + inflight`, handed to `Encoder::submit_indexed`) is
/// exactly what the packetizer stamps, keeping the encoder's RFI bookkeeping 1:1 with the
/// wire across encoder rebuilds/resets. Sealed via `Session::seal_frame_at`.
frame_index: u32,
/// When this frame's packets should have fully left (the next frame's due time) = the pacing
/// budget. In the past when the send thread is behind → immediate send (catch up).
deadline: std::time::Instant,
@@ -3117,6 +3161,9 @@ fn send_loop(
// `Some` = the client advertised VIDEO_CAP_HOST_TIMING: emit one 0xCF datagram per AU right
// after its last packet left the socket (capture→sent, the whole host pipeline incl. pacing).
timing_conn: Option<quinn::Connection>,
// The client advertised VIDEO_CAP_PROBE_SEQ — mid-session speed-test bursts may run in the
// probe index space (else they're declined; see `run_probe_burst`).
probe_seq: bool,
) {
boost_thread_priority(false); // transmit thread: above-normal (Apollo's encoder-thread level)
let mut last_perf = std::time::Instant::now();
@@ -3145,7 +3192,7 @@ fn send_loop(
}
// Probes run here (they need the Session); a burst pauses video — the encode thread blocks
// on the full frame channel meanwhile, which is exactly the intended pause.
service_probes(&mut session, &stop, &probe_rx, &probe_result_tx);
service_probes(&mut session, &stop, &probe_rx, &probe_result_tx, probe_seq);
// Adaptive FEC: pick up any new recovery target the control task set from client LossReports.
apply_fec_target(&mut session, &fec_target);
// Short timeout so we keep re-checking `stop` + probes when no frames are flowing.
@@ -3155,6 +3202,7 @@ fn send_loop(
&msg.data,
msg.capture_ns,
msg.flags,
msg.frame_index,
msg.deadline,
burst_cap,
) {
@@ -3472,6 +3520,12 @@ struct SessionContext {
/// thread emits one 0xCF datagram per AU (capture→sent µs) on it, so the client can split its
/// `host+network` latency stage. `None` = older client, no emission.
timing_conn: Option<quinn::Connection>,
/// The client advertised [`punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ`]: speed-test bursts may
/// run mid-session in the probe index space (its reassembler keeps a separate probe window).
/// `false` = older client whose single-window reassembler would drop probe-space frames as
/// stale — mid-session probes are DECLINED for it (a zeroed [`ProbeResult`]) rather than
/// consuming video frame indexes its gap detectors can't see (the phantom-gap freeze).
probe_seq: bool,
/// Shared streaming-stats recorder. The capture loop reads `is_armed()` per frame to decide
/// whether to measure the per-stage split; the send thread builds + pushes the aggregated
/// `StatsSample` at its 2 s boundary.
@@ -3527,6 +3581,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
fec_target,
conn,
timing_conn,
probe_seq,
stats,
client_label,
launch,
@@ -3664,6 +3719,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
fec_target,
send_stats,
timing_conn,
probe_seq,
)
}
})
@@ -3689,6 +3745,16 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(seconds as u64);
let mut next = std::time::Instant::now();
let mut sent: u64 = 0;
// The session's video frame numbering, owned HERE (the wire `frame_index` of the next AU this
// loop hands to the send thread; the packetizer seals with exactly this via `seal_frame_at`).
// A submission's future index is predicted as `au_seq + inflight.len()` — exact because AUs
// are emitted FIFO, one per submission, and every event that forfeits in-flight frames
// (reset/rebuild/teardown) clears `inflight` AND the encoder's reference state, so the reused
// predictions can never meet stale bookkeeping. Passing it to `Encoder::submit_indexed` keeps
// the RFI backends' frame numbers 1:1 with the client's across encoder rebuilds — an
// encoder-internal counter desyncs on the first adaptive-bitrate rebuild (NVENC RFI then
// silently dies; AMF may anchor onto a post-loss LTR).
let mut au_seq: u32 = 0;
// Rebuild-in-place on capture loss: track the live mode (a mode switch updates it) so a rebuild
// targets the CURRENT mode, and cap consecutive rebuilds so a flapping source can't loop the
// client through endless cold restarts.
@@ -3967,7 +4033,19 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
}
if !want_kf {
if let Some((first, last)) = rfi_range {
if enc.caps().supports_rfi && enc.invalidate_ref_frames(first as i64, last as i64) {
// Sanity-cap the range before consulting the encoder: RFI can only re-reference
// history the encoder still holds (NVENC: a 5-frame DPB; AMD LTR: ~1 s of marks).
// A range wider than RFI_MAX_RANGE is either a seconds-long outage (no valid
// reference anywhere) or a phantom jump from a desynced counter — both belong on
// the keyframe path, never a force-reference that could ship corruption as a
// recovery anchor. Wrapping width: frame indexes are u32 counters.
let width = last.wrapping_sub(first);
if width > punktfunk_core::packet::RFI_MAX_RANGE {
tracing::debug!(first, last, width, "RFI range too wide — keyframe instead");
want_kf = true;
} else if enc.caps().supports_rfi
&& enc.invalidate_ref_frames(first as i64, last as i64)
{
// The RFI recovered the loss with a clean re-anchor P-frame (no IDR). Anchor the
// keyframe cooldown so the client's echo of the SAME loss — its frames_dropped-
// driven keyframe request, arriving ~one loss-window later — is coalesced away
@@ -4234,7 +4312,11 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
st_queue.push(queue_us);
}
let t_submit = std::time::Instant::now();
if let Err(e) = enc.submit(&frame) {
// This submission's future wire frame index (see `au_seq`): AUs are emitted FIFO one per
// submission, so it lands `inflight.len()` AUs after the `au_seq` the loop is about to
// assign next. The RFI backends pin their frame numbering to it.
let wire_index = au_seq.wrapping_add(inflight.len() as u32);
if let Err(e) = enc.submit_indexed(&frame, wire_index) {
// The input half of an encode stall: once the driver stops draining AUs, libavcodec's
// one-frame buffer fills and avcodec_send_frame starts failing (EAGAIN) — the same
// wedge the watchdog below catches, seen from submit. Rebuild the encoder in place
@@ -4339,6 +4421,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
data: au.data,
capture_ns: cap_ns,
flags,
frame_index: au_seq,
deadline,
encode_us,
queue_us,
@@ -4354,6 +4437,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
send_gone = true;
break;
}
au_seq = au_seq.wrapping_add(1);
sent += 1;
}
if send_gone {
@@ -4414,6 +4498,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
data: au.data,
capture_ns: cap_ns,
flags,
frame_index: au_seq,
deadline,
encode_us,
queue_us: 0,
@@ -4426,6 +4511,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
if frame_tx.send(msg).is_err() {
break;
}
au_seq = au_seq.wrapping_add(1);
sent += 1;
}
// Signal the send thread to drain + exit (drop the channel), then join it.
+24
View File
@@ -274,6 +274,15 @@
// `AV_FRAME_FLAG_KEY` — this host flag is the only signal.
#define USER_FLAG_RECOVERY_ANCHOR 32
// Widest lost-frame range (frames, wrapping `last - first`) a reference-frame-invalidation
// recovery may be asked to repair; anything wider goes straight to the keyframe path on BOTH
// ends. RFI can only re-reference history the encoder still holds — NVENC keeps a 5-frame DPB,
// AMD LTR ~1 s of marks — and a genuine loss this wide (>1 s even at 240 fps) has no valid
// reference anywhere, so an RFI request for it is either hopeless or (worse) a phantom range
// from a desynced counter. Shared by the host's RFI dispatch (range → keyframe fallback) and the
// client-side gap detectors (huge gap → resync + keyframe request, no RFI).
#define RFI_MAX_RANGE 256
// Largest UDP datagram the core will send or accept. `Config::validate` bounds
// `shard_payload` so `HEADER_LEN + shard_payload + CRYPTO_OVERHEAD ≤ MAX_DATAGRAM_BYTES`.
#define MAX_DATAGRAM_BYTES 2048
@@ -382,6 +391,21 @@
#define VIDEO_CAP_HOST_TIMING 8
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_caps`] bit: the client's reassembler keeps **speed-test probe filler in its own
// frame-index space** (a second reassembly window keyed on the [`crate::packet::FLAG_PROBE`]
// user-flag), so probe bursts no longer consume video `frame_index`es. Without this, a mid-session
// speed test burns thousands of video indexes that are invisible to every client-side gap detector
// (probe frames are filtered before the pump sees them) — the first real AU afterwards reads as a
// phantom multi-thousand-frame loss (spurious freeze + a nonsense RFI). It also lets the host's
// encode loop own the video numbering outright (the wire-index contract
// [`crate::packet::Packetizer::packetize_each`] documents), which reference-frame invalidation
// depends on. The host runs mid-session probe bursts ONLY against clients that set this bit — an
// older client gets a declined (zeroed) [`ProbeResult`] instead of a measurement its single-window
// reassembler would silently drop as stale.
#define VIDEO_CAP_PROBE_SEQ 16
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// QUIC application error code a punktfunk/1 client closes the control connection with on a
// **deliberate quit** (a user "stop", not a network drop). The host reads it off the connection's