feat(core,host,client): PyroWave datagram-aligned packets + partial-frame delivery (Phase 4, §4.4)
ci / web (push) Successful in 1m10s
ci / docs-site (push) Successful in 1m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
apple / swift (push) Successful in 4m59s
ci / rust (push) Failing after 6m17s
ci / bench (push) Successful in 6m36s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m29s
arch / build-publish (push) Successful in 11m11s
android / android (push) Successful in 12m33s
deb / build-publish (push) Successful in 14m8s
windows-host / package (push) Successful in 14m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m4s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m6s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 5m15s
release / apple (push) Successful in 26m8s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 5m48s
apple / screenshots (push) Successful in 20m29s

PyroWave AUs now packetize on the negotiated shard payload, so a lost datagram
costs a few wavelet blocks of localized blur rather than a whole frame — and the
client can render an aged-out lossy frame instead of freezing until the next one.

Host (opt-in, PyroWave only):
- The encoder packetizes at the shard payload behind a 4-byte window prefix
  (used-len u16 + kind u16). Whole packets pack into WIN_PACKED windows; a packet
  too large for one shard (PyroWave 32x32 blocks are atomic and can exceed a
  shard) rides a WIN_FRAG_FIRST/CONT/LAST chain. `set_wire_chunking()` joins the
  Encoder trait (forwarded through TrackedEncoder — the silent-no-op trap);
  EncodedFrame.chunk_aligned marks the AU.
- virtual_stream tags the AU with USER_FLAG_CHUNK_ALIGNED and re-applies chunking
  after every encoder (re)build, the adaptive-bitrate rebuild included.

Core:
- USER_FLAG_CHUNK_ALIGNED (0x40) wire bit. Reassembler opt-in
  (set_deliver_partial): a chunk-aligned frame that ages out with holes is handed
  over as Frame{complete:false} — received shards at their exact offsets, missing
  ranges zero-filled — instead of being dropped. Partials age out on a tight 30ms
  fuse (PARTIAL_WINDOW_NS) instead of the 120ms loss window: each frame is
  independently decodable, so an ancient partial has no value in a live stream.
  Newest-wins. A partial still counts as dropped for loss reporting.

Client (PyroWave decode):
- The session opts in when codec == PyroWave. The decoder walks the AU
  window-by-window, skipping zero (missing) windows and reassembling FRAG chains,
  then decodes whatever survived. A newest-decoded-index guard drops partials the
  pump has already moved past (no time-travel present).

Also fixes a redundant-closure clippy nit in the PyroWave planar-present path.

Validated on an RTX 5070 Ti under 2% netem loss with FEC pinned off: 60fps
sustained entirely via partials, e2e 43ms p50 (146ms before the fuse) vs 23ms
lossless, no keyframe-recovery chatter. Tests green: core 149, host 310 + the
GPU-gated encoder smoke (framed-window walk + FRAG reassembly + upstream
round-trip), client 26; clippy clean on the pyrowave feature combos.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-15 11:14:24 +02:00
parent 1fc9ef0050
commit 705a8baddf
21 changed files with 619 additions and 33 deletions
+74 -1
View File
@@ -54,6 +54,14 @@ pub const USER_FLAG_RECOVERY_POINT: u32 = 0x10;
/// `AV_FRAME_FLAG_KEY` — this host flag is the only signal.
pub const USER_FLAG_RECOVERY_ANCHOR: u32 = 0x20;
/// `user_flags` bit: the AU's content is **shard-aligned self-delimiting chunks** — every
/// `shard_payload`-sized window of the frame buffer starts a fresh codec packet, padded to the
/// window with zeros (PyroWave datagram-aligned mode, design/pyrowave-codec-plan.md §4.4). Two
/// consequences: a receiver that opted into partial delivery can use an aged-out frame's buffer
/// AS-IS (missing shards stay zeroed; the codec's block walk skips zero windows), and even a
/// COMPLETE frame must be consumed window-by-window (the padding is not part of the stream).
pub const USER_FLAG_CHUNK_ALIGNED: u32 = 0x40;
/// Widest lost-frame range (frames, wrapping `last - first`) a reference-frame-invalidation
/// recovery may be asked to repair; anything wider goes straight to the keyframe path on BOTH
/// ends. RFI can only re-reference history the encoder still holds — NVENC keeps a 5-frame DPB,
@@ -89,6 +97,13 @@ const LOSS_WINDOW_NS: u64 = 120_000_000;
/// 120 ms ≈ 14 indices, so 64 leaves ample slack up to ~500 fps.
const HARD_LOSS_WINDOW: u32 = 64;
/// The much tighter fuse for PARTIAL-deliverable frames (chunk-aligned AUs with a consumer
/// that opted in): once anything newer exists and this much capture time passed, the frame
/// is delivered as-is — its stragglers can only make it less late, and each frame is
/// independently decodable, so waiting the full loss window (120 ms) would inject ancient
/// frames into a live stream. ~2 frame periods at 60 fps rides out normal reorder.
const PARTIAL_WINDOW_NS: u64 = 30_000_000;
/// How many frames behind the newest the reassembler remembers emitted/abandoned frame indices
/// (`completed`), so a straggler shard can neither resurrect an abandoned frame nor re-open an
/// emitted one. Must cover at least [`HARD_LOSS_WINDOW`]: stragglers can trickle in later than the
@@ -451,6 +466,13 @@ const RECOVERY_POOL_MAX: usize = 512;
/// Client-side only.
pub struct Reassembler {
limits: ReassemblerLimits,
/// Deliver aged-out incomplete frames whose AUs are [`USER_FLAG_CHUNK_ALIGNED`] instead of
/// silently dropping them (client opt-in — the PyroWave decode path): the frame buffer is
/// already the right shape (received shards at their final offsets, zeros elsewhere).
/// They still count into `frames_dropped` — a partial IS lost data for the loss reports.
deliver_partial: bool,
/// The newest such partial awaiting pickup (newest-wins: partials are a lossy byproduct).
pending_partial: Option<Frame>,
/// The video stream's window — its aged-out incomplete frames count into `frames_dropped`
/// (the client's loss-recovery trigger).
video: ReassemblyWindow,
@@ -472,6 +494,8 @@ impl Reassembler {
pub fn new(limits: ReassemblerLimits) -> Self {
Reassembler {
limits,
deliver_partial: false,
pending_partial: None,
video: ReassemblyWindow::default(),
probe: ReassemblyWindow::default(),
recovery_pool: Vec::new(),
@@ -479,6 +503,19 @@ impl Reassembler {
}
}
/// Opt into partial delivery of chunk-aligned frames (see [`Reassembler::deliver_partial`]).
pub fn set_deliver_partial(&mut self, on: bool) {
self.deliver_partial = on;
if !on {
self.pending_partial = None;
}
}
/// Take the newest aged-out partial frame, if one is pending (see `set_deliver_partial`).
pub fn take_partial(&mut self) -> Option<Frame> {
self.pending_partial.take()
}
/// Ingest one (already-decrypted) packet. Returns the access unit when its last
/// block completes, otherwise `None`.
pub fn push(
@@ -507,6 +544,8 @@ impl Reassembler {
// in-flight budget are all touched while a frame entry is mutably borrowed.
let Reassembler {
limits,
deliver_partial,
pending_partial,
video,
probe,
recovery_pool,
@@ -579,6 +618,7 @@ impl Reassembler {
recovery_pool,
in_flight_bytes,
lim.max_data_shards,
(*deliver_partial && !is_probe).then_some(pending_partial),
);
// Drop shards for frames already terminated (emitted — e.g. the recovery shards of a
@@ -756,6 +796,7 @@ impl Reassembler {
frame_index: hdr.frame_index,
pts_ns: done.pts_ns,
flags: done.user_flags,
complete: true,
}));
}
Ok(None)
@@ -809,6 +850,8 @@ impl ReassemblyWindow {
recovery_pool: &mut Vec<Vec<u8>>,
in_flight_bytes: &mut usize,
max_data_shards: usize,
// `Some(sink)` = deliver aged-out CHUNK_ALIGNED frames instead of only dropping them.
mut partial_sink: Option<&mut Option<Frame>>,
) {
let (newest, newest_pts) = match self.newest_frame {
// `frame_index` is newer iff it's within the forward half of the index space.
@@ -819,9 +862,17 @@ impl ReassemblyWindow {
let before = self.frames.len();
let completed = &mut self.completed;
let partial_on = partial_sink.is_some();
self.frames.retain(|&idx, f| {
// Partial-deliverable frames age out on the TIGHT fuse (see PARTIAL_WINDOW_NS);
// everything else keeps the full loss window.
let window_ns = if partial_on && f.user_flags & USER_FLAG_CHUNK_ALIGNED != 0 {
PARTIAL_WINDOW_NS
} else {
LOSS_WINDOW_NS
};
let keep = newest.wrapping_sub(idx) <= HARD_LOSS_WINDOW
&& newest_pts.saturating_sub(f.pts_ns) <= LOSS_WINDOW_NS;
&& newest_pts.saturating_sub(f.pts_ns) <= window_ns;
if !keep {
// Remember the abandoned index so a straggler shard is dropped (below, and in
// `push`) instead of resurrecting the frame — which would re-allocate its buffers
@@ -831,6 +882,28 @@ impl ReassemblyWindow {
completed.insert(idx, reconstructed_shards(&f.blocks, max_data_shards));
// Release its buffer budget and reclaim its parity bufs for the pool.
*in_flight_bytes -= f.buf.len();
// Partial delivery (chunk-aligned AUs only): the buffer is already exactly
// what the consumer needs — received shards at their final offsets, zeros
// where shards are missing (the codec's block walk skips zero windows).
// Newest-wins if several age out in one prune. Still counted dropped below.
if let Some(sink) = partial_sink.as_deref_mut() {
if f.user_flags & USER_FLAG_CHUNK_ALIGNED != 0 {
let mut buf = std::mem::take(&mut f.buf);
buf.truncate(f.frame_bytes);
let newer = sink
.as_ref()
.is_none_or(|p| idx.wrapping_sub(p.frame_index) <= u32::MAX / 2);
if newer {
*sink = Some(Frame {
data: buf,
frame_index: idx,
pts_ns: f.pts_ns,
flags: f.user_flags,
complete: false,
});
}
}
}
for block in f.blocks.values_mut() {
for slot in block.recovery.iter_mut() {
if let Some(rb) = slot.take() {