feat(core,host,client): PyroWave datagram-aligned packets + partial-frame delivery (Phase 4, §4.4)
ci / web (push) Successful in 1m10s
ci / docs-site (push) Successful in 1m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
apple / swift (push) Successful in 4m59s
ci / rust (push) Failing after 6m17s
ci / bench (push) Successful in 6m36s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m29s
arch / build-publish (push) Successful in 11m11s
android / android (push) Successful in 12m33s
deb / build-publish (push) Successful in 14m8s
windows-host / package (push) Successful in 14m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m4s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m6s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 5m15s
release / apple (push) Successful in 26m8s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 5m48s
apple / screenshots (push) Successful in 20m29s

PyroWave AUs now packetize on the negotiated shard payload, so a lost datagram
costs a few wavelet blocks of localized blur rather than a whole frame — and the
client can render an aged-out lossy frame instead of freezing until the next one.

Host (opt-in, PyroWave only):
- The encoder packetizes at the shard payload behind a 4-byte window prefix
  (used-len u16 + kind u16). Whole packets pack into WIN_PACKED windows; a packet
  too large for one shard (PyroWave 32x32 blocks are atomic and can exceed a
  shard) rides a WIN_FRAG_FIRST/CONT/LAST chain. `set_wire_chunking()` joins the
  Encoder trait (forwarded through TrackedEncoder — the silent-no-op trap);
  EncodedFrame.chunk_aligned marks the AU.
- virtual_stream tags the AU with USER_FLAG_CHUNK_ALIGNED and re-applies chunking
  after every encoder (re)build, the adaptive-bitrate rebuild included.

Core:
- USER_FLAG_CHUNK_ALIGNED (0x40) wire bit. Reassembler opt-in
  (set_deliver_partial): a chunk-aligned frame that ages out with holes is handed
  over as Frame{complete:false} — received shards at their exact offsets, missing
  ranges zero-filled — instead of being dropped. Partials age out on a tight 30ms
  fuse (PARTIAL_WINDOW_NS) instead of the 120ms loss window: each frame is
  independently decodable, so an ancient partial has no value in a live stream.
  Newest-wins. A partial still counts as dropped for loss reporting.

Client (PyroWave decode):
- The session opts in when codec == PyroWave. The decoder walks the AU
  window-by-window, skipping zero (missing) windows and reassembling FRAG chains,
  then decodes whatever survived. A newest-decoded-index guard drops partials the
  pump has already moved past (no time-travel present).

Also fixes a redundant-closure clippy nit in the PyroWave planar-present path.

Validated on an RTX 5070 Ti under 2% netem loss with FEC pinned off: 60fps
sustained entirely via partials, e2e 43ms p50 (146ms before the fuse) vs 23ms
lossless, no keyframe-recovery chatter. Tests green: core 149, host 310 + the
GPU-gated encoder smoke (framed-window walk + FRAG reassembly + upstream
round-trip), client 26; clippy clean on the pyrowave feature combos.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-15 11:14:24 +02:00
parent 1fc9ef0050
commit 705a8baddf
21 changed files with 619 additions and 33 deletions
+16 -1
View File
@@ -71,6 +71,8 @@ enum CtrlRequest {
#[derive(Clone, Copy)]
struct Negotiated {
mode: Mode,
/// Wire shard payload — the chunk-aligned parse window (plan §4.4).
shard_payload: u16,
compositor: CompositorPref,
gamepad: GamepadPref,
/// SHA-256 of the certificate the host actually presented (TOFU callers persist this).
@@ -513,6 +515,9 @@ pub struct NativeClient {
/// requested rate clamped to the host's range, or its default if we requested `0`. `0` = an
/// older host that didn't report it.
pub resolved_bitrate_kbps: u32,
/// The session's wire shard payload (bytes of AU per datagram) — the parse-window size
/// for chunk-aligned AUs ([`crate::packet::USER_FLAG_CHUNK_ALIGNED`], plan §4.4).
pub shard_payload: u16,
/// Host clock minus client clock (ns), from the connect-time skew handshake. Add it to a local
/// receive/present timestamp to express it in the host's capture clock (the AU `pts_ns`), making
/// glass-to-glass latency valid across machines. `0` = no correction (an old host that didn't
@@ -778,6 +783,7 @@ impl NativeClient {
resolved_compositor: negotiated.compositor,
resolved_gamepad: negotiated.gamepad,
resolved_bitrate_kbps: negotiated.bitrate_kbps,
shard_payload: negotiated.shard_payload,
clock_offset_ns: negotiated.clock_offset_ns,
bit_depth: negotiated.bit_depth,
color: negotiated.color,
@@ -1540,7 +1546,14 @@ async fn worker_main(args: WorkerArgs) {
if let Ok(sock) = transport.try_clone_socket() {
crate::transport::spawn_data_punch(sock, shutdown.clone());
}
let session = Session::new(welcome.session_config(Role::Client), Box::new(transport))?;
let mut session =
Session::new(welcome.session_config(Role::Client), Box::new(transport))?;
// PyroWave sessions opt into partial delivery (plan §4.4): an aged-out lossy
// frame arrives as blocks-with-holes instead of vanishing — the all-intra codec
// renders it as one frame of localized blur, strictly better than a freeze.
if welcome.codec == crate::quic::CODEC_PYROWAVE {
session.set_deliver_partial_frames(true);
}
Ok::<_, PunktfunkError>((
session,
send,
@@ -1558,6 +1571,7 @@ async fn worker_main(args: WorkerArgs) {
chroma_format: welcome.chroma_format,
audio_channels: welcome.audio_channels,
codec: welcome.codec,
shard_payload: welcome.shard_payload,
},
welcome.host_caps,
))
@@ -2490,6 +2504,7 @@ mod frame_channel_tests {
frame_index: i,
pts_ns: i as u64,
flags: 0,
complete: true,
}
}
+74 -1
View File
@@ -54,6 +54,14 @@ pub const USER_FLAG_RECOVERY_POINT: u32 = 0x10;
/// `AV_FRAME_FLAG_KEY` — this host flag is the only signal.
pub const USER_FLAG_RECOVERY_ANCHOR: u32 = 0x20;
/// `user_flags` bit: the AU's content is **shard-aligned self-delimiting chunks** — every
/// `shard_payload`-sized window of the frame buffer starts a fresh codec packet, padded to the
/// window with zeros (PyroWave datagram-aligned mode, design/pyrowave-codec-plan.md §4.4). Two
/// consequences: a receiver that opted into partial delivery can use an aged-out frame's buffer
/// AS-IS (missing shards stay zeroed; the codec's block walk skips zero windows), and even a
/// COMPLETE frame must be consumed window-by-window (the padding is not part of the stream).
pub const USER_FLAG_CHUNK_ALIGNED: u32 = 0x40;
/// Widest lost-frame range (frames, wrapping `last - first`) a reference-frame-invalidation
/// recovery may be asked to repair; anything wider goes straight to the keyframe path on BOTH
/// ends. RFI can only re-reference history the encoder still holds — NVENC keeps a 5-frame DPB,
@@ -89,6 +97,13 @@ const LOSS_WINDOW_NS: u64 = 120_000_000;
/// 120 ms ≈ 14 indices, so 64 leaves ample slack up to ~500 fps.
const HARD_LOSS_WINDOW: u32 = 64;
/// The much tighter fuse for PARTIAL-deliverable frames (chunk-aligned AUs with a consumer
/// that opted in): once anything newer exists and this much capture time passed, the frame
/// is delivered as-is — its stragglers can only make it less late, and each frame is
/// independently decodable, so waiting the full loss window (120 ms) would inject ancient
/// frames into a live stream. ~2 frame periods at 60 fps rides out normal reorder.
const PARTIAL_WINDOW_NS: u64 = 30_000_000;
/// How many frames behind the newest the reassembler remembers emitted/abandoned frame indices
/// (`completed`), so a straggler shard can neither resurrect an abandoned frame nor re-open an
/// emitted one. Must cover at least [`HARD_LOSS_WINDOW`]: stragglers can trickle in later than the
@@ -451,6 +466,13 @@ const RECOVERY_POOL_MAX: usize = 512;
/// Client-side only.
pub struct Reassembler {
limits: ReassemblerLimits,
/// Deliver aged-out incomplete frames whose AUs are [`USER_FLAG_CHUNK_ALIGNED`] instead of
/// silently dropping them (client opt-in — the PyroWave decode path): the frame buffer is
/// already the right shape (received shards at their final offsets, zeros elsewhere).
/// They still count into `frames_dropped` — a partial IS lost data for the loss reports.
deliver_partial: bool,
/// The newest such partial awaiting pickup (newest-wins: partials are a lossy byproduct).
pending_partial: Option<Frame>,
/// The video stream's window — its aged-out incomplete frames count into `frames_dropped`
/// (the client's loss-recovery trigger).
video: ReassemblyWindow,
@@ -472,6 +494,8 @@ impl Reassembler {
pub fn new(limits: ReassemblerLimits) -> Self {
Reassembler {
limits,
deliver_partial: false,
pending_partial: None,
video: ReassemblyWindow::default(),
probe: ReassemblyWindow::default(),
recovery_pool: Vec::new(),
@@ -479,6 +503,19 @@ impl Reassembler {
}
}
/// Opt into partial delivery of chunk-aligned frames (see [`Reassembler::deliver_partial`]).
pub fn set_deliver_partial(&mut self, on: bool) {
self.deliver_partial = on;
if !on {
self.pending_partial = None;
}
}
/// Take the newest aged-out partial frame, if one is pending (see `set_deliver_partial`).
pub fn take_partial(&mut self) -> Option<Frame> {
self.pending_partial.take()
}
/// Ingest one (already-decrypted) packet. Returns the access unit when its last
/// block completes, otherwise `None`.
pub fn push(
@@ -507,6 +544,8 @@ impl Reassembler {
// in-flight budget are all touched while a frame entry is mutably borrowed.
let Reassembler {
limits,
deliver_partial,
pending_partial,
video,
probe,
recovery_pool,
@@ -579,6 +618,7 @@ impl Reassembler {
recovery_pool,
in_flight_bytes,
lim.max_data_shards,
(*deliver_partial && !is_probe).then_some(pending_partial),
);
// Drop shards for frames already terminated (emitted — e.g. the recovery shards of a
@@ -756,6 +796,7 @@ impl Reassembler {
frame_index: hdr.frame_index,
pts_ns: done.pts_ns,
flags: done.user_flags,
complete: true,
}));
}
Ok(None)
@@ -809,6 +850,8 @@ impl ReassemblyWindow {
recovery_pool: &mut Vec<Vec<u8>>,
in_flight_bytes: &mut usize,
max_data_shards: usize,
// `Some(sink)` = deliver aged-out CHUNK_ALIGNED frames instead of only dropping them.
mut partial_sink: Option<&mut Option<Frame>>,
) {
let (newest, newest_pts) = match self.newest_frame {
// `frame_index` is newer iff it's within the forward half of the index space.
@@ -819,9 +862,17 @@ impl ReassemblyWindow {
let before = self.frames.len();
let completed = &mut self.completed;
let partial_on = partial_sink.is_some();
self.frames.retain(|&idx, f| {
// Partial-deliverable frames age out on the TIGHT fuse (see PARTIAL_WINDOW_NS);
// everything else keeps the full loss window.
let window_ns = if partial_on && f.user_flags & USER_FLAG_CHUNK_ALIGNED != 0 {
PARTIAL_WINDOW_NS
} else {
LOSS_WINDOW_NS
};
let keep = newest.wrapping_sub(idx) <= HARD_LOSS_WINDOW
&& newest_pts.saturating_sub(f.pts_ns) <= LOSS_WINDOW_NS;
&& newest_pts.saturating_sub(f.pts_ns) <= window_ns;
if !keep {
// Remember the abandoned index so a straggler shard is dropped (below, and in
// `push`) instead of resurrecting the frame — which would re-allocate its buffers
@@ -831,6 +882,28 @@ impl ReassemblyWindow {
completed.insert(idx, reconstructed_shards(&f.blocks, max_data_shards));
// Release its buffer budget and reclaim its parity bufs for the pool.
*in_flight_bytes -= f.buf.len();
// Partial delivery (chunk-aligned AUs only): the buffer is already exactly
// what the consumer needs — received shards at their final offsets, zeros
// where shards are missing (the codec's block walk skips zero windows).
// Newest-wins if several age out in one prune. Still counted dropped below.
if let Some(sink) = partial_sink.as_deref_mut() {
if f.user_flags & USER_FLAG_CHUNK_ALIGNED != 0 {
let mut buf = std::mem::take(&mut f.buf);
buf.truncate(f.frame_bytes);
let newer = sink
.as_ref()
.is_none_or(|p| idx.wrapping_sub(p.frame_index) <= u32::MAX / 2);
if newer {
*sink = Some(Frame {
data: buf,
frame_index: idx,
pts_ns: f.pts_ns,
flags: f.user_flags,
complete: false,
});
}
}
}
for block in f.blocks.values_mut() {
for slot in block.recovery.iter_mut() {
if let Some(rb) = slot.take() {
+125
View File
@@ -24,6 +24,12 @@ pub struct Frame {
pub frame_index: u32,
pub pts_ns: u64,
pub flags: u32,
/// `false` = a partial delivery: the frame aged out of the loss window with shards
/// missing, and the session opted into receiving it anyway
/// ([`Session::set_deliver_partial_frames`]). Only chunk-aligned AUs
/// ([`crate::packet::USER_FLAG_CHUNK_ALIGNED`]) are ever delivered partial; missing
/// shard ranges are zero-filled at their exact offsets.
pub complete: bool,
}
/// One end of a stream. Constructed for a single [`Role`]; calling the other role's
@@ -634,6 +640,20 @@ impl Session {
/// Client: drain the transport until a whole access unit is recovered, or no more
/// packets are pending ([`PunktfunkError::NoFrame`]).
/// Client opt-in: deliver aged-out incomplete chunk-aligned frames as
/// [`Frame`]`{ complete: false }` instead of only dropping them (the PyroWave
/// datagram-aligned mode, plan §4.4 — a lost datagram costs a few blocks of blur,
/// not the frame). No effect on other codecs' AUs (they never carry the flag).
pub fn set_deliver_partial_frames(&mut self, on: bool) {
self.reassembler.set_deliver_partial(on);
}
/// The session's negotiated wire shard payload size (bytes of AU per datagram) —
/// the window size for chunk-aligned AUs (`USER_FLAG_CHUNK_ALIGNED`).
pub fn shard_payload(&self) -> usize {
self.config.shard_payload
}
pub fn poll_frame(&mut self) -> Result<Frame> {
if self.config.role != Role::Client {
return Err(PunktfunkError::InvalidArg(
@@ -661,6 +681,11 @@ impl Session {
}
self.recv_idx = 0;
if self.recv_count == 0 {
// Nothing new on the wire — hand over an aged-out partial if one is
// waiting (it can only get staler).
if let Some(p) = self.reassembler.take_partial() {
return Ok(p);
}
return Err(PunktfunkError::NoFrame);
}
}
@@ -725,6 +750,11 @@ impl Session {
StatsCounters::add(&self.stats.frames_completed, 1);
return Ok(frame);
}
// A push that completed nothing may still have aged a partial out — deliver it
// ahead of further draining (its successors are already arriving).
if let Some(p) = self.reassembler.take_partial() {
return Ok(p);
}
}
}
@@ -980,6 +1010,101 @@ mod wire_equivalence_tests {
fn pattern(len: usize) -> Vec<u8> {
(0..len).map(|i| (i * 31 + 7) as u8).collect()
}
/// Partial delivery (plan §4.4): a chunk-aligned frame that loses shards past FEC's
/// reach is DELIVERED once it ages out — `complete: false`, received shards at their
/// exact offsets, missing ranges zero-filled — instead of silently dropping. Plain
/// AUs (no flag) keep the drop behavior even with the opt-in enabled.
#[test]
fn partial_delivery_of_chunk_aligned_frames() {
use crate::packet::USER_FLAG_CHUNK_ALIGNED;
let mk = |role| Config {
role,
phase: ProtocolPhase::P2Punktfunk,
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 0, // no parity — any drop leaves a hole
max_data_per_block: 64,
},
shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024,
encrypt: false,
key: [0u8; 16],
salt: [0u8; 4],
loopback_drop_period: 0,
};
// Drop exactly one datagram (the 3rd) out of the first frame's shards.
let (h, c) = crate::transport::loopback_pair(3, 1);
let mut host = Session::new(mk(Role::Host), Box::new(h)).unwrap();
let mut client = Session::new(mk(Role::Client), Box::new(c)).unwrap();
client.set_deliver_partial_frames(true);
// 8 shards of chunk-aligned payload.
let frame = pattern(8 * 1024);
host.submit_frame(&frame, 1_000, USER_FLAG_CHUNK_ALIGNED)
.unwrap();
// The incomplete frame ages out on the HARD index window — push enough newer
// (complete) frames past it. Collect everything the client emits.
let mut got_partial = None;
let mut completes = 0;
for i in 0..80u64 {
let filler = pattern(1024);
host.submit_frame(&filler, 2_000 + i, USER_FLAG_CHUNK_ALIGNED)
.unwrap();
loop {
match client.poll_frame() {
Ok(f) if !f.complete => got_partial = Some(f),
Ok(_) => completes += 1,
Err(PunktfunkError::NoFrame) => break,
Err(e) => panic!("unexpected: {e}"),
}
}
}
let p = got_partial.expect("the lossy frame must be delivered partial");
assert_eq!(p.pts_ns, 1_000);
assert_eq!(p.data.len(), frame.len());
assert!(p.flags & USER_FLAG_CHUNK_ALIGNED != 0);
// Exactly one 1024-byte shard is zeroed; every other offset matches the original.
let mut zero_windows = 0;
for w in 0..8 {
let win = &p.data[w * 1024..(w + 1) * 1024];
if win.iter().all(|&b| b == 0) {
zero_windows += 1;
} else {
assert_eq!(win, &frame[w * 1024..(w + 1) * 1024], "window {w} corrupt");
}
}
// loopback_pair(3, _) drops every 3rd datagram, so several of the 8 shards are
// gone — the exact count depends on phase; what matters is that SOME are zeroed
// and every survivor is intact.
assert!(
(1..8).contains(&zero_windows),
"dropped shards zero-filled (got {zero_windows})"
);
assert!(completes > 40, "surviving filler frames flow normally");
// Control: WITHOUT the flag the same loss is a plain drop, opt-in or not.
let (h2, c2) = crate::transport::loopback_pair(3, 1);
let mut host2 = Session::new(mk(Role::Host), Box::new(h2)).unwrap();
let mut client2 = Session::new(mk(Role::Client), Box::new(c2)).unwrap();
client2.set_deliver_partial_frames(true);
host2.submit_frame(&pattern(8 * 1024), 1_000, 0).unwrap();
let mut saw_partial = false;
for i in 0..80u64 {
host2.submit_frame(&pattern(1024), 2_000 + i, 0).unwrap();
loop {
match client2.poll_frame() {
Ok(f) => saw_partial |= !f.complete,
Err(PunktfunkError::NoFrame) => break,
Err(e) => panic!("unexpected: {e}"),
}
}
}
assert!(
!saw_partial,
"unflagged AUs must never be delivered partial"
);
}
}
#[cfg(test)]