feat(core,host,client): PyroWave datagram-aligned packets + partial-frame delivery (Phase 4, §4.4)
ci / web (push) Successful in 1m10s
ci / docs-site (push) Successful in 1m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
apple / swift (push) Successful in 4m59s
ci / rust (push) Failing after 6m17s
ci / bench (push) Successful in 6m36s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m29s
arch / build-publish (push) Successful in 11m11s
android / android (push) Successful in 12m33s
deb / build-publish (push) Successful in 14m8s
windows-host / package (push) Successful in 14m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m4s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m6s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 5m15s
release / apple (push) Successful in 26m8s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 5m48s
apple / screenshots (push) Successful in 20m29s
ci / web (push) Successful in 1m10s
ci / docs-site (push) Successful in 1m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
apple / swift (push) Successful in 4m59s
ci / rust (push) Failing after 6m17s
ci / bench (push) Successful in 6m36s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m29s
arch / build-publish (push) Successful in 11m11s
android / android (push) Successful in 12m33s
deb / build-publish (push) Successful in 14m8s
windows-host / package (push) Successful in 14m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m4s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m6s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 5m15s
release / apple (push) Successful in 26m8s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 5m48s
apple / screenshots (push) Successful in 20m29s
PyroWave AUs now packetize on the negotiated shard payload, so a lost datagram
costs a few wavelet blocks of localized blur rather than a whole frame — and the
client can render an aged-out lossy frame instead of freezing until the next one.
Host (opt-in, PyroWave only):
- The encoder packetizes at the shard payload behind a 4-byte window prefix
(used-len u16 + kind u16). Whole packets pack into WIN_PACKED windows; a packet
too large for one shard (PyroWave 32x32 blocks are atomic and can exceed a
shard) rides a WIN_FRAG_FIRST/CONT/LAST chain. `set_wire_chunking()` joins the
Encoder trait (forwarded through TrackedEncoder — the silent-no-op trap);
EncodedFrame.chunk_aligned marks the AU.
- virtual_stream tags the AU with USER_FLAG_CHUNK_ALIGNED and re-applies chunking
after every encoder (re)build, the adaptive-bitrate rebuild included.
Core:
- USER_FLAG_CHUNK_ALIGNED (0x40) wire bit. Reassembler opt-in
(set_deliver_partial): a chunk-aligned frame that ages out with holes is handed
over as Frame{complete:false} — received shards at their exact offsets, missing
ranges zero-filled — instead of being dropped. Partials age out on a tight 30ms
fuse (PARTIAL_WINDOW_NS) instead of the 120ms loss window: each frame is
independently decodable, so an ancient partial has no value in a live stream.
Newest-wins. A partial still counts as dropped for loss reporting.
Client (PyroWave decode):
- The session opts in when codec == PyroWave. The decoder walks the AU
window-by-window, skipping zero (missing) windows and reassembling FRAG chains,
then decodes whatever survived. A newest-decoded-index guard drops partials the
pump has already moved past (no time-travel present).
Also fixes a redundant-closure clippy nit in the PyroWave planar-present path.
Validated on an RTX 5070 Ti under 2% netem loss with FEC pinned off: 60fps
sustained entirely via partials, e2e 43ms p50 (146ms before the fuse) vs 23ms
lossless, no keyframe-recovery chatter. Tests green: core 149, host 310 + the
GPU-gated encoder smoke (framed-window walk + FRAG reassembly + upstream
round-trip), client 26; clippy clean on the pyrowave feature combos.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -652,6 +652,7 @@ impl Encoder for NvencEncoder {
|
||||
pts_ns,
|
||||
keyframe: pkt.is_key(),
|
||||
recovery_anchor: false,
|
||||
chunk_aligned: false,
|
||||
}))
|
||||
}
|
||||
// No packet ready yet (need another input frame).
|
||||
|
||||
@@ -1148,6 +1148,7 @@ impl Encoder for NvencCudaEncoder {
|
||||
pts_ns,
|
||||
keyframe,
|
||||
recovery_anchor: anchor,
|
||||
chunk_aligned: false,
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -43,6 +43,13 @@ const IMPORT_CACHE_CAP: usize = 16;
|
||||
/// Headroom over the per-frame rate budget for the packetized bitstream (block headers + meta;
|
||||
/// the rate controller itself never exceeds the budget).
|
||||
const BS_SLACK: usize = 256 * 1024;
|
||||
/// Chunked-mode window framing (§4.4): 4-byte prefix per shard-sized window.
|
||||
const WINDOW_PREFIX: usize = 4;
|
||||
/// Window kinds: whole packets / an oversized packet's fragments.
|
||||
const WIN_PACKED: u16 = 0;
|
||||
const WIN_FRAG_FIRST: u16 = 1;
|
||||
const WIN_FRAG_CONT: u16 = 2;
|
||||
const WIN_FRAG_LAST: u16 = 3;
|
||||
|
||||
/// The DRM modifiers the PyroWave device can import as a SAMPLED image of the capture's
|
||||
/// packed-RGB format. The capture advertises these for the pyrowave passthrough instead of
|
||||
@@ -178,6 +185,10 @@ pub struct PyroWaveEncoder {
|
||||
fps: u32,
|
||||
/// Per-frame bitstream budget (hard CBR): `bitrate / (8 * fps)`.
|
||||
frame_budget: usize,
|
||||
/// Datagram-aligned mode (plan §4.4): packetize at this boundary and pad every codec
|
||||
/// packet to it, so each wire shard carries whole self-delimiting packets. `None` =
|
||||
/// one packet per AU (the dense MVP shape).
|
||||
wire_chunk: Option<usize>,
|
||||
bitstream: Vec<u8>,
|
||||
pending: VecDeque<EncodedFrame>,
|
||||
frame_count: u64,
|
||||
@@ -532,6 +543,7 @@ impl PyroWaveEncoder {
|
||||
height: h,
|
||||
fps,
|
||||
frame_budget,
|
||||
wire_chunk: None,
|
||||
bitstream: Vec::new(),
|
||||
pending: VecDeque::new(),
|
||||
frame_count: 0,
|
||||
@@ -865,31 +877,100 @@ impl PyroWaveEncoder {
|
||||
dev.wait_for_fences(&[self.fence], true, 5_000_000_000)
|
||||
.context("pyrowave encode fence")?;
|
||||
|
||||
// ---- packetize: boundary = whole buffer, so the AU is exactly one pyrowave packet ----
|
||||
// ---- packetize ----
|
||||
// Dense (default): boundary = whole buffer → the AU is exactly one pyrowave packet.
|
||||
// Datagram-aligned (§4.4, `set_wire_chunking`): boundary = the wire shard payload;
|
||||
// each codec packet is zero-padded to the boundary so every shard carries whole
|
||||
// self-delimiting packets — the client windows its parse and a lost shard costs
|
||||
// only those blocks. Padding cost is small: the packetizer fills close to the
|
||||
// boundary by design.
|
||||
let cap = self.frame_budget + BS_SLACK;
|
||||
self.bitstream.resize(cap, 0);
|
||||
// Chunked mode reserves 4 bytes per window for the framing prefix.
|
||||
let boundary = self.wire_chunk.map(|c| c - WINDOW_PREFIX).unwrap_or(cap);
|
||||
let mut n: usize = 0;
|
||||
pw_check(
|
||||
pw::pyrowave_encoder_compute_num_packets(self.pw_enc, cap, &mut n),
|
||||
pw::pyrowave_encoder_compute_num_packets(self.pw_enc, boundary, &mut n),
|
||||
"compute_num_packets",
|
||||
)?;
|
||||
if n != 1 {
|
||||
bail!("pyrowave: expected a single packet at boundary {cap}, got {n}");
|
||||
if n == 0 || (self.wire_chunk.is_none() && n != 1) {
|
||||
bail!("pyrowave: unexpected packet count {n} at boundary {boundary}");
|
||||
}
|
||||
let mut packet = pw::pyrowave_packet { offset: 0, size: 0 };
|
||||
let mut packets = vec![pw::pyrowave_packet { offset: 0, size: 0 }; n];
|
||||
let mut out_n: usize = 0;
|
||||
pw_check(
|
||||
pw::pyrowave_encoder_packetize(
|
||||
self.pw_enc,
|
||||
&mut packet,
|
||||
cap,
|
||||
packets.as_mut_ptr(),
|
||||
boundary,
|
||||
&mut out_n,
|
||||
self.bitstream.as_mut_ptr() as *mut std::ffi::c_void,
|
||||
cap,
|
||||
),
|
||||
"packetize",
|
||||
)?;
|
||||
let au = self.bitstream[packet.offset..packet.offset + packet.size].to_vec();
|
||||
packets.truncate(out_n.max(1));
|
||||
let au = if let Some(chunk) = self.wire_chunk {
|
||||
// Window framing (§4.4): each `chunk`-sized window opens with a 4-byte prefix
|
||||
// (u16 used-length + u16 kind) and carries either WHOLE self-delimiting codec
|
||||
// packets (PACKED — several small ones share a window) or one fragment of an
|
||||
// oversized packet (FRAG chain — pyrowave 32×32 blocks are atomic and may
|
||||
// exceed a shard). A lost shard zeroes its window (used = 0) — the receiver
|
||||
// skips it and drops any fragment chain it interrupts.
|
||||
let payload_max = chunk - WINDOW_PREFIX;
|
||||
let mut au: Vec<u8> = Vec::with_capacity((packets.len() + 1) * chunk);
|
||||
// The currently-open PACKED window: (start offset of its prefix, bytes used).
|
||||
let mut open: Option<(usize, usize)> = None;
|
||||
let close = |au: &mut Vec<u8>, open: &mut Option<(usize, usize)>, chunk: usize| {
|
||||
if let Some((start, used)) = open.take() {
|
||||
au[start..start + 2].copy_from_slice(&(used as u16).to_le_bytes());
|
||||
au[start + 2..start + 4].copy_from_slice(&WIN_PACKED.to_le_bytes());
|
||||
au.resize(start + chunk, 0);
|
||||
}
|
||||
};
|
||||
for p in &packets {
|
||||
let bytes = &self.bitstream[p.offset..p.offset + p.size];
|
||||
if p.size <= payload_max {
|
||||
let fits = open.is_some_and(|(_, used)| used + p.size <= payload_max);
|
||||
if !fits {
|
||||
close(&mut au, &mut open, chunk);
|
||||
let start = au.len();
|
||||
au.resize(start + WINDOW_PREFIX, 0);
|
||||
open = Some((start, 0));
|
||||
}
|
||||
au.extend_from_slice(bytes);
|
||||
if let Some((_, used)) = open.as_mut() {
|
||||
*used += p.size;
|
||||
}
|
||||
} else {
|
||||
// Oversized packet: its own FRAG chain of full windows.
|
||||
close(&mut au, &mut open, chunk);
|
||||
let mut off = 0usize;
|
||||
while off < p.size {
|
||||
let take = (p.size - off).min(payload_max);
|
||||
let kind = if off == 0 {
|
||||
WIN_FRAG_FIRST
|
||||
} else if off + take == p.size {
|
||||
WIN_FRAG_LAST
|
||||
} else {
|
||||
WIN_FRAG_CONT
|
||||
};
|
||||
let start = au.len();
|
||||
au.resize(start + WINDOW_PREFIX, 0);
|
||||
au[start..start + 2].copy_from_slice(&(take as u16).to_le_bytes());
|
||||
au[start + 2..start + 4].copy_from_slice(&kind.to_le_bytes());
|
||||
au.extend_from_slice(&bytes[off..off + take]);
|
||||
au.resize(start + chunk, 0);
|
||||
off += take;
|
||||
}
|
||||
}
|
||||
}
|
||||
close(&mut au, &mut open, chunk);
|
||||
au
|
||||
} else {
|
||||
let p = &packets[0];
|
||||
self.bitstream[p.offset..p.offset + p.size].to_vec()
|
||||
};
|
||||
self.frame_count += 1;
|
||||
self.pending.push_back(EncodedFrame {
|
||||
data: au,
|
||||
@@ -898,6 +979,7 @@ impl PyroWaveEncoder {
|
||||
// whole recovery story (plan §1.2).
|
||||
keyframe: true,
|
||||
recovery_anchor: false,
|
||||
chunk_aligned: self.wire_chunk.is_some(),
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
@@ -960,6 +1042,17 @@ impl Encoder for PyroWaveEncoder {
|
||||
true
|
||||
}
|
||||
|
||||
fn set_wire_chunking(&mut self, shard_payload: usize) {
|
||||
// Sanity floor: a boundary below one block header + payload word is meaningless.
|
||||
if shard_payload >= 64 {
|
||||
self.wire_chunk = Some(shard_payload);
|
||||
tracing::info!(
|
||||
shard_payload,
|
||||
"pyrowave: datagram-aligned packetization on (partial-frame loss mode)"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn flush(&mut self) -> Result<()> {
|
||||
// Synchronous per-frame encode: nothing buffered beyond `pending`.
|
||||
Ok(())
|
||||
@@ -1127,6 +1220,83 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
// Datagram-aligned mode (§4.4): every emitted AU is a whole number of framed
|
||||
// windows — 4-byte prefix (used-length + kind), whole packets or FRAG chains for
|
||||
// oversized atomic blocks, zero padding after `used`. Walking + reassembling the
|
||||
// fragments must reproduce a decodable packet stream.
|
||||
enc.set_wire_chunking(1408);
|
||||
enc.submit(&cpu_frame(w, h, 500, [90, 60, 30, 255]))
|
||||
.expect("chunked submit");
|
||||
let au = enc.poll().expect("poll").expect("chunked AU");
|
||||
assert!(au.chunk_aligned);
|
||||
assert_eq!(au.data.len() % 1408, 0, "AU is a whole number of windows");
|
||||
// SAFETY: test-only FFI with locally-owned buffers.
|
||||
unsafe {
|
||||
let mut dev: pw::pyrowave_device = std::ptr::null_mut();
|
||||
assert_eq!(
|
||||
pw::pyrowave_create_default_device(&mut dev),
|
||||
pw::pyrowave_result_PYROWAVE_SUCCESS
|
||||
);
|
||||
let dinfo = pw::pyrowave_decoder_create_info {
|
||||
device: dev,
|
||||
width: w as i32,
|
||||
height: h as i32,
|
||||
chroma: pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420,
|
||||
fragment_path: false,
|
||||
};
|
||||
let mut dec: pw::pyrowave_decoder = std::ptr::null_mut();
|
||||
assert_eq!(
|
||||
pw::pyrowave_decoder_create(&dinfo, &mut dec),
|
||||
pw::pyrowave_result_PYROWAVE_SUCCESS
|
||||
);
|
||||
let mut frag: Vec<u8> = Vec::new();
|
||||
let mut pushed = 0usize;
|
||||
for win in au.data.chunks(1408) {
|
||||
let used = u16::from_le_bytes([win[0], win[1]]) as usize;
|
||||
let kind = u16::from_le_bytes([win[2], win[3]]);
|
||||
assert!(4 + used <= win.len(), "window overrun");
|
||||
assert!(win[4 + used..].iter().all(|&b| b == 0), "non-zero padding");
|
||||
let body = &win[4..4 + used];
|
||||
match kind {
|
||||
0 => {
|
||||
assert_eq!(
|
||||
pw::pyrowave_decoder_push_packet(
|
||||
dec,
|
||||
body.as_ptr() as *const _,
|
||||
body.len()
|
||||
),
|
||||
pw::pyrowave_result_PYROWAVE_SUCCESS
|
||||
);
|
||||
pushed += body.len();
|
||||
}
|
||||
1 => frag = body.to_vec(),
|
||||
2 => frag.extend_from_slice(body),
|
||||
3 => {
|
||||
frag.extend_from_slice(body);
|
||||
assert_eq!(
|
||||
pw::pyrowave_decoder_push_packet(
|
||||
dec,
|
||||
frag.as_ptr() as *const _,
|
||||
frag.len()
|
||||
),
|
||||
pw::pyrowave_result_PYROWAVE_SUCCESS
|
||||
);
|
||||
pushed += frag.len();
|
||||
frag.clear();
|
||||
}
|
||||
k => panic!("unknown window kind {k}"),
|
||||
}
|
||||
}
|
||||
assert!(pushed > 0, "chunked AU carries real packets");
|
||||
assert!(
|
||||
pw::pyrowave_decoder_decode_is_ready(dec, false),
|
||||
"chunked AU incomplete after framed walk"
|
||||
);
|
||||
pw::pyrowave_decoder_destroy(dec);
|
||||
pw::pyrowave_device_destroy(dev);
|
||||
}
|
||||
enc.set_wire_chunking(0); // below the floor — back to dense
|
||||
|
||||
// In-place rate retarget + encoder rebuild both keep encoding.
|
||||
assert!(enc.reconfigure_bitrate(100_000_000));
|
||||
assert!(enc.reset());
|
||||
|
||||
@@ -297,6 +297,7 @@ fn poll_encoder(enc: &mut encoder::video::Encoder, fps: u32) -> Result<Option<En
|
||||
pts_ns: pts * 1_000_000_000 / fps as u64,
|
||||
keyframe: pkt.is_key(),
|
||||
recovery_anchor: false,
|
||||
chunk_aligned: false,
|
||||
}))
|
||||
}
|
||||
Err(ffmpeg::Error::Other { errno })
|
||||
|
||||
@@ -1713,6 +1713,7 @@ impl VulkanVideoEncoder {
|
||||
pts_ns: f.pts_ns,
|
||||
keyframe: f.keyframe,
|
||||
recovery_anchor: f.recovery_anchor,
|
||||
chunk_aligned: false,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user