feat(core,host,client): PyroWave datagram-aligned packets + partial-frame delivery (Phase 4, §4.4)
ci / web (push) Successful in 1m10s
ci / docs-site (push) Successful in 1m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
apple / swift (push) Successful in 4m59s
ci / rust (push) Failing after 6m17s
ci / bench (push) Successful in 6m36s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m29s
arch / build-publish (push) Successful in 11m11s
android / android (push) Successful in 12m33s
deb / build-publish (push) Successful in 14m8s
windows-host / package (push) Successful in 14m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m4s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m6s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 5m15s
release / apple (push) Successful in 26m8s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 5m48s
apple / screenshots (push) Successful in 20m29s

PyroWave AUs now packetize on the negotiated shard payload, so a lost datagram
costs a few wavelet blocks of localized blur rather than a whole frame — and the
client can render an aged-out lossy frame instead of freezing until the next one.

Host (opt-in, PyroWave only):
- The encoder packetizes at the shard payload behind a 4-byte window prefix
  (used-len u16 + kind u16). Whole packets pack into WIN_PACKED windows; a packet
  too large for one shard (PyroWave 32x32 blocks are atomic and can exceed a
  shard) rides a WIN_FRAG_FIRST/CONT/LAST chain. `set_wire_chunking()` joins the
  Encoder trait (forwarded through TrackedEncoder — the silent-no-op trap);
  EncodedFrame.chunk_aligned marks the AU.
- virtual_stream tags the AU with USER_FLAG_CHUNK_ALIGNED and re-applies chunking
  after every encoder (re)build, the adaptive-bitrate rebuild included.

Core:
- USER_FLAG_CHUNK_ALIGNED (0x40) wire bit. Reassembler opt-in
  (set_deliver_partial): a chunk-aligned frame that ages out with holes is handed
  over as Frame{complete:false} — received shards at their exact offsets, missing
  ranges zero-filled — instead of being dropped. Partials age out on a tight 30ms
  fuse (PARTIAL_WINDOW_NS) instead of the 120ms loss window: each frame is
  independently decodable, so an ancient partial has no value in a live stream.
  Newest-wins. A partial still counts as dropped for loss reporting.

Client (PyroWave decode):
- The session opts in when codec == PyroWave. The decoder walks the AU
  window-by-window, skipping zero (missing) windows and reassembling FRAG chains,
  then decodes whatever survived. A newest-decoded-index guard drops partials the
  pump has already moved past (no time-travel present).

Also fixes a redundant-closure clippy nit in the PyroWave planar-present path.

Validated on an RTX 5070 Ti under 2% netem loss with FEC pinned off: 60fps
sustained entirely via partials, e2e 43ms p50 (146ms before the fuse) vs 23ms
lossless, no keyframe-recovery chatter. Tests green: core 149, host 310 + the
GPU-gated encoder smoke (framed-window walk + FRAG reassembly + upstream
round-trip), client 26; clippy clean on the pyrowave feature combos.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-15 11:14:24 +02:00
parent 1fc9ef0050
commit 705a8baddf
21 changed files with 619 additions and 33 deletions
+110 -13
View File
@@ -190,6 +190,9 @@ pub struct PyroWaveDecoder {
fence: vk::Fence,
width: u32,
height: u32,
/// The wire shard payload — the parse-window size for chunk-aligned AUs (§4.4): each
/// window holds whole self-delimiting codec packets, zero-padded to the window.
wire_window: usize,
}
// SAFETY: used only from the single decode thread; the shared-queue accesses go through
@@ -197,7 +200,12 @@ pub struct PyroWaveDecoder {
unsafe impl Send for PyroWaveDecoder {}
impl PyroWaveDecoder {
pub fn new(vkd: &VulkanDecodeDevice, width: u32, height: u32) -> Result<PyroWaveDecoder> {
pub fn new(
vkd: &VulkanDecodeDevice,
width: u32,
height: u32,
shard_payload: usize,
) -> Result<PyroWaveDecoder> {
if !vkd.pyrowave_decode {
bail!("presenter device lacks the PyroWave compute feature set");
}
@@ -207,13 +215,14 @@ impl PyroWaveDecoder {
// SAFETY: the handles in `vkd` are the presenter's live instance/device (it
// outlives the decoder — same contract the FFmpeg Vulkan backend relies on);
// `Hold` pins the reconstructed create-infos for the pyrowave device's lifetime.
unsafe { Self::new_inner(vkd, width, height) }
unsafe { Self::new_inner(vkd, width, height, shard_payload) }
}
unsafe fn new_inner(
vkd: &VulkanDecodeDevice,
width: u32,
height: u32,
shard_payload: usize,
) -> Result<PyroWaveDecoder> {
let static_fn = ash::StaticFn {
get_instance_proc_addr: std::mem::transmute::<usize, vk::PFN_vkGetInstanceProcAddr>(
@@ -380,25 +389,113 @@ impl PyroWaveDecoder {
fence,
width,
height,
wire_window: shard_payload.max(64),
})
}
/// One AU in → one frame out (the AU is a complete pyrowave frame: one packet).
pub fn decode(&mut self, au: &[u8]) -> Result<Option<PyroWavePlanarFrame>> {
/// One AU in → one frame out. `aligned` = the AU is shard-window chunked (each
/// `wire_window` holds whole self-delimiting packets, zero-padded — walk and strip);
/// `complete` = every shard arrived (a partial decodes anyway: missing blocks are
/// localized blur for exactly this frame, §4.4).
pub fn decode_frame(
&mut self,
au: &[u8],
aligned: bool,
complete: bool,
) -> Result<Option<PyroWavePlanarFrame>> {
// SAFETY: single decode thread; all handles owned/pinned by `self`; queue access
// serialized under the device-wide QueueLock; the fence bounds GPU completion
// before the frame is handed to the presenter.
unsafe { self.decode_inner(au) }
unsafe { self.decode_inner(au, aligned, complete) }
}
unsafe fn decode_inner(&mut self, au: &[u8]) -> Result<Option<PyroWavePlanarFrame>> {
pw_check(
pw::pyrowave_decoder_push_packet(self.pw_dec, au.as_ptr() as *const c_void, au.len()),
"push_packet",
)?;
// The reassembler delivers complete AUs only, so a frame is ready per push; a
// stale/duplicate packet (sequence rewind) simply isn't — skip, no error.
if !pw::pyrowave_decoder_decode_is_ready(self.pw_dec, false) {
/// Consume one framed shard window (§4.4): a 4-byte prefix (u16 used-length + u16
/// kind) then either WHOLE self-delimiting codec packets (PACKED) or one fragment of
/// an oversized packet (FRAG chain). A lost shard arrives as a zeroed window
/// (used = 0) — skipped, and it breaks any fragment chain it interrupts (that
/// packet's blocks are unusable without their end; dropping them is the §4.4 blur).
unsafe fn push_window(&mut self, win: &[u8], frag: &mut Vec<u8>) -> Result<()> {
if win.len() < 4 {
return Ok(());
}
let used = u16::from_le_bytes([win[0], win[1]]) as usize;
let kind = u16::from_le_bytes([win[2], win[3]]);
if used == 0 || 4 + used > win.len() {
frag.clear(); // missing / garbage window — drop any chain in progress
return Ok(());
}
let body = &win[4..4 + used];
match kind {
0 => {
frag.clear();
pw_check(
pw::pyrowave_decoder_push_packet(
self.pw_dec,
body.as_ptr() as *const c_void,
body.len(),
),
"push_packet",
)
}
1 => {
frag.clear();
frag.extend_from_slice(body);
Ok(())
}
2 => {
if !frag.is_empty() {
frag.extend_from_slice(body);
}
Ok(())
}
3 => {
if !frag.is_empty() {
frag.extend_from_slice(body);
let r = pw_check(
pw::pyrowave_decoder_push_packet(
self.pw_dec,
frag.as_ptr() as *const c_void,
frag.len(),
),
"push_packet (fragmented)",
);
frag.clear();
return r;
}
Ok(())
}
_ => {
frag.clear();
Ok(())
}
}
}
unsafe fn decode_inner(
&mut self,
au: &[u8],
aligned: bool,
complete: bool,
) -> Result<Option<PyroWavePlanarFrame>> {
if aligned {
let mut frag: Vec<u8> = Vec::new();
for win in au.chunks(self.wire_window) {
self.push_window(win, &mut frag)?;
}
} else {
pw_check(
pw::pyrowave_decoder_push_packet(
self.pw_dec,
au.as_ptr() as *const c_void,
au.len(),
),
"push_packet",
)?;
}
// A complete AU that isn't ready is a stale/duplicate (sequence rewind) — skip.
// A PARTIAL is decoded regardless: missing wavelet blocks reconstruct as zeros,
// i.e. localized blur for exactly this one frame (the next is complete again).
if complete && !pw::pyrowave_decoder_decode_is_ready(self.pw_dec, false) {
return Ok(None);
}