feat(apple): PyroWave Phase 5 — native Metal decode on Mac / Apple TV / iPad (§4.7)
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 58s
decky / build-publish (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
ci / bench (push) Successful in 5m21s
arch / build-publish (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 28s
android / android (push) Successful in 13m23s
deb / build-publish (push) Successful in 12m58s
ci / rust (push) Successful in 18m5s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m31s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m56s
flatpak / build-publish (push) Successful in 5m53s
windows-host / package (push) Successful in 14m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m12s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m6s
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
release / apple (push) Has been cancelled

The Apple client now decodes PyroWave natively on the presenter's own MTLDevice —
no MoltenVK, no upstream C++ in the app. Completes and wires up the decoder whose
early working-tree snapshot rode along in 9127c346:

- MetalWaveletShaders.swift: wavelet_dequant + idwt hand-ported from the vendored
  GLSL (STORAGE_MODE 0 only; subgroup scans → 32-wide simdgroups; DCShift spec
  constant → function constant; precision-1 split: fp16 levels 0-1 / fp32 2-4).
- MetalWaveletDecoder.swift: Swift reimplementation of push_packet/decode_packet
  incl. the Phase-4 chunk-aligned window walk (FRAG chains, zeroed missing shards,
  the >half-blocks partial rule), init_block_meta's block-index space, and the
  42-dequant + 13-idwt dispatch structure with encoder-boundary barriers. SOF-dims
  changes rebuild the size-dependent resources, which is also the mid-stream
  resize path. Ring of 4 output plane sets on the presenter's queue.
- Presenter: pf_frag_planar (3xR8, the planar_csc.frag twin) + renderPlanar with
  a shared present tail; ReadyFrame carries an image enum (.video | .planar).
- Stage2Pipeline: a dedicated PyroWave pump — no VideoToolbox machinery, no
  keyframe/re-anchor recovery (all-intra; partials render as localized blur by
  design), newest-frame-index staleness guard for late partials.
- Opt-in: "PyroWave (wired LAN)" codec entry (probe-gated, ≈A13 floor via a real
  kernel-compile probe), selecting it advertises + prefers the codec and forces
  the session SDR (HDR/10-bit/4:4:4 caps dropped, plan contract).
- Core ABI: punktfunk_connection_shard_payload() — the Welcome's negotiated shard
  payload, needed by native decoders to walk chunk-aligned AUs.
- Validation: golden fixtures generated by the host encoder + upstream's own
  decoder (pyrowave_dump_golden, RTX 5070 Ti); the Metal decode PSNR-matches at
  77-88 dB across all planes for dense AND chunk-aligned AUs, and a hole-punched
  partial still decodes. Parser unit tests cover the window walk, FRAG chains,
  broken chains, the half-blocks gate, and the block-index layout.

Tests: apple 134 green (mac; iOS/tvOS build), host 312 w/ pyrowave on .21,
core 148 w/ quic; clippy/fmt clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-15 12:16:43 +02:00
parent a70811043e
commit 739a5f76bf
28 changed files with 826 additions and 81 deletions
+27
View File
@@ -2230,6 +2230,33 @@ pub unsafe extern "C" fn punktfunk_connection_codec(
})
}
/// Read the session's negotiated wire shard payload (the `Welcome`'s value, bytes). This is the
/// parse-window size of a [`USER_FLAG_CHUNK_ALIGNED`] AU (PyroWave datagram-aligned mode,
/// design/pyrowave-codec-plan.md §4.4): every `shard_payload`-sized window of the frame buffer
/// starts a fresh self-delimiting chunk. Clients that decode PyroWave natively (the Apple Metal
/// port) need it to walk those AUs; other codecs never need this.
///
/// # Safety
/// `c` is a valid connection handle; `out` is NULL or writable for one `u32`.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_shard_payload(
c: *mut PunktfunkConnection,
out: *mut u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if !out.is_null() {
// SAFETY: `out` is non-null and the caller guarantees it is writable for one `u32`.
unsafe { *out = u32::from(c.inner.shard_payload) };
}
PunktfunkStatus::Ok
})
}
/// Send one input event to the host as a QUIC datagram (non-blocking enqueue).
///
/// # Safety
@@ -1130,10 +1130,10 @@ mod tests {
)
}
/// Decode an AU with a standalone pyrowave CPU decoder and return plane means (Y, Cb, Cr).
/// This is the Phase-1 "golden frames" oracle: the host-encoded bitstream must round-trip
/// through upstream's own decoder to the CSC's expected values.
unsafe fn decode_plane_means(w: u32, h: u32, au: &[u8]) -> (f64, f64, f64) {
/// Decode an AU with a standalone pyrowave decoder and return the full YUV420P planes.
/// This is the golden oracle for both the Phase-1 smoke check (plane means) and the Apple
/// Metal port's committed PSNR fixtures (`pyrowave_dump_golden`).
unsafe fn decode_planes(w: u32, h: u32, au: &[u8]) -> (Vec<u8>, Vec<u8>, Vec<u8>) {
let mut dev: pw::pyrowave_device = std::ptr::null_mut();
assert_eq!(
pw::pyrowave_create_default_device(&mut dev),
@@ -1177,7 +1177,13 @@ mod tests {
);
pw::pyrowave_decoder_destroy(dec);
pw::pyrowave_device_destroy(dev);
(y, cb, cr)
}
/// Plane means of an upstream-decoded AU — the Phase-1 smoke assertion.
unsafe fn decode_plane_means(w: u32, h: u32, au: &[u8]) -> (f64, f64, f64) {
// SAFETY: forwarded — same contract as the caller.
let (y, cb, cr) = unsafe { decode_planes(w, h, au) };
let mean = |v: &[u8]| v.iter().map(|&x| x as f64).sum::<f64>() / v.len() as f64;
(mean(&y), mean(&cb), mean(&cr))
}
@@ -1304,4 +1310,107 @@ mod tests {
.expect("submit after reset");
assert!(enc.poll().expect("poll").is_some());
}
/// A deterministic busy BGRA test card (gradients + checker + LCG noise) — flat fills
/// exercise almost none of the entropy decoder, this hits every subband.
fn test_card(w: u32, h: u32, seed: u32) -> CapturedFrame {
let mut rng = seed | 1;
let mut buf = vec![0u8; (w * h * 4) as usize];
for y in 0..h {
for x in 0..w {
rng = rng.wrapping_mul(1664525).wrapping_add(1013904223);
let i = ((y * w + x) * 4) as usize;
let checker = if (x / 16 + y / 16) % 2 == 0 { 48 } else { 0 };
let noise = (rng >> 24) as u8 / 8;
buf[i] = ((x * 255 / w) as u8).saturating_add(noise); // B
buf[i + 1] = ((y * 255 / h) as u8).saturating_add(checker); // G
buf[i + 2] = (((x + y) * 255 / (w + h)) as u8).saturating_add(noise); // R
buf[i + 3] = 255;
}
}
CapturedFrame {
width: w,
height: h,
pts_ns: seed as u64 * 16_666_667,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(buf),
}
}
/// Dump the Apple Metal port's golden fixtures (plan §4.7): host-encoded AUs (dense AND
/// chunk-aligned) plus upstream's own decode of each as raw YUV420P planes. The Swift test
/// (PyroWaveGoldenTests.swift) PSNR-matches the Metal decode against these — float wavelet
/// math is not bit-exact across implementations, upstream itself ships precision variants.
/// `#[ignore]`d GPU test; regenerate on a Vulkan 1.3 host:
/// cargo test -p punktfunk-host --features pyrowave --no-run
/// PYROWAVE_GOLDEN_DIR=/tmp/golden <bin> --ignored --nocapture pyrowave_dump_golden
/// then copy the files into clients/apple/Tests/PunktfunkKitTests/PyroWaveFixtures/.
#[test]
#[ignore = "fixture generator — needs a real Vulkan 1.3 compute device"]
fn pyrowave_dump_golden() {
let dir = match std::env::var("PYROWAVE_GOLDEN_DIR") {
Ok(d) => std::path::PathBuf::from(d),
Err(_) => {
eprintln!("PYROWAVE_GOLDEN_DIR not set — skipping dump");
return;
}
};
std::fs::create_dir_all(&dir).expect("create golden dir");
// Odd-block geometry on purpose: 256 aligns clean, 144 → aligned 160 exercises the
// block-grid overhang. ~1.6 bpp at 60 fps.
let (w, h) = (256u32, 144u32);
let mut enc = PyroWaveEncoder::open(w, h, 60, 4_000_000).expect("open");
let dump = |name: &str, bytes: &[u8]| {
std::fs::write(dir.join(name), bytes).expect("write fixture");
eprintln!("wrote {name}: {} bytes", bytes.len());
};
// Dense AU + upstream-decoded reference planes.
enc.submit(&test_card(w, h, 7)).expect("submit");
let au = enc.poll().expect("poll").expect("AU");
assert!(!au.chunk_aligned);
dump("au-dense.bin", &au.data);
// SAFETY: test-only FFI with locally-owned buffers.
let (y, cb, cr) = unsafe { decode_planes(w, h, &au.data) };
dump("ref-dense-y.bin", &y);
dump("ref-dense-cb.bin", &cb);
dump("ref-dense-cr.bin", &cr);
// Chunk-aligned AU of a DIFFERENT frame (its own reference): the Swift window walk +
// FRAG reassembly must reproduce the packet stream.
enc.set_wire_chunking(1408);
enc.submit(&test_card(w, h, 11)).expect("chunked submit");
let au = enc.poll().expect("poll").expect("chunked AU");
assert!(au.chunk_aligned);
assert_eq!(au.data.len() % 1408, 0);
dump("au-chunked.bin", &au.data);
// SAFETY: test-only FFI with locally-owned buffers.
let (y, cb, cr) = unsafe {
// Feed upstream through the same framed walk the clients use.
let mut stream = Vec::new();
let mut frag: Vec<u8> = Vec::new();
for win in au.data.chunks(1408) {
let used = u16::from_le_bytes([win[0], win[1]]) as usize;
let kind = u16::from_le_bytes([win[2], win[3]]);
let body = &win[4..4 + used];
match kind {
0 => stream.extend_from_slice(body),
1 => frag = body.to_vec(),
2 => frag.extend_from_slice(body),
3 => {
frag.extend_from_slice(body);
stream.extend_from_slice(&frag);
frag.clear();
}
k => panic!("unknown window kind {k}"),
}
}
decode_planes(w, h, &stream)
};
dump("ref-chunked-y.bin", &y);
dump("ref-chunked-cb.bin", &cb);
dump("ref-chunked-cr.bin", &cr);
}
}