feat(apple): PyroWave Phase 5 — native Metal decode on Mac / Apple TV / iPad (§4.7)
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 58s
decky / build-publish (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
ci / bench (push) Successful in 5m21s
arch / build-publish (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 28s
android / android (push) Successful in 13m23s
deb / build-publish (push) Successful in 12m58s
ci / rust (push) Successful in 18m5s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m31s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m56s
flatpak / build-publish (push) Successful in 5m53s
windows-host / package (push) Successful in 14m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m12s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m6s
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
release / apple (push) Has been cancelled
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 58s
decky / build-publish (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
ci / bench (push) Successful in 5m21s
arch / build-publish (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 28s
android / android (push) Successful in 13m23s
deb / build-publish (push) Successful in 12m58s
ci / rust (push) Successful in 18m5s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m31s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m56s
flatpak / build-publish (push) Successful in 5m53s
windows-host / package (push) Successful in 14m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m12s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m6s
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
release / apple (push) Has been cancelled
The Apple client now decodes PyroWave natively on the presenter's own MTLDevice —
no MoltenVK, no upstream C++ in the app. Completes and wires up the decoder whose
early working-tree snapshot rode along in 9127c346:
- MetalWaveletShaders.swift: wavelet_dequant + idwt hand-ported from the vendored
GLSL (STORAGE_MODE 0 only; subgroup scans → 32-wide simdgroups; DCShift spec
constant → function constant; precision-1 split: fp16 levels 0-1 / fp32 2-4).
- MetalWaveletDecoder.swift: Swift reimplementation of push_packet/decode_packet
incl. the Phase-4 chunk-aligned window walk (FRAG chains, zeroed missing shards,
the >half-blocks partial rule), init_block_meta's block-index space, and the
42-dequant + 13-idwt dispatch structure with encoder-boundary barriers. SOF-dims
changes rebuild the size-dependent resources, which is also the mid-stream
resize path. Ring of 4 output plane sets on the presenter's queue.
- Presenter: pf_frag_planar (3xR8, the planar_csc.frag twin) + renderPlanar with
a shared present tail; ReadyFrame carries an image enum (.video | .planar).
- Stage2Pipeline: a dedicated PyroWave pump — no VideoToolbox machinery, no
keyframe/re-anchor recovery (all-intra; partials render as localized blur by
design), newest-frame-index staleness guard for late partials.
- Opt-in: "PyroWave (wired LAN)" codec entry (probe-gated, ≈A13 floor via a real
kernel-compile probe), selecting it advertises + prefers the codec and forces
the session SDR (HDR/10-bit/4:4:4 caps dropped, plan contract).
- Core ABI: punktfunk_connection_shard_payload() — the Welcome's negotiated shard
payload, needed by native decoders to walk chunk-aligned AUs.
- Validation: golden fixtures generated by the host encoder + upstream's own
decoder (pyrowave_dump_golden, RTX 5070 Ti); the Metal decode PSNR-matches at
77-88 dB across all planes for dense AND chunk-aligned AUs, and a hole-punched
partial still decodes. Parser unit tests cover the window walk, FRAG chains,
broken chains, the half-blocks gate, and the block-index layout.
Tests: apple 134 green (mac; iOS/tvOS build), host 312 w/ pyrowave on .21,
core 148 w/ quic; clippy/fmt clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -2230,6 +2230,33 @@ pub unsafe extern "C" fn punktfunk_connection_codec(
|
||||
})
|
||||
}
|
||||
|
||||
/// Read the session's negotiated wire shard payload (the `Welcome`'s value, bytes). This is the
|
||||
/// parse-window size of a [`USER_FLAG_CHUNK_ALIGNED`] AU (PyroWave datagram-aligned mode,
|
||||
/// design/pyrowave-codec-plan.md §4.4): every `shard_payload`-sized window of the frame buffer
|
||||
/// starts a fresh self-delimiting chunk. Clients that decode PyroWave natively (the Apple Metal
|
||||
/// port) need it to walk those AUs; other codecs never need this.
|
||||
///
|
||||
/// # Safety
|
||||
/// `c` is a valid connection handle; `out` is NULL or writable for one `u32`.
|
||||
#[cfg(feature = "quic")]
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_connection_shard_payload(
|
||||
c: *mut PunktfunkConnection,
|
||||
out: *mut u32,
|
||||
) -> PunktfunkStatus {
|
||||
guard(|| {
|
||||
let c = match unsafe { c.as_ref() } {
|
||||
Some(c) => c,
|
||||
None => return PunktfunkStatus::NullPointer,
|
||||
};
|
||||
if !out.is_null() {
|
||||
// SAFETY: `out` is non-null and the caller guarantees it is writable for one `u32`.
|
||||
unsafe { *out = u32::from(c.inner.shard_payload) };
|
||||
}
|
||||
PunktfunkStatus::Ok
|
||||
})
|
||||
}
|
||||
|
||||
/// Send one input event to the host as a QUIC datagram (non-blocking enqueue).
|
||||
///
|
||||
/// # Safety
|
||||
|
||||
@@ -1130,10 +1130,10 @@ mod tests {
|
||||
)
|
||||
}
|
||||
|
||||
/// Decode an AU with a standalone pyrowave CPU decoder and return plane means (Y, Cb, Cr).
|
||||
/// This is the Phase-1 "golden frames" oracle: the host-encoded bitstream must round-trip
|
||||
/// through upstream's own decoder to the CSC's expected values.
|
||||
unsafe fn decode_plane_means(w: u32, h: u32, au: &[u8]) -> (f64, f64, f64) {
|
||||
/// Decode an AU with a standalone pyrowave decoder and return the full YUV420P planes.
|
||||
/// This is the golden oracle for both the Phase-1 smoke check (plane means) and the Apple
|
||||
/// Metal port's committed PSNR fixtures (`pyrowave_dump_golden`).
|
||||
unsafe fn decode_planes(w: u32, h: u32, au: &[u8]) -> (Vec<u8>, Vec<u8>, Vec<u8>) {
|
||||
let mut dev: pw::pyrowave_device = std::ptr::null_mut();
|
||||
assert_eq!(
|
||||
pw::pyrowave_create_default_device(&mut dev),
|
||||
@@ -1177,7 +1177,13 @@ mod tests {
|
||||
);
|
||||
pw::pyrowave_decoder_destroy(dec);
|
||||
pw::pyrowave_device_destroy(dev);
|
||||
(y, cb, cr)
|
||||
}
|
||||
|
||||
/// Plane means of an upstream-decoded AU — the Phase-1 smoke assertion.
|
||||
unsafe fn decode_plane_means(w: u32, h: u32, au: &[u8]) -> (f64, f64, f64) {
|
||||
// SAFETY: forwarded — same contract as the caller.
|
||||
let (y, cb, cr) = unsafe { decode_planes(w, h, au) };
|
||||
let mean = |v: &[u8]| v.iter().map(|&x| x as f64).sum::<f64>() / v.len() as f64;
|
||||
(mean(&y), mean(&cb), mean(&cr))
|
||||
}
|
||||
@@ -1304,4 +1310,107 @@ mod tests {
|
||||
.expect("submit after reset");
|
||||
assert!(enc.poll().expect("poll").is_some());
|
||||
}
|
||||
|
||||
/// A deterministic busy BGRA test card (gradients + checker + LCG noise) — flat fills
|
||||
/// exercise almost none of the entropy decoder, this hits every subband.
|
||||
fn test_card(w: u32, h: u32, seed: u32) -> CapturedFrame {
|
||||
let mut rng = seed | 1;
|
||||
let mut buf = vec![0u8; (w * h * 4) as usize];
|
||||
for y in 0..h {
|
||||
for x in 0..w {
|
||||
rng = rng.wrapping_mul(1664525).wrapping_add(1013904223);
|
||||
let i = ((y * w + x) * 4) as usize;
|
||||
let checker = if (x / 16 + y / 16) % 2 == 0 { 48 } else { 0 };
|
||||
let noise = (rng >> 24) as u8 / 8;
|
||||
buf[i] = ((x * 255 / w) as u8).saturating_add(noise); // B
|
||||
buf[i + 1] = ((y * 255 / h) as u8).saturating_add(checker); // G
|
||||
buf[i + 2] = (((x + y) * 255 / (w + h)) as u8).saturating_add(noise); // R
|
||||
buf[i + 3] = 255;
|
||||
}
|
||||
}
|
||||
CapturedFrame {
|
||||
width: w,
|
||||
height: h,
|
||||
pts_ns: seed as u64 * 16_666_667,
|
||||
format: PixelFormat::Bgrx,
|
||||
payload: FramePayload::Cpu(buf),
|
||||
}
|
||||
}
|
||||
|
||||
/// Dump the Apple Metal port's golden fixtures (plan §4.7): host-encoded AUs (dense AND
|
||||
/// chunk-aligned) plus upstream's own decode of each as raw YUV420P planes. The Swift test
|
||||
/// (PyroWaveGoldenTests.swift) PSNR-matches the Metal decode against these — float wavelet
|
||||
/// math is not bit-exact across implementations, upstream itself ships precision variants.
|
||||
/// `#[ignore]`d GPU test; regenerate on a Vulkan 1.3 host:
|
||||
/// cargo test -p punktfunk-host --features pyrowave --no-run
|
||||
/// PYROWAVE_GOLDEN_DIR=/tmp/golden <bin> --ignored --nocapture pyrowave_dump_golden
|
||||
/// then copy the files into clients/apple/Tests/PunktfunkKitTests/PyroWaveFixtures/.
|
||||
#[test]
|
||||
#[ignore = "fixture generator — needs a real Vulkan 1.3 compute device"]
|
||||
fn pyrowave_dump_golden() {
|
||||
let dir = match std::env::var("PYROWAVE_GOLDEN_DIR") {
|
||||
Ok(d) => std::path::PathBuf::from(d),
|
||||
Err(_) => {
|
||||
eprintln!("PYROWAVE_GOLDEN_DIR not set — skipping dump");
|
||||
return;
|
||||
}
|
||||
};
|
||||
std::fs::create_dir_all(&dir).expect("create golden dir");
|
||||
|
||||
// Odd-block geometry on purpose: 256 aligns clean, 144 → aligned 160 exercises the
|
||||
// block-grid overhang. ~1.6 bpp at 60 fps.
|
||||
let (w, h) = (256u32, 144u32);
|
||||
let mut enc = PyroWaveEncoder::open(w, h, 60, 4_000_000).expect("open");
|
||||
|
||||
let dump = |name: &str, bytes: &[u8]| {
|
||||
std::fs::write(dir.join(name), bytes).expect("write fixture");
|
||||
eprintln!("wrote {name}: {} bytes", bytes.len());
|
||||
};
|
||||
|
||||
// Dense AU + upstream-decoded reference planes.
|
||||
enc.submit(&test_card(w, h, 7)).expect("submit");
|
||||
let au = enc.poll().expect("poll").expect("AU");
|
||||
assert!(!au.chunk_aligned);
|
||||
dump("au-dense.bin", &au.data);
|
||||
// SAFETY: test-only FFI with locally-owned buffers.
|
||||
let (y, cb, cr) = unsafe { decode_planes(w, h, &au.data) };
|
||||
dump("ref-dense-y.bin", &y);
|
||||
dump("ref-dense-cb.bin", &cb);
|
||||
dump("ref-dense-cr.bin", &cr);
|
||||
|
||||
// Chunk-aligned AU of a DIFFERENT frame (its own reference): the Swift window walk +
|
||||
// FRAG reassembly must reproduce the packet stream.
|
||||
enc.set_wire_chunking(1408);
|
||||
enc.submit(&test_card(w, h, 11)).expect("chunked submit");
|
||||
let au = enc.poll().expect("poll").expect("chunked AU");
|
||||
assert!(au.chunk_aligned);
|
||||
assert_eq!(au.data.len() % 1408, 0);
|
||||
dump("au-chunked.bin", &au.data);
|
||||
// SAFETY: test-only FFI with locally-owned buffers.
|
||||
let (y, cb, cr) = unsafe {
|
||||
// Feed upstream through the same framed walk the clients use.
|
||||
let mut stream = Vec::new();
|
||||
let mut frag: Vec<u8> = Vec::new();
|
||||
for win in au.data.chunks(1408) {
|
||||
let used = u16::from_le_bytes([win[0], win[1]]) as usize;
|
||||
let kind = u16::from_le_bytes([win[2], win[3]]);
|
||||
let body = &win[4..4 + used];
|
||||
match kind {
|
||||
0 => stream.extend_from_slice(body),
|
||||
1 => frag = body.to_vec(),
|
||||
2 => frag.extend_from_slice(body),
|
||||
3 => {
|
||||
frag.extend_from_slice(body);
|
||||
stream.extend_from_slice(&frag);
|
||||
frag.clear();
|
||||
}
|
||||
k => panic!("unknown window kind {k}"),
|
||||
}
|
||||
}
|
||||
decode_planes(w, h, &stream)
|
||||
};
|
||||
dump("ref-chunked-y.bin", &y);
|
||||
dump("ref-chunked-cb.bin", &cb);
|
||||
dump("ref-chunked-cr.bin", &cr);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user