feat(apple): PyroWave Phase 5 — native Metal decode on Mac / Apple TV / iPad (§4.7)
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 58s
decky / build-publish (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
ci / bench (push) Successful in 5m21s
arch / build-publish (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 28s
android / android (push) Successful in 13m23s
deb / build-publish (push) Successful in 12m58s
ci / rust (push) Successful in 18m5s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m31s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m56s
flatpak / build-publish (push) Successful in 5m53s
windows-host / package (push) Successful in 14m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m12s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m6s
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
release / apple (push) Has been cancelled

The Apple client now decodes PyroWave natively on the presenter's own MTLDevice —
no MoltenVK, no upstream C++ in the app. Completes and wires up the decoder whose
early working-tree snapshot rode along in 9127c346:

- MetalWaveletShaders.swift: wavelet_dequant + idwt hand-ported from the vendored
  GLSL (STORAGE_MODE 0 only; subgroup scans → 32-wide simdgroups; DCShift spec
  constant → function constant; precision-1 split: fp16 levels 0-1 / fp32 2-4).
- MetalWaveletDecoder.swift: Swift reimplementation of push_packet/decode_packet
  incl. the Phase-4 chunk-aligned window walk (FRAG chains, zeroed missing shards,
  the >half-blocks partial rule), init_block_meta's block-index space, and the
  42-dequant + 13-idwt dispatch structure with encoder-boundary barriers. SOF-dims
  changes rebuild the size-dependent resources, which is also the mid-stream
  resize path. Ring of 4 output plane sets on the presenter's queue.
- Presenter: pf_frag_planar (3xR8, the planar_csc.frag twin) + renderPlanar with
  a shared present tail; ReadyFrame carries an image enum (.video | .planar).
- Stage2Pipeline: a dedicated PyroWave pump — no VideoToolbox machinery, no
  keyframe/re-anchor recovery (all-intra; partials render as localized blur by
  design), newest-frame-index staleness guard for late partials.
- Opt-in: "PyroWave (wired LAN)" codec entry (probe-gated, ≈A13 floor via a real
  kernel-compile probe), selecting it advertises + prefers the codec and forces
  the session SDR (HDR/10-bit/4:4:4 caps dropped, plan contract).
- Core ABI: punktfunk_connection_shard_payload() — the Welcome's negotiated shard
  payload, needed by native decoders to walk chunk-aligned AUs.
- Validation: golden fixtures generated by the host encoder + upstream's own
  decoder (pyrowave_dump_golden, RTX 5070 Ti); the Metal decode PSNR-matches at
  77-88 dB across all planes for dense AND chunk-aligned AUs, and a hole-punched
  partial still decodes. Parser unit tests cover the window walk, FRAG chains,
  broken chains, the half-blocks gate, and the block-index layout.

Tests: apple 134 green (mac; iOS/tvOS build), host 312 w/ pyrowave on .21,
core 148 w/ quic; clippy/fmt clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-15 12:16:43 +02:00
parent a70811043e
commit 739a5f76bf
28 changed files with 826 additions and 81 deletions
@@ -12,7 +12,23 @@ import CoreVideo
import Foundation
import VideoToolbox
/// One decoded frame waiting to be presented. Owns a retained `CVPixelBuffer` until shown.
/// A decoded frame's pixels which present path they take. VideoToolbox codecs deliver a
/// biplanar `CVPixelBuffer` (NV12/P010/444v/x444); the PyroWave Metal decoder delivers three
/// separate R8 plane textures straight off its compute pass (there is no CVPixelBuffer the
/// planes never leave the GPU).
public enum ReadyImage: @unchecked Sendable {
/// 8-bit NV12 / 4:4:4 biplanar (SDR) or 10-bit P010 / x444 (HDR), Metal-compatible.
/// `isHDR` = the stream is BT.2020 PQ and the presenter must configure EDR output.
case video(CVPixelBuffer, isHDR: Bool)
#if canImport(Metal)
/// PyroWave planar output (Y full-res + Cb/Cr half-res, 8-bit SDR) with its precomputed
/// CSC rows presented by `MetalVideoPresenter.renderPlanar`.
case planar(WaveletPlanes)
#endif
}
/// One decoded frame waiting to be presented. Owns its image (a retained `CVPixelBuffer`, or
/// the PyroWave ring textures) until shown.
public struct ReadyFrame: @unchecked Sendable {
/// Host capture clock (the AU's pts), in nanoseconds.
public let ptsNs: UInt64
@@ -22,15 +38,26 @@ public struct ReadyFrame: @unchecked Sendable {
public let receivedNs: Int64
/// Client `CLOCK_REALTIME` instant decode completed, in nanoseconds.
public let decodedNs: Int64
/// The decoded image 8-bit NV12 biplanar (SDR) or 10-bit P010 biplanar (HDR), Metal-compatible.
public let pixelBuffer: CVPixelBuffer
/// True when the stream is HDR (BT.2020 PQ): the buffer is 10-bit P010 and the presenter must
/// configure EDR + BT.2020 PQ output. Derived from the decoded buffer's pixel format.
public let isHDR: Bool
/// The decoded image and which present path it takes.
public let image: ReadyImage
/// The AU's wire `user_flags` (`AccessUnit.flags`), threaded through the decode via the frame
/// context so the re-anchor gate can classify this decoded frame (IDR / RFI anchor / recovery
/// mark) at present time the async decode callback has no other access to it. 0 when unknown.
public let flags: UInt32
/// The VideoToolbox path's buffer; nil for a PyroWave planar frame. (Kept as the accessor
/// the decode round-trip tests assert against.)
public var pixelBuffer: CVPixelBuffer? {
if case .video(let buffer, _) = image { return buffer }
return nil
}
/// Whether this frame presents on the HDR path. PyroWave planar frames are 8-bit SDR by
/// contract.
public var isHDR: Bool {
if case .video(_, let hdr) = image { return hdr }
return false
}
}
/// Per-frame context threaded through the VideoToolbox frame refcon: the AU's receipt instant (for
@@ -286,6 +313,6 @@ public final class VideoDecoder: @unchecked Sendable {
onDecoded(
ReadyFrame(
ptsNs: ptsNs, receivedNs: receivedNs, decodedNs: decodedNs,
pixelBuffer: imageBuffer, isHDR: isHDR, flags: flags))
image: .video(imageBuffer, isHDR: isHDR), flags: flags))
}
}