feat(apple): PyroWave Phase 5 — native Metal decode on Mac / Apple TV / iPad (§4.7)
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 58s
decky / build-publish (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
ci / bench (push) Successful in 5m21s
arch / build-publish (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 28s
android / android (push) Successful in 13m23s
deb / build-publish (push) Successful in 12m58s
ci / rust (push) Successful in 18m5s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m31s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m56s
flatpak / build-publish (push) Successful in 5m53s
windows-host / package (push) Successful in 14m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m12s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m6s
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
release / apple (push) Has been cancelled
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 58s
decky / build-publish (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
ci / bench (push) Successful in 5m21s
arch / build-publish (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 28s
android / android (push) Successful in 13m23s
deb / build-publish (push) Successful in 12m58s
ci / rust (push) Successful in 18m5s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m31s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m56s
flatpak / build-publish (push) Successful in 5m53s
windows-host / package (push) Successful in 14m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m12s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m6s
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
release / apple (push) Has been cancelled
The Apple client now decodes PyroWave natively on the presenter's own MTLDevice —
no MoltenVK, no upstream C++ in the app. Completes and wires up the decoder whose
early working-tree snapshot rode along in 9127c346:
- MetalWaveletShaders.swift: wavelet_dequant + idwt hand-ported from the vendored
GLSL (STORAGE_MODE 0 only; subgroup scans → 32-wide simdgroups; DCShift spec
constant → function constant; precision-1 split: fp16 levels 0-1 / fp32 2-4).
- MetalWaveletDecoder.swift: Swift reimplementation of push_packet/decode_packet
incl. the Phase-4 chunk-aligned window walk (FRAG chains, zeroed missing shards,
the >half-blocks partial rule), init_block_meta's block-index space, and the
42-dequant + 13-idwt dispatch structure with encoder-boundary barriers. SOF-dims
changes rebuild the size-dependent resources, which is also the mid-stream
resize path. Ring of 4 output plane sets on the presenter's queue.
- Presenter: pf_frag_planar (3xR8, the planar_csc.frag twin) + renderPlanar with
a shared present tail; ReadyFrame carries an image enum (.video | .planar).
- Stage2Pipeline: a dedicated PyroWave pump — no VideoToolbox machinery, no
keyframe/re-anchor recovery (all-intra; partials render as localized blur by
design), newest-frame-index staleness guard for late partials.
- Opt-in: "PyroWave (wired LAN)" codec entry (probe-gated, ≈A13 floor via a real
kernel-compile probe), selecting it advertises + prefers the codec and forces
the session SDR (HDR/10-bit/4:4:4 caps dropped, plan contract).
- Core ABI: punktfunk_connection_shard_payload() — the Welcome's negotiated shard
payload, needed by native decoders to walk chunk-aligned AUs.
- Validation: golden fixtures generated by the host encoder + upstream's own
decoder (pyrowave_dump_golden, RTX 5070 Ti); the Metal decode PSNR-matches at
77-88 dB across all planes for dense AND chunk-aligned AUs, and a hole-punched
partial still decodes. Parser unit tests cover the window walk, FRAG chains,
broken chains, the half-blocks gate, and the block-index layout.
Tests: apple 134 green (mac; iOS/tvOS build), host 312 w/ pyrowave on .21,
core 148 w/ quic; clippy/fmt clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -12,7 +12,23 @@ import CoreVideo
|
||||
import Foundation
|
||||
import VideoToolbox
|
||||
|
||||
/// One decoded frame waiting to be presented. Owns a retained `CVPixelBuffer` until shown.
|
||||
/// A decoded frame's pixels — which present path they take. VideoToolbox codecs deliver a
|
||||
/// biplanar `CVPixelBuffer` (NV12/P010/444v/x444); the PyroWave Metal decoder delivers three
|
||||
/// separate R8 plane textures straight off its compute pass (there is no CVPixelBuffer — the
|
||||
/// planes never leave the GPU).
|
||||
public enum ReadyImage: @unchecked Sendable {
|
||||
/// 8-bit NV12 / 4:4:4 biplanar (SDR) or 10-bit P010 / x444 (HDR), Metal-compatible.
|
||||
/// `isHDR` = the stream is BT.2020 PQ and the presenter must configure EDR output.
|
||||
case video(CVPixelBuffer, isHDR: Bool)
|
||||
#if canImport(Metal)
|
||||
/// PyroWave planar output (Y full-res + Cb/Cr half-res, 8-bit SDR) with its precomputed
|
||||
/// CSC rows — presented by `MetalVideoPresenter.renderPlanar`.
|
||||
case planar(WaveletPlanes)
|
||||
#endif
|
||||
}
|
||||
|
||||
/// One decoded frame waiting to be presented. Owns its image (a retained `CVPixelBuffer`, or
|
||||
/// the PyroWave ring textures) until shown.
|
||||
public struct ReadyFrame: @unchecked Sendable {
|
||||
/// Host capture clock (the AU's pts), in nanoseconds.
|
||||
public let ptsNs: UInt64
|
||||
@@ -22,15 +38,26 @@ public struct ReadyFrame: @unchecked Sendable {
|
||||
public let receivedNs: Int64
|
||||
/// Client `CLOCK_REALTIME` instant decode completed, in nanoseconds.
|
||||
public let decodedNs: Int64
|
||||
/// The decoded image — 8-bit NV12 biplanar (SDR) or 10-bit P010 biplanar (HDR), Metal-compatible.
|
||||
public let pixelBuffer: CVPixelBuffer
|
||||
/// True when the stream is HDR (BT.2020 PQ): the buffer is 10-bit P010 and the presenter must
|
||||
/// configure EDR + BT.2020 PQ output. Derived from the decoded buffer's pixel format.
|
||||
public let isHDR: Bool
|
||||
/// The decoded image and which present path it takes.
|
||||
public let image: ReadyImage
|
||||
/// The AU's wire `user_flags` (`AccessUnit.flags`), threaded through the decode via the frame
|
||||
/// context so the re-anchor gate can classify this decoded frame (IDR / RFI anchor / recovery
|
||||
/// mark) at present time — the async decode callback has no other access to it. 0 when unknown.
|
||||
public let flags: UInt32
|
||||
|
||||
/// The VideoToolbox path's buffer; nil for a PyroWave planar frame. (Kept as the accessor
|
||||
/// the decode round-trip tests assert against.)
|
||||
public var pixelBuffer: CVPixelBuffer? {
|
||||
if case .video(let buffer, _) = image { return buffer }
|
||||
return nil
|
||||
}
|
||||
|
||||
/// Whether this frame presents on the HDR path. PyroWave planar frames are 8-bit SDR by
|
||||
/// contract.
|
||||
public var isHDR: Bool {
|
||||
if case .video(_, let hdr) = image { return hdr }
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-frame context threaded through the VideoToolbox frame refcon: the AU's receipt instant (for
|
||||
@@ -286,6 +313,6 @@ public final class VideoDecoder: @unchecked Sendable {
|
||||
onDecoded(
|
||||
ReadyFrame(
|
||||
ptsNs: ptsNs, receivedNs: receivedNs, decodedNs: decodedNs,
|
||||
pixelBuffer: imageBuffer, isHDR: isHDR, flags: flags))
|
||||
image: .video(imageBuffer, isHDR: isHDR), flags: flags))
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user