feat(pyrowave): Apple Metal 4:4:4 + HDR decode, EDR present — self-configured in-band
docker / deploy-docs (push) Successful in 11s
flatpak / build-publish (push) Successful in 6m25s
deb / build-publish (push) Successful in 9m7s
windows-host / package (push) Successful in 16m12s
deb / build-publish-host (push) Successful in 9m31s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m41s
ci / rust (push) Successful in 25m11s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m45s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m2s
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 9m36s
apple / screenshots (push) Successful in 6m43s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 50s
android / android (push) Successful in 13m41s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 14m23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
ci / bench (push) Successful in 5m58s

Phases 4+5 of design/pyrowave-444-hdr.md. The Metal decoder needs NO new ABI:
every frame's sequence header carries chroma (444) and, since the Phase-3
stamps, the PQ/BT.2020 bits — so the decoder self-configures per session.

Decoder: WaveletLayout grows the 4:4:4 block space (chroma runs the full
pyramid like luma — no level-0 skip, no early half-res emit; the Metal
kernels were already chroma-agnostic, only the dispatch structure changes);
the parser accepts chroma_resolution=444, reads the PQ transfer bit, and
lifts the even-dims rule for 444; the plane ring allocates full-res chroma
and r16Unorm for PQ streams; CSC rows switch to depth-10 MSB-packed.

Presenter: planar HDR passthrough reuses pf_frag_planar on an rgba16Float
drawable (itur_2100_PQ + EDR metadata interpret the samples — same split as
pf_frag/pf_frag_hdr), plus a new pf_frag_planar_tm PQ->SDR tone-map (shared
pqToSdr tail refactored out of pf_frag_hdr_tv) for tvOS-without-headroom AND
macOS WINDOWED sessions, whose IOSurface present path (the DCP-panic
mitigation) is BGRA8-only. SessionModel stops stripping the HDR/10-bit/444
caps on the PyroWave opt-in.

New golden: au-dense444 + upstream's own 4:4:4 reference planes (regenerated
via the extended pyrowave_dump_golden); Metal decode matches at 64-67 dB
(420 fixtures re-verify 77-88 dB). Full Apple suite 157 tests green on a
real M-series GPU. Docs updated: the 8-bit-SDR-only wording is gone, the
Windows host is no longer 'on the roadmap', bpp scaling documented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-18 13:21:23 +02:00
parent 188edde2b3
commit 3ff1973d7f
9 changed files with 167 additions and 57 deletions
@@ -264,14 +264,11 @@ final class SessionModel: ObservableObject {
// PyroWave (wired LAN) is a pure opt-in: picking it in the codec setting both
// advertises the bit and prefers it the host never auto-selects it, and the
// picker only offers it when the Metal decode probe passed (simdgroup floor A13;
// every M-series Mac and the ATV 4K gen 3 pass). The codec is 8-bit 4:2:0 SDR
// BT.709 by contract, so the opt-in also drops the HDR/10-bit/4:4:4 caps for this
// session HDR sessions stay HEVC/AV1 (plan §4.7).
// every M-series Mac and the ATV 4K gen 3 pass). The decoder self-configures from
// the per-frame sequence header (4:2:0/4:4:4, SDR/PQ design/pyrowave-444-hdr.md),
// so the session keeps the user's HDR/10-bit/4:4:4 caps exactly like HEVC/AV1.
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
videoCodecs |= PunktfunkConnection.codecPyroWave
videoCaps &= ~(PunktfunkConnection.videoCap10Bit
| PunktfunkConnection.videoCapHDR
| PunktfunkConnection.videoCap444)
}
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
@@ -196,13 +196,11 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// display-referred SDR. (When the display IS in an HDR mode — requested per session via
// AVDisplayManager, see StreamViewIOS — tvOS presents pf_frag_hdr's PQ passthrough instead:
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]],
constant CscUniform& csc [[buffer(0)]]) {
// YCbCr → full-range PQ RGB via the per-frame rows (as pf_frag_hdr).
float3 pq = sampleRgb(lumaTex, chromaTex, in.uv, csc);
// ST 2084 EOTF: PQ code value → linear light, 1.0 = 10,000 nits.
// The shared PQ→display-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF → 203-nit-anchored scene light → BT.2020→709 primaries → extended-Reinhard rolloff →
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map — the
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only.
static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0;
const float m2 = 78.84375;
const float c1 = 3424.0/4096.0;
@@ -210,20 +208,49 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
const float c3 = 18.6875;
float3 p = pow(pq, 1.0/m2);
float3 lin = pow(max(p - c1, 0.0) / (c2 - c3 * p), 1.0/m1);
// Scene-referred with diffuse white at 1.0 (the same 203-nit anchor the EDR path uses).
float3 t = lin * (10000.0/203.0);
// BT.2020 → BT.709 primaries while still linear; negatives are out-of-gamut, floor them.
float3 t709 = float3(
dot(t, float3( 1.6605, -0.5876, -0.0728)),
dot(t, float3(-0.1246, 1.1329, -0.0083)),
dot(t, float3(-0.0182, -0.1006, 1.1187)));
t709 = max(t709, 0.0);
// Extended Reinhard: 1.0 stays put, the 1000-nit knee lands at display white, above rolls off.
const float w = 1000.0/203.0;
float3 mapped = saturate(t709 * (1.0 + t709 / (w * w)) / (1.0 + t709));
// BT.709 OETF — the same encoding the SDR stream arrives in, so both paths present alike.
float3 e = select(1.099 * pow(mapped, 0.45) - 0.099, 4.5 * mapped, mapped < 0.018);
return float4(e, 1.0);
return e;
}
fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]],
constant CscUniform& csc [[buffer(0)]]) {
// YCbCr → full-range PQ RGB via the per-frame rows (as pf_frag_hdr), then the tail.
return float4(pqToSdr(sampleRgb(lumaTex, chromaTex, in.uv, csc)), 1.0);
}
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) → PQ RGB → the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions
// (the IOSurface present path — the DCP-panic mitigation — is BGRA8). The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math — the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> cbTex [[texture(1)]],
texture2d<float> crTex [[texture(2)]],
constant CscUniform& csc [[buffer(0)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
#ifdef PF_BILINEAR_LUMA
float lumaY = lumaTex.sample(s, in.uv).r;
#else
float lumaY = catmullRomLuma(lumaTex, s, in.uv);
#endif
float2 cuv = chromaUV(lumaTex, cbTex, in.uv);
float3 yuv = float3(lumaY, cbTex.sample(s, cuv).r, crTex.sample(s, cuv).r);
float3 pq = saturate(float3(dot(csc.r0.xyz, yuv) + csc.r0.w,
dot(csc.r1.xyz, yuv) + csc.r1.w,
dot(csc.r2.xyz, yuv) + csc.r2.w));
return float4(pqToSdr(pq), 1.0);
}
"""
@@ -287,6 +314,11 @@ public final class MetalVideoPresenter {
private let pipelineHDRToneMap: MTLRenderPipelineState?
/// PyroWave's 3-plane SDR path (pf_frag_planar bgra8) see `renderPlanar`.
private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents).
private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache?
/// The PyroWave Metal decoder records on the presenter's device + queue: one device means
@@ -337,6 +369,8 @@ public final class MetalVideoPresenter {
let pipelineHDR: MTLRenderPipelineState
let pipelineHDRToneMap: MTLRenderPipelineState?
let pipelinePlanar: MTLRenderPipelineState
let pipelinePlanarHDR: MTLRenderPipelineState
let pipelinePlanarToneMap: MTLRenderPipelineState
do {
// DEBUG A/B lever: PUNKTFUNK_BILINEAR_LUMA=1 compiles the shader with Catmull-Rom OFF
// (plain bilinear luma) by prepending a #define ahead of the source. Default (unset) is
@@ -374,8 +408,18 @@ public final class MetalVideoPresenter {
let planar = MTLRenderPipelineDescriptor()
planar.vertexFunction = vtx
planar.fragmentFunction = library.makeFunction(name: "pf_frag_planar")
planar.colorAttachments[0].pixelFormat = .bgra8Unorm // PyroWave is 8-bit SDR
planar.colorAttachments[0].pixelFormat = .bgra8Unorm
pipelinePlanar = try device.makeRenderPipelineState(descriptor: planar)
let planarHdr = MTLRenderPipelineDescriptor()
planarHdr.vertexFunction = vtx
planarHdr.fragmentFunction = library.makeFunction(name: "pf_frag_planar")
planarHdr.colorAttachments[0].pixelFormat = .rgba16Float // PQ passthrough
pipelinePlanarHDR = try device.makeRenderPipelineState(descriptor: planarHdr)
let planarTm = MTLRenderPipelineDescriptor()
planarTm.vertexFunction = vtx
planarTm.fragmentFunction = library.makeFunction(name: "pf_frag_planar_tm")
planarTm.colorAttachments[0].pixelFormat = .bgra8Unorm
pipelinePlanarToneMap = try device.makeRenderPipelineState(descriptor: planarTm)
} catch {
return nil
}
@@ -416,6 +460,7 @@ public final class MetalVideoPresenter {
return MetalVideoPresenter(
device: device, queue: queue, pipelineSDR: pipelineSDR, pipelineHDR: pipelineHDR,
pipelineHDRToneMap: pipelineHDRToneMap, pipelinePlanar: pipelinePlanar,
pipelinePlanarHDR: pipelinePlanarHDR, pipelinePlanarToneMap: pipelinePlanarToneMap,
textureCache: textureCache, layer: layer)
}
@@ -423,6 +468,8 @@ public final class MetalVideoPresenter {
device: MTLDevice, queue: MTLCommandQueue, pipelineSDR: MTLRenderPipelineState,
pipelineHDR: MTLRenderPipelineState, pipelineHDRToneMap: MTLRenderPipelineState?,
pipelinePlanar: MTLRenderPipelineState,
pipelinePlanarHDR: MTLRenderPipelineState,
pipelinePlanarToneMap: MTLRenderPipelineState,
textureCache: CVMetalTextureCache, layer: CAMetalLayer
) {
self.device = device
@@ -431,6 +478,8 @@ public final class MetalVideoPresenter {
self.pipelineHDR = pipelineHDR
self.pipelineHDRToneMap = pipelineHDRToneMap
self.pipelinePlanar = pipelinePlanar
self.pipelinePlanarHDR = pipelinePlanarHDR
self.pipelinePlanarToneMap = pipelinePlanarToneMap
self.textureCache = textureCache
self.layer = layer
}
@@ -652,7 +701,13 @@ public final class MetalVideoPresenter {
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock()
configure(hdr: false)
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path;
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader).
#if os(macOS)
configure(hdr: planes.pq && !surfaceMode)
#else
configure(hdr: planes.pq)
#endif
var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
@@ -672,9 +727,21 @@ public final class MetalVideoPresenter {
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format).
#if os(tvOS)
let planarPassthrough = hdrActive && hdrPassthroughActive
#else
let planarPassthrough = hdrActive
#endif
let planarPipeline: MTLRenderPipelineState =
planes.pq
? (planarPassthrough ? pipelinePlanarHDR : pipelinePlanarToneMap)
: pipelinePlanar
return encodePresent(
decodedSize: CGSize(width: planes.width, height: planes.height),
targetFromLayout: targetFromLayout, pipeline: pipelinePlanar,
targetFromLayout: targetFromLayout, pipeline: planarPipeline,
presentAtMediaTime: presentAtMediaTime, onPresented: onPresented,
// The ring textures stay valid by ring depth; retaining them here also pins the
// slot's set until the sample completes (mirrors the biplanar keep-alive).
@@ -716,7 +783,7 @@ public final class MetalVideoPresenter {
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipelinePlanar)
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
@@ -47,6 +47,9 @@ struct WaveletLayout {
let width: Int
let height: Int
/// Full-res chroma (4:4:4): chroma components get the full band set including level 0,
/// exactly like luma upstream `init_block_meta` with `Chroma444`.
let chroma444: Bool
let alignedWidth: Int
let alignedHeight: Int
/// blockMeta[component][level][band] = (blockOffset32x32, blockStride32x32); -1 offset =
@@ -59,9 +62,10 @@ struct WaveletLayout {
func levelWidth(_ level: Int) -> Int { (alignedWidth / 2) >> level }
func levelHeight(_ level: Int) -> Int { (alignedHeight / 2) >> level }
init(width: Int, height: Int) {
init(width: Int, height: Int, chroma444: Bool) {
self.width = width
self.height = height
self.chroma444 = chroma444
let align = { (v: Int) in
max((v + Self.alignment - 1) & ~(Self.alignment - 1), Self.minimumImageSize)
}
@@ -78,7 +82,7 @@ struct WaveletLayout {
let ah = alignedHeight
for level in stride(from: Self.decompositionLevels - 1, through: 0, by: -1) {
for component in 0..<3 {
if level == 0 && component != 0 { continue } // 4:2:0: no top-level chroma
if level == 0 && component != 0 && !chroma444 { continue } // 4:2:0: no top-level chroma
for band in (level == Self.decompositionLevels - 1 ? 0 : 1)..<4 {
let levelW = (aw / 2) >> level
let levelH = (ah / 2) >> level
@@ -108,6 +112,9 @@ struct ParsedWaveletFrame {
var decodedBlocks: Int
/// VUI bits from the sequence header (BitstreamSequenceHeader).
var bt2020: Bool
/// PQ transfer HDR session: 16-bit studio-code planes + EDR present (the host stamps
/// this bit iff the session negotiated 10-bit the depth is coupled to the transfer).
var pq: Bool
var fullRange: Bool
/// The frame's YCbCrRGB signal for the presenter's planar CSC. PyroWave today is always
@@ -203,6 +210,7 @@ enum WaveletBitstream {
var totalBlocks = 0
var decodedBlocks = 0
var bt2020 = false
var pq = false
var fullRange = false
var sawSOF = false
@@ -220,22 +228,28 @@ enum WaveletBitstream {
// siting[31].
let code = (word1 >> 24) & 0x3
guard code == 0 else { return false } // only START_OF_FRAME is defined
let chromaRes = (word1 >> 26) & 1
guard chromaRes == 0 else { return false } // host contract: 4:2:0
let chroma444 = (word1 >> 26) & 1 != 0
let w = Int(word0 & 0x3fff) + 1
let h = Int((word0 >> 14) & 0x3fff) + 1
guard w >= 2, h >= 2, w % 2 == 0, h % 2 == 0 else { return false }
guard w >= 2, h >= 2, chroma444 || (w % 2 == 0 && h % 2 == 0) else {
return false
}
if sawSOF {
// One frame, one geometry a second SOF must agree.
guard layout?.width == w, layout?.height == h else { return false }
guard layout?.width == w, layout?.height == h,
layout?.chroma444 == chroma444
else { return false }
} else {
sawSOF = true
let l = WaveletLayout(width: w, height: h)
let l = WaveletLayout(width: w, height: h, chroma444: chroma444)
layout = l
offsets = [UInt32](repeating: .max, count: l.blockCount32)
payload.reserveCapacity(64 * 1024 / 4)
totalBlocks = Int(word1 & 0xff_ffff)
bt2020 = (word1 >> 29) & 1 != 0
// transfer_function bit: PQ an HDR session (16-bit studio-code
// planes by the negotiated coupling design/pyrowave-444-hdr.md).
pq = (word1 >> 28) & 1 != 0
fullRange = (word1 >> 30) & 1 == 0 // YCBCR_RANGE_FULL = 0
}
pos += 8
@@ -280,7 +294,7 @@ enum WaveletBitstream {
return ParsedWaveletFrame(
layout: layout, offsets: offsets, payload: payload,
totalBlocks: totalBlocks, decodedBlocks: decodedBlocks,
bt2020: bt2020, fullRange: fullRange)
bt2020: bt2020, pq: pq, fullRange: fullRange)
}
}
}
@@ -293,6 +307,8 @@ public struct WaveletPlanes: @unchecked Sendable {
public let cb: MTLTexture
public let cr: MTLTexture
public let csc: CscUniform
/// PQ (HDR) stream: the presenter picks the HDR/tone-map planar pipeline + EDR config.
public let pq: Bool
public var width: Int { y.width }
public var height: Int { y.height }
}
@@ -351,6 +367,8 @@ public final class MetalWaveletDecoder {
private var slots: [Slot] = []
private var nextSlot = 0
/// The ring's plane format facts (from the last SOF): PQ 16-bit UNORM planes.
private var hdr16 = false
/// The current geometry (from the last SOF that built the resources) the pump reports
/// decoded-size changes to the resize overlay from this. PUMP THREAD.
@@ -409,8 +427,9 @@ public final class MetalWaveletDecoder {
au: au, chunkAligned: chunkAligned, windowSize: windowSize)
else { return false }
if layout?.width != frame.layout.width || layout?.height != frame.layout.height {
guard rebuild(layout: frame.layout) else { return false }
if layout?.width != frame.layout.width || layout?.height != frame.layout.height
|| layout?.chroma444 != frame.layout.chroma444 || hdr16 != frame.pq {
guard rebuild(layout: frame.layout, hdr16: frame.pq) else { return false }
}
guard let layout, !slots.isEmpty else { return false }
@@ -450,7 +469,7 @@ public final class MetalWaveletDecoder {
dequant.setBuffer(slot.payload, offset: 0, index: 1)
for level in 0..<WaveletLayout.decompositionLevels {
for component in 0..<3 {
if level == 0 && component != 0 { continue } // 4:2:0
if level == 0 && component != 0 && !layout.chroma444 { continue } // 4:2:0
for band in (level == WaveletLayout.decompositionLevels - 1 ? 0 : 1)..<4 {
let meta = layout.blockMeta[component][level][band]
let w = layout.levelWidth(level)
@@ -489,15 +508,20 @@ public final class MetalWaveletDecoder {
let grid = MTLSize(width: (rx + 15) / 16, height: (ry + 15) / 16, depth: 1)
let group = MTLSize(width: 64, height: 1, depth: 1)
if inputLevel == 0 {
// 4:2:0: the final full-res pass is luma only (chroma finished at level 1).
// Final full-res pass: luma only in 4:2:0 (chroma finished at level 1); all
// three components in 4:4:4 (chroma runs the full pyramid like luma).
idwt.setComputePipelineState(idwtShiftPipeline)
idwt.setTexture(coefficients[0][0], index: 0)
idwt.setTexture(slot.y, index: 1)
let components = layout.chroma444 ? 3 : 1
for component in 0..<components {
idwt.setTexture(coefficients[component][0], index: 0)
let out = component == 0 ? slot.y : (component == 1 ? slot.cb : slot.cr)
idwt.setTexture(out, index: 1)
idwt.dispatchThreadgroups(grid, threadsPerThreadgroup: group)
}
} else {
for component in 0..<3 {
idwt.setTexture(coefficients[component][inputLevel], index: 0)
if component != 0 && inputLevel == 1 {
if component != 0 && inputLevel == 1 && !layout.chroma444 {
// 4:2:0 chroma emits its final half-res plane one level early.
idwt.setComputePipelineState(idwtShiftPipeline)
idwt.setTexture(component == 1 ? slot.cb : slot.cr, index: 1)
@@ -513,7 +537,9 @@ public final class MetalWaveletDecoder {
let planes = WaveletPlanes(
y: slot.y, cb: slot.cb, cr: slot.cr,
csc: CscRows.rows(frame.cscSignal, depth: 8, msbPacked: false))
csc: CscRows.rows(
frame.cscSignal, depth: frame.pq ? 10 : 8, msbPacked: frame.pq),
pq: frame.pq)
cmd.addCompletedHandler { buffer in
completion(buffer.error == nil ? planes : nil)
}
@@ -524,9 +550,9 @@ public final class MetalWaveletDecoder {
/// (Re)allocate every size-dependent resource for `layout`'s geometry. Also the mid-stream
/// resize path: a Reconfigure shows up here as new SOF dims.
private func rebuild(layout newLayout: WaveletLayout) -> Bool {
private func rebuild(layout newLayout: WaveletLayout, hdr16 newHdr16: Bool) -> Bool {
waveletLog.info(
"pyrowave: building decoder \(newLayout.width)x\(newLayout.height) (aligned \(newLayout.alignedWidth)x\(newLayout.alignedHeight), \(newLayout.blockCount32) blocks)")
"pyrowave: building decoder \(newLayout.width)x\(newLayout.height) (aligned \(newLayout.alignedWidth)x\(newLayout.alignedHeight), \(newLayout.blockCount32) blocks, \(newLayout.chroma444 ? "4:4:4" : "4:2:0", privacy: .public)\(newHdr16 ? " HDR16" : "", privacy: .public))")
var coeff: [[MTLTexture]] = []
var lls: [[MTLTexture]] = []
for component in 0..<3 {
@@ -560,19 +586,22 @@ public final class MetalWaveletDecoder {
var newSlots: [Slot] = []
for i in 0..<Self.ringDepth {
let planeFormat: MTLPixelFormat = newHdr16 ? .r16Unorm : .r8Unorm
let plane = { (w: Int, h: Int, name: String) -> MTLTexture? in
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .r8Unorm, width: w, height: h, mipmapped: false)
pixelFormat: planeFormat, width: w, height: h, mipmapped: false)
desc.usage = [.shaderRead, .shaderWrite]
desc.storageMode = .private
let t = self.device.makeTexture(descriptor: desc)
t?.label = name
return t
}
let cw = newLayout.chroma444 ? newLayout.width : newLayout.width / 2
let ch = newLayout.chroma444 ? newLayout.height : newLayout.height / 2
guard
let y = plane(newLayout.width, newLayout.height, "pyrowave Y[\(i)]"),
let cb = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cb[\(i)]"),
let cr = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cr[\(i)]"),
let cb = plane(cw, ch, "pyrowave Cb[\(i)]"),
let cr = plane(cw, ch, "pyrowave Cr[\(i)]"),
let offsets = device.makeBuffer(
length: max(newLayout.blockCount32 * 4, 4), options: .storageModeShared),
let payload = device.makeBuffer(length: 64 * 1024, options: .storageModeShared)
@@ -585,6 +614,7 @@ public final class MetalWaveletDecoder {
slots = newSlots
nextSlot = 0
layout = newLayout
hdr16 = newHdr16
return true
}
@@ -57,7 +57,7 @@ final class PyroWaveParserTests: XCTestCase {
func testLayoutMatchesUpstreamBlockSpace() {
// init_block_meta's walk for 256x144 (aligned 256x160): level extents halve from
// 128x80; per (comp,level,band) count32 = ceil(ceil(w/8)/4) * ceil(ceil(h/8)/4).
let layout = WaveletLayout(width: width, height: height)
let layout = WaveletLayout(width: width, height: height, chroma444: false)
XCTAssertEqual(layout.alignedWidth, 256)
XCTAssertEqual(layout.alignedHeight, 160)
XCTAssertEqual(layout.levelWidth(0), 128)
@@ -85,7 +85,7 @@ final class PyroWaveParserTests: XCTestCase {
}
func testDenseParseFillsOffsetsAndCountsBlocks() throws {
let layout = WaveletLayout(width: width, height: height)
let layout = WaveletLayout(width: width, height: height, chroma444: false)
var au = sof(totalBlocks: 4)
au += packet(blockIndex: 0)
au += packet(blockIndex: 3)
@@ -273,6 +273,17 @@ final class PyroWaveGoldenTests: XCTestCase {
try assertMatchesReference(decoded, prefix: "ref-chunked")
}
/// 4:4:4: the chroma components run the full pyramid like luma (no level-0 skip, no
/// early half-res emit) the layout + dispatch structure Phase 4 added
/// (design/pyrowave-444-hdr.md). The fixture comes from the 4:4:4 host encoder; the
/// reference is upstream's own 4:4:4 decode (full-res chroma planes).
func testDense444GoldenFrame() throws {
try XCTSkipIf(!MetalWaveletDecoder.supported, "no capable Metal device")
let au = try fixture("au-dense444")
let decoded = try decode(au: au, chunkAligned: false, windowSize: 0)
try assertMatchesReference(decoded, prefix: "ref-dense444")
}
/// Phase-4 partial delivery: zero a mid-AU window (a lost shard) the frame must still
/// decode (blocks > half) and stay recognizably the same picture (holes reconstruct as
/// localized blur, not garbage).
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+13 -11
View File
@@ -36,11 +36,14 @@ Bandwidth. At the codec's ~1.6 bits-per-pixel operating point (4:2:0, 60 fps):
| 2560×1440 @ 60 | ≈ 355 Mbps |
| 3840×2160 @ 60 | ≈ 800 Mbps |
120 Hz doubles these. Gigabit Ethernet tops out around 940 Mbps of payload, so 4K60 wants
2.5GbE (or a lower rate). **Do not run this over Wi-Fi** — that's what HEVC/AV1 are for.
120 Hz doubles these, 4:4:4 multiplies them by ~1.6, and an HDR (10-bit) session adds ~15 %.
Gigabit Ethernet tops out around 940 Mbps of payload, so 4K60 wants 2.5GbE (or a lower
rate). **Do not run this over Wi-Fi** — that's what HEVC/AV1 are for.
Also: PyroWave is **8-bit SDR only**. An HDR session stays on HEVC/AV1 regardless of this
setting.
PyroWave follows the same 4:4:4 and HDR settings as HEVC/AV1: a session negotiates
full-chroma 4:4:4 when your client's 4:4:4 setting is on, and HDR (BT.2020 PQ, carried in
16-bit planes) when the host's display pipeline is HDR — currently the Windows host only
(the Linux host's capture path has no HDR source yet, so Linux-hosted sessions are SDR).
## Turning it on
@@ -53,10 +56,10 @@ setting.
gamepad console, or launch with `PUNKTFUNK_PREFER_PYROWAVE=1`.
- Apple (Mac, Apple TV 4K, iPad — wired networking strongly recommended): set
**Settings → Codec → PyroWave (wired LAN)**. The option appears only on devices whose
GPU passes the decode probe (Apple Silicon and A13-class or newer); picking it forces
the session SDR.
GPU passes the decode probe (Apple Silicon and A13-class or newer). The decoder follows
the stream: 4:2:0 or 4:4:4, SDR or HDR, per what the session negotiated.
3. Leave the bitrate on Automatic: a PyroWave session pins itself to the ~1.6 bpp rate for
your mode (≈200 Mbps at 1080p60). An explicit bitrate is honored if you set one, but the
your mode (≈200 Mbps at 1080p60; ~2.6 bpp for 4:4:4, +15 % for HDR). An explicit bitrate is honored if you set one, but the
adaptive-bitrate controller stays off either way — this codec has no useful low-rate
regime, so under sustained loss the right move is switching back to HEVC, not degrading.
The pin follows the resolution: a mid-stream resize (e.g. Match window) re-pins the rate
@@ -66,7 +69,6 @@ The stats overlay shows `pyrowave` as the decode path when the mode is active.
## Current limits
- Linux host + Linux client (including docked Deck) and Apple clients (native Metal decode
on Mac / Apple TV 4K / iPad) today; the Windows host is tracked on the
[roadmap](/docs/roadmap).
- 8-bit SDR, 4:2:0 only.
- Linux and Windows hosts; Linux clients (including docked Deck) and Apple clients (native
Metal decode on Mac / Apple TV 4K / iPad) today.
- HDR needs a Windows host (the Linux host's capture path has no HDR source yet).