feat(pyrowave): Apple Metal 4:4:4 + HDR decode, EDR present — self-configured in-band
docker / deploy-docs (push) Successful in 11s
flatpak / build-publish (push) Successful in 6m25s
deb / build-publish (push) Successful in 9m7s
windows-host / package (push) Successful in 16m12s
deb / build-publish-host (push) Successful in 9m31s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m41s
ci / rust (push) Successful in 25m11s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m45s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m2s
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 9m36s
apple / screenshots (push) Successful in 6m43s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 50s
android / android (push) Successful in 13m41s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 14m23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
ci / bench (push) Successful in 5m58s
docker / deploy-docs (push) Successful in 11s
flatpak / build-publish (push) Successful in 6m25s
deb / build-publish (push) Successful in 9m7s
windows-host / package (push) Successful in 16m12s
deb / build-publish-host (push) Successful in 9m31s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m41s
ci / rust (push) Successful in 25m11s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m45s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m2s
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 9m36s
apple / screenshots (push) Successful in 6m43s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 50s
android / android (push) Successful in 13m41s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 14m23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
ci / bench (push) Successful in 5m58s
Phases 4+5 of design/pyrowave-444-hdr.md. The Metal decoder needs NO new ABI: every frame's sequence header carries chroma (444) and, since the Phase-3 stamps, the PQ/BT.2020 bits — so the decoder self-configures per session. Decoder: WaveletLayout grows the 4:4:4 block space (chroma runs the full pyramid like luma — no level-0 skip, no early half-res emit; the Metal kernels were already chroma-agnostic, only the dispatch structure changes); the parser accepts chroma_resolution=444, reads the PQ transfer bit, and lifts the even-dims rule for 444; the plane ring allocates full-res chroma and r16Unorm for PQ streams; CSC rows switch to depth-10 MSB-packed. Presenter: planar HDR passthrough reuses pf_frag_planar on an rgba16Float drawable (itur_2100_PQ + EDR metadata interpret the samples — same split as pf_frag/pf_frag_hdr), plus a new pf_frag_planar_tm PQ->SDR tone-map (shared pqToSdr tail refactored out of pf_frag_hdr_tv) for tvOS-without-headroom AND macOS WINDOWED sessions, whose IOSurface present path (the DCP-panic mitigation) is BGRA8-only. SessionModel stops stripping the HDR/10-bit/444 caps on the PyroWave opt-in. New golden: au-dense444 + upstream's own 4:4:4 reference planes (regenerated via the extended pyrowave_dump_golden); Metal decode matches at 64-67 dB (420 fixtures re-verify 77-88 dB). Full Apple suite 157 tests green on a real M-series GPU. Docs updated: the 8-bit-SDR-only wording is gone, the Windows host is no longer 'on the roadmap', bpp scaling documented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -264,14 +264,11 @@ final class SessionModel: ObservableObject {
|
||||
// PyroWave (wired LAN) is a pure opt-in: picking it in the codec setting both
|
||||
// advertises the bit and prefers it — the host never auto-selects it, and the
|
||||
// picker only offers it when the Metal decode probe passed (simdgroup floor ≈ A13;
|
||||
// every M-series Mac and the ATV 4K gen 3 pass). The codec is 8-bit 4:2:0 SDR
|
||||
// BT.709 by contract, so the opt-in also drops the HDR/10-bit/4:4:4 caps for this
|
||||
// session — HDR sessions stay HEVC/AV1 (plan §4.7).
|
||||
// every M-series Mac and the ATV 4K gen 3 pass). The decoder self-configures from
|
||||
// the per-frame sequence header (4:2:0/4:4:4, SDR/PQ — design/pyrowave-444-hdr.md),
|
||||
// so the session keeps the user's HDR/10-bit/4:4:4 caps exactly like HEVC/AV1.
|
||||
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
|
||||
videoCodecs |= PunktfunkConnection.codecPyroWave
|
||||
videoCaps &= ~(PunktfunkConnection.videoCap10Bit
|
||||
| PunktfunkConnection.videoCapHDR
|
||||
| PunktfunkConnection.videoCap444)
|
||||
}
|
||||
let result = Result { try PunktfunkConnection(
|
||||
host: host.address, port: host.port,
|
||||
|
||||
@@ -196,13 +196,11 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
|
||||
// display-referred SDR. (When the display IS in an HDR mode — requested per session via
|
||||
// AVDisplayManager, see StreamViewIOS — tvOS presents pf_frag_hdr's PQ passthrough instead:
|
||||
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
|
||||
fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
|
||||
texture2d<float> lumaTex [[texture(0)]],
|
||||
texture2d<float> chromaTex [[texture(1)]],
|
||||
constant CscUniform& csc [[buffer(0)]]) {
|
||||
// Y′CbCr → full-range PQ R′G′B′ via the per-frame rows (as pf_frag_hdr).
|
||||
float3 pq = sampleRgb(lumaTex, chromaTex, in.uv, csc);
|
||||
// ST 2084 EOTF: PQ code value → linear light, 1.0 = 10,000 nits.
|
||||
// The shared PQ→display-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
|
||||
// EOTF → 203-nit-anchored scene light → BT.2020→709 primaries → extended-Reinhard rolloff →
|
||||
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map — the
|
||||
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only.
|
||||
static inline float3 pqToSdr(float3 pq) {
|
||||
const float m1 = 2610.0/16384.0;
|
||||
const float m2 = 78.84375;
|
||||
const float c1 = 3424.0/4096.0;
|
||||
@@ -210,20 +208,49 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
|
||||
const float c3 = 18.6875;
|
||||
float3 p = pow(pq, 1.0/m2);
|
||||
float3 lin = pow(max(p - c1, 0.0) / (c2 - c3 * p), 1.0/m1);
|
||||
// Scene-referred with diffuse white at 1.0 (the same 203-nit anchor the EDR path uses).
|
||||
float3 t = lin * (10000.0/203.0);
|
||||
// BT.2020 → BT.709 primaries while still linear; negatives are out-of-gamut, floor them.
|
||||
float3 t709 = float3(
|
||||
dot(t, float3( 1.6605, -0.5876, -0.0728)),
|
||||
dot(t, float3(-0.1246, 1.1329, -0.0083)),
|
||||
dot(t, float3(-0.0182, -0.1006, 1.1187)));
|
||||
t709 = max(t709, 0.0);
|
||||
// Extended Reinhard: 1.0 stays put, the 1000-nit knee lands at display white, above rolls off.
|
||||
const float w = 1000.0/203.0;
|
||||
float3 mapped = saturate(t709 * (1.0 + t709 / (w * w)) / (1.0 + t709));
|
||||
// BT.709 OETF — the same encoding the SDR stream arrives in, so both paths present alike.
|
||||
float3 e = select(1.099 * pow(mapped, 0.45) - 0.099, 4.5 * mapped, mapped < 0.018);
|
||||
return float4(e, 1.0);
|
||||
return e;
|
||||
}
|
||||
|
||||
fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
|
||||
texture2d<float> lumaTex [[texture(0)]],
|
||||
texture2d<float> chromaTex [[texture(1)]],
|
||||
constant CscUniform& csc [[buffer(0)]]) {
|
||||
// Y′CbCr → full-range PQ R′G′B′ via the per-frame rows (as pf_frag_hdr), then the tail.
|
||||
return float4(pqToSdr(sampleRgb(lumaTex, chromaTex, in.uv, csc)), 1.0);
|
||||
}
|
||||
|
||||
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
|
||||
// fold in depth-10 MSB packing) → PQ R′G′B′ → the shared SDR tail. Used when a PQ pyrowave
|
||||
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions
|
||||
// (the IOSurface present path — the DCP-panic mitigation — is BGRA8). The passthrough planar
|
||||
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math — the
|
||||
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
|
||||
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
|
||||
texture2d<float> lumaTex [[texture(0)]],
|
||||
texture2d<float> cbTex [[texture(1)]],
|
||||
texture2d<float> crTex [[texture(2)]],
|
||||
constant CscUniform& csc [[buffer(0)]]) {
|
||||
constexpr sampler s(filter::linear, address::clamp_to_edge);
|
||||
#ifdef PF_BILINEAR_LUMA
|
||||
float lumaY = lumaTex.sample(s, in.uv).r;
|
||||
#else
|
||||
float lumaY = catmullRomLuma(lumaTex, s, in.uv);
|
||||
#endif
|
||||
float2 cuv = chromaUV(lumaTex, cbTex, in.uv);
|
||||
float3 yuv = float3(lumaY, cbTex.sample(s, cuv).r, crTex.sample(s, cuv).r);
|
||||
float3 pq = saturate(float3(dot(csc.r0.xyz, yuv) + csc.r0.w,
|
||||
dot(csc.r1.xyz, yuv) + csc.r1.w,
|
||||
dot(csc.r2.xyz, yuv) + csc.r2.w));
|
||||
return float4(pqToSdr(pq), 1.0);
|
||||
}
|
||||
"""
|
||||
|
||||
@@ -287,6 +314,11 @@ public final class MetalVideoPresenter {
|
||||
private let pipelineHDRToneMap: MTLRenderPipelineState?
|
||||
/// PyroWave's 3-plane SDR path (pf_frag_planar → bgra8) — see `renderPlanar`.
|
||||
private let pipelinePlanar: MTLRenderPipelineState
|
||||
/// PyroWave planar HDR passthrough (pf_frag_planar → rgba16Float; the layer's PQ colour
|
||||
/// space + EDR interpret the samples) and the planar PQ→SDR tone-map (pf_frag_planar_tm →
|
||||
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents).
|
||||
private let pipelinePlanarHDR: MTLRenderPipelineState
|
||||
private let pipelinePlanarToneMap: MTLRenderPipelineState
|
||||
private var textureCache: CVMetalTextureCache?
|
||||
|
||||
/// The PyroWave Metal decoder records on the presenter's device + queue: one device means
|
||||
@@ -337,6 +369,8 @@ public final class MetalVideoPresenter {
|
||||
let pipelineHDR: MTLRenderPipelineState
|
||||
let pipelineHDRToneMap: MTLRenderPipelineState?
|
||||
let pipelinePlanar: MTLRenderPipelineState
|
||||
let pipelinePlanarHDR: MTLRenderPipelineState
|
||||
let pipelinePlanarToneMap: MTLRenderPipelineState
|
||||
do {
|
||||
// DEBUG A/B lever: PUNKTFUNK_BILINEAR_LUMA=1 compiles the shader with Catmull-Rom OFF
|
||||
// (plain bilinear luma) by prepending a #define ahead of the source. Default (unset) is
|
||||
@@ -374,8 +408,18 @@ public final class MetalVideoPresenter {
|
||||
let planar = MTLRenderPipelineDescriptor()
|
||||
planar.vertexFunction = vtx
|
||||
planar.fragmentFunction = library.makeFunction(name: "pf_frag_planar")
|
||||
planar.colorAttachments[0].pixelFormat = .bgra8Unorm // PyroWave is 8-bit SDR
|
||||
planar.colorAttachments[0].pixelFormat = .bgra8Unorm
|
||||
pipelinePlanar = try device.makeRenderPipelineState(descriptor: planar)
|
||||
let planarHdr = MTLRenderPipelineDescriptor()
|
||||
planarHdr.vertexFunction = vtx
|
||||
planarHdr.fragmentFunction = library.makeFunction(name: "pf_frag_planar")
|
||||
planarHdr.colorAttachments[0].pixelFormat = .rgba16Float // PQ passthrough
|
||||
pipelinePlanarHDR = try device.makeRenderPipelineState(descriptor: planarHdr)
|
||||
let planarTm = MTLRenderPipelineDescriptor()
|
||||
planarTm.vertexFunction = vtx
|
||||
planarTm.fragmentFunction = library.makeFunction(name: "pf_frag_planar_tm")
|
||||
planarTm.colorAttachments[0].pixelFormat = .bgra8Unorm
|
||||
pipelinePlanarToneMap = try device.makeRenderPipelineState(descriptor: planarTm)
|
||||
} catch {
|
||||
return nil
|
||||
}
|
||||
@@ -416,6 +460,7 @@ public final class MetalVideoPresenter {
|
||||
return MetalVideoPresenter(
|
||||
device: device, queue: queue, pipelineSDR: pipelineSDR, pipelineHDR: pipelineHDR,
|
||||
pipelineHDRToneMap: pipelineHDRToneMap, pipelinePlanar: pipelinePlanar,
|
||||
pipelinePlanarHDR: pipelinePlanarHDR, pipelinePlanarToneMap: pipelinePlanarToneMap,
|
||||
textureCache: textureCache, layer: layer)
|
||||
}
|
||||
|
||||
@@ -423,6 +468,8 @@ public final class MetalVideoPresenter {
|
||||
device: MTLDevice, queue: MTLCommandQueue, pipelineSDR: MTLRenderPipelineState,
|
||||
pipelineHDR: MTLRenderPipelineState, pipelineHDRToneMap: MTLRenderPipelineState?,
|
||||
pipelinePlanar: MTLRenderPipelineState,
|
||||
pipelinePlanarHDR: MTLRenderPipelineState,
|
||||
pipelinePlanarToneMap: MTLRenderPipelineState,
|
||||
textureCache: CVMetalTextureCache, layer: CAMetalLayer
|
||||
) {
|
||||
self.device = device
|
||||
@@ -431,6 +478,8 @@ public final class MetalVideoPresenter {
|
||||
self.pipelineHDR = pipelineHDR
|
||||
self.pipelineHDRToneMap = pipelineHDRToneMap
|
||||
self.pipelinePlanar = pipelinePlanar
|
||||
self.pipelinePlanarHDR = pipelinePlanarHDR
|
||||
self.pipelinePlanarToneMap = pipelinePlanarToneMap
|
||||
self.textureCache = textureCache
|
||||
self.layer = layer
|
||||
}
|
||||
@@ -652,7 +701,13 @@ public final class MetalVideoPresenter {
|
||||
let surfaceMode = surfacePresentsStaged
|
||||
#endif
|
||||
stagingLock.unlock()
|
||||
configure(hdr: false)
|
||||
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path;
|
||||
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader).
|
||||
#if os(macOS)
|
||||
configure(hdr: planes.pq && !surfaceMode)
|
||||
#else
|
||||
configure(hdr: planes.pq)
|
||||
#endif
|
||||
var csc = planes.csc
|
||||
#if os(macOS)
|
||||
if surfaceMode != surfacePresentsActive {
|
||||
@@ -672,9 +727,21 @@ public final class MetalVideoPresenter {
|
||||
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
|
||||
}
|
||||
#endif
|
||||
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
|
||||
// 8-bit — tvOS without display headroom, or a not-yet-flipped layer — tone-maps
|
||||
// in-shader instead (the pipeline must match the drawable's pixel format).
|
||||
#if os(tvOS)
|
||||
let planarPassthrough = hdrActive && hdrPassthroughActive
|
||||
#else
|
||||
let planarPassthrough = hdrActive
|
||||
#endif
|
||||
let planarPipeline: MTLRenderPipelineState =
|
||||
planes.pq
|
||||
? (planarPassthrough ? pipelinePlanarHDR : pipelinePlanarToneMap)
|
||||
: pipelinePlanar
|
||||
return encodePresent(
|
||||
decodedSize: CGSize(width: planes.width, height: planes.height),
|
||||
targetFromLayout: targetFromLayout, pipeline: pipelinePlanar,
|
||||
targetFromLayout: targetFromLayout, pipeline: planarPipeline,
|
||||
presentAtMediaTime: presentAtMediaTime, onPresented: onPresented,
|
||||
// The ring textures stay valid by ring depth; retaining them here also pins the
|
||||
// slot's set until the sample completes (mirrors the biplanar keep-alive).
|
||||
@@ -716,7 +783,7 @@ public final class MetalVideoPresenter {
|
||||
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
|
||||
return false
|
||||
}
|
||||
encoder.setRenderPipelineState(pipelinePlanar)
|
||||
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
|
||||
encoder.setFragmentTexture(planes.y, index: 0)
|
||||
encoder.setFragmentTexture(planes.cb, index: 1)
|
||||
encoder.setFragmentTexture(planes.cr, index: 2)
|
||||
|
||||
@@ -47,6 +47,9 @@ struct WaveletLayout {
|
||||
|
||||
let width: Int
|
||||
let height: Int
|
||||
/// Full-res chroma (4:4:4): chroma components get the full band set including level 0,
|
||||
/// exactly like luma — upstream `init_block_meta` with `Chroma444`.
|
||||
let chroma444: Bool
|
||||
let alignedWidth: Int
|
||||
let alignedHeight: Int
|
||||
/// blockMeta[component][level][band] = (blockOffset32x32, blockStride32x32); -1 offset =
|
||||
@@ -59,9 +62,10 @@ struct WaveletLayout {
|
||||
func levelWidth(_ level: Int) -> Int { (alignedWidth / 2) >> level }
|
||||
func levelHeight(_ level: Int) -> Int { (alignedHeight / 2) >> level }
|
||||
|
||||
init(width: Int, height: Int) {
|
||||
init(width: Int, height: Int, chroma444: Bool) {
|
||||
self.width = width
|
||||
self.height = height
|
||||
self.chroma444 = chroma444
|
||||
let align = { (v: Int) in
|
||||
max((v + Self.alignment - 1) & ~(Self.alignment - 1), Self.minimumImageSize)
|
||||
}
|
||||
@@ -78,7 +82,7 @@ struct WaveletLayout {
|
||||
let ah = alignedHeight
|
||||
for level in stride(from: Self.decompositionLevels - 1, through: 0, by: -1) {
|
||||
for component in 0..<3 {
|
||||
if level == 0 && component != 0 { continue } // 4:2:0: no top-level chroma
|
||||
if level == 0 && component != 0 && !chroma444 { continue } // 4:2:0: no top-level chroma
|
||||
for band in (level == Self.decompositionLevels - 1 ? 0 : 1)..<4 {
|
||||
let levelW = (aw / 2) >> level
|
||||
let levelH = (ah / 2) >> level
|
||||
@@ -108,6 +112,9 @@ struct ParsedWaveletFrame {
|
||||
var decodedBlocks: Int
|
||||
/// VUI bits from the sequence header (BitstreamSequenceHeader).
|
||||
var bt2020: Bool
|
||||
/// PQ transfer ⇒ HDR session: 16-bit studio-code planes + EDR present (the host stamps
|
||||
/// this bit iff the session negotiated 10-bit — the depth is coupled to the transfer).
|
||||
var pq: Bool
|
||||
var fullRange: Bool
|
||||
|
||||
/// The frame's Y′CbCr→RGB signal for the presenter's planar CSC. PyroWave today is always
|
||||
@@ -203,6 +210,7 @@ enum WaveletBitstream {
|
||||
var totalBlocks = 0
|
||||
var decodedBlocks = 0
|
||||
var bt2020 = false
|
||||
var pq = false
|
||||
var fullRange = false
|
||||
var sawSOF = false
|
||||
|
||||
@@ -220,22 +228,28 @@ enum WaveletBitstream {
|
||||
// siting[31].
|
||||
let code = (word1 >> 24) & 0x3
|
||||
guard code == 0 else { return false } // only START_OF_FRAME is defined
|
||||
let chromaRes = (word1 >> 26) & 1
|
||||
guard chromaRes == 0 else { return false } // host contract: 4:2:0
|
||||
let chroma444 = (word1 >> 26) & 1 != 0
|
||||
let w = Int(word0 & 0x3fff) + 1
|
||||
let h = Int((word0 >> 14) & 0x3fff) + 1
|
||||
guard w >= 2, h >= 2, w % 2 == 0, h % 2 == 0 else { return false }
|
||||
guard w >= 2, h >= 2, chroma444 || (w % 2 == 0 && h % 2 == 0) else {
|
||||
return false
|
||||
}
|
||||
if sawSOF {
|
||||
// One frame, one geometry — a second SOF must agree.
|
||||
guard layout?.width == w, layout?.height == h else { return false }
|
||||
guard layout?.width == w, layout?.height == h,
|
||||
layout?.chroma444 == chroma444
|
||||
else { return false }
|
||||
} else {
|
||||
sawSOF = true
|
||||
let l = WaveletLayout(width: w, height: h)
|
||||
let l = WaveletLayout(width: w, height: h, chroma444: chroma444)
|
||||
layout = l
|
||||
offsets = [UInt32](repeating: .max, count: l.blockCount32)
|
||||
payload.reserveCapacity(64 * 1024 / 4)
|
||||
totalBlocks = Int(word1 & 0xff_ffff)
|
||||
bt2020 = (word1 >> 29) & 1 != 0
|
||||
// transfer_function bit: PQ ⇒ an HDR session (16-bit studio-code
|
||||
// planes by the negotiated coupling — design/pyrowave-444-hdr.md).
|
||||
pq = (word1 >> 28) & 1 != 0
|
||||
fullRange = (word1 >> 30) & 1 == 0 // YCBCR_RANGE_FULL = 0
|
||||
}
|
||||
pos += 8
|
||||
@@ -280,7 +294,7 @@ enum WaveletBitstream {
|
||||
return ParsedWaveletFrame(
|
||||
layout: layout, offsets: offsets, payload: payload,
|
||||
totalBlocks: totalBlocks, decodedBlocks: decodedBlocks,
|
||||
bt2020: bt2020, fullRange: fullRange)
|
||||
bt2020: bt2020, pq: pq, fullRange: fullRange)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -293,6 +307,8 @@ public struct WaveletPlanes: @unchecked Sendable {
|
||||
public let cb: MTLTexture
|
||||
public let cr: MTLTexture
|
||||
public let csc: CscUniform
|
||||
/// PQ (HDR) stream: the presenter picks the HDR/tone-map planar pipeline + EDR config.
|
||||
public let pq: Bool
|
||||
public var width: Int { y.width }
|
||||
public var height: Int { y.height }
|
||||
}
|
||||
@@ -351,6 +367,8 @@ public final class MetalWaveletDecoder {
|
||||
|
||||
private var slots: [Slot] = []
|
||||
private var nextSlot = 0
|
||||
/// The ring's plane format facts (from the last SOF): PQ ⇒ 16-bit UNORM planes.
|
||||
private var hdr16 = false
|
||||
|
||||
/// The current geometry (from the last SOF that built the resources) — the pump reports
|
||||
/// decoded-size changes to the resize overlay from this. PUMP THREAD.
|
||||
@@ -409,8 +427,9 @@ public final class MetalWaveletDecoder {
|
||||
au: au, chunkAligned: chunkAligned, windowSize: windowSize)
|
||||
else { return false }
|
||||
|
||||
if layout?.width != frame.layout.width || layout?.height != frame.layout.height {
|
||||
guard rebuild(layout: frame.layout) else { return false }
|
||||
if layout?.width != frame.layout.width || layout?.height != frame.layout.height
|
||||
|| layout?.chroma444 != frame.layout.chroma444 || hdr16 != frame.pq {
|
||||
guard rebuild(layout: frame.layout, hdr16: frame.pq) else { return false }
|
||||
}
|
||||
guard let layout, !slots.isEmpty else { return false }
|
||||
|
||||
@@ -450,7 +469,7 @@ public final class MetalWaveletDecoder {
|
||||
dequant.setBuffer(slot.payload, offset: 0, index: 1)
|
||||
for level in 0..<WaveletLayout.decompositionLevels {
|
||||
for component in 0..<3 {
|
||||
if level == 0 && component != 0 { continue } // 4:2:0
|
||||
if level == 0 && component != 0 && !layout.chroma444 { continue } // 4:2:0
|
||||
for band in (level == WaveletLayout.decompositionLevels - 1 ? 0 : 1)..<4 {
|
||||
let meta = layout.blockMeta[component][level][band]
|
||||
let w = layout.levelWidth(level)
|
||||
@@ -489,15 +508,20 @@ public final class MetalWaveletDecoder {
|
||||
let grid = MTLSize(width: (rx + 15) / 16, height: (ry + 15) / 16, depth: 1)
|
||||
let group = MTLSize(width: 64, height: 1, depth: 1)
|
||||
if inputLevel == 0 {
|
||||
// 4:2:0: the final full-res pass is luma only (chroma finished at level 1).
|
||||
// Final full-res pass: luma only in 4:2:0 (chroma finished at level 1); all
|
||||
// three components in 4:4:4 (chroma runs the full pyramid like luma).
|
||||
idwt.setComputePipelineState(idwtShiftPipeline)
|
||||
idwt.setTexture(coefficients[0][0], index: 0)
|
||||
idwt.setTexture(slot.y, index: 1)
|
||||
idwt.dispatchThreadgroups(grid, threadsPerThreadgroup: group)
|
||||
let components = layout.chroma444 ? 3 : 1
|
||||
for component in 0..<components {
|
||||
idwt.setTexture(coefficients[component][0], index: 0)
|
||||
let out = component == 0 ? slot.y : (component == 1 ? slot.cb : slot.cr)
|
||||
idwt.setTexture(out, index: 1)
|
||||
idwt.dispatchThreadgroups(grid, threadsPerThreadgroup: group)
|
||||
}
|
||||
} else {
|
||||
for component in 0..<3 {
|
||||
idwt.setTexture(coefficients[component][inputLevel], index: 0)
|
||||
if component != 0 && inputLevel == 1 {
|
||||
if component != 0 && inputLevel == 1 && !layout.chroma444 {
|
||||
// 4:2:0 chroma emits its final half-res plane one level early.
|
||||
idwt.setComputePipelineState(idwtShiftPipeline)
|
||||
idwt.setTexture(component == 1 ? slot.cb : slot.cr, index: 1)
|
||||
@@ -513,7 +537,9 @@ public final class MetalWaveletDecoder {
|
||||
|
||||
let planes = WaveletPlanes(
|
||||
y: slot.y, cb: slot.cb, cr: slot.cr,
|
||||
csc: CscRows.rows(frame.cscSignal, depth: 8, msbPacked: false))
|
||||
csc: CscRows.rows(
|
||||
frame.cscSignal, depth: frame.pq ? 10 : 8, msbPacked: frame.pq),
|
||||
pq: frame.pq)
|
||||
cmd.addCompletedHandler { buffer in
|
||||
completion(buffer.error == nil ? planes : nil)
|
||||
}
|
||||
@@ -524,9 +550,9 @@ public final class MetalWaveletDecoder {
|
||||
|
||||
/// (Re)allocate every size-dependent resource for `layout`'s geometry. Also the mid-stream
|
||||
/// resize path: a Reconfigure shows up here as new SOF dims.
|
||||
private func rebuild(layout newLayout: WaveletLayout) -> Bool {
|
||||
private func rebuild(layout newLayout: WaveletLayout, hdr16 newHdr16: Bool) -> Bool {
|
||||
waveletLog.info(
|
||||
"pyrowave: building decoder \(newLayout.width)x\(newLayout.height) (aligned \(newLayout.alignedWidth)x\(newLayout.alignedHeight), \(newLayout.blockCount32) blocks)")
|
||||
"pyrowave: building decoder \(newLayout.width)x\(newLayout.height) (aligned \(newLayout.alignedWidth)x\(newLayout.alignedHeight), \(newLayout.blockCount32) blocks, \(newLayout.chroma444 ? "4:4:4" : "4:2:0", privacy: .public)\(newHdr16 ? " HDR16" : "", privacy: .public))")
|
||||
var coeff: [[MTLTexture]] = []
|
||||
var lls: [[MTLTexture]] = []
|
||||
for component in 0..<3 {
|
||||
@@ -560,19 +586,22 @@ public final class MetalWaveletDecoder {
|
||||
|
||||
var newSlots: [Slot] = []
|
||||
for i in 0..<Self.ringDepth {
|
||||
let planeFormat: MTLPixelFormat = newHdr16 ? .r16Unorm : .r8Unorm
|
||||
let plane = { (w: Int, h: Int, name: String) -> MTLTexture? in
|
||||
let desc = MTLTextureDescriptor.texture2DDescriptor(
|
||||
pixelFormat: .r8Unorm, width: w, height: h, mipmapped: false)
|
||||
pixelFormat: planeFormat, width: w, height: h, mipmapped: false)
|
||||
desc.usage = [.shaderRead, .shaderWrite]
|
||||
desc.storageMode = .private
|
||||
let t = self.device.makeTexture(descriptor: desc)
|
||||
t?.label = name
|
||||
return t
|
||||
}
|
||||
let cw = newLayout.chroma444 ? newLayout.width : newLayout.width / 2
|
||||
let ch = newLayout.chroma444 ? newLayout.height : newLayout.height / 2
|
||||
guard
|
||||
let y = plane(newLayout.width, newLayout.height, "pyrowave Y[\(i)]"),
|
||||
let cb = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cb[\(i)]"),
|
||||
let cr = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cr[\(i)]"),
|
||||
let cb = plane(cw, ch, "pyrowave Cb[\(i)]"),
|
||||
let cr = plane(cw, ch, "pyrowave Cr[\(i)]"),
|
||||
let offsets = device.makeBuffer(
|
||||
length: max(newLayout.blockCount32 * 4, 4), options: .storageModeShared),
|
||||
let payload = device.makeBuffer(length: 64 * 1024, options: .storageModeShared)
|
||||
@@ -585,6 +614,7 @@ public final class MetalWaveletDecoder {
|
||||
slots = newSlots
|
||||
nextSlot = 0
|
||||
layout = newLayout
|
||||
hdr16 = newHdr16
|
||||
return true
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user