feat(clients/apple): AV1 decode support — OBU plumbing, hardware-gated advertisement
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 15m35s
apple / screenshots (push) Successful in 4m35s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m51s
ci / web (push) Successful in 1m5s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 5m14s
deb / build-publish (push) Successful in 3m31s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 5m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m20s
docker / deploy-docs (push) Successful in 6s

The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs
exclusively, so AV1 was never advertised and the codec picker hid it. Add
the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift):
a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C
CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and
the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack
(temporal delimiter stripped, everything size-fielded, one copy per AU).

VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route
through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) —
keyframe gating keys on the in-band sequence header exactly as the NAL
codecs key on in-band parameter sets, so loss recovery and mid-session
reconfigure work unchanged. AV1 sessions require a hardware decoder
(VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and
both the Hello advertisement and the Settings picker are gated on
VTIsHardwareDecodeSupported — AV1 only appears on devices that can
actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV).

Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk,
the parse against an independent reference, av1C bytes, delta-TU gating,
repack byte-exactness, and — on AV1 hardware — a real
VTDecompressionSession decode through VideoDecoder. Host precedence stays
HEVC > AV1 > H.264, so AV1 engages only when explicitly picked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-09 01:22:49 +02:00
parent bf9be59f0b
commit c3fa6c1514
10 changed files with 927 additions and 34 deletions
+2 -1
View File
@@ -10,7 +10,8 @@ Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (stat
## Features
- **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
- **Hardware decode** — VideoToolbox H.264/HEVC (plus **AV1** on devices with an AV1 hardware
decoder — M3-class Macs, A17 Pro-class iPhones), with a low-latency **stage-2 presenter**
(`VTDecompressionSession``CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
default and an `AVSampleBufferDisplayLayer` fallback.
- **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
@@ -194,11 +194,13 @@ final class SessionModel: ObservableObject {
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
// This client's VideoToolbox path decodes H.264 and HEVC (AV1 depacketization isn't
// wired AnnexB.swift is NAL-only so it must never be advertised here). The host
// resolves the emitted codec from these + the soft `preferredCodec`; `resolvedCodec`
// reflects what it chose.
let videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
// This client's VideoToolbox path decodes H.264 and HEVC everywhere, and AV1 when
// this device has an AV1 hardware decoder (M3-class Macs, A17 Pro-class iPhones
// VideoToolbox has no software AV1 decoder, so advertising it elsewhere would invite
// a stream that can't decode; see AV1.swift). The host resolves the emitted codec
// from these + the soft `preferredCodec`; `resolvedCodec` reflects what it chose.
var videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
if AV1.hardwareDecodeSupported { videoCodecs |= PunktfunkConnection.codecAV1 }
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
@@ -38,13 +38,21 @@ enum SettingsOptions {
HUDPlacement.allCases.map { ($0.label, $0.rawValue) }
/// Video-codec preference (`DefaultsKey.codec`) a soft preference the host falls back from.
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (AnnexB.swift is NAL-only),
/// so it never advertises AV1 offering it here would be a dead setting.
static let codecs: [(label: String, tag: String)] = [
/// AV1 appears only on devices with an AV1 hardware decoder (the same
/// `AV1.hardwareDecodeSupported` gate SessionModel advertises by) elsewhere it would be a
/// dead setting the host could never honor. Ordered by the host's resolve precedence
/// (HEVC > AV1 > H.264).
static let codecs: [(label: String, tag: String)] = {
var options: [(label: String, tag: String)] = [
("Automatic", "auto"),
("HEVC (H.265)", "hevc"),
("H.264 (AVC)", "h264"),
]
if AV1.hardwareDecodeSupported {
options.insert(("AV1", "av1"), at: 2)
}
return options
}()
// MARK: - Bitrate
@@ -269,7 +269,8 @@ public final class PunktfunkConnection {
/// `2` = HEVC (default / older host), `1` = H.264, `4` = AV1. Build the decoder from THIS. The
/// resolved value honors the client's `preferredCodec` when the host could emit it.
public private(set) var resolvedCodec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
/// The resolved codec as an `AnnexB.VideoCodec` (H.264 vs HEVC) drives the NAL parsing.
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
/// Connect and start a session at the requested mode (the host creates a native virtual
@@ -0,0 +1,561 @@
// AV1 (low-overhead OBU bitstream) CoreMedia plumbing the AV1 sibling of AnnexB.swift.
//
// The punktfunk host emits AV1 access units as low-overhead temporal units (the raw encoder
// output every other client feeds ffmpeg): a temporal-delimiter OBU, then on every keyframe,
// per the same in-band-config policy as the NAL codecs a sequence-header OBU, then the frame
// OBUs. VideoToolbox instead wants the ISOBMFF 'av01' flavor: a CMVideoFormatDescription
// carrying an `av1C` configuration record (built from the sequence header), and sample buffers
// holding the temporal unit with the temporal delimiter stripped and every OBU size-fielded.
// This file converts between the two.
//
// HOT PATH: like AnnexB, both pumps run `formatDescription(fromKeyframe:)` +
// `sampleBuffer(au:format:)` once per AU, so everything is built on `forEachOBU` a zero-copy
// scan over the AU's bytes (ranges, not materialized Datas). A delta AU (no sequence header)
// costs a few OBU-header reads; the sample repack leaves exactly one copy (source block
// buffer), mirroring AnnexB.sampleBuffer.
//
// The full sequence-header parse (AV1 spec 5.5.1) runs only when a keyframe actually carries
// one it exists to fill the `av1C` record fields (profile/level/tier/depth/chroma) and the
// colorimetry extensions (so VideoDecoder.isHDRFormat and the presenter's color handling work
// identically across codecs). The host currently gates 10-bit and 4:4:4 to HEVC, so an AV1
// stream is 8-bit 4:2:0 today; the parser still reads depth/chroma/color faithfully so nothing
// here needs touching when that gate lifts.
import CoreMedia
import Foundation
import VideoToolbox
public enum AV1 {
/// True when this device can hardware-decode AV1 (M3-class Macs, A17 Pro-class iPhones,
/// current iPads; false on every Apple TV to date). VideoToolbox has no software AV1
/// decoder, so this is the advertisement gate: a client must never invite a stream it
/// can't decode in real time.
public static let hardwareDecodeSupported: Bool =
VTIsHardwareDecodeSupported(kCMVideoCodecType_AV1)
// MARK: - OBU walking
/// OBU types (AV1 spec 6.2.2) only the ones this file dispatches on.
enum OBUType {
static let sequenceHeader: UInt8 = 1
static let temporalDelimiter: UInt8 = 2
static let padding: UInt8 = 15
}
/// Walk the OBUs of a low-overhead temporal unit without copying: `body` receives the buffer
/// base, each OBU's header range (header byte + optional extension byte + size field, i.e.
/// everything before the payload), payload range, and type and returns false to stop early.
/// The walk ends at the first malformed OBU (forbidden bit set, truncated header, or a size
/// field overrunning the buffer): a torn AU decodes as garbage anyway and the pumps' keyframe
/// recovery re-anchors, so bailing beats guessing at boundaries. An OBU with
/// `obu_has_size_field == 0` extends to the end of the buffer (legal only for the last one).
/// The base pointer is only valid inside `body`.
static func forEachOBU(
in data: Data,
_ body: (
_ base: UnsafePointer<UInt8>, _ header: Range<Int>, _ payload: Range<Int>,
_ type: UInt8
) -> Bool
) {
data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return }
let count = raw.count
var i = 0
while i < count {
let start = i
let h = base[i]
guard h & 0x80 == 0 else { return } // obu_forbidden_bit not an OBU stream
let type = (h >> 3) & 0x0F
let hasExtension = h & 0x04 != 0
let hasSize = h & 0x02 != 0
i += 1
if hasExtension {
guard i < count else { return }
i += 1
}
let payloadLen: Int
if hasSize {
guard let (size, sizeLen) = leb128(base: base, at: i, count: count)
else { return }
i += sizeLen
payloadLen = size
} else {
payloadLen = count - i // no size field: extends to the end (must be last)
}
guard i + payloadLen <= count else { return }
if !body(base, start..<i, i..<(i + payloadLen), type) { return }
i += payloadLen
}
}
}
/// Decode a leb128 value at `at` (AV1 spec 4.10.5). Returns (value, encoded length) or nil
/// on truncation / a value past 32 bits (sizes beyond that are nonsense for an OBU).
private static func leb128(
base: UnsafePointer<UInt8>, at: Int, count: Int
) -> (Int, Int)? {
var value: UInt64 = 0
for i in 0..<8 {
guard at + i < count else { return nil }
let byte = base[at + i]
value |= UInt64(byte & 0x7F) << (7 * i)
if byte & 0x80 == 0 {
guard value <= UInt64(UInt32.max) else { return nil }
return (Int(value), i + 1)
}
}
return nil
}
/// leb128-encoded byte length of `value`.
private static func leb128Length(_ value: Int) -> Int {
var v = UInt32(value)
var n = 1
while v >= 0x80 {
v >>= 7
n += 1
}
return n
}
/// Encode `value` as leb128 into `dst`; returns the byte count written.
private static func putLeb128(_ value: Int, into dst: UnsafeMutableRawPointer) -> Int {
var v = UInt32(value)
var n = 0
repeat {
var byte = UInt8(v & 0x7F)
v >>= 7
if v != 0 { byte |= 0x80 }
dst.storeBytes(of: byte, toByteOffset: n, as: UInt8.self)
n += 1
} while v != 0
return n
}
// MARK: - Sequence header
/// The sequence-header fields the `av1C` record and the colorimetry extensions need
/// (AV1 spec 5.5; color codes are ITU-T H.273, shared with the HEVC VUI).
struct SequenceHeader {
var profile: UInt8 = 0
var levelIdx0: UInt8 = 0
var tier0: UInt8 = 0
var highBitdepth = false
var twelveBit = false
var monochrome = false
var subsamplingX = true
var subsamplingY = true
var chromaSamplePosition: UInt8 = 0
/// H.273 codes; 2 = unspecified (the spec default when no color description is coded).
var colorPrimaries: UInt8 = 2
var transferCharacteristics: UInt8 = 2
var matrixCoefficients: UInt8 = 2
var fullRange = false
var maxWidth = 0
var maxHeight = 0
}
/// MSB-first bit reader over the sequence-header payload. Every read is bounds-checked and
/// returns nil on overrun the parser guard-lets each field so a truncated header yields
/// nil rather than garbage.
private struct BitReader {
private let bytes: UnsafePointer<UInt8>
private let bitCount: Int
private var pos = 0
init(bytes: UnsafePointer<UInt8>, count: Int) {
self.bytes = bytes
self.bitCount = count * 8
}
mutating func f(_ n: Int) -> UInt32? {
guard n <= 32, pos + n <= bitCount else { return nil }
var v: UInt32 = 0
for _ in 0..<n {
let bit = (bytes[pos >> 3] >> (7 - UInt8(pos & 7))) & 1
v = (v << 1) | UInt32(bit)
pos += 1
}
return v
}
mutating func flag() -> Bool? { f(1).map { $0 == 1 } }
/// uvlc() (spec 4.10.3) only `num_ticks_per_picture_minus_1` uses it here.
mutating func uvlc() -> UInt32? {
var leadingZeros = 0
while true {
guard let b = f(1) else { return nil }
if b == 1 { break }
leadingZeros += 1
if leadingZeros >= 32 { return nil }
}
if leadingZeros == 0 { return 0 }
guard let v = f(leadingZeros) else { return nil }
return v + (1 << leadingZeros) - 1
}
}
/// Parse a sequence-header OBU payload (spec 5.5.1 the full walk down to color_config,
/// which is what `av1C` + the colorimetry extensions are built from). Returns nil on any
/// truncation or spec violation.
static func parseSequenceHeader(_ payload: Data) -> SequenceHeader? {
payload.withUnsafeBytes { (raw: UnsafeRawBufferPointer) -> SequenceHeader? in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return nil }
var r = BitReader(bytes: base, count: raw.count)
var sh = SequenceHeader()
guard let profile = r.f(3), profile <= 2 else { return nil }
sh.profile = UInt8(profile)
guard r.flag() != nil else { return nil } // still_picture
guard let reduced = r.flag() else { return nil }
var decoderModelInfoPresent = false
var bufferDelayLengthMinus1 = 0
if reduced {
guard let level = r.f(5) else { return nil }
sh.levelIdx0 = UInt8(level)
sh.tier0 = 0
} else {
guard let timingInfoPresent = r.flag() else { return nil }
if timingInfoPresent {
guard r.f(32) != nil, r.f(32) != nil, // num_units_in_display_tick, time_scale
let equalPictureInterval = r.flag()
else { return nil }
if equalPictureInterval, r.uvlc() == nil { return nil }
guard let dmip = r.flag() else { return nil }
decoderModelInfoPresent = dmip
if decoderModelInfoPresent {
guard let bdl = r.f(5), r.f(32) != nil, r.f(5) != nil, r.f(5) != nil
else { return nil }
bufferDelayLengthMinus1 = Int(bdl)
}
}
guard let initialDisplayDelayPresent = r.flag(),
let opCountMinus1 = r.f(5)
else { return nil }
for i in 0...Int(opCountMinus1) {
guard r.f(12) != nil, let level = r.f(5) else { return nil }
var tier: UInt32 = 0
if level > 7 {
guard let t = r.f(1) else { return nil }
tier = t
}
if i == 0 {
sh.levelIdx0 = UInt8(level)
sh.tier0 = UInt8(tier)
}
if decoderModelInfoPresent {
guard let present = r.flag() else { return nil }
if present {
let n = bufferDelayLengthMinus1 + 1
guard r.f(n) != nil, r.f(n) != nil, r.f(1) != nil else { return nil }
}
}
if initialDisplayDelayPresent {
guard let present = r.flag() else { return nil }
if present, r.f(4) == nil { return nil }
}
}
}
guard let widthBitsMinus1 = r.f(4), let heightBitsMinus1 = r.f(4),
let maxWidthMinus1 = r.f(Int(widthBitsMinus1) + 1),
let maxHeightMinus1 = r.f(Int(heightBitsMinus1) + 1)
else { return nil }
sh.maxWidth = Int(maxWidthMinus1) + 1
sh.maxHeight = Int(maxHeightMinus1) + 1
if !reduced {
guard let frameIdNumbersPresent = r.flag() else { return nil }
if frameIdNumbersPresent {
guard r.f(4) != nil, r.f(3) != nil else { return nil }
}
}
// use_128x128_superblock, enable_filter_intra, enable_intra_edge_filter
guard r.f(3) != nil else { return nil }
if !reduced {
// enable_interintra_compound enable_dual_filter
guard r.f(4) != nil, let enableOrderHint = r.flag() else { return nil }
if enableOrderHint {
guard r.f(2) != nil else { return nil } // jnt_comp, ref_frame_mvs
}
guard let chooseScreenContentTools = r.flag() else { return nil }
let forceScreenContentTools: UInt32
if chooseScreenContentTools {
forceScreenContentTools = 2 // SELECT_SCREEN_CONTENT_TOOLS
} else {
guard let v = r.f(1) else { return nil }
forceScreenContentTools = v
}
if forceScreenContentTools > 0 {
guard let chooseIntegerMv = r.flag() else { return nil }
if !chooseIntegerMv, r.f(1) == nil { return nil }
}
if enableOrderHint, r.f(3) == nil { return nil } // order_hint_bits_minus_1
}
// enable_superres, enable_cdef, enable_restoration
guard r.f(3) != nil else { return nil }
// color_config() (spec 5.5.2)
guard let highBitdepth = r.flag() else { return nil }
sh.highBitdepth = highBitdepth
if sh.profile == 2, highBitdepth {
guard let twelveBit = r.flag() else { return nil }
sh.twelveBit = twelveBit
}
if sh.profile == 1 {
sh.monochrome = false
} else {
guard let mono = r.flag() else { return nil }
sh.monochrome = mono
}
guard let colorDescriptionPresent = r.flag() else { return nil }
if colorDescriptionPresent {
guard let cp = r.f(8), let tc = r.f(8), let mc = r.f(8) else { return nil }
sh.colorPrimaries = UInt8(cp)
sh.transferCharacteristics = UInt8(tc)
sh.matrixCoefficients = UInt8(mc)
}
if sh.monochrome {
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
sh.subsamplingX = true
sh.subsamplingY = true
sh.chromaSamplePosition = 0
return sh
}
if sh.colorPrimaries == 1, sh.transferCharacteristics == 13,
sh.matrixCoefficients == 0 {
// BT.709 + sRGB + identity forces full-range 4:4:4.
sh.fullRange = true
sh.subsamplingX = false
sh.subsamplingY = false
return sh
}
guard let fullRange = r.flag() else { return nil }
sh.fullRange = fullRange
switch sh.profile {
case 0:
sh.subsamplingX = true
sh.subsamplingY = true
case 1:
sh.subsamplingX = false
sh.subsamplingY = false
default: // profile 2
if sh.highBitdepth, sh.twelveBit {
guard let ssx = r.flag() else { return nil }
sh.subsamplingX = ssx
if ssx {
guard let ssy = r.flag() else { return nil }
sh.subsamplingY = ssy
} else {
sh.subsamplingY = false
}
} else {
sh.subsamplingX = true
sh.subsamplingY = false
}
}
if sh.subsamplingX, sh.subsamplingY {
guard let csp = r.f(2) else { return nil }
sh.chromaSamplePosition = UInt8(csp)
}
return sh
}
}
// MARK: - Format description
/// Build a format description from a keyframe AU's in-band sequence header the AV1
/// equivalent of `AnnexB.formatDescription(fromIDR:)`. Returns nil when the AU carries no
/// sequence-header OBU (a delta frame): the pumps latch the previous description exactly as
/// they do for the NAL codecs. The description carries the `av1C` record (with the sequence
/// header as its configOBUs) plus colorimetry extensions mapped from color_config, so
/// `VideoDecoder.isHDRFormat` and the presenter treat AV1 like any other stream.
public static func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
// The sequence-header OBU, re-emitted with a size field (encoders size-field everything
// in practice; the rewrap also covers a last-OBU-without-size corner).
var seqHeaderOBU: Data?
var seqHeaderPayload: Data?
forEachOBU(in: au) { base, header, payload, type in
guard type == OBUType.sequenceHeader else { return true }
var obu = Data(capacity: 2 + leb128Length(payload.count) + payload.count)
obu.append(base[header.lowerBound] | 0x02) // has_size_field set
if base[header.lowerBound] & 0x04 != 0 { // extension byte rides along
obu.append(base[header.lowerBound + 1])
}
var lenBuf = [UInt8](repeating: 0, count: 8)
let lenLen = lenBuf.withUnsafeMutableBytes {
putLeb128(payload.count, into: $0.baseAddress!)
}
obu.append(contentsOf: lenBuf[0..<lenLen])
obu.append(UnsafeBufferPointer(start: base + payload.lowerBound, count: payload.count))
seqHeaderOBU = obu
seqHeaderPayload = Data(bytes: base + payload.lowerBound, count: payload.count)
return false
}
guard let seqHeaderOBU, let seqHeaderPayload,
let sh = parseSequenceHeader(seqHeaderPayload),
sh.maxWidth > 0, sh.maxHeight > 0
else { return nil }
// AV1CodecConfigurationRecord (AV1-ISOBMFF §2.3): 4 fixed bytes + configOBUs.
var av1C = Data(capacity: 4 + seqHeaderOBU.count)
av1C.append(0x81) // marker=1, version=1
av1C.append((sh.profile << 5) | sh.levelIdx0)
av1C.append(
(sh.tier0 << 7)
| ((sh.highBitdepth ? 1 : 0) << 6)
| ((sh.twelveBit ? 1 : 0) << 5)
| ((sh.monochrome ? 1 : 0) << 4)
| ((sh.subsamplingX ? 1 : 0) << 3)
| ((sh.subsamplingY ? 1 : 0) << 2)
| sh.chromaSamplePosition)
av1C.append(0) // no initial_presentation_delay
av1C.append(seqHeaderOBU)
// Colorimetry from color_config's H.273 codes; unspecified (2) falls back to BT.709
// the host's SDR default, same policy the presenter applies elsewhere.
let primaries: CFString = {
switch sh.colorPrimaries {
case 9: return kCMFormatDescriptionColorPrimaries_ITU_R_2020
case 6: return kCMFormatDescriptionColorPrimaries_SMPTE_C
case 5: return kCMFormatDescriptionColorPrimaries_EBU_3213
default: return kCMFormatDescriptionColorPrimaries_ITU_R_709_2
}
}()
let transfer: CFString = {
switch sh.transferCharacteristics {
case 16: return kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ
case 18: return kCMFormatDescriptionTransferFunction_ITU_R_2100_HLG
case 13: return kCMFormatDescriptionTransferFunction_sRGB
case 8: return kCMFormatDescriptionTransferFunction_Linear
default: return kCMFormatDescriptionTransferFunction_ITU_R_709_2
}
}()
let matrix: CFString = {
switch sh.matrixCoefficients {
case 9, 10: return kCMFormatDescriptionYCbCrMatrix_ITU_R_2020
case 5, 6: return kCMFormatDescriptionYCbCrMatrix_ITU_R_601_4
default: return kCMFormatDescriptionYCbCrMatrix_ITU_R_709_2
}
}()
let extensions: [CFString: Any] = [
kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: ["av1C": av1C],
kCMFormatDescriptionExtension_ColorPrimaries: primaries,
kCMFormatDescriptionExtension_TransferFunction: transfer,
kCMFormatDescriptionExtension_YCbCrMatrix: matrix,
kCMFormatDescriptionExtension_FullRangeVideo: sh.fullRange,
]
var format: CMVideoFormatDescription?
let status = CMVideoFormatDescriptionCreate(
allocator: kCFAllocatorDefault,
codecType: kCMVideoCodecType_AV1,
width: Int32(sh.maxWidth), height: Int32(sh.maxHeight),
extensions: extensions as CFDictionary,
formatDescriptionOut: &format)
return status == noErr ? format : nil
}
// MARK: - Sample buffers
/// Wrap one temporal unit as a decode-ready CMSampleBuffer in the ISOBMFF 'av01' sample
/// format: the temporal-delimiter (and padding) OBUs are dropped, every remaining OBU is
/// re-emitted with a size field, and mirroring AnnexB.sampleBuffer the result is packed
/// straight into the CMBlockBuffer's allocation (sized by a first cheap scan). The sequence
/// header stays in-band (spec-legal: it's bit-identical to the one in `av1C`, which is
/// rebuilt from the same keyframe), preserving the host's self-contained-keyframe policy.
public static func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
// Pass 1: byte scan only total repacked size of the kept OBUs.
var total = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
let headerLen = base[header.lowerBound] & 0x04 != 0 ? 2 : 1
total += headerLen + leb128Length(payload.count) + payload.count
return true
}
// Nothing decodable (a delimiter-only AU our host never sends one): drop it rather
// than hand the decoder an empty sample.
guard total > 0 else { return nil }
var blockBuffer: CMBlockBuffer?
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault, memoryBlock: nil,
blockLength: total, blockAllocator: kCFAllocatorDefault,
customBlockSource: nil, offsetToData: 0, dataLength: total,
flags: kCMBlockBufferAssureMemoryNowFlag, blockBufferOut: &blockBuffer) == noErr,
let block = blockBuffer
else { return nil }
var dstLen = 0
var dstPtr: UnsafeMutablePointer<CChar>?
guard CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &dstLen,
dataPointerOut: &dstPtr) == noErr,
dstLen == total, let dstPtr
else { return nil }
// Pass 2: the single copy header (+extension) byte, size field, payload per OBU.
let dst = UnsafeMutableRawPointer(dstPtr)
var off = 0
forEachOBU(in: au.data) { base, header, payload, type in
if type == OBUType.temporalDelimiter || type == OBUType.padding { return true }
dst.storeBytes(
of: base[header.lowerBound] | 0x02, toByteOffset: off, as: UInt8.self)
off += 1
if base[header.lowerBound] & 0x04 != 0 {
dst.storeBytes(
of: base[header.lowerBound + 1], toByteOffset: off, as: UInt8.self)
off += 1
}
off += putLeb128(payload.count, into: dst.advanced(by: off))
dst.advanced(by: off)
.copyMemory(from: base + payload.lowerBound, byteCount: payload.count)
off += payload.count
return true
}
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: CMTime(value: Int64(au.ptsNs), timescale: 1_000_000_000),
decodeTimeStamp: .invalid)
var sampleSize = total
var sample: CMSampleBuffer?
guard CMSampleBufferCreate(
allocator: kCFAllocatorDefault, dataBuffer: block, dataReady: true,
makeDataReadyCallback: nil, refcon: nil, formatDescription: format,
sampleCount: 1, sampleTimingEntryCount: 1, sampleTimingArray: &timing,
sampleSizeEntryCount: 1, sampleSizeArray: &sampleSize,
sampleBufferOut: &sample) == noErr
else { return nil }
// Low-latency display: render on arrival, don't wait for a clock.
if let attachments = CMSampleBufferGetSampleAttachmentsArray(sample!, createIfNecessary: true) {
let dict = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self)
CFDictionarySetValue(
dict,
Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(),
Unmanaged.passUnretained(kCFBooleanTrue).toOpaque())
}
return sample
}
}
extension VideoCodec {
/// Codec-dispatching format-description refresh: the AV1 path keys on an in-band sequence
/// header, the NAL codecs on in-band parameter sets one call site in each pump.
public func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
self == .av1
? AV1.formatDescription(fromKeyframe: au)
: AnnexB.formatDescription(fromIDR: au, codec: self)
}
/// Codec-dispatching sample wrap (see `formatDescription(fromKeyframe:)`).
public func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
self == .av1
? AV1.sampleBuffer(au: au, format: format)
: AnnexB.sampleBuffer(au: au, format: format, codec: self)
}
}
@@ -6,8 +6,10 @@
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
// the codec the host resolved in the Welcome (`connection.videoCodec`) HEVC and H.264
// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
// is not an Annex-B/NAL codec and isn't handled here this client never advertises it in
// the Hello, so a host never emits it at us.
// is not an Annex-B/NAL codec and isn't handled here its OBU flavor of the same plumbing
// lives in AV1.swift, and the pumps reach both through `VideoCodec`'s dispatching
// `formatDescription(fromKeyframe:)` / `sampleBuffer(au:format:)`, so nothing below is ever
// called with `.av1`.
//
// HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
// once per AU, so the conversion is built on `forEachNAL` a zero-copy scan over the AU's bytes
@@ -23,10 +25,15 @@ import Foundation
public enum VideoCodec: Equatable {
case h264
case hevc
case av1
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown HEVC).
public init(wire: UInt8) {
self = wire == 0x01 ? .h264 : .hevc // 0x01 = PUNKTFUNK_CODEC_H264
switch wire {
case 0x01: self = .h264 // PUNKTFUNK_CODEC_H264
case 0x04: self = .av1 // PUNKTFUNK_CODEC_AV1
default: self = .hevc // PUNKTFUNK_CODEC_HEVC the default / older-host codec
}
}
/// NAL unit type from a NAL's first byte. HEVC: bits 1..6; H.264: bits 0..4.
@@ -140,6 +147,8 @@ public enum AnnexB {
sets = [vps, sps, pps]
case .h264:
sets = [sps, pps]
case .av1:
return nil // OBU stream, no parameter-set NALs handled in AV1.swift, never here
}
var format: CMVideoFormatDescription?
@@ -175,6 +184,8 @@ public enum AnnexB {
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
formatDescriptionOut: &format)
case .av1:
break // unreachable the .av1 arm above already returned
}
}
return status == noErr ? format : nil
@@ -277,7 +277,7 @@ public final class Stage2Pipeline {
}
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec) {
if let f = connection.videoCodec.formatDescription(fromKeyframe: au.data) {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
@@ -71,7 +71,7 @@ final class StreamPump {
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
let idrFormat = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec)
let idrFormat = connection.videoCodec.formatDescription(fromKeyframe: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
@@ -95,7 +95,7 @@ final class StreamPump {
}
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f, codec: connection.videoCodec),
let sample = connection.videoCodec.sampleBuffer(au: au, format: f),
!token.isStopped // don't enqueue a stale frame after a restart
else { continue }
layer.enqueue(sample)
@@ -1,4 +1,5 @@
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's HEVC AUs.
// Stage-2 presenter, decode half: explicit VideoToolbox decode of the host's AUs (H.264 /
// HEVC / AV1 whatever the Welcome resolved).
//
// Stage-1 hands compressed samples to AVSampleBufferDisplayLayer, which decodes AND presents
// internally with no per-frame callback so neither decode-completion nor present can be
@@ -61,8 +62,8 @@ public final class VideoDecoder: @unchecked Sendable {
/// depth / HDR). Read inside `createSessionLocked` under `lock`.
private var chroma444 = false
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the AnnexB
/// NAL parsing (H.264 vs HEVC parameter sets). Read under `lock`.
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the
/// bitstream framing (H.264/HEVC NAL parsing vs AV1 OBU repack). Read under `lock`.
private var codec: VideoCodec = .hevc
public init(
@@ -84,8 +85,8 @@ public final class VideoDecoder: @unchecked Sendable {
lock.unlock()
}
/// Select the negotiated codec (H.264 vs HEVC). Call once at session start, before decoding,
/// from `connection.videoCodec`. Thread-safe.
/// Select the negotiated codec (H.264 / HEVC / AV1). Call once at session start, before
/// decoding, from `connection.videoCodec`. Thread-safe.
public func setCodec(_ c: VideoCodec) {
lock.lock()
codec = c
@@ -93,8 +94,9 @@ public final class VideoDecoder: @unchecked Sendable {
}
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
/// caller resolves `format` from the IDR exactly as stage-1 does (`AnnexB.formatDescription`).
/// Returns false if the session couldn't be created or the frame couldn't be submitted.
/// caller resolves `format` from the keyframe exactly as stage-1 does
/// (`VideoCodec.formatDescription(fromKeyframe:)`). Returns false if the session couldn't be
/// created or the frame couldn't be submitted.
@discardableResult
public func decode(au: AccessUnit, format newFormat: CMVideoFormatDescription) -> Bool {
lock.lock()
@@ -112,7 +114,7 @@ public final class VideoDecoder: @unchecked Sendable {
// invalidate the session between here and DecodeFrame. The VT output callback takes the
// ring lock, not this one, so there's no re-entrancy. DecodeFrame is async non-blocking.
guard let session,
let sample = AnnexB.sampleBuffer(au: au, format: newFormat, codec: codec)
let sample = codec.sampleBuffer(au: au, format: newFormat)
else { lock.unlock(); return false }
var infoOut = VTDecodeInfoFlags()
let status = VTDecompressionSessionDecodeFrame(
@@ -199,13 +201,14 @@ public final class VideoDecoder: @unchecked Sendable {
var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decoderOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
// 4:4:4 sessions REQUIRE a hardware decoder: we only advertise 4:4:4 when the hardware probe
// passed, so a hardware-incapable mode (e.g. a resolution past the HW 4:4:4 ceiling) must fail
// HERE, synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software 4:4:4 decoder far too slow for a real-time stream. 4:2:0 keeps the
// software fallback (nil spec) as a robustness net.
// 4:4:4 and AV1 sessions REQUIRE a hardware decoder: both are only advertised when the
// hardware gate passed (the 4:4:4 probe / `AV1.hardwareDecodeSupported`), so a
// hardware-incapable mode (e.g. a resolution past a HW ceiling) must fail HERE,
// synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software decoder far too slow for a real-time stream. 4:2:0
// H.264/HEVC keeps the software fallback (nil spec) as a robustness net.
let spec: CFDictionary? =
chroma444
chroma444 || codec == .av1
? [kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true] as CFDictionary
: nil
var newSession: VTDecompressionSession?
@@ -0,0 +1,306 @@
// Real-bitstream tests for the AV1 OBU CoreMedia plumbing (AV1.swift): the OBU walk, the
// sequence-header parse, the `av1C` format description, the sample repack, the VideoCodec
// dispatch and, on AV1-hardware devices, a real VTDecompressionSession decode of the blob
// (the AV1 counterpart of VideoToolboxRoundTripTests; there is no VT AV1 *encoder*, so the
// bitstream is generated offline).
//
// Blobs: the first two temporal units of an SVT-AV1 clip a keyframe TU (temporal delimiter +
// sequence header + frame) and a delta TU (temporal delimiter + frame), exactly the wire shape
// the punktfunk host emits. Generated with:
// ffmpeg -f lavfi -i testsrc2=size=320x180:rate=30 -frames:v 2 \
// -c:v libsvtav1 -preset 12 -crf 63 -g 30 -f obu out.obu
// then split on the temporal-delimiter OBUs. 320×180 clears the hardware decoder's
// minimum-dimension floor (see Probe444Blobs). Ground truth (ffprobe + a reference parse):
// Main profile (0), level_idx 0, tier 0, 8-bit, 4:2:0, no color description (unspecified),
// studio range, max frame 320×180, chroma sample position 0.
import CoreMedia
import VideoToolbox
import XCTest
@testable import PunktfunkKit
/// Sendable holder for the values the (background-thread) decode callback writes.
private final class FrameBox: @unchecked Sendable {
let lock = NSLock()
var frame: ReadyFrame?
var error: OSStatus?
}
final class AV1Tests: XCTestCase {
// MARK: - OBU walk
func testOBUWalkKeyframe() {
var seen: [(type: UInt8, payloadCount: Int)] = []
var lastEnd = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, header, payload, type in
XCTAssertEqual(header.lowerBound, lastEnd, "OBUs must be contiguous")
XCTAssertEqual(header.upperBound, payload.lowerBound)
lastEnd = payload.upperBound
seen.append((type, payload.count))
return true
}
XCTAssertEqual(lastEnd, Self.keyframeTU.count, "walk must cover the whole TU")
XCTAssertEqual(seen.map(\.type), [
AV1.OBUType.temporalDelimiter, AV1.OBUType.sequenceHeader, 6, // 6 = OBU_FRAME
])
XCTAssertEqual(seen[0].payloadCount, 0)
XCTAssertEqual(seen[1].payloadCount, 11)
}
func testOBUWalkStopsEarly() {
var calls = 0
AV1.forEachOBU(in: Data(Self.keyframeTU)) { _, _, _, _ in
calls += 1
return false
}
XCTAssertEqual(calls, 1)
}
func testOBUWalkRejectsGarbage() {
// 0x80 = forbidden bit set: not an OBU stream, the walk must not call the body.
AV1.forEachOBU(in: Data([0x80, 0x00, 0x01])) { _, _, _, _ in
XCTFail("garbage must not yield OBUs")
return true
}
// A size field overrunning the buffer stops the walk at the previous OBU.
var truncated = Data([0x12, 0x00]) // valid TD
truncated.append(contentsOf: [0x0A, 0x7F, 0x01]) // seq header claiming 127 bytes, 1 present
var types: [UInt8] = []
AV1.forEachOBU(in: truncated) { _, _, _, type in
types.append(type)
return true
}
XCTAssertEqual(types, [AV1.OBUType.temporalDelimiter])
}
// MARK: - Sequence header
func testSequenceHeaderParse() throws {
// The sequence-header OBU payload sits at bytes 4..<15 (TD 2 bytes, header+size 2 bytes).
let payload = Data(Self.keyframeTU[4..<15])
let sh = try XCTUnwrap(AV1.parseSequenceHeader(payload))
XCTAssertEqual(sh.profile, 0) // Main
XCTAssertEqual(sh.levelIdx0, 0)
XCTAssertEqual(sh.tier0, 0)
XCTAssertFalse(sh.highBitdepth)
XCTAssertFalse(sh.twelveBit)
XCTAssertFalse(sh.monochrome)
XCTAssertTrue(sh.subsamplingX) // profile 0 4:2:0
XCTAssertTrue(sh.subsamplingY)
XCTAssertEqual(sh.chromaSamplePosition, 0)
XCTAssertEqual(sh.colorPrimaries, 2) // no color description unspecified
XCTAssertEqual(sh.transferCharacteristics, 2)
XCTAssertEqual(sh.matrixCoefficients, 2)
XCTAssertFalse(sh.fullRange)
XCTAssertEqual(sh.maxWidth, 320)
XCTAssertEqual(sh.maxHeight, 180)
}
func testSequenceHeaderRejectsTruncation() {
// The parse consumes exactly 79 bits of this header (it stops after
// chroma_sample_position the last field av1C needs), so 10 bytes suffice and the
// 11th only carries fields past the parse. Everything shorter must fail cleanly.
let payload = Data(Self.keyframeTU[4..<15])
for cut in 0..<10 {
XCTAssertNil(
AV1.parseSequenceHeader(payload.prefix(cut)),
"a header truncated to \(cut) bytes must not parse")
}
XCTAssertNotNil(AV1.parseSequenceHeader(payload.prefix(10)))
}
// MARK: - Format description
func testFormatDescriptionFromKeyframe() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
XCTAssertEqual(CMFormatDescriptionGetMediaSubType(format), kCMVideoCodecType_AV1)
let dims = CMVideoFormatDescriptionGetDimensions(format)
XCTAssertEqual(dims.width, 320)
XCTAssertEqual(dims.height, 180)
// The av1C record: marker/version, profile+level, the packed flags byte (4:2:0, 8-bit,
// csp 0 0x0C), no presentation delay then the sequence-header OBU verbatim.
let atoms = try XCTUnwrap(
CMFormatDescriptionGetExtension(
format,
extensionKey: kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms)
as? [String: Any])
let av1C = try XCTUnwrap(atoms["av1C"] as? Data)
XCTAssertEqual([UInt8](av1C.prefix(4)), [0x81, 0x00, 0x0C, 0x00])
XCTAssertEqual([UInt8](av1C.dropFirst(4)), [UInt8](Self.keyframeTU[2..<15]),
"configOBUs must be the size-fielded sequence-header OBU")
// Unspecified color codes fall back to BT.709 studio range and the transfer-function
// extension is what keeps VideoDecoder.isHDRFormat working for AV1.
let transfer = CMFormatDescriptionGetExtension(
format, extensionKey: kCMFormatDescriptionExtension_TransferFunction) as? String
XCTAssertEqual(transfer, kCMFormatDescriptionTransferFunction_ITU_R_709_2 as String)
XCTAssertFalse(VideoDecoder.isHDRFormat(format))
}
func testDeltaTUYieldsNoFormat() {
XCTAssertNil(AV1.formatDescription(fromKeyframe: Data(Self.deltaTU)),
"a delta TU has no sequence header — the pumps must latch the previous one")
}
// MARK: - Sample repack
func testSampleRepack() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 1_000_000, frameIndex: 0, flags: 0, receivedNs: 0)
let sample = try XCTUnwrap(AV1.sampleBuffer(au: au, format: format))
// The blob is already fully size-fielded, so the repack is byte-identical minus the
// 2-byte temporal delimiter.
let block = try XCTUnwrap(CMSampleBufferGetDataBuffer(sample))
var length = 0
var ptr: UnsafeMutablePointer<CChar>?
XCTAssertEqual(CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &length,
dataPointerOut: &ptr), noErr)
let bytes = UnsafeRawBufferPointer(start: ptr, count: length)
XCTAssertEqual([UInt8](bytes), [UInt8](Self.keyframeTU[2...]))
// No temporal delimiter survives, and the pts round-trips at nanosecond scale.
AV1.forEachOBU(in: Data(bytes)) { _, _, _, type in
XCTAssertNotEqual(type, AV1.OBUType.temporalDelimiter)
return true
}
let pts = CMSampleBufferGetPresentationTimeStamp(sample)
XCTAssertEqual(pts.value, 1_000_000)
XCTAssertEqual(pts.timescale, 1_000_000_000)
}
func testSampleRepackDelimiterOnlyIsDropped() throws {
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data([0x12, 0x00]), ptsNs: 0, frameIndex: 0, flags: 0, receivedNs: 0)
XCTAssertNil(AV1.sampleBuffer(au: au, format: format))
}
// MARK: - VideoCodec dispatch
func testWireCodecResolution() {
XCTAssertEqual(VideoCodec(wire: 0x01), .h264)
XCTAssertEqual(VideoCodec(wire: 0x02), .hevc)
XCTAssertEqual(VideoCodec(wire: 0x04), .av1)
XCTAssertEqual(VideoCodec(wire: 0xFF), .hevc) // unknown the default codec
}
func testCodecDispatch() {
let au = Data(Self.keyframeTU)
XCTAssertNotNil(VideoCodec.av1.formatDescription(fromKeyframe: au))
// The same bytes through the NAL paths must not parse proves the dispatch matters.
XCTAssertNil(VideoCodec.hevc.formatDescription(fromKeyframe: au))
XCTAssertNil(VideoCodec.h264.formatDescription(fromKeyframe: au))
}
// MARK: - Hardware decode (end to end)
/// The AV1 counterpart of VideoToolboxRoundTripTests' decode half: the keyframe blob through
/// the REAL VideoDecoder (format description repack hardware VTDecompressionSession).
/// Pixels out = the whole AV1 decode path is sound. Skipped on devices without AV1 hardware
/// (exactly the devices the client never advertises AV1 from).
func testHardwareDecodeEndToEnd() throws {
try XCTSkipUnless(
AV1.hardwareDecodeSupported, "no AV1 hardware decoder — AV1 is never advertised here")
let box = FrameBox()
let decoded = expectation(description: "decoded frame")
let decoder = VideoDecoder(
onDecoded: { frame in
box.lock.lock()
box.frame = frame
box.lock.unlock()
decoded.fulfill()
},
onDecodeError: { status in
box.lock.lock()
box.error = status
box.lock.unlock()
decoded.fulfill()
})
decoder.setCodec(.av1)
let format = try XCTUnwrap(AV1.formatDescription(fromKeyframe: Data(Self.keyframeTU)))
let au = AccessUnit(
data: Data(Self.keyframeTU), ptsNs: 42_000_000, frameIndex: 0, flags: 0, receivedNs: 1)
XCTAssertTrue(decoder.decode(au: au, format: format), "hardware session must accept the keyframe")
wait(for: [decoded], timeout: 5)
box.lock.lock()
let frame = box.frame
let error = box.error
box.lock.unlock()
XCTAssertNil(error.map { "decode error \($0)" })
let ready = try XCTUnwrap(frame)
XCTAssertEqual(ready.ptsNs, 42_000_000)
XCTAssertFalse(ready.isHDR)
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), 320)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), 180)
XCTAssertEqual(
CVPixelBufferGetPixelFormatType(ready.pixelBuffer),
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, "SDR AV1 must decode to NV12")
decoder.reset()
}
// MARK: - Blobs
/// Keyframe temporal unit (740 bytes).
static let keyframeTU: [UInt8] = [
0x12, 0x00, 0x0a, 0x0b, 0x00, 0x00, 0x00, 0x04, 0x3c, 0xfe, 0xcc, 0x4a, 0xf9, 0x00, 0x40, 0x32,
0xd2, 0x05, 0x10, 0x00, 0x9b, 0xa0, 0x8f, 0xbe, 0x7d, 0xf0, 0xdf, 0xbc, 0xf8, 0x00, 0xdd, 0x6d,
0x69, 0x7f, 0xb3, 0x26, 0x63, 0x5e, 0x79, 0xf4, 0xf5, 0xc5, 0x84, 0xda, 0xcf, 0xec, 0xd8, 0xa6,
0xc1, 0x56, 0x99, 0x43, 0xf0, 0xff, 0x31, 0xd5, 0x41, 0xd8, 0xbb, 0x07, 0x10, 0x5e, 0x42, 0xc5,
0x9b, 0x63, 0xc0, 0x14, 0xe7, 0x28, 0x73, 0xf3, 0x50, 0x12, 0x02, 0x8e, 0x2b, 0x91, 0xd7, 0x3d,
0xc5, 0x33, 0xc9, 0x9b, 0xd9, 0xea, 0xfb, 0xc3, 0x5a, 0xdd, 0xb3, 0x0b, 0x5d, 0xc6, 0xde, 0x1a,
0xca, 0x90, 0x61, 0x0a, 0x77, 0x83, 0xb5, 0x8f, 0x9a, 0x88, 0xf0, 0x3d, 0xa2, 0x78, 0x81, 0x6c,
0xb6, 0x8e, 0x85, 0x90, 0x44, 0xa1, 0xda, 0xa8, 0xf6, 0xcb, 0xa2, 0xf2, 0xb8, 0xb9, 0xac, 0x6b,
0xba, 0xfd, 0x8f, 0x7e, 0x04, 0x32, 0x0d, 0x90, 0xed, 0x5a, 0xe2, 0x1d, 0x8a, 0x03, 0x29, 0x47,
0xde, 0xf6, 0x9a, 0xd4, 0x45, 0x91, 0x17, 0x5c, 0x7b, 0xdf, 0xa3, 0xe2, 0x7a, 0xd9, 0x93, 0xfd,
0x55, 0xeb, 0xd9, 0x3f, 0xa6, 0xec, 0xdd, 0x2a, 0x00, 0xbc, 0x8d, 0x1d, 0x98, 0xa2, 0x39, 0x88,
0x3f, 0x32, 0x3e, 0x18, 0x85, 0x12, 0x46, 0x57, 0x1d, 0x25, 0x8e, 0xcc, 0x41, 0x37, 0xc3, 0x9f,
0xbd, 0x7b, 0xf9, 0xb9, 0xa4, 0x9d, 0x86, 0xf4, 0xcc, 0x2b, 0x5a, 0xf3, 0xbc, 0x77, 0x80, 0xcb,
0xc4, 0xf3, 0x02, 0x91, 0xf6, 0xb8, 0x59, 0x93, 0x33, 0xbe, 0xe2, 0x23, 0xac, 0xb9, 0xe2, 0x69,
0x67, 0xad, 0x63, 0x45, 0x35, 0x94, 0x9e, 0x2e, 0xfa, 0x2b, 0xb9, 0xc3, 0xf9, 0x39, 0x5e, 0xa8,
0x33, 0x4d, 0xf3, 0x5c, 0xbd, 0xe6, 0x5b, 0x50, 0x19, 0xe6, 0xd3, 0xf2, 0x01, 0xcf, 0x35, 0x09,
0xd6, 0x2a, 0x67, 0x17, 0xd2, 0xbd, 0x91, 0xc3, 0x91, 0x17, 0x4a, 0xbc, 0x29, 0xf0, 0xb8, 0xd4,
0xfc, 0x04, 0xac, 0x63, 0xfb, 0x2f, 0xc5, 0xe9, 0xb2, 0x06, 0xac, 0x3c, 0x79, 0x33, 0x5c, 0x73,
0x80, 0x95, 0x0f, 0xad, 0xff, 0xee, 0xed, 0x78, 0xaf, 0xc6, 0x1b, 0xb4, 0xc2, 0x96, 0x5f, 0x7f,
0x20, 0x5f, 0xb6, 0xdb, 0x70, 0xab, 0x60, 0x0b, 0xea, 0xd1, 0xaf, 0x57, 0x71, 0xeb, 0x3b, 0xef,
0xb1, 0x3c, 0x01, 0x72, 0x5b, 0x59, 0x6d, 0x36, 0xe3, 0x16, 0xda, 0x0a, 0x6b, 0xc7, 0x0b, 0xa0,
0xa6, 0x6f, 0x77, 0x4f, 0x0b, 0xe6, 0x62, 0x34, 0x0e, 0xdd, 0xfa, 0xbe, 0x4f, 0x67, 0x21, 0x40,
0xc2, 0xcd, 0xf3, 0x63, 0x9e, 0xb2, 0x28, 0xaf, 0x5b, 0x0e, 0x28, 0xba, 0x91, 0xec, 0xec, 0xf2,
0xf4, 0xdb, 0x9c, 0x51, 0x24, 0x67, 0x77, 0xe0, 0x67, 0x70, 0x51, 0xfb, 0x00, 0x61, 0x50, 0x9f,
0xae, 0x5e, 0x96, 0x2d, 0x1e, 0xa2, 0xab, 0x49, 0x90, 0x90, 0x1c, 0x04, 0x93, 0x8b, 0xc2, 0xee,
0x53, 0x61, 0x62, 0x19, 0x62, 0x5c, 0xff, 0x15, 0x84, 0x7a, 0x5c, 0x70, 0xaa, 0x6d, 0x39, 0xb2,
0xe9, 0x19, 0xd1, 0x9f, 0xf3, 0xb3, 0xb7, 0x05, 0xd2, 0xef, 0x5f, 0xe9, 0x2a, 0x25, 0x55, 0x0a,
0xf3, 0xd2, 0x95, 0xba, 0x22, 0x5f, 0x49, 0x9b, 0x5d, 0xae, 0xeb, 0x51, 0x63, 0x51, 0xa0, 0x85,
0xd6, 0xb6, 0x23, 0x6f, 0x92, 0xbf, 0x99, 0xf3, 0xf9, 0xbf, 0x07, 0xd4, 0x05, 0x1c, 0x6b, 0xe1,
0x42, 0x49, 0xfe, 0x99, 0x4c, 0x6f, 0x34, 0xea, 0x29, 0x14, 0xd5, 0x92, 0x17, 0xfa, 0xc4, 0x35,
0xef, 0x97, 0x31, 0x06, 0xdc, 0xc7, 0x57, 0xe7, 0x79, 0x3d, 0x64, 0xf3, 0x12, 0x0d, 0x60, 0x5d,
0x9d, 0xa5, 0x11, 0xb9, 0xf9, 0x75, 0x3a, 0x59, 0x42, 0x2a, 0xd0, 0xed, 0xb4, 0xa8, 0xee, 0x87,
0x9e, 0x3d, 0x52, 0x47, 0x6c, 0x8f, 0x43, 0x98, 0xd0, 0x72, 0x8b, 0x16, 0xef, 0x2a, 0xaa, 0x9c,
0x42, 0x91, 0xc8, 0x92, 0x85, 0x79, 0xc0, 0xf8, 0xc0, 0x12, 0x44, 0x38, 0x49, 0xdb, 0x5f, 0xfc,
0x7b, 0x6e, 0xac, 0xea, 0x38, 0x3c, 0x3f, 0x49, 0xc2, 0xe7, 0x82, 0xb3, 0x30, 0xf2, 0x7e, 0x31,
0xc2, 0xf8, 0xfc, 0x9a, 0x9e, 0x6d, 0x4c, 0x5f, 0xd4, 0xc0, 0x22, 0xd3, 0x40, 0xc9, 0x66, 0x59,
0x38, 0x14, 0x64, 0x24, 0xc0, 0x0d, 0x56, 0x88, 0x6d, 0x9f, 0x80, 0x71, 0xc8, 0x26, 0x3e, 0x2f,
0xd1, 0xd2, 0x6d, 0x8a, 0xf2, 0x2c, 0x01, 0xbf, 0x89, 0x15, 0xc7, 0x66, 0x3d, 0x19, 0x9f, 0xb8,
0x4c, 0xb9, 0x6f, 0xd7, 0xe8, 0x59, 0xf3, 0xe7, 0xdd, 0x14, 0x3e, 0x99, 0x37, 0x90, 0xb5, 0x2d,
0x49, 0xcc, 0x40, 0xd6, 0xe1, 0x29, 0x4e, 0x31, 0x7c, 0xef, 0xdb, 0x74, 0xf0, 0x9f, 0xa6, 0xdd,
0xbc, 0xd9, 0x65, 0x0f, 0xf7, 0x22, 0xa8, 0xd0, 0xfb, 0x78, 0x49, 0x40, 0xbf, 0x96, 0xa1, 0x5a,
0x7b, 0xc6, 0x60, 0x63, 0x22, 0xad, 0x5d, 0x97, 0xa2, 0x77, 0x3f, 0x58, 0x3b, 0x94, 0xa4, 0xd9,
0xe3, 0xe5, 0xae, 0x0c, 0x30, 0x91, 0xdd, 0x5a, 0xbb, 0xfd, 0x10, 0x6a, 0x22, 0x8f, 0xbd, 0x8c,
0x41, 0xf5, 0xa2, 0x29, 0xd7, 0xad, 0x4e, 0x58, 0xfb, 0x1e, 0x9a, 0x0e, 0x98, 0x54, 0xc1, 0xd7,
0xfb, 0xdf, 0xcc, 0x1d, 0x5d, 0xe3, 0x25, 0x6b, 0x57, 0x69, 0x67, 0x80, 0x0c, 0xb5, 0xcb, 0x01,
0x5a, 0x56, 0x56, 0x01, 0x47, 0xad, 0x6b, 0x26, 0x28, 0x30, 0x36, 0x79, 0x91, 0x62, 0x52, 0x93,
0xa4, 0xe8, 0x52, 0x30,
]
/// Delta temporal unit (27 bytes).
static let deltaTU: [UInt8] = [
0x12, 0x00, 0x32, 0x17, 0x30, 0x02, 0x04, 0x09, 0x24, 0x92, 0x22, 0x7f, 0x80, 0x00, 0x01, 0x9f,
0x00, 0x00, 0x00, 0x8b, 0x07, 0x27, 0x7a, 0x64, 0x4c, 0xec, 0xf4,
]
}