feat(clients/apple): AV1 decode support — OBU plumbing, hardware-gated advertisement
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 15m35s
apple / screenshots (push) Successful in 4m35s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m51s
ci / web (push) Successful in 1m5s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 5m14s
deb / build-publish (push) Successful in 3m31s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 5m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m20s
docker / deploy-docs (push) Successful in 6s
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 15m35s
apple / screenshots (push) Successful in 4m35s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m51s
ci / web (push) Successful in 1m5s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 5m14s
deb / build-publish (push) Successful in 3m31s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 5m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m20s
docker / deploy-docs (push) Successful in 6s
The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs exclusively, so AV1 was never advertised and the codec picker hid it. Add the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift): a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack (temporal delimiter stripped, everything size-fielded, one copy per AU). VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) — keyframe gating keys on the in-band sequence header exactly as the NAL codecs key on in-band parameter sets, so loss recovery and mid-session reconfigure work unchanged. AV1 sessions require a hardware decoder (VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and both the Hello advertisement and the Settings picker are gated on VTIsHardwareDecodeSupported — AV1 only appears on devices that can actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV). Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk, the parse against an independent reference, av1C bytes, delta-TU gating, repack byte-exactness, and — on AV1 hardware — a real VTDecompressionSession decode through VideoDecoder. Host precedence stays HEVC > AV1 > H.264, so AV1 engages only when explicitly picked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -6,8 +6,10 @@
|
||||
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
|
||||
// the codec the host resolved in the Welcome (`connection.videoCodec`) — HEVC and H.264
|
||||
// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
|
||||
// is not an Annex-B/NAL codec and isn't handled here — this client never advertises it in
|
||||
// the Hello, so a host never emits it at us.
|
||||
// is not an Annex-B/NAL codec and isn't handled here — its OBU flavor of the same plumbing
|
||||
// lives in AV1.swift, and the pumps reach both through `VideoCodec`'s dispatching
|
||||
// `formatDescription(fromKeyframe:)` / `sampleBuffer(au:format:)`, so nothing below is ever
|
||||
// called with `.av1`.
|
||||
//
|
||||
// HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
|
||||
// once per AU, so the conversion is built on `forEachNAL` — a zero-copy scan over the AU's bytes
|
||||
@@ -23,10 +25,15 @@ import Foundation
|
||||
public enum VideoCodec: Equatable {
|
||||
case h264
|
||||
case hevc
|
||||
case av1
|
||||
|
||||
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown → HEVC).
|
||||
public init(wire: UInt8) {
|
||||
self = wire == 0x01 ? .h264 : .hevc // 0x01 = PUNKTFUNK_CODEC_H264
|
||||
switch wire {
|
||||
case 0x01: self = .h264 // PUNKTFUNK_CODEC_H264
|
||||
case 0x04: self = .av1 // PUNKTFUNK_CODEC_AV1
|
||||
default: self = .hevc // PUNKTFUNK_CODEC_HEVC — the default / older-host codec
|
||||
}
|
||||
}
|
||||
|
||||
/// NAL unit type from a NAL's first byte. HEVC: bits 1..6; H.264: bits 0..4.
|
||||
@@ -140,6 +147,8 @@ public enum AnnexB {
|
||||
sets = [vps, sps, pps]
|
||||
case .h264:
|
||||
sets = [sps, pps]
|
||||
case .av1:
|
||||
return nil // OBU stream, no parameter-set NALs — handled in AV1.swift, never here
|
||||
}
|
||||
|
||||
var format: CMVideoFormatDescription?
|
||||
@@ -175,6 +184,8 @@ public enum AnnexB {
|
||||
parameterSetSizes: sizes,
|
||||
nalUnitHeaderLength: 4,
|
||||
formatDescriptionOut: &format)
|
||||
case .av1:
|
||||
break // unreachable — the .av1 arm above already returned
|
||||
}
|
||||
}
|
||||
return status == noErr ? format : nil
|
||||
|
||||
Reference in New Issue
Block a user