feat(clients/apple): AV1 decode support — OBU plumbing, hardware-gated advertisement
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 15m35s
apple / screenshots (push) Successful in 4m35s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 5m51s
ci / web (push) Successful in 1m5s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 5m14s
deb / build-publish (push) Successful in 3m31s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 5m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m20s
docker / deploy-docs (push) Successful in 6s

The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs
exclusively, so AV1 was never advertised and the codec picker hid it. Add
the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift):
a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C
CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and
the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack
(temporal delimiter stripped, everything size-fielded, one copy per AU).

VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route
through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) —
keyframe gating keys on the in-band sequence header exactly as the NAL
codecs key on in-band parameter sets, so loss recovery and mid-session
reconfigure work unchanged. AV1 sessions require a hardware decoder
(VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and
both the Hello advertisement and the Settings picker are gated on
VTIsHardwareDecodeSupported — AV1 only appears on devices that can
actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV).

Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk,
the parse against an independent reference, av1C bytes, delta-TU gating,
repack byte-exactness, and — on AV1 hardware — a real
VTDecompressionSession decode through VideoDecoder. Host precedence stays
HEVC > AV1 > H.264, so AV1 engages only when explicitly picked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-09 01:22:49 +02:00
parent bf9be59f0b
commit c3fa6c1514
10 changed files with 927 additions and 34 deletions
@@ -71,7 +71,7 @@ final class StreamPump {
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
let idrFormat = AnnexB.formatDescription(fromIDR: au.data, codec: connection.videoCodec)
let idrFormat = connection.videoCodec.formatDescription(fromKeyframe: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
@@ -95,7 +95,7 @@ final class StreamPump {
}
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f, codec: connection.videoCodec),
let sample = connection.videoCodec.sampleBuffer(au: au, format: f),
!token.isStopped // don't enqueue a stale frame after a restart
else { continue }
layer.enqueue(sample)