feat(apple): stage-2 default + pixel-perfect, decode robustness, UI/rumble polish
apple / swift (push) Successful in 1m4s
android / android (push) Successful in 4m33s
ci / rust (push) Successful in 5m4s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 3m12s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
release / apple (push) Successful in 8m30s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m48s
apple / screenshots (push) Successful in 5m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
apple / swift (push) Successful in 1m4s
android / android (push) Successful in 4m33s
ci / rust (push) Successful in 5m4s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 3m12s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
release / apple (push) Successful in 8m30s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m48s
apple / screenshots (push) Successful in 5m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
Stream reliability - Default to the stage-2 presenter (VTDecompressionSession + CAMetalLayer): it detects and recovers a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference frame with no app-side recovery (confirmed Apple limitation). Stage 1 is now a DEBUG-only presenter toggle, plus the automatic no-Metal fallback. - Stage-2 pixel-perfect: render the drawable at the decoded size (shader stays 1:1 = identity) and let the layer's contentsGravity scale via the system compositor — the same path stage-1's videoGravity used — instead of scaling in-shader. - Loss recovery in both pumps is now a persistent awaitingIDR want, retried until an IDR actually lands, so a keyframe request swallowed by the throttle can't strand a frozen frame; 100 ms keyframe throttle to match the Android path. - Fix "Publishing changes from within view updates": defer the HostStore writes out of the .onChange(of: model.phase) callback. - Move AVAudioSession setActive/setCategory off the main thread (async on a shared serial queue) to stop the UI-stall warning. Controllers - Rumble: capped-exponential backoff when the gamecontrollerd.haptics XPC breaks (-4811) so a transient server interruption self-heals instead of cascading; playsHapticsOnly so a controller engine doesn't join the always-active streaming audio session. - Host cards: iPad pointer "magnet" hover effect; iPhone press scale + light haptic. UI / design - Ship Geist (SIL OFL 1.1) as the app font (bundled OTFs + registration), with the license surfaced in Acknowledgements. - Restructure iOS/iPadOS Settings into a category NavigationSplitView; resolution wheel with custom-resolution entry; 10-bit HDR toggle in Display. - Industrial host-card redesign (left-aligned, bold, brand monogram tiles). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -6,6 +6,9 @@
|
||||
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
import os
|
||||
|
||||
private let pumpLog = Logger(subsystem: "io.unom.punktfunk", category: "video")
|
||||
|
||||
/// Cancellation handle owned by exactly one pump thread — a restart hands the old pump
|
||||
/// its own token, so it can never be revived by a newer start().
|
||||
@@ -47,44 +50,74 @@ final class StreamPump {
|
||||
var format: CMVideoFormatDescription?
|
||||
var lastKeyframeRequest = Date.distantPast
|
||||
var lastFramesDropped = connection.framesDropped()
|
||||
// Coalesced host keyframe request: the decode stays wedged for several frames until
|
||||
// the IDR lands, so requesting on every frame would flood the control stream.
|
||||
// Recovery is a persistent WANT, not a one-shot edge: set it on detected loss (or a
|
||||
// decoder reset), retry the throttled request EVERY iteration, and clear it only when a
|
||||
// fresh IDR actually re-anchors decode. The old code advanced `lastFramesDropped` on the
|
||||
// same edge it fired the throttled request — so a request swallowed by the throttle (a
|
||||
// second drop within the window, e.g. the lost recovery IDR itself being pruned) was
|
||||
// never re-sent: the counter went flat, the climb never re-fired, and the picture stayed
|
||||
// frozen for good while audio kept playing. The iPhone's lossy Wi-Fi hits this where the
|
||||
// Mac's Ethernet never does.
|
||||
var awaitingIDR = false
|
||||
var awaitingSince = Date.distantPast // when the current recovery began (for the resume log)
|
||||
var wasFailed = false
|
||||
// Coalesced host keyframe request. 100 ms throttle (matches the working Android path):
|
||||
// fast enough that a lost recovery IDR is re-requested promptly, bounded so a sustained
|
||||
// freeze can't flood the control stream.
|
||||
func requestKeyframeThrottled() {
|
||||
let now = Date()
|
||||
if now.timeIntervalSince(lastKeyframeRequest) > 0.25 {
|
||||
if now.timeIntervalSince(lastKeyframeRequest) > 0.1 {
|
||||
connection.requestKeyframe()
|
||||
lastKeyframeRequest = now
|
||||
}
|
||||
}
|
||||
while token.isLive {
|
||||
do {
|
||||
// Loss recovery (the primary recovery path). Under the host's infinite GOP the
|
||||
// only recovery keyframe is one we request. The reassembler drops unrecoverable
|
||||
// AUs (framesDropped); the decoder then *conceals* the reference-missing delta
|
||||
// frames that follow — a frozen / garbage picture, WITHOUT flipping the layer to
|
||||
// .failed — so the .failed check below rarely fires after a real network blip.
|
||||
// Ask the host for a fresh IDR whenever the drop count climbs. Polled every
|
||||
// iteration (not just per AU) so a total-loss drought still recovers the moment
|
||||
// packets resume and the reassembler counts the gap.
|
||||
// Loss recovery (the primary path). Under the host's infinite GOP the only
|
||||
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
|
||||
// (framesDropped); the decoder then *conceals* the reference-missing deltas — a
|
||||
// frozen / garbage picture that never flips the layer to .failed — so key off the
|
||||
// drop count climbing, then keep asking (awaitingIDR) until an IDR lands. Polled
|
||||
// every iteration so a total-loss drought still recovers when packets resume.
|
||||
let dropped = connection.framesDropped()
|
||||
if dropped > lastFramesDropped {
|
||||
// Log only on the false→true transition (once per recovery cycle), not per
|
||||
// dropped AU, so heavy loss doesn't spam the log.
|
||||
if !awaitingIDR {
|
||||
awaitingSince = Date()
|
||||
pumpLog.notice(
|
||||
"video: unrecoverable drop (framesDropped=\(dropped, privacy: .public)) — requesting recovery IDR")
|
||||
}
|
||||
lastFramesDropped = dropped
|
||||
requestKeyframeThrottled()
|
||||
awaitingIDR = true
|
||||
}
|
||||
if awaitingIDR { requestKeyframeThrottled() }
|
||||
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
|
||||
onFrame?(au)
|
||||
if let f = AnnexB.formatDescription(fromIDR: au.data) {
|
||||
format = f // refreshed on every IDR (mode changes included)
|
||||
let idrFormat = AnnexB.formatDescription(fromIDR: au.data)
|
||||
if let f = idrFormat {
|
||||
format = f // refreshed on every IDR (mode changes included)
|
||||
if awaitingIDR {
|
||||
let ms = Int(Date().timeIntervalSince(awaitingSince) * 1000)
|
||||
pumpLog.notice("video: recovery IDR received — resumed after \(ms, privacy: .public) ms")
|
||||
}
|
||||
awaitingIDR = false // a fresh IDR re-anchored decode — recovery complete
|
||||
}
|
||||
if layer.status == .failed {
|
||||
let failed = layer.status == .failed
|
||||
if failed {
|
||||
// Decode wedged hard (the cold-first-connect case — a lost/corrupt opening
|
||||
// IDR): flush and re-gate on the next in-band parameter sets (resuming with
|
||||
// a delta frame can't recover), AND ask the host for a fresh IDR. Throttled:
|
||||
// the layer stays .failed across several polls until the IDR lands.
|
||||
// IDR): flush and, unless THIS AU is the recovering IDR (re-anchored above),
|
||||
// re-gate on the next in-band parameter sets and keep asking — enqueuing a
|
||||
// delta into a failed layer can't recover it.
|
||||
if !wasFailed { pumpLog.warning("video: display layer .failed — flushing + re-anchoring") }
|
||||
layer.flush()
|
||||
format = AnnexB.formatDescription(fromIDR: au.data)
|
||||
requestKeyframeThrottled()
|
||||
if idrFormat == nil {
|
||||
format = nil
|
||||
awaitingIDR = true
|
||||
}
|
||||
}
|
||||
wasFailed = failed
|
||||
guard let f = format,
|
||||
let sample = AnnexB.sampleBuffer(au: au, format: f),
|
||||
token.isLive // don't enqueue a stale frame after a restart
|
||||
|
||||
Reference in New Issue
Block a user