c56b1b455a
apple / swift (push) Successful in 1m17s
ci / rust (push) Failing after 31s
ci / web (push) Failing after 42s
ci / docs-site (push) Failing after 40s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
docker / deploy-docs (push) Has been skipped
rpm / build-publish (push) Failing after 15s
deb / build-publish (push) Failing after 43s
Fixes the intermittent first-connect freeze. The host streams infinite GOP — one opening IDR, then P-frames only (recovery keyframes just on loss) — so when the client's decoder wedges on the cold first session (a lost/corrupt opening IDR, a bad early P-frame) the picture stays frozen until the far-off next keyframe. The client had no way to ask for one; now it does. Add a RequestKeyframe control message (client -> host, reliable control stream), mirroring Reconfigure: - core: quic.rs RequestKeyframe (type 0x03) + roundtrip test; client.rs CtrlRequest::Keyframe + NativeClient::request_keyframe; abi.rs punktfunk_connection_request_keyframe (header regenerated). - host: m3.rs decodes it in the control loop and signals the encode loop, which coalesces a burst and calls enc.request_keyframe() — wiring the existing NvencEncoder hook (force_kf -> next frame pict_type=I), the same recovery the GameStream path already had via force_idr. - apple: PunktfunkConnection.requestKeyframe(); StreamPump (stage-1) requests on layer.status==.failed; Stage2Pipeline (stage-2) on a sync submit failure and on the async decode-error callback via a thread-safe KeyframeRecovery. All throttled to <=1/250ms (the decode stays wedged for several frames until the IDR lands, so per-frame requests would flood the control stream). Self-healing: a lost recovery IDR is re-requested after the throttle; the host coalesces bursts into a single IDR. Validated: cargo fmt + clippy clean; core + host test suites green (incl. new request_keyframe_roundtrip); swift build + test (39 passed); xcframework rebuilt (all 5 slices), header regenerated with no unrelated drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
92 lines
3.7 KiB
Swift
92 lines
3.7 KiB
Swift
// The platform-independent heart of the presenters: one thread pulling AUs from the
|
|
// connection into an AVSampleBufferDisplayLayer, with the format description refreshed
|
|
// on every IDR (the host opens with an IDR carrying in-band parameter sets; recovery
|
|
// keyframes re-send them — there is no out-of-band extradata, ever). Shared by the
|
|
// macOS StreamLayerView and the iOS/iPadOS stream view.
|
|
|
|
import AVFoundation
|
|
import Foundation
|
|
|
|
/// Cancellation handle owned by exactly one pump thread — a restart hands the old pump
|
|
/// its own token, so it can never be revived by a newer start().
|
|
private final class PumpToken: @unchecked Sendable {
|
|
private let lock = NSLock()
|
|
private var live = true
|
|
var isLive: Bool {
|
|
lock.lock()
|
|
defer { lock.unlock() }
|
|
return live
|
|
}
|
|
func cancel() {
|
|
lock.lock()
|
|
live = false
|
|
lock.unlock()
|
|
}
|
|
}
|
|
|
|
/// One pump per instance; create a fresh StreamPump per start (cancel is permanent).
|
|
final class StreamPump {
|
|
private let token = PumpToken()
|
|
|
|
/// Pump thread: pull AUs, wrap, enqueue. Non-IDR AUs before the first format
|
|
/// description are dropped. `onFrame`/`onSessionEnd` fire on the pump thread.
|
|
func start(
|
|
connection: PunktfunkConnection,
|
|
layer: AVSampleBufferDisplayLayer,
|
|
onFrame: (@Sendable (AccessUnit) -> Void)?,
|
|
onSessionEnd: (@Sendable () -> Void)?
|
|
) {
|
|
let token = token
|
|
layer.flush() // drop any frames a previous connection left queued
|
|
|
|
let thread = Thread {
|
|
var format: CMVideoFormatDescription?
|
|
var lastKeyframeRequest = Date.distantPast
|
|
while token.isLive {
|
|
do {
|
|
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
|
|
onFrame?(au)
|
|
if let f = AnnexB.formatDescription(fromIDR: au.data) {
|
|
format = f // refreshed on every IDR (mode changes included)
|
|
}
|
|
if layer.status == .failed {
|
|
// Decode wedged: flush and re-gate on the next in-band parameter sets
|
|
// (resuming with a delta frame can't recover), AND ask the host for a
|
|
// fresh IDR. With the host's infinite GOP the next keyframe could be
|
|
// far off, so without the request the picture stays frozen — the
|
|
// intermittent first-connect freeze. Throttled: the layer stays .failed
|
|
// across several polls until the IDR lands, and one request suffices.
|
|
layer.flush()
|
|
format = AnnexB.formatDescription(fromIDR: au.data)
|
|
let now = Date()
|
|
if now.timeIntervalSince(lastKeyframeRequest) > 0.25 {
|
|
connection.requestKeyframe()
|
|
lastKeyframeRequest = now
|
|
}
|
|
}
|
|
guard let f = format,
|
|
let sample = AnnexB.sampleBuffer(au: au, format: f),
|
|
token.isLive // don't enqueue a stale frame after a restart
|
|
else { continue }
|
|
layer.enqueue(sample)
|
|
} catch {
|
|
if token.isLive {
|
|
onSessionEnd?()
|
|
}
|
|
break // session closed
|
|
}
|
|
}
|
|
}
|
|
thread.name = "punktfunk-pump"
|
|
thread.qualityOfService = .userInteractive
|
|
thread.start()
|
|
}
|
|
|
|
/// Stop pumping (≤ one poll timeout). Does not close the connection.
|
|
func stop() {
|
|
token.cancel()
|
|
}
|
|
|
|
deinit { token.cancel() }
|
|
}
|