feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout)
Backgrounding a live session no longer freezes it when the user opts in: audio keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live, video decode is DROPPED, and a bounded timer auto-disconnects. Off by default. - PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT + PyroWave) drain nextAU() for flow control but DISCARD the AU before any VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen). - SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire (keeps host linger → fast late reconnect). exitBackground clears the flag and requestKeyframe()s; the pump's freeze gate auto-arms on the resumed frame-index gap so concealed frames are withheld until the IDR re-anchors. disconnect() cancels the timer + clears isBackgrounded. - ContentView scenePhase driver (iOS): .background+streaming+setting → enterBackground; .active → exitBackground. scenePhase (not willResignActive) so Control-Center/app-switcher peeks don't start the timer. - Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout; new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10). - Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3). macOS swift build + swift test green (142 tests). The iOS-gated scenePhase handler + settings section are not exercised by the macOS CI target (known §9 gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN, timeout ends the session, phone-call audio-steal degrades gracefully). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -180,6 +180,23 @@ public final class SessionAudio {
|
||||
}
|
||||
}
|
||||
|
||||
/// Background keep-alive: silence the mic uplink while backgrounded (privacy — no room audio
|
||||
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
|
||||
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
|
||||
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
|
||||
/// — this stops the actual capture, which is the privacy-relevant part. Main thread.
|
||||
public func setMicMuted(_ muted: Bool) {
|
||||
stateLock.lock()
|
||||
let capture = captureEngine
|
||||
stateLock.unlock()
|
||||
guard let capture else { return }
|
||||
if muted {
|
||||
capture.pause()
|
||||
} else if !flag.isStopped {
|
||||
try? capture.start()
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Playback (host → speaker)
|
||||
|
||||
private func startPlayback(speakerUID: String) {
|
||||
|
||||
@@ -534,6 +534,23 @@ public final class PunktfunkConnection {
|
||||
_ = punktfunk_connection_request_keyframe(h)
|
||||
}
|
||||
|
||||
/// Background-keep-alive video drop (opt-in). While true, both video pumps keep DRAINING
|
||||
/// `nextAU()` (so QUIC flow control and host pacing stay healthy) but DISCARD each AU before any
|
||||
/// VideoToolbox/Metal decode or render — the crash/jetsam-safe way to hold a backgrounded
|
||||
/// session (audio keeps rendering; no GPU work off-screen). Set on `SessionModel.enterBackground`,
|
||||
/// cleared on `exitBackground` (which then requests a fresh IDR; the pump's re-anchor gate
|
||||
/// auto-arms on the resumed frame-index gap). Its own tiny lock — read on the pump thread every
|
||||
/// iteration, written on the main actor; never contends the ABI/plane locks.
|
||||
private let videoDropLock = NSLock()
|
||||
private var videoDropped = false
|
||||
public var isVideoDropped: Bool {
|
||||
videoDropLock.lock(); defer { videoDropLock.unlock() }
|
||||
return videoDropped
|
||||
}
|
||||
public func setVideoDropped(_ dropped: Bool) {
|
||||
videoDropLock.lock(); videoDropped = dropped; videoDropLock.unlock()
|
||||
}
|
||||
|
||||
/// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a
|
||||
/// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap —
|
||||
/// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a
|
||||
|
||||
@@ -432,6 +432,15 @@ public final class Stage2Pipeline {
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Background keep-alive: drain one AU (flow control + host pacing) and discard it
|
||||
// BEFORE any VideoToolbox decode or Metal render — no GPU work off-screen. The
|
||||
// decoder session is left intact; exitBackground requests a fresh IDR and the
|
||||
// re-anchor gate arms on the resumed frame-index gap so concealed frames are
|
||||
// withheld until it lands.
|
||||
if connection.isVideoDropped {
|
||||
_ = try connection.nextAU(timeoutMs: 100)
|
||||
return true
|
||||
}
|
||||
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
|
||||
// decoder conceals the reference-missing deltas — often WITHOUT an error callback —
|
||||
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
|
||||
@@ -687,6 +696,13 @@ public final class Stage2Pipeline {
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Background keep-alive: drain + discard before the Metal wavelet decode
|
||||
// (PyroWave is all-intra, so the resumed frame heals on its own — no IDR
|
||||
// request needed, just no GPU work off-screen).
|
||||
if connection.isVideoDropped {
|
||||
_ = try connection.nextAU(timeoutMs: 100)
|
||||
return true
|
||||
}
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
|
||||
onFrame?(au)
|
||||
if let newest = newestIndex,
|
||||
|
||||
@@ -63,6 +63,14 @@ final class StreamPump {
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Background keep-alive: drain one AU to keep QUIC flow control + host pacing
|
||||
// healthy, then discard it BEFORE any decode/enqueue — no VideoToolbox/Metal work
|
||||
// off-screen. Skips all recovery/gate bookkeeping too; exitBackground requests a
|
||||
// fresh IDR and the re-anchor gate re-arms on the resumed frame-index gap.
|
||||
if connection.isVideoDropped {
|
||||
_ = try connection.nextAU(timeoutMs: 100)
|
||||
return true
|
||||
}
|
||||
// Loss recovery (the primary path). Under the host's infinite GOP the only
|
||||
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
|
||||
// (framesDropped); the decoder then *conceals* the reference-missing deltas — a
|
||||
|
||||
Reference in New Issue
Block a user