feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout)

Backgrounding a live session no longer freezes it when the user opts in: audio
keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live,
video decode is DROPPED, and a bounded timer auto-disconnects. Off by default.

- PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag
  both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT +
  PyroWave) drain nextAU() for flow control but DISCARD the AU before any
  VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen).
- SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop
  flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture
  engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire
  (keeps host linger → fast late reconnect). exitBackground clears the flag and
  requestKeyframe()s; the pump's freeze gate auto-arms on the resumed
  frame-index gap so concealed frames are withheld until the IDR re-anchors.
  disconnect() cancels the timer + clears isBackgrounded.
- ContentView scenePhase driver (iOS): .background+streaming+setting →
  enterBackground; .active → exitBackground. scenePhase (not willResignActive)
  so Control-Center/app-switcher peeks don't start the timer.
- Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout;
  new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10).
- Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3).

macOS swift build + swift test green (142 tests). The iOS-gated scenePhase
handler + settings section are not exercised by the macOS CI target (known §9
gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN,
timeout ends the session, phone-call audio-steal degrades gracefully).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-16 18:41:46 +02:00
parent 4cae1b8bb8
commit 14c5e7c11c
10 changed files with 187 additions and 0 deletions
@@ -180,6 +180,23 @@ public final class SessionAudio {
}
}
/// Background keep-alive: silence the mic uplink while backgrounded (privacy no room audio
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
/// this stops the actual capture, which is the privacy-relevant part. Main thread.
public func setMicMuted(_ muted: Bool) {
stateLock.lock()
let capture = captureEngine
stateLock.unlock()
guard let capture else { return }
if muted {
capture.pause()
} else if !flag.isStopped {
try? capture.start()
}
}
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
@@ -534,6 +534,23 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_request_keyframe(h)
}
/// Background-keep-alive video drop (opt-in). While true, both video pumps keep DRAINING
/// `nextAU()` (so QUIC flow control and host pacing stay healthy) but DISCARD each AU before any
/// VideoToolbox/Metal decode or render the crash/jetsam-safe way to hold a backgrounded
/// session (audio keeps rendering; no GPU work off-screen). Set on `SessionModel.enterBackground`,
/// cleared on `exitBackground` (which then requests a fresh IDR; the pump's re-anchor gate
/// auto-arms on the resumed frame-index gap). Its own tiny lock read on the pump thread every
/// iteration, written on the main actor; never contends the ABI/plane locks.
private let videoDropLock = NSLock()
private var videoDropped = false
public var isVideoDropped: Bool {
videoDropLock.lock(); defer { videoDropLock.unlock() }
return videoDropped
}
public func setVideoDropped(_ dropped: Bool) {
videoDropLock.lock(); videoDropped = dropped; videoDropLock.unlock()
}
/// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a
/// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap
/// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a
@@ -432,6 +432,15 @@ public final class Stage2Pipeline {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain one AU (flow control + host pacing) and discard it
// BEFORE any VideoToolbox decode or Metal render no GPU work off-screen. The
// decoder session is left intact; exitBackground requests a fresh IDR and the
// re-anchor gate arms on the resumed frame-index gap so concealed frames are
// withheld until it lands.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
@@ -687,6 +696,13 @@ public final class Stage2Pipeline {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain + discard before the Metal wavelet decode
// (PyroWave is all-intra, so the resumed frame heals on its own no IDR
// request needed, just no GPU work off-screen).
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
onFrame?(au)
if let newest = newestIndex,
@@ -63,6 +63,14 @@ final class StreamPump {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain one AU to keep QUIC flow control + host pacing
// healthy, then discard it BEFORE any decode/enqueue no VideoToolbox/Metal work
// off-screen. Skips all recovery/gate bookkeeping too; exitBackground requests a
// fresh IDR and the re-anchor gate re-arms on the resumed frame-index gap.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
// (framesDropped); the decoder then *conceals* the reference-missing deltas a