feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout)

Backgrounding a live session no longer freezes it when the user opts in: audio
keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live,
video decode is DROPPED, and a bounded timer auto-disconnects. Off by default.

- PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag
  both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT +
  PyroWave) drain nextAU() for flow control but DISCARD the AU before any
  VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen).
- SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop
  flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture
  engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire
  (keeps host linger → fast late reconnect). exitBackground clears the flag and
  requestKeyframe()s; the pump's freeze gate auto-arms on the resumed
  frame-index gap so concealed frames are withheld until the IDR re-anchors.
  disconnect() cancels the timer + clears isBackgrounded.
- ContentView scenePhase driver (iOS): .background+streaming+setting →
  enterBackground; .active → exitBackground. scenePhase (not willResignActive)
  so Control-Center/app-switcher peeks don't start the timer.
- Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout;
  new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10).
- Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3).

macOS swift build + swift test green (142 tests). The iOS-gated scenePhase
handler + settings section are not exercised by the macOS CI target (known §9
gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN,
timeout ends the session, phone-call audio-steal degrades gracefully).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-16 18:41:46 +02:00
parent 4cae1b8bb8
commit 14c5e7c11c
10 changed files with 187 additions and 0 deletions
+13
View File
@@ -19,6 +19,19 @@
<array> <array>
<string>_punktfunk._udp</string> <string>_punktfunk._udp</string>
</array> </array>
<!-- Background keep-alive (opt-in, iOS/iPadOS): the ONLY sanctioned way to keep the long-lived
QUIC socket + pump-thread set alive while backgrounded is the audio background mode, backed
by the session's real, audible remote audio (AVAudioEngine keeps rendering). Video decode is
dropped; a bounded timer auto-disconnects. Never silence-as-keepalive (App Review 2.5.4).
tvOS ignores/tolerates the key; macOS is not gated by it. -->
<key>UIBackgroundModes</key>
<array>
<string>audio</string>
</array>
<!-- Live Activities (iOS/iPadOS): the Lock-Screen / Dynamic-Island session surface. Updated
locally (pushType nil) from the alive app process — no aps-environment. tvOS/macOS ignore it. -->
<key>NSSupportsLiveActivities</key>
<true/>
<!-- Deep links: punktfunk://connect/<host-uuid>[?launch=<GameEntry.id>]. Emitted by the <!-- Deep links: punktfunk://connect/<host-uuid>[?launch=<GameEntry.id>]. Emitted by the
launcher widget and Siri/Shortcuts; routed by ContentView.onOpenURL into the existing launcher widget and Siri/Shortcuts; routed by ContentView.onOpenURL into the existing
connect path. Shared across all three targets (tvOS/macOS accept it harmlessly). --> connect path. Shared across all three targets (tvOS/macOS accept it harmlessly). -->
@@ -96,6 +96,14 @@ struct ContentView: View {
/// fires Wake-on-LAN up front and falls into the "Waking" wait if the dial fails. Off: connects /// fires Wake-on-LAN up front and falls into the "Waking" wait if the dial fails. Off: connects
/// go straight through with no wake. The explicit "Wake Host" action is unaffected either way. /// go straight through with no wake. The explicit "Wake Host" action is unaffected either way.
@AppStorage(DefaultsKey.autoWake) private var autoWakeEnabled = true @AppStorage(DefaultsKey.autoWake) private var autoWakeEnabled = true
/// Background keep-alive (Settings General, iOS-only). Default OFF (today's freeze-on-background
/// is the default). When on, backgrounding a live session keeps audio + the connection alive and
/// drops video, auto-disconnecting after `backgroundTimeoutMinutes`.
@AppStorage(DefaultsKey.backgroundKeepAlive) private var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) private var backgroundTimeoutMinutes = 10
/// scenePhase drives the keep-alive: use THIS, not the willResignActive observers resign-active
/// also fires for Control Center / app-switcher peeks, where the disconnect timer must not start.
@Environment(\.scenePhase) private var scenePhase
private var gamepadUIActive: Bool { private var gamepadUIActive: Bool {
GamepadUIEnvironment.isActive( GamepadUIEnvironment.isActive(
gamepadConnected: gamepadManager.active != nil, enabledSetting: gamepadUIEnabled) gamepadConnected: gamepadManager.active != nil, enabledSetting: gamepadUIEnabled)
@@ -122,6 +130,22 @@ struct ContentView: View {
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a // tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns. // parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) } .onOpenURL { handleDeepLink($0) }
#if os(iOS)
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
.onChange(of: scenePhase) { _, phase in
switch phase {
case .background:
if backgroundKeepAlive, model.phase == .streaming {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
}
case .active:
model.exitBackground()
default:
break
}
}
#endif
.onChange(of: model.phase) { _, phase in .onChange(of: model.phase) { _, phase in
switch phase { switch phase {
case .streaming: case .streaming:
@@ -148,6 +148,13 @@ final class SessionModel: ObservableObject {
var isBusy: Bool { phase != .idle } var isBusy: Bool { phase != .idle }
/// True while a streaming session is running in the background under the opt-in keep-alive
/// (audio plays, video dropped, timeout armed). Drives the Live Activity's stage/countdown (M3)
/// and is cleared on foreground or teardown. iOS/iPadOS only in practice.
@Published private(set) var isBackgrounded = false
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer?
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true /// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host /// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair` /// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -332,6 +339,46 @@ final class SessionModel: ObservableObject {
} }
} }
// MARK: - Background keep-alive (opt-in, iOS)
/// Enter the backgrounded keep-alive state: keep audio playing, DROP video decode (no GPU work
/// off-screen), mute the mic (privacy), and arm a bounded auto-disconnect. The caller
/// (ContentView's scenePhase driver) gates this on the setting + `.streaming`; a no-op otherwise.
/// The video-drop seam is read by both pumps every iteration (`connection.isVideoDropped`).
func enterBackground(timeoutMinutes: Int) {
guard phase == .streaming, let conn = connection, !isBackgrounded else { return }
isBackgrounded = true
conn.setVideoDropped(true)
audio?.setMicMuted(true)
// Non-deliberate on fire (keep the host linger) so a user who returns late reconnects fast,
// exactly like today's network-drop path. min 1 minute guards a nonsense setting.
let minutes = max(1, timeoutMinutes)
let timer = DispatchSource.makeTimerSource(queue: .main)
timer.schedule(deadline: .now() + .seconds(minutes * 60))
timer.setEventHandler { [weak self] in
// The timer fires on `.main`, so the actor's executor is the main thread here.
MainActor.assumeIsolated { self?.disconnect(deliberate: false) }
}
backgroundTimer?.cancel()
backgroundTimer = timer
timer.resume()
}
/// Return to foreground: cancel the timeout, resume mic + video, and force a clean re-anchor
/// request a fresh IDR (infinite GOP: it won't come on its own) and let the pump's freeze gate
/// withhold the concealed frames until it lands (it auto-arms on the resumed frame-index gap).
func exitBackground() {
guard isBackgrounded else { return }
isBackgrounded = false
backgroundTimer?.cancel()
backgroundTimer = nil
audio?.setMicMuted(false)
if let conn = connection {
conn.setVideoDropped(false)
conn.requestKeyframe()
}
}
/// The user confirmed the fingerprint: returns it for pinning and enters streaming. /// The user confirmed the fingerprint: returns it for pinning and enters streaming.
func confirmTrust() -> Data? { func confirmTrust() -> Data? {
guard case .awaitingTrust(let fingerprint) = phase else { return nil } guard case .awaitingTrust(let fingerprint) = phase else { return nil }
@@ -349,6 +396,10 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) { func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate() statsTimer?.invalidate()
statsTimer = nil statsTimer = nil
// Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel()
backgroundTimer = nil
isBackgrounded = false
let audio = self.audio let audio = self.audio
self.audio = nil self.audio = nil
// Gamepad capture is main-actor (releases held buttons on the wire while the // Gamepad capture is main-actor (releases held buttons on the wire while the
@@ -440,6 +440,34 @@ extension SettingsView {
} }
} }
/// iOS/iPadOS only: keep a backgrounded session alive (audio background mode). Empty elsewhere
/// (tvOS backgrounding semantics differ; macOS isn't gated by the mode) so the shared `.general`
/// detail can reference it unconditionally.
@ViewBuilder var keepAliveSection: some View {
#if os(iOS)
Section {
Toggle("Keep streaming in background", isOn: $backgroundKeepAlive)
if backgroundKeepAlive {
Picker("Disconnect after", selection: $backgroundTimeoutMinutes) {
Text("1 minute").tag(1)
Text("5 minutes").tag(5)
Text("10 minutes").tag(10)
Text("30 minutes").tag(30)
}
}
} header: {
Text("Background")
} footer: {
Text("Off by default: backgrounding the app freezes the session. When on, audio keeps "
+ "playing and the connection stays live (video is dropped to save power) after you "
+ "switch away — and the session auto-disconnects after the time above so it can't "
+ "run down your battery. Returning to the app resumes video instantly.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder var experimentalSection: some View { @ViewBuilder var experimentalSection: some View {
Section { Section {
Toggle("Show game library", isOn: $libraryEnabled) Toggle("Show game library", isOn: $libraryEnabled)
@@ -49,6 +49,8 @@ struct SettingsView: View {
@ObservedObject var gamepads = GamepadManager.shared @ObservedObject var gamepads = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true @AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true
@AppStorage(DefaultsKey.autoWake) var autoWakeEnabled = true @AppStorage(DefaultsKey.autoWake) var autoWakeEnabled = true
@AppStorage(DefaultsKey.backgroundKeepAlive) var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) var backgroundTimeoutMinutes = 10
#if DEBUG && !os(tvOS) #if DEBUG && !os(tvOS)
@State var showControllerTest = false @State var showControllerTest = false
#endif #endif
@@ -242,6 +244,7 @@ struct SettingsView: View {
pointerSection pointerSection
compositorSection compositorSection
wakeSection wakeSection
keepAliveSection // iOS-only content; empty on tvOS
} }
.formStyle(.grouped) .formStyle(.grouped)
.navigationTitle("General") .navigationTitle("General")
@@ -180,6 +180,23 @@ public final class SessionAudio {
} }
} }
/// Background keep-alive: silence the mic uplink while backgrounded (privacy no room audio
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
/// this stops the actual capture, which is the privacy-relevant part. Main thread.
public func setMicMuted(_ muted: Bool) {
stateLock.lock()
let capture = captureEngine
stateLock.unlock()
guard let capture else { return }
if muted {
capture.pause()
} else if !flag.isStopped {
try? capture.start()
}
}
// MARK: - Playback (host speaker) // MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) { private func startPlayback(speakerUID: String) {
@@ -534,6 +534,23 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_request_keyframe(h) _ = punktfunk_connection_request_keyframe(h)
} }
/// Background-keep-alive video drop (opt-in). While true, both video pumps keep DRAINING
/// `nextAU()` (so QUIC flow control and host pacing stay healthy) but DISCARD each AU before any
/// VideoToolbox/Metal decode or render the crash/jetsam-safe way to hold a backgrounded
/// session (audio keeps rendering; no GPU work off-screen). Set on `SessionModel.enterBackground`,
/// cleared on `exitBackground` (which then requests a fresh IDR; the pump's re-anchor gate
/// auto-arms on the resumed frame-index gap). Its own tiny lock read on the pump thread every
/// iteration, written on the main actor; never contends the ABI/plane locks.
private let videoDropLock = NSLock()
private var videoDropped = false
public var isVideoDropped: Bool {
videoDropLock.lock(); defer { videoDropLock.unlock() }
return videoDropped
}
public func setVideoDropped(_ dropped: Bool) {
videoDropLock.lock(); videoDropped = dropped; videoDropLock.unlock()
}
/// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a /// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a
/// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap /// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap
/// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a /// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a
@@ -432,6 +432,15 @@ public final class Stage2Pipeline {
while alive, !token.isStopped { while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in alive = autoreleasepool { () -> Bool in
do { do {
// Background keep-alive: drain one AU (flow control + host pacing) and discard it
// BEFORE any VideoToolbox decode or Metal render no GPU work off-screen. The
// decoder session is left intact; exitBackground requests a fresh IDR and the
// re-anchor gate arms on the resumed frame-index gap so concealed frames are
// withheld until it lands.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the // Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback // decoder conceals the reference-missing deltas often WITHOUT an error callback
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh // so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
@@ -687,6 +696,13 @@ public final class Stage2Pipeline {
while alive, !token.isStopped { while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in alive = autoreleasepool { () -> Bool in
do { do {
// Background keep-alive: drain + discard before the Metal wavelet decode
// (PyroWave is all-intra, so the resumed frame heals on its own no IDR
// request needed, just no GPU work off-screen).
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
guard let au = try connection.nextAU(timeoutMs: 100) else { return true } guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
onFrame?(au) onFrame?(au)
if let newest = newestIndex, if let newest = newestIndex,
@@ -63,6 +63,14 @@ final class StreamPump {
while alive, !token.isStopped { while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in alive = autoreleasepool { () -> Bool in
do { do {
// Background keep-alive: drain one AU to keep QUIC flow control + host pacing
// healthy, then discard it BEFORE any decode/enqueue no VideoToolbox/Metal work
// off-screen. Skips all recovery/gate bookkeeping too; exitBackground requests a
// fresh IDR and the re-anchor gate re-arms on the resumed frame-index gap.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). Under the host's infinite GOP the only // Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs // recovery keyframe is one we request. The reassembler drops unrecoverable AUs
// (framesDropped); the decoder then *conceals* the reference-missing deltas a // (framesDropped); the decoder then *conceals* the reference-missing deltas a
@@ -112,6 +112,16 @@ public enum DefaultsKey {
/// routed/VPN host), so connects go straight through instead of waiting out the wake timeout. /// routed/VPN host), so connects go straight through instead of waiting out the wake timeout.
/// The explicit "Wake Host" action stays available regardless. Read by ContentView.startSession. /// The explicit "Wake Host" action stays available regardless. Read by ContentView.startSession.
public static let autoWake = "punktfunk.autoWake" public static let autoWake = "punktfunk.autoWake"
/// iOS/iPadOS: keep a streaming session ALIVE when the app is backgrounded (audio background
/// mode). Off by default (today's freeze-on-background is the default). When on, backgrounding a
/// live session keeps audio playing and the QUIC/pump live while DROPPING video decode, and a
/// bounded timer (`backgroundTimeoutMinutes`) auto-disconnects if the user doesn't return. Read
/// by ContentView's scenePhase driver. Hidden on tvOS/macOS.
public static let backgroundKeepAlive = "punktfunk.backgroundKeepAlive"
/// iOS/iPadOS: minutes a backgrounded keep-alive session runs before auto-disconnecting (a
/// battery/thermal/bandwidth backstop). Default 10; the UI offers 1/5/10/30. The auto-disconnect
/// is non-deliberate (host linger kept), so a late return reconnects fast. Read on enterBackground.
public static let backgroundTimeoutMinutes = "punktfunk.backgroundTimeoutMinutes"
} }
extension Notification.Name { extension Notification.Name {