feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout)
Backgrounding a live session no longer freezes it when the user opts in: audio keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live, video decode is DROPPED, and a bounded timer auto-disconnects. Off by default. - PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT + PyroWave) drain nextAU() for flow control but DISCARD the AU before any VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen). - SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire (keeps host linger → fast late reconnect). exitBackground clears the flag and requestKeyframe()s; the pump's freeze gate auto-arms on the resumed frame-index gap so concealed frames are withheld until the IDR re-anchors. disconnect() cancels the timer + clears isBackgrounded. - ContentView scenePhase driver (iOS): .background+streaming+setting → enterBackground; .active → exitBackground. scenePhase (not willResignActive) so Control-Center/app-switcher peeks don't start the timer. - Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout; new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10). - Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3). macOS swift build + swift test green (142 tests). The iOS-gated scenePhase handler + settings section are not exercised by the macOS CI target (known §9 gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN, timeout ends the session, phone-call audio-steal degrades gracefully). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -19,6 +19,19 @@
|
||||
<array>
|
||||
<string>_punktfunk._udp</string>
|
||||
</array>
|
||||
<!-- Background keep-alive (opt-in, iOS/iPadOS): the ONLY sanctioned way to keep the long-lived
|
||||
QUIC socket + pump-thread set alive while backgrounded is the audio background mode, backed
|
||||
by the session's real, audible remote audio (AVAudioEngine keeps rendering). Video decode is
|
||||
dropped; a bounded timer auto-disconnects. Never silence-as-keepalive (App Review 2.5.4).
|
||||
tvOS ignores/tolerates the key; macOS is not gated by it. -->
|
||||
<key>UIBackgroundModes</key>
|
||||
<array>
|
||||
<string>audio</string>
|
||||
</array>
|
||||
<!-- Live Activities (iOS/iPadOS): the Lock-Screen / Dynamic-Island session surface. Updated
|
||||
locally (pushType nil) from the alive app process — no aps-environment. tvOS/macOS ignore it. -->
|
||||
<key>NSSupportsLiveActivities</key>
|
||||
<true/>
|
||||
<!-- Deep links: punktfunk://connect/<host-uuid>[?launch=<GameEntry.id>]. Emitted by the
|
||||
launcher widget and Siri/Shortcuts; routed by ContentView.onOpenURL into the existing
|
||||
connect path. Shared across all three targets (tvOS/macOS accept it harmlessly). -->
|
||||
|
||||
@@ -96,6 +96,14 @@ struct ContentView: View {
|
||||
/// fires Wake-on-LAN up front and falls into the "Waking…" wait if the dial fails. Off: connects
|
||||
/// go straight through with no wake. The explicit "Wake Host" action is unaffected either way.
|
||||
@AppStorage(DefaultsKey.autoWake) private var autoWakeEnabled = true
|
||||
/// Background keep-alive (Settings → General, iOS-only). Default OFF (today's freeze-on-background
|
||||
/// is the default). When on, backgrounding a live session keeps audio + the connection alive and
|
||||
/// drops video, auto-disconnecting after `backgroundTimeoutMinutes`.
|
||||
@AppStorage(DefaultsKey.backgroundKeepAlive) private var backgroundKeepAlive = false
|
||||
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) private var backgroundTimeoutMinutes = 10
|
||||
/// scenePhase drives the keep-alive: use THIS, not the willResignActive observers — resign-active
|
||||
/// also fires for Control Center / app-switcher peeks, where the disconnect timer must not start.
|
||||
@Environment(\.scenePhase) private var scenePhase
|
||||
private var gamepadUIActive: Bool {
|
||||
GamepadUIEnvironment.isActive(
|
||||
gamepadConnected: gamepadManager.active != nil, enabledSetting: gamepadUIEnabled)
|
||||
@@ -122,6 +130,22 @@ struct ContentView: View {
|
||||
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
|
||||
// parallel session — this drives the one `model` ContentView owns.
|
||||
.onOpenURL { handleDeepLink($0) }
|
||||
#if os(iOS)
|
||||
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
|
||||
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
|
||||
.onChange(of: scenePhase) { _, phase in
|
||||
switch phase {
|
||||
case .background:
|
||||
if backgroundKeepAlive, model.phase == .streaming {
|
||||
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
|
||||
}
|
||||
case .active:
|
||||
model.exitBackground()
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
#endif
|
||||
.onChange(of: model.phase) { _, phase in
|
||||
switch phase {
|
||||
case .streaming:
|
||||
|
||||
@@ -148,6 +148,13 @@ final class SessionModel: ObservableObject {
|
||||
|
||||
var isBusy: Bool { phase != .idle }
|
||||
|
||||
/// True while a streaming session is running in the background under the opt-in keep-alive
|
||||
/// (audio plays, video dropped, timeout armed). Drives the Live Activity's stage/countdown (M3)
|
||||
/// and is cleared on foreground or teardown. iOS/iPadOS only in practice.
|
||||
@Published private(set) var isBackgrounded = false
|
||||
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
|
||||
private var backgroundTimer: DispatchSourceTimer?
|
||||
|
||||
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
|
||||
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
|
||||
/// — `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
|
||||
@@ -332,6 +339,46 @@ final class SessionModel: ObservableObject {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Background keep-alive (opt-in, iOS)
|
||||
|
||||
/// Enter the backgrounded keep-alive state: keep audio playing, DROP video decode (no GPU work
|
||||
/// off-screen), mute the mic (privacy), and arm a bounded auto-disconnect. The caller
|
||||
/// (ContentView's scenePhase driver) gates this on the setting + `.streaming`; a no-op otherwise.
|
||||
/// The video-drop seam is read by both pumps every iteration (`connection.isVideoDropped`).
|
||||
func enterBackground(timeoutMinutes: Int) {
|
||||
guard phase == .streaming, let conn = connection, !isBackgrounded else { return }
|
||||
isBackgrounded = true
|
||||
conn.setVideoDropped(true)
|
||||
audio?.setMicMuted(true)
|
||||
// Non-deliberate on fire (keep the host linger) so a user who returns late reconnects fast,
|
||||
// exactly like today's network-drop path. min 1 minute guards a nonsense setting.
|
||||
let minutes = max(1, timeoutMinutes)
|
||||
let timer = DispatchSource.makeTimerSource(queue: .main)
|
||||
timer.schedule(deadline: .now() + .seconds(minutes * 60))
|
||||
timer.setEventHandler { [weak self] in
|
||||
// The timer fires on `.main`, so the actor's executor is the main thread here.
|
||||
MainActor.assumeIsolated { self?.disconnect(deliberate: false) }
|
||||
}
|
||||
backgroundTimer?.cancel()
|
||||
backgroundTimer = timer
|
||||
timer.resume()
|
||||
}
|
||||
|
||||
/// Return to foreground: cancel the timeout, resume mic + video, and force a clean re-anchor —
|
||||
/// request a fresh IDR (infinite GOP: it won't come on its own) and let the pump's freeze gate
|
||||
/// withhold the concealed frames until it lands (it auto-arms on the resumed frame-index gap).
|
||||
func exitBackground() {
|
||||
guard isBackgrounded else { return }
|
||||
isBackgrounded = false
|
||||
backgroundTimer?.cancel()
|
||||
backgroundTimer = nil
|
||||
audio?.setMicMuted(false)
|
||||
if let conn = connection {
|
||||
conn.setVideoDropped(false)
|
||||
conn.requestKeyframe()
|
||||
}
|
||||
}
|
||||
|
||||
/// The user confirmed the fingerprint: returns it for pinning and enters streaming.
|
||||
func confirmTrust() -> Data? {
|
||||
guard case .awaitingTrust(let fingerprint) = phase else { return nil }
|
||||
@@ -349,6 +396,10 @@ final class SessionModel: ObservableObject {
|
||||
func disconnect(deliberate: Bool = true) {
|
||||
statsTimer?.invalidate()
|
||||
statsTimer = nil
|
||||
// Drop any armed background keep-alive (incl. the timeout that just fired us).
|
||||
backgroundTimer?.cancel()
|
||||
backgroundTimer = nil
|
||||
isBackgrounded = false
|
||||
let audio = self.audio
|
||||
self.audio = nil
|
||||
// Gamepad capture is main-actor (releases held buttons on the wire while the
|
||||
|
||||
@@ -440,6 +440,34 @@ extension SettingsView {
|
||||
}
|
||||
}
|
||||
|
||||
/// iOS/iPadOS only: keep a backgrounded session alive (audio background mode). Empty elsewhere
|
||||
/// (tvOS backgrounding semantics differ; macOS isn't gated by the mode) so the shared `.general`
|
||||
/// detail can reference it unconditionally.
|
||||
@ViewBuilder var keepAliveSection: some View {
|
||||
#if os(iOS)
|
||||
Section {
|
||||
Toggle("Keep streaming in background", isOn: $backgroundKeepAlive)
|
||||
if backgroundKeepAlive {
|
||||
Picker("Disconnect after", selection: $backgroundTimeoutMinutes) {
|
||||
Text("1 minute").tag(1)
|
||||
Text("5 minutes").tag(5)
|
||||
Text("10 minutes").tag(10)
|
||||
Text("30 minutes").tag(30)
|
||||
}
|
||||
}
|
||||
} header: {
|
||||
Text("Background")
|
||||
} footer: {
|
||||
Text("Off by default: backgrounding the app freezes the session. When on, audio keeps "
|
||||
+ "playing and the connection stays live (video is dropped to save power) after you "
|
||||
+ "switch away — and the session auto-disconnects after the time above so it can't "
|
||||
+ "run down your battery. Returning to the app resumes video instantly.")
|
||||
.font(.geist(12, relativeTo: .caption))
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
@ViewBuilder var experimentalSection: some View {
|
||||
Section {
|
||||
Toggle("Show game library", isOn: $libraryEnabled)
|
||||
|
||||
@@ -49,6 +49,8 @@ struct SettingsView: View {
|
||||
@ObservedObject var gamepads = GamepadManager.shared
|
||||
@AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true
|
||||
@AppStorage(DefaultsKey.autoWake) var autoWakeEnabled = true
|
||||
@AppStorage(DefaultsKey.backgroundKeepAlive) var backgroundKeepAlive = false
|
||||
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) var backgroundTimeoutMinutes = 10
|
||||
#if DEBUG && !os(tvOS)
|
||||
@State var showControllerTest = false
|
||||
#endif
|
||||
@@ -242,6 +244,7 @@ struct SettingsView: View {
|
||||
pointerSection
|
||||
compositorSection
|
||||
wakeSection
|
||||
keepAliveSection // iOS-only content; empty on tvOS
|
||||
}
|
||||
.formStyle(.grouped)
|
||||
.navigationTitle("General")
|
||||
|
||||
@@ -180,6 +180,23 @@ public final class SessionAudio {
|
||||
}
|
||||
}
|
||||
|
||||
/// Background keep-alive: silence the mic uplink while backgrounded (privacy — no room audio
|
||||
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
|
||||
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
|
||||
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
|
||||
/// — this stops the actual capture, which is the privacy-relevant part. Main thread.
|
||||
public func setMicMuted(_ muted: Bool) {
|
||||
stateLock.lock()
|
||||
let capture = captureEngine
|
||||
stateLock.unlock()
|
||||
guard let capture else { return }
|
||||
if muted {
|
||||
capture.pause()
|
||||
} else if !flag.isStopped {
|
||||
try? capture.start()
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Playback (host → speaker)
|
||||
|
||||
private func startPlayback(speakerUID: String) {
|
||||
|
||||
@@ -534,6 +534,23 @@ public final class PunktfunkConnection {
|
||||
_ = punktfunk_connection_request_keyframe(h)
|
||||
}
|
||||
|
||||
/// Background-keep-alive video drop (opt-in). While true, both video pumps keep DRAINING
|
||||
/// `nextAU()` (so QUIC flow control and host pacing stay healthy) but DISCARD each AU before any
|
||||
/// VideoToolbox/Metal decode or render — the crash/jetsam-safe way to hold a backgrounded
|
||||
/// session (audio keeps rendering; no GPU work off-screen). Set on `SessionModel.enterBackground`,
|
||||
/// cleared on `exitBackground` (which then requests a fresh IDR; the pump's re-anchor gate
|
||||
/// auto-arms on the resumed frame-index gap). Its own tiny lock — read on the pump thread every
|
||||
/// iteration, written on the main actor; never contends the ABI/plane locks.
|
||||
private let videoDropLock = NSLock()
|
||||
private var videoDropped = false
|
||||
public var isVideoDropped: Bool {
|
||||
videoDropLock.lock(); defer { videoDropLock.unlock() }
|
||||
return videoDropped
|
||||
}
|
||||
public func setVideoDropped(_ dropped: Bool) {
|
||||
videoDropLock.lock(); videoDropped = dropped; videoDropLock.unlock()
|
||||
}
|
||||
|
||||
/// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a
|
||||
/// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap —
|
||||
/// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a
|
||||
|
||||
@@ -432,6 +432,15 @@ public final class Stage2Pipeline {
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Background keep-alive: drain one AU (flow control + host pacing) and discard it
|
||||
// BEFORE any VideoToolbox decode or Metal render — no GPU work off-screen. The
|
||||
// decoder session is left intact; exitBackground requests a fresh IDR and the
|
||||
// re-anchor gate arms on the resumed frame-index gap so concealed frames are
|
||||
// withheld until it lands.
|
||||
if connection.isVideoDropped {
|
||||
_ = try connection.nextAU(timeoutMs: 100)
|
||||
return true
|
||||
}
|
||||
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
|
||||
// decoder conceals the reference-missing deltas — often WITHOUT an error callback —
|
||||
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
|
||||
@@ -687,6 +696,13 @@ public final class Stage2Pipeline {
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Background keep-alive: drain + discard before the Metal wavelet decode
|
||||
// (PyroWave is all-intra, so the resumed frame heals on its own — no IDR
|
||||
// request needed, just no GPU work off-screen).
|
||||
if connection.isVideoDropped {
|
||||
_ = try connection.nextAU(timeoutMs: 100)
|
||||
return true
|
||||
}
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
|
||||
onFrame?(au)
|
||||
if let newest = newestIndex,
|
||||
|
||||
@@ -63,6 +63,14 @@ final class StreamPump {
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Background keep-alive: drain one AU to keep QUIC flow control + host pacing
|
||||
// healthy, then discard it BEFORE any decode/enqueue — no VideoToolbox/Metal work
|
||||
// off-screen. Skips all recovery/gate bookkeeping too; exitBackground requests a
|
||||
// fresh IDR and the re-anchor gate re-arms on the resumed frame-index gap.
|
||||
if connection.isVideoDropped {
|
||||
_ = try connection.nextAU(timeoutMs: 100)
|
||||
return true
|
||||
}
|
||||
// Loss recovery (the primary path). Under the host's infinite GOP the only
|
||||
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
|
||||
// (framesDropped); the decoder then *conceals* the reference-missing deltas — a
|
||||
|
||||
@@ -112,6 +112,16 @@ public enum DefaultsKey {
|
||||
/// routed/VPN host), so connects go straight through instead of waiting out the wake timeout.
|
||||
/// The explicit "Wake Host" action stays available regardless. Read by ContentView.startSession.
|
||||
public static let autoWake = "punktfunk.autoWake"
|
||||
/// iOS/iPadOS: keep a streaming session ALIVE when the app is backgrounded (audio background
|
||||
/// mode). Off by default (today's freeze-on-background is the default). When on, backgrounding a
|
||||
/// live session keeps audio playing and the QUIC/pump live while DROPPING video decode, and a
|
||||
/// bounded timer (`backgroundTimeoutMinutes`) auto-disconnects if the user doesn't return. Read
|
||||
/// by ContentView's scenePhase driver. Hidden on tvOS/macOS.
|
||||
public static let backgroundKeepAlive = "punktfunk.backgroundKeepAlive"
|
||||
/// iOS/iPadOS: minutes a backgrounded keep-alive session runs before auto-disconnecting (a
|
||||
/// battery/thermal/bandwidth backstop). Default 10; the UI offers 1/5/10/30. The auto-disconnect
|
||||
/// is non-deliberate (host linger kept), so a late return reconnects fast. Read on enterBackground.
|
||||
public static let backgroundTimeoutMinutes = "punktfunk.backgroundTimeoutMinutes"
|
||||
}
|
||||
|
||||
extension Notification.Name {
|
||||
|
||||
Reference in New Issue
Block a user