fix(apple/iOS): immersive streaming — edge-to-edge, no status bar, hidden cursor, native default mode
ci / rust (push) Has been cancelled

Streaming on iPad left the status bar up and the video boxed inside the safe areas, on
top of a 16:9 default mode letterboxing on the 4:3 screen, with the iPadOS cursor
hovering over the video. The session view is now immersive on iOS:

- .ignoresSafeArea + .statusBarHidden + .persistentSystemOverlays(.hidden) for the
  session only (home gets its chrome back on disconnect).
- First run seeds the stream mode from the device's native screen
  (UIScreen.nativeBounds + maximumFramesPerSecond) instead of 1920×1080 — verified
  live: a fresh install negotiated the iPad's 2752×2064 with the host. macOS keeps the
  1080p default (a desktop window is not the screen).
- The iPadOS cursor hides while over the video (UIPointerInteraction .hidden(),
  re-resolved on capture toggles) — the host renders its own cursor from our deltas;
  true pointer lock through UIHostingController remains the documented gap.

Found along the way (host-side, not fixed here): at very high modes a keyframe burst
can fill the UDP send buffer and m3 treats the sendmmsg WouldBlock as fatal
("session ended with error: submit_frame: WouldBlock") instead of backpressuring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-11 12:44:29 +02:00
parent 57f7e32c24
commit 154da2dc58
3 changed files with 51 additions and 8 deletions
+8 -5
View File
@@ -188,11 +188,14 @@ signing, bundle id `io.unom.punktfunk`. Notes:
pickers are macOS-only). For the iPad-with-external-display setup: the target
enables multiple scenes + indirect input events — on Stage Manager iPads, drag the
punktfunk window onto the external screen and the stream runs there with full
keyboard/mouse/touch. Known gaps: `prefersPointerLocked` is declared on the stream
view controller but UIHostingController doesn't forward the preference from
representable children, so the system cursor stays visible (relative-mouse
forwarding works regardless — fixing it means putting the controller into the UIKit
presentation chain, e.g. a full-screen UIKit presentation on session start); and
keyboard/mouse/touch. While streaming the session is immersive (edge-to-edge,
status bar + home indicator hidden) and the iPadOS cursor is hidden over the video
(`UIPointerInteraction` `.hidden()` — visible again when ⌘⎋ releases capture); on
iOS first run the stream mode defaults to the device's native screen so the video
fills the display. Known gaps: true pointer LOCK (`prefersPointerLocked`) isn't
consulted through UIHostingController, so the hidden cursor can still drift onto a
second screen (fixing it means putting the controller into the UIKit presentation
chain); and
AVAudioSession interruptions (calls, Siri) don't auto-restart the audio engines yet
(reconnect recovers).
@@ -40,7 +40,10 @@ struct ContentView: View {
home
}
}
.onAppear { autoConnectIfAsked() }
.onAppear {
seedDefaultModeIfNeeded()
autoConnectIfAsked()
}
.onDisappear { model.disconnect() } // window closed mid-session (Cmd+N spawns more)
// On the outer Group so the sheet survives the trust-prompt home transition
// (the "Pair with PIN instead" path disconnects first the host's accept loop
@@ -77,8 +80,15 @@ struct ContentView: View {
}
#if os(macOS)
.frame(minWidth: 640, minHeight: 360)
#endif
.background(Color.black)
#else
// Streaming is immersive: edge-to-edge under the status bar and home
// indicator, both hidden for the session (they return with the hosts grid).
.background(Color.black)
.ignoresSafeArea()
.statusBarHidden(true)
.persistentSystemOverlays(.hidden)
#endif
}
// MARK: - Home (hosts grid)
@@ -243,6 +253,21 @@ struct ContentView: View {
}
}
/// First run on iOS: default the stream mode to this device's native screen so the
/// video fills the display instead of letterboxing 1920×1080 onto a 4:3 iPad. (The
/// compiled-in AppStorage defaults only apply until any value is saved; macOS keeps
/// 1080p a desktop window is not the screen.)
private func seedDefaultModeIfNeeded() {
#if os(iOS)
let defaults = UserDefaults.standard
guard defaults.object(forKey: "punktfunk.width") == nil else { return }
let bounds = UIScreen.main.nativeBounds // portrait-oriented pixels
defaults.set(Int(max(bounds.width, bounds.height)), forKey: "punktfunk.width")
defaults.set(Int(min(bounds.width, bounds.height)), forKey: "punktfunk.height")
defaults.set(UIScreen.main.maximumFramesPerSecond, forKey: "punktfunk.hz")
#endif
}
private func connect(_ host: StoredHost) {
model.connect(
to: host,
@@ -66,12 +66,13 @@ public struct StreamView: UIViewControllerRepresentable {
}
}
public final class StreamViewController: UIViewController {
public final class StreamViewController: UIViewController, UIPointerInteractionDelegate {
public private(set) var connection: PunktfunkConnection?
private var pump: StreamPump?
private var inputCapture: InputCapture?
private var captured = false
private var observers: [NSObjectProtocol] = []
private var pointerInteraction: UIPointerInteraction?
var onCaptureChange: ((Bool) -> Void)?
@@ -89,6 +90,19 @@ public final class StreamViewController: UIViewController {
public override func loadView() {
view = StreamLayerUIView()
// Hide the iPadOS cursor while it hovers the video: the host renders its own
// cursor from our raw deltas, so the local one only diverges from it. (True
// pointer LOCK prefersPointerLocked isn't consulted through
// UIHostingController; this hides the pointer without locking it.)
let interaction = UIPointerInteraction(delegate: self)
view.addInteraction(interaction)
pointerInteraction = interaction
}
public func pointerInteraction(
_ interaction: UIPointerInteraction, styleFor region: UIPointerRegion
) -> UIPointerStyle? {
captured ? .hidden() : nil
}
public override var prefersPointerLocked: Bool { captured }
@@ -168,6 +182,7 @@ public final class StreamViewController: UIViewController {
captured = false
}
setNeedsUpdateOfPrefersPointerLocked()
pointerInteraction?.invalidate() // re-resolve the hidden/visible pointer style
let onCaptureChange = onCaptureChange
let captured = captured
DispatchQueue.main.async { onCaptureChange?(captured) }