feat(apple): the gamepad UI comes to tvOS - focus-driven, with real session controls
ci / rust (push) Failing after 50s
ci / web (push) Successful in 57s
decky / build-publish (push) Successful in 14s
ci / docs-site (push) Successful in 1m2s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
flatpak / build-publish (push) Failing after 2m14s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m46s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 11m40s
arch / build-publish (push) Successful in 18m10s
deb / build-publish (push) Successful in 18m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m21s
android / android (push) Successful in 21m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 5m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 1m2s
release / apple (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / rust (push) Failing after 50s
ci / web (push) Successful in 57s
decky / build-publish (push) Successful in 14s
ci / docs-site (push) Successful in 1m2s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
flatpak / build-publish (push) Failing after 2m14s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m46s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 11m40s
arch / build-publish (push) Successful in 18m10s
deb / build-publish (push) Successful in 18m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m21s
android / android (push) Successful in 21m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 5m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 1m2s
release / apple (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
The console UI now runs on tvOS through the NATIVE focus engine: carousel cards and settings rows are focusable Buttons (Siri Remote and pads both navigate; imperative scrollTo replaces the drop-prone scrollPosition binding), while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what focus has no concept of: X/Y screen actions and left/right value adjust with the poll's dominant-axis feel (onMoveCommand proved input-source-dependent: keyboard intercepted, pad dpad not -> double steps). Text entry uses the system fullscreen keyboard (TVTextEntry); pairing + library present as covers under the launcher; the game library defaults ON; settings values slide a quiet 14 pt in the step's direction. Session controls: controller/remote input routes EXCLUSIVELY through GameController during a stream (GCEventViewController, interaction disabled) - a pad's B no longer doubles as a UIKit menu press that ended sessions mid-game. Deliberate exits only: the cross-client escape chord (hold L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream banner (now also on tvOS) teaches both. The Siri Remote's touch surface drives the host pointer - press = left click, Play/Pause = right click, release-tail jumps gated so motion stays truly relative. tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root font shrank the whole platform (29 pt there), plus 10-foot sizing across host cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu" hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops instead of suspending; full-width focusSections make the home actions reachable from any column. The presenter defaults to stage-3 glass pacing on tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind the 50 ms display stage) and is pickable from the gamepad settings; HDR capability advertises from AVPlayer.eligibleForHDRPlayback instead of the current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -263,19 +263,24 @@ public final class SessionAudio {
|
||||
defer { drainDone.signal() }
|
||||
// Decode happens IN-CORE (libopus multistream) — AudioToolbox's Opus path is
|
||||
// stereo-only — and is handed back as interleaved f32 PCM in wire channel order.
|
||||
while !flag.isStopped {
|
||||
// Per-iteration autorelease pool: no runloop on this thread (see Stage2Pipeline).
|
||||
var alive = true
|
||||
while alive, !flag.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
let pcm: PunktfunkConnection.AudioPCM?
|
||||
do {
|
||||
pcm = try connection.nextAudioPcm(timeoutMs: 100)
|
||||
} catch {
|
||||
break // session closed
|
||||
return false // session closed
|
||||
}
|
||||
guard let pcm, pcm.frameCount > 0 else { continue }
|
||||
guard let pcm, pcm.frameCount > 0 else { return true }
|
||||
pcm.samples.withUnsafeBufferPointer { p in
|
||||
if let base = p.baseAddress {
|
||||
ring.write(base, count: pcm.frameCount * pcm.channels)
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
thread.name = "punktfunk-audio"
|
||||
|
||||
@@ -48,6 +48,23 @@ public final class GamepadCapture {
|
||||
/// Motion forwarding floor: ≥ 4 ms between samples (≈ 250 Hz, the DualSense's own rate).
|
||||
private static let motionIntervalNs: UInt64 = 4_000_000
|
||||
|
||||
/// The cross-client controller escape chord (pf-client-core's `ESCAPE_CHORD`):
|
||||
/// L1+R1+Start+Select held together — four simultaneous buttons no game uses, so normal
|
||||
/// play can't trip it. Held for `disconnectHold` it ends the session via
|
||||
/// `onDisconnectRequest`; the chord keeps forwarding to the host meanwhile (the user is
|
||||
/// leaving anyway). The desktop clients' quick-press step (leave fullscreen / release
|
||||
/// capture) has no Apple equivalent worth wiring — macOS has ⌃⌥⇧Q/D, touch has the HUD.
|
||||
private static let escapeChord: UInt32 =
|
||||
GamepadWire.leftShoulder | GamepadWire.rightShoulder | GamepadWire.start | GamepadWire.back
|
||||
/// pf-client-core's `DISCONNECT_HOLD` — the same 1.5 s on every client.
|
||||
private static let disconnectHold: TimeInterval = 1.5
|
||||
private var chordTimer: Timer?
|
||||
/// Fired ON MAIN once the escape chord has been held `disconnectHold` — the session owner
|
||||
/// disconnects. On tvOS this (plus the Siri Remote's hold-Back) is the ONLY way out of a
|
||||
/// stream with a controller: B/Menu presses are deliberately swallowed during a session so
|
||||
/// gameplay can't end it (see ContentView's tvOS session branch).
|
||||
public var onDisconnectRequest: (() -> Void)?
|
||||
|
||||
public init(connection: PunktfunkConnection, manager: GamepadManager) {
|
||||
self.connection = connection
|
||||
self.manager = manager
|
||||
@@ -165,6 +182,7 @@ public final class GamepadCapture {
|
||||
private func sync(_ g: GCExtendedGamepad) {
|
||||
guard !suspended else { return }
|
||||
let newButtons = Self.buttonMask(g)
|
||||
updateEscapeChord(newButtons)
|
||||
let changed = newButtons ^ buttons
|
||||
if changed != 0 {
|
||||
for bit in GamepadWire.allButtons where changed & bit != 0 {
|
||||
@@ -297,7 +315,26 @@ public final class GamepadCapture {
|
||||
|
||||
/// Unwind everything held on the wire: button-ups, neutral axes, lifted fingers. The
|
||||
/// host's virtual pad returns to rest instead of running with the last state.
|
||||
/// Arm the disconnect timer when the full chord lands, disarm the moment any of the four
|
||||
/// releases. Events only arrive on state CHANGES, so a held chord needs the timer — the
|
||||
/// handler won't fire again until something moves.
|
||||
private func updateEscapeChord(_ newButtons: UInt32) {
|
||||
let held = newButtons & Self.escapeChord == Self.escapeChord
|
||||
if held, chordTimer == nil {
|
||||
let timer = Timer(timeInterval: Self.disconnectHold, repeats: false) { [weak self] _ in
|
||||
Task { @MainActor in self?.onDisconnectRequest?() }
|
||||
}
|
||||
RunLoop.main.add(timer, forMode: .common)
|
||||
chordTimer = timer
|
||||
} else if !held, chordTimer != nil {
|
||||
chordTimer?.invalidate()
|
||||
chordTimer = nil
|
||||
}
|
||||
}
|
||||
|
||||
private func releaseAll() {
|
||||
chordTimer?.invalidate()
|
||||
chordTimer = nil
|
||||
for bit in GamepadWire.allButtons where buttons & bit != 0 {
|
||||
connection.send(.gamepadButton(bit, down: false, pad: 0))
|
||||
}
|
||||
|
||||
@@ -74,7 +74,11 @@ public final class GamepadFeedback {
|
||||
// session — a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and
|
||||
// let rumble own the wait elsewhere; on an Xbox session it stays nonblocking.
|
||||
let thread = Thread { [connection, flag, drainDone, weak self] in
|
||||
while !flag.isStopped {
|
||||
// Per-iteration autorelease pool: no runloop on this thread, and the haptics/HID
|
||||
// rendering below autoreleases ObjC temporaries. `false` = session over.
|
||||
var alive = true
|
||||
while alive, !flag.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Poll the feedback planes NON-BLOCKING. A blocking poll (timeoutMs > 0) holds
|
||||
// the connection's shared feedback lock for its whole wait; the video pump drains
|
||||
@@ -106,12 +110,14 @@ public final class GamepadFeedback {
|
||||
self?.render(ev)
|
||||
burst += 1
|
||||
}
|
||||
return true
|
||||
} catch {
|
||||
break // .closed (or fatal) — the session is over
|
||||
return false // .closed (or fatal) — the session is over
|
||||
}
|
||||
}
|
||||
// ~8 ms poll cadence (≈125 Hz), slept OUTSIDE the feedback lock — low rumble/HID
|
||||
// latency without holding the lock the HDR-meta drain needs.
|
||||
if !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
|
||||
if alive, !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
|
||||
}
|
||||
drainDone.signal()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
// The Siri Remote as a pointing device during a tvOS streaming session — the remote's touch
|
||||
// surface drives the HOST cursor (relative deltas, like a laptop trackpad), a surface press
|
||||
// clicks (left button), and Play/Pause right-clicks. It also owns the remote's DELIBERATE
|
||||
// session exit: hold Back/Menu ≥ `disconnectHold`. A short Back press does nothing — the
|
||||
// UIKit menu press it also generates is swallowed by ContentView's session branch, so neither
|
||||
// a trackpad fumble nor a game-controller B press can end the session (the pad's exit is the
|
||||
// L1+R1+Start+Select chord in GamepadCapture).
|
||||
//
|
||||
// The remote is read through GameController as a GCMicroGamepad with
|
||||
// `reportsAbsoluteDpadValues = true`: the dpad axes then report the finger's ABSOLUTE position
|
||||
// on the surface (±1, +y up) while touched, and snap to exactly (0, 0) on lift. Successive
|
||||
// positions are differenced into relative mouse deltas; the exact-zero snap is treated as a
|
||||
// lift (a real touch at the mathematical centre is measure-zero, and one dropped delta there
|
||||
// is imperceptible). Handlers (not a poll) — the same in-session delivery GamepadCapture
|
||||
// relies on.
|
||||
//
|
||||
// Lifecycle mirrors GamepadCapture: started by SessionModel when streaming begins (never
|
||||
// during the trust prompt), stopped on disconnect; held buttons are released on stop so the
|
||||
// host never keeps a stuck click.
|
||||
|
||||
#if os(tvOS)
|
||||
import Foundation
|
||||
import GameController
|
||||
import UIKit
|
||||
|
||||
@MainActor
|
||||
public final class SiriRemotePointer {
|
||||
private let connection: PunktfunkConnection
|
||||
private var observers: [NSObjectProtocol] = []
|
||||
private var bound: GCController?
|
||||
/// Finger position (±1 axes) at the last dpad callback while touched; nil = lifted.
|
||||
private var lastTouch: (x: Float, y: Float)?
|
||||
/// Wire buttons currently held (1 = left, 3 = right) — released on stop/unbind.
|
||||
private var heldButtons: Set<UInt32> = []
|
||||
/// When Back/Menu went down; a release after `disconnectHold` fires the exit.
|
||||
private var menuDownAt: Date?
|
||||
|
||||
/// Hold Back/Menu at least this long (then release) to end the session. Shorter than the
|
||||
/// controller chord's 1.5 s — the remote has no way to trip this during gameplay.
|
||||
private static let disconnectHold: TimeInterval = 1.0
|
||||
/// A full edge-to-edge swipe moves the host cursor about this many pixels. The surface is
|
||||
/// small; two comfortable swipes should cross a 1080p desktop.
|
||||
private static let pointerScale: Float = 1100
|
||||
/// Largest single-callback finger travel accepted as real motion (surface units; the axes
|
||||
/// span ±1, so 0.4 ≈ a fifth of the pad). On RELEASE the hardware slides the reported
|
||||
/// position back to (0, 0) through intermediate callbacks — naive differencing turns that
|
||||
/// tail into reverse deltas that RETRACE the whole swipe, so the cursor springs back to its
|
||||
/// anchor and the pointer feels absolute. Real finger motion arrives as many small steps
|
||||
/// (even a fast flick stays well under this per callback); the release tail arrives as one
|
||||
/// or two huge jumps — discard those (the anchor still follows, so nothing accumulates).
|
||||
private static let maxStep: Float = 0.4
|
||||
|
||||
/// Fired ON MAIN after Back/Menu was held ≥ `disconnectHold` and released.
|
||||
public var onDisconnectRequest: (() -> Void)?
|
||||
|
||||
public init(connection: PunktfunkConnection) {
|
||||
self.connection = connection
|
||||
}
|
||||
|
||||
public func start() {
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
forName: .GCControllerDidConnect, object: nil, queue: .main
|
||||
) { [weak self] _ in
|
||||
MainActor.assumeIsolated { self?.rebind() }
|
||||
})
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
forName: .GCControllerDidDisconnect, object: nil, queue: .main
|
||||
) { [weak self] _ in
|
||||
MainActor.assumeIsolated { self?.rebind() }
|
||||
})
|
||||
rebind()
|
||||
}
|
||||
|
||||
public func stop() {
|
||||
observers.forEach(NotificationCenter.default.removeObserver(_:))
|
||||
observers.removeAll()
|
||||
bind(nil)
|
||||
}
|
||||
|
||||
/// The Siri Remote is the non-extended controller carrying a microGamepad — a full gamepad
|
||||
/// (which also EXPOSES a microGamepad view of itself) must never be captured here, its
|
||||
/// buttons belong to GamepadCapture.
|
||||
private func rebind() {
|
||||
let remote = GCController.controllers().first {
|
||||
$0.extendedGamepad == nil && $0.microGamepad != nil
|
||||
}
|
||||
bind(remote)
|
||||
}
|
||||
|
||||
private func bind(_ controller: GCController?) {
|
||||
guard controller !== bound else { return }
|
||||
if let old = bound?.microGamepad {
|
||||
old.dpad.valueChangedHandler = nil
|
||||
old.buttonA.pressedChangedHandler = nil
|
||||
old.buttonX.pressedChangedHandler = nil
|
||||
old.buttonMenu.pressedChangedHandler = nil
|
||||
}
|
||||
releaseHeld()
|
||||
lastTouch = nil
|
||||
menuDownAt = nil
|
||||
bound = controller
|
||||
guard let micro = controller?.microGamepad else { return }
|
||||
|
||||
// Absolute finger position instead of the emulated dpad — the raw surface is what a
|
||||
// trackpad needs. Rotation stays off: the remote's natural grip is the coordinate frame.
|
||||
micro.reportsAbsoluteDpadValues = true
|
||||
micro.allowsRotation = false
|
||||
|
||||
micro.dpad.valueChangedHandler = { [weak self] _, x, y in
|
||||
MainActor.assumeIsolated { self?.touchMoved(x: x, y: y) }
|
||||
}
|
||||
// Surface click = left button; Play/Pause = right (the remote's only spare face button).
|
||||
micro.buttonA.pressedChangedHandler = { [weak self] _, _, pressed in
|
||||
MainActor.assumeIsolated { self?.setButton(1, down: pressed) }
|
||||
}
|
||||
micro.buttonX.pressedChangedHandler = { [weak self] _, _, pressed in
|
||||
MainActor.assumeIsolated { self?.setButton(3, down: pressed) }
|
||||
}
|
||||
micro.buttonMenu.pressedChangedHandler = { [weak self] _, _, pressed in
|
||||
MainActor.assumeIsolated { self?.menuChanged(pressed: pressed) }
|
||||
}
|
||||
}
|
||||
|
||||
private func touchMoved(x: Float, y: Float) {
|
||||
// Exact (0, 0) is the lift snap — drop the anchor so the next touch starts a fresh
|
||||
// gesture instead of a jump-delta from the old position.
|
||||
guard x != 0 || y != 0 else {
|
||||
lastTouch = nil
|
||||
return
|
||||
}
|
||||
defer { lastTouch = (x, y) }
|
||||
guard let last = lastTouch else { return } // first contact anchors, moves nothing
|
||||
let stepX = x - last.x
|
||||
let stepY = y - last.y
|
||||
// The release tail (and any tracking glitch) shows up as a single impossible jump —
|
||||
// see `maxStep`. Skip the emission; the deferred anchor update above still follows the
|
||||
// reported position, so the gesture cleanly re-anchors instead of retracing.
|
||||
guard abs(stepX) < Self.maxStep, abs(stepY) < Self.maxStep else { return }
|
||||
let dx = stepX * Self.pointerScale / 2 // axes span ±1 → full swipe = 2.0
|
||||
let dy = -stepY * Self.pointerScale / 2 // GC +y is up; mouse +y is down
|
||||
let ix = Int32(dx.rounded())
|
||||
let iy = Int32(dy.rounded())
|
||||
guard ix != 0 || iy != 0 else { return }
|
||||
connection.send(.mouseMove(dx: ix, dy: iy))
|
||||
}
|
||||
|
||||
private func setButton(_ button: UInt32, down: Bool) {
|
||||
if down { heldButtons.insert(button) } else { heldButtons.remove(button) }
|
||||
connection.send(.mouseButton(button, down: down))
|
||||
}
|
||||
|
||||
private func menuChanged(pressed: Bool) {
|
||||
if pressed {
|
||||
menuDownAt = Date()
|
||||
return
|
||||
}
|
||||
let heldFor = menuDownAt.map { Date().timeIntervalSince($0) } ?? 0
|
||||
menuDownAt = nil
|
||||
if heldFor >= Self.disconnectHold {
|
||||
onDisconnectRequest?()
|
||||
}
|
||||
// A short press is deliberately nothing: the accompanying UIKit menu press is swallowed
|
||||
// in ContentView, and forwarding it as a host key would make trackpad fumbles type.
|
||||
}
|
||||
|
||||
private func releaseHeld() {
|
||||
for button in heldButtons {
|
||||
connection.send(.mouseButton(button, down: false))
|
||||
}
|
||||
heldButtons.removeAll()
|
||||
}
|
||||
}
|
||||
#endif
|
||||
@@ -13,14 +13,14 @@ public enum Licenses {
|
||||
return text
|
||||
}
|
||||
|
||||
/// punktfunk's own license — MIT OR Apache-2.0, at your option.
|
||||
/// Punktfunk's own license — MIT OR Apache-2.0, at your option.
|
||||
public static var appLicense: String {
|
||||
let mit = resource("LICENSE-MIT")
|
||||
let apache = resource("LICENSE-APACHE")
|
||||
if mit.isEmpty && apache.isEmpty {
|
||||
return "punktfunk is licensed under MIT OR Apache-2.0, at your option."
|
||||
return "Punktfunk is licensed under MIT OR Apache-2.0, at your option."
|
||||
}
|
||||
return "punktfunk is licensed under MIT OR Apache-2.0, at your option.\n\n"
|
||||
return "Punktfunk is licensed under MIT OR Apache-2.0, at your option.\n\n"
|
||||
+ "================================ MIT ================================\n\n"
|
||||
+ mit
|
||||
+ "\n\n============================== Apache-2.0 ==============================\n\n"
|
||||
@@ -51,11 +51,27 @@ public enum Licenses {
|
||||
/// Acknowledgements screen renders these chunks in a `LazyVStack` (only on-screen chunks lay
|
||||
/// out, and no chunk is tall enough to clip). Split at line boundaries and joined with "\n";
|
||||
/// the inter-chunk break is the `LazyVStack` row boundary, so no text is lost. Computed once.
|
||||
public static let thirdPartyNoticesChunks: [String] = {
|
||||
let lines = thirdPartyNotices.split(separator: "\n", omittingEmptySubsequences: false)
|
||||
let chunkSize = 200
|
||||
return stride(from: 0, to: lines.count, by: chunkSize).map { start in
|
||||
lines[start..<min(start + chunkSize, lines.count)].joined(separator: "\n")
|
||||
public static let thirdPartyNoticesChunks: [String] = chunked(thirdPartyNotices)
|
||||
|
||||
/// Lines per chunk: tvOS reads much smaller chunks — focus is how tvOS scrolls, and each
|
||||
/// chunk is one focus stop, so a 200-line chunk (~5 screens tall there) would skip most of
|
||||
/// itself per step; ~24 lines ≈ two thirds of a screen reads like a page turn. Elsewhere the
|
||||
/// only constraint is the text-render height limit, so chunks stay big.
|
||||
private static var chunkLines: Int {
|
||||
#if os(tvOS)
|
||||
24
|
||||
#else
|
||||
200
|
||||
#endif
|
||||
}
|
||||
|
||||
/// `text` split at line boundaries into render/focus-sized chunks (joined with "\n"; the
|
||||
/// inter-chunk break is the caller's stack-row boundary, so no text is lost). tvOS pages
|
||||
/// focus through these — every license wall on the Acknowledgements screen renders this way.
|
||||
public static func chunked(_ text: String) -> [String] {
|
||||
let lines = text.split(separator: "\n", omittingEmptySubsequences: false)
|
||||
return stride(from: 0, to: lines.count, by: chunkLines).map { start in
|
||||
lines[start..<min(start + chunkLines, lines.count)].joined(separator: "\n")
|
||||
}
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,6 +10,9 @@
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
import QuartzCore
|
||||
#if os(tvOS)
|
||||
import UIKit
|
||||
#endif
|
||||
|
||||
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view or
|
||||
/// presenter directly makes a `owner → link → owner` cycle that only `invalidate()` breaks — if a
|
||||
@@ -34,16 +37,31 @@ enum PresenterChoice: Equatable {
|
||||
|
||||
/// Resolve from the `PUNKTFUNK_PRESENTER` env override (A/B without touching settings) first,
|
||||
/// then the persisted `DefaultsKey.presenter` setting; anything unknown (or an empty env var)
|
||||
/// falls back to stage-2. `allowStage1` is false in release builds, where a leftover DEBUG
|
||||
/// "stage1" value silently maps to stage-2 rather than reviving the freeze-prone fallback.
|
||||
/// falls back to the platform default. `allowStage1` is false in release builds, where a
|
||||
/// leftover DEBUG "stage1" value silently maps to the default rather than reviving the
|
||||
/// freeze-prone fallback.
|
||||
static func resolve(setting: String?, env: String?, allowStage1: Bool) -> PresenterChoice {
|
||||
let raw = env.flatMap { $0.isEmpty ? nil : $0 } ?? setting
|
||||
switch raw {
|
||||
case "stage1": return allowStage1 ? .stage1 : .stage2
|
||||
case "stage1": return allowStage1 ? .stage1 : platformDefault
|
||||
case "stage2": return .stage2
|
||||
case "stage3": return .stage3
|
||||
default: return .stage2
|
||||
default: return platformDefault
|
||||
}
|
||||
}
|
||||
|
||||
/// tvOS defaults to GLASS pacing: an Apple TV is the sticky-FIFO worst case by construction —
|
||||
/// a fixed 60 Hz panel fed a 60 fps stream, where arrival pacing pins the layer's image queue
|
||||
/// at ~3 drawables and every frame rides ~50 ms of queue (the measured display stage there).
|
||||
/// The Settings picker can still force stage-2 for an A/B. Everything else keeps stage-2 (the
|
||||
/// proven default; ProMotion/desktop panels out-tick the stream often enough to drain).
|
||||
static var platformDefault: PresenterChoice {
|
||||
#if os(tvOS)
|
||||
.stage3
|
||||
#else
|
||||
.stage2
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
final class SessionPresenter {
|
||||
@@ -179,6 +197,13 @@ final class SessionPresenter {
|
||||
stage2?.setDrawableTarget(CGSize(
|
||||
width: (fit.width * contentsScale).rounded(),
|
||||
height: (fit.height * contentsScale).rounded()))
|
||||
#if os(tvOS)
|
||||
// Push the display's live EDR headroom alongside: > 1 means the TV is composited in an
|
||||
// HDR mode (the session's AVDisplayManager request landed — see StreamViewIOS), and HDR
|
||||
// frames flip to PQ passthrough. The stream view also re-layouts on mode-switch/screen-
|
||||
// mode notifications, so a mid-session switch reaches here without a bounds change.
|
||||
stage2?.setDisplayHeadroom(UIScreen.main.currentEDRHeadroom)
|
||||
#endif
|
||||
}
|
||||
|
||||
/// Stop the active pump/pipeline (≤ one poll timeout; stage-2 joins its pump) and detach the
|
||||
|
||||
@@ -358,7 +358,12 @@ public final class Stage2Pipeline {
|
||||
// decode 4:4:4 at the negotiated resolution (the HW probe clears the common case but not a
|
||||
// resolution-ceiling miss). End cleanly instead of looping on a black screen.
|
||||
var decodeFailRun = 0
|
||||
while !token.isStopped {
|
||||
// Every iteration drains its own autorelease pool: this thread has no runloop, so
|
||||
// autoreleased VT/CM temporaries would otherwise accumulate until session end.
|
||||
// `false` = session over — exit the loop (the closure can't `break` across itself).
|
||||
var alive = true
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
|
||||
// decoder conceals the reference-missing deltas — often WITHOUT an error callback —
|
||||
@@ -378,13 +383,13 @@ public final class Stage2Pipeline {
|
||||
if let meta = try? connection.nextHdrMeta(timeoutMs: 0) {
|
||||
presenter.setHdrMeta(meta)
|
||||
}
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
|
||||
onFrame?(au)
|
||||
if let f = connection.videoCodec.formatDescription(fromKeyframe: au.data) {
|
||||
format = f // refreshed on every IDR (mode changes included)
|
||||
awaitingIDR = false // a fresh IDR re-anchored decode — recovery complete
|
||||
}
|
||||
guard let f = format, !token.isStopped else { continue }
|
||||
guard let f = format, !token.isStopped else { return true }
|
||||
if decoder.decode(au: au, format: f) {
|
||||
decodeFailRun = 0
|
||||
} else {
|
||||
@@ -397,12 +402,14 @@ public final class Stage2Pipeline {
|
||||
// recovers within a GOP) ⇒ 4:4:4 isn't decodable here; end the session.
|
||||
if connection.isChroma444, decodeFailRun >= 180 {
|
||||
if !token.isStopped { onSessionEnd?() }
|
||||
break
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
} catch {
|
||||
if !token.isStopped { onSessionEnd?() }
|
||||
break // session closed
|
||||
return false // session closed
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -435,10 +442,14 @@ public final class Stage2Pipeline {
|
||||
let gate: PresentGate? = pacing == .glass ? PresentGate() : nil
|
||||
let renderThread = Thread {
|
||||
defer { renderStopped.signal() }
|
||||
while !token.isStopped {
|
||||
// Every iteration drains its own autorelease pool (`return` = the old `continue`):
|
||||
// this thread has no runloop, and `nextDrawable()` AUTORELEASES each CAMetalDrawable —
|
||||
// without a per-iteration pool every presented frame's drawable object (plus its
|
||||
// texture-descriptor/array retinue, ~2 MB/min at 120 fps) piles up until session end.
|
||||
while !token.isStopped { autoreleasepool {
|
||||
if renderSignal.wait(timeout: .now() + .milliseconds(100)) == .timedOut {
|
||||
debugStats?.flushIfDue(ring: ring, gate: gate)
|
||||
continue
|
||||
return
|
||||
}
|
||||
// Stage-3: while a present is in flight, don't take from the ring at all — frames
|
||||
// keep coalescing there (newest wins, the intended drop point) and the presented
|
||||
@@ -447,13 +458,13 @@ public final class Stage2Pipeline {
|
||||
if let gate, !gate.tryAcquire(now: CACurrentMediaTime()) {
|
||||
debugStats?.gatedWake()
|
||||
debugStats?.flushIfDue(ring: ring, gate: gate)
|
||||
continue
|
||||
return
|
||||
}
|
||||
guard !token.isStopped, let frame = ring.take() else {
|
||||
gate?.release() // armed but nothing to render — don't hold the gate stale
|
||||
debugStats?.emptyWake()
|
||||
debugStats?.flushIfDue(ring: ring, gate: gate)
|
||||
continue
|
||||
return
|
||||
}
|
||||
// V-Sync ON: flip on the next predicted vsync (< one period out, stale link ⇒
|
||||
// immediate — see VsyncClock). OFF: flip as soon as the GPU finishes.
|
||||
@@ -488,7 +499,7 @@ public final class Stage2Pipeline {
|
||||
ring.putBack(frame)
|
||||
}
|
||||
debugStats?.flushIfDue(ring: ring, gate: gate)
|
||||
}
|
||||
} }
|
||||
}
|
||||
renderThread.name = "punktfunk-stage2-render"
|
||||
renderThread.qualityOfService = .userInteractive
|
||||
@@ -512,6 +523,13 @@ public final class Stage2Pipeline {
|
||||
presenter.setDrawableTarget(size)
|
||||
}
|
||||
|
||||
/// Forward the display's current EDR headroom to the presenter (MAIN thread — a `UIScreen`
|
||||
/// read). tvOS flips HDR presentation between PQ passthrough and the in-shader tone-map on
|
||||
/// it; see `MetalVideoPresenter.setDisplayHeadroom`.
|
||||
public func setDisplayHeadroom(_ headroom: CGFloat) {
|
||||
presenter.setDisplayHeadroom(headroom)
|
||||
}
|
||||
|
||||
/// Stop the pump + render thread (≤ one poll timeout each) and drop the decode session. MAIN
|
||||
/// THREAD; idempotent. Does not close the connection. A restart needs a fresh Stage2Pipeline
|
||||
/// (the stop is permanent).
|
||||
|
||||
@@ -47,7 +47,12 @@ final class StreamPump {
|
||||
var awaitingIDR = false
|
||||
var awaitingSince = Date.distantPast // when the current recovery began (for the resume log)
|
||||
var wasFailed = false
|
||||
while !token.isStopped {
|
||||
// Every iteration drains its own autorelease pool: this thread has no runloop, so
|
||||
// autoreleased CM/layer temporaries would otherwise accumulate until session end.
|
||||
// `false` = session over — exit the loop (the closure can't `break` across itself).
|
||||
var alive = true
|
||||
while alive, !token.isStopped {
|
||||
alive = autoreleasepool { () -> Bool in
|
||||
do {
|
||||
// Loss recovery (the primary path). Under the host's infinite GOP the only
|
||||
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
|
||||
@@ -69,7 +74,7 @@ final class StreamPump {
|
||||
}
|
||||
if awaitingIDR { recovery.request() }
|
||||
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
|
||||
onFrame?(au)
|
||||
let idrFormat = connection.videoCodec.formatDescription(fromKeyframe: au.data)
|
||||
if let f = idrFormat {
|
||||
@@ -97,13 +102,15 @@ final class StreamPump {
|
||||
guard let f = format,
|
||||
let sample = connection.videoCodec.sampleBuffer(au: au, format: f),
|
||||
!token.isStopped // don't enqueue a stale frame after a restart
|
||||
else { continue }
|
||||
else { return true }
|
||||
layer.enqueue(sample)
|
||||
return true
|
||||
} catch {
|
||||
if !token.isStopped {
|
||||
onSessionEnd?()
|
||||
}
|
||||
break // session closed
|
||||
return false // session closed
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -36,6 +36,9 @@ import PunktfunkCore
|
||||
import SwiftUI
|
||||
import UIKit
|
||||
import os
|
||||
#if os(tvOS)
|
||||
import AVKit // AVDisplayManager — the per-session display-mode (HDR10/refresh) request
|
||||
#endif
|
||||
|
||||
/// Same diagnostic switch as InputCapture (PUNKTFUNK_INPUT_DEBUG=1): on iOS we log the
|
||||
/// resolved pointer-lock state each time capture engages, so the user can see whether the
|
||||
@@ -108,7 +111,20 @@ public struct StreamView: UIViewControllerRepresentable {
|
||||
}
|
||||
}
|
||||
|
||||
public final class StreamViewController: UIViewController {
|
||||
#if os(tvOS)
|
||||
/// tvOS: a GCEventViewController with `controllerUserInteractionEnabled = false` routes game-
|
||||
/// controller (and Siri Remote) input EXCLUSIVELY to the GameController framework while the
|
||||
/// stream is up. Without it a pad's B/Menu press doubles as a UIKit menu press — which ended
|
||||
/// the session (or suspended the whole app) from ordinary gameplay; a SwiftUI
|
||||
/// `.onExitCommand {}` swallow proved unreliable with nothing focusable on screen. Every
|
||||
/// in-session exit is GC-level by design: the pad's escape chord (GamepadCapture) and the
|
||||
/// remote's hold-Back (SiriRemotePointer).
|
||||
public typealias StreamViewControllerBase = GCEventViewController
|
||||
#else
|
||||
public typealias StreamViewControllerBase = UIViewController
|
||||
#endif
|
||||
|
||||
public final class StreamViewController: StreamViewControllerBase {
|
||||
public private(set) var connection: PunktfunkConnection?
|
||||
private var observers: [NSObjectProtocol] = []
|
||||
/// Record the unified latency stages (end-to-end / decode / display) when the stage-2
|
||||
@@ -119,6 +135,11 @@ public final class StreamViewController: UIViewController {
|
||||
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
|
||||
/// stage-1 StreamPump → displayLayer path as the Metal-unavailable / DEBUG fallback.
|
||||
private let presenter = SessionPresenter()
|
||||
#if os(tvOS)
|
||||
/// The window's display manager the session's mode request was set on — held weakly so
|
||||
/// stop() can clear the request even after the view has left the window.
|
||||
private weak var sessionDisplayManager: AVDisplayManager?
|
||||
#endif
|
||||
#if os(iOS)
|
||||
private var inputCapture: InputCapture?
|
||||
fileprivate var captured = false
|
||||
@@ -157,6 +178,12 @@ public final class StreamViewController: UIViewController {
|
||||
|
||||
public override func loadView() {
|
||||
view = StreamLayerUIView()
|
||||
#if os(tvOS)
|
||||
// Kill the pad/remote → UIKit press path at the source for the whole session (see the
|
||||
// GCEventViewController typealias above). GC delivery is untouched: GamepadCapture
|
||||
// forwards the pad, SiriRemotePointer drives the pointer and owns the remote exit.
|
||||
controllerUserInteractionEnabled = false
|
||||
#endif
|
||||
// Re-size the stage-2 drawable if the display scale changes without a bounds change (e.g.
|
||||
// moving to an external display at a different scale) — the iOS analogue of macOS's
|
||||
// viewDidChangeBackingProperties relayout. The handler takes the VC as its argument, so it
|
||||
@@ -230,6 +257,18 @@ public final class StreamViewController: UIViewController {
|
||||
}
|
||||
#endif
|
||||
|
||||
#if os(tvOS)
|
||||
// The GCEventViewController's interaction flag applies to the deepest such controller
|
||||
// CONTAINING THE FIRST RESPONDER — inside SwiftUI's hosting-controller sandwich that is not
|
||||
// guaranteed to be us unless we anchor the responder chain here explicitly.
|
||||
public override var canBecomeFirstResponder: Bool { true }
|
||||
|
||||
public override func viewDidAppear(_ animated: Bool) {
|
||||
super.viewDidAppear(animated)
|
||||
becomeFirstResponder()
|
||||
}
|
||||
#endif
|
||||
|
||||
func start(
|
||||
connection: PunktfunkConnection,
|
||||
onFrame: (@Sendable (AccessUnit) -> Void)?,
|
||||
@@ -342,6 +381,19 @@ public final class StreamViewController: UIViewController {
|
||||
setCaptured(true) // entering a session is the deliberate "capture me" moment
|
||||
}
|
||||
#endif
|
||||
|
||||
#if os(tvOS)
|
||||
// The TV's mode switch (requested in applyDisplayCriteriaIfNeeded) completes
|
||||
// asynchronously, and a dynamic-range-only switch doesn't re-layout by itself —
|
||||
// re-layout on the switch/mode notifications so the presenter sees the new EDR
|
||||
// headroom immediately (layout pushes UIScreen.currentEDRHeadroom down).
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
forName: .AVDisplayManagerModeSwitchEnd, object: nil, queue: .main
|
||||
) { [weak self] _ in self?.layoutMetalLayer() })
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
forName: UIScreen.modeDidChangeNotification, object: nil, queue: .main
|
||||
) { [weak self] _ in self?.layoutMetalLayer() })
|
||||
#endif
|
||||
}
|
||||
|
||||
func stop() {
|
||||
@@ -360,6 +412,12 @@ public final class StreamViewController: UIViewController {
|
||||
streamView.onScroll = nil
|
||||
streamView.currentHostMode = nil
|
||||
#endif
|
||||
#if os(tvOS)
|
||||
// Return the TV to the user's preferred mode — the home screen must not stay in the
|
||||
// session's HDR10/refresh mode.
|
||||
sessionDisplayManager?.preferredDisplayCriteria = nil
|
||||
sessionDisplayManager = nil
|
||||
#endif
|
||||
presenter.stop()
|
||||
connection = nil
|
||||
}
|
||||
@@ -367,8 +425,50 @@ public final class StreamViewController: UIViewController {
|
||||
public override func viewDidLayoutSubviews() {
|
||||
super.viewDidLayoutSubviews()
|
||||
layoutMetalLayer()
|
||||
#if os(tvOS)
|
||||
applyDisplayCriteriaIfNeeded()
|
||||
#endif
|
||||
}
|
||||
|
||||
#if os(tvOS)
|
||||
/// Ask the TV for a display mode matching the session — HDR10 at the stream's refresh rate —
|
||||
/// via AVDisplayManager, the tvOS mechanism custom renderers use for HDR output (AVFoundation
|
||||
/// playback layers do this implicitly). Honored only when the user allows matching (tvOS
|
||||
/// Settings → Video and Audio → Match Content); the presenter reads the RESULT off UIScreen's
|
||||
/// EDR headroom (pushed in SessionPresenter.layout) and keeps the in-shader tone-map whenever
|
||||
/// the switch never lands, so an SDR-composited display can't show blown-out PQ either way.
|
||||
/// Applied once per session, as soon as the window and the negotiated mode both exist; the
|
||||
/// stop() teardown clears it.
|
||||
private func applyDisplayCriteriaIfNeeded() {
|
||||
guard let manager = view.window?.avDisplayManager, let connection,
|
||||
manager.preferredDisplayCriteria == nil,
|
||||
UserDefaults.standard.object(forKey: DefaultsKey.hdrEnabled) as? Bool ?? true
|
||||
else { return }
|
||||
let mode = connection.currentMode()
|
||||
guard mode.width > 0, mode.height > 0, mode.refreshHz > 0 else { return }
|
||||
// A synthetic HDR10-HEVC format description carrying the negotiated mode — what the
|
||||
// stream decodes to. AVDisplayCriteria(refreshRate:formatDescription:) matches the
|
||||
// display to it (tvOS 17+, our deployment floor).
|
||||
let ext: [CFString: Any] = [
|
||||
kCMFormatDescriptionExtension_ColorPrimaries:
|
||||
kCMFormatDescriptionColorPrimaries_ITU_R_2020,
|
||||
kCMFormatDescriptionExtension_TransferFunction:
|
||||
kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ,
|
||||
kCMFormatDescriptionExtension_YCbCrMatrix:
|
||||
kCMFormatDescriptionYCbCrMatrix_ITU_R_2020,
|
||||
]
|
||||
var desc: CMFormatDescription?
|
||||
CMVideoFormatDescriptionCreate(
|
||||
allocator: kCFAllocatorDefault, codecType: kCMVideoCodecType_HEVC,
|
||||
width: Int32(mode.width), height: Int32(mode.height),
|
||||
extensions: ext as CFDictionary, formatDescriptionOut: &desc)
|
||||
guard let desc else { return }
|
||||
manager.preferredDisplayCriteria = AVDisplayCriteria(
|
||||
refreshRate: Float(mode.refreshHz), formatDescription: desc)
|
||||
sessionDisplayManager = manager
|
||||
}
|
||||
#endif
|
||||
|
||||
/// The display scale to render the metal drawable at. `traitCollection.displayScale` is the
|
||||
/// canonical render scale and is reliable once the controller is in the hierarchy;
|
||||
/// `view.contentScaleFactor` can read 1.0 before the view attaches to a window/screen, which
|
||||
|
||||
Reference in New Issue
Block a user