feat(apple): the gamepad UI comes to tvOS - focus-driven, with real session controls
ci / rust (push) Failing after 50s
ci / web (push) Successful in 57s
decky / build-publish (push) Successful in 14s
ci / docs-site (push) Successful in 1m2s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
flatpak / build-publish (push) Failing after 2m14s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m46s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 11m40s
arch / build-publish (push) Successful in 18m10s
deb / build-publish (push) Successful in 18m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m21s
android / android (push) Successful in 21m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 5m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 1m2s
release / apple (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled

The console UI now runs on tvOS through the NATIVE focus engine: carousel
cards and settings rows are focusable Buttons (Siri Remote and pads both
navigate; imperative scrollTo replaces the drop-prone scrollPosition binding),
while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what
focus has no concept of: X/Y screen actions and left/right value adjust with
the poll's dominant-axis feel (onMoveCommand proved input-source-dependent:
keyboard intercepted, pad dpad not -> double steps). Text entry uses the
system fullscreen keyboard (TVTextEntry); pairing + library present as covers
under the launcher; the game library defaults ON; settings values slide a
quiet 14 pt in the step's direction.

Session controls: controller/remote input routes EXCLUSIVELY through
GameController during a stream (GCEventViewController, interaction disabled) -
a pad's B no longer doubles as a UIKit menu press that ended sessions
mid-game. Deliberate exits only: the cross-client escape chord (hold
L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all
Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream
banner (now also on tvOS) teaches both. The Siri Remote's touch surface
drives the host pointer - press = left click, Play/Pause = right click,
release-tail jumps gated so motion stays truly relative.

tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every
unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root
font shrank the whole platform (29 pt there), plus 10-foot sizing across host
cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu"
hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops
instead of suspending; full-width focusSections make the home actions
reachable from any column. The presenter defaults to stage-3 glass pacing on
tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind
the 50 ms display stage) and is pickable from the gamepad settings; HDR
capability advertises from AVPlayer.eligibleForHDRPlayback instead of the
current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-10 16:58:35 +02:00
parent 1fcf9e11ec
commit 3ba19f28a2
29 changed files with 1208 additions and 242 deletions
@@ -263,19 +263,24 @@ public final class SessionAudio {
defer { drainDone.signal() }
// Decode happens IN-CORE (libopus multistream) AudioToolbox's Opus path is
// stereo-only and is handed back as interleaved f32 PCM in wire channel order.
while !flag.isStopped {
// Per-iteration autorelease pool: no runloop on this thread (see Stage2Pipeline).
var alive = true
while alive, !flag.isStopped {
alive = autoreleasepool { () -> Bool in
let pcm: PunktfunkConnection.AudioPCM?
do {
pcm = try connection.nextAudioPcm(timeoutMs: 100)
} catch {
break // session closed
return false // session closed
}
guard let pcm, pcm.frameCount > 0 else { continue }
guard let pcm, pcm.frameCount > 0 else { return true }
pcm.samples.withUnsafeBufferPointer { p in
if let base = p.baseAddress {
ring.write(base, count: pcm.frameCount * pcm.channels)
}
}
return true
}
}
}
thread.name = "punktfunk-audio"
@@ -48,6 +48,23 @@ public final class GamepadCapture {
/// Motion forwarding floor: 4 ms between samples ( 250 Hz, the DualSense's own rate).
private static let motionIntervalNs: UInt64 = 4_000_000
/// The cross-client controller escape chord (pf-client-core's `ESCAPE_CHORD`):
/// L1+R1+Start+Select held together four simultaneous buttons no game uses, so normal
/// play can't trip it. Held for `disconnectHold` it ends the session via
/// `onDisconnectRequest`; the chord keeps forwarding to the host meanwhile (the user is
/// leaving anyway). The desktop clients' quick-press step (leave fullscreen / release
/// capture) has no Apple equivalent worth wiring macOS has Q/D, touch has the HUD.
private static let escapeChord: UInt32 =
GamepadWire.leftShoulder | GamepadWire.rightShoulder | GamepadWire.start | GamepadWire.back
/// pf-client-core's `DISCONNECT_HOLD` the same 1.5 s on every client.
private static let disconnectHold: TimeInterval = 1.5
private var chordTimer: Timer?
/// Fired ON MAIN once the escape chord has been held `disconnectHold` the session owner
/// disconnects. On tvOS this (plus the Siri Remote's hold-Back) is the ONLY way out of a
/// stream with a controller: B/Menu presses are deliberately swallowed during a session so
/// gameplay can't end it (see ContentView's tvOS session branch).
public var onDisconnectRequest: (() -> Void)?
public init(connection: PunktfunkConnection, manager: GamepadManager) {
self.connection = connection
self.manager = manager
@@ -165,6 +182,7 @@ public final class GamepadCapture {
private func sync(_ g: GCExtendedGamepad) {
guard !suspended else { return }
let newButtons = Self.buttonMask(g)
updateEscapeChord(newButtons)
let changed = newButtons ^ buttons
if changed != 0 {
for bit in GamepadWire.allButtons where changed & bit != 0 {
@@ -297,7 +315,26 @@ public final class GamepadCapture {
/// Unwind everything held on the wire: button-ups, neutral axes, lifted fingers. The
/// host's virtual pad returns to rest instead of running with the last state.
/// Arm the disconnect timer when the full chord lands, disarm the moment any of the four
/// releases. Events only arrive on state CHANGES, so a held chord needs the timer the
/// handler won't fire again until something moves.
private func updateEscapeChord(_ newButtons: UInt32) {
let held = newButtons & Self.escapeChord == Self.escapeChord
if held, chordTimer == nil {
let timer = Timer(timeInterval: Self.disconnectHold, repeats: false) { [weak self] _ in
Task { @MainActor in self?.onDisconnectRequest?() }
}
RunLoop.main.add(timer, forMode: .common)
chordTimer = timer
} else if !held, chordTimer != nil {
chordTimer?.invalidate()
chordTimer = nil
}
}
private func releaseAll() {
chordTimer?.invalidate()
chordTimer = nil
for bit in GamepadWire.allButtons where buttons & bit != 0 {
connection.send(.gamepadButton(bit, down: false, pad: 0))
}
@@ -74,7 +74,11 @@ public final class GamepadFeedback {
// session a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and
// let rumble own the wait elsewhere; on an Xbox session it stays nonblocking.
let thread = Thread { [connection, flag, drainDone, weak self] in
while !flag.isStopped {
// Per-iteration autorelease pool: no runloop on this thread, and the haptics/HID
// rendering below autoreleases ObjC temporaries. `false` = session over.
var alive = true
while alive, !flag.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Poll the feedback planes NON-BLOCKING. A blocking poll (timeoutMs > 0) holds
// the connection's shared feedback lock for its whole wait; the video pump drains
@@ -106,12 +110,14 @@ public final class GamepadFeedback {
self?.render(ev)
burst += 1
}
return true
} catch {
break // .closed (or fatal) the session is over
return false // .closed (or fatal) the session is over
}
}
// ~8 ms poll cadence (125 Hz), slept OUTSIDE the feedback lock low rumble/HID
// latency without holding the lock the HDR-meta drain needs.
if !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
if alive, !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
}
drainDone.signal()
}
@@ -0,0 +1,173 @@
// The Siri Remote as a pointing device during a tvOS streaming session the remote's touch
// surface drives the HOST cursor (relative deltas, like a laptop trackpad), a surface press
// clicks (left button), and Play/Pause right-clicks. It also owns the remote's DELIBERATE
// session exit: hold Back/Menu `disconnectHold`. A short Back press does nothing the
// UIKit menu press it also generates is swallowed by ContentView's session branch, so neither
// a trackpad fumble nor a game-controller B press can end the session (the pad's exit is the
// L1+R1+Start+Select chord in GamepadCapture).
//
// The remote is read through GameController as a GCMicroGamepad with
// `reportsAbsoluteDpadValues = true`: the dpad axes then report the finger's ABSOLUTE position
// on the surface (±1, +y up) while touched, and snap to exactly (0, 0) on lift. Successive
// positions are differenced into relative mouse deltas; the exact-zero snap is treated as a
// lift (a real touch at the mathematical centre is measure-zero, and one dropped delta there
// is imperceptible). Handlers (not a poll) the same in-session delivery GamepadCapture
// relies on.
//
// Lifecycle mirrors GamepadCapture: started by SessionModel when streaming begins (never
// during the trust prompt), stopped on disconnect; held buttons are released on stop so the
// host never keeps a stuck click.
#if os(tvOS)
import Foundation
import GameController
import UIKit
@MainActor
public final class SiriRemotePointer {
private let connection: PunktfunkConnection
private var observers: [NSObjectProtocol] = []
private var bound: GCController?
/// Finger position (±1 axes) at the last dpad callback while touched; nil = lifted.
private var lastTouch: (x: Float, y: Float)?
/// Wire buttons currently held (1 = left, 3 = right) released on stop/unbind.
private var heldButtons: Set<UInt32> = []
/// When Back/Menu went down; a release after `disconnectHold` fires the exit.
private var menuDownAt: Date?
/// Hold Back/Menu at least this long (then release) to end the session. Shorter than the
/// controller chord's 1.5 s the remote has no way to trip this during gameplay.
private static let disconnectHold: TimeInterval = 1.0
/// A full edge-to-edge swipe moves the host cursor about this many pixels. The surface is
/// small; two comfortable swipes should cross a 1080p desktop.
private static let pointerScale: Float = 1100
/// Largest single-callback finger travel accepted as real motion (surface units; the axes
/// span ±1, so 0.4 a fifth of the pad). On RELEASE the hardware slides the reported
/// position back to (0, 0) through intermediate callbacks naive differencing turns that
/// tail into reverse deltas that RETRACE the whole swipe, so the cursor springs back to its
/// anchor and the pointer feels absolute. Real finger motion arrives as many small steps
/// (even a fast flick stays well under this per callback); the release tail arrives as one
/// or two huge jumps discard those (the anchor still follows, so nothing accumulates).
private static let maxStep: Float = 0.4
/// Fired ON MAIN after Back/Menu was held `disconnectHold` and released.
public var onDisconnectRequest: (() -> Void)?
public init(connection: PunktfunkConnection) {
self.connection = connection
}
public func start() {
observers.append(NotificationCenter.default.addObserver(
forName: .GCControllerDidConnect, object: nil, queue: .main
) { [weak self] _ in
MainActor.assumeIsolated { self?.rebind() }
})
observers.append(NotificationCenter.default.addObserver(
forName: .GCControllerDidDisconnect, object: nil, queue: .main
) { [weak self] _ in
MainActor.assumeIsolated { self?.rebind() }
})
rebind()
}
public func stop() {
observers.forEach(NotificationCenter.default.removeObserver(_:))
observers.removeAll()
bind(nil)
}
/// The Siri Remote is the non-extended controller carrying a microGamepad a full gamepad
/// (which also EXPOSES a microGamepad view of itself) must never be captured here, its
/// buttons belong to GamepadCapture.
private func rebind() {
let remote = GCController.controllers().first {
$0.extendedGamepad == nil && $0.microGamepad != nil
}
bind(remote)
}
private func bind(_ controller: GCController?) {
guard controller !== bound else { return }
if let old = bound?.microGamepad {
old.dpad.valueChangedHandler = nil
old.buttonA.pressedChangedHandler = nil
old.buttonX.pressedChangedHandler = nil
old.buttonMenu.pressedChangedHandler = nil
}
releaseHeld()
lastTouch = nil
menuDownAt = nil
bound = controller
guard let micro = controller?.microGamepad else { return }
// Absolute finger position instead of the emulated dpad the raw surface is what a
// trackpad needs. Rotation stays off: the remote's natural grip is the coordinate frame.
micro.reportsAbsoluteDpadValues = true
micro.allowsRotation = false
micro.dpad.valueChangedHandler = { [weak self] _, x, y in
MainActor.assumeIsolated { self?.touchMoved(x: x, y: y) }
}
// Surface click = left button; Play/Pause = right (the remote's only spare face button).
micro.buttonA.pressedChangedHandler = { [weak self] _, _, pressed in
MainActor.assumeIsolated { self?.setButton(1, down: pressed) }
}
micro.buttonX.pressedChangedHandler = { [weak self] _, _, pressed in
MainActor.assumeIsolated { self?.setButton(3, down: pressed) }
}
micro.buttonMenu.pressedChangedHandler = { [weak self] _, _, pressed in
MainActor.assumeIsolated { self?.menuChanged(pressed: pressed) }
}
}
private func touchMoved(x: Float, y: Float) {
// Exact (0, 0) is the lift snap drop the anchor so the next touch starts a fresh
// gesture instead of a jump-delta from the old position.
guard x != 0 || y != 0 else {
lastTouch = nil
return
}
defer { lastTouch = (x, y) }
guard let last = lastTouch else { return } // first contact anchors, moves nothing
let stepX = x - last.x
let stepY = y - last.y
// The release tail (and any tracking glitch) shows up as a single impossible jump
// see `maxStep`. Skip the emission; the deferred anchor update above still follows the
// reported position, so the gesture cleanly re-anchors instead of retracing.
guard abs(stepX) < Self.maxStep, abs(stepY) < Self.maxStep else { return }
let dx = stepX * Self.pointerScale / 2 // axes span ±1 full swipe = 2.0
let dy = -stepY * Self.pointerScale / 2 // GC +y is up; mouse +y is down
let ix = Int32(dx.rounded())
let iy = Int32(dy.rounded())
guard ix != 0 || iy != 0 else { return }
connection.send(.mouseMove(dx: ix, dy: iy))
}
private func setButton(_ button: UInt32, down: Bool) {
if down { heldButtons.insert(button) } else { heldButtons.remove(button) }
connection.send(.mouseButton(button, down: down))
}
private func menuChanged(pressed: Bool) {
if pressed {
menuDownAt = Date()
return
}
let heldFor = menuDownAt.map { Date().timeIntervalSince($0) } ?? 0
menuDownAt = nil
if heldFor >= Self.disconnectHold {
onDisconnectRequest?()
}
// A short press is deliberately nothing: the accompanying UIKit menu press is swallowed
// in ContentView, and forwarding it as a host key would make trackpad fumbles type.
}
private func releaseHeld() {
for button in heldButtons {
connection.send(.mouseButton(button, down: false))
}
heldButtons.removeAll()
}
}
#endif
@@ -13,14 +13,14 @@ public enum Licenses {
return text
}
/// punktfunk's own license MIT OR Apache-2.0, at your option.
/// Punktfunk's own license MIT OR Apache-2.0, at your option.
public static var appLicense: String {
let mit = resource("LICENSE-MIT")
let apache = resource("LICENSE-APACHE")
if mit.isEmpty && apache.isEmpty {
return "punktfunk is licensed under MIT OR Apache-2.0, at your option."
return "Punktfunk is licensed under MIT OR Apache-2.0, at your option."
}
return "punktfunk is licensed under MIT OR Apache-2.0, at your option.\n\n"
return "Punktfunk is licensed under MIT OR Apache-2.0, at your option.\n\n"
+ "================================ MIT ================================\n\n"
+ mit
+ "\n\n============================== Apache-2.0 ==============================\n\n"
@@ -51,11 +51,27 @@ public enum Licenses {
/// Acknowledgements screen renders these chunks in a `LazyVStack` (only on-screen chunks lay
/// out, and no chunk is tall enough to clip). Split at line boundaries and joined with "\n";
/// the inter-chunk break is the `LazyVStack` row boundary, so no text is lost. Computed once.
public static let thirdPartyNoticesChunks: [String] = {
let lines = thirdPartyNotices.split(separator: "\n", omittingEmptySubsequences: false)
let chunkSize = 200
return stride(from: 0, to: lines.count, by: chunkSize).map { start in
lines[start..<min(start + chunkSize, lines.count)].joined(separator: "\n")
public static let thirdPartyNoticesChunks: [String] = chunked(thirdPartyNotices)
/// Lines per chunk: tvOS reads much smaller chunks focus is how tvOS scrolls, and each
/// chunk is one focus stop, so a 200-line chunk (~5 screens tall there) would skip most of
/// itself per step; ~24 lines two thirds of a screen reads like a page turn. Elsewhere the
/// only constraint is the text-render height limit, so chunks stay big.
private static var chunkLines: Int {
#if os(tvOS)
24
#else
200
#endif
}
/// `text` split at line boundaries into render/focus-sized chunks (joined with "\n"; the
/// inter-chunk break is the caller's stack-row boundary, so no text is lost). tvOS pages
/// focus through these every license wall on the Acknowledgements screen renders this way.
public static func chunked(_ text: String) -> [String] {
let lines = text.split(separator: "\n", omittingEmptySubsequences: false)
return stride(from: 0, to: lines.count, by: chunkLines).map { start in
lines[start..<min(start + chunkLines, lines.count)].joined(separator: "\n")
}
}()
}
}
@@ -10,6 +10,9 @@
import AVFoundation
import Foundation
import QuartzCore
#if os(tvOS)
import UIKit
#endif
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view or
/// presenter directly makes a `owner link owner` cycle that only `invalidate()` breaks if a
@@ -34,16 +37,31 @@ enum PresenterChoice: Equatable {
/// Resolve from the `PUNKTFUNK_PRESENTER` env override (A/B without touching settings) first,
/// then the persisted `DefaultsKey.presenter` setting; anything unknown (or an empty env var)
/// falls back to stage-2. `allowStage1` is false in release builds, where a leftover DEBUG
/// "stage1" value silently maps to stage-2 rather than reviving the freeze-prone fallback.
/// falls back to the platform default. `allowStage1` is false in release builds, where a
/// leftover DEBUG "stage1" value silently maps to the default rather than reviving the
/// freeze-prone fallback.
static func resolve(setting: String?, env: String?, allowStage1: Bool) -> PresenterChoice {
let raw = env.flatMap { $0.isEmpty ? nil : $0 } ?? setting
switch raw {
case "stage1": return allowStage1 ? .stage1 : .stage2
case "stage1": return allowStage1 ? .stage1 : platformDefault
case "stage2": return .stage2
case "stage3": return .stage3
default: return .stage2
default: return platformDefault
}
}
/// tvOS defaults to GLASS pacing: an Apple TV is the sticky-FIFO worst case by construction
/// a fixed 60 Hz panel fed a 60 fps stream, where arrival pacing pins the layer's image queue
/// at ~3 drawables and every frame rides ~50 ms of queue (the measured display stage there).
/// The Settings picker can still force stage-2 for an A/B. Everything else keeps stage-2 (the
/// proven default; ProMotion/desktop panels out-tick the stream often enough to drain).
static var platformDefault: PresenterChoice {
#if os(tvOS)
.stage3
#else
.stage2
#endif
}
}
final class SessionPresenter {
@@ -179,6 +197,13 @@ final class SessionPresenter {
stage2?.setDrawableTarget(CGSize(
width: (fit.width * contentsScale).rounded(),
height: (fit.height * contentsScale).rounded()))
#if os(tvOS)
// Push the display's live EDR headroom alongside: > 1 means the TV is composited in an
// HDR mode (the session's AVDisplayManager request landed see StreamViewIOS), and HDR
// frames flip to PQ passthrough. The stream view also re-layouts on mode-switch/screen-
// mode notifications, so a mid-session switch reaches here without a bounds change.
stage2?.setDisplayHeadroom(UIScreen.main.currentEDRHeadroom)
#endif
}
/// Stop the active pump/pipeline ( one poll timeout; stage-2 joins its pump) and detach the
@@ -358,7 +358,12 @@ public final class Stage2Pipeline {
// decode 4:4:4 at the negotiated resolution (the HW probe clears the common case but not a
// resolution-ceiling miss). End cleanly instead of looping on a black screen.
var decodeFailRun = 0
while !token.isStopped {
// Every iteration drains its own autorelease pool: this thread has no runloop, so
// autoreleased VT/CM temporaries would otherwise accumulate until session end.
// `false` = session over exit the loop (the closure can't `break` across itself).
var alive = true
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback
@@ -378,13 +383,13 @@ public final class Stage2Pipeline {
if let meta = try? connection.nextHdrMeta(timeoutMs: 0) {
presenter.setHdrMeta(meta)
}
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
onFrame?(au)
if let f = connection.videoCodec.formatDescription(fromKeyframe: au.data) {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
guard let f = format, !token.isStopped else { continue }
guard let f = format, !token.isStopped else { return true }
if decoder.decode(au: au, format: f) {
decodeFailRun = 0
} else {
@@ -397,12 +402,14 @@ public final class Stage2Pipeline {
// recovers within a GOP) 4:4:4 isn't decodable here; end the session.
if connection.isChroma444, decodeFailRun >= 180 {
if !token.isStopped { onSessionEnd?() }
break
return false
}
}
return true
} catch {
if !token.isStopped { onSessionEnd?() }
break // session closed
return false // session closed
}
}
}
}
@@ -435,10 +442,14 @@ public final class Stage2Pipeline {
let gate: PresentGate? = pacing == .glass ? PresentGate() : nil
let renderThread = Thread {
defer { renderStopped.signal() }
while !token.isStopped {
// Every iteration drains its own autorelease pool (`return` = the old `continue`):
// this thread has no runloop, and `nextDrawable()` AUTORELEASES each CAMetalDrawable
// without a per-iteration pool every presented frame's drawable object (plus its
// texture-descriptor/array retinue, ~2 MB/min at 120 fps) piles up until session end.
while !token.isStopped { autoreleasepool {
if renderSignal.wait(timeout: .now() + .milliseconds(100)) == .timedOut {
debugStats?.flushIfDue(ring: ring, gate: gate)
continue
return
}
// Stage-3: while a present is in flight, don't take from the ring at all frames
// keep coalescing there (newest wins, the intended drop point) and the presented
@@ -447,13 +458,13 @@ public final class Stage2Pipeline {
if let gate, !gate.tryAcquire(now: CACurrentMediaTime()) {
debugStats?.gatedWake()
debugStats?.flushIfDue(ring: ring, gate: gate)
continue
return
}
guard !token.isStopped, let frame = ring.take() else {
gate?.release() // armed but nothing to render don't hold the gate stale
debugStats?.emptyWake()
debugStats?.flushIfDue(ring: ring, gate: gate)
continue
return
}
// V-Sync ON: flip on the next predicted vsync (< one period out, stale link
// immediate see VsyncClock). OFF: flip as soon as the GPU finishes.
@@ -488,7 +499,7 @@ public final class Stage2Pipeline {
ring.putBack(frame)
}
debugStats?.flushIfDue(ring: ring, gate: gate)
}
} }
}
renderThread.name = "punktfunk-stage2-render"
renderThread.qualityOfService = .userInteractive
@@ -512,6 +523,13 @@ public final class Stage2Pipeline {
presenter.setDrawableTarget(size)
}
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
/// read). tvOS flips HDR presentation between PQ passthrough and the in-shader tone-map on
/// it; see `MetalVideoPresenter.setDisplayHeadroom`.
public func setDisplayHeadroom(_ headroom: CGFloat) {
presenter.setDisplayHeadroom(headroom)
}
/// Stop the pump + render thread ( one poll timeout each) and drop the decode session. MAIN
/// THREAD; idempotent. Does not close the connection. A restart needs a fresh Stage2Pipeline
/// (the stop is permanent).
@@ -47,7 +47,12 @@ final class StreamPump {
var awaitingIDR = false
var awaitingSince = Date.distantPast // when the current recovery began (for the resume log)
var wasFailed = false
while !token.isStopped {
// Every iteration drains its own autorelease pool: this thread has no runloop, so
// autoreleased CM/layer temporaries would otherwise accumulate until session end.
// `false` = session over exit the loop (the closure can't `break` across itself).
var alive = true
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
@@ -69,7 +74,7 @@ final class StreamPump {
}
if awaitingIDR { recovery.request() }
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
onFrame?(au)
let idrFormat = connection.videoCodec.formatDescription(fromKeyframe: au.data)
if let f = idrFormat {
@@ -97,13 +102,15 @@ final class StreamPump {
guard let f = format,
let sample = connection.videoCodec.sampleBuffer(au: au, format: f),
!token.isStopped // don't enqueue a stale frame after a restart
else { continue }
else { return true }
layer.enqueue(sample)
return true
} catch {
if !token.isStopped {
onSessionEnd?()
}
break // session closed
return false // session closed
}
}
}
}
@@ -36,6 +36,9 @@ import PunktfunkCore
import SwiftUI
import UIKit
import os
#if os(tvOS)
import AVKit // AVDisplayManager the per-session display-mode (HDR10/refresh) request
#endif
/// Same diagnostic switch as InputCapture (PUNKTFUNK_INPUT_DEBUG=1): on iOS we log the
/// resolved pointer-lock state each time capture engages, so the user can see whether the
@@ -108,7 +111,20 @@ public struct StreamView: UIViewControllerRepresentable {
}
}
public final class StreamViewController: UIViewController {
#if os(tvOS)
/// tvOS: a GCEventViewController with `controllerUserInteractionEnabled = false` routes game-
/// controller (and Siri Remote) input EXCLUSIVELY to the GameController framework while the
/// stream is up. Without it a pad's B/Menu press doubles as a UIKit menu press which ended
/// the session (or suspended the whole app) from ordinary gameplay; a SwiftUI
/// `.onExitCommand {}` swallow proved unreliable with nothing focusable on screen. Every
/// in-session exit is GC-level by design: the pad's escape chord (GamepadCapture) and the
/// remote's hold-Back (SiriRemotePointer).
public typealias StreamViewControllerBase = GCEventViewController
#else
public typealias StreamViewControllerBase = UIViewController
#endif
public final class StreamViewController: StreamViewControllerBase {
public private(set) var connection: PunktfunkConnection?
private var observers: [NSObjectProtocol] = []
/// Record the unified latency stages (end-to-end / decode / display) when the stage-2
@@ -119,6 +135,11 @@ public final class StreamViewController: UIViewController {
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
/// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback.
private let presenter = SessionPresenter()
#if os(tvOS)
/// The window's display manager the session's mode request was set on held weakly so
/// stop() can clear the request even after the view has left the window.
private weak var sessionDisplayManager: AVDisplayManager?
#endif
#if os(iOS)
private var inputCapture: InputCapture?
fileprivate var captured = false
@@ -157,6 +178,12 @@ public final class StreamViewController: UIViewController {
public override func loadView() {
view = StreamLayerUIView()
#if os(tvOS)
// Kill the pad/remote UIKit press path at the source for the whole session (see the
// GCEventViewController typealias above). GC delivery is untouched: GamepadCapture
// forwards the pad, SiriRemotePointer drives the pointer and owns the remote exit.
controllerUserInteractionEnabled = false
#endif
// Re-size the stage-2 drawable if the display scale changes without a bounds change (e.g.
// moving to an external display at a different scale) the iOS analogue of macOS's
// viewDidChangeBackingProperties relayout. The handler takes the VC as its argument, so it
@@ -230,6 +257,18 @@ public final class StreamViewController: UIViewController {
}
#endif
#if os(tvOS)
// The GCEventViewController's interaction flag applies to the deepest such controller
// CONTAINING THE FIRST RESPONDER inside SwiftUI's hosting-controller sandwich that is not
// guaranteed to be us unless we anchor the responder chain here explicitly.
public override var canBecomeFirstResponder: Bool { true }
public override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
becomeFirstResponder()
}
#endif
func start(
connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?,
@@ -342,6 +381,19 @@ public final class StreamViewController: UIViewController {
setCaptured(true) // entering a session is the deliberate "capture me" moment
}
#endif
#if os(tvOS)
// The TV's mode switch (requested in applyDisplayCriteriaIfNeeded) completes
// asynchronously, and a dynamic-range-only switch doesn't re-layout by itself
// re-layout on the switch/mode notifications so the presenter sees the new EDR
// headroom immediately (layout pushes UIScreen.currentEDRHeadroom down).
observers.append(NotificationCenter.default.addObserver(
forName: .AVDisplayManagerModeSwitchEnd, object: nil, queue: .main
) { [weak self] _ in self?.layoutMetalLayer() })
observers.append(NotificationCenter.default.addObserver(
forName: UIScreen.modeDidChangeNotification, object: nil, queue: .main
) { [weak self] _ in self?.layoutMetalLayer() })
#endif
}
func stop() {
@@ -360,6 +412,12 @@ public final class StreamViewController: UIViewController {
streamView.onScroll = nil
streamView.currentHostMode = nil
#endif
#if os(tvOS)
// Return the TV to the user's preferred mode the home screen must not stay in the
// session's HDR10/refresh mode.
sessionDisplayManager?.preferredDisplayCriteria = nil
sessionDisplayManager = nil
#endif
presenter.stop()
connection = nil
}
@@ -367,8 +425,50 @@ public final class StreamViewController: UIViewController {
public override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
layoutMetalLayer()
#if os(tvOS)
applyDisplayCriteriaIfNeeded()
#endif
}
#if os(tvOS)
/// Ask the TV for a display mode matching the session HDR10 at the stream's refresh rate
/// via AVDisplayManager, the tvOS mechanism custom renderers use for HDR output (AVFoundation
/// playback layers do this implicitly). Honored only when the user allows matching (tvOS
/// Settings Video and Audio Match Content); the presenter reads the RESULT off UIScreen's
/// EDR headroom (pushed in SessionPresenter.layout) and keeps the in-shader tone-map whenever
/// the switch never lands, so an SDR-composited display can't show blown-out PQ either way.
/// Applied once per session, as soon as the window and the negotiated mode both exist; the
/// stop() teardown clears it.
private func applyDisplayCriteriaIfNeeded() {
guard let manager = view.window?.avDisplayManager, let connection,
manager.preferredDisplayCriteria == nil,
UserDefaults.standard.object(forKey: DefaultsKey.hdrEnabled) as? Bool ?? true
else { return }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0, mode.refreshHz > 0 else { return }
// A synthetic HDR10-HEVC format description carrying the negotiated mode what the
// stream decodes to. AVDisplayCriteria(refreshRate:formatDescription:) matches the
// display to it (tvOS 17+, our deployment floor).
let ext: [CFString: Any] = [
kCMFormatDescriptionExtension_ColorPrimaries:
kCMFormatDescriptionColorPrimaries_ITU_R_2020,
kCMFormatDescriptionExtension_TransferFunction:
kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ,
kCMFormatDescriptionExtension_YCbCrMatrix:
kCMFormatDescriptionYCbCrMatrix_ITU_R_2020,
]
var desc: CMFormatDescription?
CMVideoFormatDescriptionCreate(
allocator: kCFAllocatorDefault, codecType: kCMVideoCodecType_HEVC,
width: Int32(mode.width), height: Int32(mode.height),
extensions: ext as CFDictionary, formatDescriptionOut: &desc)
guard let desc else { return }
manager.preferredDisplayCriteria = AVDisplayCriteria(
refreshRate: Float(mode.refreshHz), formatDescription: desc)
sessionDisplayManager = manager
}
#endif
/// The display scale to render the metal drawable at. `traitCollection.displayScale` is the
/// canonical render scale and is reliable once the controller is in the hierarchy;
/// `view.contentScaleFactor` can read 1.0 before the view attaches to a window/screen, which