feat(apple): the gamepad UI comes to tvOS - focus-driven, with real session controls
ci / rust (push) Failing after 50s
ci / web (push) Successful in 57s
decky / build-publish (push) Successful in 14s
ci / docs-site (push) Successful in 1m2s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
flatpak / build-publish (push) Failing after 2m14s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m46s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 11m40s
arch / build-publish (push) Successful in 18m10s
deb / build-publish (push) Successful in 18m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m21s
android / android (push) Successful in 21m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 5m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 1m2s
release / apple (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled

The console UI now runs on tvOS through the NATIVE focus engine: carousel
cards and settings rows are focusable Buttons (Siri Remote and pads both
navigate; imperative scrollTo replaces the drop-prone scrollPosition binding),
while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what
focus has no concept of: X/Y screen actions and left/right value adjust with
the poll's dominant-axis feel (onMoveCommand proved input-source-dependent:
keyboard intercepted, pad dpad not -> double steps). Text entry uses the
system fullscreen keyboard (TVTextEntry); pairing + library present as covers
under the launcher; the game library defaults ON; settings values slide a
quiet 14 pt in the step's direction.

Session controls: controller/remote input routes EXCLUSIVELY through
GameController during a stream (GCEventViewController, interaction disabled) -
a pad's B no longer doubles as a UIKit menu press that ended sessions
mid-game. Deliberate exits only: the cross-client escape chord (hold
L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all
Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream
banner (now also on tvOS) teaches both. The Siri Remote's touch surface
drives the host pointer - press = left click, Play/Pause = right click,
release-tail jumps gated so motion stays truly relative.

tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every
unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root
font shrank the whole platform (29 pt there), plus 10-foot sizing across host
cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu"
hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops
instead of suspending; full-width focusSections make the home actions
reachable from any column. The presenter defaults to stage-3 glass pacing on
tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind
the 50 ms display stage) and is pickable from the gamepad settings; HDR
capability advertises from AVPlayer.eligibleForHDRPlayback instead of the
current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-10 16:58:35 +02:00
parent 1fcf9e11ec
commit 3ba19f28a2
29 changed files with 1208 additions and 242 deletions
@@ -36,6 +36,9 @@ import PunktfunkCore
import SwiftUI
import UIKit
import os
#if os(tvOS)
import AVKit // AVDisplayManager the per-session display-mode (HDR10/refresh) request
#endif
/// Same diagnostic switch as InputCapture (PUNKTFUNK_INPUT_DEBUG=1): on iOS we log the
/// resolved pointer-lock state each time capture engages, so the user can see whether the
@@ -108,7 +111,20 @@ public struct StreamView: UIViewControllerRepresentable {
}
}
public final class StreamViewController: UIViewController {
#if os(tvOS)
/// tvOS: a GCEventViewController with `controllerUserInteractionEnabled = false` routes game-
/// controller (and Siri Remote) input EXCLUSIVELY to the GameController framework while the
/// stream is up. Without it a pad's B/Menu press doubles as a UIKit menu press which ended
/// the session (or suspended the whole app) from ordinary gameplay; a SwiftUI
/// `.onExitCommand {}` swallow proved unreliable with nothing focusable on screen. Every
/// in-session exit is GC-level by design: the pad's escape chord (GamepadCapture) and the
/// remote's hold-Back (SiriRemotePointer).
public typealias StreamViewControllerBase = GCEventViewController
#else
public typealias StreamViewControllerBase = UIViewController
#endif
public final class StreamViewController: StreamViewControllerBase {
public private(set) var connection: PunktfunkConnection?
private var observers: [NSObjectProtocol] = []
/// Record the unified latency stages (end-to-end / decode / display) when the stage-2
@@ -119,6 +135,11 @@ public final class StreamViewController: UIViewController {
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
/// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback.
private let presenter = SessionPresenter()
#if os(tvOS)
/// The window's display manager the session's mode request was set on held weakly so
/// stop() can clear the request even after the view has left the window.
private weak var sessionDisplayManager: AVDisplayManager?
#endif
#if os(iOS)
private var inputCapture: InputCapture?
fileprivate var captured = false
@@ -157,6 +178,12 @@ public final class StreamViewController: UIViewController {
public override func loadView() {
view = StreamLayerUIView()
#if os(tvOS)
// Kill the pad/remote UIKit press path at the source for the whole session (see the
// GCEventViewController typealias above). GC delivery is untouched: GamepadCapture
// forwards the pad, SiriRemotePointer drives the pointer and owns the remote exit.
controllerUserInteractionEnabled = false
#endif
// Re-size the stage-2 drawable if the display scale changes without a bounds change (e.g.
// moving to an external display at a different scale) the iOS analogue of macOS's
// viewDidChangeBackingProperties relayout. The handler takes the VC as its argument, so it
@@ -230,6 +257,18 @@ public final class StreamViewController: UIViewController {
}
#endif
#if os(tvOS)
// The GCEventViewController's interaction flag applies to the deepest such controller
// CONTAINING THE FIRST RESPONDER inside SwiftUI's hosting-controller sandwich that is not
// guaranteed to be us unless we anchor the responder chain here explicitly.
public override var canBecomeFirstResponder: Bool { true }
public override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
becomeFirstResponder()
}
#endif
func start(
connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?,
@@ -342,6 +381,19 @@ public final class StreamViewController: UIViewController {
setCaptured(true) // entering a session is the deliberate "capture me" moment
}
#endif
#if os(tvOS)
// The TV's mode switch (requested in applyDisplayCriteriaIfNeeded) completes
// asynchronously, and a dynamic-range-only switch doesn't re-layout by itself
// re-layout on the switch/mode notifications so the presenter sees the new EDR
// headroom immediately (layout pushes UIScreen.currentEDRHeadroom down).
observers.append(NotificationCenter.default.addObserver(
forName: .AVDisplayManagerModeSwitchEnd, object: nil, queue: .main
) { [weak self] _ in self?.layoutMetalLayer() })
observers.append(NotificationCenter.default.addObserver(
forName: UIScreen.modeDidChangeNotification, object: nil, queue: .main
) { [weak self] _ in self?.layoutMetalLayer() })
#endif
}
func stop() {
@@ -360,6 +412,12 @@ public final class StreamViewController: UIViewController {
streamView.onScroll = nil
streamView.currentHostMode = nil
#endif
#if os(tvOS)
// Return the TV to the user's preferred mode the home screen must not stay in the
// session's HDR10/refresh mode.
sessionDisplayManager?.preferredDisplayCriteria = nil
sessionDisplayManager = nil
#endif
presenter.stop()
connection = nil
}
@@ -367,8 +425,50 @@ public final class StreamViewController: UIViewController {
public override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
layoutMetalLayer()
#if os(tvOS)
applyDisplayCriteriaIfNeeded()
#endif
}
#if os(tvOS)
/// Ask the TV for a display mode matching the session HDR10 at the stream's refresh rate
/// via AVDisplayManager, the tvOS mechanism custom renderers use for HDR output (AVFoundation
/// playback layers do this implicitly). Honored only when the user allows matching (tvOS
/// Settings Video and Audio Match Content); the presenter reads the RESULT off UIScreen's
/// EDR headroom (pushed in SessionPresenter.layout) and keeps the in-shader tone-map whenever
/// the switch never lands, so an SDR-composited display can't show blown-out PQ either way.
/// Applied once per session, as soon as the window and the negotiated mode both exist; the
/// stop() teardown clears it.
private func applyDisplayCriteriaIfNeeded() {
guard let manager = view.window?.avDisplayManager, let connection,
manager.preferredDisplayCriteria == nil,
UserDefaults.standard.object(forKey: DefaultsKey.hdrEnabled) as? Bool ?? true
else { return }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0, mode.refreshHz > 0 else { return }
// A synthetic HDR10-HEVC format description carrying the negotiated mode what the
// stream decodes to. AVDisplayCriteria(refreshRate:formatDescription:) matches the
// display to it (tvOS 17+, our deployment floor).
let ext: [CFString: Any] = [
kCMFormatDescriptionExtension_ColorPrimaries:
kCMFormatDescriptionColorPrimaries_ITU_R_2020,
kCMFormatDescriptionExtension_TransferFunction:
kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ,
kCMFormatDescriptionExtension_YCbCrMatrix:
kCMFormatDescriptionYCbCrMatrix_ITU_R_2020,
]
var desc: CMFormatDescription?
CMVideoFormatDescriptionCreate(
allocator: kCFAllocatorDefault, codecType: kCMVideoCodecType_HEVC,
width: Int32(mode.width), height: Int32(mode.height),
extensions: ext as CFDictionary, formatDescriptionOut: &desc)
guard let desc else { return }
manager.preferredDisplayCriteria = AVDisplayCriteria(
refreshRate: Float(mode.refreshHz), formatDescription: desc)
sessionDisplayManager = manager
}
#endif
/// The display scale to render the metal drawable at. `traitCollection.displayScale` is the
/// canonical render scale and is reliable once the controller is in the hierarchy;
/// `view.contentScaleFactor` can read 1.0 before the view attaches to a window/screen, which