feat(apple): the gamepad UI comes to tvOS - focus-driven, with real session controls
apple / swift (push) Waiting to run
apple / screenshots (push) Blocked by required conditions
release / apple (push) Waiting to run
ci / rust (push) Failing after 50s
ci / web (push) Successful in 57s
decky / build-publish (push) Successful in 14s
ci / docs-site (push) Successful in 1m2s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
flatpak / build-publish (push) Failing after 2m14s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m46s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 11m40s
arch / build-publish (push) Successful in 18m10s
deb / build-publish (push) Successful in 18m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m21s
android / android (push) Successful in 21m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 5m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 1m2s
apple / swift (push) Waiting to run
apple / screenshots (push) Blocked by required conditions
release / apple (push) Waiting to run
ci / rust (push) Failing after 50s
ci / web (push) Successful in 57s
decky / build-publish (push) Successful in 14s
ci / docs-site (push) Successful in 1m2s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
flatpak / build-publish (push) Failing after 2m14s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m46s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 11m40s
arch / build-publish (push) Successful in 18m10s
deb / build-publish (push) Successful in 18m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m21s
android / android (push) Successful in 21m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 5m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 1m2s
The console UI now runs on tvOS through the NATIVE focus engine: carousel cards and settings rows are focusable Buttons (Siri Remote and pads both navigate; imperative scrollTo replaces the drop-prone scrollPosition binding), while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what focus has no concept of: X/Y screen actions and left/right value adjust with the poll's dominant-axis feel (onMoveCommand proved input-source-dependent: keyboard intercepted, pad dpad not -> double steps). Text entry uses the system fullscreen keyboard (TVTextEntry); pairing + library present as covers under the launcher; the game library defaults ON; settings values slide a quiet 14 pt in the step's direction. Session controls: controller/remote input routes EXCLUSIVELY through GameController during a stream (GCEventViewController, interaction disabled) - a pad's B no longer doubles as a UIKit menu press that ended sessions mid-game. Deliberate exits only: the cross-client escape chord (hold L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream banner (now also on tvOS) teaches both. The Siri Remote's touch surface drives the host pointer - press = left click, Play/Pause = right click, release-tail jumps gated so motion stays truly relative. tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root font shrank the whole platform (29 pt there), plus 10-foot sizing across host cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu" hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops instead of suspending; full-width focusSections make the home actions reachable from any column. The presenter defaults to stage-3 glass pacing on tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind the 50 ms display stage) and is pickable from the gamepad settings; HDR capability advertises from AVPlayer.eligibleForHDRPlayback instead of the current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -36,6 +36,9 @@ import PunktfunkCore
|
||||
import SwiftUI
|
||||
import UIKit
|
||||
import os
|
||||
#if os(tvOS)
|
||||
import AVKit // AVDisplayManager — the per-session display-mode (HDR10/refresh) request
|
||||
#endif
|
||||
|
||||
/// Same diagnostic switch as InputCapture (PUNKTFUNK_INPUT_DEBUG=1): on iOS we log the
|
||||
/// resolved pointer-lock state each time capture engages, so the user can see whether the
|
||||
@@ -108,7 +111,20 @@ public struct StreamView: UIViewControllerRepresentable {
|
||||
}
|
||||
}
|
||||
|
||||
public final class StreamViewController: UIViewController {
|
||||
#if os(tvOS)
|
||||
/// tvOS: a GCEventViewController with `controllerUserInteractionEnabled = false` routes game-
|
||||
/// controller (and Siri Remote) input EXCLUSIVELY to the GameController framework while the
|
||||
/// stream is up. Without it a pad's B/Menu press doubles as a UIKit menu press — which ended
|
||||
/// the session (or suspended the whole app) from ordinary gameplay; a SwiftUI
|
||||
/// `.onExitCommand {}` swallow proved unreliable with nothing focusable on screen. Every
|
||||
/// in-session exit is GC-level by design: the pad's escape chord (GamepadCapture) and the
|
||||
/// remote's hold-Back (SiriRemotePointer).
|
||||
public typealias StreamViewControllerBase = GCEventViewController
|
||||
#else
|
||||
public typealias StreamViewControllerBase = UIViewController
|
||||
#endif
|
||||
|
||||
public final class StreamViewController: StreamViewControllerBase {
|
||||
public private(set) var connection: PunktfunkConnection?
|
||||
private var observers: [NSObjectProtocol] = []
|
||||
/// Record the unified latency stages (end-to-end / decode / display) when the stage-2
|
||||
@@ -119,6 +135,11 @@ public final class StreamViewController: UIViewController {
|
||||
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
|
||||
/// stage-1 StreamPump → displayLayer path as the Metal-unavailable / DEBUG fallback.
|
||||
private let presenter = SessionPresenter()
|
||||
#if os(tvOS)
|
||||
/// The window's display manager the session's mode request was set on — held weakly so
|
||||
/// stop() can clear the request even after the view has left the window.
|
||||
private weak var sessionDisplayManager: AVDisplayManager?
|
||||
#endif
|
||||
#if os(iOS)
|
||||
private var inputCapture: InputCapture?
|
||||
fileprivate var captured = false
|
||||
@@ -157,6 +178,12 @@ public final class StreamViewController: UIViewController {
|
||||
|
||||
public override func loadView() {
|
||||
view = StreamLayerUIView()
|
||||
#if os(tvOS)
|
||||
// Kill the pad/remote → UIKit press path at the source for the whole session (see the
|
||||
// GCEventViewController typealias above). GC delivery is untouched: GamepadCapture
|
||||
// forwards the pad, SiriRemotePointer drives the pointer and owns the remote exit.
|
||||
controllerUserInteractionEnabled = false
|
||||
#endif
|
||||
// Re-size the stage-2 drawable if the display scale changes without a bounds change (e.g.
|
||||
// moving to an external display at a different scale) — the iOS analogue of macOS's
|
||||
// viewDidChangeBackingProperties relayout. The handler takes the VC as its argument, so it
|
||||
@@ -230,6 +257,18 @@ public final class StreamViewController: UIViewController {
|
||||
}
|
||||
#endif
|
||||
|
||||
#if os(tvOS)
|
||||
// The GCEventViewController's interaction flag applies to the deepest such controller
|
||||
// CONTAINING THE FIRST RESPONDER — inside SwiftUI's hosting-controller sandwich that is not
|
||||
// guaranteed to be us unless we anchor the responder chain here explicitly.
|
||||
public override var canBecomeFirstResponder: Bool { true }
|
||||
|
||||
public override func viewDidAppear(_ animated: Bool) {
|
||||
super.viewDidAppear(animated)
|
||||
becomeFirstResponder()
|
||||
}
|
||||
#endif
|
||||
|
||||
func start(
|
||||
connection: PunktfunkConnection,
|
||||
onFrame: (@Sendable (AccessUnit) -> Void)?,
|
||||
@@ -342,6 +381,19 @@ public final class StreamViewController: UIViewController {
|
||||
setCaptured(true) // entering a session is the deliberate "capture me" moment
|
||||
}
|
||||
#endif
|
||||
|
||||
#if os(tvOS)
|
||||
// The TV's mode switch (requested in applyDisplayCriteriaIfNeeded) completes
|
||||
// asynchronously, and a dynamic-range-only switch doesn't re-layout by itself —
|
||||
// re-layout on the switch/mode notifications so the presenter sees the new EDR
|
||||
// headroom immediately (layout pushes UIScreen.currentEDRHeadroom down).
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
forName: .AVDisplayManagerModeSwitchEnd, object: nil, queue: .main
|
||||
) { [weak self] _ in self?.layoutMetalLayer() })
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
forName: UIScreen.modeDidChangeNotification, object: nil, queue: .main
|
||||
) { [weak self] _ in self?.layoutMetalLayer() })
|
||||
#endif
|
||||
}
|
||||
|
||||
func stop() {
|
||||
@@ -360,6 +412,12 @@ public final class StreamViewController: UIViewController {
|
||||
streamView.onScroll = nil
|
||||
streamView.currentHostMode = nil
|
||||
#endif
|
||||
#if os(tvOS)
|
||||
// Return the TV to the user's preferred mode — the home screen must not stay in the
|
||||
// session's HDR10/refresh mode.
|
||||
sessionDisplayManager?.preferredDisplayCriteria = nil
|
||||
sessionDisplayManager = nil
|
||||
#endif
|
||||
presenter.stop()
|
||||
connection = nil
|
||||
}
|
||||
@@ -367,8 +425,50 @@ public final class StreamViewController: UIViewController {
|
||||
public override func viewDidLayoutSubviews() {
|
||||
super.viewDidLayoutSubviews()
|
||||
layoutMetalLayer()
|
||||
#if os(tvOS)
|
||||
applyDisplayCriteriaIfNeeded()
|
||||
#endif
|
||||
}
|
||||
|
||||
#if os(tvOS)
|
||||
/// Ask the TV for a display mode matching the session — HDR10 at the stream's refresh rate —
|
||||
/// via AVDisplayManager, the tvOS mechanism custom renderers use for HDR output (AVFoundation
|
||||
/// playback layers do this implicitly). Honored only when the user allows matching (tvOS
|
||||
/// Settings → Video and Audio → Match Content); the presenter reads the RESULT off UIScreen's
|
||||
/// EDR headroom (pushed in SessionPresenter.layout) and keeps the in-shader tone-map whenever
|
||||
/// the switch never lands, so an SDR-composited display can't show blown-out PQ either way.
|
||||
/// Applied once per session, as soon as the window and the negotiated mode both exist; the
|
||||
/// stop() teardown clears it.
|
||||
private func applyDisplayCriteriaIfNeeded() {
|
||||
guard let manager = view.window?.avDisplayManager, let connection,
|
||||
manager.preferredDisplayCriteria == nil,
|
||||
UserDefaults.standard.object(forKey: DefaultsKey.hdrEnabled) as? Bool ?? true
|
||||
else { return }
|
||||
let mode = connection.currentMode()
|
||||
guard mode.width > 0, mode.height > 0, mode.refreshHz > 0 else { return }
|
||||
// A synthetic HDR10-HEVC format description carrying the negotiated mode — what the
|
||||
// stream decodes to. AVDisplayCriteria(refreshRate:formatDescription:) matches the
|
||||
// display to it (tvOS 17+, our deployment floor).
|
||||
let ext: [CFString: Any] = [
|
||||
kCMFormatDescriptionExtension_ColorPrimaries:
|
||||
kCMFormatDescriptionColorPrimaries_ITU_R_2020,
|
||||
kCMFormatDescriptionExtension_TransferFunction:
|
||||
kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ,
|
||||
kCMFormatDescriptionExtension_YCbCrMatrix:
|
||||
kCMFormatDescriptionYCbCrMatrix_ITU_R_2020,
|
||||
]
|
||||
var desc: CMFormatDescription?
|
||||
CMVideoFormatDescriptionCreate(
|
||||
allocator: kCFAllocatorDefault, codecType: kCMVideoCodecType_HEVC,
|
||||
width: Int32(mode.width), height: Int32(mode.height),
|
||||
extensions: ext as CFDictionary, formatDescriptionOut: &desc)
|
||||
guard let desc else { return }
|
||||
manager.preferredDisplayCriteria = AVDisplayCriteria(
|
||||
refreshRate: Float(mode.refreshHz), formatDescription: desc)
|
||||
sessionDisplayManager = manager
|
||||
}
|
||||
#endif
|
||||
|
||||
/// The display scale to render the metal drawable at. `traitCollection.displayScale` is the
|
||||
/// canonical render scale and is reliable once the controller is in the hierarchy;
|
||||
/// `view.contentScaleFactor` can read 1.0 before the view attaches to a window/screen, which
|
||||
|
||||
Reference in New Issue
Block a user