feat(apple): the gamepad UI comes to tvOS - focus-driven, with real session controls
apple / swift (push) Waiting to run
apple / screenshots (push) Blocked by required conditions
release / apple (push) Waiting to run
ci / rust (push) Failing after 50s
ci / web (push) Successful in 57s
decky / build-publish (push) Successful in 14s
ci / docs-site (push) Successful in 1m2s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
flatpak / build-publish (push) Failing after 2m14s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m46s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 11m40s
arch / build-publish (push) Successful in 18m10s
deb / build-publish (push) Successful in 18m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m21s
android / android (push) Successful in 21m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 5m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 1m2s

The console UI now runs on tvOS through the NATIVE focus engine: carousel
cards and settings rows are focusable Buttons (Siri Remote and pads both
navigate; imperative scrollTo replaces the drop-prone scrollPosition binding),
while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what
focus has no concept of: X/Y screen actions and left/right value adjust with
the poll's dominant-axis feel (onMoveCommand proved input-source-dependent:
keyboard intercepted, pad dpad not -> double steps). Text entry uses the
system fullscreen keyboard (TVTextEntry); pairing + library present as covers
under the launcher; the game library defaults ON; settings values slide a
quiet 14 pt in the step's direction.

Session controls: controller/remote input routes EXCLUSIVELY through
GameController during a stream (GCEventViewController, interaction disabled) -
a pad's B no longer doubles as a UIKit menu press that ended sessions
mid-game. Deliberate exits only: the cross-client escape chord (hold
L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all
Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream
banner (now also on tvOS) teaches both. The Siri Remote's touch surface
drives the host pointer - press = left click, Play/Pause = right click,
release-tail jumps gated so motion stays truly relative.

tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every
unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root
font shrank the whole platform (29 pt there), plus 10-foot sizing across host
cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu"
hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops
instead of suspending; full-width focusSections make the home actions
reachable from any column. The presenter defaults to stage-3 glass pacing on
tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind
the 50 ms display stage) and is pickable from the gamepad settings; HDR
capability advertises from AVPlayer.eligibleForHDRPlayback instead of the
current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-10 16:58:35 +02:00
parent 1fcf9e11ec
commit 3ba19f28a2
29 changed files with 1208 additions and 242 deletions
@@ -2,6 +2,7 @@
// handshake phase, and the pump-thread main-actor stats relay.
import Foundation
import os
import PunktfunkKit
import SwiftUI
@@ -10,6 +11,15 @@ import SwiftUI
#elseif canImport(UIKit)
import UIKit
#endif
#if os(tvOS)
import AVFoundation // AVPlayer.eligibleForHDRPlayback the TV-capability HDR gate
#endif
/// 1 Hz latency-stage line mirrored to the unified log so the stages can be read WITHOUT the
/// on-screen HUD (Console.app, wirelessly on an iPad/Apple TV). The HUD is not a neutral
/// instrument: any visible overlay forces the metal layer through the compositor, which costs a
/// refresh period on the vsync-latched platforms this is how to measure with it off.
private let statsLog = Logger(subsystem: "io.unom.punktfunk", category: "stats")
/// Pump-thread-side frame counters; a 1 Hz main-actor timer drains them into @Published
/// values. NSLock instead of an actor the writer is the (non-async) pump thread.
@@ -119,6 +129,12 @@ final class SessionModel: ObservableObject {
private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture?
private var gamepadFeedback: GamepadFeedback?
#if os(tvOS)
/// Siri Remote host pointer while streaming (touch surface moves, press = left click,
/// Play/Pause = right click) + the remote's deliberate exit (hold Back 1 s). See
/// SiriRemotePointer same trust gate/lifecycle as the gamepad capture above.
private var remotePointer: SiriRemotePointer?
#endif
var isBusy: Bool { phase != .idle }
@@ -163,6 +179,14 @@ final class SessionModel: ObservableObject {
let displayHDR: Bool = {
#if os(macOS)
return (NSScreen.main?.maximumExtendedDynamicRangeColorComponentValue ?? 1.0) > 1.0
#elseif os(tvOS)
// NOT the EDR headroom here: on tvOS that reflects the CURRENT output mode, and
// Apple's recommended setup runs an SDR home screen with Match Content an
// HDR-capable TV would read 1.0 at connect time and never be advertised. The
// session switches the display to HDR10 itself once streaming (AVDisplayManager
// see StreamViewIOS), so gate on the TV's mode-independent capability; if the
// switch never lands, the presenter's in-shader tone-map keeps PQ safe anyway.
return AVPlayer.eligibleForHDRPlayback
#else
return UIScreen.main.potentialEDRHeadroom > 1.0
#endif
@@ -300,6 +324,10 @@ final class SessionModel: ObservableObject {
// connection is still up); the feedback drain joins off-main like audio.
gamepadCapture?.stop()
gamepadCapture = nil
#if os(tvOS)
remotePointer?.stop() // releases any held click while the connection is still up
remotePointer = nil
#endif
let feedback = gamepadFeedback
gamepadFeedback = nil
if let conn = connection {
@@ -363,11 +391,20 @@ final class SessionModel: ObservableObject {
// session's virtual pad is a DualSense). Same trust gate as audio nothing is
// forwarded during the trust prompt.
let capture = GamepadCapture(connection: conn, manager: .shared)
// The cross-client escape chord (hold L1+R1+Start+Select 1.5 s) on tvOS the only
// controller way out of a stream (B/Menu is swallowed during sessions; see ContentView).
capture.onDisconnectRequest = { [weak self] in self?.disconnect() }
capture.start()
gamepadCapture = capture
let feedback = GamepadFeedback(connection: conn, manager: .shared)
feedback.start()
gamepadFeedback = feedback
#if os(tvOS)
let pointer = SiriRemotePointer(connection: conn)
pointer.onDisconnectRequest = { [weak self] in self?.disconnect() }
pointer.start()
remotePointer = pointer
#endif
}
private func startStatsTimer() {
@@ -429,12 +466,32 @@ final class SessionModel: ObservableObject {
} else {
self.decodeValid = false
}
if let d = self.displayStage.drain() {
let displayWindow = self.displayStage.drain()
if let d = displayWindow {
self.displayP50Ms = d.p50Ms
self.displayValid = true
} else {
self.displayValid = false
}
// Mirror the window to the unified log (see statsLog) one line per second,
// stages in ms, only while frames actually flowed. `fps` counts RECEIVED AUs;
// `presents` counts frames that reached glass (the display meter's sample count)
// a presentsfps gap is the presenter dropping/serializing, an fps deficit is
// upstream (host capture/encode or the network).
if frames > 0 {
let line = String(
format: "fps=%d presents=%d e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%d",
frames,
displayWindow?.count ?? 0,
self.endToEndValid ? self.endToEndP50Ms : -1,
self.endToEndValid ? self.endToEndP95Ms : -1,
self.hostNetworkValid ? self.hostNetworkP50Ms : -1,
self.decodeValid ? self.decodeP50Ms : -1,
self.displayValid ? self.displayP50Ms : -1,
lost)
statsLog.info("\(line, privacy: .public)")
}
}
}
// .common so the HUD keeps updating during window drags / menu tracking.