feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS

Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit:
Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split
along the same seams.

The gamepad mode is couch-complete, and now on macOS too (the living-room
Mac case), not just iOS/iPadOS:

- GamepadSettingsView: a console-style, fully controller-navigable settings
  screen (X from the launcher) — up/down moves focus, left/right steps values
  (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a
  one-line description. Backed by GamepadMenuList, the vertical sibling of
  GamepadCarousel, and SettingsOptions — the option lists hoisted out of
  SettingsView statics and shared by the touch, tvOS and gamepad settings.
- GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad
  — field rows open an on-screen controller keyboard (dpad grid, A types,
  X backspaces, B done); the launcher carousel ends in an Add Host tile, so
  the dead-end "add one with touch first" empty state is gone.
- Launcher polish: contextual hint bar with the pad's real button glyphs,
  controller name + battery chip, one shared console chrome.
- GamepadScreenBackground: an animated aurora (TimelineView-driven drifting
  blobs in the brand's violet family, breathing radii, slow hue shift,
  legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a
  .metal library only bundles reliably in one of the two build systems (SPM vs
  the xcodeproj's synced folders) these sources compile under.
- macOS port: settings/add-host/library present as sized sheets (a macOS sheet
  takes its content's IDEAL size, and the GeometryReader-driven screens
  collapsed to nothing), NSScreen-based mode lists, scroll indicators .never
  (the "always show scroll bars" setting overrides .hidden), tray scrims so
  scrolled rows dim under the pinned title/hints, extra title clearance, and a
  PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/
  library render-verified live on a real Mac + LAN hosts.
- GamepadMenuInput: X button support, and (re)start now snapshots held buttons
  so a controller handoff press never fires twice (the B that closed the
  keyboard no longer also cancels the screen underneath).
- Cleanups: one "Connection failed" alert in ContentView instead of one per
  home screen; HostDiscovery.advertises/unsaved shared by both home screens.
- host: can_encode_444 stub for the non-Linux/Windows host build (the macOS
  synthetic-source loopback used by the Swift tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-02 11:05:10 +02:00
parent e925d00194
commit 133e25849d
84 changed files with 4231 additions and 2698 deletions
+5
View File
@@ -16,5 +16,10 @@
compliance question. -->
<key>ITSAppUsesNonExemptEncryption</key>
<false/>
<!-- Allow CADisplayLink above 60 Hz on ProMotion iPhones: without this key the system
silently caps the link at 60 even when SessionPresenter asks for the stream's rate
via preferredFrameRateRange, so a 120 fps stream would present at half rate. -->
<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>
</dict>
</plist>
+11
View File
@@ -84,6 +84,17 @@ PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkCli
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
On iOS/iPadOS **and macOS** a connected controller swaps the whole home for the **gamepad UI**
(`Home/Gamepad*`, `Settings/GamepadSettingsView`): a console-style host carousel (A connect · Y
library · X settings), a controller-navigable settings screen, an add-host flow with an
on-screen controller keyboard (no touch required anywhere), and the coverflow library browser —
all driven by the shared `GamepadMenuInput` poller + `GamepadCarousel`/`GamepadMenuList` focus
machinery, with dual-channel haptics (device Taptic + controller `MenuHaptics`), over an
animated "aurora" backdrop (`GamepadScreenBackground` — TimelineView-driven drifting color
blobs; deliberately pure SwiftUI, since a .metal library only reliably bundles in one of the
two build systems these sources compile under). macOS presents the settings/add-host screens as
sheets (no `fullScreenCover` there); `PUNKTFUNK_FORCE_GAMEPAD_UI=1` forces the mode without a
physical pad (dev/screenshots).
- **Tests** (`swift test`) — Annex-B units, a real-codec VideoToolbox round trip, DualSense
trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the
remote first-light test.
@@ -55,9 +55,9 @@ struct ContentView: View {
#if !os(macOS)
@State private var showSettings = false
#endif
#if os(iOS)
#if os(iOS) || os(macOS)
// A connected controller (+ the Settings toggle) swaps the whole home screen for
// GamepadHomeView instead of retrofitting HomeView's touch UI see `home` below.
// GamepadHomeView instead of retrofitting HomeView's touch/desktop UI see `home` below.
@ObservedObject private var gamepadManager = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) private var gamepadUIEnabled = true
private var gamepadUIActive: Bool {
@@ -137,12 +137,16 @@ struct ContentView: View {
// The library is a full-screen presentation, not a sheet: on iPad a sheet is a centered page
// card, but the gamepad coverflow is meant to be an immersive, full-bleed screen (and the
// launcher behind it stops consuming the controller see GamepadHomeView's `isActive`).
// macOS has no `fullScreenCover`, so it keeps the sheet there.
// macOS has no `fullScreenCover`, so it keeps the sheet there with an explicit size: a
// macOS sheet takes its content's IDEAL size, and both library layouts are geometry-driven
// (the coverflow is a GeometryReader, ideal zero), so without a frame it collapses to a
// tiny panel.
#if os(macOS)
.sheet(item: $libraryTarget) { host in
NavigationStack {
LibraryView(store: store, host: host, onLaunch: { launchTitle(host, $0) })
}
.frame(minWidth: 940, minHeight: 620)
}
#else
.fullScreenCover(item: $libraryTarget) { host in
@@ -176,6 +180,18 @@ struct ContentView: View {
+ "device in the host's web console (port 3000 → Pairing) — no PIN needed. Or "
+ "pair with the 4-digit PIN it can display.")
}
// One "Connection failed" surface for every home screen (touch grid, gamepad launcher) and
// platform SessionModel funnels all connect/session errors into `errorMessage`.
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
set: { if !$0 { model.errorMessage = nil } })
) {
Button("OK", role: .cancel) {}
} message: {
Text(model.errorMessage ?? "")
}
// The delegated-approval wait: the host holds the connection open until the operator
// approves it. Cancel returns the UI at once; the in-flight connect is left to time out
// and its late result is discarded by SessionModel's connect guard (disconnect resets the
@@ -197,12 +213,21 @@ struct ContentView: View {
private var home: some View {
#if os(macOS)
HomeView(
store: store, model: model, discovery: discovery,
showAddHost: $showAddHost, pairingTarget: $pairingTarget,
speedTestTarget: $speedTestTarget, libraryTarget: $libraryTarget,
connect: { connect($0) }, connectDiscovered: connectDiscovered,
onPaired: handlePaired, onLaunchTitle: launchTitle)
Group {
if gamepadUIActive {
GamepadHomeView(
store: store, model: model, discovery: discovery,
libraryTarget: $libraryTarget,
connect: { connect($0) }, connectDiscovered: connectDiscovered)
} else {
HomeView(
store: store, model: model, discovery: discovery,
showAddHost: $showAddHost, pairingTarget: $pairingTarget,
speedTestTarget: $speedTestTarget, libraryTarget: $libraryTarget,
connect: { connect($0) }, connectDiscovered: connectDiscovered,
onPaired: handlePaired, onLaunchTitle: launchTitle)
}
}
#elseif os(iOS)
Group {
if gamepadUIActive {
@@ -308,7 +333,8 @@ struct ContentView: View {
onSessionEnd: { [weak model] in
Task { @MainActor in model?.sessionEnded() }
},
presentMeter: model.presentLatency
presentMeter: model.presentLatency,
presentTailMeter: model.presentTail
)
.overlay(alignment: placement.alignment) {
if captureEnabled && hudEnabled {
@@ -565,23 +591,3 @@ private struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
private extension Data {
/// Parse an even-length hex string into bytes; nil on any non-hex character or odd length.
/// Used to turn an mDNS-advertised cert fingerprint into a connect pin.
init?(hexString: String) {
let chars = Array(hexString)
guard chars.count.isMultiple(of: 2) else { return nil }
var bytes = [UInt8]()
bytes.reserveCapacity(chars.count / 2)
var i = 0
while i < chars.count {
guard let hi = chars[i].hexDigitValue, let lo = chars[i + 1].hexDigitValue else {
return nil
}
bytes.append(UInt8(hi << 4 | lo))
i += 2
}
self = Data(bytes)
}
}
@@ -0,0 +1,234 @@
// The gamepad-driven "Add Host" screen (iOS/iPadOS/macOS) the controller counterpart of
// AddHostSheet, reached from the launcher's Add Host tile. Three field rows (name / address /
// port) plus the Add action, navigated with the same vertical focus list as the gamepad settings;
// A on a field opens GamepadKeyboard in a bottom tray, so a host can be registered end to end
// without touching the screen. Field edits are live (the row shows every keystroke); B closes the
// keyboard first, then cancels the screen the same "back peels one layer" rule as a console UI.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
struct GamepadAddHostView: View {
@Environment(\.dismiss) private var dismiss
let onAdd: (StoredHost) -> Void
#if os(iOS)
/// `.compact` in a landscape phone window tighter chrome so the keyboard tray still fits.
@Environment(\.verticalSizeClass) private var vSizeClass
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the sheet is sized to fit the tray
#endif
@State private var name = ""
@State private var address = ""
@State private var port = "9777"
@State private var focusID: String?
/// The field row the keyboard tray is editing; nil the row list owns the controller.
@State private var editing: String?
var body: some View {
GamepadMenuList(
items: rows,
focusID: $focusID,
onActivate: { activate(id: $0.id) },
onBack: { dismiss() },
isActive: editing == nil
) { row, focused in
rowView(row, focused: focused)
.frame(maxWidth: 620)
.padding(.horizontal, 24)
}
.frame(maxWidth: .infinity)
.safeAreaInset(edge: .top, spacing: 0) {
VStack(spacing: 4) {
Text("Add Host")
.font(.geist(compact ? 20 : 30, .bold, relativeTo: .title))
.foregroundStyle(.white)
if !compact {
Text("Hosts on this network appear automatically — add one by address "
+ "for everything else.")
.font(.geist(13, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.55))
.multilineTextAlignment(.center)
.frame(maxWidth: 440)
}
}
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.frame(maxWidth: .infinity)
.overlay(alignment: .topTrailing) { closeButton.padding(.trailing, 20) }
.background { GamepadTrayScrim(edge: .top) }
}
.safeAreaInset(edge: .bottom, spacing: 0) {
bottomTray
.padding(.horizontal, 22)
.padding(.vertical, compact ? 6 : 10)
.background { GamepadTrayScrim(edge: .bottom) }
}
.background { GamepadScreenBackground() }
// A port can't exceed 5 digits cap while typing so the row can't grow absurd.
.onChange(of: port) { _, value in
if value.count > 5 { port = String(value.prefix(5)) }
}
}
/// The keyboard tray while editing, the controls legend otherwise.
@ViewBuilder private var bottomTray: some View {
if let editing {
VStack(spacing: 10) {
GamepadKeyboard(
text: editingBinding(editing),
allowed: allowedCharacters(editing),
onDone: { closeKeyboard() })
// Fresh keyboard per field: a touch user can retarget the tray by tapping
// another field row, and the keyboard's input wiring captured the previous
// binding on appear new identity forces a rewire to the new field.
.id(editing)
GamepadHintBar(hints: [
.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Type"),
.init(glyph: buttonGlyph(\.buttonX, fallback: "x.circle"), text: "Delete"),
.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Done"),
])
.frame(maxWidth: .infinity, alignment: .leading)
}
.transition(.move(edge: .bottom).combined(with: .opacity))
} else {
GamepadHintBar(hints: [
.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Select"),
.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Cancel"),
])
.frame(maxWidth: .infinity, alignment: .leading)
}
}
/// Touch/click fallback for closing the controller path is B, a hardware keyboard's Esc
/// rides the cancel action.
private var closeButton: some View {
Button { dismiss() } label: {
Image(systemName: "xmark")
.font(.system(size: 14, weight: .semibold))
.foregroundStyle(.white)
.frame(width: 34, height: 34)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
.buttonStyle(.plain)
.keyboardShortcut(.cancelAction)
.accessibilityLabel("Cancel")
}
// MARK: - Rows
private struct Row: Identifiable {
let id: String
let label: String
var value = ""
var placeholder = ""
var isAction = false
}
private var rows: [Row] {
[
Row(id: "name", label: "Name", value: name, placeholder: "Optional — e.g. Living Room"),
Row(id: "address", label: "Address", value: address, placeholder: "IP or hostname"),
Row(id: "port", label: "Port", value: port, placeholder: "9777"),
Row(id: "add", label: "Add Host", isAction: true),
]
}
private func rowView(_ row: Row, focused: Bool) -> some View {
HStack(spacing: 14) {
if row.isAction {
Label("Add Host", systemImage: "plus.circle.fill")
.font(.geist(16, .semibold, relativeTo: .body))
.foregroundStyle(canAdd ? Color.brand : .white.opacity(0.35))
.frame(maxWidth: .infinity)
} else {
Text(row.label)
.font(.geist(16, .semibold, relativeTo: .body))
.foregroundStyle(.white)
Spacer(minLength: 12)
Text(row.value.isEmpty ? row.placeholder : row.value)
.font(.geistFixed(15, .medium))
.foregroundStyle(row.value.isEmpty ? .white.opacity(0.35) : .white)
.lineLimit(1)
.truncationMode(.head) // keep the end of a long address visible while typing
if editing == row.id {
// The live-edit caret: this row is what the keyboard tray is typing into.
Rectangle()
.fill(Color.brand)
.frame(width: 2, height: 18)
}
}
}
.padding(.horizontal, 16)
.padding(.vertical, 13)
.background {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.fill(.white.opacity(focused || editing == row.id ? 0.1 : 0))
}
.overlay {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.strokeBorder(
editing == row.id ? Color.brand.opacity(0.7) : .white.opacity(focused ? 0.22 : 0),
lineWidth: 1)
}
.scaleEffect(focused ? 1.0 : 0.98)
.animation(.smooth(duration: 0.18), value: focused)
}
// MARK: - Actions
private func activate(id: String) {
switch id {
case "add":
guard canAdd else {
// Not addable yet jump straight to what's missing instead of a dead press.
focusID = "address"
openKeyboard("address")
return
}
onAdd(StoredHost(
name: name.trimmingCharacters(in: .whitespaces),
address: address.trimmingCharacters(in: .whitespaces),
port: UInt16(port) ?? 9777))
dismiss()
default:
openKeyboard(id)
}
}
private var canAdd: Bool {
!address.trimmingCharacters(in: .whitespaces).isEmpty
&& UInt16(port).map { $0 > 0 } == true
}
private func openKeyboard(_ id: String) {
withAnimation(.spring(response: 0.32, dampingFraction: 0.86)) { editing = id }
}
private func closeKeyboard() {
withAnimation(.spring(response: 0.32, dampingFraction: 0.86)) { editing = nil }
}
private func editingBinding(_ id: String) -> Binding<String> {
switch id {
case "name": return $name
case "port": return $port
default: return $address
}
}
/// What the keyboard may type per field: a port is digits, an address never contains spaces;
/// a name is free-form.
private func allowedCharacters(_ id: String) -> CharacterSet? {
switch id {
case "port": return CharacterSet(charactersIn: "0123456789")
case "address": return CharacterSet(charactersIn: " ").inverted
default: return nil
}
}
}
#endif
@@ -1,6 +1,6 @@
// The one piece of gamepad-menu machinery shared by the host launcher (GamepadHomeView) and the
// library coverflow (LibraryCoverflowView): a horizontal, center-snapping carousel driven entirely
// by a controller (iOS/iPadOS only).
// by a controller (iOS/iPadOS/macOS).
//
// The scrolling is pure native SwiftUI `.scrollTargetLayout()` + `.scrollTargetBehavior(.viewAligned)`
// snap exactly one item to center, and symmetric `.safeAreaPadding(.horizontal)` (sized off the live
@@ -24,8 +24,7 @@
import PunktfunkKit
import SwiftUI
#if os(iOS)
import UIKit
#if os(iOS) || os(macOS)
struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hashable {
let items: [Item]
@@ -40,6 +39,8 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
let onActivate: (Item) -> Void
/// Y the screen's secondary action (e.g. open a host's library); nil disables it.
var onSecondary: (() -> Void)?
/// X the screen's tertiary action (e.g. open settings); nil disables it.
var onTertiary: (() -> Void)?
/// B back/dismiss; nil disables it (e.g. the root launcher has nowhere to go back to).
var onBack: (() -> Void)?
/// L1/R1 jump this many items at once (clamped to the ends); 0 disables the shoulders.
@@ -94,7 +95,9 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
}
.scrollPosition(id: $scrolledID)
.scrollTargetBehavior(.viewAligned)
.scrollIndicators(.hidden)
// .never, not .hidden macOS's "always show scroll bars" setting overrides .hidden
// and paints a scroller across the console strip.
.scrollIndicators(.never)
.scrollClipDisabled() // let the focused card scale up past the strip bounds
.safeAreaPadding(.horizontal, inset)
.offset(x: bumpOffset)
@@ -147,6 +150,7 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
input.onMove = { move($0) }
input.onConfirm = { activate() }
input.onSecondary = onSecondary
input.onTertiary = onTertiary
input.onBack = onBack
input.onShoulder = shoulderJump > 0 ? { shoulder(right: $0) } : nil
}
@@ -0,0 +1,232 @@
// Chrome shared by the gamepad-driven screens (GamepadHomeView, GamepadSettingsView,
// GamepadAddHostView, LibraryCoverflowView): the full-bleed console backdrop, the
// controller-glyph hint bar, and the connected-controller status chip. One look across every
// screen is what makes the gamepad UI read as a coherent mode rather than a set of themed pages.
// iOS/iPadOS and macOS (the couch Mac-mini case); tvOS keeps its native focus engine instead.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
import GameController
/// The active controller's real glyph for a button (Xbox "A", DualSense , ) via
/// `sfSymbolsName`; a generic fallback before a controller profile resolves.
/// @MainActor: GamepadManager is main-actor-bound (inside a View body this was implicit).
@MainActor
func buttonGlyph(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, fallback: String
) -> String {
GamepadManager.shared.active?.controller.extendedGamepad?[keyPath: button].sfSymbolsName
?? fallback
}
/// Top padding for a gamepad screen's pinned title. macOS gets extra clearance the launcher
/// title sits right under the window titlebar and the settings/add-host sheets have no titlebar
/// at all, so the iOS value hugs the top edge there.
func gamepadTitleTopPadding(compact: Bool) -> CGFloat {
#if os(macOS)
26
#else
compact ? 4 : 10
#endif
}
/// One glyph + label cell in a hint bar.
struct GamepadHint: Identifiable {
let glyph: String
let text: String
var id: String { glyph + text }
}
/// The pinned controls legend every gamepad screen shows bottom-leading (via `.safeAreaInset`).
/// Same font/spacing everywhere so the legend reads as system chrome, not per-screen decoration.
struct GamepadHintBar: View {
let hints: [GamepadHint]
var body: some View {
HStack(spacing: 18) {
ForEach(hints) { hint in
HStack(spacing: 7) {
Image(systemName: hint.glyph)
.font(.system(size: 19))
.foregroundStyle(.white)
Text(hint.text)
}
.fixedSize() // keep glyph + label together; never truncate a hint mid-word
}
}
.font(.geist(14, .semibold, relativeTo: .subheadline))
.foregroundStyle(.white.opacity(0.85))
}
}
/// The console backdrop: a living "aurora" field in the brand's violet family soft color blobs
/// drifting on slow Lissajous paths over black, in the direction of Apple Music's animated player
/// background but calmer (long 3090 s periods, muted opacities, a legibility scrim on top, so it
/// reads as ambience behind the cards, never as content). Deliberately pure SwiftUI rather than a
/// .metal shader: these sources are built both by SwiftPM (`swift run`/tests) and by the Xcode
/// project's synchronized folders, and a compiled metallib is only reliably bundled in one of the
/// two radial gradients driven by a TimelineView give the same look with none of that risk.
///
/// Applied via `.background { }` NOT as a ZStack sibling so the `.ignoresSafeArea()` here
/// can't inflate the caller's layout past the safe area (see the layout discipline note in
/// GamepadHomeView's header). Honors Reduce Motion by freezing the field at a fixed phase.
struct GamepadScreenBackground: View {
@Environment(\.accessibilityReduceMotion) private var reduceMotion
/// One drifting color blob: a base position + drift ellipse (unit coordinates), angular
/// speeds (rad/s periods of 3090 s), and a radius that slowly breathes.
private struct Blob {
let color: Color
let center: CGPoint
let drift: CGSize
let speed: (x: Double, y: Double)
let phase: (x: Double, y: Double)
/// Radius as a fraction of the view's larger dimension (+ breathing amplitude/speed).
let radius: CGFloat
let breathe: (amount: CGFloat, speed: Double)
let opacity: Double
}
/// The brand violet, a deeper indigo, a warmer plum, and a cool blue related hues so the
/// field shifts within one temperature instead of strobing through the rainbow.
private static let blobs: [Blob] = [
Blob(color: Color(red: 0.53, green: 0.47, blue: 0.96), // brand violet
center: CGPoint(x: 0.30, y: 0.24), drift: CGSize(width: 0.16, height: 0.10),
speed: (0.111, 0.083), phase: (0.0, 1.9),
radius: 0.52, breathe: (0.07, 0.061), opacity: 0.52),
Blob(color: Color(red: 0.24, green: 0.20, blue: 0.72), // deep indigo
center: CGPoint(x: 0.78, y: 0.66), drift: CGSize(width: 0.13, height: 0.14),
speed: (0.071, 0.096), phase: (2.4, 0.7),
radius: 0.58, breathe: (0.08, 0.049), opacity: 0.55),
Blob(color: Color(red: 0.62, green: 0.30, blue: 0.80), // plum
center: CGPoint(x: 0.16, y: 0.82), drift: CGSize(width: 0.12, height: 0.09),
speed: (0.089, 0.067), phase: (4.1, 3.2),
radius: 0.44, breathe: (0.09, 0.078), opacity: 0.42),
Blob(color: Color(red: 0.22, green: 0.38, blue: 0.86), // cool blue
center: CGPoint(x: 0.70, y: 0.12), drift: CGSize(width: 0.10, height: 0.08),
speed: (0.059, 0.104), phase: (1.2, 5.0),
radius: 0.40, breathe: (0.06, 0.055), opacity: 0.38),
]
var body: some View {
Group {
if reduceMotion {
field(at: 0)
} else {
// 30 Hz is plenty for centimeters-per-minute drift, and halves the redraw cost
// of a battery-fed couch device vs. the default display rate.
TimelineView(.animation(minimumInterval: 1.0 / 30.0)) { context in
field(at: context.date.timeIntervalSinceReferenceDate)
}
}
}
.ignoresSafeArea()
}
private func field(at t: TimeInterval) -> some View {
GeometryReader { geo in
let side = max(geo.size.width, geo.size.height)
ZStack {
Color.black
ZStack {
ForEach(Self.blobs.indices, id: \.self) { i in
blobView(Self.blobs[i], at: t, in: geo.size, side: side)
}
}
// ±10° over ~5 min the whole field very slowly warms and cools.
.hueRotation(.degrees(sin(t * 0.021) * 10))
// Composite the additive blobs offscreen once instead of per-layer.
.drawingGroup()
// Legibility scrim: the title (top) and detail/hints (bottom) always sit on
// near-black, whatever the blobs are doing behind them.
LinearGradient(
stops: [
.init(color: .black.opacity(0.55), location: 0),
.init(color: .black.opacity(0.15), location: 0.35),
.init(color: .black.opacity(0.20), location: 0.65),
.init(color: .black.opacity(0.60), location: 1),
],
startPoint: .top, endPoint: .bottom)
}
}
}
private func blobView(_ blob: Blob, at t: TimeInterval, in size: CGSize, side: CGFloat) -> some View {
let x = blob.center.x + blob.drift.width * CGFloat(sin(t * blob.speed.x + blob.phase.x))
let y = blob.center.y + blob.drift.height * CGFloat(cos(t * blob.speed.y + blob.phase.y))
let r = side * blob.radius
* (1 + blob.breathe.amount * CGFloat(sin(t * blob.breathe.speed + blob.phase.x)))
return Circle()
.fill(RadialGradient(
colors: [blob.color, blob.color.opacity(0)],
center: .center, startRadius: 0, endRadius: r / 2))
.frame(width: r, height: r)
.position(x: x * size.width, y: y * size.height)
.opacity(blob.opacity)
.blendMode(.plusLighter)
}
}
/// A darkening scrim behind a pinned tray (a screen title, the hints/detail bar, the keyboard
/// tray): scrollable rows pass beneath those insets, so without this the tray text and the row
/// underneath render interleaved. Fades toward the content so it reads as depth, not a bar.
struct GamepadTrayScrim: View {
let edge: VerticalEdge
var body: some View {
LinearGradient(
stops: [
.init(color: .black.opacity(0.92), location: 0),
.init(color: .black.opacity(0.85), location: 0.55),
.init(color: .black.opacity(0), location: 1),
],
startPoint: edge == .top ? .top : .bottom,
endPoint: edge == .top ? .bottom : .top)
// Grow past the tray so the fade-to-clear happens OUTSIDE its bounds the tray's own
// text always sits on the near-opaque part, rows dim before they reach it.
.padding(edge == .top ? .bottom : .top, -32)
.ignoresSafeArea()
}
}
/// "Which pad is driving this UI" the active controller's name and battery, worn as a quiet
/// chip in the launcher's top bar. Callers observe GamepadManager already, so this re-renders
/// when the pad or its battery state changes.
struct ControllerStatusChip: View {
let controller: GamepadManager.DiscoveredController
var body: some View {
HStack(spacing: 7) {
Image(systemName: controller.hasTouchpadAndMotion
? "playstation.logo" : "gamecontroller.fill")
.font(.system(size: 12))
Text(controller.name)
.lineLimit(1)
if let level = controller.batteryLevel {
Image(systemName: batterySymbol(level))
.font(.system(size: 12))
.foregroundStyle(level <= 0.2 && !controller.isCharging
? AnyShapeStyle(.red) : AnyShapeStyle(.white.opacity(0.7)))
}
}
.font(.geist(12, .medium, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.7))
.padding(.horizontal, 12)
.padding(.vertical, 7)
.background(Capsule().fill(.white.opacity(0.08)))
.overlay(Capsule().strokeBorder(.white.opacity(0.12), lineWidth: 1))
}
private func batterySymbol(_ level: Float) -> String {
if controller.isCharging { return "battery.100.bolt" }
switch level {
case ..<0.125: return "battery.0"
case ..<0.375: return "battery.25"
case ..<0.625: return "battery.50"
case ..<0.875: return "battery.75"
default: return "battery.100"
}
}
}
#endif
@@ -1,8 +1,9 @@
// The gamepad-driven home screen (iOS/iPadOS only): a distinct, "10-foot" console-style host
// launcher shown INSTEAD of HomeView while GamepadUIEnvironment is active a separate screen built
// around a center-snapping carousel of hosts, driven from the couch with a controller. No touch is
// required (a tap still works as a fallback). Scope: browse saved + discovered hosts, connect, and
// when the library flag is on jump into a saved host's library (Y).
// required anywhere: A connects, Y opens a saved host's library (when the flag is on), X opens the
// gamepad settings screen, and the carousel always ends in an Add Host tile that opens the
// controller-keyboard add flow. (A tap still works as a fallback for all of it.)
//
// All the scrolling/snapping/navigation/haptics live in GamepadCarousel; this file is the launcher's
// chrome. Layout discipline (so nothing is EVER clipped, portrait or landscape): the gradient is a
@@ -11,18 +12,21 @@
// status bar / home indicator. As a background it draws behind without affecting layout, so the
// GeometryReader is sized to the safe area. The title and the controller-glyph hints are pinned with
// `.safeAreaInset` (top / bottom-leading) guaranteed inside the safe area and out of the carousel's
// vertical budget and the card is sized off the remaining height. tvOS/macOS never mount this view.
// vertical budget and the card is sized off the remaining height. macOS mounts it too (the
// couch Mac-mini case) same screen, with the settings/add-host covers presented as sheets
// (macOS has no fullScreenCover). tvOS never mounts this view (native focus engine instead).
import PunktfunkKit
import SwiftUI
#if os(iOS)
#if os(iOS) || os(macOS)
import GameController
/// One navigable tile: a saved host or a discovered-but-unsaved one. Hashable so it can be the
/// carousel's scroll-position identity.
/// One navigable tile: a saved host, a discovered-but-unsaved one, or the trailing Add Host
/// action. Hashable so it can be the carousel's scroll-position identity.
private enum GamepadHomeTarget: Hashable {
case saved(UUID)
case discovered(String)
case addHost
}
/// A fully-resolved launcher tile display fields + the activate action, built fresh each render
@@ -31,13 +35,17 @@ private struct HomeTile: Identifiable {
let id: GamepadHomeTarget
let title: String
let subtitle: String
let isOnline: Bool
let isPaired: Bool
let isConnecting: Bool
/// Saved (solid monogram) vs. discovered-but-unsaved (tinted outline).
let filled: Bool
var isOnline = false
var isPaired = false
var isConnecting = false
/// Saved (solid monogram) vs. discovered-but-unsaved / action (tinted outline).
var filled = false
/// Only saved hosts have a library (matches the touch grid's context-menu gate).
let hasLibrary: Bool
var hasLibrary = false
/// Shows this SF symbol in the badge instead of the title monogram (the Add Host tile).
var icon: String?
/// Whether the detail panel shows the online/paired pill (hosts yes, actions no).
var showsStatus = true
let activate: () -> Void
}
@@ -51,12 +59,18 @@ struct GamepadHomeView: View {
/// Same experimental gate the touch grid's "Browse Library" context-menu item uses.
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
#if os(iOS)
/// `.compact` in a landscape phone window drives tighter chrome so everything still fits.
@Environment(\.verticalSizeClass) private var vSizeClass
@State private var selection: GamepadHomeTarget?
@State private var breathe = false
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the window minimum keeps room
#endif
@ObservedObject private var gamepads = GamepadManager.shared
@State private var selection: GamepadHomeTarget?
@State private var showSettings = false
@State private var showAddHost = false
var body: some View {
GeometryReader { geo in
@@ -64,97 +78,70 @@ struct GamepadHomeView: View {
}
// Pinned inside the safe area, out of the carousel's vertical budget never clipped.
.safeAreaInset(edge: .top, spacing: 0) {
titleView
.padding(.top, compact ? 4 : 10)
titleBar
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.frame(maxWidth: .infinity)
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
if !tiles.isEmpty {
hintBar
.padding(.leading, 22)
.padding(.vertical, compact ? 6 : 10)
}
}
.background { background }
.onAppear {
discovery.start()
withAnimation(.easeInOut(duration: 4).repeatForever(autoreverses: true)) { breathe = true }
GamepadHintBar(hints: hints)
.padding(.leading, 22)
.padding(.vertical, compact ? 6 : 10)
}
.background { GamepadScreenBackground() }
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
set: { if !$0 { model.errorMessage = nil } })
) {
Button("OK", role: .cancel) {}
} message: {
Text(model.errorMessage ?? "")
// The settings / add-host screens take over the controller (the carousel's `isActive`
// gate above). iOS presents them full screen the immersive console feel; macOS has no
// fullScreenCover, so they become generously sized sheets over the dimmed launcher.
#if os(macOS)
.sheet(isPresented: $showSettings) {
GamepadSettingsView()
.frame(width: 720, height: 640)
}
.sheet(isPresented: $showAddHost) {
GamepadAddHostView { store.add($0) }
.frame(width: 660, height: 620)
}
.frame(minWidth: 640, minHeight: 420)
#else
.fullScreenCover(isPresented: $showSettings) { GamepadSettingsView() }
.fullScreenCover(isPresented: $showAddHost) {
GamepadAddHostView { store.add($0) }
}
#endif
}
// MARK: - Hero (carousel + detail), sized to fit the space between the pinned title and hints
@ViewBuilder private func hero(for size: CGSize) -> some View {
if tiles.isEmpty {
emptyState.frame(maxWidth: .infinity, maxHeight: .infinity)
} else {
let cardWidth = min(340, size.width * 0.84)
// 96 the carousel's own vertical breathing (+40) plus the detail line (~54); clamp so
// the strip + detail always fit the region the safe-area insets leave.
let cardHeight = min(compact ? 170 : 210, max(118, size.height - 96))
VStack(spacing: compact ? 8 : 10) {
Spacer(minLength: 0)
carousel(cardWidth: cardWidth, cardHeight: cardHeight)
detailPanel
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
let cardWidth = min(340, size.width * 0.84)
// 96 the carousel's own vertical breathing (+40) plus the detail line (~54); clamp so
// the strip + detail always fit the region the safe-area insets leave.
let cardHeight = min(compact ? 170 : 210, max(118, size.height - 96))
VStack(spacing: compact ? 8 : 10) {
Spacer(minLength: 0)
carousel(cardWidth: cardWidth, cardHeight: cardHeight)
detailPanel
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
// MARK: - Chrome
private var background: some View {
ZStack {
LinearGradient(
colors: [.black, Color.brand.opacity(0.22), .black],
startPoint: .top, endPoint: .bottom)
// A soft brand orb behind the strip gives the flat gradient depth; it breathes slowly.
Circle()
.fill(RadialGradient(
colors: [Color.brand.opacity(0.55), .clear],
center: .center, startRadius: 0, endRadius: 300))
.frame(width: 560, height: 560)
.blur(radius: 70)
.scaleEffect(breathe ? 1.08 : 0.92)
.opacity(breathe ? 0.5 : 0.32)
.offset(y: -20)
}
.ignoresSafeArea()
}
private var titleView: some View {
private var titleBar: some View {
Text("Select a Host")
.font(.geist(compact ? 20 : 30, .bold, relativeTo: .title))
.foregroundStyle(.white)
}
private var emptyState: some View {
VStack(spacing: 14) {
Image(systemName: "gamecontroller")
.font(.system(size: 46, weight: .light))
.foregroundStyle(Color.brand)
Text("No hosts yet")
.font(.geist(20, .semibold, relativeTo: .title3))
.foregroundStyle(.white)
Text("Add one with touch first — it'll show up here for the controller.")
.font(.geist(15, relativeTo: .body))
.foregroundStyle(.white.opacity(0.6))
.multilineTextAlignment(.center)
.frame(maxWidth: 320)
}
.frame(maxWidth: .infinity)
.overlay(alignment: .trailing) {
// Which pad is driving this UI (name + battery) quiet, and only where there's
// room; a compact-height phone gives the pixels to the carousel instead.
if !compact, let active = gamepads.active {
ControllerStatusChip(controller: active)
.padding(.trailing, 20)
}
}
}
// MARK: - Carousel
@@ -167,9 +154,10 @@ struct GamepadHomeView: View {
spacing: 30,
onActivate: { $0.activate() },
onSecondary: { openLibraryForSelected() },
// Stop consuming the controller while the library is presented on top otherwise the
// launcher navigates behind it (invisibly on iPhone, visibly on iPad's page sheet).
isActive: libraryTarget == nil
onTertiary: { showSettings = true },
// Stop consuming the controller while another screen is presented on top otherwise
// the launcher navigates behind it (invisibly on iPhone, visibly on iPad's page sheet).
isActive: libraryTarget == nil && !showSettings && !showAddHost
) { tile in
hostCard(tile, size: CGSize(width: cardWidth, height: cardHeight))
}
@@ -211,7 +199,7 @@ struct GamepadHomeView: View {
Text(tile?.subtitle ?? " ")
.font(.geist(13, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.6))
if let tile {
if let tile, tile.showsStatus {
statusPill(online: tile.isOnline, paired: tile.isPaired)
}
}
@@ -236,71 +224,52 @@ struct GamepadHomeView: View {
// MARK: - Hint bar (pinned bottom-leading via safeAreaInset)
private var hintBar: some View {
HStack(spacing: 18) {
hint(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Connect")
if showsLibraryHint {
hint(glyph: buttonGlyph(\.buttonY, fallback: "y.circle"), text: "Library")
}
private var hints: [GamepadHint] {
let selected = tiles.first { $0.id == selection }
var hints = [GamepadHint(
glyph: buttonGlyph(\.buttonA, fallback: "a.circle"),
text: selected?.id == .addHost ? "Add Host" : "Connect")]
if libraryEnabled, selected?.hasLibrary == true {
hints.append(.init(glyph: buttonGlyph(\.buttonY, fallback: "y.circle"), text: "Library"))
}
.font(.geist(14, .semibold, relativeTo: .subheadline))
.foregroundStyle(.white.opacity(0.85))
}
private func hint(glyph: String, text: String) -> some View {
HStack(spacing: 7) {
Image(systemName: glyph)
.font(.system(size: 19))
.foregroundStyle(.white)
Text(text)
}
.fixedSize() // keep glyph + label together; never truncate a hint mid-word
}
private var showsLibraryHint: Bool {
guard libraryEnabled else { return false }
return tiles.first { $0.id == selection }?.hasLibrary ?? false
}
/// The active controller's real glyph for a button (Xbox "A", DualSense , ) via
/// `sfSymbolsName`; a generic fallback before a controller profile resolves.
private func buttonGlyph(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, fallback: String
) -> String {
GamepadManager.shared.active?.controller.extendedGamepad?[keyPath: button].sfSymbolsName
?? fallback
hints.append(.init(glyph: buttonGlyph(\.buttonX, fallback: "x.circle"), text: "Settings"))
return hints
}
// MARK: - Data + actions
/// Built fresh each render from the live stores (no stale value capture) saved hosts first,
/// then discovered-but-unsaved ones.
/// then discovered-but-unsaved ones, then the Add Host action tile (so the strip is never
/// empty and manual entry is always one press away).
private var tiles: [HomeTile] {
let saved = store.hosts.map { host in
HomeTile(
id: .saved(host.id),
title: host.displayName,
subtitle: "\(host.address):\(String(host.port))",
isOnline: isOnline(host),
isOnline: discovery.advertises(host),
isPaired: host.pinnedSHA256 != nil,
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
filled: true,
hasLibrary: true,
activate: { connect(host) })
}
let discovered = discoveredUnsaved.map { d in
let discovered = discovery.unsaved(among: store.hosts).map { d in
HomeTile(
id: .discovered(d.id),
title: d.name,
subtitle: "\(d.host):\(String(d.port))",
isOnline: true,
isPaired: false,
isConnecting: false,
filled: false,
hasLibrary: false,
activate: { connectDiscovered(d) })
}
return saved + discovered
let add = HomeTile(
id: .addHost,
title: "Add Host",
subtitle: "Register a host by address",
icon: "plus",
showsStatus: false,
activate: { showAddHost = true })
return saved + discovered + [add]
}
/// Only saved hosts have a library matches the touch grid, where "Browse Library" is a
@@ -311,14 +280,6 @@ struct GamepadHomeView: View {
else { return }
libraryTarget = host
}
private func isOnline(_ host: StoredHost) -> Bool {
discovery.hosts.contains { host.matches($0) }
}
private var discoveredUnsaved: [DiscoveredHost] {
discovery.hosts.filter { d in !store.hosts.contains { $0.matches(d) } }
}
}
/// One "console tile" in the host carousel a dark-glass landscape card, bigger and bolder than the
@@ -381,6 +342,10 @@ private struct GamepadHostTile: View {
: AnyShapeStyle(Color.brand.opacity(0.16)))
if tile.isConnecting {
ProgressView().tint(.white)
} else if let icon = tile.icon {
Image(systemName: icon)
.font(.system(size: 24, weight: .semibold))
.foregroundStyle(Color.brand)
} else {
Text(monogram(tile.title))
.font(.geistFixed(25, .bold))
@@ -0,0 +1,182 @@
// A controller-driven on-screen keyboard for the gamepad UI's text fields (iOS/iPadOS only)
// iOS has no system keyboard a game controller can drive (the tvOS fullscreen entry doesn't
// exist here), so without this, adding a host from the couch would end with "now touch the
// screen". Dpad/stick moves a key cursor over a fixed grid, A types, X backspaces, B/Y confirms.
// Lowercase + digits + the hostname/address punctuation is deliberately the whole character set:
// these fields hold names, addresses and ports, not prose.
//
// Edits are applied to the binding live (the caller's field row shows every keystroke), so
// closing the keyboard is always "done" there is no separate cancel/commit step to get wrong.
// Touch stays a fallback: every keycap is tappable.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
struct GamepadKeyboard: View {
@Binding var text: String
/// Restricts typed characters (e.g. digits for a port field); backspace always works.
var allowed: CharacterSet?
/// B / Y / the Done key the binding already holds the final text.
let onDone: () -> Void
@State private var input = GamepadMenuInput(manager: .shared)
@State private var haptics = MenuHaptics(manager: .shared)
@State private var cursor = GridPos(row: 1, col: 0) // opens on "q"
@State private var pressTick = 0
@State private var boundaryTick = 0
#if os(iOS)
/// `.compact` (landscape phone): shorter keycaps so the tray leaves room for the field rows.
@Environment(\.verticalSizeClass) private var vSizeClass
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the sheet is sized generously
#endif
private struct GridPos: Hashable {
var row: Int
var col: Int
}
private enum Key: Hashable {
case char(Character)
case space
case backspace
case done
}
/// Digits first (addresses/ports), then letters; the last char column carries the
/// hostname/address punctuation.
private static let rows: [[Key]] = [
Array("1234567890").map(Key.char),
Array("qwertyuiop").map(Key.char),
Array("asdfghjkl-").map(Key.char),
Array("zxcvbnm._:").map(Key.char),
[.space, .backspace, .done],
]
var body: some View {
VStack(spacing: compact ? 5 : 7) {
ForEach(Self.rows.indices, id: \.self) { r in
HStack(spacing: compact ? 5 : 7) {
ForEach(Self.rows[r].indices, id: \.self) { c in
keycap(Self.rows[r][c], focused: cursor == GridPos(row: r, col: c))
.onTapGesture {
cursor = GridPos(row: r, col: c)
press(Self.rows[r][c])
}
}
}
}
}
.frame(maxWidth: 560)
.padding(compact ? 10 : 14)
.background {
RoundedRectangle(cornerRadius: 22, style: .continuous)
.fill(.ultraThinMaterial)
.environment(\.colorScheme, .dark)
}
.overlay {
RoundedRectangle(cornerRadius: 22, style: .continuous)
.strokeBorder(.white.opacity(0.12), lineWidth: 1)
}
.sensoryFeedback(.selection, trigger: cursor)
.sensoryFeedback(.impact(weight: .light), trigger: pressTick)
.sensoryFeedback(.impact(flexibility: .rigid, intensity: 0.7), trigger: boundaryTick)
.onAppear {
wire()
input.start()
}
.onDisappear {
input.stop()
haptics.stop()
}
}
// MARK: - Keycaps
@ViewBuilder private func keycap(_ key: Key, focused: Bool) -> some View {
Group {
switch key {
case .char(let c):
Text(String(c)).font(.geistFixed(18, .medium))
case .space:
Image(systemName: "space")
case .backspace:
Image(systemName: "delete.left")
case .done:
Label("Done", systemImage: "checkmark")
.font(.geist(15, .semibold, relativeTo: .callout))
}
}
.foregroundStyle(focused ? Color.black : .white)
.frame(maxWidth: .infinity, minHeight: compact ? 34 : 42)
.background {
RoundedRectangle(cornerRadius: 9, style: .continuous)
.fill(focused ? AnyShapeStyle(Color.brand) : AnyShapeStyle(.white.opacity(0.08)))
}
.animation(.smooth(duration: 0.12), value: focused)
.contentShape(Rectangle())
}
// MARK: - Input
private func wire() {
input.onMove = { move($0) }
input.onConfirm = { press(Self.rows[cursor.row][cursor.col]) }
input.onTertiary = { press(.backspace) }
input.onSecondary = onDone
input.onBack = onDone
}
private func move(_ direction: GamepadMenuInput.Direction) {
var next = cursor
switch direction {
case .left: next.col -= 1
case .right: next.col += 1
case .up, .down:
let row = cursor.row + (direction == .down ? 1 : -1)
guard row >= 0, row < Self.rows.count else { return refuse() }
// Map the column proportionally between rows of different widths, so e.g. Done
// (rightmost of 3) goes up to the rightmost letters, not to "e".
let from = max(1, Self.rows[cursor.row].count - 1)
let to = Self.rows[row].count - 1
next = GridPos(
row: row,
col: Int((Double(cursor.col) * Double(to) / Double(from)).rounded()))
}
guard next.row >= 0, next.row < Self.rows.count,
next.col >= 0, next.col < Self.rows[next.row].count
else { return refuse() }
cursor = next
haptics.move()
}
private func press(_ key: Key) {
switch key {
case .char(let c):
if let allowed, !c.unicodeScalars.allSatisfy(allowed.contains) { return refuse() }
text.append(c)
case .space:
if let allowed, !allowed.contains(" ") { return refuse() }
text.append(" ")
case .backspace:
guard !text.isEmpty else { return refuse() }
text.removeLast()
case .done:
haptics.confirm()
onDone()
return
}
pressTick &+= 1
haptics.move()
}
/// Refused input (edge of the grid, a disallowed character, deleting nothing).
private func refuse() {
boundaryTick &+= 1
haptics.boundary()
}
}
#endif
@@ -0,0 +1,178 @@
// The vertical sibling of GamepadCarousel (iOS/iPadOS/macOS): a controller-driven focus list for
// the gamepad UI's form-like screens (GamepadSettingsView, GamepadAddHostView). Up/down moves a
// focus bar through the rows, left/right adjusts the focused row's value, A activates it, B backs
// out. The CALLER owns each row's look (it gets the focused flag); this component owns the focus
// cursor, controller polling, haptics, and keeping the focused row scrolled into view.
//
// Unlike the carousel there is no snapping and no `.scrollPosition` two-way binding to fight: the
// cursor is plainly authoritative, the scroll view just chases it with `scrollTo`. Touch stays a
// first-class fallback tapping a row focuses AND activates it (rows are always fully visible, so
// the carousel's "first tap re-centers" step would only add friction here), and free finger
// scrolling is never hijacked back to the focused row until the next controller move.
//
// Feedback is dual-channel like the carousel: `.sensoryFeedback` ticks the DEVICE Taptic engine,
// `MenuHaptics` ticks the CONTROLLER. Moves and value changes get the crisp detent; a refused
// move at either end gets the dull boundary thud plus a short vertical recoil.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
struct GamepadMenuList<Item: Identifiable, Row: View>: View where Item.ID: Hashable {
let items: [Item]
/// Output only: the list WRITES the focused item's id here (e.g. for a caller's hint bar).
@Binding var focusID: Item.ID?
/// Left/right on the focused row. Return whether the value actually changed true plays the
/// move detent, false the boundary thud (end of a clamped range, or nothing to adjust).
var onAdjust: ((Item, Int) -> Bool)?
/// A activate the focused row (toggle it, open it, run it the caller decides).
let onActivate: (Item) -> Void
/// B back/dismiss; nil disables it.
var onBack: (() -> Void)?
/// Whether this list currently owns controller input same handoff contract as
/// GamepadCarousel's `isActive` (a covered screen must stop polling the shared pad).
var isActive: Bool = true
@ViewBuilder let row: (Item, _ focused: Bool) -> Row
@State private var input = GamepadMenuInput(manager: .shared)
@State private var haptics = MenuHaptics(manager: .shared)
/// Authoritative focus cursor (index into `items`).
@State private var cursor = 0
/// A short vertical recoil when a move is refused at a list end.
@State private var bumpOffset: CGFloat = 0
/// `.sensoryFeedback` counters (see GamepadCarousel): device ticks for activate / value-change
/// / end-stop events; moves trigger on `cursor` itself.
@State private var activateTick = 0
@State private var adjustTick = 0
@State private var boundaryTick = 0
var body: some View {
ScrollViewReader { proxy in
ScrollView(.vertical) {
LazyVStack(spacing: 6) {
ForEach(Array(items.enumerated()), id: \.element.id) { idx, item in
row(item, idx == cursor && isActive)
.contentShape(Rectangle())
.onTapGesture { tap(idx) }
.id(item.id)
}
}
.padding(.vertical, 10)
}
// .never, not .hidden macOS's "always show scroll bars" setting overrides .hidden.
.scrollIndicators(.never)
.offset(y: bumpOffset)
.onChange(of: cursor) { _, newValue in
guard newValue >= 0, newValue < items.count else { return }
withAnimation(.easeOut(duration: 0.2)) {
proxy.scrollTo(items[newValue].id)
}
}
}
.sensoryFeedback(.selection, trigger: cursor)
.sensoryFeedback(.selection, trigger: adjustTick)
.sensoryFeedback(.impact(weight: .medium), trigger: activateTick)
.sensoryFeedback(.impact(flexibility: .rigid, intensity: 0.7), trigger: boundaryTick)
.onAppear {
reconcile()
wire()
if isActive { input.start() }
}
.onDisappear {
input.stop()
haptics.stop()
}
.onChange(of: isActive) { _, active in
if active {
wire()
input.start()
} else {
input.stop()
haptics.stop()
}
}
// Re-seed a dropped focus AND re-wire the input callbacks so they capture the current
// `items` value (a plain array it would otherwise go stale in the stored closures).
.onChange(of: items.map(\.id)) { _, _ in
reconcile()
wire()
}
}
// MARK: - Input wiring
private func wire() {
input.onMove = { direction in
switch direction {
case .up: step(by: -1)
case .down: step(by: 1)
case .left: adjust(by: -1)
case .right: adjust(by: 1)
}
}
input.onConfirm = { activate() }
input.onBack = onBack
}
private func step(by delta: Int) {
guard !items.isEmpty else { return }
let target = cursor + delta
guard target >= 0, target < items.count else { return boundaryBump(forward: delta > 0) }
cursor = target
focusID = items[target].id
haptics.move()
}
private func adjust(by delta: Int) {
guard let onAdjust, cursor >= 0, cursor < items.count else { return }
if onAdjust(items[cursor], delta) {
adjustTick &+= 1
haptics.move()
} else {
boundaryTick &+= 1
haptics.boundary()
}
}
private func activate() {
guard cursor >= 0, cursor < items.count else { return }
activateTick &+= 1
haptics.confirm()
onActivate(items[cursor])
}
/// Touch fallback: a tap focuses the row and activates it in one go.
private func tap(_ idx: Int) {
guard idx >= 0, idx < items.count else { return }
if cursor != idx {
cursor = idx
focusID = items[idx].id
}
activate()
}
/// Keep `cursor`/`focusID` consistent with `items`: seed on appear; on a list change keep the
/// same focused item when it survives, else clamp the cursor into range.
private func reconcile() {
guard !items.isEmpty else {
cursor = 0
if focusID != nil { focusID = nil }
return
}
if let id = focusID, let idx = items.firstIndex(where: { $0.id == id }) {
cursor = idx
} else {
cursor = min(max(cursor, 0), items.count - 1)
focusID = items[cursor].id
}
}
private func boundaryBump(forward: Bool) {
boundaryTick &+= 1
haptics.boundary()
let recoil: CGFloat = forward ? -14 : 14
withAnimation(.spring(response: 0.16, dampingFraction: 0.42)) { bumpOffset = recoil }
withAnimation(.spring(response: 0.34, dampingFraction: 0.7).delay(0.1)) { bumpOffset = 0 }
}
}
#endif
@@ -137,17 +137,6 @@ struct HomeView: View {
}
#endif
#endif
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
set: { if !$0 { model.errorMessage = nil } }
)
) {
Button("OK", role: .cancel) {}
} message: {
Text(model.errorMessage ?? "")
}
}
// MARK: - Cards
@@ -156,7 +145,7 @@ struct HomeView: View {
let onBrowseLibrary: (() -> Void)? = libraryEnabled ? { libraryTarget = host } : nil
return HostCardView(
host: host,
isOnline: isOnline(host),
isOnline: discovery.advertises(host),
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
isMostRecent: host.id == mostRecentHostID,
isBusy: model.isBusy,
@@ -186,18 +175,10 @@ struct HomeView: View {
.padding(.top, store.hosts.isEmpty ? 0 : 8)
}
/// A saved host is "online" iff a live mDNS advert currently matches it (see
/// `StoredHost.matches`). Recomputed on every discovery change (the @Published set), so the
/// dot tracks hosts appearing/leaving the network live.
private func isOnline(_ host: StoredHost) -> Bool {
discovery.hosts.contains { host.matches($0) }
}
/// Discovered hosts not already saved the saved grid shows the rest, so this section only
/// surfaces genuinely-new hosts on the network. Same match as the online dot, so a saved host
/// whose IP changed (still fingerprint-matched) doesn't also appear here as a stranger.
/// Discovered hosts not already saved (see `HostDiscovery.unsaved` shared with the gamepad
/// launcher so both screens classify hosts identically).
private var discoveredUnsaved: [DiscoveredHost] {
discovery.hosts.filter { d in !store.hosts.contains { $0.matches(d) } }
discovery.unsaved(among: store.hosts)
}
/// The host of the most recent session its card carries the accent ring.
@@ -1,4 +1,4 @@
// The gamepad-driven presentation of the game library (iOS/iPadOS only see LibraryView's
// The gamepad-driven presentation of the game library (iOS/iPadOS/macOS see LibraryView's
// `gamepadUIActive` branch): a classic coverflow instead of the touch grid. All the
// scrolling/snapping/navigation/haptics live in GamepadCarousel; this file is the coverflow card
// (poster + the 3D recede treatment via `.scrollTransition`), the "now focused" detail panel, and
@@ -15,9 +15,8 @@
import PunktfunkKit
import SwiftUI
#if os(iOS)
#if os(iOS) || os(macOS)
import GameController
import UIKit
struct LibraryCoverflowView: View {
let games: [GameEntry]
@@ -27,27 +26,26 @@ struct LibraryCoverflowView: View {
/// Close button already covers that); this is what makes gamepad-only exit possible.
var onDismiss: (() -> Void)?
#if os(iOS)
/// `.compact` in a landscape phone window drives a tighter poster so everything still fits.
@Environment(\.verticalSizeClass) private var vSizeClass
@State private var selection: String?
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS
#endif
@State private var selection: String?
var body: some View {
GeometryReader { geo in
content(for: geo.size)
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
hintBar
GamepadHintBar(hints: hints)
.padding(.leading, 22)
.padding(.vertical, compact ? 6 : 10)
}
.background {
LinearGradient(
colors: [.black, Color.brand.opacity(0.16), .black],
startPoint: .top, endPoint: .bottom)
.ignoresSafeArea()
}
.background { GamepadScreenBackground() }
}
@ViewBuilder private func content(for size: CGSize) -> some View {
@@ -138,34 +136,13 @@ struct LibraryCoverflowView: View {
// MARK: - Hint bar (pinned bottom-leading via safeAreaInset)
private var hintBar: some View {
HStack(spacing: 18) {
if onLaunch != nil {
hint(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Launch")
}
hint(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Close")
private var hints: [GamepadHint] {
var hints: [GamepadHint] = []
if onLaunch != nil {
hints.append(.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Launch"))
}
.font(.geist(14, .semibold, relativeTo: .subheadline))
.foregroundStyle(.white.opacity(0.85))
}
private func hint(glyph: String, text: String) -> some View {
HStack(spacing: 7) {
Image(systemName: glyph)
.font(.system(size: 19))
.foregroundStyle(.white)
Text(text)
}
.fixedSize() // keep glyph + label together; never truncate a hint mid-word
}
/// The active controller's real glyph for a button (Xbox "B", DualSense , ) via
/// `sfSymbolsName`; a generic fallback before a controller profile resolves.
private func buttonGlyph(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, fallback: String
) -> String {
GamepadManager.shared.active?.controller.extendedGamepad?[keyPath: button].sfSymbolsName
?? fallback
hints.append(.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Close"))
return hints
}
}
#endif
@@ -5,11 +5,6 @@
import PunktfunkKit
import SwiftUI
#if canImport(UIKit)
import UIKit
#elseif canImport(AppKit)
import AppKit
#endif
struct LibraryView: View {
@ObservedObject var store: HostStore
@@ -26,9 +21,9 @@ struct LibraryView: View {
/// list fetch, reused across every poster in the grid). Built alongside `games` in `load()`;
/// torn down on disappear since it isn't one-shot like `LibraryClient.fetch`'s own session.
@State private var imageSession: URLSession?
#if os(iOS)
// Gamepad-driven browsing is iOS/iPadOS-only see HomeView's identical gate. tvOS keeps its
// existing plain-grid presentation of this same view unchanged.
#if os(iOS) || os(macOS)
// Gamepad-driven browsing (iOS/iPadOS/macOS) see ContentView's identical gate. tvOS keeps
// its existing plain-grid presentation of this same view unchanged.
@ObservedObject private var gamepadManager = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) private var gamepadUIEnabled = true
private var gamepadUIActive: Bool {
@@ -74,7 +69,7 @@ struct LibraryView: View {
} else if games.isEmpty {
emptyState
} else {
#if os(iOS)
#if os(iOS) || os(macOS)
if gamepadUIActive {
LibraryCoverflowView(
games: games, imageSession: imageSession, onLaunch: onLaunch,
@@ -202,88 +197,3 @@ private struct GameCard: View {
}
}
}
/// The store-provenance badge (Steam vs. a user-curated custom entry) overlaid on a poster
/// shared by the touch grid's `GameCard` and the gamepad coverflow's cover cell.
struct StoreBadge: View {
let isCustom: Bool
var body: some View {
Text(isCustom ? "Custom" : "Steam")
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 6)
.padding(.vertical, 3)
.background(.ultraThinMaterial, in: Capsule())
.padding(6)
}
}
#if canImport(UIKit)
private typealias PlatformImage = UIImage
#elseif canImport(AppKit)
private typealias PlatformImage = NSImage
#endif
private extension Image {
init(platformImage: PlatformImage) {
#if canImport(UIKit)
self.init(uiImage: platformImage)
#elseif canImport(AppKit)
self.init(nsImage: platformImage)
#endif
}
}
/// Sequentially tries cover-art URLs over `session` (so a paired client can reach the host's own
/// art proxy, not just public CDNs see `LibraryImageLoader`), advancing past any that fail to
/// load, then a placeholder. The loaded image is hard-clipped to fill the card's actual frame
/// regardless of its own aspect ratio: a portrait capsule fills it as intended, but a fallback
/// banner (wide hero/header art, used when a title has no portrait capsule) would otherwise report
/// a much wider intrinsic size than the card and overflow into neighboring cards. Not `private`
/// the gamepad coverflow (`LibraryCoverflowView`) reuses it directly rather than re-fetching art.
struct PosterImage: View {
let candidates: [URL]
let title: String
let session: URLSession?
@State private var index = 0
@State private var image: PlatformImage?
var body: some View {
Group {
if let image {
Image(platformImage: image)
.resizable()
.scaledToFill()
} else if index < candidates.count {
ZStack { placeholder; ProgressView() }
} else {
placeholder
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.clipped()
.task(id: index) { await loadCurrent() }
}
private func loadCurrent() async {
guard index < candidates.count else { return }
guard let session, let data = try? await session.data(from: candidates[index]).0,
let loaded = PlatformImage(data: data)
else {
index += 1 // advance to the next candidate (or past the end placeholder)
return
}
image = loaded
}
private var placeholder: some View {
ZStack {
Rectangle().fill(.quaternary)
Text(title)
.font(.geist(17, .semibold, relativeTo: .headline))
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
.padding(8)
}
}
}
@@ -0,0 +1,95 @@
// Reusable library widgets, shared by the touch grid (LibraryView's `GameCard`) and the gamepad
// coverflow (LibraryCoverflowView's cover cell).
import PunktfunkKit
import SwiftUI
#if canImport(UIKit)
import UIKit
#elseif canImport(AppKit)
import AppKit
#endif
/// The store-provenance badge (Steam vs. a user-curated custom entry) overlaid on a poster
/// shared by the touch grid's `GameCard` and the gamepad coverflow's cover cell.
struct StoreBadge: View {
let isCustom: Bool
var body: some View {
Text(isCustom ? "Custom" : "Steam")
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 6)
.padding(.vertical, 3)
.background(.ultraThinMaterial, in: Capsule())
.padding(6)
}
}
#if canImport(UIKit)
private typealias PlatformImage = UIImage
#elseif canImport(AppKit)
private typealias PlatformImage = NSImage
#endif
private extension Image {
init(platformImage: PlatformImage) {
#if canImport(UIKit)
self.init(uiImage: platformImage)
#elseif canImport(AppKit)
self.init(nsImage: platformImage)
#endif
}
}
/// Sequentially tries cover-art URLs over `session` (so a paired client can reach the host's own
/// art proxy, not just public CDNs see `LibraryImageLoader`), advancing past any that fail to
/// load, then a placeholder. The loaded image is hard-clipped to fill the card's actual frame
/// regardless of its own aspect ratio: a portrait capsule fills it as intended, but a fallback
/// banner (wide hero/header art, used when a title has no portrait capsule) would otherwise report
/// a much wider intrinsic size than the card and overflow into neighboring cards. Not `private`
/// the gamepad coverflow (`LibraryCoverflowView`) reuses it directly rather than re-fetching art.
struct PosterImage: View {
let candidates: [URL]
let title: String
let session: URLSession?
@State private var index = 0
@State private var image: PlatformImage?
var body: some View {
Group {
if let image {
Image(platformImage: image)
.resizable()
.scaledToFill()
} else if index < candidates.count {
ZStack { placeholder; ProgressView() }
} else {
placeholder
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.clipped()
.task(id: index) { await loadCurrent() }
}
private func loadCurrent() async {
guard index < candidates.count else { return }
guard let session, let data = try? await session.data(from: candidates[index]).0,
let loaded = PlatformImage(data: data)
else {
index += 1 // advance to the next candidate (or past the end placeholder)
return
}
image = loaded
}
private var placeholder: some View {
ZStack {
Rectangle().fill(.quaternary)
Text(title)
.font(.geist(17, .semibold, relativeTo: .headline))
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
.padding(8)
}
}
}
@@ -0,0 +1,35 @@
// The HUD-corner model persisted by Settings and read wherever the overlay is placed
// (ContentView, StreamHUDView).
import SwiftUI
/// Which corner the HUD overlay occupies (persisted as `DefaultsKey.hudPlacement`). The raw
/// values are stable on disk rename the cases freely, never the strings.
enum HUDPlacement: String, CaseIterable, Identifiable {
case topLeading, topTrailing, bottomLeading, bottomTrailing
var id: String { rawValue }
/// SwiftUI overlay alignment for `.overlay(alignment:)`.
var alignment: Alignment {
switch self {
case .topLeading: return .topLeading
case .topTrailing: return .topTrailing
case .bottomLeading: return .bottomLeading
case .bottomTrailing: return .bottomTrailing
}
}
/// The HUD's own stack hugs the screen edge it sits against, so its text aligns outward.
var isTrailing: Bool { self == .topTrailing || self == .bottomTrailing }
/// User-facing corner label.
var label: String {
switch self {
case .topLeading: return "Top Left"
case .topTrailing: return "Top Right"
case .bottomLeading: return "Bottom Left"
case .bottomTrailing: return "Bottom Right"
}
}
}
@@ -74,6 +74,11 @@ final class SessionModel: ObservableObject {
@Published var presentLatencyP95Ms = 0.0
@Published var presentLatencyValid = false
@Published var presentLatencySkewCorrected = false
/// Decode-completionpresent (the "present tail": ring wait + render + vsync) the term the
/// stage-2 presenter exists to shorten. Both instants are client-side, so no skew applies.
@Published var presentTailP50Ms = 0.0
@Published var presentTailP95Ms = 0.0
@Published var presentTailValid = false
/// Mirrors StreamView's capture state (it owns the input capture; this drives the
/// HUD's "click to capture" / " releases" hint).
@Published var mouseCaptured = false
@@ -82,6 +87,8 @@ final class SessionModel: ObservableObject {
let latency = LatencyMeter()
/// Fed by the stage-2 presenter's display link (capturepresent). Passed to StreamView.
let presentLatency = LatencyMeter()
/// Fed by the same present stamp (decode-completionpresent). Passed to StreamView.
let presentTail = LatencyMeter()
private var statsTimer: Timer?
private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture?
@@ -337,6 +344,13 @@ final class SessionModel: ObservableObject {
} else {
self.presentLatencyValid = false
}
if let t = self.presentTail.drain() {
self.presentTailP50Ms = t.p50Ms
self.presentTailP95Ms = t.p95Ms
self.presentTailValid = true
} else {
self.presentTailValid = false
}
}
}
// .common so the HUD keeps updating during window drags / menu tracking.
@@ -4,37 +4,6 @@
import PunktfunkKit
import SwiftUI
/// Which corner the HUD overlay occupies (persisted as `DefaultsKey.hudPlacement`). The raw
/// values are stable on disk rename the cases freely, never the strings.
enum HUDPlacement: String, CaseIterable, Identifiable {
case topLeading, topTrailing, bottomLeading, bottomTrailing
var id: String { rawValue }
/// SwiftUI overlay alignment for `.overlay(alignment:)`.
var alignment: Alignment {
switch self {
case .topLeading: return .topLeading
case .topTrailing: return .topTrailing
case .bottomLeading: return .bottomLeading
case .bottomTrailing: return .bottomTrailing
}
}
/// The HUD's own stack hugs the screen edge it sits against, so its text aligns outward.
var isTrailing: Bool { self == .topTrailing || self == .bottomTrailing }
/// User-facing corner label.
var label: String {
switch self {
case .topLeading: return "Top Left"
case .topTrailing: return "Top Right"
case .bottomLeading: return "Bottom Left"
case .bottomTrailing: return "Bottom Right"
}
}
}
struct StreamHUDView: View {
@ObservedObject var model: SessionModel
let connection: PunktfunkConnection
@@ -63,6 +32,13 @@ struct StreamHUDView: View {
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
if model.presentTailValid {
// Decodepresent (the client-local "present tail": ring wait + render + vsync)
// the term the stage-2 presenter shortens; no skew applies (one clock).
Text("decode→present \(model.presentTailP50Ms, specifier: "%.1f")/\(model.presentTailP95Ms, specifier: "%.1f") ms p50/p95")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
// While captured the cursor is hidden+frozen, so the button is keyboard-only
// ( or Cmd+Tab release the cursor; released, it's clickable again).
#if os(macOS)
@@ -71,11 +47,6 @@ struct StreamHUDView: View {
: "Click the stream to capture input")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
// The client-side cursor (C) draws the local cursor over the stream instead of
// capturing it the only accurate cursor for gamescope, whose capture has none.
Text("⌘⇧C toggles the on-screen cursor")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#elseif os(iOS)
// Touch always plays directly; (hardware keyboard) toggles kb/mouse.
Text(model.mouseCaptured
@@ -0,0 +1,357 @@
// The gamepad-driven settings screen (iOS/iPadOS/macOS): the couch-relevant subset of SettingsView,
// restyled as a console settings page and fully navigable with a controller up/down moves the
// focus bar, left/right steps the focused value, A cycles/toggles it, B closes. Shown from the
// gamepad home launcher (X); the touch SettingsView remains the full-fidelity editor (custom
// resolutions, the log bitrate slider, debug tools), and both write the same DefaultsKey storage,
// so values round-trip freely between the two.
//
// Rows are rebuilt from live @AppStorage on every render; the focus list dispatches adjust/
// activate back here BY ROW ID (see `adjust`/`activate`), so a stored input callback can never act
// on stale captured state. Left/right CLAMPS at a choice list's ends (the dull boundary thud tells
// the thumb it's the last option); A always cycles forward, wrapping, so every option is reachable
// with one button. Toggles read left = off, right = on refusing a no-op with the same thud.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
import GameController
struct GamepadSettingsView: View {
@Environment(\.dismiss) private var dismiss
@AppStorage(DefaultsKey.streamWidth) private var width = 1920
@AppStorage(DefaultsKey.streamHeight) private var height = 1080
@AppStorage(DefaultsKey.streamHz) private var hz = 60
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) private var enable444 = true
@AppStorage(DefaultsKey.codec) private var codec = "auto"
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
@AppStorage(DefaultsKey.gamepadUIEnabled) private var gamepadUIEnabled = true
@ObservedObject private var gamepads = GamepadManager.shared
#if os(iOS)
/// `.compact` in a landscape phone window tighter chrome so more rows fit.
@Environment(\.verticalSizeClass) private var vSizeClass
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the sheet is sized generously
#endif
@State private var focusID: String?
var body: some View {
GamepadMenuList(
items: rows,
focusID: $focusID,
onAdjust: { row, delta in adjust(id: row.id, by: delta) },
onActivate: { activate(id: $0.id) },
onBack: { dismiss() }
) { row, focused in
rowView(row, focused: focused)
.frame(maxWidth: 620)
.padding(.horizontal, 24)
}
.frame(maxWidth: .infinity)
.safeAreaInset(edge: .top, spacing: 0) {
Text("Settings")
.font(.geist(compact ? 20 : 30, .bold, relativeTo: .title))
.foregroundStyle(.white)
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.frame(maxWidth: .infinity)
.overlay(alignment: .trailing) { closeButton.padding(.trailing, 20) }
.background { GamepadTrayScrim(edge: .top) }
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
VStack(alignment: .leading, spacing: 8) {
Text(focusedDetail)
.font(.geist(13, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.55))
.lineLimit(2, reservesSpace: true)
.animation(.smooth(duration: 0.2), value: focusID)
GamepadHintBar(hints: [
.init(glyph: "arrow.left.and.right", text: "Adjust"),
.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Change"),
.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Done"),
])
}
.padding(.leading, 22)
.padding(.trailing, 22)
.padding(.vertical, compact ? 6 : 10)
.frame(maxWidth: .infinity, alignment: .leading)
.background { GamepadTrayScrim(edge: .bottom) }
}
.background { GamepadScreenBackground() }
.onAppear {
gamepads.refresh()
gamepads.startDiscovery()
}
.onDisappear { gamepads.stopDiscovery() }
}
/// Touch/click fallback for closing the controller path is B, a hardware keyboard's Esc
/// rides the cancel action.
private var closeButton: some View {
Button { dismiss() } label: {
Image(systemName: "xmark")
.font(.system(size: 14, weight: .semibold))
.foregroundStyle(.white)
.frame(width: 34, height: 34)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
.buttonStyle(.plain)
.keyboardShortcut(.cancelAction)
.accessibilityLabel("Close settings")
}
// MARK: - Row rendering
private func rowView(_ row: Row, focused: Bool) -> some View {
VStack(alignment: .leading, spacing: 6) {
if let header = row.header {
Text(header)
.font(.geist(12, .semibold, relativeTo: .caption))
.tracking(1.4)
.foregroundStyle(.white.opacity(0.45))
.padding(.leading, 16)
.padding(.top, 14)
}
HStack(spacing: 14) {
Image(systemName: row.icon)
.font(.system(size: 17))
.foregroundStyle(focused ? Color.brand : .white.opacity(0.55))
.frame(width: 28)
Text(row.label)
.font(.geist(16, .semibold, relativeTo: .body))
.foregroundStyle(.white)
.lineLimit(1)
Spacer(minLength: 12)
HStack(spacing: 9) {
Image(systemName: "chevron.left")
.font(.system(size: 12, weight: .semibold))
.foregroundStyle(.white.opacity(focused ? 0.6 : 0))
Text(row.value)
.font(.geist(15, .medium, relativeTo: .callout))
.foregroundStyle(focused ? .white : .white.opacity(0.6))
.lineLimit(1)
Image(systemName: "chevron.right")
.font(.system(size: 12, weight: .semibold))
.foregroundStyle(.white.opacity(focused ? 0.6 : 0))
}
}
.padding(.horizontal, 16)
.padding(.vertical, 13)
.background {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.fill(.white.opacity(focused ? 0.1 : 0))
}
.overlay {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.strokeBorder(.white.opacity(focused ? 0.22 : 0), lineWidth: 1)
}
.scaleEffect(focused ? 1.0 : 0.98)
.animation(.smooth(duration: 0.18), value: focused)
}
}
private var focusedDetail: String {
rows.first { $0.id == focusID }?.detail ?? " "
}
// MARK: - Row model
private struct Row: Identifiable {
let id: String
/// Section header drawn above this row (the first row of each group carries it).
var header: String?
let icon: String
let label: String
let value: String
/// One-line explanation shown near the hint bar while this row is focused.
let detail: String
/// Left/right step; returns whether the value actually changed (false boundary thud).
let adjust: (Int) -> Bool
/// A cycle forward (wrapping) / flip.
let activate: () -> Void
}
/// Dispatch by id so the focus list's stored input callbacks always act on freshly built rows
/// (never on state captured at wire time).
private func adjust(id: String, by delta: Int) -> Bool {
rows.first { $0.id == id }?.adjust(delta) ?? false
}
private func activate(id: String) {
rows.first { $0.id == id }?.activate()
}
private var rows: [Row] {
let resolution = resolutionOptions
let refresh = SettingsOptions.refreshRates(including: hz)
.map { (label: "\($0) Hz", tag: $0) }
let bitrate = SettingsOptions.bitrateOptions(current: bitrateKbps)
let controllers = SettingsOptions.controllerOptions(gamepads)
return [
choiceRow(
id: "resolution", header: "Stream", icon: "aspectratio",
label: "Resolution",
detail: "The host creates a virtual display at exactly this size — no scaling.",
options: resolution, current: "\(width)x\(height)"
) { tag in
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 2 else { return }
width = parts[0]
height = parts[1]
},
choiceRow(
id: "refresh", icon: "gauge.with.needle", label: "Refresh rate",
detail: "Rates this display can actually show.",
options: refresh, current: hz
) { hz = $0 },
choiceRow(
id: "bitrate", icon: "speedometer", label: "Bitrate",
detail: "Automatic uses the host's default (20 Mbps). "
+ "Run a speed test from the touch UI for an informed value.",
options: bitrate, current: bitrateKbps
) { bitrateKbps = $0 },
choiceRow(
id: "compositor", icon: "macwindow", label: "Compositor",
detail: "Which compositor drives the virtual output — honored only if "
+ "available on the host.",
options: SettingsOptions.compositors, current: compositor
) { compositor = $0 },
choiceRow(
id: "codec", header: "Video", icon: "film", label: "Video codec",
detail: "A preference — the host falls back if it can't encode this one "
+ "(10-bit and 4:4:4 are HEVC-only).",
options: SettingsOptions.codecs, current: codec
) { codec = $0 },
toggleRow(
id: "hdr", icon: "sun.max", label: "10-bit HDR",
detail: "HDR10 — engages when the host sends HDR content and this display "
+ "supports it.",
value: $hdrEnabled),
toggleRow(
id: "chroma", icon: "textformat", label: "Full chroma (4:4:4)",
detail: "Sharper text and UI at more bandwidth — needs host opt-in and "
+ "hardware decode.",
value: $enable444),
choiceRow(
id: "audio", header: "Audio", icon: "speaker.wave.2", label: "Audio channels",
detail: "The speaker layout requested from the host.",
options: SettingsOptions.audioChannels, current: audioChannels
) { audioChannels = $0 },
toggleRow(
id: "mic", icon: "mic", label: "Microphone",
detail: "Send this device's microphone to the host's virtual mic.",
value: $micEnabled),
choiceRow(
id: "pad", header: "Controller", icon: "gamecontroller", label: "Use controller",
detail: "Which pad is forwarded to the host, as player 1.",
options: controllers, current: gamepads.preferredID
) { gamepads.preferredID = $0 },
choiceRow(
id: "padType", icon: "dpad", label: "Controller type",
detail: "The virtual pad the host creates — Automatic matches this controller.",
options: SettingsOptions.padTypes, current: gamepadType
) { gamepadType = $0 },
toggleRow(
id: "hud", header: "Interface", icon: "chart.bar", label: "Statistics overlay",
detail: "Resolution, frame rate, throughput and latency while streaming.",
value: $hudEnabled),
choiceRow(
id: "hudPlacement", icon: "rectangle.inset.topright.filled", label: "Overlay position",
detail: "Which corner the statistics overlay sits in.",
options: SettingsOptions.hudPlacements, current: hudPlacement
) { hudPlacement = $0 },
toggleRow(
id: "library", icon: "square.grid.2x2", label: "Game library",
detail: "Browse and launch the host's games with \(buttonName(\.buttonY, "Y")) "
+ "(experimental).",
value: $libraryEnabled),
toggleRow(
id: "gamepadUI", icon: "hand.tap", label: "Controller-optimized UI",
detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled),
]
}
/// Resolution choices as "WxH" tags the current size is inserted when it's a custom mode
/// (set via the touch settings), so cycling starts from it instead of jumping.
private var resolutionOptions: [(label: String, tag: String)] {
var options = SettingsOptions.resolutionModes()
.map { (label: "\($0.name) · \($0.w) × \($0.h)", tag: "\($0.w)x\($0.h)") }
let current = "\(width)x\(height)"
if !options.contains(where: { $0.tag == current }) {
options.insert((label: "Custom · \(width) × \(height)", tag: current), at: 0)
}
return options
}
/// The active controller's user-facing name for a button (for detail strings).
private func buttonName(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, _ fallback: String
) -> String {
gamepads.active?.controller.extendedGamepad?[keyPath: button].localizedName ?? fallback
}
// MARK: - Row builders
private func choiceRow<T: Equatable>(
id: String, header: String? = nil, icon: String, label: String, detail: String,
options: [(label: String, tag: T)], current: T, write: @escaping (T) -> Void
) -> Row {
let index = options.firstIndex { $0.tag == current }
return Row(
id: id, header: header, icon: icon, label: label,
value: index.map { options[$0].label } ?? "",
detail: detail,
adjust: { delta in
// Unknown current value: snap to the first option on any step.
guard let index else {
guard let first = options.first else { return false }
write(first.tag)
return true
}
let target = index + delta
guard target >= 0, target < options.count else { return false }
write(options[target].tag)
return true
},
activate: {
guard let index else { return write(options.first?.tag ?? current) }
write(options[(index + 1) % options.count].tag)
})
}
private func toggleRow(
id: String, header: String? = nil, icon: String, label: String, detail: String,
value: Binding<Bool>
) -> Row {
Row(
id: id, header: header, icon: icon, label: label,
value: value.wrappedValue ? "On" : "Off",
detail: detail,
adjust: { delta in
// Directional semantics: left = off, right = on; a no-op reads as a boundary.
let target = delta > 0
guard value.wrappedValue != target else { return false }
value.wrappedValue = target
return true
},
activate: { value.wrappedValue.toggle() })
}
}
#endif
@@ -0,0 +1,60 @@
// SettingsView's navigation and presentation helpers: the iOS settings categories, the iPad
// sheet sizing, and the bounded-slider clamp.
import SwiftUI
#if os(iOS)
/// The settings groups, mirroring the macOS preference tabs. On iPad each is a sidebar row that
/// drives the detail pane; on iPhone the same list collapses to pushed sub-pages. Internal (not
/// private) so the screenshot harness can open SettingsView on a specific category.
enum SettingsCategory: String, CaseIterable, Identifiable {
case general, display, audio, controllers, advanced, about
var id: Self { self }
var title: String {
switch self {
case .general: return "General"
case .display: return "Display"
case .audio: return "Audio"
case .controllers: return "Controllers"
case .advanced: return "Advanced"
case .about: return "About"
}
}
var symbol: String {
switch self {
case .general: return "gearshape"
case .display: return "display"
case .audio: return "speaker.wave.2"
case .controllers: return "gamecontroller"
case .advanced: return "slider.horizontal.3"
case .about: return "info.circle"
}
}
}
extension View {
/// Present the settings sheet large on iPad so the NavigationSplitView has room for its
/// sidebar + detail a default form sheet is too narrow and the split view would collapse to
/// the iPhone push list. No-op on iPhone (the standard sheet is already right) and on iOS 17
/// (no `presentationSizing` it falls back to the default sheet, which still degrades cleanly
/// to the push list).
@ViewBuilder
func settingsSheetSizing() -> some View {
if UIDevice.current.userInterfaceIdiom == .pad, #available(iOS 18, *) {
presentationSizing(.page)
} else {
self
}
}
}
#endif
extension Double {
/// The log-scale slider mapping needs a bounded input (Automatic stores 0).
func clamped(_ lo: Double, _ hi: Double) -> Double {
Swift.min(Swift.max(self, lo), hi)
}
}
@@ -0,0 +1,147 @@
// The option lists every settings surface renders from one source of truth shared by the
// touch/desktop SettingsView (Pickers), the tvOS pushed selection rows, and the gamepad settings
// screen (GamepadSettingsView's left/right cycling). Pure data + small pure helpers; anything that
// reads live view state (e.g. the bitrate slider mapping) stays on SettingsView.
#if os(macOS)
import AppKit
#endif
import PunktfunkKit
import SwiftUI
enum SettingsOptions {
/// Compositor choices the `tag` is the wire value (`PunktfunkConnection.Compositor` raw).
static let compositors: [(label: String, tag: Int)] = [
("Automatic", 0),
("KWin (KDE Plasma)", 1),
("wlroots (Sway / Hyprland)", 2),
("Mutter (GNOME)", 3),
("gamescope", 4),
]
static let audioChannels: [(label: String, tag: Int)] = [
("Stereo", 2),
("5.1 Surround", 6),
("7.1 Surround", 8),
]
/// Virtual-pad types the `tag` is the wire value (`PunktfunkConnection.GamepadType` raw).
static let padTypes: [(label: String, tag: Int)] = [
("Automatic", 0),
("Xbox 360", 1),
("Xbox One", 3),
("DualSense", 2),
("DualShock 4", 4),
]
static let hudPlacements: [(label: String, tag: String)] =
HUDPlacement.allCases.map { ($0.label, $0.rawValue) }
/// Video-codec preference (`DefaultsKey.codec`) a soft preference the host falls back from.
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (hosts don't emit AV1 on
/// the native path yet).
static let codecs: [(label: String, tag: String)] = [
("Automatic", "auto"),
("HEVC (H.265)", "hevc"),
("H.264 (AVC)", "h264"),
]
// MARK: - Bitrate
/// Discrete bitrate steps for the surfaces with no Slider (tvOS pushed pickers, the gamepad
/// settings' left/right cycling), up to the same 3 Gbps ceiling the slider has.
static let bitratePresets: [(label: String, tag: Int)] = [
("Automatic", 0),
("10 Mbps", 10_000),
("20 Mbps", 20_000),
("40 Mbps", 40_000),
("80 Mbps", 80_000),
("150 Mbps", 150_000),
("300 Mbps", 300_000),
("500 Mbps", 500_000),
("1 Gbps", 1_000_000),
("1.5 Gbps", 1_500_000),
("2 Gbps", 2_000_000),
("3 Gbps", 3_000_000),
]
/// The presets plus the currently stored value when it isn't one of them (set via the touch
/// slider or a synced device) so the current choice stays visible/selectable.
static func bitrateOptions(current: Int) -> [(label: String, tag: Int)] {
var options = bitratePresets
if !options.contains(where: { $0.tag == current }) {
options.insert(
(SpeedTestSheet.mbpsLabel(kbps: current) + " (custom)", current), at: 1)
}
return options
}
// MARK: - Controllers
/// "Use controller" choices: Automatic, every forwardable controller, and so a stale pin
/// stays visible instead of leaving the selection tag-less any pinned id that is NOT among
/// the selectable (extended) entries, present-but-unusable included.
@MainActor
static func controllerOptions(_ gamepads: GamepadManager) -> [(label: String, tag: String)] {
let selectable = gamepads.controllers.filter(\.isExtended)
var options: [(label: String, tag: String)] = [("Automatic", "")]
options += selectable.map { ($0.name, $0.id) }
if !gamepads.preferredID.isEmpty,
!selectable.contains(where: { $0.id == gamepads.preferredID }) {
options.append(("Unavailable controller", gamepads.preferredID))
}
return options
}
#if os(iOS) || os(macOS)
// MARK: - Stream mode (iOS + macOS pickers; tvOS builds its own preset list)
/// 16:9 then ultrawide presets; the device's native mode is prepended by `resolutionModes`.
static let resolutionPresets: [(name: String, w: Int, h: Int)] = [
("720p", 1280, 720),
("1080p", 1920, 1080),
("1440p", 2560, 1440),
("4K", 3840, 2160),
("Ultrawide 1080p", 2560, 1080),
("Ultrawide 1440p", 3440, 1440),
("Super ultrawide", 5120, 1440),
]
/// This device's native mode first, then the presets, deduped by dimensions (native wins a
/// tie).
@MainActor
static func resolutionModes() -> [(name: String, w: Int, h: Int)] {
var native: [(name: String, w: Int, h: Int)] = []
#if os(iOS)
let bounds = UIScreen.main.nativeBounds // portrait-oriented pixels
native = [("This device",
Int(max(bounds.width, bounds.height)),
Int(min(bounds.width, bounds.height)))]
#else
if let screen = NSScreen.main {
let scale = screen.backingScaleFactor
native = [("This display",
Int(screen.frame.width * scale),
Int(screen.frame.height * scale))]
}
#endif
var seen = Set<String>()
return (native + resolutionPresets).filter { seen.insert("\($0.w)x\($0.h)").inserted }
}
/// Refresh rates the device can actually display (no point asking the host to render frames
/// the screen can't show), plus any stored custom value so it stays selectable.
@MainActor
static func refreshRates(including current: Int) -> [Int] {
#if os(iOS)
let maxHz = UIScreen.main.maximumFramesPerSecond
#else
let maxHz = NSScreen.main?.maximumFramesPerSecond ?? 60
#endif
var rates = [60, 120, 240].filter { $0 <= maxHz }
if rates.isEmpty { rates = [maxHz] }
if !rates.contains(current) { rates.append(current) }
return rates.sorted()
}
#endif
}
@@ -0,0 +1,385 @@
// SettingsView's shared sections each setting's Section is defined exactly once here and
// composed by the per-platform bodies in SettingsView.swift.
import PunktfunkKit
import SwiftUI
extension SettingsView {
// MARK: - Sections (shared)
@ViewBuilder var streamModeSection: some View {
Section {
#if os(iOS)
// Touch-first: a rotating wheel of common resolutions (this device's own mode first) and
// a segmented refresh-rate control the same family as the Clock/Timer pickers. The host
// renders a virtual output at exactly the chosen mode, so these are real pixel sizes. The
// last wheel row, "Custom", reveals width/height/refresh fields for an arbitrary mode.
VStack(alignment: .leading, spacing: 4) {
Text("Resolution")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Resolution", selection: resolutionSelection) {
ForEach(resolutionChoices, id: \.tag) { choice in
Text(choice.label).tag(choice.tag)
}
}
.labelsHidden()
.pickerStyle(.wheel)
.frame(maxHeight: 140)
}
if isCustomResolution {
// Arbitrary entry: type the exact width × height (and refresh) the host should drive.
HStack {
TextField("Width", value: $width, format: .number.grouping(.never))
.keyboardType(.numberPad)
Text("×")
TextField("Height", value: $height, format: .number.grouping(.never))
.labelsHidden()
.keyboardType(.numberPad)
}
// A row built from an HStack of TextFields otherwise insets its bottom separator to
// the inner content, clipping the hairline under "Width"; pin it to the cell edge.
.alignmentGuide(.listRowSeparatorLeading) { _ in 0 }
LabeledContent("Refresh rate") {
TextField("Hz", value: $hz, format: .number.grouping(.never))
.keyboardType(.numberPad)
.multilineTextAlignment(.trailing)
}
} else if refreshChoices.count > 1 {
VStack(alignment: .leading, spacing: 6) {
Text("Refresh rate")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Refresh rate", selection: $hz) {
ForEach(refreshChoices, id: \.self) { rate in
Text("\(rate) Hz").tag(rate)
}
}
.labelsHidden()
.pickerStyle(.segmented)
}
} else {
// A device with a single supported rate (e.g. 60 Hz) has nothing to pick.
LabeledContent("Refresh rate") {
Text("\(hz) Hz").foregroundStyle(.secondary)
}
}
Button("Use this display's mode") { fillFromMainScreen() }
#elseif os(macOS)
HStack {
TextField("Resolution", value: $width, format: .number.grouping(.never))
Text("×")
TextField("", value: $height, format: .number.grouping(.never))
.labelsHidden()
}
TextField("Refresh rate (Hz)", value: $hz, format: .number.grouping(.never))
LabeledContent("") {
Button("Use this display's mode") { fillFromMainScreen() }
}
#endif
#if !os(tvOS)
Toggle("Automatic bitrate", isOn: automaticBitrate)
if bitrateKbps != 0 {
HStack(spacing: 12) {
Slider(value: bitrateSlider, in: 0...1) {
Text("Bitrate")
}
Text(SpeedTestSheet.mbpsLabel(kbps: bitrateKbps))
.monospacedDigit()
.foregroundStyle(.secondary)
.frame(minWidth: 76, alignment: .trailing)
}
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
}
}
#endif
} header: {
Text("Stream mode")
} footer: {
Text("The host creates a virtual output at exactly this mode — "
+ "native resolution, no scaling. \(Self.bitrateFooter)")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
// MARK: - Stream mode (iOS wheel)
/// Sentinel wheel tag for the "Custom" row. Real tags are "WxH" (digits + "x"), so this can't
/// collide with a resolution.
private static let customResolutionTag = "custom"
/// Wheel rows: the resolution modes (device native first see `SettingsOptions`), then a
/// "Custom" row that reveals the numeric fields.
private var resolutionChoices: [(label: String, tag: String)] {
SettingsOptions.resolutionModes()
.map { (label: "\($0.name) · \($0.w) × \($0.h)", tag: "\($0.w)x\($0.h)") }
+ [(label: "Custom…", tag: Self.customResolutionTag)]
}
private var presetResolutionTags: Set<String> {
Set(SettingsOptions.resolutionModes().map { "\($0.w)x\($0.h)" })
}
/// True when the editable custom fields should show: the wheel is parked on "Custom" (sticky),
/// or the stored size simply isn't one of the presets (e.g. a value synced from a Mac) so a
/// non-preset mode stays editable across relaunches without a persisted flag.
private var isCustomResolution: Bool {
customMode || !presetResolutionTags.contains("\(width)x\(height)")
}
/// The wheel works in "WxH" tags so one selection drives both width and height; the custom
/// sentinel toggles `customMode` instead of writing a size.
private var resolutionSelection: Binding<String> {
Binding(
get: { isCustomResolution ? Self.customResolutionTag : "\(width)x\(height)" },
set: { tag in
if tag == Self.customResolutionTag {
customMode = true
return
}
customMode = false
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 2 else { return }
width = parts[0]
height = parts[1]
})
}
/// Refresh rates this device can display, plus any stored custom value (see `SettingsOptions`).
private var refreshChoices: [Int] {
SettingsOptions.refreshRates(including: hz)
}
#endif
@ViewBuilder var audioSection: some View {
Section {
Picker("Audio channels", selection: $audioChannels) {
ForEach(SettingsOptions.audioChannels, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
#if os(macOS)
Picker("Speaker", selection: $speakerUID) {
Text("System default").tag("")
ForEach(outputDevices) { device in
Text(device.name).tag(device.uid)
}
if !speakerUID.isEmpty,
!outputDevices.contains(where: { $0.uid == speakerUID }) {
Text("Unavailable device").tag(speakerUID)
}
}
#endif
Toggle("Send microphone to the host", isOn: $micEnabled)
#if os(macOS)
Picker("Microphone", selection: $micUID) {
Text("System default").tag("")
ForEach(inputDevices) { device in
Text(device.name).tag(device.uid)
}
if !micUID.isEmpty,
!inputDevices.contains(where: { $0.uid == micUID }) {
Text("Unavailable device").tag(micUID)
}
}
.disabled(!micEnabled)
#endif
} header: {
Text("Audio")
} footer: {
Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
/// iPad-only pointer-capture toggle: lock the mouse/trackpad for relative movement (games) vs
/// forward an absolute cursor position (desktop). Empty on iPhone (no hardware-pointer lock
/// the mouse path there is always the absolute fallback).
@ViewBuilder var pointerSection: some View {
if UIDevice.current.userInterfaceIdiom == .pad {
Section {
Toggle("Capture pointer for games", isOn: $pointerCapture)
} header: {
Text("Pointer")
} footer: {
Text("With a mouse or trackpad connected, lock the pointer and send relative "
+ "movement — the expected behavior for games (mouse-look). Turn this off for "
+ "desktop use to keep the pointer free and send its absolute position instead. "
+ "The lock needs the stream full-screen and frontmost; it falls back to the "
+ "absolute pointer automatically (Stage Manager, Slide Over). Finger touch is "
+ "unaffected. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
}
#endif
@ViewBuilder var compositorSection: some View {
Section {
Picker("Compositor", selection: $compositor) {
ForEach(SettingsOptions.compositors, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
} header: {
Text("Host compositor")
} footer: {
Text("Which compositor drives the virtual output on the host. A specific "
+ "choice is honored only if that backend is available there — "
+ "otherwise the host falls back to auto-detection.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var windowSection: some View {
#if os(macOS)
Section {
Toggle("Fullscreen while streaming", isOn: $fullscreenWhileStreaming)
} header: {
Text("Window")
} footer: {
Text("Take the window fullscreen when a session starts and restore it on the host "
+ "list, so only the stream is fullscreen — not the picker.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
// the controller test. Empty in release builds (no presenter UI; stage-2 always).
@ViewBuilder var presenterSection: some View {
#if DEBUG
Section {
Picker("Presenter", selection: $presenter) {
Text("Stage 2 (default)").tag("stage2")
Text("Stage 1 (debug)").tag("stage1")
}
} header: {
Text("Video presenter · debug")
} footer: {
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD and "
+ "self-recovers from decode stalls. Stage 1 feeds compressed video straight to the "
+ "system display layer; it freezes on a lost HEVC reference frame, so it's a debug "
+ "fallback only. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder var hdrSection: some View {
Section {
Picker("Video codec", selection: $codec) {
ForEach(SettingsOptions.codecs, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
Toggle("10-bit HDR", isOn: $hdrEnabled)
Toggle("Full chroma (4:4:4)", isOn: $enable444)
} header: {
Text("Video quality")
} footer: {
Text("Codec is a preference — the host falls back if it can't encode the one you pick "
+ "(and 10-bit/4:4:4 are HEVC-only). HDR requests a 10-bit BT.2020 PQ (HDR10) stream — "
+ "it only engages when the host is sending HDR content AND this display supports HDR. "
+ "4:4:4 requests full chroma (sharper text/UI, more bandwidth) — it only engages when "
+ "this device can hardware-decode it AND the host opted in. Otherwise the stream stays "
+ "8-bit 4:2:0 SDR. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var statisticsSection: some View {
Section {
Toggle("Show statistics overlay", isOn: $hudEnabled)
Picker("Position", selection: $hudPlacement) {
ForEach(HUDPlacement.allCases) { placement in
Text(placement.label).tag(placement.rawValue)
}
}
.disabled(!hudEnabled)
} header: {
Text("Statistics")
} footer: {
Text(Self.statisticsFooter)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var experimentalSection: some View {
Section {
Toggle("Show game library", isOn: $libraryEnabled)
} header: {
Text("Experimental")
} footer: {
Text("Adds a “Browse Library…” action to each host that lists its games "
+ "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "Works once you've paired with the host — the library is authorized by this "
+ "device's certificate, with no extra host setup.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var controllersSection: some View {
Section {
if gamepads.controllers.isEmpty {
Text("No controllers detected")
.foregroundStyle(.secondary)
} else {
ForEach(gamepads.controllers) { controller in
controllerRow(controller)
}
}
Picker("Use controller", selection: $gamepads.preferredID) {
ForEach(controllerOptions, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
Picker("Controller type", selection: $gamepadType) {
ForEach(SettingsOptions.padTypes, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
#if !os(tvOS)
Toggle("Gamepad-optimized browsing", isOn: $gamepadUIEnabled)
#endif
#if DEBUG && !os(tvOS)
Button("Test Controller…") { showControllerTest = true }
.disabled(gamepads.active == nil)
.sheet(isPresented: $showControllerTest) { ControllerTestView() }
#endif
} header: {
Text("Controllers")
} footer: {
// The gamepad-UI blurb is appended here, not merged into the shared
// `controllersFooter` constant tvOS's `tvBody` reuses that exact string (line ~348)
// for its own footer and has no such toggle to describe.
VStack(alignment: .leading, spacing: 6) {
Text(Self.controllersFooter)
#if !os(tvOS)
Text(Self.gamepadUIFooter)
#endif
}
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
}
@@ -0,0 +1,153 @@
// SettingsView's footers and stateful helpers, used by both the section builders
// (SettingsView+Sections.swift) and the per-platform bodies (SettingsView.swift). The option
// LISTS live in SettingsOptions they're shared with the gamepad settings screen too.
#if os(macOS)
import AppKit
#endif
import PunktfunkKit
import SwiftUI
extension SettingsView {
// MARK: - Bitrate
/// Slider domain, log-scale: the useful range spans three orders of magnitude
/// (a few Mbps 3 Gbps) linear would cram everything below 100 Mbps into the
/// first pixels.
private static let minSliderKbps = 2_000.0
private static let maxSliderKbps = 3_000_000.0
static let bitrateFooter =
"Automatic uses the host's default bitrate (20 Mbps); the host clamps any choice "
+ "to its supported range. Run a speed test from a host card's context menu to "
+ "pick an informed value. Applies from the next session."
static let gigabitWarning =
"Above 1 Gbps — test the network speed first (a host card's context menu → "
+ "Test Network Speed…). A bitrate beyond what the link sustains causes loss "
+ "and stutter."
/// `bitrateKbps == 0` is Automatic; switching to manual lands on the host default.
var automaticBitrate: Binding<Bool> {
Binding(
get: { bitrateKbps == 0 },
set: { bitrateKbps = $0 ? 0 : 20_000 })
}
/// Slider position 0...1 kbps on the log scale, snapped to two significant figures
/// so the readout shows round numbers instead of 47_322.
var bitrateSlider: Binding<Double> {
Binding(
get: {
let v = Double(bitrateKbps).clamped(Self.minSliderKbps, Self.maxSliderKbps)
return log(v / Self.minSliderKbps)
/ log(Self.maxSliderKbps / Self.minSliderKbps)
},
set: { pos in
let raw = Self.minSliderKbps
* pow(Self.maxSliderKbps / Self.minSliderKbps, pos)
let mag = pow(10, floor(log10(raw)) - 1)
bitrateKbps = Int((raw / mag).rounded() * mag)
})
}
// MARK: - Statistics
static var statisticsFooter: String {
let base = "The overlay shows resolution, frame rate, throughput and latency while "
+ "streaming, in the chosen corner."
#if os(macOS) || os(iOS)
return base + " Toggle it any time with ⌘⇧S."
#else
return base
#endif
}
// MARK: - Controllers
static let controllersFooter =
"One controller is forwarded to the host, as player 1 — Automatic picks the most "
+ "recently connected one. The type is the virtual pad the host creates: Automatic "
+ "matches the controller (a DualSense gets adaptive triggers, lightbar, touchpad "
+ "and motion; a DualShock 4 the same minus adaptive triggers), and changes apply "
+ "from the next session. Two identical controllers may swap a manual selection "
+ "after reconnecting."
#if !os(tvOS)
static let gamepadUIFooter =
"When a controller is connected, the host list and game library switch to a "
+ "controller-friendly layout — larger focus targets, controller-navigable settings, "
+ "and a swipeable cover browser for the library. Turn this off to always use the "
+ "standard layout. (The system may still move basic focus with a controller "
+ "connected even with this off — that's outside the app's control.)"
#endif
/// "Use controller" choices for this view's manager (see `SettingsOptions.controllerOptions`).
var controllerOptions: [(label: String, tag: String)] {
SettingsOptions.controllerOptions(gamepads)
}
func controllerRow(_ controller: GamepadManager.DiscoveredController) -> some View {
HStack(spacing: 10) {
Image(systemName: controller.hasTouchpadAndMotion ? "playstation.logo" : "gamecontroller.fill")
.foregroundStyle(.secondary)
VStack(alignment: .leading, spacing: 2) {
Text(controller.name)
HStack(spacing: 8) {
if !controller.isExtended {
Text(controller.productCategory)
}
if controller.hasAdaptiveTriggers {
Image(systemName: "r2.button.roundedtop.horizontal")
}
if controller.hasLight {
Image(systemName: "lightbulb.fill")
}
if controller.hasMotion {
Image(systemName: "gyroscope")
}
if controller.hasHaptics {
Image(systemName: "waveform")
}
if let level = controller.batteryLevel {
Text("\(Int(level * 100))%")
if controller.isCharging {
Image(systemName: "bolt.fill")
}
}
}
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
}
Spacer()
if gamepads.active?.id == controller.id {
Text("In use")
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 8)
.padding(.vertical, 3)
.background(Capsule().fill(.green.opacity(0.2)))
.foregroundStyle(.green)
}
}
}
func fillFromMainScreen() {
#if os(macOS)
guard let screen = NSScreen.main else { return }
let scale = screen.backingScaleFactor
width = Int(screen.frame.width * scale)
height = Int(screen.frame.height * scale)
hz = screen.maximumFramesPerSecond
#else
// nativeBounds is portrait-oriented pixels streams are landscape.
let bounds = UIScreen.main.nativeBounds
width = Int(max(bounds.width, bounds.height))
height = Int(min(bounds.width, bounds.height))
hz = UIScreen.main.maximumFramesPerSecond
#if os(iOS)
// The native mode is the "This device" wheel row, so leave Custom mode if it was on.
customMode = false
#endif
#endif
}
}
@@ -0,0 +1,369 @@
// App settings. The host creates a native virtual output at exactly the chosen size/refresh
// there is no scaling anywhere in the pipeline.
//
// Navigation differs per platform, but all three group the same categories (General, Display,
// Audio, Controllers, Advanced, About): macOS uses a tabbed preferences window; iOS/iPadOS uses
// an adaptive NavigationSplitView a category sidebar + detail pane on iPad, auto-collapsing to
// a hierarchical push list on iPhone (the system Settings idiom on each); tvOS uses a
// focus-native pushed-picker layout. The individual sections (`streamModeSection`,
// `audioSection`, ) are shared across all three so a setting is defined exactly once they
// live in SettingsView+Sections.swift, with their helpers in SettingsView+Support.swift.
#if os(macOS)
import AppKit
#endif
import PunktfunkKit
import SwiftUI
@MainActor
struct SettingsView: View {
@Environment(\.dismiss) private var dismiss
@AppStorage(DefaultsKey.streamWidth) var width = 1920
@AppStorage(DefaultsKey.streamHeight) var height = 1080
@AppStorage(DefaultsKey.streamHz) var hz = 60
@AppStorage(DefaultsKey.compositor) var compositor = 0
@AppStorage(DefaultsKey.gamepadType) var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) var presenter = "stage2"
@AppStorage(DefaultsKey.hdrEnabled) var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) var libraryEnabled = false
@AppStorage(DefaultsKey.fullscreenWhileStreaming) var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) var micEnabled = true
@AppStorage(DefaultsKey.audioChannels) var audioChannels = 2
@AppStorage(DefaultsKey.codec) var codec = "auto"
@AppStorage(DefaultsKey.hudEnabled) var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) var hudPlacement = HUDPlacement.topTrailing.rawValue
@ObservedObject var gamepads = GamepadManager.shared
#if !os(tvOS)
@AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true
#endif
#if DEBUG && !os(tvOS)
@State var showControllerTest = false
#endif
#if os(iOS)
@AppStorage(DefaultsKey.pointerCapture) var pointerCapture = true
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
@State private var settingsSelection: SettingsCategory?
// Tracked so the detail can show its own Done whenever the sidebar (and its Done) is off screen
// not just on iPhone, but on any iPad layout that collapses the sidebar to an overlay. Starts
// .doubleColumn so iPad reliably opens with the sidebar (and its Done) visible.
@State private var columnVisibility: NavigationSplitViewVisibility = .doubleColumn
// Sticky once the wheel lands on "Custom", so editing a width/height that briefly equals a
// preset doesn't snap the wheel back off Custom. A stored non-preset value reads as custom even
// when this is false (see `isCustomResolution`), so it survives relaunches without persisting.
@State var customMode = false
#endif
#if os(macOS)
@AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = ""
@State var outputDevices: [AudioDevice] = []
@State var inputDevices: [AudioDevice] = []
#endif
#if os(iOS)
/// `initialCategory` is nil in the app (the list opens un-selected on iPhone; iPad lands on
/// General via `onAppear`). The screenshot harness passes an explicit category so the captured
/// shot opens on a real settings page (a populated detail) rather than the bare category list.
init(initialCategory: SettingsCategory? = nil) {
_settingsSelection = State(initialValue: initialCategory)
}
#endif
var body: some View {
#if os(tvOS)
// Native tv pattern: no inline text entry (typing numbers with a remote is
// miserable and the inline field chrome fights the focus system). Modes are
// preset pickers that push selection lists like the system Settings app.
tvBody
#elseif os(macOS)
macBody
#else
iosBody
#endif
}
// MARK: - macOS: tabbed preferences
#if os(macOS)
private var macBody: some View {
TabView {
Form {
streamModeSection
compositorSection
}
.formStyle(.grouped)
.tabItem { Label("General", systemImage: "gearshape") }
Form {
presenterSection
hdrSection
windowSection
statisticsSection
}
.formStyle(.grouped)
.tabItem { Label("Display", systemImage: "display") }
Form {
audioSection
}
.formStyle(.grouped)
.onAppear {
outputDevices = AudioDevices.outputs()
inputDevices = AudioDevices.inputs()
}
.tabItem { Label("Audio", systemImage: "speaker.wave.2") }
Form {
controllersSection
}
.formStyle(.grouped)
.onAppear {
gamepads.refresh()
gamepads.startDiscovery()
}
.onDisappear { gamepads.stopDiscovery() }
.tabItem { Label("Controllers", systemImage: "gamecontroller") }
Form {
experimentalSection
}
.formStyle(.grouped)
.tabItem { Label("Advanced", systemImage: "slider.horizontal.3") }
AcknowledgementsView()
.tabItem { Label("About", systemImage: "info.circle") }
}
.frame(width: 480, height: 460)
}
#endif
// MARK: - iOS / iPadOS: adaptive split view
#if os(iOS)
private var iosBody: some View {
NavigationSplitView(columnVisibility: $columnVisibility) {
List(selection: $settingsSelection) {
ForEach(SettingsCategory.allCases) { category in
// On iPhone the split view collapses to a push list, but a selection List
// draws no disclosure indicator of its own add one in compact width for the
// expected drill-in affordance. On iPad the selected row highlights instead, so
// the chevron is omitted there.
HStack {
Label(category.title, systemImage: category.symbol)
if horizontalSizeClass == .compact {
Spacer()
Image(systemName: "chevron.forward")
.font(.footnote.weight(.semibold))
.foregroundStyle(.tertiary)
// Purely a drill-in affordance the row's button trait already
// conveys "opens"; keep it out of the VoiceOver announcement.
.accessibilityHidden(true)
}
}
.tag(category)
}
}
.navigationTitle("Settings")
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
} detail: {
// NavigationSplitView hosts the detail in its own navigation context (its title bar),
// so no inner NavigationStack that would double the bar on iPad. On iPhone the split
// view collapses to one stack and pushes this when a row is tapped. `?? .general` only
// backs the brief pre-selection window; the list never auto-pushes on a nil selection.
settingsDetail(settingsSelection ?? .general)
// Keep a Done on the detail whenever the sidebar (and its Done) isn't on screen: the
// iPhone push, or any iPad layout that collapsed the sidebar to an overlay. When the
// sidebar is showing, its Done is the only one so this stays hidden to avoid two.
.toolbar {
if horizontalSizeClass == .compact || columnVisibility == .detailOnly {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
}
}
.onAppear {
if horizontalSizeClass == .regular, settingsSelection == nil {
settingsSelection = .general
}
gamepads.refresh()
gamepads.startDiscovery()
}
// A regularregular launch sets the default above; this catches a compactregular change
// (e.g. an iPad leaving narrow split-screen multitasking) so the detail pane fills in.
.onChange(of: horizontalSizeClass) { _, newValue in
if newValue == .regular, settingsSelection == nil {
settingsSelection = .general
}
}
.onDisappear { gamepads.stopDiscovery() }
}
@ViewBuilder
private func settingsDetail(_ category: SettingsCategory) -> some View {
switch category {
case .general:
Form {
streamModeSection
pointerSection
compositorSection
}
.formStyle(.grouped)
.navigationTitle("General")
.navigationBarTitleDisplayMode(.inline)
case .display:
Form {
presenterSection
hdrSection
statisticsSection
}
.formStyle(.grouped)
.navigationTitle("Display")
.navigationBarTitleDisplayMode(.inline)
case .audio:
Form { audioSection }
.formStyle(.grouped)
.navigationTitle("Audio")
.navigationBarTitleDisplayMode(.inline)
case .controllers:
Form { controllersSection }
.formStyle(.grouped)
.navigationTitle("Controllers")
.navigationBarTitleDisplayMode(.inline)
case .advanced:
Form { experimentalSection }
.formStyle(.grouped)
.navigationTitle("Advanced")
.navigationBarTitleDisplayMode(.inline)
case .about:
// Already a full scrollable view that sets its own "Acknowledgements" title; pin the
// display mode inline to match the five sibling detail pages (it would otherwise inherit
// the large title from the "Settings" sidebar root).
AcknowledgementsView()
.navigationBarTitleDisplayMode(.inline)
}
}
#endif
// MARK: - tvOS
#if os(tvOS)
private static let presets: [(label: String, tag: String)] = [
("720p @ 60", "1280x720x60"),
("1080p @ 60", "1920x1080x60"),
("4K @ 60", "3840x2160x60"),
]
private var modeTag: Binding<String> {
Binding(
get: { "\(width)x\(height)x\(hz)" },
set: { tag in
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 3 else { return }
width = parts[0]
height = parts[1]
hz = parts[2]
})
}
private var hudEnabledTag: Binding<String> {
Binding(get: { hudEnabled ? "on" : "off" }, set: { hudEnabled = $0 == "on" })
}
private var hdrEnabledTag: Binding<String> {
Binding(get: { hdrEnabled ? "on" : "off" }, set: { hdrEnabled = $0 == "on" })
}
private var tvBody: some View {
let currentTag = "\(width)x\(height)x\(hz)"
let bounds = UIScreen.main.nativeBounds
let nativeTag = "\(Int(max(bounds.width, bounds.height)))x"
+ "\(Int(min(bounds.width, bounds.height)))x\(UIScreen.main.maximumFramesPerSecond)"
var options = Self.presets
if !options.contains(where: { $0.tag == nativeTag }) {
options.insert(("This TV (native)", nativeTag), at: 0)
}
if !options.contains(where: { $0.tag == currentTag }) {
options.insert(("Custom (\(width)×\(height) @ \(hz))", currentTag), at: 0)
}
return ScrollView {
VStack(spacing: 16) {
TVSelectionRow(title: "Stream mode", options: options, selection: modeTag)
TVSelectionRow(
title: "Bitrate",
options: SettingsOptions.bitrateOptions(current: bitrateKbps),
selection: $bitrateKbps)
TVSelectionRow(
title: "Audio channels",
options: SettingsOptions.audioChannels,
selection: $audioChannels)
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
.multilineTextAlignment(.center)
}
TVSelectionRow(
title: "Compositor", options: SettingsOptions.compositors,
selection: $compositor)
#if DEBUG
TVSelectionRow(
title: "Presenter (debug)",
options: [("Stage 2 (default)", "stage2"), ("Stage 1 (debug)", "stage1")],
selection: $presenter)
#endif
TVSelectionRow(
title: "10-bit HDR",
options: [("On", "on"), ("Off", "off")], selection: hdrEnabledTag)
Text("The host creates a virtual output at exactly this mode — native "
+ "resolution, no scaling. \(Self.bitrateFooter) A specific compositor "
+ "is honored only if available on the host.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
TVSelectionRow(
title: "Statistics overlay",
options: [("On", "on"), ("Off", "off")], selection: hudEnabledTag)
TVSelectionRow(
title: "Statistics position", options: SettingsOptions.hudPlacements,
selection: $hudPlacement)
ForEach(gamepads.controllers) { controller in
controllerRow(controller)
.padding(.horizontal, 24)
}
TVSelectionRow(
title: "Use controller", options: controllerOptions,
selection: $gamepads.preferredID)
TVSelectionRow(
title: "Controller type", options: SettingsOptions.padTypes,
selection: $gamepadType)
Text(Self.controllersFooter)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
NavigationLink("Acknowledgements") { AcknowledgementsView() }
.padding(.top, 8)
}
.frame(maxWidth: 1000)
.frame(maxWidth: .infinity)
.padding(60)
}
.navigationTitle("Settings")
.onAppear {
gamepads.refresh()
gamepads.startDiscovery()
}
.onDisappear { gamepads.stopDiscovery() }
}
#endif
}
File diff suppressed because it is too large Load Diff
@@ -46,9 +46,24 @@ extension StoredHost {
}
}
private extension Data {
/// Lowercase hex, no separators to compare a pinned fingerprint against the mDNS `fp`.
var hexLower: String { map { String(format: "%02x", $0) }.joined() }
/// The two joins of live mDNS discovery against the saved-host store, shared by the touch grid
/// (HomeView) and the gamepad launcher (GamepadHomeView) so both screens classify hosts the same
/// way. LAN-scoped like the underlying match: a host that isn't advertising here is "not seen",
/// not proven off.
extension HostDiscovery {
/// A saved host is "online" iff a live advert currently matches it (see `StoredHost.matches`).
/// Recomputed on every discovery change (the @Published set), so it tracks hosts
/// appearing/leaving the network live.
func advertises(_ host: StoredHost) -> Bool {
hosts.contains { host.matches($0) }
}
/// Discovered hosts not already saved the saved list shows the rest, so this only surfaces
/// genuinely-new hosts on the network. Same match as `advertises`, so a saved host whose IP
/// changed (still fingerprint-matched) doesn't also appear as a stranger.
func unsaved(among saved: [StoredHost]) -> [DiscoveredHost] {
hosts.filter { d in !saved.contains { $0.matches(d) } }
}
}
@MainActor
@@ -0,0 +1,27 @@
// Hex encode/decode for the trust surface pinned certificate fingerprints and the mDNS `fp`
// TXT value travel as lowercase hex.
import Foundation
extension Data {
/// Lowercase hex, no separators to compare a pinned fingerprint against the mDNS `fp`.
var hexLower: String { map { String(format: "%02x", $0) }.joined() }
/// Parse an even-length hex string into bytes; nil on any non-hex character or odd length.
/// Used to turn an mDNS-advertised cert fingerprint into a connect pin.
init?(hexString: String) {
let chars = Array(hexString)
guard chars.count.isMultiple(of: 2) else { return nil }
var bytes = [UInt8]()
bytes.reserveCapacity(chars.count / 2)
var i = 0
while i < chars.count {
guard let hi = chars[i].hexDigitValue, let lo = chars[i + 1].hexDigitValue else {
return nil
}
bytes.append(UInt8(hi << 4 | lo))
i += 2
}
self = Data(bytes)
}
}
@@ -70,7 +70,7 @@ struct TrustCardView: View {
/// 64 hex chars four groups per line, two lines easy to eyeball against the log.
private static func format(fingerprint: Data) -> String {
let hex = fingerprint.map { String(format: "%02x", $0) }.joined()
let hex = fingerprint.hexLower
let groups = stride(from: 0, to: hex.count, by: 8).map { i -> String in
let start = hex.index(hex.startIndex, offsetBy: i)
let end = hex.index(start, offsetBy: min(8, hex.count - i))
@@ -1,202 +0,0 @@
// Annex-B HEVC CoreMedia plumbing.
//
// The punktfunk host emits Annex-B access units with in-band VPS/SPS/PPS on every IDR
// (deliberately the client needs no out-of-band extradata). VideoToolbox wants the AVCC
// flavor instead: a CMVideoFormatDescription built from the parameter sets, and sample
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two.
//
// SCAFFOLD: written on the Linux host, not yet compiled against Xcode.
import CoreMedia
import Foundation
/// The video codec of the host's elementary stream negotiated in the Welcome and read via
/// `punktfunk_connection_codec`. Both are Annex-B with in-band parameter sets on every IDR; they
/// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1 is not an
/// Annex-B/NAL codec and isn't handled here (hosts don't emit it on the native path yet).
public enum VideoCodec: Equatable {
case h264
case hevc
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown HEVC).
public init(wire: UInt8) {
self = wire == 0x01 ? .h264 : .hevc // 0x01 = PUNKTFUNK_CODEC_H264
}
}
public enum AnnexB {
/// Split an Annex-B stream into NAL units (start codes 00 00 01 / 00 00 00 01 stripped).
/// All zeros immediately preceding a start code are dropped: they're either the
/// 4-byte-code prefix or `trailing_zero_8bits` padding, never NAL payload (emulation
/// prevention keeps 00 00 0x out of conforming NAL bytes) same policy as ffmpeg.
public static func nalUnits(in data: Data) -> [Data] {
var nals: [Data] = []
let bytes = [UInt8](data)
var i = 0
var start = -1
while i + 2 < bytes.count {
if bytes[i] == 0, bytes[i + 1] == 0, bytes[i + 2] == 1 {
var codeStart = i
while codeStart > 0, bytes[codeStart - 1] == 0 {
codeStart -= 1
}
if start >= 0, start < codeStart {
nals.append(Data(bytes[start..<codeStart]))
}
start = i + 3
i += 3
} else {
i += 1
}
}
if start >= 0, start < bytes.count {
nals.append(Data(bytes[start...]))
}
return nals
}
/// HEVC NAL unit type (bits 1..6 of the first byte).
public static func hevcNalType(_ nal: Data) -> UInt8 {
guard let first = nal.first else { return 0xFF }
return (first >> 1) & 0x3F
}
/// H.264 NAL unit type (bits 0..4 of the first byte).
public static func h264NalType(_ nal: Data) -> UInt8 {
guard let first = nal.first else { return 0xFF }
return first & 0x1F
}
/// True if this NAL is a parameter set for `codec` (dropped from AVCC; kept for the format desc).
/// HEVC: VPS 32 / SPS 33 / PPS 34. H.264: SPS 7 / PPS 8 (no VPS).
private static func isParameterSet(_ nal: Data, _ codec: VideoCodec) -> Bool {
switch codec {
case .hevc: let t = hevcNalType(nal); return t == 32 || t == 33 || t == 34
case .h264: let t = h264NalType(nal); return t == 7 || t == 8
}
}
/// Build a format description from an IDR AU's in-band parameter sets (HEVC: VPS/SPS/PPS;
/// H.264: SPS/PPS). Returns nil when the AU carries no parameter sets (non-IDR).
public static func formatDescription(fromIDR au: Data, codec: VideoCodec)
-> CMVideoFormatDescription?
{
// Collect the parameter-set NALs in the order VideoToolbox wants them (HEVC: VPS,SPS,PPS;
// H.264: SPS,PPS).
var vps: Data?, sps: Data?, pps: Data?
for nal in nalUnits(in: au) {
switch codec {
case .hevc:
switch hevcNalType(nal) {
case 32: vps = nal
case 33: sps = nal
case 34: pps = nal
default: break
}
case .h264:
switch h264NalType(nal) {
case 7: sps = nal
case 8: pps = nal
default: break
}
}
}
guard let sps, let pps else { return nil }
let sets: [Data] = codec == .hevc ? [vps, sps, pps].compactMap { $0 } : [sps, pps]
guard codec == .h264 || sets.count == 3 else { return nil } // HEVC needs the VPS too
var format: CMVideoFormatDescription?
// Pin every parameter set's bytes for the duration of the create call, then hand
// VideoToolbox parallel pointer/size arrays.
var pointers: [UnsafePointer<UInt8>] = []
var sizes: [Int] = []
func withAll(_ i: Int, _ body: () -> Void) {
if i == sets.count { body(); return }
sets[i].withUnsafeBytes { raw in
pointers.append(raw.bindMemory(to: UInt8.self).baseAddress!)
sizes.append(sets[i].count)
withAll(i + 1, body)
}
}
var status: OSStatus = -1
withAll(0) {
switch codec {
case .hevc:
status = CMVideoFormatDescriptionCreateFromHEVCParameterSets(
allocator: kCFAllocatorDefault,
parameterSetCount: pointers.count,
parameterSetPointers: pointers,
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
extensions: nil,
formatDescriptionOut: &format)
case .h264:
status = CMVideoFormatDescriptionCreateFromH264ParameterSets(
allocator: kCFAllocatorDefault,
parameterSetCount: pointers.count,
parameterSetPointers: pointers,
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
formatDescriptionOut: &format)
}
}
return status == noErr ? format : nil
}
/// Re-pack an Annex-B AU as AVCC (4-byte big-endian length before each NAL), dropping
/// the parameter-set NALs (they live in the format description).
public static func avcc(from au: Data, codec: VideoCodec) -> Data {
var out = Data(capacity: au.count + 16)
for nal in nalUnits(in: au) {
if isParameterSet(nal, codec) { continue }
var len = UInt32(nal.count).bigEndian
withUnsafeBytes(of: &len) { out.append(contentsOf: $0) }
out.append(nal)
}
return out
}
/// Wrap one AU as a decode-ready CMSampleBuffer.
public static func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription, codec: VideoCodec
) -> CMSampleBuffer? {
let avccData = avcc(from: au.data, codec: codec)
var blockBuffer: CMBlockBuffer?
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault, memoryBlock: nil,
blockLength: avccData.count, blockAllocator: kCFAllocatorDefault,
customBlockSource: nil, offsetToData: 0, dataLength: avccData.count,
flags: 0, blockBufferOut: &blockBuffer) == noErr,
let block = blockBuffer
else { return nil }
let copied = avccData.withUnsafeBytes { raw in
CMBlockBufferReplaceDataBytes(
with: raw.baseAddress!, blockBuffer: block,
offsetIntoDestination: 0, dataLength: avccData.count)
}
guard copied == noErr else { return nil }
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: CMTime(value: Int64(au.ptsNs), timescale: 1_000_000_000),
decodeTimeStamp: .invalid)
var sampleSize = avccData.count
var sample: CMSampleBuffer?
guard CMSampleBufferCreate(
allocator: kCFAllocatorDefault, dataBuffer: block, dataReady: true,
makeDataReadyCallback: nil, refcon: nil, formatDescription: format,
sampleCount: 1, sampleTimingEntryCount: 1, sampleTimingArray: &timing,
sampleSizeEntryCount: 1, sampleSizeArray: &sampleSize,
sampleBufferOut: &sample) == noErr
else { return nil }
// Low-latency display: render on arrival, don't wait for a clock.
if let attachments = CMSampleBufferGetSampleAttachmentsArray(sample!, createIfNecessary: true) {
let dict = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self)
CFDictionarySetValue(
dict,
Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(),
Unmanaged.passUnretained(kCFBooleanTrue).toOpaque())
}
return sample
}
}
@@ -0,0 +1,129 @@
import AVFoundation
import os
/// SPSC-ish jitter ring (interleaved float, `channels` per frame), drain thread render
/// callback. The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// reads return silence until enough is buffered (at least `prefill`, and at least one
/// packet more than the device's render quantum large-buffer devices would otherwise
/// chronically out-demand the prefill and oscillate prime dropout re-prime), and an
/// underrun re-primes, concealing jitter as one short dip instead of sustained crackle.
/// All counts stay whole frames (multiples of `channels`), so the interleave can never slip.
final class AudioRing: @unchecked Sendable {
private var buf: [Float]
private var readIdx = 0
private var writeIdx = 0
private var primed = false
private var renderQuantum = 0
private let prefill: Int
private let highWater: Int
private let channels: Int
private let lock = OSAllocatedUnfairLock()
/// `capacity`/`prefill` in samples (interleaved `channels` per frame, both whole frames).
init(capacity: Int, prefill: Int, channels: Int) {
buf = [Float](repeating: 0, count: capacity)
self.prefill = prefill
self.channels = channels
highWater = prefill * 4
}
func write(_ samples: UnsafePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
let capacity = buf.count
// A single write larger than the whole ring would push readIdx PAST writeIdx below
// (inverting the valid range corruption). It never happens (one decoded packet is far
// under capacity), but guard rather than corrupt.
guard count <= capacity else { return }
if writeIdx + count - readIdx > capacity {
readIdx = writeIdx + count - capacity // overflow: drop oldest
}
for i in 0..<count {
buf[(writeIdx + i) % capacity] = samples[i]
}
writeIdx += count
// Latency clamp: both ends run at 48 kHz, so backlog from a network stall (or
// creeping host-vs-DAC clock skew) never drains on its own without this, one
// 300 ms hiccup leaves audio 300 ms behind video for the rest of the session.
// Shedding down to 2× prefill costs one audible blip instead.
if writeIdx - readIdx > highWater {
readIdx = writeIdx - prefill * 2
}
}
/// Fills `out` completely (silence beyond what's buffered).
func read(into out: UnsafeMutablePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
renderQuantum = max(renderQuantum, count)
let available = writeIdx - readIdx
if !primed {
// One 5 ms host packet (240 frames × channels) of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 240 * channels) {
primed = true
} else {
for i in 0..<count { out[i] = 0 }
return
}
}
let n = min(available, count)
let capacity = buf.count
for i in 0..<n {
out[i] = buf[(readIdx + i) % capacity]
}
readIdx += n
if n < count {
for i in n..<count { out[i] = 0 }
primed = false // underrun re-prime before resuming
}
}
}
/// CoreAudio channel layout for the canonical wire order FL FR FC LFE RL RR [SL SR]. nil for
/// stereo (the standard layout is correct). For 5.1/7.1 we list explicit channel labels via
/// `kAudioChannelLayoutTag_UseChannelDescriptions` preset tags (DTS_5_1 etc.) don't reliably
/// match Moonlight's order. NB the 7.1 mapping (verified against the WASAPI 0x63F + SPA orderings):
/// wire idx 4-5 = RL/RR = the WAVE *back* pair LeftSurround/RightSurround; idx 6-7 = SL/SR = the
/// WAVE *side* pair LeftSurroundDirect/RightSurroundDirect. (Using RearSurround* for 6-7 would
/// swap side/back vs the Windows/Linux clients.)
func wireChannelLayout(channels: Int) -> AVAudioChannelLayout? {
let labels: [AudioChannelLabel]
switch channels {
case 6:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen, kAudioChannelLabel_LeftSurround,
kAudioChannelLabel_RightSurround,
]
case 8:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen,
kAudioChannelLabel_LeftSurround, kAudioChannelLabel_RightSurround, // wire RL/RR (back)
kAudioChannelLabel_LeftSurroundDirect, kAudioChannelLabel_RightSurroundDirect, // wire SL/SR (side)
]
default:
return nil
}
let size = MemoryLayout<AudioChannelLayout>.size
+ (labels.count - 1) * MemoryLayout<AudioChannelDescription>.stride
let raw = UnsafeMutableRawPointer.allocate(byteCount: size, alignment: 16)
defer { raw.deallocate() }
let layout = raw.bindMemory(to: AudioChannelLayout.self, capacity: 1)
layout.pointee.mChannelLayoutTag = kAudioChannelLayoutTag_UseChannelDescriptions
layout.pointee.mChannelBitmap = AudioChannelBitmap(rawValue: 0)
layout.pointee.mNumberChannelDescriptions = UInt32(labels.count)
// `mChannelDescriptions` is the C variable-length tail array (declared `[1]`, over-allocated
// above). Scope the pointer with `withUnsafeMutablePointer` taking `&mChannelDescriptions`
// inline yields a pointer valid only for that expression, so building a buffer from it that
// outlives the call is a dangling-pointer bug. Inside the closure it stays valid while we fill it.
withUnsafeMutablePointer(to: &layout.pointee.mChannelDescriptions) { tail in
let descs = UnsafeMutableBufferPointer(start: tail, count: labels.count)
for (i, lbl) in labels.enumerated() {
descs[i] = AudioChannelDescription(
mChannelLabel: lbl, mChannelFlags: AudioChannelFlags(rawValue: 0),
mCoordinates: (0, 0, 0))
}
}
return AVAudioChannelLayout(layout: layout)
}
@@ -19,99 +19,6 @@ import os
private let log = Logger(subsystem: "io.unom.punktfunk", category: "audio")
/// SPSC-ish jitter ring (interleaved float, `channels` per frame), drain thread render
/// callback. The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// reads return silence until enough is buffered (at least `prefill`, and at least one
/// packet more than the device's render quantum large-buffer devices would otherwise
/// chronically out-demand the prefill and oscillate prime dropout re-prime), and an
/// underrun re-primes, concealing jitter as one short dip instead of sustained crackle.
/// All counts stay whole frames (multiples of `channels`), so the interleave can never slip.
final class AudioRing: @unchecked Sendable {
private var buf: [Float]
private var readIdx = 0
private var writeIdx = 0
private var primed = false
private var renderQuantum = 0
private let prefill: Int
private let highWater: Int
private let channels: Int
private let lock = OSAllocatedUnfairLock()
/// `capacity`/`prefill` in samples (interleaved `channels` per frame, both whole frames).
init(capacity: Int, prefill: Int, channels: Int) {
buf = [Float](repeating: 0, count: capacity)
self.prefill = prefill
self.channels = channels
highWater = prefill * 4
}
func write(_ samples: UnsafePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
let capacity = buf.count
// A single write larger than the whole ring would push readIdx PAST writeIdx below
// (inverting the valid range corruption). It never happens (one decoded packet is far
// under capacity), but guard rather than corrupt.
guard count <= capacity else { return }
if writeIdx + count - readIdx > capacity {
readIdx = writeIdx + count - capacity // overflow: drop oldest
}
for i in 0..<count {
buf[(writeIdx + i) % capacity] = samples[i]
}
writeIdx += count
// Latency clamp: both ends run at 48 kHz, so backlog from a network stall (or
// creeping host-vs-DAC clock skew) never drains on its own without this, one
// 300 ms hiccup leaves audio 300 ms behind video for the rest of the session.
// Shedding down to 2× prefill costs one audible blip instead.
if writeIdx - readIdx > highWater {
readIdx = writeIdx - prefill * 2
}
}
/// Fills `out` completely (silence beyond what's buffered).
func read(into out: UnsafeMutablePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
renderQuantum = max(renderQuantum, count)
let available = writeIdx - readIdx
if !primed {
// One 5 ms host packet (240 frames × channels) of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 240 * channels) {
primed = true
} else {
for i in 0..<count { out[i] = 0 }
return
}
}
let n = min(available, count)
let capacity = buf.count
for i in 0..<n {
out[i] = buf[(readIdx + i) % capacity]
}
readIdx += n
if n < count {
for i in n..<count { out[i] = 0 }
primed = false // underrun re-prime before resuming
}
}
}
private final class StopFlag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
}
/// Render-block-owned scratch storage: freed exactly when the closure (and thus the
/// last possible render call) is released never racing CoreAudio.
private final class ScratchBuffer {
@@ -120,55 +27,6 @@ private final class ScratchBuffer {
deinit { ptr.deallocate() }
}
/// CoreAudio channel layout for the canonical wire order FL FR FC LFE RL RR [SL SR]. nil for
/// stereo (the standard layout is correct). For 5.1/7.1 we list explicit channel labels via
/// `kAudioChannelLayoutTag_UseChannelDescriptions` preset tags (DTS_5_1 etc.) don't reliably
/// match Moonlight's order. NB the 7.1 mapping (verified against the WASAPI 0x63F + SPA orderings):
/// wire idx 4-5 = RL/RR = the WAVE *back* pair LeftSurround/RightSurround; idx 6-7 = SL/SR = the
/// WAVE *side* pair LeftSurroundDirect/RightSurroundDirect. (Using RearSurround* for 6-7 would
/// swap side/back vs the Windows/Linux clients.)
private func wireChannelLayout(channels: Int) -> AVAudioChannelLayout? {
let labels: [AudioChannelLabel]
switch channels {
case 6:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen, kAudioChannelLabel_LeftSurround,
kAudioChannelLabel_RightSurround,
]
case 8:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen,
kAudioChannelLabel_LeftSurround, kAudioChannelLabel_RightSurround, // wire RL/RR (back)
kAudioChannelLabel_LeftSurroundDirect, kAudioChannelLabel_RightSurroundDirect, // wire SL/SR (side)
]
default:
return nil
}
let size = MemoryLayout<AudioChannelLayout>.size
+ (labels.count - 1) * MemoryLayout<AudioChannelDescription>.stride
let raw = UnsafeMutableRawPointer.allocate(byteCount: size, alignment: 16)
defer { raw.deallocate() }
let layout = raw.bindMemory(to: AudioChannelLayout.self, capacity: 1)
layout.pointee.mChannelLayoutTag = kAudioChannelLayoutTag_UseChannelDescriptions
layout.pointee.mChannelBitmap = AudioChannelBitmap(rawValue: 0)
layout.pointee.mNumberChannelDescriptions = UInt32(labels.count)
// `mChannelDescriptions` is the C variable-length tail array (declared `[1]`, over-allocated
// above). Scope the pointer with `withUnsafeMutablePointer` taking `&mChannelDescriptions`
// inline yields a pointer valid only for that expression, so building a buffer from it that
// outlives the call is a dangling-pointer bug. Inside the closure it stays valid while we fill it.
withUnsafeMutablePointer(to: &layout.pointee.mChannelDescriptions) { tail in
let descs = UnsafeMutableBufferPointer(start: tail, count: labels.count)
for (i, lbl) in labels.enumerated() {
descs[i] = AudioChannelDescription(
mChannelLabel: lbl, mChannelFlags: AudioChannelFlags(rawValue: 0),
mCoordinates: (0, 0, 0))
}
}
return AVAudioChannelLayout(layout: layout)
}
public final class SessionAudio {
private let connection: PunktfunkConnection
private let flag = StopFlag()
@@ -0,0 +1,59 @@
// The client's persistent identity + the SPAKE2 PIN pairing ceremony the trust
// bootstrap that precedes any pinned PunktfunkConnection.
import Foundation
import PunktfunkCore
/// This client's persistent self-signed identity. Generate ONCE with `generateIdentity()`,
/// store both PEMs (Keychain), present on every connect the certificate's fingerprint is
/// how hosts recognize this client after pairing.
public struct ClientIdentity: Sendable {
public let certPEM: String
public let keyPEM: String
public init(certPEM: String, keyPEM: String) {
self.certPEM = certPEM
self.keyPEM = keyPEM
}
}
/// Generate a fresh client identity (self-signed cert + key, PEM).
public func generateIdentity() throws -> ClientIdentity {
var cert = [CChar](repeating: 0, count: 4096)
var key = [CChar](repeating: 0, count: 4096)
let rc = punktfunk_generate_identity(&cert, UInt(cert.count), &key, UInt(key.count))
guard rc == PUNKTFUNK_STATUS_OK.rawValue else {
throw PunktfunkClientError.status(rc)
}
return ClientIdentity(certPEM: String(cString: cert), keyPEM: String(cString: key))
}
/// Run the PIN pairing ceremony: the host displays a 4-digit PIN (its log/UI), the user
/// types it here. On success the host stores this client's identity and the returned
/// fingerprint is the host's now-VERIFIED identity persist it and pass it as `pinSHA256`
/// to every later connect. Throws `.wrongPIN` when the proof is rejected.
public func pair(
host: String, port: UInt16 = 9777,
identity: ClientIdentity, pin: String, name: String,
timeoutMs: UInt32 = 90_000
) throws -> Data {
var observed = [UInt8](repeating: 0, count: 32)
// The C header types PunktfunkStatus as a bare int32 (C17, no enum import), so the ABI
// functions return Int32 directly compare against the enum constants' rawValue, the
// same bridging the connection methods use (statusOK etc.).
let rc = host.withCString { cs in
identity.certPEM.withCString { cert in
identity.keyPEM.withCString { key in
pin.withCString { p in
name.withCString { n in
punktfunk_pair(cs, port, cert, key, p, n, &observed, timeoutMs)
}
}
}
}
}
switch rc {
case PUNKTFUNK_STATUS_OK.rawValue: return Data(observed)
case PUNKTFUNK_STATUS_CRYPTO.rawValue: throw PunktfunkClientError.wrongPIN
default: throw PunktfunkClientError.status(rc)
}
}
@@ -0,0 +1,87 @@
// Convenience constructors for the wire input events (field semantics match
// punktfunk_core::input::InputEvent; see punktfunk_core.h).
import Foundation
import PunktfunkCore
public extension PunktfunkInputEvent {
private static func make(
_ kind: UInt32, code: UInt32, x: Int32, y: Int32, flags: UInt32 = 0
) -> PunktfunkInputEvent {
PunktfunkInputEvent(kind: UInt8(kind), _pad: (0, 0, 0), code: code, x: x, y: y, flags: flags)
}
static func mouseMove(dx: Int32, dy: Int32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_MOVE.rawValue, code: 0, x: dx, y: dy)
}
/// Absolute cursor position in client-surface pixels the host places its cursor
/// there (same letterbox mapping and `flags` surface-dims packing as the touch events).
/// Used by the iPad pointer fallback when the scene can't pointer-lock and GCMouse's
/// relative deltas aren't available; the surface dimensions must each fit in 16 bits.
static func mouseMoveAbs(
x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_MOUSE_MOVE_ABS.rawValue, code: 0, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
/// GameStream button ids: 1=left 2=middle 3=right 4=X1 5=X2 (host maps to evdev BTN_*).
static func mouseButton(_ button: UInt32, down: Bool) -> PunktfunkInputEvent {
make(
(down ? PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_DOWN : PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_UP).rawValue,
code: button, x: 0, y: 0)
}
/// `vk` is a Windows virtual-key code (the host's vk_to_evdev table consumes these).
static func key(_ vk: UInt32, down: Bool) -> PunktfunkInputEvent {
make((down ? PUNKTFUNK_INPUT_KIND_KEY_DOWN : PUNKTFUNK_INPUT_KIND_KEY_UP).rawValue, code: vk, x: 0, y: 0)
}
/// WHEEL_DELTA(120)-scaled; positive = up (vertical) / right (horizontal) the
/// convention Moonlight/SDL use; the host maps onto the ei/wl axes.
static func scroll(_ delta: Int32, horizontal: Bool = false) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_SCROLL.rawValue, code: horizontal ? 1 : 0, x: delta, y: 0)
}
// Gamepad (wire contract in punktfunk_core::input::gamepad): one transition per event,
// `pad` = controller index, accumulated host-side into a virtual Xbox 360 or DualSense
// pad (the session's negotiated `GamepadType`).
/// `button` is a GameStream buttonFlags bit (A=0x1000 B=0x2000 X=0x4000 Y=0x8000,
/// dpad=0x1/2/4/8, start=0x10 back=0x20 LS=0x40 RS=0x80 LB=0x100 RB=0x200 guide=0x400,
/// touchpad click=0x100000 DualSense sessions only, the xpad has no such button).
static func gamepadButton(_ button: UInt32, down: Bool, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_GAMEPAD_BUTTON.rawValue,
code: button, x: down ? 1 : 0, y: 0, flags: pad)
}
/// Axis ids: 0=LSX 1=LSY 2=RSX 3=RSY (32768...32767, XInput convention: +y = UP
/// `GCControllerDirectionPad.yAxis` already matches, no flip), 4=LT 5=RT (0...255).
static func gamepadAxis(_ axis: UInt32, value: Int32, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_GAMEPAD_AXIS.rawValue, code: axis, x: value, y: 0, flags: pad)
}
// Touch (host-side: libei ei_touchscreen on the virtual output). `id` distinguishes
// fingers and is reusable after touchUp; coordinates are absolute pixels on the
// client's touch surface, whose size rides in `flags` so the host can rescale
// the surface dimensions must each fit in 16 bits. Built for the iOS variant
// (UITouch these); nothing on macOS emits them yet.
static func touchDown(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_DOWN.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchMove(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_MOVE.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchUp(id: UInt32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_TOUCH_UP.rawValue, code: id, x: 0, y: 0)
}
}
@@ -57,60 +57,6 @@ public enum PunktfunkClientError: Error {
case status(Int32)
}
/// This client's persistent self-signed identity. Generate ONCE with `generateIdentity()`,
/// store both PEMs (Keychain), present on every connect the certificate's fingerprint is
/// how hosts recognize this client after pairing.
public struct ClientIdentity: Sendable {
public let certPEM: String
public let keyPEM: String
public init(certPEM: String, keyPEM: String) {
self.certPEM = certPEM
self.keyPEM = keyPEM
}
}
/// Generate a fresh client identity (self-signed cert + key, PEM).
public func generateIdentity() throws -> ClientIdentity {
var cert = [CChar](repeating: 0, count: 4096)
var key = [CChar](repeating: 0, count: 4096)
let rc = punktfunk_generate_identity(&cert, UInt(cert.count), &key, UInt(key.count))
guard rc == PUNKTFUNK_STATUS_OK.rawValue else {
throw PunktfunkClientError.status(rc)
}
return ClientIdentity(certPEM: String(cString: cert), keyPEM: String(cString: key))
}
/// Run the PIN pairing ceremony: the host displays a 4-digit PIN (its log/UI), the user
/// types it here. On success the host stores this client's identity and the returned
/// fingerprint is the host's now-VERIFIED identity persist it and pass it as `pinSHA256`
/// to every later connect. Throws `.wrongPIN` when the proof is rejected.
public func pair(
host: String, port: UInt16 = 9777,
identity: ClientIdentity, pin: String, name: String,
timeoutMs: UInt32 = 90_000
) throws -> Data {
var observed = [UInt8](repeating: 0, count: 32)
// The C header types PunktfunkStatus as a bare int32 (C17, no enum import), so the ABI
// functions return Int32 directly compare against the enum constants' rawValue, the
// same bridging the connection methods use (statusOK etc.).
let rc = host.withCString { cs in
identity.certPEM.withCString { cert in
identity.keyPEM.withCString { key in
pin.withCString { p in
name.withCString { n in
punktfunk_pair(cs, port, cert, key, p, n, &observed, timeoutMs)
}
}
}
}
}
switch rc {
case PUNKTFUNK_STATUS_OK.rawValue: return Data(observed)
case PUNKTFUNK_STATUS_CRYPTO.rawValue: throw PunktfunkClientError.wrongPIN
default: throw PunktfunkClientError.status(rc)
}
}
/// `withCString` over an optional nil maps to a NULL C pointer.
func withOptionalCString<R>(_ s: String?, _ body: (UnsafePointer<CChar>?) -> R) -> R {
guard let s else { return body(nil) }
@@ -803,87 +749,3 @@ public final class PunktfunkConnection {
return closeRequested ? nil : handle
}
}
// Convenience constructors for the wire input events (field semantics match
// punktfunk_core::input::InputEvent; see punktfunk_core.h).
public extension PunktfunkInputEvent {
private static func make(
_ kind: UInt32, code: UInt32, x: Int32, y: Int32, flags: UInt32 = 0
) -> PunktfunkInputEvent {
PunktfunkInputEvent(kind: UInt8(kind), _pad: (0, 0, 0), code: code, x: x, y: y, flags: flags)
}
static func mouseMove(dx: Int32, dy: Int32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_MOVE.rawValue, code: 0, x: dx, y: dy)
}
/// Absolute cursor position in client-surface pixels the host places its cursor
/// there (same letterbox mapping and `flags` surface-dims packing as the touch events).
/// Used by the iPad pointer fallback when the scene can't pointer-lock and GCMouse's
/// relative deltas aren't available; the surface dimensions must each fit in 16 bits.
static func mouseMoveAbs(
x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_MOUSE_MOVE_ABS.rawValue, code: 0, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
/// GameStream button ids: 1=left 2=middle 3=right 4=X1 5=X2 (host maps to evdev BTN_*).
static func mouseButton(_ button: UInt32, down: Bool) -> PunktfunkInputEvent {
make(
(down ? PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_DOWN : PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_UP).rawValue,
code: button, x: 0, y: 0)
}
/// `vk` is a Windows virtual-key code (the host's vk_to_evdev table consumes these).
static func key(_ vk: UInt32, down: Bool) -> PunktfunkInputEvent {
make((down ? PUNKTFUNK_INPUT_KIND_KEY_DOWN : PUNKTFUNK_INPUT_KIND_KEY_UP).rawValue, code: vk, x: 0, y: 0)
}
/// WHEEL_DELTA(120)-scaled; positive = up (vertical) / right (horizontal) the
/// convention Moonlight/SDL use; the host maps onto the ei/wl axes.
static func scroll(_ delta: Int32, horizontal: Bool = false) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_SCROLL.rawValue, code: horizontal ? 1 : 0, x: delta, y: 0)
}
// Gamepad (wire contract in punktfunk_core::input::gamepad): one transition per event,
// `pad` = controller index, accumulated host-side into a virtual Xbox 360 or DualSense
// pad (the session's negotiated `GamepadType`).
/// `button` is a GameStream buttonFlags bit (A=0x1000 B=0x2000 X=0x4000 Y=0x8000,
/// dpad=0x1/2/4/8, start=0x10 back=0x20 LS=0x40 RS=0x80 LB=0x100 RB=0x200 guide=0x400,
/// touchpad click=0x100000 DualSense sessions only, the xpad has no such button).
static func gamepadButton(_ button: UInt32, down: Bool, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_GAMEPAD_BUTTON.rawValue,
code: button, x: down ? 1 : 0, y: 0, flags: pad)
}
/// Axis ids: 0=LSX 1=LSY 2=RSX 3=RSY (32768...32767, XInput convention: +y = UP
/// `GCControllerDirectionPad.yAxis` already matches, no flip), 4=LT 5=RT (0...255).
static func gamepadAxis(_ axis: UInt32, value: Int32, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_GAMEPAD_AXIS.rawValue, code: axis, x: value, y: 0, flags: pad)
}
// Touch (host-side: libei ei_touchscreen on the virtual output). `id` distinguishes
// fingers and is reusable after touchUp; coordinates are absolute pixels on the
// client's touch surface, whose size rides in `flags` so the host can rescale
// the surface dimensions must each fit in 16 bits. Built for the iOS variant
// (UITouch these); nothing on macOS emits them yet.
static func touchDown(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_DOWN.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchMove(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_MOVE.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchUp(id: UInt32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_TOUCH_UP.rawValue, code: id, x: 0, y: 0)
}
}
@@ -0,0 +1,73 @@
#if DEBUG
import Combine
import GameController
/// Local feedback driver for the Settings Controllers "Test Controller" panel (DEBUG builds
/// only). It drives the SAME CoreHaptics rumble renderer and `DualSenseTriggerEffect` path a
/// live session uses just aimed at the physically-connected controller instead of the
/// hostclient feedback planes so rumble, the adaptive triggers, the lightbar and the player
/// LEDs can be confirmed on-device without a host. Reusing the real renderers is the point:
/// a passing test exercises the exact code a session runs.
@MainActor
public final class ControllerTester: ObservableObject {
private let renderer = RumbleRenderer()
private weak var controller: GCController?
/// The rumble backend now in use "DualSense HID · USB/Bluetooth", "CoreHaptics", or ""
/// for the test panel to display so it's obvious which path a given pad takes.
@Published public private(set) var rumbleBackend = ""
public init() {}
/// Aim the feedback at a controller (nil releases it). Idempotent safe to call on every
/// active-controller change.
public func target(_ c: GCController?) {
guard c !== controller else { return }
controller = c
renderer.retarget(c) { [weak self] note in
Task { @MainActor in self?.rumbleBackend = note }
}
}
/// Drive both motors at 0...1 amplitudes low = left/heavy, high = right/light mapped to
/// the 0...0xFFFF wire range the session carries, through the real `RumbleRenderer`.
public func rumble(low: Float, high: Float) {
func u16(_ v: Float) -> UInt16 { UInt16((min(max(v, 0), 1) * 65535).rounded()) }
renderer.apply(low: u16(low), high: u16(high))
}
public func stopRumble() { renderer.apply(low: 0, high: 0) }
/// Replay an adaptive-trigger effect on a DualSense via the real `DualSenseTriggerEffect`
/// renderer. `right == false` L2, `true` R2. No-op on a non-DualSense pad.
public func applyTrigger(_ effect: DualSenseTriggerEffect, right: Bool) {
guard let ds = controller?.extendedGamepad as? GCDualSenseGamepad else { return }
effect.apply(to: right ? ds.rightTrigger : ds.leftTrigger)
}
public func resetTriggers() {
guard let ds = controller?.extendedGamepad as? GCDualSenseGamepad else { return }
ds.leftTrigger.setModeOff()
ds.rightTrigger.setModeOff()
}
/// Lightbar colour (DualSense / DualShock 4); nil turns it off. No-op without a light.
public func setLight(_ color: GCColor?) {
controller?.light?.color = color ?? GCColor(red: 0, green: 0, blue: 0)
}
/// Player-indicator LEDs (`.index1`...`.index4`, or `.indexUnset` to clear).
public func setPlayerIndex(_ index: GCControllerPlayerIndex) {
controller?.playerIndex = index
}
/// Silence every channel and release the controller call on the panel's disappear.
public func stop() {
resetTriggers()
setPlayerIndex(.indexUnset)
setLight(nil)
renderer.retarget(nil) // async teardown: stops the motors + drops the controller ref
controller = nil
}
}
#endif
@@ -29,64 +29,6 @@ import Combine
import Foundation
import GameController
/// The gamepad wire contract (mirrors `punktfunk_core::input::gamepad`).
public enum GamepadWire {
public static let dpadUp: UInt32 = 0x0001
public static let dpadDown: UInt32 = 0x0002
public static let dpadLeft: UInt32 = 0x0004
public static let dpadRight: UInt32 = 0x0008
public static let start: UInt32 = 0x0010
public static let back: UInt32 = 0x0020
public static let leftStickClick: UInt32 = 0x0040
public static let rightStickClick: UInt32 = 0x0080
public static let leftShoulder: UInt32 = 0x0100
public static let rightShoulder: UInt32 = 0x0200
public static let guide: UInt32 = 0x0400
public static let a: UInt32 = 0x1000
public static let b: UInt32 = 0x2000
public static let x: UInt32 = 0x4000
public static let y: UInt32 = 0x8000
/// DualSense touchpad click (Moonlight's extended-button bit position).
public static let touchpadClick: UInt32 = 0x10_0000
public static let allButtons: [UInt32] = [
dpadUp, dpadDown, dpadLeft, dpadRight, start, back,
leftStickClick, rightStickClick, leftShoulder, rightShoulder, guide,
a, b, x, y, touchpadClick,
]
public static let axisLSX: UInt32 = 0
public static let axisLSY: UInt32 = 1
public static let axisRSX: UInt32 = 2
public static let axisRSY: UInt32 = 3
public static let axisLT: UInt32 = 4
public static let axisRT: UInt32 = 5
/// Raw DualSense gyro units per rad/s: hid-playstation's calibration over the host's
/// fixed blob resolves to 20 LSB per deg/s.
public static let gyroLSBPerRadS: Float = 20 * 180 / .pi
/// Raw DualSense accelerometer units per g (same derivation).
public static let accelLSBPerG: Float = 10_000
/// GC touchpad coordinates (±1, +y up) wire (0...65535, origin top-left, +y down).
public static func touchpad(x: Float, y: Float) -> (x: UInt16, y: UInt16) {
let wx = ((x.clamped(to: -1...1) + 1) / 2 * 65535).rounded()
let wy = ((1 - y.clamped(to: -1...1)) / 2 * 65535).rounded()
return (UInt16(wx), UInt16(wy))
}
/// Scale + clamp one motion component into the raw signed-16 sensor domain.
public static func motionRaw(_ value: Float, scale: Float) -> Int16 {
Int16((value * scale).rounded().clamped(to: Float(Int16.min)...Float(Int16.max)))
}
}
extension Float {
fileprivate func clamped(to range: ClosedRange<Float>) -> Float {
Swift.min(Swift.max(self, range.lowerBound), range.upperBound)
}
}
@MainActor
public final class GamepadCapture {
private let connection: PunktfunkConnection
@@ -0,0 +1,195 @@
// Hostclient gamepad feedback rendering: one drain thread polls the rumble (0xCA) and
// HID-output (0xCD) planes and replays them on the active physical controller
//
// rumble CHHapticEngine players (per-handle localities when the pad has them,
// one combined engine otherwise),
// lightbar GCDeviceLight,
// player LEDs GCController.playerIndex (the DS bit patterns map to player 14),
// trigger FX DualSenseTriggerEffect.parse GCDualSenseAdaptiveTrigger.
//
// Only pad 0 is rendered (exactly one controller is forwarded). HID-output traffic exists
// only on PlayStation-pad sessions (a DualSense, or a DualShock 4 = lightbar only) the
// drain always polls both planes with short timeouts and never spins, so an Xbox session
// just renders rumble. GameController profile mutation
// happens on main; CHHapticEngine work on its own serial queue; the drain thread itself
// touches neither. When GamepadManager switches the active controller mid-session, the
// old pad is reset (triggers off, player index unset) and the last known feedback state
// is replayed onto the new one.
import Combine
import Foundation
import GameController
public final class GamepadFeedback {
private let connection: PunktfunkConnection
private let flag = StopFlag()
private let drainDone = DispatchSemaphore(value: 0)
private var drainStarted = false
private let rumble = RumbleRenderer()
private var activeSub: AnyCancellable?
// Last applied feedback (main-actor) replayed when the active controller changes.
@MainActor private var target: GCController?
@MainActor private var lastLight: (r: UInt8, g: UInt8, b: UInt8)?
@MainActor private var lastPlayerBits: UInt8?
@MainActor private var lastTrigger: [DualSenseTriggerEffect?] = [nil, nil]
public init(connection: PunktfunkConnection, manager: GamepadManager) {
self.connection = connection
// Capture self weakly in the hop too, so the inner sink's weak capture isn't shadowing
// an implicit strong one and the subscription (stored on self) never retain-cycles.
Task { @MainActor [weak self] in
guard let self else { return }
self.activeSub = manager.$active.sink { [weak self] dc in
MainActor.assumeIsolated { self?.retarget(dc?.controller) }
}
}
}
/// Safety net: the drain thread captures `connection` strongly and only `self` weakly, so if
/// this is dropped without `stop()` (an abrupt teardown) the thread would poll forever and
/// leak the connection signal it to exit. (`stop()` is the normal path and also joins it.)
deinit { flag.stop() }
/// Map the DualSense player-LED bit patterns (5 LEDs, hid-playstation's player
/// conventions) onto GCControllerPlayerIndex. Unknown patterns fall back to the lit
/// count, clamped to the four indices GC offers.
public static func playerIndex(forBits bits: UInt8) -> GCControllerPlayerIndex {
switch bits & 0x1F {
case 0: return .indexUnset
case 0b00100: return .index1
case 0b01010: return .index2
case 0b10101: return .index3
case 0b11011: return .index4
default:
let lit = (bits & 0x1F).nonzeroBitCount
return GCControllerPlayerIndex(rawValue: min(lit, 4) - 1) ?? .index1
}
}
public func start() {
guard !drainStarted else { return }
drainStarted = true
// Hidout traffic (lightbar / player LEDs / triggers) only exists on a PlayStation-pad
// session a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and
// let rumble own the wait elsewhere; on an Xbox session it stays nonblocking.
let thread = Thread { [connection, flag, drainDone, weak self] in
while !flag.isStopped {
do {
// Poll the feedback planes NON-BLOCKING. A blocking poll (timeoutMs > 0) holds
// the connection's shared feedback lock for its whole wait; the video pump drains
// HDR mastering metadata (nextHdrMeta) on the SAME lock every frame, so a blocking
// poll here starved it and throttled HDR to ~1 fps (SDR, which never drains HDR
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
if let r = try connection.nextRumble(timeoutMs: 0), r.pad == 0 {
self?.rumble.apply(low: r.low, high: r.high)
}
// Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
var burst = 0
while burst < 64, !flag.isStopped,
let ev = try connection.nextHidOutput(timeoutMs: 0) {
self?.render(ev)
burst += 1
}
} catch {
break // .closed (or fatal) the session is over
}
// ~8 ms poll cadence (125 Hz), slept OUTSIDE the feedback lock low rumble/HID
// latency without holding the lock the HDR-meta drain needs.
if !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
}
drainDone.signal()
}
thread.name = "punktfunk-feedback"
thread.qualityOfService = .userInteractive
thread.start()
}
/// Stop the drain and silence the motors. Blocks until the drain thread exits ( one
/// poll cycle) call off the main actor, before `connection.close()`.
public func stop() {
flag.stop()
if drainStarted {
drainDone.wait()
drainStarted = false
}
rumble.stop()
// Drop the retarget subscription and the dead session's cached feedback a
// controller change after teardown must not replay this session's triggers/LEDs.
Task { @MainActor in
self.activeSub = nil
self.lastLight = nil
self.lastPlayerBits = nil
self.lastTrigger = [nil, nil]
self.reset(self.target)
self.target = nil
}
}
private func render(_ ev: PunktfunkConnection.HidOutputEvent) {
DispatchQueue.main.async {
MainActor.assumeIsolated { self.apply(ev) }
}
}
@MainActor
private func apply(_ ev: PunktfunkConnection.HidOutputEvent) {
switch ev {
case let .led(pad, r, g, b):
guard pad == 0 else { return }
lastLight = (r, g, b)
target?.light?.color = GCColor(
red: Float(r) / 255, green: Float(g) / 255, blue: Float(b) / 255)
case let .playerLEDs(pad, bits):
guard pad == 0 else { return }
lastPlayerBits = bits
target?.playerIndex = Self.playerIndex(forBits: bits)
case let .triggerEffect(pad, which, effect):
guard pad == 0, which < 2 else { return }
let parsed = DualSenseTriggerEffect.parse(effect)
lastTrigger[Int(which)] = parsed
if let trigger = adaptiveTrigger(which) {
parsed.apply(to: trigger)
}
}
}
@MainActor
private func retarget(_ controller: GCController?) {
guard controller !== target else { return }
reset(target)
target = controller
rumble.retarget(controller)
// Replay the session's feedback state so a swapped-in controller looks the same.
if let (r, g, b) = lastLight {
controller?.light?.color = GCColor(
red: Float(r) / 255, green: Float(g) / 255, blue: Float(b) / 255)
}
if let bits = lastPlayerBits {
controller?.playerIndex = Self.playerIndex(forBits: bits)
}
for which in 0..<2 {
if let effect = lastTrigger[which], let trigger = adaptiveTrigger(UInt8(which)) {
effect.apply(to: trigger)
}
}
}
@MainActor
private func reset(_ controller: GCController?) {
guard let c = controller else { return }
c.playerIndex = .indexUnset
if let ds = c.extendedGamepad as? GCDualSenseGamepad {
ds.leftTrigger.setModeOff()
ds.rightTrigger.setModeOff()
}
}
@MainActor
private func adaptiveTrigger(_ which: UInt8) -> GCDualSenseAdaptiveTrigger? {
guard let ds = target?.extendedGamepad as? GCDualSenseGamepad else { return nil }
return which == 0 ? ds.leftTrigger : ds.rightTrigger
}
}
@@ -14,8 +14,9 @@
// off or race over.
//
// The button set mirrors a console launcher: A confirms, B backs out, Y is a screen's secondary
// action, and the shoulders (L1/R1) are optional fast "jump" steps. Directional moves auto-repeat
// on a held stick/dpad after an initial delay; every button is edge-triggered (fires once per press).
// action, X a tertiary one, and the shoulders (L1/R1) are optional fast "jump" steps. Directional
// moves auto-repeat on a held stick/dpad after an initial delay; every button is edge-triggered
// (fires once per press).
import Foundation
import GameController
@@ -29,10 +30,18 @@ public final class GamepadMenuInput {
private let manager: GamepadManager
private var pollTimer: Timer?
private var isActive = false
/// Seed the pressed-state trackers from the LIVE controller on the first poll after a
/// (re)start, firing nothing. Screens hand the controller off (a keyboard closes, a cover
/// dismisses) while the user is still holding the very button that triggered the handoff
/// without this, the next screen's first poll would read that held button as a fresh edge
/// and act on the same press twice (e.g. the B that closed the keyboard also backing out
/// of the screen underneath).
private var needsSnapshot = false
private var currentDirection: Direction?
private var repeatTimer: Timer?
private var wasConfirmPressed = false
private var wasSecondaryPressed = false
private var wasTertiaryPressed = false
private var wasBackPressed = false
private var wasLeftShoulderPressed = false
private var wasRightShoulderPressed = false
@@ -44,6 +53,8 @@ public final class GamepadMenuInput {
public var onConfirm: (() -> Void)?
/// Button Y (or equivalent secondary action, e.g. "open library") edge-triggered.
public var onSecondary: (() -> Void)?
/// Button X (or equivalent tertiary action, e.g. "settings" / "delete") edge-triggered.
public var onTertiary: (() -> Void)?
/// Button B (or equivalent back/dismiss) edge-triggered.
public var onBack: (() -> Void)?
/// Shoulder buttons (L1 `false` / R1 `true`) edge-triggered fast-jump steps, optional per
@@ -63,6 +74,7 @@ public final class GamepadMenuInput {
public func start() {
guard !isActive else { return }
isActive = true
needsSnapshot = true
let timer = Timer(timeInterval: pollInterval, repeats: true) { [weak self] _ in
Task { @MainActor in self?.poll() }
}
@@ -79,6 +91,7 @@ public final class GamepadMenuInput {
currentDirection = nil
wasConfirmPressed = false
wasSecondaryPressed = false
wasTertiaryPressed = false
wasBackPressed = false
wasLeftShoulderPressed = false
wasRightShoulderPressed = false
@@ -89,8 +102,24 @@ public final class GamepadMenuInput {
private func poll() {
guard isActive, let gamepad = manager.active?.controller.extendedGamepad else { return }
if needsSnapshot {
// Adopt whatever is held right now without firing (see `needsSnapshot`): a button
// must be RELEASED after a handoff before it can act here, and a held direction only
// keeps moving once it changes or re-engages.
needsSnapshot = false
wasConfirmPressed = gamepad.buttonA.isPressed
wasSecondaryPressed = gamepad.buttonY.isPressed
wasTertiaryPressed = gamepad.buttonX.isPressed
wasBackPressed = gamepad.buttonB.isPressed
wasLeftShoulderPressed = gamepad.leftShoulder.isPressed
wasRightShoulderPressed = gamepad.rightShoulder.isPressed
currentDirection = directionFrom(gamepad)
return
}
edge(gamepad.buttonA.isPressed, &wasConfirmPressed) { onConfirm?() }
edge(gamepad.buttonY.isPressed, &wasSecondaryPressed) { onSecondary?() }
edge(gamepad.buttonX.isPressed, &wasTertiaryPressed) { onTertiary?() }
edge(gamepad.buttonB.isPressed, &wasBackPressed) { onBack?() }
edge(gamepad.leftShoulder.isPressed, &wasLeftShoulderPressed) { onShoulder?(false) }
edge(gamepad.rightShoulder.isPressed, &wasRightShoulderPressed) { onShoulder?(true) }
@@ -0,0 +1,26 @@
// Whether the iOS/iPadOS/macOS UI should be in its controller-friendly mode (the console-style
// host launcher, gamepad settings, and the coverflow library browser instead of the touch/desktop
// layouts). A pure function, not a singleton: the reactivity comes from callers already observing
// `GamepadManager.shared` and the `DefaultsKey.gamepadUIEnabled` @AppStorage themselves (the same
// local-read pattern SettingsView already uses for GamepadManager), so this stays the single place
// the two combine without adding a second ObservableObject or an environment key nobody else needs.
import Foundation
public enum GamepadUIEnvironment {
/// `enabledSetting` is the user's Settings toggle (`DefaultsKey.gamepadUIEnabled`);
/// `gamepadConnected` is `GamepadManager.shared.active != nil` active only once a usable
/// controller is actually attached (a non-extended-profile device leaves `active` nil, which
/// keeps the touch UI). A `Bool` rather than the `DiscoveredController` itself: this function's
/// whole job is the AND, so there's nothing else to inspect, and it keeps the helper testable
/// without a real `GCController` (which XCTest can't construct).
public static func isActive(gamepadConnected: Bool, enabledSetting: Bool) -> Bool {
enabledSetting && (gamepadConnected || forced)
}
/// Dev-only escape hatch (like ContentView's `PUNKTFUNK_AUTOCONNECT`): pretend a controller is
/// attached so the gamepad UI can be exercised/screenshotted without physical hardware
/// essential on a headless CI Mac and for `swift run` UI work. Never set in production.
private static let forced =
ProcessInfo.processInfo.environment["PUNKTFUNK_FORCE_GAMEPAD_UI"] == "1"
}
@@ -0,0 +1,62 @@
// The gamepad wire contract shared by capture (GamepadCapture), feedback (GamepadFeedback),
// and the tests button bits, axis ids, and the touchpad/motion unit conversions.
import Foundation
/// The gamepad wire contract (mirrors `punktfunk_core::input::gamepad`).
public enum GamepadWire {
public static let dpadUp: UInt32 = 0x0001
public static let dpadDown: UInt32 = 0x0002
public static let dpadLeft: UInt32 = 0x0004
public static let dpadRight: UInt32 = 0x0008
public static let start: UInt32 = 0x0010
public static let back: UInt32 = 0x0020
public static let leftStickClick: UInt32 = 0x0040
public static let rightStickClick: UInt32 = 0x0080
public static let leftShoulder: UInt32 = 0x0100
public static let rightShoulder: UInt32 = 0x0200
public static let guide: UInt32 = 0x0400
public static let a: UInt32 = 0x1000
public static let b: UInt32 = 0x2000
public static let x: UInt32 = 0x4000
public static let y: UInt32 = 0x8000
/// DualSense touchpad click (Moonlight's extended-button bit position).
public static let touchpadClick: UInt32 = 0x10_0000
public static let allButtons: [UInt32] = [
dpadUp, dpadDown, dpadLeft, dpadRight, start, back,
leftStickClick, rightStickClick, leftShoulder, rightShoulder, guide,
a, b, x, y, touchpadClick,
]
public static let axisLSX: UInt32 = 0
public static let axisLSY: UInt32 = 1
public static let axisRSX: UInt32 = 2
public static let axisRSY: UInt32 = 3
public static let axisLT: UInt32 = 4
public static let axisRT: UInt32 = 5
/// Raw DualSense gyro units per rad/s: hid-playstation's calibration over the host's
/// fixed blob resolves to 20 LSB per deg/s.
public static let gyroLSBPerRadS: Float = 20 * 180 / .pi
/// Raw DualSense accelerometer units per g (same derivation).
public static let accelLSBPerG: Float = 10_000
/// GC touchpad coordinates (±1, +y up) wire (0...65535, origin top-left, +y down).
public static func touchpad(x: Float, y: Float) -> (x: UInt16, y: UInt16) {
let wx = ((x.clamped(to: -1...1) + 1) / 2 * 65535).rounded()
let wy = ((1 - y.clamped(to: -1...1)) / 2 * 65535).rounded()
return (UInt16(wx), UInt16(wy))
}
/// Scale + clamp one motion component into the raw signed-16 sensor domain.
public static func motionRaw(_ value: Float, scale: Float) -> Int16 {
Int16((value * scale).rounded().clamped(to: Float(Int16.min)...Float(Int16.max)))
}
}
extension Float {
fileprivate func clamped(to range: ClosedRange<Float>) -> Float {
Swift.min(Swift.max(self, range.lowerBound), range.upperBound)
}
}
@@ -1,22 +1,3 @@
// Hostclient gamepad feedback rendering: one drain thread polls the rumble (0xCA) and
// HID-output (0xCD) planes and replays them on the active physical controller
//
// rumble CHHapticEngine players (per-handle localities when the pad has them,
// one combined engine otherwise),
// lightbar GCDeviceLight,
// player LEDs GCController.playerIndex (the DS bit patterns map to player 14),
// trigger FX DualSenseTriggerEffect.parse GCDualSenseAdaptiveTrigger.
//
// Only pad 0 is rendered (exactly one controller is forwarded). HID-output traffic exists
// only on PlayStation-pad sessions (a DualSense, or a DualShock 4 = lightbar only) the
// drain always polls both planes with short timeouts and never spins, so an Xbox session
// just renders rumble. GameController profile mutation
// happens on main; CHHapticEngine work on its own serial queue; the drain thread itself
// touches neither. When GamepadManager switches the active controller mid-session, the
// old pad is reset (triggers off, player index unset) and the last known feedback state
// is replayed onto the new one.
import Combine
import CoreHaptics
import Foundation
import GameController
@@ -24,21 +5,6 @@ import os
private let log = Logger(subsystem: "io.unom.punktfunk", category: "gamepad")
private final class FeedbackStopFlag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
}
/// Rumble CoreHaptics, isolated on one serial queue (CHHapticEngine is not main-bound,
/// but it isn't a free-for-all either). Engines are created lazily on the first nonzero
/// amplitude and torn down on retarget; players run only while their motor is on, so an
@@ -47,7 +13,7 @@ private final class FeedbackStopFlag: @unchecked Sendable {
///
/// `@unchecked Sendable` is sound because every property (`controller`/`low`/`high`/`broken`) is
/// read and written only inside `queue` closures the serial queue is the synchronization.
private final class RumbleRenderer: @unchecked Sendable {
final class RumbleRenderer: @unchecked Sendable {
private let queue = DispatchQueue(label: "io.unom.punktfunk.haptics", qos: .userInteractive)
/// One actuator's started engine plus the player currently driving it (nil = idle). The
@@ -316,248 +282,3 @@ private final class RumbleRenderer: @unchecked Sendable {
return c == nil ? "" : "CoreHaptics"
}
}
public final class GamepadFeedback {
private let connection: PunktfunkConnection
private let flag = FeedbackStopFlag()
private let drainDone = DispatchSemaphore(value: 0)
private var drainStarted = false
private let rumble = RumbleRenderer()
private var activeSub: AnyCancellable?
// Last applied feedback (main-actor) replayed when the active controller changes.
@MainActor private var target: GCController?
@MainActor private var lastLight: (r: UInt8, g: UInt8, b: UInt8)?
@MainActor private var lastPlayerBits: UInt8?
@MainActor private var lastTrigger: [DualSenseTriggerEffect?] = [nil, nil]
public init(connection: PunktfunkConnection, manager: GamepadManager) {
self.connection = connection
// Capture self weakly in the hop too, so the inner sink's weak capture isn't shadowing
// an implicit strong one and the subscription (stored on self) never retain-cycles.
Task { @MainActor [weak self] in
guard let self else { return }
self.activeSub = manager.$active.sink { [weak self] dc in
MainActor.assumeIsolated { self?.retarget(dc?.controller) }
}
}
}
/// Safety net: the drain thread captures `connection` strongly and only `self` weakly, so if
/// this is dropped without `stop()` (an abrupt teardown) the thread would poll forever and
/// leak the connection signal it to exit. (`stop()` is the normal path and also joins it.)
deinit { flag.stop() }
/// Map the DualSense player-LED bit patterns (5 LEDs, hid-playstation's player
/// conventions) onto GCControllerPlayerIndex. Unknown patterns fall back to the lit
/// count, clamped to the four indices GC offers.
public static func playerIndex(forBits bits: UInt8) -> GCControllerPlayerIndex {
switch bits & 0x1F {
case 0: return .indexUnset
case 0b00100: return .index1
case 0b01010: return .index2
case 0b10101: return .index3
case 0b11011: return .index4
default:
let lit = (bits & 0x1F).nonzeroBitCount
return GCControllerPlayerIndex(rawValue: min(lit, 4) - 1) ?? .index1
}
}
public func start() {
guard !drainStarted else { return }
drainStarted = true
// Hidout traffic (lightbar / player LEDs / triggers) only exists on a PlayStation-pad
// session a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and
// let rumble own the wait elsewhere; on an Xbox session it stays nonblocking.
let thread = Thread { [connection, flag, drainDone, weak self] in
while !flag.isStopped {
do {
// Poll the feedback planes NON-BLOCKING. A blocking poll (timeoutMs > 0) holds
// the connection's shared feedback lock for its whole wait; the video pump drains
// HDR mastering metadata (nextHdrMeta) on the SAME lock every frame, so a blocking
// poll here starved it and throttled HDR to ~1 fps (SDR, which never drains HDR
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
if let r = try connection.nextRumble(timeoutMs: 0), r.pad == 0 {
self?.rumble.apply(low: r.low, high: r.high)
}
// Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
var burst = 0
while burst < 64, !flag.isStopped,
let ev = try connection.nextHidOutput(timeoutMs: 0) {
self?.render(ev)
burst += 1
}
} catch {
break // .closed (or fatal) the session is over
}
// ~8 ms poll cadence (125 Hz), slept OUTSIDE the feedback lock low rumble/HID
// latency without holding the lock the HDR-meta drain needs.
if !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
}
drainDone.signal()
}
thread.name = "punktfunk-feedback"
thread.qualityOfService = .userInteractive
thread.start()
}
/// Stop the drain and silence the motors. Blocks until the drain thread exits ( one
/// poll cycle) call off the main actor, before `connection.close()`.
public func stop() {
flag.stop()
if drainStarted {
drainDone.wait()
drainStarted = false
}
rumble.stop()
// Drop the retarget subscription and the dead session's cached feedback a
// controller change after teardown must not replay this session's triggers/LEDs.
Task { @MainActor in
self.activeSub = nil
self.lastLight = nil
self.lastPlayerBits = nil
self.lastTrigger = [nil, nil]
self.reset(self.target)
self.target = nil
}
}
private func render(_ ev: PunktfunkConnection.HidOutputEvent) {
DispatchQueue.main.async {
MainActor.assumeIsolated { self.apply(ev) }
}
}
@MainActor
private func apply(_ ev: PunktfunkConnection.HidOutputEvent) {
switch ev {
case let .led(pad, r, g, b):
guard pad == 0 else { return }
lastLight = (r, g, b)
target?.light?.color = GCColor(
red: Float(r) / 255, green: Float(g) / 255, blue: Float(b) / 255)
case let .playerLEDs(pad, bits):
guard pad == 0 else { return }
lastPlayerBits = bits
target?.playerIndex = Self.playerIndex(forBits: bits)
case let .triggerEffect(pad, which, effect):
guard pad == 0, which < 2 else { return }
let parsed = DualSenseTriggerEffect.parse(effect)
lastTrigger[Int(which)] = parsed
if let trigger = adaptiveTrigger(which) {
parsed.apply(to: trigger)
}
}
}
@MainActor
private func retarget(_ controller: GCController?) {
guard controller !== target else { return }
reset(target)
target = controller
rumble.retarget(controller)
// Replay the session's feedback state so a swapped-in controller looks the same.
if let (r, g, b) = lastLight {
controller?.light?.color = GCColor(
red: Float(r) / 255, green: Float(g) / 255, blue: Float(b) / 255)
}
if let bits = lastPlayerBits {
controller?.playerIndex = Self.playerIndex(forBits: bits)
}
for which in 0..<2 {
if let effect = lastTrigger[which], let trigger = adaptiveTrigger(UInt8(which)) {
effect.apply(to: trigger)
}
}
}
@MainActor
private func reset(_ controller: GCController?) {
guard let c = controller else { return }
c.playerIndex = .indexUnset
if let ds = c.extendedGamepad as? GCDualSenseGamepad {
ds.leftTrigger.setModeOff()
ds.rightTrigger.setModeOff()
}
}
@MainActor
private func adaptiveTrigger(_ which: UInt8) -> GCDualSenseAdaptiveTrigger? {
guard let ds = target?.extendedGamepad as? GCDualSenseGamepad else { return nil }
return which == 0 ? ds.leftTrigger : ds.rightTrigger
}
}
#if DEBUG
/// Local feedback driver for the Settings Controllers "Test Controller" panel (DEBUG builds
/// only). It drives the SAME CoreHaptics rumble renderer and `DualSenseTriggerEffect` path a
/// live session uses just aimed at the physically-connected controller instead of the
/// hostclient feedback planes so rumble, the adaptive triggers, the lightbar and the player
/// LEDs can be confirmed on-device without a host. Reusing the real renderers is the point:
/// a passing test exercises the exact code a session runs.
@MainActor
public final class ControllerTester: ObservableObject {
private let renderer = RumbleRenderer()
private weak var controller: GCController?
/// The rumble backend now in use "DualSense HID · USB/Bluetooth", "CoreHaptics", or ""
/// for the test panel to display so it's obvious which path a given pad takes.
@Published public private(set) var rumbleBackend = ""
public init() {}
/// Aim the feedback at a controller (nil releases it). Idempotent safe to call on every
/// active-controller change.
public func target(_ c: GCController?) {
guard c !== controller else { return }
controller = c
renderer.retarget(c) { [weak self] note in
Task { @MainActor in self?.rumbleBackend = note }
}
}
/// Drive both motors at 0...1 amplitudes low = left/heavy, high = right/light mapped to
/// the 0...0xFFFF wire range the session carries, through the real `RumbleRenderer`.
public func rumble(low: Float, high: Float) {
func u16(_ v: Float) -> UInt16 { UInt16((min(max(v, 0), 1) * 65535).rounded()) }
renderer.apply(low: u16(low), high: u16(high))
}
public func stopRumble() { renderer.apply(low: 0, high: 0) }
/// Replay an adaptive-trigger effect on a DualSense via the real `DualSenseTriggerEffect`
/// renderer. `right == false` L2, `true` R2. No-op on a non-DualSense pad.
public func applyTrigger(_ effect: DualSenseTriggerEffect, right: Bool) {
guard let ds = controller?.extendedGamepad as? GCDualSenseGamepad else { return }
effect.apply(to: right ? ds.rightTrigger : ds.leftTrigger)
}
public func resetTriggers() {
guard let ds = controller?.extendedGamepad as? GCDualSenseGamepad else { return }
ds.leftTrigger.setModeOff()
ds.rightTrigger.setModeOff()
}
/// Lightbar colour (DualSense / DualShock 4); nil turns it off. No-op without a light.
public func setLight(_ color: GCColor?) {
controller?.light?.color = color ?? GCColor(red: 0, green: 0, blue: 0)
}
/// Player-indicator LEDs (`.index1`...`.index4`, or `.indexUnset` to clear).
public func setPlayerIndex(_ index: GCControllerPlayerIndex) {
controller?.playerIndex = index
}
/// Silence every channel and release the controller call on the panel's disappear.
public func stop() {
resetTriggers()
setPlayerIndex(.indexUnset)
setLight(nil)
renderer.retarget(nil) // async teardown: stops the motors + drops the controller ref
controller = nil
}
}
#endif
@@ -1,20 +0,0 @@
// Whether the iOS/iPadOS UI should be in its controller-friendly mode (larger focus targets on
// the host grid, the coverflow library browser instead of the plain grid). A pure function, not a
// singleton: the reactivity comes from callers already observing `GamepadManager.shared` and the
// `DefaultsKey.gamepadUIEnabled` @AppStorage themselves (the same local-read pattern SettingsView
// already uses for GamepadManager), so this stays the single place the two combine without adding
// a second ObservableObject or an environment key nobody else needs.
import Foundation
public enum GamepadUIEnvironment {
/// `enabledSetting` is the user's Settings toggle (`DefaultsKey.gamepadUIEnabled`);
/// `gamepadConnected` is `GamepadManager.shared.active != nil` active only once a usable
/// controller is actually attached (a non-extended-profile device leaves `active` nil, which
/// keeps the touch UI). A `Bool` rather than the `DiscoveredController` itself: this function's
/// whole job is the AND, so there's nothing else to inspect, and it keeps the helper testable
/// without a real `GCController` (which XCTest can't construct).
public static func isActive(gamepadConnected: Bool, enabledSetting: Bool) -> Bool {
enabledSetting && gamepadConnected
}
}
@@ -571,102 +571,4 @@ public final class InputCapture {
}
#endif
}
/// HID usage (GCKeyCode raw) Windows VK (the host maps VK evdev; every VK emitted
/// here exists in punktfunk-host/src/inject.rs::vk_to_evdev extend the two together).
static let hidToVK: [Int: UInt32] = {
var m: [Int: UInt32] = [:]
// az: HID 0x04..0x1D VK 'A'..'Z'.
for i in 0..<26 { m[0x04 + i] = UInt32(0x41 + i) }
// 19: HID 0x1E..0x26 VK '1'..'9'; then 0: HID 0x27 VK '0' (set separately
// the '0' key sits AFTER '9' in HID but its VK 0x30 sits BEFORE '1' (0x31)).
for i in 0..<9 { m[0x1E + i] = UInt32(0x31 + i) }
m[0x27] = 0x30
m[0x28] = 0x0D // return
m[0x29] = 0x1B // escape
m[0x2A] = 0x08 // backspace
m[0x2B] = 0x09 // tab
m[0x2C] = 0x20 // space
m[0x2D] = 0xBD; m[0x2E] = 0xBB // - =
m[0x2F] = 0xDB; m[0x30] = 0xDD; m[0x31] = 0xDC // [ ] backslash
m[0x33] = 0xBA; m[0x34] = 0xDE; m[0x35] = 0xC0 // ; ' `
m[0x36] = 0xBC; m[0x37] = 0xBE; m[0x38] = 0xBF // , . /
m[0x39] = 0x14 // caps lock
// F1..F12: HID 0x3A..0x45 VK 0x70..0x7B.
for i in 0..<12 { m[0x3A + i] = UInt32(0x70 + i) }
m[0x46] = 0x2C; m[0x47] = 0x91; m[0x48] = 0x13 // printscreen scrolllock pause
m[0x4F] = 0x27; m[0x50] = 0x25; m[0x51] = 0x28; m[0x52] = 0x26 // arrows R L D U
m[0x49] = 0x2D; m[0x4A] = 0x24; m[0x4B] = 0x21 // insert home pageup
m[0x4C] = 0x2E; m[0x4D] = 0x23; m[0x4E] = 0x22 // delete end pagedown
// Keypad: NumLock, / * - +, Enter, 1..9, 0, decimal. KP Enter goes as
// VK_SEPARATOR (0x6C) this host maps it to KEY_KPENTER (Windows itself would
// send VK_RETURN+extended, which vk_to_evdev can't distinguish).
m[0x53] = 0x90
m[0x54] = 0x6F; m[0x55] = 0x6A; m[0x56] = 0x6D; m[0x57] = 0x6B
m[0x58] = 0x6C
for i in 0..<9 { m[0x59 + i] = UInt32(0x61 + i) }
m[0x62] = 0x60; m[0x63] = 0x6E
m[0x64] = 0xE2 // ISO 102nd key (<> next to left shift on ISO layouts)
m[0x65] = 0x5D // menu/application
m[0xE0] = 0xA2; m[0xE1] = 0xA0; m[0xE2] = 0xA4; m[0xE3] = 0x5B // Lctrl Lshift Lalt Lcmd
m[0xE4] = 0xA3; m[0xE5] = 0xA1; m[0xE6] = 0xA5; m[0xE7] = 0x5C // Rctrl Rshift Ralt Rcmd
return m
}()
#if os(macOS)
/// NSEvent.keyCode (Carbon virtual keycode, kVK_*) Windows VK. The macOS NSEvent key
/// path is keyed by keyCode (a layout-independent hardware position), NOT by HID usage,
/// so it needs its own table but it emits the EXACT SAME Windows VK integers `hidToVK`
/// already produces for each physical key (A0x41, Return0x0D, KeypadEnter0x6C, ), so
/// the host's vk_to_evdev (inject.rs) accepts both with zero change. Modifier keys come
/// via flagsChanged (handleFlagsChanged), not keyDown, so they're absent here. Keys with
/// no host evdev arm (F13F20, KeypadEquals, the Fn key) are omitted nil swallowed.
static let keyCodeToVK: [UInt16: UInt32] = {
var m: [UInt16: UInt32] = [:]
// Letters kVK_ANSI_A..Z (scattered keycodes) VK 'A'..'Z'.
m[0x00] = 0x41; m[0x01] = 0x53; m[0x02] = 0x44; m[0x03] = 0x46 // A S D F
m[0x04] = 0x48; m[0x05] = 0x47; m[0x06] = 0x5A; m[0x07] = 0x58 // H G Z X
m[0x08] = 0x43; m[0x09] = 0x56; m[0x0B] = 0x42; m[0x0C] = 0x51 // C V B Q
m[0x0D] = 0x57; m[0x0E] = 0x45; m[0x0F] = 0x52; m[0x10] = 0x59 // W E R Y
m[0x11] = 0x54; m[0x1F] = 0x4F; m[0x20] = 0x55; m[0x22] = 0x49 // T O U I
m[0x23] = 0x50; m[0x25] = 0x4C; m[0x26] = 0x4A; m[0x28] = 0x4B // P L J K
m[0x2D] = 0x4E; m[0x2E] = 0x4D // N M
// Digit row kVK_ANSI_1..0 (scattered) VK '1'..'9','0'.
m[0x12] = 0x31; m[0x13] = 0x32; m[0x14] = 0x33; m[0x15] = 0x34 // 1 2 3 4
m[0x16] = 0x36; m[0x17] = 0x35; m[0x19] = 0x39; m[0x1A] = 0x37 // 6 5 9 7
m[0x1C] = 0x38; m[0x1D] = 0x30 // 8 0
// Whitespace / control.
m[0x24] = 0x0D // return
m[0x30] = 0x09 // tab
m[0x31] = 0x20 // space
m[0x33] = 0x08 // delete (backspace)
m[0x35] = 0x1B // escape
m[0x75] = 0x2E // forward delete (VK_DELETE)
m[0x39] = 0x14 // caps lock
// Punctuation (US ANSI) + ISO 102nd key.
m[0x1B] = 0xBD; m[0x18] = 0xBB // - = (OEM_MINUS OEM_PLUS)
m[0x21] = 0xDB; m[0x1E] = 0xDD; m[0x2A] = 0xDC // [ ] backslash (OEM_4 6 5)
m[0x29] = 0xBA; m[0x27] = 0xDE; m[0x32] = 0xC0 // ; ' ` (OEM_1 7 3)
m[0x2B] = 0xBC; m[0x2F] = 0xBE; m[0x2C] = 0xBF // , . / (OEM_COMMA PERIOD 2)
m[0x0A] = 0xE2 // ISO 102nd key (<> next to left shift; OEM_102)
// Function keys F1..F12 (scattered) VK 0x70..0x7B. F13+ omitted (no host arm).
m[0x7A] = 0x70; m[0x78] = 0x71; m[0x63] = 0x72; m[0x76] = 0x73 // F1 F2 F3 F4
m[0x60] = 0x74; m[0x61] = 0x75; m[0x62] = 0x76; m[0x64] = 0x77 // F5 F6 F7 F8
m[0x65] = 0x78; m[0x6D] = 0x79; m[0x67] = 0x7A; m[0x6F] = 0x7B // F9 F10 F11 F12
// Arrows.
m[0x7B] = 0x25; m[0x7C] = 0x27; m[0x7D] = 0x28; m[0x7E] = 0x26 // left right down up
// Nav cluster (Apple keycodes; Help sits where Insert is).
m[0x72] = 0x2D; m[0x73] = 0x24; m[0x74] = 0x21 // insert home pageup
m[0x77] = 0x23; m[0x79] = 0x22 // end pagedown (forward-delete handled above)
// Keypad kVK_ANSI_Keypad0..9 (scattered) VK_NUMPAD0..9, plus the operators.
m[0x52] = 0x60; m[0x53] = 0x61; m[0x54] = 0x62; m[0x55] = 0x63 // KP0 KP1 KP2 KP3
m[0x56] = 0x64; m[0x57] = 0x65; m[0x58] = 0x66; m[0x59] = 0x67 // KP4 KP5 KP6 KP7
m[0x5B] = 0x68; m[0x5C] = 0x69 // KP8 KP9
m[0x41] = 0x6E; m[0x43] = 0x6A; m[0x45] = 0x6B // KP decimal multiply plus
m[0x4E] = 0x6D; m[0x4B] = 0x6F // KP minus divide
m[0x4C] = 0x6C // KP enter VK_SEPARATOR (host maps to KEY_KPENTER, matching hidToVK)
m[0x47] = 0x90 // KP clear sits where NumLock is VK_NUMLOCK. (KP equals 0x51 dropped.)
return m
}()
#endif
}
@@ -0,0 +1,102 @@
// InputCapture's static keymap tables: HID usage Windows VK (the GCKeyboard path on all
// platforms) and, on macOS, NSEvent.keyCode Windows VK (the NSEvent key path).
extension InputCapture {
/// HID usage (GCKeyCode raw) Windows VK (the host maps VK evdev; every VK emitted
/// here exists in punktfunk-host/src/inject.rs::vk_to_evdev extend the two together).
static let hidToVK: [Int: UInt32] = {
var m: [Int: UInt32] = [:]
// az: HID 0x04..0x1D VK 'A'..'Z'.
for i in 0..<26 { m[0x04 + i] = UInt32(0x41 + i) }
// 19: HID 0x1E..0x26 VK '1'..'9'; then 0: HID 0x27 VK '0' (set separately
// the '0' key sits AFTER '9' in HID but its VK 0x30 sits BEFORE '1' (0x31)).
for i in 0..<9 { m[0x1E + i] = UInt32(0x31 + i) }
m[0x27] = 0x30
m[0x28] = 0x0D // return
m[0x29] = 0x1B // escape
m[0x2A] = 0x08 // backspace
m[0x2B] = 0x09 // tab
m[0x2C] = 0x20 // space
m[0x2D] = 0xBD; m[0x2E] = 0xBB // - =
m[0x2F] = 0xDB; m[0x30] = 0xDD; m[0x31] = 0xDC // [ ] backslash
m[0x33] = 0xBA; m[0x34] = 0xDE; m[0x35] = 0xC0 // ; ' `
m[0x36] = 0xBC; m[0x37] = 0xBE; m[0x38] = 0xBF // , . /
m[0x39] = 0x14 // caps lock
// F1..F12: HID 0x3A..0x45 VK 0x70..0x7B.
for i in 0..<12 { m[0x3A + i] = UInt32(0x70 + i) }
m[0x46] = 0x2C; m[0x47] = 0x91; m[0x48] = 0x13 // printscreen scrolllock pause
m[0x4F] = 0x27; m[0x50] = 0x25; m[0x51] = 0x28; m[0x52] = 0x26 // arrows R L D U
m[0x49] = 0x2D; m[0x4A] = 0x24; m[0x4B] = 0x21 // insert home pageup
m[0x4C] = 0x2E; m[0x4D] = 0x23; m[0x4E] = 0x22 // delete end pagedown
// Keypad: NumLock, / * - +, Enter, 1..9, 0, decimal. KP Enter goes as
// VK_SEPARATOR (0x6C) this host maps it to KEY_KPENTER (Windows itself would
// send VK_RETURN+extended, which vk_to_evdev can't distinguish).
m[0x53] = 0x90
m[0x54] = 0x6F; m[0x55] = 0x6A; m[0x56] = 0x6D; m[0x57] = 0x6B
m[0x58] = 0x6C
for i in 0..<9 { m[0x59 + i] = UInt32(0x61 + i) }
m[0x62] = 0x60; m[0x63] = 0x6E
m[0x64] = 0xE2 // ISO 102nd key (<> next to left shift on ISO layouts)
m[0x65] = 0x5D // menu/application
m[0xE0] = 0xA2; m[0xE1] = 0xA0; m[0xE2] = 0xA4; m[0xE3] = 0x5B // Lctrl Lshift Lalt Lcmd
m[0xE4] = 0xA3; m[0xE5] = 0xA1; m[0xE6] = 0xA5; m[0xE7] = 0x5C // Rctrl Rshift Ralt Rcmd
return m
}()
#if os(macOS)
/// NSEvent.keyCode (Carbon virtual keycode, kVK_*) Windows VK. The macOS NSEvent key
/// path is keyed by keyCode (a layout-independent hardware position), NOT by HID usage,
/// so it needs its own table but it emits the EXACT SAME Windows VK integers `hidToVK`
/// already produces for each physical key (A0x41, Return0x0D, KeypadEnter0x6C, ), so
/// the host's vk_to_evdev (inject.rs) accepts both with zero change. Modifier keys come
/// via flagsChanged (handleFlagsChanged), not keyDown, so they're absent here. Keys with
/// no host evdev arm (F13F20, KeypadEquals, the Fn key) are omitted nil swallowed.
static let keyCodeToVK: [UInt16: UInt32] = {
var m: [UInt16: UInt32] = [:]
// Letters kVK_ANSI_A..Z (scattered keycodes) VK 'A'..'Z'.
m[0x00] = 0x41; m[0x01] = 0x53; m[0x02] = 0x44; m[0x03] = 0x46 // A S D F
m[0x04] = 0x48; m[0x05] = 0x47; m[0x06] = 0x5A; m[0x07] = 0x58 // H G Z X
m[0x08] = 0x43; m[0x09] = 0x56; m[0x0B] = 0x42; m[0x0C] = 0x51 // C V B Q
m[0x0D] = 0x57; m[0x0E] = 0x45; m[0x0F] = 0x52; m[0x10] = 0x59 // W E R Y
m[0x11] = 0x54; m[0x1F] = 0x4F; m[0x20] = 0x55; m[0x22] = 0x49 // T O U I
m[0x23] = 0x50; m[0x25] = 0x4C; m[0x26] = 0x4A; m[0x28] = 0x4B // P L J K
m[0x2D] = 0x4E; m[0x2E] = 0x4D // N M
// Digit row kVK_ANSI_1..0 (scattered) VK '1'..'9','0'.
m[0x12] = 0x31; m[0x13] = 0x32; m[0x14] = 0x33; m[0x15] = 0x34 // 1 2 3 4
m[0x16] = 0x36; m[0x17] = 0x35; m[0x19] = 0x39; m[0x1A] = 0x37 // 6 5 9 7
m[0x1C] = 0x38; m[0x1D] = 0x30 // 8 0
// Whitespace / control.
m[0x24] = 0x0D // return
m[0x30] = 0x09 // tab
m[0x31] = 0x20 // space
m[0x33] = 0x08 // delete (backspace)
m[0x35] = 0x1B // escape
m[0x75] = 0x2E // forward delete (VK_DELETE)
m[0x39] = 0x14 // caps lock
// Punctuation (US ANSI) + ISO 102nd key.
m[0x1B] = 0xBD; m[0x18] = 0xBB // - = (OEM_MINUS OEM_PLUS)
m[0x21] = 0xDB; m[0x1E] = 0xDD; m[0x2A] = 0xDC // [ ] backslash (OEM_4 6 5)
m[0x29] = 0xBA; m[0x27] = 0xDE; m[0x32] = 0xC0 // ; ' ` (OEM_1 7 3)
m[0x2B] = 0xBC; m[0x2F] = 0xBE; m[0x2C] = 0xBF // , . / (OEM_COMMA PERIOD 2)
m[0x0A] = 0xE2 // ISO 102nd key (<> next to left shift; OEM_102)
// Function keys F1..F12 (scattered) VK 0x70..0x7B. F13+ omitted (no host arm).
m[0x7A] = 0x70; m[0x78] = 0x71; m[0x63] = 0x72; m[0x76] = 0x73 // F1 F2 F3 F4
m[0x60] = 0x74; m[0x61] = 0x75; m[0x62] = 0x76; m[0x64] = 0x77 // F5 F6 F7 F8
m[0x65] = 0x78; m[0x6D] = 0x79; m[0x67] = 0x7A; m[0x6F] = 0x7B // F9 F10 F11 F12
// Arrows.
m[0x7B] = 0x25; m[0x7C] = 0x27; m[0x7D] = 0x28; m[0x7E] = 0x26 // left right down up
// Nav cluster (Apple keycodes; Help sits where Insert is).
m[0x72] = 0x2D; m[0x73] = 0x24; m[0x74] = 0x21 // insert home pageup
m[0x77] = 0x23; m[0x79] = 0x22 // end pagedown (forward-delete handled above)
// Keypad kVK_ANSI_Keypad0..9 (scattered) VK_NUMPAD0..9, plus the operators.
m[0x52] = 0x60; m[0x53] = 0x61; m[0x54] = 0x62; m[0x55] = 0x63 // KP0 KP1 KP2 KP3
m[0x56] = 0x64; m[0x57] = 0x65; m[0x58] = 0x66; m[0x59] = 0x67 // KP4 KP5 KP6 KP7
m[0x5B] = 0x68; m[0x5C] = 0x69 // KP8 KP9
m[0x41] = 0x6E; m[0x43] = 0x6A; m[0x45] = 0x6B // KP decimal multiply plus
m[0x4E] = 0x6D; m[0x4B] = 0x6F // KP minus divide
m[0x4C] = 0x6C // KP enter VK_SEPARATOR (host maps to KEY_KPENTER, matching hidToVK)
m[0x47] = 0x90 // KP clear sits where NumLock is VK_NUMLOCK. (KP equals 0x51 dropped.)
return m
}()
#endif
}
@@ -0,0 +1,9 @@
import CoreGraphics
extension CGFloat {
/// Clamp into `range` keeps the absolute-cursor mapping inside the host's pixel
/// bounds even if a stray event reports a point a hair past the video rect.
func clamped(to range: ClosedRange<CGFloat>) -> CGFloat {
Swift.min(Swift.max(self, range.lowerBound), range.upperBound)
}
}
@@ -51,8 +51,8 @@ public enum DefaultsKey {
/// Which corner the statistics overlay sits in a `HUDPlacement` raw value
/// ("topLeading"/"topTrailing"/"bottomLeading"/"bottomTrailing"). Default top-trailing.
public static let hudPlacement = "punktfunk.hudPlacement"
/// iOS/iPadOS: switch the host list and game library to a controller-friendly layout
/// (larger focus targets, a coverflow-style library) whenever a gamepad is connected. On by
/// default; see `GamepadUIEnvironment.isActive`.
/// iOS/iPadOS/macOS: switch the host list, settings and game library to a controller-friendly
/// layout (the console launcher, gamepad-navigable settings, a coverflow-style library)
/// whenever a gamepad is connected. On by default; see `GamepadUIEnvironment.isActive`.
public static let gamepadUIEnabled = "punktfunk.gamepadUIEnabled"
}
@@ -0,0 +1,21 @@
// One NSLock-guarded boolean, set once: the cancellation handle shared by the session services
// (the two video pumps, audio playback/mic, gamepad feedback). Each start() creates a fresh flag
// and hands it to its worker thread(s); stop() sets it permanently, so a stale worker can never
// be revived by a newer start.
import Foundation
final class StopFlag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
}
@@ -0,0 +1,265 @@
// Annex-B (HEVC / H.264) CoreMedia plumbing.
//
// The punktfunk host emits Annex-B access units with in-band parameter sets on every IDR
// (deliberately the client needs no out-of-band extradata). VideoToolbox wants the AVCC
// flavor instead: a CMVideoFormatDescription built from the parameter sets, and sample
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
// the codec the host resolved in the Welcome (`connection.videoCodec`) HEVC and H.264
// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
// is not an Annex-B/NAL codec and isn't handled here (hosts don't emit it on the native
// path yet).
//
// HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
// once per AU, so the conversion is built on `forEachNAL` a zero-copy scan over the AU's bytes
// (ranges, not materialized Datas) and `sampleBuffer` packs the AVCC form straight into
// the CMBlockBuffer's own allocation. Per AU that leaves exactly one copy here (source
// block buffer) instead of the naive scan-copy-slice-repack chain.
import CoreMedia
import Foundation
/// The video codec of the host's elementary stream negotiated in the Welcome and read via
/// `punktfunk_connection_codec`.
public enum VideoCodec: Equatable {
case h264
case hevc
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown HEVC).
public init(wire: UInt8) {
self = wire == 0x01 ? .h264 : .hevc // 0x01 = PUNKTFUNK_CODEC_H264
}
/// NAL unit type from a NAL's first byte. HEVC: bits 1..6; H.264: bits 0..4.
fileprivate func nalType(_ first: UInt8) -> UInt8 {
self == .hevc ? (first >> 1) & 0x3F : first & 0x1F
}
/// True for a parameter-set NAL (dropped from AVCC; kept for the format description).
/// HEVC: VPS 32 / SPS 33 / PPS 34. H.264: SPS 7 / PPS 8 (no VPS).
fileprivate func isParameterSet(_ first: UInt8) -> Bool {
let t = nalType(first)
return self == .hevc ? (32...34).contains(t) : t == 7 || t == 8
}
/// True for a VCL (slice) NAL in a conforming AU no parameter set follows the first one,
/// so the format-description scan can stop there.
fileprivate func isVCL(_ first: UInt8) -> Bool {
let t = nalType(first)
return self == .hevc ? t <= 31 : (1...5).contains(t)
}
}
public enum AnnexB {
/// Walk the NAL units of `data` without copying: `body` receives the buffer base and each
/// NAL's byte range (start codes 00 00 01 / 00 00 00 01 excluded), and returns false to
/// stop the walk early (e.g. at the first VCL NAL). All zeros immediately preceding a
/// start code are dropped: they're either the 4-byte-code prefix or `trailing_zero_8bits`
/// padding, never NAL payload (emulation prevention keeps 00 00 0x out of conforming NAL
/// bytes) same policy as ffmpeg. The base pointer is only valid inside `body`.
static func forEachNAL(
in data: Data, _ body: (_ base: UnsafePointer<UInt8>, _ range: Range<Int>) -> Bool
) {
data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return }
let count = raw.count
var i = 0
var start = -1
while i + 2 < count {
if base[i] == 0, base[i + 1] == 0, base[i + 2] == 1 {
var codeStart = i
while codeStart > 0, base[codeStart - 1] == 0 {
codeStart -= 1
}
if start >= 0, start < codeStart, !body(base, start..<codeStart) { return }
start = i + 3
i += 3
} else {
i += 1
}
}
if start >= 0, start < count {
_ = body(base, start..<count)
}
}
}
/// Split an Annex-B stream into NAL units (start codes stripped see `forEachNAL` for
/// the boundary policy). Materializes a Data per NAL; the streaming paths use
/// `forEachNAL` directly instead.
public static func nalUnits(in data: Data) -> [Data] {
var nals: [Data] = []
forEachNAL(in: data) { base, range in
nals.append(Data(bytes: base + range.lowerBound, count: range.count))
return true
}
return nals
}
/// HEVC NAL unit type (bits 1..6 of the first byte).
public static func hevcNalType(_ nal: Data) -> UInt8 {
guard let first = nal.first else { return 0xFF }
return (first >> 1) & 0x3F
}
/// H.264 NAL unit type (bits 0..4 of the first byte).
public static func h264NalType(_ nal: Data) -> UInt8 {
guard let first = nal.first else { return 0xFF }
return first & 0x1F
}
/// Build a format description from an IDR AU's in-band parameter sets (HEVC: VPS/SPS/PPS;
/// H.264: SPS/PPS). Returns nil when the AU carries no parameter sets (non-IDR). Runs per
/// AU on the pump thread: parameter sets precede the first VCL NAL in a conforming AU, so
/// the scan stops there a delta frame (no leading parameter sets) costs a few byte
/// compares, no copies.
public static func formatDescription(
fromIDR au: Data, codec: VideoCodec
) -> CMVideoFormatDescription? {
var vps: Data?, sps: Data?, pps: Data?
forEachNAL(in: au) { base, range in
let first = base[range.lowerBound]
switch codec.nalType(first) {
case 32 where codec == .hevc:
vps = Data(bytes: base + range.lowerBound, count: range.count)
case 33 where codec == .hevc, 7 where codec == .h264:
sps = Data(bytes: base + range.lowerBound, count: range.count)
case 34 where codec == .hevc, 8 where codec == .h264:
pps = Data(bytes: base + range.lowerBound, count: range.count)
default:
if codec.isVCL(first) { return false } // no parameter sets can follow
// AUD/SEI/ may precede the slices; keep scanning.
}
return true
}
guard let sps, let pps else { return nil }
// In the order VideoToolbox wants them: HEVC VPS,SPS,PPS (VPS required); H.264 SPS,PPS.
let sets: [Data]
switch codec {
case .hevc:
guard let vps else { return nil }
sets = [vps, sps, pps]
case .h264:
sets = [sps, pps]
}
var format: CMVideoFormatDescription?
// Pin every parameter set's bytes for the duration of the create call, then hand
// VideoToolbox parallel pointer/size arrays.
var pointers: [UnsafePointer<UInt8>] = []
var sizes: [Int] = []
func withAll(_ i: Int, _ body: () -> Void) {
if i == sets.count { body(); return }
sets[i].withUnsafeBytes { raw in
pointers.append(raw.bindMemory(to: UInt8.self).baseAddress!)
sizes.append(sets[i].count)
withAll(i + 1, body)
}
}
var status: OSStatus = -1
withAll(0) {
switch codec {
case .hevc:
status = CMVideoFormatDescriptionCreateFromHEVCParameterSets(
allocator: kCFAllocatorDefault,
parameterSetCount: pointers.count,
parameterSetPointers: pointers,
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
extensions: nil,
formatDescriptionOut: &format)
case .h264:
status = CMVideoFormatDescriptionCreateFromH264ParameterSets(
allocator: kCFAllocatorDefault,
parameterSetCount: pointers.count,
parameterSetPointers: pointers,
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
formatDescriptionOut: &format)
}
}
return status == noErr ? format : nil
}
/// Re-pack an Annex-B AU as AVCC (4-byte big-endian length before each NAL), dropping
/// the parameter-set NALs (they live in the format description).
public static func avcc(from au: Data, codec: VideoCodec) -> Data {
var out = Data(capacity: au.count + 16)
forEachNAL(in: au) { base, range in
if codec.isParameterSet(base[range.lowerBound]) { return true }
var len = UInt32(range.count).bigEndian
withUnsafeBytes(of: &len) { out.append(contentsOf: $0) }
out.append(UnsafeBufferPointer(start: base + range.lowerBound, count: range.count))
return true
}
return out
}
/// Wrap one AU as a decode-ready CMSampleBuffer. The AVCC form is packed directly into
/// the CMBlockBuffer's allocation (sized by a first cheap scan) no intermediate Data.
public static func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription, codec: VideoCodec
) -> CMSampleBuffer? {
// Pass 1: byte scan only total AVCC size of the payload (non-parameter-set) NALs.
var total = 0
forEachNAL(in: au.data) { base, range in
if !codec.isParameterSet(base[range.lowerBound]) { total += 4 + range.count }
return true
}
// Nothing decodable (a parameter-set-only AU our host never sends one): drop it
// rather than hand the decoder an empty sample.
guard total > 0 else { return nil }
var blockBuffer: CMBlockBuffer?
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault, memoryBlock: nil,
blockLength: total, blockAllocator: kCFAllocatorDefault,
customBlockSource: nil, offsetToData: 0, dataLength: total,
flags: kCMBlockBufferAssureMemoryNowFlag, blockBufferOut: &blockBuffer) == noErr,
let block = blockBuffer
else { return nil }
var dstLen = 0
var dstPtr: UnsafeMutablePointer<CChar>?
guard CMBlockBufferGetDataPointer(
block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &dstLen,
dataPointerOut: &dstPtr) == noErr,
dstLen == total, let dstPtr
else { return nil }
// Pass 2: the single copy length prefix + payload per NAL, straight into the block.
let dst = UnsafeMutableRawPointer(dstPtr)
var off = 0
forEachNAL(in: au.data) { base, range in
if codec.isParameterSet(base[range.lowerBound]) { return true }
var len = UInt32(range.count).bigEndian
withUnsafeBytes(of: &len) {
dst.advanced(by: off).copyMemory(from: $0.baseAddress!, byteCount: 4)
}
dst.advanced(by: off + 4)
.copyMemory(from: base + range.lowerBound, byteCount: range.count)
off += 4 + range.count
return true
}
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: CMTime(value: Int64(au.ptsNs), timescale: 1_000_000_000),
decodeTimeStamp: .invalid)
var sampleSize = total
var sample: CMSampleBuffer?
guard CMSampleBufferCreate(
allocator: kCFAllocatorDefault, dataBuffer: block, dataReady: true,
makeDataReadyCallback: nil, refcon: nil, formatDescription: format,
sampleCount: 1, sampleTimingEntryCount: 1, sampleTimingArray: &timing,
sampleSizeEntryCount: 1, sampleSizeArray: &sampleSize,
sampleBufferOut: &sample) == noErr
else { return nil }
// Low-latency display: render on arrival, don't wait for a clock.
if let attachments = CMSampleBufferGetSampleAttachmentsArray(sample!, createIfNecessary: true) {
let dict = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self)
CFDictionarySetValue(
dict,
Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(),
Unmanaged.passUnretained(kCFBooleanTrue).toOpaque())
}
return sample
}
}
@@ -0,0 +1,29 @@
// Throttled host keyframe requests for decode recovery, shared by both pumps (StreamPump /
// Stage2Pipeline). Wedge signals arrive from several threads the decoder's async error callback
// (a VT thread), a submit failure on the pump thread, the framesDropped poll and the decode stays
// stalled for several frames until the requested IDR lands, so requests are coalesced (100 ms, the
// throttle the working Android path uses: fast enough that a lost recovery IDR is re-requested
// promptly, bounded so a sustained freeze can't flood the control stream). Bound to the live
// connection at pump start, unbound on stop.
import Foundation
final class KeyframeRecovery: @unchecked Sendable {
private let lock = NSLock()
private var connection: PunktfunkConnection?
private var lastNs: UInt64 = 0
func bind(_ c: PunktfunkConnection?) {
lock.lock(); connection = c; lastNs = 0; lock.unlock()
}
func request() {
lock.lock()
let now = DispatchTime.now().uptimeNanoseconds
let due = lastNs == 0 || now &- lastNs > 100_000_000 // 100 ms since the last request
if due { lastNs = now }
let conn = due ? connection : nil
lock.unlock()
conn?.requestKeyframe()
}
}
@@ -44,10 +44,11 @@ vertex VOut pf_vtx(uint vid [[vertex_id]]) {
return o;
}
// Bicubic (Catmull-Rom) sampling of the single-channel luma plane. When the drawable is larger
// than the decoded frame (a window/view bigger than the host's fixed mode), a bilinear upscale
// looks soft; Catmull-Rom keeps edges crisp matching AVSampleBufferDisplayLayer's (stage-1)
// scaler and reduces to the exact texel at 1:1, so a native-resolution present stays pixel-exact.
// Bicubic (Catmull-Rom) sampling of the single-channel luma plane. The drawable is sized to the
// LAYER's pixels (see `render`), so this kernel performs the decodedon-screen scale: when the
// window/view is bigger than the host's fixed mode a bilinear upscale looks soft; Catmull-Rom
// keeps edges crisp matching AVSampleBufferDisplayLayer's (stage-1) scaler and reduces to the
// exact texel at 1:1, so a native-resolution present stays pixel-exact.
// Nine bilinear taps (TheRealMJP's optimisation of the 16-tap kernel); `s` MUST be a linear
// sampler. Luma carries the perceived detail, so only it gets bicubic; chroma stays bilinear.
float catmullRomLuma(texture2d<float> tex, sampler s, float2 uv) {
@@ -77,14 +78,27 @@ float catmullRomLuma(texture2d<float> tex, sampler s, float2 uv) {
return r;
}
// 4:2:0 chroma is left-cosited horizontally (H.273 chroma_loc type 0 the MPEG convention the
// host encodes and VideoToolbox decodes as-is), but sampling the half-res plane at the luma UV
// assumes CENTER siting a ~0.5-luma-px rightward chroma shift on hard colored edges. Offset the
// sample by +0.25 chroma texels to re-align (libplacebo/mpv's correction). Vertical siting for
// type 0 is centered, which plain sampling already matches. A full-size 4:4:4 plane has no
// subsampling to correct the offset self-disables when the plane widths match.
float2 chromaUV(texture2d<float> lumaTex, texture2d<float> chromaTex, float2 uv) {
if (chromaTex.get_width() < lumaTex.get_width()) {
uv.x += 0.25 / float(chromaTex.get_width());
}
return uv;
}
// SDR: 8-bit NV12 / 4:4:4 (BT.709, limited/video range) full-range RGB. Chroma is sampled at the
// same UV as luma, so a full-size 4:4:4 chroma plane needs no shader change vs 4:2:0.
// (siting-corrected) luma UV, so a full-size 4:4:4 chroma plane needs no shader change vs 4:2:0.
fragment float4 pf_frag(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg;
float2 c = chromaTex.sample(s, chromaUV(lumaTex, chromaTex, in.uv)).rg;
// BT.709, 8-bit limited (video) range full-range RGB.
y = (y - 16.0/255.0) * (255.0/219.0);
float u = (c.x - 128.0/255.0) * (255.0/224.0);
@@ -105,7 +119,7 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg;
float2 c = chromaTex.sample(s, chromaUV(lumaTex, chromaTex, in.uv)).rg;
// BT.2020 10-bit limited (video) range full-range PQ RGB.
y = (y - 64.0/1023.0) * (1023.0/876.0);
float u = (c.x - 512.0/1023.0) * (1023.0/896.0);
@@ -185,10 +199,11 @@ public final class MetalVideoPresenter {
// (the display link is the pacing source) the fix for the fullscreen stutter. macOS-only.
layer.displaySyncEnabled = false
#endif
// Render the drawable at the DECODED frame's resolution (set per-frame in `render`) and let the
// system compositor scale it to the layer's bounds the same `.resizeAspect` path stage-1's
// AVSampleBufferDisplayLayer uses. A native-resolution present is then pixel-exact (1:1, no
// shader scaling); a resized window rescales via the system's scaler.
// The drawable is rendered at the LAYER's pixel size (set per-frame in `render`), so the
// shader not the compositor performs the decodedon-screen scale (bicubic luma; the
// compositor's contentsGravity path is plain bilinear). The gravity stays aspect-fit as a
// transient fallback: during a live resize the compositor may composite a drawable from
// the previous layout before the next render catches up.
layer.contentsGravity = .resizeAspect
// Triple-buffer: more in-flight drawables before `nextDrawable()` (called on the display-link /
// MAIN thread) has to block waiting for one to free.
@@ -277,9 +292,15 @@ public final class MetalVideoPresenter {
/// Draw one decoded frame to the next drawable and present it. MAIN THREAD (the display link).
/// `isHDR` selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// layer config via `configure`. Returns true on success; false when there's no drawable yet, a
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present.
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present (and can
/// requeue the frame). `onPresented` fires once the drawable actually reached glass, with the
/// `CLOCK_REALTIME` instant from the drawable's `presentedTime` or nil when the system reports
/// none (a dropped drawable). It runs on a Metal callback thread; keep the handler thread-safe.
@discardableResult
public func render(_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false) -> Bool {
public func render(
_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false,
onPresented: ((Int64?) -> Void)? = nil
) -> Bool {
// Reconcile the layer with the decoded frame's HDR-ness (handles a mid-session SDRHDR flip).
configure(hdr: isHDR)
@@ -298,15 +319,25 @@ public final class MetalVideoPresenter {
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false }
// Size the drawable to the decoded frame so the fullscreen triangle samples 1:1 (pixel-exact);
// the layer's contentsGravity then scales it to the on-screen bounds via the system compositor
// (matching stage-1). drawableSize does NOT track bounds (defaults to 0), so set it BEFORE
// nextDrawable; re-set only on a change (first frame / Reconfigure / HDR flip).
// Size the drawable to the LAYER's pixels (bounds × contentsScale, both set by the hosting
// view's layout) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
// a native-mode session stays exactly 1:1 (the kernel reduces to the identity texel), and a
// window bigger than the host's mode gets bicubic luma instead of the compositor's bilinear.
// Before the first layout (empty bounds) fall back to the decoded size. drawableSize does NOT
// track bounds (defaults to 0), so set it BEFORE nextDrawable; re-set only on a change
// (layout / Reconfigure / HDR flip and every frame of a live resize, which is fine).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
if layer.drawableSize != decodedSize { layer.drawableSize = decodedSize }
let scale = layer.contentsScale
let boundsSize = layer.bounds.size
let targetSize = (boundsSize.width > 0 && boundsSize.height > 0)
? CGSize(
width: (boundsSize.width * scale).rounded(),
height: (boundsSize.height * scale).rounded())
: decodedSize
if layer.drawableSize != targetSize { layer.drawableSize = targetSize }
#if DEBUG
logSizeIfChanged(decoded: decodedSize)
logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
#endif
guard let drawable = layer.nextDrawable(),
let commandBuffer = queue.makeCommandBuffer()
@@ -325,6 +356,24 @@ public final class MetalVideoPresenter {
encoder.setFragmentTexture(CVMetalTextureGetTexture(chroma), index: 1)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
if let onPresented {
#if targetEnvironment(simulator)
// The simulator SDK exposes neither addPresentedHandler nor presentedTime report
// nil so the caller stamps with its display-link estimate (the pre-presentedTime
// behavior; simulator numbers are indicative only anyway).
onPresented(nil)
#else
// Registered BEFORE present. presentedTime is CACurrentMediaTime-based; 0 means the
// system never put this drawable on glass (dropped) report nil, the caller falls
// back to its display-link estimate.
drawable.addPresentedHandler { d in
onPresented(
d.presentedTime > 0
? Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: d.presentedTime)
: nil)
}
#endif
}
commandBuffer.present(drawable) // present at the next vsync lowest latency
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// sampling releasing them at scope exit could free the backing mid-read.
@@ -350,11 +399,12 @@ public final class MetalVideoPresenter {
}
#if DEBUG
private func logSizeIfChanged(decoded: CGSize) {
let sig = "\(Int(decoded.width))x\(Int(decoded.height))|hdr\(hdrActive ? 1 : 0)"
private func logSizeIfChanged(decoded: CGSize, drawable: CGSize) {
let sig = "\(Int(decoded.width))x\(Int(decoded.height))\(Int(drawable.width))x\(Int(drawable.height))|hdr\(hdrActive ? 1 : 0)"
if sig != lastSizeSig {
lastSizeSig = sig
let msg = "stage2: decoded \(Int(decoded.width))x\(Int(decoded.height)) hdr=\(hdrActive)"
let msg =
"stage2: decoded \(Int(decoded.width))x\(Int(decoded.height)) → drawable \(Int(drawable.width))x\(Int(drawable.height)) hdr=\(hdrActive)"
presenterLog.info("\(msg, privacy: .public)")
}
}
@@ -0,0 +1,153 @@
// Per-session presenter stack shared by the macOS and iOS/tvOS stream views: stage-2 (explicit
// VTDecompressionSession decode CAMetalLayer, driven by the hosting view's CADisplayLink) is the
// default; stage-1 (StreamPump AVSampleBufferDisplayLayer) is the Metal-unavailable / DEBUG
// fallback. The views own the platform bits capture, window/scale tracking, and constructing the
// display link and delegate the shared presenter lifecycle here.
//
// Main-thread only: start/layout/stop and the display-link tick all run on the main runloop.
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import QuartzCore
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view or
/// presenter directly makes a `owner link owner` cycle that only `invalidate()` breaks if a
/// teardown is ever missed the owner leaks and keeps ticking. The proxy is what the link retains;
/// the handler closure captures the owner `[weak]`, so the owner can deallocate and its `deinit`
/// invalidate the link.
public final class DisplayLinkProxy: NSObject {
private let onTick: (CADisplayLink) -> Void
public init(_ onTick: @escaping (CADisplayLink) -> Void) { self.onTick = onTick }
@objc public func tick(_ link: CADisplayLink) { onTick(link) }
}
final class SessionPresenter {
private var pump: StreamPump?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer?
private var connection: PunktfunkConnection?
/// Start the presenter for `connection`. `baseLayer` is the view's AVSampleBufferDisplayLayer:
/// stage-1 enqueues into it; stage-2 leaves it idle and composites an opaque CAMetalLayer
/// sublayer over it. `makeDisplayLink` supplies the platform link (macOS `NSView.displayLink`
/// tracks the view's display; iOS/tvOS uses the plain `CADisplayLink` init) only called when
/// stage-2 engages. Call `layout(in:contentsScale:)` right after so the sublayer has a frame
/// before the first tick.
func start(
connection: PunktfunkConnection,
baseLayer: AVSampleBufferDisplayLayer,
presentMeter: LatencyMeter?,
presentTailMeter: LatencyMeter? = nil,
makeDisplayLink: (AnyObject, Selector) -> CADisplayLink,
onFrame: (@Sendable (AccessUnit) -> Void)?,
onSessionEnd: (@Sendable () -> Void)?
) {
stop()
self.connection = connection
// Presenter choice stage-2 is the DEFAULT (explicit VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Stage-1 is
// reachable only via the DEBUG presenter toggle; release always takes stage-2 (the stage-1
// pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let pipeline = Stage2Pipeline(
presentMeter: presentMeter, presentTailMeter: presentTailMeter) {
let metal = pipeline.layer
// The opaque metal layer composites OVER the AVSampleBufferDisplayLayer base, which
// sits idle (un-enqueued) in stage-2. contentsScale + frame are set in layout().
baseLayer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
let proxy = DisplayLinkProxy { [weak self] link in
self?.stage2?.renderTick(
targetPresentNs: Stage2Pipeline.realtimeNs(
forDisplayLinkTimestamp: link.targetTimestamp))
}
let link = makeDisplayLink(proxy, #selector(DisplayLinkProxy.tick(_:)))
link.add(to: .main, forMode: .common)
stage2Link = link
syncFrameRate(hz: connection.currentMode().refreshHz)
pipeline.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
} else {
let pump = StreamPump()
pump.start(
connection: connection, layer: baseLayer,
onFrame: onFrame, onSessionEnd: onSessionEnd)
self.pump = pump
}
}
/// Ask the display link for the stream's own cadence. iOS/tvOS-only: without an explicit
/// range, ProMotion devices cap CADisplayLink at 60 Hz (iPhones additionally need
/// `CADisableMinimumFrameDurationOnPhone` in Info.plist), so a 120 fps stream would present
/// at half rate with the ring silently dropping every other frame. `maximum` allows up to
/// 120 so the system MAY tick faster than a sub-120 stream (each extra tick is a near-free
/// empty `renderTick`, and presenting on a denser grid shortens the decodeglass wait); the
/// macOS NSView link already tracks its display and must NOT be capped to the stream rate.
/// Re-applied from `layout` so a mid-session `Reconfigure` picks up a new refresh.
private func syncFrameRate(hz: UInt32) {
#if !os(macOS)
guard hz > 0, let link = stage2Link else { return }
let hzF = Float(hz)
if link.preferredFrameRateRange.preferred != hzF {
link.preferredFrameRateRange = CAFrameRateRange(
minimum: min(30, hzF), maximum: max(hzF, 120), preferred: hzF)
}
#endif
}
/// Position the stage-2 metal sublayer aspect-fit in the hosting view (the host streams at the
/// client's native mode, so this is usually the full bounds; it letterboxes a resized window).
/// The layer FRAME + contentsScale set here are what the presenter sizes its drawable from
/// (frame × scale) the shader then performs the decodedon-screen scale (bicubic luma), so a
/// native-mode session stays pixel-exact 1:1 and a mismatched window beats the compositor's
/// bilinear. No-op for stage-1 or before start.
func layout(in bounds: CGRect, contentsScale: CGFloat) {
guard let metalLayer, let connection else { return }
let mode = connection.currentMode()
syncFrameRate(hz: mode.refreshHz) // track a mid-session Reconfigure's new refresh
let fit: CGRect = (mode.width > 0 && mode.height > 0)
? AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
: bounds
// No implicit resize animation; contentsScale tracks the view's backing/display scale.
CATransaction.begin()
CATransaction.setDisableActions(true)
metalLayer.contentsScale = contentsScale
metalLayer.frame = fit
CATransaction.commit()
}
/// Stop the active pump/pipeline ( one poll timeout; stage-2 joins its pump) and detach the
/// stage-2 layer + link. Does not close the connection that stays with whoever owns it.
/// Idempotent.
func stop() {
pump?.stop()
pump = nil
stage2Link?.invalidate()
stage2Link = nil
stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
connection = nil
}
deinit {
// The owning view's stop() normally ran already; this covers a missed teardown so the
// display link can't keep ticking a deallocated pipeline.
stage2Link?.invalidate()
stage2?.stop()
pump?.stop()
}
}
#endif
@@ -12,16 +12,6 @@ import AVFoundation
import Foundation
import QuartzCore
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view directly
/// makes a `view link view` cycle that only `invalidate()` breaks if a teardown is ever missed
/// the view leaks and keeps ticking. This proxy holds the handler weakly, so the view can deallocate
/// and its `deinit` invalidate the link.
public final class DisplayLinkProxy: NSObject {
private let onTick: (CADisplayLink) -> Void
public init(_ onTick: @escaping (CADisplayLink) -> Void) { self.onTick = onTick }
@objc public func tick(_ link: CADisplayLink) { onTick(link) }
}
/// Newest-ready 1-slot ring: the decoder overwrites (drops the older undisplayed frame lowest
/// latency, no smoothing buffer), the display link takes-and-clears. Sendable; lock-guarded.
private final class ReadyRing: @unchecked Sendable {
@@ -34,37 +24,15 @@ private final class ReadyRing: @unchecked Sendable {
lock.lock(); defer { lock.unlock() }
let f = frame; frame = nil; return f
}
}
/// Cancellation handle owned by one pump thread (same pattern as StreamPump).
private final class PumpToken: @unchecked Sendable {
private let lock = NSLock()
private var live = true
var isLive: Bool { lock.lock(); defer { lock.unlock() }; return live }
func cancel() { lock.lock(); live = false; lock.unlock() }
}
/// Throttled host keyframe requests for decode recovery. The decoder's async error callback (a VT
/// thread) and the pump thread (a submit failure) both signal a wedge; this coalesces them so the
/// control stream isn't flooded while the decode stays stalled for several frames until the requested
/// IDR lands. Bound to the live connection in `start`, unbound in `stop`.
private final class KeyframeRecovery: @unchecked Sendable {
private let lock = NSLock()
private var connection: PunktfunkConnection?
private var lastNs: UInt64 = 0
func bind(_ c: PunktfunkConnection?) {
lock.lock(); connection = c; lastNs = 0; lock.unlock()
}
func request() {
/// Return a frame the display link took but could not present (a transient `nextDrawable`
/// failure). Kept only while the slot is still empty a newer decoded frame wins, so
/// newest-ready ordering is preserved. Without this, a failed render silently LOSES the
/// frame, and under the host's infinite GOP a static scene sends no replacement until the
/// next damage the stale picture would persist.
func putBack(_ f: ReadyFrame) {
lock.lock()
let now = DispatchTime.now().uptimeNanoseconds
let due = lastNs == 0 || now &- lastNs > 100_000_000 // 100 ms since the last request
if due { lastNs = now }
let conn = due ? connection : nil
if frame == nil { frame = f }
lock.unlock()
conn?.requestKeyframe()
}
}
@@ -72,12 +40,13 @@ public final class Stage2Pipeline {
private let ring = ReadyRing()
private let presenter: MetalVideoPresenter
private let decoder: VideoDecoder
private let presentMeter: LatencyMeter
private let presentMeter: LatencyMeter?
private let presentTailMeter: LatencyMeter?
private let recovery = KeyframeRecovery()
private var token = PumpToken()
private var token = StopFlag()
private var offsetNs: Int64 = 0
/// Signalled when the pump thread exits, so `stop()` can join it (bounded) before `decoder.reset()`
/// otherwise a pump iteration already past its `token.isLive` check can rebuild a decode session
/// otherwise a pump iteration already past its `token.isStopped` check can rebuild a decode session
/// right after the reset (a brief orphan session). `pumpJoinable` is armed by `start`, consumed by
/// the first `stop` (so the idempotent second `stop`/deinit doesn't block on an already-drained
/// semaphore). start/stop are sequential lifecycle calls, so the plain flag is safe.
@@ -87,12 +56,15 @@ public final class Stage2Pipeline {
/// The Metal layer the hosting view installs + sizes.
public var layer: CAMetalLayer { presenter.layer }
/// `presentMeter` records capturepresent (the glass-to-glass term). Returns nil if Metal can't be
/// set up (headless / no GPU) caller falls back to the stage-1 presenter.
public init?(presentMeter: LatencyMeter) {
/// `presentMeter` records capturepresent (the glass-to-glass term); `presentTailMeter`
/// records decode-completionpresent (the ring wait + render the tail stage-2 exists to
/// shorten). Both optional: metering never gates the presenter choice. Returns nil if Metal
/// can't be set up (headless / no GPU) caller falls back to the stage-1 presenter.
public init?(presentMeter: LatencyMeter?, presentTailMeter: LatencyMeter? = nil) {
guard let presenter = MetalVideoPresenter.make() else { return nil }
self.presenter = presenter
self.presentMeter = presentMeter
self.presentTailMeter = presentTailMeter
let ring = ring
let recovery = recovery
self.decoder = VideoDecoder(
@@ -113,7 +85,7 @@ public final class Stage2Pipeline {
) {
offsetNs = connection.clockOffsetNs
recovery.bind(connection) // arm host-keyframe recovery for this session
token = PumpToken() // fresh token per start cancel is permanent (like StreamPump)
token = StopFlag() // fresh token per start a stop is permanent (like StreamPump)
// Configure the decoder's chroma + the layer's initial colorimetry before the first frame. The
// chroma subsampling drives only the decode pixel format (orthogonal to HDR/depth); the HDR
@@ -138,7 +110,7 @@ public final class Stage2Pipeline {
// decode 4:4:4 at the negotiated resolution (the HW probe clears the common case but not a
// resolution-ceiling miss). End cleanly instead of looping on a black screen.
var decodeFailRun = 0
while token.isLive {
while !token.isStopped {
do {
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback
@@ -164,7 +136,7 @@ public final class Stage2Pipeline {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
guard let f = format, token.isLive else { continue }
guard let f = format, !token.isStopped else { continue }
if decoder.decode(au: au, format: f) {
decodeFailRun = 0
} else {
@@ -176,12 +148,12 @@ public final class Stage2Pipeline {
// ~3 s of solid failure in a 4:4:4 session (and only there a 4:2:0 loss
// recovers within a GOP) 4:4:4 isn't decodable here; end the session.
if connection.isChroma444, decodeFailRun >= 180 {
if token.isLive { onSessionEnd?() }
if !token.isStopped { onSessionEnd?() }
break
}
}
} catch {
if token.isLive { onSessionEnd?() }
if !token.isStopped { onSessionEnd?() }
break // session closed
}
}
@@ -192,19 +164,32 @@ public final class Stage2Pipeline {
thread.start()
}
/// MAIN thread, once per vsync. Present the newest ready frame (if any) and stamp capturepresent at
/// MAIN thread, once per vsync. Present the newest ready frame (if any). The latency stamps
/// use the drawable's ACTUAL on-glass instant (`addPresentedHandler`/`presentedTime` the
/// handler fires on a Metal callback thread; the meters are thread-safe), falling back to
/// `targetPresentNs` the display link's target present instant, already converted to
/// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`).
/// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`) when the system reports
/// no presented time (a dropped drawable). A frame that could not be rendered (no drawable
/// yet) goes back into the ring so the next tick retries it.
public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return }
guard presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) else { return }
presentMeter.record(ptsNs: frame.ptsNs, atNs: targetPresentNs, offsetNs: offsetNs)
let offsetNs = offsetNs
let presentMeter = presentMeter
let presentTailMeter = presentTailMeter
let rendered = presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) { presentedNs in
let atNs = presentedNs ?? targetPresentNs
presentMeter?.record(ptsNs: frame.ptsNs, atNs: atNs, offsetNs: offsetNs)
// Present tail = decode-completion on-glass. Both instants are client
// CLOCK_REALTIME, so no skew offset applies.
presentTailMeter?.record(ptsNs: UInt64(frame.decodedNs), atNs: atNs, offsetNs: 0)
}
if !rendered { ring.putBack(frame) }
}
/// Stop the pump ( one poll timeout) and drop the decode session. MAIN THREAD; idempotent. Does not
/// close the connection. A restart needs a fresh Stage2Pipeline (cancel is permanent).
/// close the connection. A restart needs a fresh Stage2Pipeline (the stop is permanent).
public func stop() {
token.cancel()
token.stop()
// Join the pump (bounded: one nextAU poll + an in-flight decode) before resetting the decoder,
// so the pump can't rebuild a session right after the reset. Only the first stop joins; a
// repeat/deinit stop skips the already-drained semaphore.
@@ -216,7 +201,7 @@ public final class Stage2Pipeline {
recovery.bind(nil) // stop requesting keyframes once the session is torn down
}
deinit { token.cancel() }
deinit { token.stop() }
/// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME`
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the target
@@ -43,7 +43,7 @@ public enum Stage444Probe {
au auBytes: [UInt8], want: OSType, fullRangeSibling: OSType
) -> Bool {
let data = Data(auBytes)
guard let format = AnnexB.formatDescription(fromIDR: data) else { return false }
guard let format = AnnexB.formatDescription(fromIDR: data, codec: .hevc) else { return false }
// Require a hardware decoder a software false-positive would make us advertise 4:4:4 and
// then decode every real frame on the CPU, blowing the latency budget.
let spec: [CFString: Any] = [
@@ -62,7 +62,7 @@ public enum Stage444Probe {
defer { VTDecompressionSessionInvalidate(session) }
let au = AccessUnit(data: data, ptsNs: 0, frameIndex: 0, flags: 0)
guard let sample = AnnexB.sampleBuffer(au: au, format: format) else { return false }
guard let sample = AnnexB.sampleBuffer(au: au, format: format, codec: .hevc) else { return false }
var produced: OSType = 0
let done = DispatchSemaphore(value: 0)
@@ -10,26 +10,10 @@ import os
private let pumpLog = Logger(subsystem: "io.unom.punktfunk", category: "video")
/// Cancellation handle owned by exactly one pump thread a restart hands the old pump
/// its own token, so it can never be revived by a newer start().
private final class PumpToken: @unchecked Sendable {
private let lock = NSLock()
private var live = true
var isLive: Bool {
lock.lock()
defer { lock.unlock() }
return live
}
func cancel() {
lock.lock()
live = false
lock.unlock()
}
}
/// One pump per instance; create a fresh StreamPump per start (cancel is permanent).
/// One pump per instance; create a fresh StreamPump per start (the stop is permanent
/// a restart hands the old pump its own token, so it can never be revived by a newer start()).
final class StreamPump {
private let token = PumpToken()
private let token = StopFlag()
/// Pump thread: pull AUs, wrap, enqueue. Non-IDR AUs before the first format
/// description are dropped. `onFrame`/`onSessionEnd` fire on the pump thread.
@@ -40,6 +24,9 @@ final class StreamPump {
onSessionEnd: (@Sendable () -> Void)?
) {
let token = token
// Coalesced host keyframe requests (100 ms throttle see KeyframeRecovery).
let recovery = KeyframeRecovery()
recovery.bind(connection)
// The layer is non-Sendable but its enqueue/flush are documented thread-safe, and after
// this point only the pump thread drives it assert that so the @Sendable Thread closure
// may capture it.
@@ -48,7 +35,6 @@ final class StreamPump {
let thread = Thread {
var format: CMVideoFormatDescription?
var lastKeyframeRequest = Date.distantPast
var lastFramesDropped = connection.framesDropped()
// Recovery is a persistent WANT, not a one-shot edge: set it on detected loss (or a
// decoder reset), retry the throttled request EVERY iteration, and clear it only when a
@@ -61,17 +47,7 @@ final class StreamPump {
var awaitingIDR = false
var awaitingSince = Date.distantPast // when the current recovery began (for the resume log)
var wasFailed = false
// Coalesced host keyframe request. 100 ms throttle (matches the working Android path):
// fast enough that a lost recovery IDR is re-requested promptly, bounded so a sustained
// freeze can't flood the control stream.
func requestKeyframeThrottled() {
let now = Date()
if now.timeIntervalSince(lastKeyframeRequest) > 0.1 {
connection.requestKeyframe()
lastKeyframeRequest = now
}
}
while token.isLive {
while !token.isStopped {
do {
// Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
@@ -91,7 +67,7 @@ final class StreamPump {
lastFramesDropped = dropped
awaitingIDR = true
}
if awaitingIDR { requestKeyframeThrottled() }
if awaitingIDR { recovery.request() }
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
@@ -120,11 +96,11 @@ final class StreamPump {
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f, codec: connection.videoCodec),
token.isLive // don't enqueue a stale frame after a restart
!token.isStopped // don't enqueue a stale frame after a restart
else { continue }
layer.enqueue(sample)
} catch {
if token.isLive {
if !token.isStopped {
onSessionEnd?()
}
break // session closed
@@ -138,8 +114,8 @@ final class StreamPump {
/// Stop pumping ( one poll timeout). Does not close the connection.
func stop() {
token.cancel()
token.stop()
}
deinit { token.cancel() }
deinit { token.stop() }
}
@@ -148,9 +148,9 @@ public final class VideoDecoder: @unchecked Sendable {
/// True when `newFormat` carries a PQ (SMPTE ST 2084) or HLG transfer function i.e. the host
/// is sending HDR (BT.2020). VideoToolbox populates the transfer-function extension from the
/// HEVC VUI, so this picks the decode bit depth (10-bit P010/x444 vs 8-bit NV12/444v) from the
/// stream. The present-side HDR config (colorspace/EDR/shader) is latched once per session from
/// the Welcome (`connection.isHDR`), which the host does NOT flip mid-session so this predicate
/// and that config agree for the session (a `#if DEBUG` assert in the presenter guards it).
/// stream and can flip mid-session (a game entering HDR re-inits the host encoder). The
/// presenter follows the decoded frame's resulting `isHDR`, not the Welcome's latched flag
/// (`render` reconciles the layer per frame via the idempotent `configure(hdr:)`).
static func isHDRFormat(_ format: CMVideoFormatDescription) -> Bool {
guard
let tf = CMFormatDescriptionGetExtension(
@@ -208,6 +208,11 @@ public final class VideoDecoder: @unchecked Sendable {
outputCallback: &callback,
decompressionSessionOut: &newSession)
guard status == noErr, let newSession else { return false }
// Real-time hint: schedule this session for live-streaming latency rather than
// throughput/efficiency. Best-effort decoders that don't support the property
// return an error, which is fine to ignore.
VTSessionSetProperty(
newSession, key: kVTDecompressionPropertyKey_RealTime, value: kCFBooleanTrue)
session = newSession
format = newFormat
return true
@@ -86,20 +86,22 @@ public struct StreamView: NSViewRepresentable {
private let onFrame: (@Sendable (AccessUnit) -> Void)?
private let onSessionEnd: (@Sendable () -> Void)?
private let presentMeter: LatencyMeter?
private let presentTailMeter: LatencyMeter?
/// `onFrame`/`onSessionEnd` fire on the pump thread hop to the main actor for UI.
/// `captureEnabled: false` disables input capture entirely while UI (e.g. a trust
/// prompt) is layered over the stream; flipping it to true auto-engages capture
/// once. `onCaptureChange` (main thread) reports engage/release drive the HUD's
/// "click to capture" / " releases" hint with it. `presentMeter` records capturepresent
/// when the stage-2 presenter is active (`punktfunk.presenter == "stage2"`).
/// and `presentTailMeter` decodepresent when the stage-2 presenter is active.
public init(
connection: PunktfunkConnection,
captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
presentMeter: LatencyMeter? = nil
presentMeter: LatencyMeter? = nil,
presentTailMeter: LatencyMeter? = nil
) {
self.connection = connection
self.captureEnabled = captureEnabled
@@ -107,6 +109,7 @@ public struct StreamView: NSViewRepresentable {
self.onFrame = onFrame
self.onSessionEnd = onSessionEnd
self.presentMeter = presentMeter
self.presentTailMeter = presentTailMeter
}
public func makeNSView(context: Context) -> StreamLayerView {
@@ -114,6 +117,7 @@ public struct StreamView: NSViewRepresentable {
view.onCaptureChange = onCaptureChange
view.captureEnabled = captureEnabled
view.presentMeter = presentMeter
view.presentTailMeter = presentTailMeter
view.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
return view
}
@@ -122,6 +126,7 @@ public struct StreamView: NSViewRepresentable {
view.onCaptureChange = onCaptureChange
view.captureEnabled = captureEnabled
view.presentMeter = presentMeter
view.presentTailMeter = presentTailMeter
// SwiftUI reuses the NSView across state changes repoint the pump only when the
// connection identity actually changed.
if view.connection !== connection {
@@ -136,13 +141,13 @@ public struct StreamView: NSViewRepresentable {
public final class StreamLayerView: NSView {
private let displayLayer = AVSampleBufferDisplayLayer()
private var pump: StreamPump?
/// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a display link instead of the
/// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
/// Record capturepresent / decodepresent when the stage-2 presenter is active.
/// Consulted at start().
var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer?
var presentTailMeter: LatencyMeter?
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
/// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback.
private let presenter = SessionPresenter()
public private(set) var connection: PunktfunkConnection?
private let cursorCapture = CursorCapture()
private var inputCapture: InputCapture?
@@ -242,16 +247,16 @@ public final class StreamLayerView: NSView {
public override func layout() {
super.layout()
attemptPendingCapture() // bounds become real here on first presentation
layoutMetalLayer() // keep the stage-2 sublayer aspect-fit to the view
layoutPresenter() // keep the stage-2 sublayer aspect-fit to the view
}
public override func setFrameSize(_ newSize: NSSize) {
super.setFrameSize(newSize)
// `layout()` isn't guaranteed on a manual-frame (no-Auto-Layout) live resize, so the
// stage-2 metal sublayer's drawableSize could stay at the old size while the view grows
// the compositor then upscales a too-small drawable and the video turns blocky. Resize the
// drawable here too so it always tracks the window's pixel size (no stale upscale).
layoutMetalLayer()
// stage-2 metal sublayer's frame could stay at the old size while the view grows
// the compositor then upscales a too-small layer and the video turns blocky. Re-fit
// here too so it always tracks the window's size (no stale upscale).
layoutPresenter()
}
// MARK: - Capture state machine
@@ -362,8 +367,7 @@ public final class StreamLayerView: NSView {
// A click is explicit intent AND may arrive mid-activation (acceptsFirstMouse:
// NSApp.isActive / isKeyWindow are still false for the click coming in from
// another app) only the auto-engage paths require already-held key status.
// `connection != nil` (not `pump`) is the session-active gate the stage-2 presenter
// runs without a StreamPump, and capture must still engage there.
// `connection != nil` is the session-active gate (presenter internals are opaque here).
guard captureEnabled, !captured, connection != nil, window != nil,
fromClick || (NSApp.isActive && window?.isKeyWindow == true)
else { return }
@@ -483,8 +487,10 @@ public final class StreamLayerView: NSView {
let u = (p.x - fit.minX) / fit.width
let v = (p.y - videoMinYTop) / fit.height
guard u >= 0, u <= 1, v >= 0, v <= 1 else { return nil } // letterbox bars
let hx = Int32((u * CGFloat(mode.width)).rounded().clamped(0, CGFloat(mode.width - 1)))
let hy = Int32((v * CGFloat(mode.height)).rounded().clamped(0, CGFloat(mode.height - 1)))
let hx = Int32((u * CGFloat(mode.width)).rounded()
.clamped(to: 0...CGFloat(mode.width - 1)))
let hy = Int32((v * CGFloat(mode.height)).rounded()
.clamped(to: 0...CGFloat(mode.height - 1)))
return HostPoint(x: hx, y: hy, w: mode.width, h: mode.height)
}
@@ -507,10 +513,10 @@ public final class StreamLayerView: NSView {
DispatchQueue.main.async { onCaptureChange(captured) }
}
// MARK: - Pump
// MARK: - Session start/stop
/// Pump thread: pull AUs from the connection, wrap, enqueue. The first IDR yields the
/// format description; non-IDR AUs before it are dropped (the host opens with an IDR).
/// Wire up input capture and start the presenter (see SessionPresenter for the
/// stage-2/stage-1 choice). `onFrame` fires per AU at receipt; `onSessionEnd` on close.
public func start(
connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
@@ -558,90 +564,31 @@ public final class StreamLayerView: NSView {
cursorVisible = false
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent)
// Presenter choice stage-2 is the DEFAULT (explicit VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Stage-1 is
// reachable only via the DEBUG presenter toggle; release always takes stage-2 (the stage-1
// pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
} else {
let pump = StreamPump()
pump.start(
connection: connection, layer: displayLayer,
onFrame: onFrame, onSessionEnd: onSessionEnd)
self.pump = pump
}
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
// default, the stage-1 pump as the Metal-missing / DEBUG fallback. The link comes from
// NSView.displayLink so it tracks the display this view is on.
presenter.start(
connection: connection,
baseLayer: displayLayer,
presentMeter: presentMeter,
presentTailMeter: presentTailMeter,
makeDisplayLink: { displayLink(target: $0, selector: $1) },
onFrame: onFrame,
onSessionEnd: onSessionEnd)
layoutPresenter()
requestAutoCapture() // entering a session is the deliberate "capture me" moment
}
// MARK: - Stage-2 presenter (VTDecompressionSession CAMetalLayer + display link)
private func startStage2(
_ pipeline: Stage2Pipeline, connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?, onSessionEnd: (@Sendable () -> Void)?
) {
let metal = pipeline.layer
// The opaque metal layer composites OVER the AVSampleBufferDisplayLayer base, which sits
// idle (un-enqueued) in stage-2. contentsScale + frame are set in layoutMetalLayer().
displayLayer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
layoutMetalLayer()
// Weak-proxy target so the link doesn't form a retain cycle with the view (see
// DisplayLinkProxy) the link retains the proxy; the proxy holds the view weakly.
let proxy = DisplayLinkProxy { [weak self] link in self?.stage2Tick(link) }
let link = displayLink(target: proxy, selector: #selector(DisplayLinkProxy.tick(_:)))
link.add(to: .main, forMode: .common)
stage2Link = link
pipeline.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
}
private func stage2Tick(_ link: CADisplayLink) {
stage2?.renderTick(
targetPresentNs: Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: link.targetTimestamp))
}
/// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// mode, so this is usually the full bounds; it letterboxes a resized window). Only the layer
/// FRAME is set here the presenter sizes the drawable to the decoded frame and the layer's
/// contentsGravity (.resizeAspect) scales it to this frame via the system compositor, so a
/// resized window rescales through the system's filter (matching stage-1) instead of the shader.
private func layoutMetalLayer() {
guard let metalLayer, let connection else { return }
let mode = connection.currentMode()
let fit: NSRect = (mode.width > 0 && mode.height > 0)
? AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
: bounds
// No implicit resize animation; refresh contentsScale on a retinanon-retina move.
CATransaction.begin()
CATransaction.setDisableActions(true)
metalLayer.contentsScale = window?.backingScaleFactor ?? 1
metalLayer.frame = fit
CATransaction.commit()
/// Aspect-fit the stage-2 metal sublayer to the view; refresh contentsScale on a
/// retinanon-retina move (see SessionPresenter.layout).
private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
}
public override func viewDidChangeBackingProperties() {
super.viewDidChangeBackingProperties()
layoutMetalLayer() // backing scale changed (e.g. moved to a non-retina display)
}
private func teardownStage2() {
stage2Link?.invalidate()
stage2Link = nil
stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
layoutPresenter() // backing scale changed (e.g. moved to a non-retina display)
}
/// Stop pumping ( one poll timeout). Does not close the connection that stays with
@@ -651,9 +598,7 @@ public final class StreamLayerView: NSView {
removeMouseMonitor() // belt-and-suspenders: releaseCapture no-ops if not captured
inputCapture?.stop()
inputCapture = nil
pump?.stop()
pump = nil
teardownStage2()
presenter.stop()
connection = nil
}
@@ -661,16 +606,7 @@ public final class StreamLayerView: NSView {
removeMouseMonitor()
appObservers.forEach(NotificationCenter.default.removeObserver(_:))
windowObservers.forEach(NotificationCenter.default.removeObserver(_:))
pump?.stop()
teardownStage2() // invalidate the display link + stop the pipeline if stop() was missed
}
}
extension CGFloat {
/// Clamp into a [lo, hi] range keeps the absolute-cursor mapping inside the host's
/// pixel bounds even if a stray event reports a point a hair past the video rect.
fileprivate func clamped(_ lo: CGFloat, _ hi: CGFloat) -> CGFloat {
Swift.min(Swift.max(self, lo), hi)
presenter.stop() // invalidate the display link + stop the pipeline if stop() was missed
}
}
#endif
@@ -51,6 +51,7 @@ public struct StreamView: UIViewControllerRepresentable {
private let onFrame: (@Sendable (AccessUnit) -> Void)?
private let onSessionEnd: (@Sendable () -> Void)?
private let presentMeter: LatencyMeter?
private let presentTailMeter: LatencyMeter?
public init(
connection: PunktfunkConnection,
@@ -58,7 +59,8 @@ public struct StreamView: UIViewControllerRepresentable {
onCaptureChange: ((Bool) -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil,
presentMeter: LatencyMeter? = nil
presentMeter: LatencyMeter? = nil,
presentTailMeter: LatencyMeter? = nil
) {
self.connection = connection
self.captureEnabled = captureEnabled
@@ -66,6 +68,7 @@ public struct StreamView: UIViewControllerRepresentable {
self.onFrame = onFrame
self.onSessionEnd = onSessionEnd
self.presentMeter = presentMeter
self.presentTailMeter = presentTailMeter
}
public func makeUIViewController(context: Context) -> StreamViewController {
@@ -73,6 +76,7 @@ public struct StreamView: UIViewControllerRepresentable {
controller.onCaptureChange = onCaptureChange
controller.captureEnabled = captureEnabled
controller.presentMeter = presentMeter
controller.presentTailMeter = presentTailMeter
controller.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
return controller
}
@@ -81,6 +85,7 @@ public struct StreamView: UIViewControllerRepresentable {
controller.onCaptureChange = onCaptureChange
controller.captureEnabled = captureEnabled
controller.presentMeter = presentMeter
controller.presentTailMeter = presentTailMeter
if controller.connection !== connection {
controller.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
}
@@ -95,14 +100,14 @@ public struct StreamView: UIViewControllerRepresentable {
public final class StreamViewController: UIViewController {
public private(set) var connection: PunktfunkConnection?
private var pump: StreamPump?
private var observers: [NSObjectProtocol] = []
/// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a CADisplayLink instead of the
/// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
/// Record capturepresent / decodepresent when the stage-2 presenter is active.
/// Consulted at start().
var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer?
var presentTailMeter: LatencyMeter?
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
/// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback.
private let presenter = SessionPresenter()
#if os(iOS)
private var inputCapture: InputCapture?
fileprivate var captured = false
@@ -274,27 +279,18 @@ public final class StreamViewController: UIViewController {
inputCapture = capture
#endif
// Presenter choice stage-2 is the DEFAULT (VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder, where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference frame with no
// way to recover. Stage-1 is reachable only via the DEBUG presenter toggle; release always
// takes stage-2 (the stage-1 pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
} else {
let pump = StreamPump()
pump.start(
connection: connection, layer: streamView.displayLayer,
onFrame: onFrame, onSessionEnd: onSessionEnd)
self.pump = pump
}
// Presenter choice + lifecycle live in SessionPresenter (shared with macOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
// default, the stage-1 pump as the Metal-missing / DEBUG fallback.
presenter.start(
connection: connection,
baseLayer: streamView.displayLayer,
presentMeter: presentMeter,
presentTailMeter: presentTailMeter,
makeDisplayLink: { CADisplayLink(target: $0, selector: $1) },
onFrame: onFrame,
onSessionEnd: onSessionEnd)
layoutMetalLayer()
#if os(iOS)
// GC only delivers while active; everything held is flushed by InputCapture's
@@ -349,39 +345,10 @@ public final class StreamViewController: UIViewController {
streamView.onScroll = nil
streamView.currentHostMode = nil
#endif
pump?.stop()
pump = nil
teardownStage2()
presenter.stop()
connection = nil
}
// MARK: - Stage-2 presenter (VTDecompressionSession CAMetalLayer + display link)
private func startStage2(
_ pipeline: Stage2Pipeline, connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?, onSessionEnd: (@Sendable () -> Void)?
) {
let metal = pipeline.layer
// Composites OVER the idle (un-enqueued in stage-2) AVSampleBufferDisplayLayer base.
// (contentsScale + frame are set by layoutMetalLayer() just below.)
streamView.layer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
layoutMetalLayer()
// Weak-proxy target so the link doesn't retain-cycle with the controller (see
// DisplayLinkProxy) the link retains the proxy; the proxy holds self weakly.
let proxy = DisplayLinkProxy { [weak self] link in self?.stage2Tick(link) }
let link = CADisplayLink(target: proxy, selector: #selector(DisplayLinkProxy.tick(_:)))
link.add(to: .main, forMode: .common)
stage2Link = link
pipeline.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
}
private func stage2Tick(_ link: CADisplayLink) {
stage2?.renderTick(
targetPresentNs: Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: link.targetTimestamp))
}
public override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
layoutMetalLayer()
@@ -397,40 +364,16 @@ public final class StreamViewController: UIViewController {
return s > 0 ? s : UIScreen.main.scale
}
/// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// mode, so this is usually the full bounds). Only the layer FRAME is set here the presenter
/// sizes the drawable to the decoded frame and the layer's contentsGravity (.resizeAspect)
/// scales it to this frame via the system compositor (matching stage-1's videoGravity).
/// Aspect-fit the stage-2 metal sublayer to the view at the canonical render scale
/// (see SessionPresenter.layout).
private func layoutMetalLayer() {
guard let metalLayer, let connection else { return }
let mode = connection.currentMode()
let bounds = streamView.bounds
let fit: CGRect = (mode.width > 0 && mode.height > 0)
? AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
: bounds
CATransaction.begin()
CATransaction.setDisableActions(true) // don't animate the resize
metalLayer.contentsScale = renderScale
metalLayer.frame = fit
CATransaction.commit()
}
private func teardownStage2() {
stage2Link?.invalidate()
stage2Link = nil
stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
presenter.layout(in: streamView.bounds, contentsScale: renderScale)
}
#if os(iOS)
private func setCaptured(_ on: Bool) {
if on {
// `connection != nil` (not `pump`) is the session-active gate the stage-2 presenter
// runs without a StreamPump.
// `connection != nil` is the session-active gate (presenter internals are opaque here).
guard captureEnabled, !captured, connection != nil else { return }
inputCapture?.setForwarding(true)
captured = true
@@ -476,8 +419,7 @@ public final class StreamViewController: UIViewController {
deinit {
observers.forEach(NotificationCenter.default.removeObserver(_:))
pump?.stop()
teardownStage2() // invalidate the display link + stop the pipeline if stop() was missed
presenter.stop() // invalidate the display link + stop the pipeline if stop() was missed
}
}
@@ -675,12 +617,4 @@ final class StreamLayerUIView: UIView {
}
#endif
}
#if os(iOS)
extension CGFloat {
fileprivate func clamped(to range: ClosedRange<CGFloat>) -> CGFloat {
Swift.min(Swift.max(self, range.lowerBound), range.upperBound)
}
}
#endif
#endif
@@ -131,40 +131,49 @@ final class LoopbackIntegrationTests: XCTestCase {
}
/// The PIN pairing ceremony + the --require-pairing gate through the Swift wrapper:
/// anonymous rejection, the single wrong-PIN online guess, the real ceremony, and a
/// paired + pinned session. Driven by test-loopback.sh, which arms a second host with
/// --require-pairing and parses its random PIN out of the log.
/// no session while unpaired, the single wrong-PIN online guess, the real ceremony, and a
/// paired + pinned session. Driven by test-loopback.sh, which arms TWO --require-pairing
/// hosts and parses their random PINs out of the logs: a pairing attempt right or wrong
/// consumes the host's one-shot arming window (SPAKE2's "one online guess"), so the wrong-PIN
/// assertion burns the GUESS host's window and the real ceremony runs against the PAIRING
/// host's untouched one.
func testPairingCeremonyAndRequirePairingGate() throws {
let env = ProcessInfo.processInfo.environment
guard let portStr = env["PUNKTFUNK_PAIRING_PORT"], let port = UInt16(portStr),
let pin = env["PUNKTFUNK_PAIRING_PIN"]
let pin = env["PUNKTFUNK_PAIRING_PIN"],
let guessPortStr = env["PUNKTFUNK_GUESS_PORT"], let guessPort = UInt16(guessPortStr),
let guessPin = env["PUNKTFUNK_GUESS_PIN"]
else {
throw XCTSkip("needs an armed punktfunk1-host — use clients/apple/test-loopback.sh")
throw XCTSkip("needs armed punktfunk1-hosts — use clients/apple/test-loopback.sh")
}
let identity = try generateIdentity()
// 1. Unpaired clients don't get sessions from a --require-pairing host.
// 1. Unpaired clients don't get sessions from a require-pairing host. The host PARKS the
// identified knock for delegated console approval (§8b-1) rather than rejecting it
// outright nobody approves here, so the connect times out client-side. Either way:
// no session while unpaired.
XCTAssertThrowsError(
try PunktfunkConnection(
host: "127.0.0.1", port: port, width: 1280, height: 720, refreshHz: 60,
identity: identity, timeoutMs: 5000),
"unpaired client must be rejected")
"unpaired client must not get a session")
// 2. A wrong PIN is exactly one failed online guess distinguishable from
// transport errors so the UI can say "try again".
// 2. A wrong PIN is exactly one failed online guess distinguishable from transport
// errors so the UI can say "try again". The attempt consumes the GUESS host's arming
// window (that is the point of the one-guess design), which is why it gets its own host.
XCTAssertThrowsError(
try pair(
host: "127.0.0.1", port: port, identity: identity,
pin: pin == "0000" ? "9999" : "0000", name: "wrong-pin", timeoutMs: 5000)
host: "127.0.0.1", port: guessPort, identity: identity,
pin: guessPin == "0000" ? "9999" : "0000", name: "wrong-pin", timeoutMs: 5000)
) { error in
guard case PunktfunkClientError.wrongPIN = error else {
return XCTFail("expected .wrongPIN, got \(error)")
}
}
// 3. The real ceremony (after the host's 2 s pairing cooldown).
Thread.sleep(forTimeInterval: 2.2)
// 3. The real ceremony the PAIRING host's first attempt, so neither its one-shot window
// nor the per-host pairing cooldown has been touched.
let fingerprint = try pair(
host: "127.0.0.1", port: port, identity: identity,
pin: pin, name: "loopback-test", timeoutMs: 5000)
+35 -18
View File
@@ -1,47 +1,64 @@
#!/usr/bin/env bash
# Loopback integration: real punktfunk/1 hosts (synthetic source — pure protocol, runs fine on
# macOS) on 127.0.0.1, then the Swift integration tests against them through the xcframework.
# Two hosts: an open one (stream round trip) and one armed with --require-pairing (the PIN
# ceremony + pairing gate — its random PIN is parsed out of its log).
# Three hosts: an OPEN one (--allow-tofu; the anonymous stream round trip — bare punktfunk1-host
# now defaults to require-pairing), one armed with --require-pairing (the PIN ceremony + pairing
# gate — its random PIN is parsed out of its log), and a GUESS host whose one-shot arming window
# the wrong-PIN test deliberately burns (a pairing attempt — right or wrong — consumes the armed
# PIN, the SPAKE2 "one online guess", so the real ceremony needs a window of its own).
set -euo pipefail
cd "$(dirname "$0")/../.."
PORT="${PUNKTFUNK_LOOPBACK_PORT:-19778}"
PAIR_PORT="${PUNKTFUNK_PAIRING_PORT:-19779}"
GUESS_PORT="${PUNKTFUNK_GUESS_PORT:-19780}"
cargo build --release -p punktfunk-host
# Each host gets a throwaway config home: the pairing host persists a trust store
# (punktfunk1-paired.json, resolved from $HOME) and both mint an identity cert on first
# Each host gets a throwaway config home: the pairing hosts persist a trust store
# (punktfunk1-paired.json, resolved from $HOME) and all mint an identity cert on first
# run — none of that belongs in the user's real ~/.config/punktfunk, and separate homes
# also keep the two first runs from racing on the same cert.pem.
# also keep the first runs from racing on the same cert.pem.
CFG="$(mktemp -d "${TMPDIR:-/tmp}/punktfunk-loopback.XXXXXX")"
PAIR_LOG="$CFG/pairing-host.log"
mkdir -p "$CFG/open" "$CFG/paired"
trap 'kill "${HOST_PID:-}" "${PAIR_PID:-}" 2>/dev/null || true' EXIT
GUESS_LOG="$CFG/guess-host.log"
mkdir -p "$CFG/open" "$CFG/paired" "$CFG/guess"
trap 'kill "${HOST_PID:-}" "${PAIR_PID:-}" "${GUESS_PID:-}" 2>/dev/null || true' EXIT
# The open host also scripts a feedback burst (rumble + DualSense hidout) right after the
# handshake, so the Swift test can assert the host→client feedback planes end to end.
HOME="$CFG/open" XDG_CONFIG_HOME="$CFG/open/.config" PUNKTFUNK_TEST_FEEDBACK=1 \
target/release/punktfunk-host punktfunk1-host --port "$PORT" --source synthetic --frames 300 &
target/release/punktfunk-host punktfunk1-host --port "$PORT" --source synthetic --frames 300 \
--allow-tofu &
HOST_PID=$!
HOME="$CFG/paired" XDG_CONFIG_HOME="$CFG/paired/.config" \
target/release/punktfunk-host punktfunk1-host --port "$PAIR_PORT" --source synthetic --frames 300 \
--require-pairing >"$PAIR_LOG" 2>&1 &
PAIR_PID=$!
HOME="$CFG/guess" XDG_CONFIG_HOME="$CFG/guess/.config" \
target/release/punktfunk-host punktfunk1-host --port "$GUESS_PORT" --source synthetic --frames 300 \
--require-pairing >"$GUESS_LOG" 2>&1 &
GUESS_PID=$!
sleep 1
PIN=""
for _ in $(seq 50); do
PIN="$(grep -oE 'pair: [0-9]+' "$PAIR_LOG" | head -1 | cut -d' ' -f2 || true)"
[ -n "$PIN" ] && break
sleep 0.2
done
if [ -z "$PIN" ]; then
echo "no arming PIN in the pairing host's log ($PAIR_LOG)" >&2
exit 1
fi
# Parse each pairing host's random arming PIN out of its startup log.
pin_from_log() {
local log="$1" pin=""
for _ in $(seq 50); do
pin="$(grep -oE 'pair: [0-9]+' "$log" | head -1 | cut -d' ' -f2 || true)"
[ -n "$pin" ] && break
sleep 0.2
done
if [ -z "$pin" ]; then
echo "no arming PIN in the pairing host's log ($log)" >&2
exit 1
fi
echo "$pin"
}
PIN="$(pin_from_log "$PAIR_LOG")"
GUESS_PIN="$(pin_from_log "$GUESS_LOG")"
cd clients/apple
PUNKTFUNK_LOOPBACK_PORT="$PORT" PUNKTFUNK_PAIRING_PORT="$PAIR_PORT" PUNKTFUNK_PAIRING_PIN="$PIN" \
PUNKTFUNK_GUESS_PORT="$GUESS_PORT" PUNKTFUNK_GUESS_PIN="$GUESS_PIN" \
PUNKTFUNK_TEST_FEEDBACK=1 \
swift test --filter LoopbackIntegrationTests
+7
View File
@@ -623,6 +623,13 @@ pub fn can_encode_444(codec: Codec) -> bool {
})
}
/// Non-Linux/Windows (the macOS dev/test build of the host — synthetic-source loopback only):
/// no GPU encode backend exists here, so 4:4:4 is never advertised.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn can_encode_444(_codec: Codec) -> bool {
false
}
// ---------------------------------------------------------------------------------------------
// Windows backend selection (the analogue of the Linux nvidia_present / linux_zero_copy_is_vaapi
// logic). NVIDIA → NVENC, AMD → AMF, Intel → QSV; `auto` (default) reads the DXGI adapter vendor.
+30 -11
View File
@@ -6,7 +6,7 @@ description: "Design rationale + open items for the explicit VTDecompressionSess
> **Status:** SHIPPED as the **default** presenter (stage-1 `AVSampleBufferDisplayLayer` is the
> Metal-unavailable / DEBUG fallback). HDR corrected and **4:4:4** added on top of the proven
> main-thread present path (the hosting view's `CADisplayLink` drives `render` per vsync). Code:
> `clients/apple/Sources/PunktfunkKit/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,Stage444Probe,LatencyMeter}.swift`.
> `clients/apple/Sources/PunktfunkKit/Video/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,SessionPresenter,Stage444Probe,LatencyMeter}.swift`.
> This doc is trimmed to design rationale + open items — the shipped `.swift` code is the source of
> truth for the decode/present/measurement walkthrough.
>
@@ -34,17 +34,29 @@ description: "Design rationale + open items for the explicit VTDecompressionSess
> metadata when it goes HDR. A ≤2-frame transition flash on the rare flip is accepted.
>
> **Pacing.** The hosting view owns a **main-runloop `CADisplayLink`** (a weak `DisplayLinkProxy`
> breaks the retain cycle) that calls `renderTick` once per vsync. `renderTick` pops the **newest**
> ready frame from the 1-slot ring (older undisplayed frames dropped — lowest latency, no smoothing
> buffer) and, if there is one, draws it via **manual `layer.nextDrawable()`** and presents at the next
> vsync; on an idle vsync (no new frame) it does nothing and the compositor holds the last presented
> breaks the retain cycle; the shared per-session lifecycle lives in `SessionPresenter`) that calls
> `renderTick` once per vsync. `renderTick` pops the **newest** ready frame from the 1-slot ring
> (older undisplayed frames dropped — lowest latency, no smoothing buffer) and, if there is one,
> draws it via **manual `layer.nextDrawable()`** and presents at the next vsync; a frame that could
> not be rendered (no drawable yet) is **put back** into the still-empty ring so the next tick
> retries it (under the infinite GOP a static scene sends no replacement — losing the frame would
> freeze stale content). On an idle vsync it does nothing and the compositor holds the last presented
> drawable (no idle re-render — matters at 5K). `drawableSize` is set **before** `nextDrawable` (it
> doesn't track bounds, defaults to 0), so allocation always uses the decoded size. `maximumDrawableCount
> = 3`. macOS `displaySyncEnabled = **false**`: the display link is the single pacing source, so leaving
> the layer's own vsync wait on would *also* block `present`/`nextDrawable` on the main thread and
> doesn't track bounds, defaults to 0) to the **layer's pixel size** (bounds × contentsScale), so the
> shader — not the compositor's bilinear — performs the decoded→on-screen scale (bicubic Catmull-Rom
> luma + siting-corrected bilinear chroma); a native-mode session is exactly 1:1 (the kernel reduces
> to the identity texel). `maximumDrawableCount = 3`. On iOS/tvOS `SessionPresenter` sets the link's
> `preferredFrameRateRange` to the negotiated refresh (+ `CADisableMinimumFrameDurationOnPhone` in
> Info.plist) — without it ProMotion devices cap the link at 60 Hz and a 120 fps stream presents at
> half rate; macOS's `NSView.displayLink` already tracks its display and is left alone. macOS
> `displaySyncEnabled = **false**`: the display link is the single pacing source, so leaving the
> layer's own vsync wait on would *also* block `present`/`nextDrawable` on the main thread and
> serialize it to the display — the cause of the fullscreen judder; disabling it lets present return
> promptly. Present is stamped at the display link's `targetTimestamp` projected to `CLOCK_REALTIME`
> (the actual on-glass instant, <1 vsync after the draw — accurate for the HUD).
> promptly. Present is stamped at the drawable's **actual `presentedTime`** (`addPresentedHandler`,
> converted to `CLOCK_REALTIME`), falling back to the display link's `targetTimestamp` projection
> when the system reports none (a dropped drawable) — so the HUD numbers reflect glass, and a missed
> vsync shows up instead of being assumed away. The same stamp feeds **decode→present**
> (`presentTailMeter` → the HUD's "decode→present" line), closing the third instant promised below.
>
> *(History: an off-main `CAMetalDisplayLink` variant and an off-main blocking-render present thread
> were both tried and **reverted** — both measured slower on macOS *and* iPad than this main-thread
@@ -105,7 +117,14 @@ Async `VTDecompressionSession` callback → **1-slot newest-ready ring** → dis
forcing a too-large 4:4:4 mode.
- **Glass-to-glass numbers via `tools/latency-probe`** — close the still-unmeasured host render→capture
term and confirm the main-thread display-link present p50 holds at ~11 ms (and isn't regressed by the
per-frame `configure` / HDR-anchor work).
per-frame `configure` / HDR-anchor work). The HUD's new decode→present line + the `presentedTime`-based
stamp make the client-side share directly visible now.
- **On-glass validation of the 2026-07 presenter batch** — the shader-side scale (drawable at layer
pixel size; bicubic luma + chroma-siting offset — compare a resized/fullscreen-on-larger-panel
window against stage-1 for sharpness, and check GPU headroom at 5K HDR), the iOS/tvOS
`preferredFrameRateRange` (a 120 fps stream on a ProMotion iPhone/iPad should now present at ~120 —
watch the HUD fps), `kVTDecompressionPropertyKey_RealTime`, and the zero-copy AnnexB → CMBlockBuffer
packing (unit/round-trip tested; confirm live).
- **Smoothing / pacing policy** — present newest-ready for lowest latency today; an optional even-pacing
policy (`present(_:afterMinimumDuration:)`) can come later if frames look uneven.
- **4:4:4 runtime downgrade-reconnect** — today a persistently-undecodable 4:4:4 session ends cleanly