feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS
Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit: Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split along the same seams. The gamepad mode is couch-complete, and now on macOS too (the living-room Mac case), not just iOS/iPadOS: - GamepadSettingsView: a console-style, fully controller-navigable settings screen (X from the launcher) — up/down moves focus, left/right steps values (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a one-line description. Backed by GamepadMenuList, the vertical sibling of GamepadCarousel, and SettingsOptions — the option lists hoisted out of SettingsView statics and shared by the touch, tvOS and gamepad settings. - GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad — field rows open an on-screen controller keyboard (dpad grid, A types, X backspaces, B done); the launcher carousel ends in an Add Host tile, so the dead-end "add one with touch first" empty state is gone. - Launcher polish: contextual hint bar with the pad's real button glyphs, controller name + battery chip, one shared console chrome. - GamepadScreenBackground: an animated aurora (TimelineView-driven drifting blobs in the brand's violet family, breathing radii, slow hue shift, legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a .metal library only bundles reliably in one of the two build systems (SPM vs the xcodeproj's synced folders) these sources compile under. - macOS port: settings/add-host/library present as sized sheets (a macOS sheet takes its content's IDEAL size, and the GeometryReader-driven screens collapsed to nothing), NSScreen-based mode lists, scroll indicators .never (the "always show scroll bars" setting overrides .hidden), tray scrims so scrolled rows dim under the pinned title/hints, extra title clearance, and a PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/ library render-verified live on a real Mac + LAN hosts. - GamepadMenuInput: X button support, and (re)start now snapshots held buttons so a controller handoff press never fires twice (the B that closed the keyboard no longer also cancels the screen underneath). - Cleanups: one "Connection failed" alert in ContentView instead of one per home screen; HostDiscovery.advertises/unsaved shared by both home screens. - host: can_encode_444 stub for the non-Linux/Windows host build (the macOS synthetic-source loopback used by the Swift tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -6,7 +6,7 @@ description: "Design rationale + open items for the explicit VTDecompressionSess
|
||||
> **Status:** SHIPPED as the **default** presenter (stage-1 `AVSampleBufferDisplayLayer` is the
|
||||
> Metal-unavailable / DEBUG fallback). HDR corrected and **4:4:4** added on top of the proven
|
||||
> main-thread present path (the hosting view's `CADisplayLink` drives `render` per vsync). Code:
|
||||
> `clients/apple/Sources/PunktfunkKit/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,Stage444Probe,LatencyMeter}.swift`.
|
||||
> `clients/apple/Sources/PunktfunkKit/Video/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,SessionPresenter,Stage444Probe,LatencyMeter}.swift`.
|
||||
> This doc is trimmed to design rationale + open items — the shipped `.swift` code is the source of
|
||||
> truth for the decode/present/measurement walkthrough.
|
||||
>
|
||||
@@ -34,17 +34,29 @@ description: "Design rationale + open items for the explicit VTDecompressionSess
|
||||
> metadata when it goes HDR. A ≤2-frame transition flash on the rare flip is accepted.
|
||||
>
|
||||
> **Pacing.** The hosting view owns a **main-runloop `CADisplayLink`** (a weak `DisplayLinkProxy`
|
||||
> breaks the retain cycle) that calls `renderTick` once per vsync. `renderTick` pops the **newest**
|
||||
> ready frame from the 1-slot ring (older undisplayed frames dropped — lowest latency, no smoothing
|
||||
> buffer) and, if there is one, draws it via **manual `layer.nextDrawable()`** and presents at the next
|
||||
> vsync; on an idle vsync (no new frame) it does nothing and the compositor holds the last presented
|
||||
> breaks the retain cycle; the shared per-session lifecycle lives in `SessionPresenter`) that calls
|
||||
> `renderTick` once per vsync. `renderTick` pops the **newest** ready frame from the 1-slot ring
|
||||
> (older undisplayed frames dropped — lowest latency, no smoothing buffer) and, if there is one,
|
||||
> draws it via **manual `layer.nextDrawable()`** and presents at the next vsync; a frame that could
|
||||
> not be rendered (no drawable yet) is **put back** into the still-empty ring so the next tick
|
||||
> retries it (under the infinite GOP a static scene sends no replacement — losing the frame would
|
||||
> freeze stale content). On an idle vsync it does nothing and the compositor holds the last presented
|
||||
> drawable (no idle re-render — matters at 5K). `drawableSize` is set **before** `nextDrawable` (it
|
||||
> doesn't track bounds, defaults to 0), so allocation always uses the decoded size. `maximumDrawableCount
|
||||
> = 3`. macOS `displaySyncEnabled = **false**`: the display link is the single pacing source, so leaving
|
||||
> the layer's own vsync wait on would *also* block `present`/`nextDrawable` on the main thread and
|
||||
> doesn't track bounds, defaults to 0) to the **layer's pixel size** (bounds × contentsScale), so the
|
||||
> shader — not the compositor's bilinear — performs the decoded→on-screen scale (bicubic Catmull-Rom
|
||||
> luma + siting-corrected bilinear chroma); a native-mode session is exactly 1:1 (the kernel reduces
|
||||
> to the identity texel). `maximumDrawableCount = 3`. On iOS/tvOS `SessionPresenter` sets the link's
|
||||
> `preferredFrameRateRange` to the negotiated refresh (+ `CADisableMinimumFrameDurationOnPhone` in
|
||||
> Info.plist) — without it ProMotion devices cap the link at 60 Hz and a 120 fps stream presents at
|
||||
> half rate; macOS's `NSView.displayLink` already tracks its display and is left alone. macOS
|
||||
> `displaySyncEnabled = **false**`: the display link is the single pacing source, so leaving the
|
||||
> layer's own vsync wait on would *also* block `present`/`nextDrawable` on the main thread and
|
||||
> serialize it to the display — the cause of the fullscreen judder; disabling it lets present return
|
||||
> promptly. Present is stamped at the display link's `targetTimestamp` projected to `CLOCK_REALTIME`
|
||||
> (the actual on-glass instant, <1 vsync after the draw — accurate for the HUD).
|
||||
> promptly. Present is stamped at the drawable's **actual `presentedTime`** (`addPresentedHandler`,
|
||||
> converted to `CLOCK_REALTIME`), falling back to the display link's `targetTimestamp` projection
|
||||
> when the system reports none (a dropped drawable) — so the HUD numbers reflect glass, and a missed
|
||||
> vsync shows up instead of being assumed away. The same stamp feeds **decode→present**
|
||||
> (`presentTailMeter` → the HUD's "decode→present" line), closing the third instant promised below.
|
||||
>
|
||||
> *(History: an off-main `CAMetalDisplayLink` variant and an off-main blocking-render present thread
|
||||
> were both tried and **reverted** — both measured slower on macOS *and* iPad than this main-thread
|
||||
@@ -105,7 +117,14 @@ Async `VTDecompressionSession` callback → **1-slot newest-ready ring** → dis
|
||||
forcing a too-large 4:4:4 mode.
|
||||
- **Glass-to-glass numbers via `tools/latency-probe`** — close the still-unmeasured host render→capture
|
||||
term and confirm the main-thread display-link present p50 holds at ~11 ms (and isn't regressed by the
|
||||
per-frame `configure` / HDR-anchor work).
|
||||
per-frame `configure` / HDR-anchor work). The HUD's new decode→present line + the `presentedTime`-based
|
||||
stamp make the client-side share directly visible now.
|
||||
- **On-glass validation of the 2026-07 presenter batch** — the shader-side scale (drawable at layer
|
||||
pixel size; bicubic luma + chroma-siting offset — compare a resized/fullscreen-on-larger-panel
|
||||
window against stage-1 for sharpness, and check GPU headroom at 5K HDR), the iOS/tvOS
|
||||
`preferredFrameRateRange` (a 120 fps stream on a ProMotion iPhone/iPad should now present at ~120 —
|
||||
watch the HUD fps), `kVTDecompressionPropertyKey_RealTime`, and the zero-copy AnnexB → CMBlockBuffer
|
||||
packing (unit/round-trip tested; confirm live).
|
||||
- **Smoothing / pacing policy** — present newest-ready for lowest latency today; an optional even-pacing
|
||||
policy (`present(_:afterMinimumDuration:)`) can come later if frames look uneven.
|
||||
- **4:4:4 runtime downgrade-reconnect** — today a persistently-undecodable 4:4:4 session ends cleanly
|
||||
|
||||
Reference in New Issue
Block a user