feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS

Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit:
Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split
along the same seams.

The gamepad mode is couch-complete, and now on macOS too (the living-room
Mac case), not just iOS/iPadOS:

- GamepadSettingsView: a console-style, fully controller-navigable settings
  screen (X from the launcher) — up/down moves focus, left/right steps values
  (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a
  one-line description. Backed by GamepadMenuList, the vertical sibling of
  GamepadCarousel, and SettingsOptions — the option lists hoisted out of
  SettingsView statics and shared by the touch, tvOS and gamepad settings.
- GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad
  — field rows open an on-screen controller keyboard (dpad grid, A types,
  X backspaces, B done); the launcher carousel ends in an Add Host tile, so
  the dead-end "add one with touch first" empty state is gone.
- Launcher polish: contextual hint bar with the pad's real button glyphs,
  controller name + battery chip, one shared console chrome.
- GamepadScreenBackground: an animated aurora (TimelineView-driven drifting
  blobs in the brand's violet family, breathing radii, slow hue shift,
  legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a
  .metal library only bundles reliably in one of the two build systems (SPM vs
  the xcodeproj's synced folders) these sources compile under.
- macOS port: settings/add-host/library present as sized sheets (a macOS sheet
  takes its content's IDEAL size, and the GeometryReader-driven screens
  collapsed to nothing), NSScreen-based mode lists, scroll indicators .never
  (the "always show scroll bars" setting overrides .hidden), tray scrims so
  scrolled rows dim under the pinned title/hints, extra title clearance, and a
  PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/
  library render-verified live on a real Mac + LAN hosts.
- GamepadMenuInput: X button support, and (re)start now snapshots held buttons
  so a controller handoff press never fires twice (the B that closed the
  keyboard no longer also cancels the screen underneath).
- Cleanups: one "Connection failed" alert in ContentView instead of one per
  home screen; HostDiscovery.advertises/unsaved shared by both home screens.
- host: can_encode_444 stub for the non-Linux/Windows host build (the macOS
  synthetic-source loopback used by the Swift tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-02 11:05:10 +02:00
parent e925d00194
commit 133e25849d
84 changed files with 4231 additions and 2698 deletions
+30 -11
View File
@@ -6,7 +6,7 @@ description: "Design rationale + open items for the explicit VTDecompressionSess
> **Status:** SHIPPED as the **default** presenter (stage-1 `AVSampleBufferDisplayLayer` is the
> Metal-unavailable / DEBUG fallback). HDR corrected and **4:4:4** added on top of the proven
> main-thread present path (the hosting view's `CADisplayLink` drives `render` per vsync). Code:
> `clients/apple/Sources/PunktfunkKit/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,Stage444Probe,LatencyMeter}.swift`.
> `clients/apple/Sources/PunktfunkKit/Video/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,SessionPresenter,Stage444Probe,LatencyMeter}.swift`.
> This doc is trimmed to design rationale + open items — the shipped `.swift` code is the source of
> truth for the decode/present/measurement walkthrough.
>
@@ -34,17 +34,29 @@ description: "Design rationale + open items for the explicit VTDecompressionSess
> metadata when it goes HDR. A ≤2-frame transition flash on the rare flip is accepted.
>
> **Pacing.** The hosting view owns a **main-runloop `CADisplayLink`** (a weak `DisplayLinkProxy`
> breaks the retain cycle) that calls `renderTick` once per vsync. `renderTick` pops the **newest**
> ready frame from the 1-slot ring (older undisplayed frames dropped — lowest latency, no smoothing
> buffer) and, if there is one, draws it via **manual `layer.nextDrawable()`** and presents at the next
> vsync; on an idle vsync (no new frame) it does nothing and the compositor holds the last presented
> breaks the retain cycle; the shared per-session lifecycle lives in `SessionPresenter`) that calls
> `renderTick` once per vsync. `renderTick` pops the **newest** ready frame from the 1-slot ring
> (older undisplayed frames dropped — lowest latency, no smoothing buffer) and, if there is one,
> draws it via **manual `layer.nextDrawable()`** and presents at the next vsync; a frame that could
> not be rendered (no drawable yet) is **put back** into the still-empty ring so the next tick
> retries it (under the infinite GOP a static scene sends no replacement — losing the frame would
> freeze stale content). On an idle vsync it does nothing and the compositor holds the last presented
> drawable (no idle re-render — matters at 5K). `drawableSize` is set **before** `nextDrawable` (it
> doesn't track bounds, defaults to 0), so allocation always uses the decoded size. `maximumDrawableCount
> = 3`. macOS `displaySyncEnabled = **false**`: the display link is the single pacing source, so leaving
> the layer's own vsync wait on would *also* block `present`/`nextDrawable` on the main thread and
> doesn't track bounds, defaults to 0) to the **layer's pixel size** (bounds × contentsScale), so the
> shader — not the compositor's bilinear — performs the decoded→on-screen scale (bicubic Catmull-Rom
> luma + siting-corrected bilinear chroma); a native-mode session is exactly 1:1 (the kernel reduces
> to the identity texel). `maximumDrawableCount = 3`. On iOS/tvOS `SessionPresenter` sets the link's
> `preferredFrameRateRange` to the negotiated refresh (+ `CADisableMinimumFrameDurationOnPhone` in
> Info.plist) — without it ProMotion devices cap the link at 60 Hz and a 120 fps stream presents at
> half rate; macOS's `NSView.displayLink` already tracks its display and is left alone. macOS
> `displaySyncEnabled = **false**`: the display link is the single pacing source, so leaving the
> layer's own vsync wait on would *also* block `present`/`nextDrawable` on the main thread and
> serialize it to the display — the cause of the fullscreen judder; disabling it lets present return
> promptly. Present is stamped at the display link's `targetTimestamp` projected to `CLOCK_REALTIME`
> (the actual on-glass instant, <1 vsync after the draw — accurate for the HUD).
> promptly. Present is stamped at the drawable's **actual `presentedTime`** (`addPresentedHandler`,
> converted to `CLOCK_REALTIME`), falling back to the display link's `targetTimestamp` projection
> when the system reports none (a dropped drawable) — so the HUD numbers reflect glass, and a missed
> vsync shows up instead of being assumed away. The same stamp feeds **decode→present**
> (`presentTailMeter` → the HUD's "decode→present" line), closing the third instant promised below.
>
> *(History: an off-main `CAMetalDisplayLink` variant and an off-main blocking-render present thread
> were both tried and **reverted** — both measured slower on macOS *and* iPad than this main-thread
@@ -105,7 +117,14 @@ Async `VTDecompressionSession` callback → **1-slot newest-ready ring** → dis
forcing a too-large 4:4:4 mode.
- **Glass-to-glass numbers via `tools/latency-probe`** — close the still-unmeasured host render→capture
term and confirm the main-thread display-link present p50 holds at ~11 ms (and isn't regressed by the
per-frame `configure` / HDR-anchor work).
per-frame `configure` / HDR-anchor work). The HUD's new decode→present line + the `presentedTime`-based
stamp make the client-side share directly visible now.
- **On-glass validation of the 2026-07 presenter batch** — the shader-side scale (drawable at layer
pixel size; bicubic luma + chroma-siting offset — compare a resized/fullscreen-on-larger-panel
window against stage-1 for sharpness, and check GPU headroom at 5K HDR), the iOS/tvOS
`preferredFrameRateRange` (a 120 fps stream on a ProMotion iPhone/iPad should now present at ~120 —
watch the HUD fps), `kVTDecompressionPropertyKey_RealTime`, and the zero-copy AnnexB → CMBlockBuffer
packing (unit/round-trip tested; confirm live).
- **Smoothing / pacing policy** — present newest-ready for lowest latency today; an optional even-pacing
policy (`present(_:afterMinimumDuration:)`) can come later if frames look uneven.
- **4:4:4 runtime downgrade-reconnect** — today a persistently-undecodable 4:4:4 session ends cleanly