The console UI answers the iPad field test — and the screen that would never open again was a UIKit read mid-render #154

Merged

enricobuehler merged 15 commits from worktree-apple-gamepad-relanded into main 2026-08-10 10:51:04 +00:00

Conversation 0 Commits 15 Files Changed 36

+2334 -295

15 Commits

Author	SHA1	Message	Date
enricobuehler	5ca0dfdcd2	fix(apple): stream audio coexists with Music instead of fighting it ... Field report, with a competitor cited doing it right: connecting to a host PAUSED the user's Apple Music, and resuming Music mid-stream silenced the stream's audio for good. Both are one defect — the audio session was EXCLUSIVE. Activating it evicted whoever was playing (the pause at connect), and Music's resume evicted us right back; iOS stops the engines on that interruption and restarts nothing, so the stream stayed silent until reconnect. `.mixWithOthers` on every category path (playAndRecord, playback, tvOS) is the heart of it: a mixable session interrupts nobody at activation and is not interrupted by another app's mixable playback — game audio rides over the playlist, the way a console does it. The trade is that a mixable session is nobody's Now Playing app, so the lock screen keeps showing the music. For a game stream that is the correct reading. The other half is `installInterruptionObserver`: interruptions still exist for a mixable session — a phone call, Siri, an app claiming a NON-mixable session of its own — and ending one restarts nothing by itself. On `.ended` the observer re-runs the full activation (an interruption can drop the category, and the earpiece steer is per-route) and revives the engines through the same conservative `reviveStoppedEngines` the route-change and media-reset observers already use — unconditionally, not only on the `.shouldResume` hint: a live stream is the one case where the user's intent to keep hearing it is not in doubt, and the revive already declines when playback never went down. The teardown's `.notifyOthersOnDeactivation` stays as a courtesy for the edge where an old exclusive install interrupted something; a mixable session has nobody to notify. macOS build (session paths are #if'd out there), tvOS typecheck, 284 tests.	2026-08-10 12:47:05 +02:00
enricobuehler	a14b000c9b	feat(apple): the tray gradient blur is back — as a real blur this time ... The form screens' pinned title and legend lost their gradient blur when GamepadTrayScrim went (2590238b): the scrim was `.ultraThinMaterial`, and a material by definition lifts and tints whatever it blurs — it read grey over the aurora, and washed with the palette's ground it read coloured. There is no public way to blur a backdrop without that tint, so the layer was deleted rather than re-tuned, and the rows have scrolled to a raw edge since. Glur's GlurView blurs the backdrop through a gradient with NO material stage on top — the rows soften as they slide under the pinned trays and nothing carries a colour. Three things about the adoption are deliberate: - It is the library's GlurBackdrop product, which uses a private API (a variable-blur CAFilter on UIKit/AppKit). The public `.glur()` modifier is a Metal shader on a view's OWN content and silently no-ops over platform-backed views like ScrollView, so it cannot reach a backdrop at all. GlurBackdrop ships as a separate module precisely so adopters can weigh the App Store exposure; it disables hit testing itself, so the band never eats a touch. - The dependency is pinned by REVISION: GlurBackdrop exists only on main — no release carries it (the newest tag, `1.1`, predates it and is not three-component semver anyway, so `from:` resolution stops at 1.0.4, which doesn't have the product). Wired into BOTH manifests — the package for `swift build`, and the xcodeproj (remote package reference + per-target product deps) for the app targets, which compile these sources themselves and know nothing of Package.swift's dependencies. - The band takes NO content-side overhang, unlike the scrim it replaces — and dropping the scrim's hard-won -44/-72 runway is the point, not an oversight. That runway existed because a material carries body at every alpha and had to dissolve OUTSIDE the tray; carried over here it blurred fully-visible rows at rest (first on-glass verdict). Glur's ramp reaches literal zero, so the band ends exactly at the tray's layout edge: open content untouched, rows soften only where they actually slide under. The 80 pt outer bleed and the pinned shape (`transaction { $0.animation = nil }`) are kept — full-bleed by LAYOUT rather than `.ignoresSafeArea()`, so a push never shows the band growing into place. Mounted exactly where the scrim was: settings, add-host and pair trays (top + bottom) and the library title (top). Glur is attributed on the Acknowledgements screen alongside the bundled font — it is a Swift package, so the Rust-closure notices generator rightly knows nothing about it. macOS build, tvOS typecheck; band placement and the blur itself verified on the iPad on glass.	2026-08-10 12:46:45 +02:00
enricobuehler	bac63059a9	feat(apple): drive the console UI from a hardware keyboard ... Asked for by a field user: "make the iPadOS client compatible with keyboard to select games with keyboard arrows, enter to launch". An iPad on a Magic Keyboard and a couch Mac are the same situation the console layout was built for — a screen driven from a distance with a fixed set of directional inputs — and the cursor/confirm/back model already exists here for the pad. A keyboard is a third input onto it, not a new navigation scheme: arrows move, Return and Space activate, Esc backs out, everywhere the controller already worked (carousel, menu lists, prompts) plus the plain poster grid. `active` mirrors each caller's existing controller gate rather than being a second, parallel notion of "who has input". Without that, a launcher sitting under an open screen would keep eating key presses and navigate behind it — the same defect the pad gate exists to prevent. Esc returns `.ignored` when a screen has no back action, so it still reaches the `.cancelAction` shortcut that closes a macOS sheet. The plain grid needed real arithmetic rather than a flat index. It renders up to TWO `LazyVGrid` sections (launchers above titles), so a flat index steps by the wrong amount at the boundary whenever the first section's last row is partial — up from the titles' first row lands mid-launcher-row instead of above. `LibraryGridNav` moves within a section and hands off at its edges preserving the column, clamping into partial rows. It lives in PunktfunkKit because it is edge-case arithmetic and that is the target tests can reach; 12 cases cover the partial row, the hand-off, a stale cursor, an empty grid and a zero column count. The column count comes from the grid's MEASURED width run through `.adaptive`'s own fitting rule, so up/down move exactly one visual row instead of a guess that drifts with window size. Measured via a background GeometryReader — a sibling inside a ScrollView would claim the whole viewport. The grid cursor starts nil and only appears on the first arrow press, so a touch user is never shown a selection they didn't ask for. tvOS is excluded throughout: its focus engine already routes hardware arrows, and these screens hand it navigation authority deliberately. 17 PunktfunkKit tests pass; macOS + tvOS typecheck; launcher and settings verified rendering and navigating in the iPad Pro 13" simulator.	2026-08-10 09:59:28 +02:00
enricobuehler	3daead7d71	fix(apple): the sub-screens' legends sit where the launcher's does ... Carries the display-corner inset (previous commit) through the three screens that pin the same legend pill — settings, add-host, pair — so they don't drift from the launcher the moment the user opens one. Each of those trays is `maxWidth: .infinity, alignment: .leading`, so the pill is left-aligned there too and the iPad's short centre indicator is never in its way; the keyboard tray sits ABOVE the legend in the stack, so pulling the tray down doesn't put keycaps over the strip. And the settings tab strip now takes its padding from the row metrics instead of two fixed numbers. It was the other half of the iPad sizing complaint: the tier work grew the fields under it while the tab pills stayed at phone scale, which read as a tab bar that belonged to a different screen. macOS + tvOS typecheck. Re-landed on the probe spelling: the corner inset is read from `\.displayBottomInset` (DisplayBottomInsetProbe via ContentView), never asked of UIKit during body — the original spelling froze the update graph of any view that called it on device (see the legend-pinning commit).	2026-08-10 09:59:25 +02:00
enricobuehler	18d0009c35	feat(apple): the trust prompt takes controller input ... Trust-on-first-use had no pad wiring at all, and it appears at the worst possible moment for that: mid-connect, with capture already disabled so ContentView can blur the stream and stop forwarding. A pad-only user pressed A to connect, got this card, and found the controller in their hands doing nothing — the only way past was to reach for the screen. A trusts, B cancels, X runs the PIN ceremony instead. The legend only appears with a pad attached; naming controller buttons to a trackpad user would be describing hardware they don't have. `GamepadManager` is observed rather than read once so the bar arrives if a pad wakes up while the card is already open. Safe to poll unconditionally while the card is up: capture is off for its whole lifetime and the home screens are unmounted behind the session view, so nothing else is reading the pad. The A press that STARTED the connect is typically still held when this mounts — `GamepadMenuInput` adopts the held state without firing on start, so it doesn't auto-trust. tvOS is untouched: the focus engine already drives these buttons. macOS + tvOS typecheck.	2026-08-10 09:58:26 +02:00
enricobuehler	8ef5350431	fix(apple): pin the launcher legend to the display's corner ... On an iPad the controls pill sat two to three times further from the bottom of the screen than from the left, so it read as floating rather than tucked into the corner. `safeAreaInset` places its content INSIDE the safe area, so the pill's bottom margin was stacking on top of the device's own ~24 pt home-indicator inset while the leading margin had nothing to stack on. Subtracting that inset is the fix; GETTING the inset is where this commit's history is. Three spellings failed before this one, each silently: - a `GeometryReader` carrying `.ignoresSafeArea()` — a proxy reports no inset for an edge it has been told to ignore, so it can only answer 0; - `.ignoresSafeArea(.container, edges: .bottom)` on the inset CONTENT, which does not move content the inset mechanism itself placed; - asking UIKit for the key window (`UIApplication.shared.connectedScenes…`) DURING body — which answers the right number and then KILLS the asking view. On a physical iPad (never the simulator) that walk re-enters UIKit layout mid-render and SwiftUI silently severs the view's update graph: from then on every `@State` write lands in storage without `body` ever running again. That is exactly how Settings and Add Host stopped opening while their triggers kept firing and `showSettings` kept reading true — no AttributeGraph warning, nothing in the log. Found by bisecting device builds; a build that opens the screen programmatically renders green with input-driven navigation completely dead, so only a real press counts. So the inset is measured where it is knowable: DisplayBottomInsetProbe, a UIView that reads its WINDOW's inset from UIKit's own callbacks (didMoveToWindow / safeAreaInsetsDidChange / layoutSubviews), hops out of the current update, and publishes through `\.displayBottomInset` from ContentView. The legend reads the environment — proven safe on glass — and `gamepadLegendBottomPadding` is pure arithmetic. Tablets only. The padding goes negative to pull the pill back down through the indicator strip, which is safe there because the pill is left-aligned and an iPad's indicator is a short bar in the middle. A phone's indicator is taller and its legend runs most of the width, so the same move would cross it — phones keep the plain margin, and the tier comes from the size classes, so an iPad in a narrow Stage Manager window is treated as the in-hand case it is. macOS + tvOS typecheck; margins and Settings/Add Host opening from a real controller press verified on the iPad itself.	2026-08-10 09:58:20 +02:00
enricobuehler	522ac7bd49	feat(apple): the connect path's modals are pad-navigable ... `.alert` and `.confirmationDialog` are UIKit/AppKit surfaces: a game controller cannot move through their buttons or press one. On iOS/macOS that made every prompt in the connect path a dead end for a pad-only user, and they are not incidental prompts — "Pairing required" is the FIRST thing an unpaired host shows (so pairing was unreachable before it even got to a PIN), "Connection failed" strands the console UI behind a modal only a finger can dismiss, and "Waiting for approval" owns the only Cancel for a connect that may never complete. GamepadPromptView renders those states as a console card with a focus list of actions; the system alerts stand down while it is up. tvOS keeps them — the focus engine drives them natively there, which is exactly why this gap was invisible from that platform. Two things are deliberate rather than incidental: The gate is "not STREAMING", not `model.connection == nil`. A connection object exists well before a stream does — through the pair-required and approval handshakes, precisely when these fire — so gating on the connection would hand those cases back to the system dialog. Streaming is the one state that must keep the alert: there the pad belongs to GamepadCapture. And the overlay hangs off `driven`, not `home`, for the same reason: `home` renders only while the connection is nil, so a prompt mounted there would be skipped in the very case it was written for. The launcher stands down from the controller poll while a prompt is up (`promptActive`) — without it the host carousel keeps scrolling underneath the modal and one A press reaches both. macOS + tvOS typecheck; console UI verified opening Settings in the iPad simulator with the prompts wired in.	2026-08-10 09:56:25 +02:00
enricobuehler	8ab4918923	fix(apple): the host strip stopped flashing as it settles ... Reported from the iPad: the cards arrive, come to rest, and then flash. They did — and it was not the motion. Every card sat at the WRONG fill for as long as the entrance animation ran, then snapped to its real one in a single frame once SwiftUI tore the animation down, roughly a third of a second AFTER the last card had stopped moving. `rotation3DEffect` was the cause. A real 3D transform renders its subtree through an offscreen layer, and these cards carry translucent glass, which resolves differently in there — so the strip showed the aurora through itself for the duration and went opaque the instant the layer was dropped. Measured on an iPad Pro 13", sampling the centred tile: it held #4a3d87 across twelve frames in which nothing moved, then stepped to #423970 — 23/255 of blue in one frame, on every card at once, while the backdrop beside them stayed exactly #713fa0 throughout. Stretching the timeline from 1.02 s to 2.82 s moved the step from 0.70 s to 2.50 s after launch: the same 0.32 s before the end in both runs, which is what proves it is the animation ENDING and not any state the tiles were waiting on. Deleting the rotation deleted the step outright. `compositingGroup()` above and below the transforms each changed nothing, so this is the 3D layer specifically, not compositing in general. The turn is now projected instead of rendered: cos(angle) as a horizontal squeeze IS the orthographic projection of a Y-axis rotation, hinged on the edge the card fans from — which also puts the code behind the doc comment that already described the fan that way (the sign of a rotation carried it before; cos is even, so the anchor has to). Affine, so there is no offscreen pass and no layer to drop. What's lost is the perspective trapezoid, which at card size was never what sold the motion; the fan-out, the stagger and the overshoot are untouched. Also drops the temporary PUNKTFUNK_DIAG_OPEN hook that gated each step of this branch in the simulator. 284 tests pass; macOS + tvOS typecheck; entrance re-recorded on the iPad Pro simulator — the tile now settles monotonically (#4153cf → #383c77 → #39396e → #393a6e) and holds, with the fan-out still reading as a turn.	2026-08-10 09:36:02 +02:00
enricobuehler	09bac99090	feat(apple): pairing you can actually do with a controller ... PairSheet is a `Form` with two `TextField`s. On tvOS the focus engine drives those natively, but on iOS/macOS a controller cannot reach a text field, type into it, or press the button underneath — so for anyone in the console UI, pairing (the ONE thing between a fresh install and a first stream) ended at "now touch the screen". GamepadPairView is the same ceremony in the gamepad UI's own vocabulary: the vertical focus list the settings and add-host screens use, A on a field to open GamepadKeyboard in a bottom tray, B to peel one layer. It mirrors GamepadAddHostView field for field, because it is the same interaction and someone who has added a host should recognise it immediately. The ceremony itself moved to a shared `PairCeremony` used by both presentations, so they can never disagree about what a wrong PIN means, what a host rejection says, or when a late result must be discarded. On iOS it is a shell layer like settings and add-host, and it LEADS the shell's screen order: it blocks a connect the user already asked for and can be raised from on top of the library (launching a title on an unpaired host), so it has to win; backing out reveals whatever it interrupted. macOS has no shell, so its sheet switches content by mode instead. tvOS is untouched. macOS + tvOS typecheck; console UI verified opening Settings in the iPad simulator with the pair screen wired into the shell.	2026-08-10 08:12:50 +02:00
enricobuehler	5db3b3c4fd	feat(apple): the controls legend is clickable ... Every hint cell that names an action is a real button on iOS/iPadOS/macOS. The legend already lists every action a screen has, in one fixed place, so a user without a pad in their hands — an iPad on a stand, a Mac driven by trackpad, anyone running `gamepadUIMode == "always"` — was reading a complete menu they could not press. tvOS keeps them inert deliberately. There is no pointer there, so a tappable cell would have to be FOCUSABLE, and that puts six new stops in the path of a focus engine whose flow on these screens is load-bearing and hard-won — while every action in the legend already has a native route (select, Menu, Play/Pause, the focusable tab pills). Cells that name an INPUT rather than an action stay labels: "↔ Adjust" is the stick itself, and "A Type" over the on-screen keyboard has no tap equivalent because a touch user types by tapping the keycap. Two details that are load-bearing rather than tidy: the decorative hairline gets `allowsHitTesting(false)` (it sits on top of the cells), and the press style's `contentShape` sits below its `scaleEffect` so shrinking the artwork cannot move the target out from under a resting finger and lose the touch-up. macOS + tvOS typecheck; 272 tests pass; console UI verified opening Settings in the iPad simulator.	2026-08-10 08:05:14 +02:00
enricobuehler	ffb1ecfebe	feat(apple): size the console UI for the iPad ... GamepadFormMetrics had exactly two tiers, tvOS and everything-else, so a 13" iPad Pro drew its settings rows, section tabs and legend at iPhone scale in the middle of a very large display — the field verdict was that the sizing "does not adapt to larger screens". There is a third tier now, between the in-hand and 10-foot sets. Chosen from the SIZE CLASSES, not the device idiom: an iPad in a narrow Stage Manager or Split View window keeps the in-hand numbers, because what is being read is the window, not the panel behind it. Phone sizing is untouched. Published from ContentView — the app root — rather than only from `gamepadPaletteInk`. A screen that applies that modifier itself sits ABOVE its own copy of the environment, so its `@Environment` resolves against its parent and would read the bare phone default no matter what the modifier published. The screenshot harness publishes it too, for the same reason: it mounts these screens with no ContentView above them, so an iPad capture used to render the whole gamepad UI at iPhone scale. macOS + tvOS typecheck; the console UI verified in the iPad simulator opening Settings at the tablet tier.	2026-08-10 07:44:36 +02:00
enricobuehler	08e462fee9	fix(apple): Graphite's selected settings tab was white on light grey ... The tab strip's selected pill is FILLED with the palette accent, and `ink.onAccent` — chosen by that accent's own luminance, and documented for "a filled pill's label" — has existed for exactly this since the palettes landed. The strip was using the plain foreground instead, which is white on every dark palette and therefore looked fine on all but one: Graphite's accent is a light grey (luma ≈ 0.80), so its selected tab read white on white.	2026-08-10 07:41:43 +02:00
enricobuehler	2590238b8f	fix(apple): the console sub-screens' backdrop, glass and option band ... Four things the field reported on glass, all in the console's visual layer. The tray scrim is GONE, not re-tuned. It laid `.ultraThinMaterial` across the top and bottom of every form screen, and a material by definition lifts and tints whatever it blurs — so it read grey, and washing it with the palette's ground only made it read coloured. There is no public way to blur a backdrop without that tint, so the layer had to go. The darkening it also provided was never its to provide: the backdrop's own legibility scrim (0.38/0.06/0.08/0.40 — the same gradient pf-console-ui bakes into its shader) has always been one layer down, which is why the console has no tray band. Pale palettes rendered every row, pill and card as a flat white slab, and lowering the tint did nothing, because the opacity came from the glass BODY: `.regular` is a bright, high-body material and a pale palette's `ink.glass` is literal white. Pale palettes take `Glass.clear` now, with a light wash to keep dark ink legible. A focused settings row changed colour in a hard jump, a beat late, while its scale animated smoothly beside it — the focus tint rode `Glass.tint`, and a Glass value is opaque to SwiftUI's animation system. It is a plain fill between the glass and the label now, so the existing animation covers it. The host cards appeared to be swapped for different ones as their entrance landed: `CardEntrance` swings each tile in on a rotation3DEffect, and Liquid Glass samples the backdrop through its own layer, which it cannot do under a 3D transform. Those tiles take the material path (`forceMaterial`). And the option band never turned like a cylinder because the band MASKED itself: a mask rasterises what it covers, flattening `rotation3DEffect`'s perspective, so the projection was computed and discarded every frame. The soft edge is folded into each option's own opacity instead. Its ±1 neighbours stay hidden at rest — showing them reproduced the documented overlap defect ("2752 × 2064" with "280 ×" through it), which is why they were dropped. Also fixes pale palettes washing out: the calm mix added a plusLighter wash of a near-white ground on top of a field already mixed toward it, saturating the form screens to white. macOS + tvOS typecheck; console UI verified opening Settings in the simulator, with no tray band.	2026-08-10 07:33:11 +02:00
enricobuehler	beb639f067	fix(ui): the OLED palette is called Eclipse now ... Every other background reads as a place or a material — Violet, Nebula, Abyss, Ember, Moss, Graphite, Holo, Sunset, Bloom, Dawn, Mint, Opal — and "OLED" was a spec sheet sitting among them, naming the panel it suits rather than what it looks like. It looks like black with a thin violet corona at one corner, which is an eclipse, and that puts it beside Nebula and Abyss. The ID stays "oled" in all three clients: it is the stored ui_palette value AND the cross-client key, so renaming it would orphan every saved choice and silently desync Apple, Android and the console UI. Only the label moved. No test or doc pinned the old string. Palette tests pass; macOS builds.	2026-08-10 07:30:30 +02:00
enricobuehler	90450ff1f6	feat(apple): the legends remember which pad you own ... `buttonGlyph` asked the LIVE controller for its `sfSymbolsName` and fell back to generic letter glyphs, so the moment a DualSense slept, disconnected or ran flat, every ✕/◯ hint turned into A/B under its owner's hands. Under `gamepadUIMode == "always"` — the console UI up with no pad by design — they were never right at all. GamepadManager now remembers the family of the last controller that was actually attached (`DefaultsKey.lastGamepadKind`, written on selection and deliberately never cleared on disconnect), and the legends resolve live → remembered → generic. Not a user-facing setting: a glyph-style picker is one more row asking a question the app can answer itself, and the remembered pad is right essentially always. GamepadGlyphs maps by POSITION, not by name, because that is what `GCExtendedGamepad`'s buttonA/B/X/Y are. So ✕ is role `.a` (the bottom button), and a Switch pad gets Nintendo's transposed labels rather than Xbox letters printed on the wrong physical buttons. The test asserts every symbol name RESOLVES on the running OS. That is the load-bearing one: `Image(systemName:)` draws an unknown name as nothing at all — no crash, no log, no red build — so a typo would silently blank a legend on device and no amount of building would catch it. macOS + tvOS typecheck; the 5 glyph tests pass; the console UI verified opening its Settings screen in the iPad simulator.	2026-08-10 07:29:33 +02:00