Compare commits

..
Author SHA1 Message Date
enricobuehler 1e243c6c93 Merge branch 'main' into worktree-shield-select-back-quit
ci / rust-arm64 (pull_request) Successful in 1m30s
ci / web (pull_request) Successful in 2m0s
ci / bun-nix (pull_request) Successful in 30s
ci / docs-site (pull_request) Successful in 1m52s
ci / rust (pull_request) Successful in 26m11s
2026-08-14 20:12:18 +00:00
enricobuehler b5cace3a00 Merge pull request 'The app menu ate ⌘Q, so the host's compositor never saw the chord' (#236) from worktree-apple-cmd-passthrough into main
apple / swift (push) Successful in 2m3s
apple / distribute (push) Successful in 10m51s
apple / screenshots (push) Successful in 8m49s
ci / rust (push) Successful in 27m48s
ci / web (push) Successful in 9m48s
ci / docs-site (push) Successful in 8m58s
ci / bun-nix (push) Successful in 27s
ci / rust-arm64 (push) Successful in 2m3s
Reviewed-on: #236
2026-08-14 20:10:16 +00:00
enricobuehler 1e5dca4c25 Merge pull request 'A pad whose Select is KEYCODE_BACK quit the session on one press' (#235) from worktree-shield-select-back-quit into main
Reviewed-on: #235
2026-08-14 20:10:04 +00:00
enricobuehler b2146f33fe fix(apple): the app menu ate ⌘Q, so the host's compositor never saw the chord
ci / rust (pull_request) Successful in 6m40s
ci / web (pull_request) Successful in 1m11s
ci / docs-site (pull_request) Successful in 2m13s
ci / bun-nix (pull_request) Successful in 46s
apple / swift (pull_request) Successful in 2m5s
apple / distribute (pull_request) Skipped
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m58s
With the default modifier layout ⌘ is Super on the host, which makes ⌘Q the chord
a Hyprland/KDE/GNOME user reaches for first. AppKit dispatches menu key
equivalents before the stream view ever sees a keyDown, so it quit the client
instead. Not Hyprland-specific.

`InputCapture`'s local keyDown monitor now claims every ⌘ chord while input is
captured and forwards it to the host itself. It has to send from there: the
monitor runs ahead of BOTH the menu and `StreamLayerView.keyDown`, and on macOS
that second one is the host's only key path (the GCKeyboard send has been
iOS-only since e414ec0) — so returning nil to keep the menu out takes the host's
copy with it. The file's own comment said the opposite, that swallowing keys
"risks starving GC's own delivery"; on macOS there is no GC delivery to starve,
which is why this could never have been a one-line `return nil`.

Verified against AppKit rather than assumed: a standalone harness posting a
synthetic ⌘Q confirms the monitor sees it first and that returning nil stops the
menu item firing, with a passed-through ⌘W as the control.

⌘⎋ and ⌃⌘F stay client-side whatever the setting says — forward those and a
captured stream is a room with no door. ⌘Tab, ⌘Space and Mission Control are out
of reach for a local monitor: macOS claims them before any app sees them, and
catching them needs a CGEventTap and an Accessibility prompt, which is a product
decision rather than a code one.

This answers to the cross-client "Capture system shortcuts"
(`Settings::inhibit_shortcuts`), which the Apple client had no answer to because
SDL's keyboard grab is what implements it everywhere else. Default on,
profileable like its siblings, and — matching the SDL clients — inert under the
desktop mouse model, which is something you ⌘Tab away from.

Two adjacent defects fixed on the way, both the same root cause. macOS stops
delivering keyUp while ⌘ is held, so a forwarded chord key is released when the
last ⌘ comes up rather than waiting for an up that may never arrive, and the
one-shot `suppressedVK` latch is cleared in the same place — left pending (⌃⌘F's
F, ⌘⎋'s Esc) it would go on to eat the next press of that key. Chord matching
also stopped comparing the raw `deviceIndependentFlagsMask`, which carries Caps
Lock and the arrows' .function/.numericPad bits: with Caps Lock on, ⌘⎋ and
⌃⌥⇧Q — both escape hatches — were not recognized at all.

On glass still owed: a clean compile proves nothing for an input grab.
2026-08-14 22:04:30 +02:00
enricobuehler 1ac6c9bf3d fix(android): a pad whose Select is KEYCODE_BACK quit the session on one press
ci / bun-nix (pull_request) Successful in 21s
ci / docs-site (pull_request) Successful in 1m23s
ci / rust-arm64 (pull_request) Successful in 2m50s
ci / web (pull_request) Successful in 5m55s
ci / rust (pull_request) Canceled after 6m47s
android / android (pull_request) Successful in 5m38s
Field report: pressing Select disconnected the stream. The host log was
unambiguous about what it was NOT — "client datagram stream ended" plus "virtual
display torn down (deliberate quit — keep-alive skipped)" is a client that said
it was leaving, not a drop and not a compositor crash.

Only two client paths raise that: StreamScreen's BackHandler, and the exit chord
(router.onExitChord). The chord is excluded by construction — `armExit` posts a
1 s timer and releasing any member calls `disarmExit`, so a tap always cancels.
That leaves the back stack, and from a SOURCE_GAMEPAD device KEYCODE_BACK is the
ONLY keycode that reaches it: a mapped button is consumed in the gamepad branch,
anything with a VK is consumed on the keycode path, volume/power go to the
system, and a FLAG_FALLBACK BACK is swallowed. So a one-press quit identifies the
button's keycode without knowing which controller was on the couch.

Plenty of pads deliver Select as the plain KEYCODE_BACK a remote's Back uses,
with no BUTTON_SELECT scancode behind it — the Android-TV shape, where every
input device is expected to offer Back, reached whether the vendor prints "Back"
on the button or "Select"/"View". `buttonBit` had no row for KEYCODE_BACK, so the
press fell through unconsumed into StreamScreen's BackHandler, which is the
deliberate-quit exit. One press, session over.

The same gap meant those pads could not produce BTN_BACK at all, so every
shortcut built on Select was unreachable on exactly the devices whose users have
no keyboard: the emergency exit chord StreamScreen's own start banner advertises
("Hold Select + Start + L1 + R1 to leave"), the mic mute, the stats tier.

New `Gamepad.padButtonBit(keyCode, flags)` — buttonBit plus that one row —
resolves a gamepad-sourced BACK to BTN_BACK, and MainActivity's streaming branch
asks it instead. It keys off the keycode, not the vendor, so it covers every pad
with this behaviour; a pad that does carry BUTTON_SELECT is unaffected in both
directions, having never had the bug. FLAG_FALLBACK events stay excluded: those
are the synthetic BACK the framework raises after an unconsumed BUTTON_* press,
and forwarding one would put a phantom Select on the wire (one landing while
Start + L1 + R1 were held would complete the exit chord out of nowhere). A
remote's or keyboard's BACK is neither mouse- nor gamepad-sourced, so it still
leaves the stream — for a device with no pad on it that is the documented way
out, and the banner says so.

The mouse-side-button hook moves above the gamepad branch so a device that can
be a mouse keeps its X1/X2 semantics; it answers null for everything that cannot
be a mouse, so nothing else changes route.

PadButtonBitTest pins the mapping, the fallback exclusion, that the three Select
chords are now reachable from a BACK-only pad, and that no other keycode moved.

Verified: :kit:testDebugUnitTest + :app:testDebugUnitTest green (PadButtonBitTest
4/4), :app:compileDebugKotlin clean. NOT yet verified on-glass — the behaviour
needs a real pad: Select reaches the game, and Back no longer quits.
2026-08-14 21:52:16 +02:00
enricobuehler 36e133ae66 Merge pull request 'Android canary ships to Play open + closed testing instead of internal' (#234) from worktree-android-canary-open-testing into main
ci / web (push) Successful in 1m30s
ci / bun-nix (push) Successful in 17s
ci / rust-arm64 (push) Failing after 2m32s
ci / docs-site (push) Successful in 1m45s
android / android (push) Successful in 7m59s
ci / rust (push) Canceled after 17m30s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 11s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 12s
docker / builders (ci/gamescope-trixie.Dockerfile, punktfunk-gamescope-trixie) (push) Successful in 12s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m20s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 52s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 27s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / builders-arm64cross (push) Successful in 9s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 48s
docker / deploy-docs (push) Successful in 36s
Reviewed-on: #234
2026-08-14 19:50:39 +00:00
enricobuehler d7e66fafe1 Merge pull request 'The store's four marketing frames become screenshot scenes on both platforms' (#233) from worktree-store-marketing-scenes into main
android / android (push) Canceled after 17s
ci / rust (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
ci / bun-nix (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/gamescope-trixie.Dockerfile, punktfunk-gamescope-trixie) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
apple / swift (push) Successful in 1m59s
apple / distribute (push) Successful in 11m3s
apple / screenshots (push) Canceled after 7m31s
Reviewed-on: #233
2026-08-14 19:49:33 +00:00
enricobuehler 026dbe6153 fix(screenshots): captures grow the status bar Robolectric never had
ci / rust-arm64 (pull_request) Successful in 1m57s
ci / bun-nix (pull_request) Successful in 48s
ci / docs-site (pull_request) Successful in 1m55s
apple / swift (pull_request) Successful in 2m3s
apple / distribute (pull_request) Skipped
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 8m58s
android / android (pull_request) Successful in 6m12s
ci / rust (pull_request) Successful in 27m10s
Robolectric renders no system UI and zero insets, so every phone capture
was missing the status bar and its content sat where the bar belongs — on
the Pixel store render the app title collided with the camera punch-hole.
ShotStatusFrame draws a plausible bar (time left, radios right, the
CENTRE left empty for the hole) and pushes the scene below it, the same
geometry real insets produce; height mirrors a Pixel's tall bar measured
off a real capture. On for the touch screens, off for the immersive
surfaces (stream, console shell, TV) that hide the real bar too.
2026-08-14 19:53:09 +02:00
enricobuehler cc70c64797 feat(screenshots): the add-host sheet becomes a scene — the blends' phone screens were three designs old
The Blender store scenes render whatever screens/ holds, and theirs were
June captures of the pre-console UI. Fresh captures existed for hosts and
pair but the add-host sheet had no scene: AddHostSheet's state is hoisted
(ConnectScreen keeps half-typed values across dismissal), so the scene
passes a filled form straight in.

Two capture-truth fixes with it: dialog scenes advance the frozen clock
1.6 s (a ModalBottomSheet's entrance spring is still mid-rise at 0.8 s),
and the add-host shot uses Pixel-like geometry (411×915dp @ 420 dpi —
same 1080×2400 px, but the dp headroom is what lets the Connect button,
the row carrying the resolution promise, fit in frame).
2026-08-14 19:15:37 +02:00
enricobuehler b66bcef528 fix(screenshots): the macOS leg built the harness out of existence
The whole shot harness is #if DEBUG, and shoot_macos built -c release —
so the binary launched as the NORMAL app, never printed PF_SHOT_WINDOW,
and every scene 'never reported a window' while the script SIGKILLed a
perfectly healthy app. Build debug: SwiftUI has no release-only visuals,
and the harness actually exists there. All eight mac scenes capture now.
2026-08-14 18:23:38 +02:00
enricobuehler f033d3f5df fix(screenshots): pin the shot palette — a reused device's saved choice shipped a sunset Apple TV set
The aurora screens read the LIVE uiPalette default, and shot mode never
forced one: the Apple TV Simulator had a sunset palette persisted from
manual use, so every tvOS capture came out pink-on-pale while the iPhone
set stayed violet. ScreenshotHostView now pins the palette (violet, or
PUNKTFUNK_SHOT_PALETTE) before the scene mounts.

Also documents the local tvOS-SIMULATOR wall in screenshots.sh: Xcode
26.6 and the 27 beta plan the macro targets swiftui-navigation-transitions
pulls in for the tvOS triple and never schedule their swift-syntax deps
('unable to resolve module dependency') — prebuilts on or off. Only the
tvOS target links that package, which is why iOS and device builds never
hit it. Local workaround, since HomeView's use is canImport-guarded:
temporarily unlink the product from the tvOS target, capture, restore.
2026-08-14 17:11:35 +02:00
enricobuehler e3443da108 fix(screenshots): the store frames go landscape — the app is built for horizontal use
Portrait captures show a layout nobody streams in. The touch controllers
frame and the library shot now render at landscape phone geometry (the
portrait library variant is gone), a console-controllers-landscape frame
joins the set, and the Apple 12-controllers scene rotates: on the
landscape canvas the two pads sit as side-by-side columns — one
ControllerTestView per pad — so neither story is cut by the short height.

Known wart, deliberate: the console landscape frame's floating legend
overlaps the second pad card mid-scroll; the styled composite crops above
it, and the touch variant carries the uncropped two-card view.
2026-08-14 15:07:23 +02:00
enricobuehler 5a4dd7423e feat(screenshots): the Apple controller panel renders a value model, so the harness can inject pads
ControllerTestView drew straight from GCController/GCExtendedGamepad, and a
GCController cannot be constructed — the store plan's FEEL THE GAME frame
had no Apple scene. Every card now renders plain values (ShotPad,
InputSnapshot): the live path flattens the active DiscoveredController and
samples the pad into a snapshot on each 30 Hz tick, the screenshot harness
hands the panel pads that were never connected via a default-nil shotPads
parameter (the seam Android's ControllersScreen grew in 0a468c96). Live
behavior is unchanged — same cards, same order, same live feeds.

The 12-controllers scene injects the two pads the listing names — the
DualSense leading with the feedback surface (adaptive-trigger effects,
rumble backend, lightbar + player LEDs), the Xbox pad carrying the input
readout frozen mid-game; transport/battery/player ride in the header's
detail line because the panel has no dedicated battery row. Registered in
the iOS/macOS block and the store set only: ControllerTestView does not
build on tvOS, so the tvOS CI scene list is untouched.
2026-08-14 15:01:20 +02:00
enricobuehler 0a468c96da feat(screenshots): the two missing marketing frames — the library shelf and pads that exist
The store plan's PICK & PLAY and FEEL THE GAME shots had no scene on any
platform: the library screen's state comes off the network, and the
controllers screens enumerate InputDevices, of which Robolectric has none
(the old shot honestly said 'no controller detected' — a palette proof
that sells nothing).

- Android library: Coverflow goes internal and LibraryScene rebuilds the
  real shell around it (aurora, header, hint bar) with a mock shelf.
  Cover art is answered synchronously by coil-test's FakeImageLoaderEngine
  with generated gradient posters, so the frozen animation clock never
  races an async load. Shot at phone portrait+landscape and TV geometry.
- Android controllers: PadRow renders a PadInfo model instead of a raw
  InputDevice (padInfoOf maps real devices; both screens take a
  padsOverride). The scenes inject the two pads the listing names —
  DualSense (player 1) and Xbox (player 2), real VID:PIDs.
- Apple library: ShotLibrary composes the real LibraryCoverflowView with
  a JSON-decoded mock shelf (GameEntry's memberwise init is internal to
  PunktfunkKit; Codable is the public construction surface), registered
  as cross-platform scene 11-library and added to the store set + the
  tvOS CI scene list. Artless entries settle to their deterministic
  fallback posters, which is also what keeps the shot offline.

Apple controllers stays a follow-up: ControllerTestView binds to live
GCController hardware and has no injection surface yet.

Verified: all 31 Roborazzi scenes render; the new tv-library,
phone-library and controllers shots reviewed by eye.
2026-08-14 14:38:08 +02:00
25 changed files with 1119 additions and 147 deletions
+1 -1
View File
@@ -755,7 +755,7 @@ jobs:
bash tools/screenshots.sh ipad || echo "::warning::iPad 13\" screenshots skipped"
# tvOS shoots only the scenes that exist there — the 0609 gamepad-console scenes are
# compiled out on tvOS (native focus engine), and an unknown name = a normal app launch.
SCENES="01-stream 02-hosts 05-settings 03-pair" \
SCENES="01-stream 02-hosts 11-library 05-settings 03-pair" \
bash tools/screenshots.sh tvos || echo "::warning::Apple TV screenshots skipped"
echo "Produced:"; ls -la screenshots || true
+4
View File
@@ -142,6 +142,10 @@ dependencies {
// job runs `:app:testDebugUnitTest -PskipRustBuild` (see kit/build.gradle.kts). ---
testImplementation(composeBom)
testImplementation("androidx.compose.ui:ui-test-junit4")
// Deterministic cover art for the library scene: FakeImageLoaderEngine answers the coverflow's
// AsyncImage synchronously with generated posters — no network, no async race under the frozen
// animation clock.
testImplementation("io.coil-kt:coil-test:2.7.0")
debugImplementation("androidx.compose.ui:ui-test-manifest") // the ComponentActivity test host
testImplementation("junit:junit:4.13.2")
// Real `org.json` for the shared-vectors test: the `org.json` inside `android.jar` is a stub
@@ -69,7 +69,7 @@ import kotlinx.coroutines.delay
* to be the same one whichever interface asked.
*/
@Composable
fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit) {
internal fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit, padsOverride: List<PadInfo>? = null) {
BackHandler(onBack = onBack)
var testing by remember { mutableStateOf(false) }
ControllersBody(
@@ -77,6 +77,7 @@ fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit) {
scroll = rememberScrollState(),
testing = testing,
onTestingChange = { testing = it },
padsOverride = padsOverride,
// The touch screen holds the probes for its whole life: events are OBSERVED (not consumed)
// while the test is off, which is what keeps the "Last input" line live while browsing.
// Nothing else here wants the pad, so there is no one to hand them to.
@@ -99,7 +100,12 @@ fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit) {
* drops out of the probe slots and B is a HOLD (below). Everything reverts the moment it ends.
*/
@Composable
fun ConsoleControllersScreen(gamepadSetting: Int, onBack: () -> Unit, navActive: Boolean = true) {
internal fun ConsoleControllersScreen(
gamepadSetting: Int,
onBack: () -> Unit,
navActive: Boolean = true,
padsOverride: List<PadInfo>? = null,
) {
BackHandler(onBack = onBack)
val landscape = LocalConfiguration.current.orientation == Configuration.ORIENTATION_LANDSCAPE
val hazeState = remember { HazeState() }
@@ -139,6 +145,7 @@ fun ConsoleControllersScreen(gamepadSetting: Int, onBack: () -> Unit, navActive:
scroll = scroll,
testing = testing,
onTestingChange = { testing = it },
padsOverride = padsOverride,
// Only while testing: the rest of the time the screen's own nav holds the
// probes, so the "Last input" line is a test-time readout here rather than
// an always-on one. A pad that reaches this screen at all has already
@@ -200,14 +207,17 @@ private fun ControllersBody(
onTestingChange: (Boolean) -> Unit,
observeInput: Boolean,
contentPadding: PaddingValues,
padsOverride: List<PadInfo>? = null,
heading: @Composable () -> Unit,
) {
val context = LocalContext.current
val activity = context as? MainActivity
// Device list, re-read on every hot-plug event.
// Device list, re-read on every hot-plug event. [padsOverride] replaces it wholesale: the
// screenshot harness runs where no InputDevice can exist, and the connected-pad card is the
// point of that shot.
var generation by remember { mutableIntStateOf(0) }
val pads = remember(generation) { Gamepad.pads() }
val pads = padsOverride ?: remember(generation) { Gamepad.pads() }.map(::padInfoOf)
val others = remember(generation) {
InputDevice.getDeviceIds()
.toList()
@@ -392,8 +402,8 @@ private fun ControllersBody(
// Every real controller is forwarded now (Automatic forwards them all, each on its own
// wire pad index) — not just the first. A joystick-only device Android doesn't classify as
// a gamepad still can't be forwarded (the host wants a gamepad), so gate the badge on it.
pads.forEach { dev ->
PadRow(dev, forwarded = isForwarded(dev), gamepadSetting = gamepadSetting)
pads.forEach { info ->
PadRow(info, gamepadSetting = gamepadSetting)
}
}
@@ -675,19 +685,19 @@ private fun DsRow(usbDev: android.hardware.usb.UsbDevice) {
/** One detected gamepad: identity, what it streams as, and a rumble test. */
@Composable
private fun PadRow(dev: InputDevice, forwarded: Boolean, gamepadSetting: Int) {
private fun PadRow(info: PadInfo, gamepadSetting: Int) {
OutlinedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(6.dp),
) {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Text(dev.name, style = MaterialTheme.typography.bodyLarge, modifier = Modifier.weight(1f))
if (forwarded) {
Text(info.name, style = MaterialTheme.typography.bodyLarge, modifier = Modifier.weight(1f))
if (info.forwarded) {
// Android's own controller number (1-based; 0 = unassigned), shown so a multi-pad
// user can tell which physical pad is which. The stream's wire pad index is
// assigned separately (lowest-free per device) once streaming starts.
val number = dev.controllerNumber
val number = info.controllerNumber
Text(
if (number > 0) "forwarded · player $number" else "forwarded to host",
style = MaterialTheme.typography.labelSmall,
@@ -696,11 +706,11 @@ private fun PadRow(dev: InputDevice, forwarded: Boolean, gamepadSetting: Int) {
}
}
Text(
deviceDetail(dev),
info.detail,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
val resolved = Gamepad.prefFor(dev)
val resolved = info.resolvedPref
Text(
if (gamepadSetting == Gamepad.PREF_AUTO) {
"Streams as: ${prefLabel(resolved)} (automatic)"
@@ -711,9 +721,8 @@ private fun PadRow(dev: InputDevice, forwarded: Boolean, gamepadSetting: Int) {
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
val canRumble = deviceHasVibrator(dev)
if (canRumble) {
OutlinedButton(onClick = { testRumble(dev) }) { Text("Test rumble") }
if (info.canRumble) {
OutlinedButton(onClick = { info.dev?.let(::testRumble) }) { Text("Test rumble") }
} else {
Text(
"No rumble motors reported — host rumble will be silent",
@@ -794,6 +803,32 @@ private fun Group(title: String, content: @Composable ColumnScope.() -> Unit) {
private fun isForwarded(dev: InputDevice): Boolean =
!dev.isVirtual && dev.sources and InputDevice.SOURCE_GAMEPAD == InputDevice.SOURCE_GAMEPAD
/**
* Everything [PadRow] renders, decoupled from [InputDevice] so the screenshot harness can compose
* the connected-pad card at all — Robolectric enumerates no input devices, and a marketing shot of
* "no controller detected" sells nothing. Production always maps a real device via [padInfoOf];
* [dev] powers the rumble test and is absent only in the harness (the button then no-ops).
*/
internal data class PadInfo(
val name: String,
val detail: String,
val forwarded: Boolean,
val controllerNumber: Int,
val resolvedPref: Int,
val canRumble: Boolean,
val dev: InputDevice? = null,
)
internal fun padInfoOf(dev: InputDevice): PadInfo = PadInfo(
name = dev.name,
detail = deviceDetail(dev),
forwarded = isForwarded(dev),
controllerNumber = dev.controllerNumber,
resolvedPref = Gamepad.prefFor(dev),
canRumble = deviceHasVibrator(dev),
dev = dev,
)
/** Whether the controller reports a rumble motor — via VibratorManager (API 31+) or the legacy Vibrator. */
private fun deviceHasVibrator(dev: InputDevice): Boolean =
if (Build.VERSION.SDK_INT >= 31) {
@@ -255,8 +255,10 @@ private fun MessageState(text: String) {
)
}
// Internal (not private): the screenshot harness composes the real coverflow with mock games —
// the library screen itself can't be shot, its state comes off the network.
@Composable
private fun Coverflow(
internal fun Coverflow(
games: List<GameEntry>,
loader: ImageLoader,
navActive: Boolean,
@@ -526,10 +526,25 @@ class MainActivity : ComponentActivity() {
override fun dispatchKeyEvent(event: KeyEvent): Boolean {
val handle = streamHandle
if (handle != 0L) {
// A mouse's side buttons, when they arrive key-shaped, are X1/X2 — not navigation.
// Resolved before the gamepad and remote-pointer hooks so neither can claim them as
// its own BACK. See [mouseSideButton] for how a mouse's BACK is told from a pad's or
// a remote's; it answers null for every device that cannot be a mouse, so asking it
// first re-routes nothing else.
mouseSideButton(event)?.let { back ->
when (event.action) {
KeyEvent.ACTION_DOWN ->
if (event.repeatCount == 0) mouseForwarder?.sideButtonKey(back, true)
KeyEvent.ACTION_UP -> mouseForwarder?.sideButtonKey(back, false)
}
return true
}
// Gamepad buttons (incl. DPAD only when truly from a gamepad — else KEYCODE_DPAD_* are
// keyboard arrows and belong to the VK path below).
// keyboard arrows and belong to the VK path below — and BACK, which is how a pad with
// no BUTTON_SELECT scancode delivers its Select: see [Gamepad.padButtonBit], which is
// why this asks it rather than `buttonBit`).
if (event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
val bit = Gamepad.buttonBit(event.keyCode)
val bit = Gamepad.padButtonBit(event.keyCode, event.flags)
if (bit != 0) {
// The router forwards the bit on this device's own wire pad index and tracks held
// state per pad. The emergency-exit chord (Select + Start + L1 + R1) is handled
@@ -540,17 +555,6 @@ class MainActivity : ComponentActivity() {
return true // consumed
}
}
// A mouse's side buttons, when they arrive key-shaped, are X1/X2 — not navigation.
// Resolved before the remote-pointer hook so pointer mode can't eat them as its own
// BACK. See [mouseSideButton] for how a mouse's BACK is told from a remote's.
mouseSideButton(event)?.let { back ->
when (event.action) {
KeyEvent.ACTION_DOWN ->
if (event.repeatCount == 0) mouseForwarder?.sideButtonKey(back, true)
KeyEvent.ACTION_UP -> mouseForwarder?.sideButtonKey(back, false)
}
return true
}
// TV remote-as-pointer sees non-gamepad keys first (SELECT long-press toggles it;
// while active it owns the D-pad/SELECT/PLAY-PAUSE/BACK).
if (!event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
@@ -567,12 +571,13 @@ class MainActivity : ComponentActivity() {
return true
}
when (event.keyCode) {
// Whatever [mouseSideButton] didn't claim. A view-level FALLBACK BACK appears when
// a BUTTON_* press goes unconsumed, and an air-mouse remote stamps its own BACK
// SOURCE_MOUSE; both are duplicates of something already handled, and letting
// either through doubles as Android navigation and yanks the user out of the
// stream. A remote/keyboard BACK is never mouse-sourced, so it still falls through
// to the BackHandler and exits.
// Whatever [mouseSideButton] and the pad branch didn't claim. A view-level FALLBACK
// BACK appears when a BUTTON_* press goes unconsumed, and an air-mouse remote stamps
// its own BACK SOURCE_MOUSE; both are duplicates of something already handled, and
// letting either through doubles as Android navigation and yanks the user out of the
// stream. A remote/keyboard BACK is never mouse-sourced and never gamepad-sourced,
// so it still falls through to the BackHandler and exits — which for a device with
// no pad on it is the documented way out.
KeyEvent.KEYCODE_BACK, KeyEvent.KEYCODE_FORWARD ->
if (event.isFromSource(InputDevice.SOURCE_MOUSE) ||
event.flags and KeyEvent.FLAG_FALLBACK != 0
@@ -34,19 +34,34 @@ class ScreenshotTest {
// cursor via an infinite animation that otherwise keeps Compose perpetually "busy", so
// setContent's wait-for-idle never returns. Frozen, the capture is also deterministic.
/** Full-screen content scenes: the compose root fills the device, so a root capture is the shot. */
private fun shootRoot(name: String, content: @androidx.compose.runtime.Composable () -> Unit) {
/**
* Full-screen content scenes: the compose root fills the device, so a root capture is the
* shot. [statusBar] draws the fake system bar and pushes content below it (see
* [ShotStatusFrame]) — off for the immersive surfaces (stream, console shell), which hide
* the real bar too.
*/
private fun shootRoot(
name: String,
statusBar: Boolean = true,
content: @androidx.compose.runtime.Composable () -> Unit,
) {
compose.mainClock.autoAdvance = false
compose.setContent { ShotTheme(content) }
compose.setContent { ShotTheme { if (statusBar) ShotStatusFrame(content) else content() } }
compose.mainClock.advanceTimeBy(800)
compose.onRoot().captureRoboImage("$out/phone-$name.png")
}
/** Dialog scenes: the AlertDialog is a separate window, so capture the whole screen (all windows). */
private fun shootScreen(name: String, content: @androidx.compose.runtime.Composable () -> Unit) {
private fun shootScreen(
name: String,
statusBar: Boolean = true,
content: @androidx.compose.runtime.Composable () -> Unit,
) {
compose.mainClock.autoAdvance = false
compose.setContent { ShotTheme(content) }
compose.mainClock.advanceTimeBy(800)
compose.setContent { ShotTheme { if (statusBar) ShotStatusFrame(content) else content() } }
// 1.6 s, not 0.8: a ModalBottomSheet's entrance spring is still mid-rise at 0.8 s and the
// add-host sheet's Connect button was captured half below the frame.
compose.mainClock.advanceTimeBy(1600)
captureScreenRoboImage("$out/phone-$name.png")
}
@@ -73,25 +88,25 @@ class ScreenshotTest {
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi") // landscape — the stream is immersive
fun stream() = shootRoot("stream") { StreamScene(io.unom.punktfunk.StatsVerbosity.DETAILED) }
fun stream() = shootRoot("stream", statusBar = false) { StreamScene(io.unom.punktfunk.StatsVerbosity.DETAILED) }
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun streamCompact() = shootRoot("stream-compact") { StreamScene(io.unom.punktfunk.StatsVerbosity.COMPACT) }
fun streamCompact() = shootRoot("stream-compact", statusBar = false) { StreamScene(io.unom.punktfunk.StatsVerbosity.COMPACT) }
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun streamNormal() = shootRoot("stream-normal") { StreamScene(io.unom.punktfunk.StatsVerbosity.NORMAL) }
fun streamNormal() = shootRoot("stream-normal", statusBar = false) { StreamScene(io.unom.punktfunk.StatsVerbosity.NORMAL) }
// Both banner texts, in the stream's own landscape geometry — it is bottom-centre, so the
// aspect is load-bearing.
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun streamBannerPad() = shootRoot("stream-banner-pad") { StreamBannerScene(pad = true) }
fun streamBannerPad() = shootRoot("stream-banner-pad", statusBar = false) { StreamBannerScene(pad = true) }
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun streamBannerTouch() = shootRoot("stream-banner-touch") { StreamBannerScene(pad = false) }
fun streamBannerTouch() = shootRoot("stream-banner-touch", statusBar = false) { StreamBannerScene(pad = false) }
// The touch flow is a Material dialog over the host grid (a separate window → shootScreen).
@Test
@@ -114,15 +129,15 @@ class ScreenshotTest {
// The console flow is the full-screen aurora takeover (a root capture).
@Test
fun connectingConsole() = shootRoot("connecting-console") { ConnectConsoleScene() }
fun connectingConsole() = shootRoot("connecting-console", statusBar = false) { ConnectConsoleScene() }
@Test
fun consoleSettings() = shootRoot("console-settings") { ConsoleSettingsScene() }
fun consoleSettings() = shootRoot("console-settings", statusBar = false) { ConsoleSettingsScene() }
/** A PALE palette: the whole UI flips to dark ink on white frost, which only a shot proves. */
@Test
fun consoleSettingsLight() =
shootRoot("console-settings-light") { ConsoleSettingsScene(paletteId = "holo") }
shootRoot("console-settings-light", statusBar = false) { ConsoleSettingsScene(paletteId = "holo") }
/**
* Landscape — the orientation the console actually runs in, and a DIFFERENT layout since the
@@ -132,16 +147,16 @@ class ScreenshotTest {
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun consoleSettingsLandscape() =
shootRoot("console-settings-landscape") { ConsoleSettingsScene() }
shootRoot("console-settings-landscape", statusBar = false) { ConsoleSettingsScene() }
// The console home, the screen the living backdrop is most of. The default sdk (36) draws the
// real AGSL MESH field; the paired API-31 shot below draws the blob fallback, so the two
// renderings of the same palette can be compared rather than assumed equivalent.
@Test
fun consoleHome() = shootRoot("console-home") { ConsoleHomeScene() }
fun consoleHome() = shootRoot("console-home", statusBar = false) { ConsoleHomeScene() }
@Test
fun consoleHomeLight() = shootRoot("console-home-light") { ConsoleHomeScene(paletteId = "holo") }
fun consoleHomeLight() = shootRoot("console-home-light", statusBar = false) { ConsoleHomeScene(paletteId = "holo") }
/**
* Landscape — the orientation the console UI actually runs in, and the only one wide enough to
@@ -149,7 +164,7 @@ class ScreenshotTest {
*/
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun consoleHomeLandscape() = shootRoot("console-home-landscape") { ConsoleHomeScene() }
fun consoleHomeLandscape() = shootRoot("console-home-landscape", statusBar = false) { ConsoleHomeScene() }
/**
* The API 31/32 field. `RuntimeShader` is API 33+, so everything below it keeps the four
@@ -158,24 +173,46 @@ class ScreenshotTest {
*/
@Test
@Config(sdk = [31], qualifiers = "w360dp-h800dp-xxhdpi")
fun consoleHomeBlobFallback() = shootRoot("console-home-blobs") { ConsoleHomeScene() }
fun consoleHomeBlobFallback() = shootRoot("console-home-blobs", statusBar = false) { ConsoleHomeScene() }
// The two screens the console reached for the first time in WP8.3. Each is shot on a dark AND a
// pale palette, because the console draws them through a ColorScheme derived from the palette's
// ink — and the pale one is the only place a grey-on-pastel slip can show up.
@Test
fun consoleLicenses() = shootRoot("console-licenses") { ConsoleLicensesScene() }
fun consoleLicenses() = shootRoot("console-licenses", statusBar = false) { ConsoleLicensesScene() }
@Test
fun consoleLicensesLight() =
shootRoot("console-licenses-light") { ConsoleLicensesScene(paletteId = "holo") }
shootRoot("console-licenses-light", statusBar = false) { ConsoleLicensesScene(paletteId = "holo") }
@Test
fun consoleControllers() = shootRoot("console-controllers") { ConsoleControllersScene() }
fun consoleControllers() = shootRoot("console-controllers", statusBar = false) { ConsoleControllersScene() }
/**
* The touch presentation, pads connected — landscape, like every store frame: the app is
* built for horizontal use, and a portrait capture shows a layout nobody streams in.
*/
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun controllers() = shootRoot("controllers") { ControllersScene() }
/** The console presentation at the same landscape geometry — the store's FEEL THE GAME frame. */
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun consoleControllersLandscape() =
shootRoot("console-controllers-landscape", statusBar = false) { ConsoleControllersScene() }
/**
* The library coverflow with a mock shelf — the store's PICK & PLAY frame. Landscape: the
* orientation the coverflow actually runs in, and the only one wide enough for neighbours.
*/
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi")
fun library() = shootRoot("library", statusBar = false) { LibraryScene() }
@Test
fun consoleControllersLight() =
shootRoot("console-controllers-light") { ConsoleControllersScene(paletteId = "holo") }
shootRoot("console-controllers-light", statusBar = false) { ConsoleControllersScene(paletteId = "holo") }
@Test
fun trust() = shootScreen("trust") {
@@ -197,4 +234,13 @@ class ScreenshotTest {
HostsScene()
PairDialog()
}
/**
* The add-host sheet (separate window → whole-screen capture). Pixel-like geometry, not the
* default 360×800dp: same 1080×2400 px, but at 420 dpi the extra dp headroom is what lets the
* sheet's Connect button — the row that carries the resolution promise — fit in frame.
*/
@Test
@Config(sdk = [36], qualifiers = "w411dp-h915dp-420dpi")
fun addHost() = shootScreen("add-host") { AddHostScene() }
}
@@ -1,14 +1,32 @@
package io.unom.punktfunk.screenshots
import android.content.Context
import android.content.res.Configuration
import android.graphics.Bitmap
import android.graphics.Canvas
import android.graphics.LinearGradient
import android.graphics.Paint
import android.graphics.Shader
import android.graphics.Typeface
import android.graphics.drawable.BitmapDrawable
import android.graphics.drawable.ColorDrawable
import android.graphics.drawable.Drawable
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.BatteryFull
import androidx.compose.material.icons.filled.SignalCellular4Bar
import androidx.compose.material.icons.filled.Wifi
import androidx.compose.material3.Icon
import androidx.compose.foundation.lazy.grid.GridCells
import androidx.compose.foundation.lazy.grid.GridItemSpan
import androidx.compose.foundation.lazy.grid.LazyVerticalGrid
@@ -35,8 +53,27 @@ import androidx.compose.runtime.CompositionLocalProvider
import io.unom.punktfunk.GamepadHome
import io.unom.punktfunk.GamepadInk
import io.unom.punktfunk.GamepadPalette
import coil.ImageLoader
import coil.test.FakeImageLoaderEngine
import dev.chrisbanes.haze.HazeState
import dev.chrisbanes.haze.hazeSource
import io.unom.punktfunk.AddHostSheet
import io.unom.punktfunk.ConsoleControllersScreen
import io.unom.punktfunk.ConsoleHeader
import io.unom.punktfunk.ConsoleLegendInset
import io.unom.punktfunk.ConsoleLicensesScreen
import io.unom.punktfunk.ControllersScreen
import io.unom.punktfunk.Coverflow
import io.unom.punktfunk.GamepadAuroraBackground
import io.unom.punktfunk.GamepadHintBar
import io.unom.punktfunk.PadGlyph
import io.unom.punktfunk.PadInfo
import io.unom.punktfunk.consoleLegendInsets
import io.unom.punktfunk.consoleSafeArea
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.library.Artwork
import io.unom.punktfunk.kit.library.GameEntry
import androidx.compose.ui.platform.LocalConfiguration
import io.unom.punktfunk.GamepadSettingsScreen
import io.unom.punktfunk.HomeTile
import io.unom.punktfunk.LocalGamepadInk
@@ -70,6 +107,51 @@ internal fun ShotTheme(content: @Composable () -> Unit) {
MaterialTheme(colorScheme = BrandDark, content = content)
}
/**
* Robolectric has no system UI, so every capture was missing the status bar and the content sat
* where the bar belongs — on the Pixel render the app title collided with the camera punch-hole.
* This frame draws a plausible bar (time left, radios right, the CENTRE left empty for the hole)
* and pushes the scene below it, the same geometry real insets produce. The height mirrors a
* Pixel's tall bar as measured off a real 1344×2992 capture (~145 px ≈ 40 dp).
*/
@Composable
internal fun ShotStatusFrame(content: @Composable () -> Unit) {
Column(Modifier.fillMaxSize().background(MaterialTheme.colorScheme.background)) {
Row(
Modifier.fillMaxWidth().height(40.dp).padding(horizontal = 28.dp),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically,
) {
Text(
"21:47",
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onBackground.copy(alpha = 0.9f),
)
Row(
horizontalArrangement = Arrangement.spacedBy(5.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Icon(
Icons.Filled.Wifi, contentDescription = null,
tint = MaterialTheme.colorScheme.onBackground.copy(alpha = 0.9f),
modifier = Modifier.size(15.dp),
)
Icon(
Icons.Filled.SignalCellular4Bar, contentDescription = null,
tint = MaterialTheme.colorScheme.onBackground.copy(alpha = 0.9f),
modifier = Modifier.size(14.dp),
)
Icon(
Icons.Filled.BatteryFull, contentDescription = null,
tint = MaterialTheme.colorScheme.onBackground.copy(alpha = 0.9f),
modifier = Modifier.size(16.dp),
)
}
}
Box(Modifier.weight(1f).fillMaxWidth()) { content() }
}
}
private data class MockHost(
val name: String,
val address: String,
@@ -510,8 +592,8 @@ internal fun ConsoleHomeScene(paletteId: String = "violet") {
* whole risk. Their touch presentation is inked by the app theme, which is always dark, so nothing
* before this could catch light-grey body text stranded on a pastel field.
*
* Robolectric enumerates no input devices, so the controllers scene renders its deterministic
* "nothing connected" state.
* Robolectric enumerates no input devices, so the controllers scenes inject [shotPads] — the
* deterministic connected-pads state the store listing needs.
*/
@Composable
internal fun ConsoleLicensesScene(paletteId: String = "violet") =
@@ -520,14 +602,147 @@ internal fun ConsoleLicensesScene(paletteId: String = "violet") =
@Composable
internal fun ConsoleControllersScene(paletteId: String = "violet") =
ConsolePalette(paletteId) {
ConsoleControllersScreen(gamepadSetting = 0, onBack = {}, navActive = false)
// Robolectric enumerates no input devices, so the shot injects the two pads the store
// listing talks about — the empty "no controller detected" state proves the palette but
// sells nothing.
ConsoleControllersScreen(
gamepadSetting = 0, onBack = {}, navActive = false, padsOverride = shotPads(),
)
}
/**
* The touch presentation of the same screen, with the same injected pads. Wrapped in a background
* [Surface]: the activity provides the dark ground in the app, and without one here the content
* color falls back to black-on-white while the cards stay dark.
*/
@Composable
internal fun ControllersScene() =
Surface(color = MaterialTheme.colorScheme.background) {
ControllersScreen(gamepadSetting = 0, onBack = {}, padsOverride = shotPads())
}
/**
* The "Add a host" bottom sheet over the host grid — the store's onboarding frame. State is
* hoisted in production (ConnectScreen), so the scene passes a filled-in form directly; the
* mode label mirrors what a paired 120 Hz phone shows on the connect button.
*/
@Composable
internal fun AddHostScene() {
HostsScene()
AddHostSheet(
hostName = "Living Room PC", onHostNameChange = {},
host = "192.168.1.42", onHostChange = {},
port = "9777", onPortChange = {},
connecting = false, modeLabel = "2992×1344@120",
onDismiss = {}, onConnect = { _, _, _ -> },
)
}
/** The two pads the store listing names: DualSense (adaptive triggers, LEDs, rumble) and Xbox. */
internal fun shotPads() = listOf(
PadInfo(
name = "DualSense Wireless Controller",
detail = "054C:0CE6 · gamepad · joystick",
forwarded = true, controllerNumber = 1,
resolvedPref = Gamepad.PREF_DUALSENSE, canRumble = true,
),
PadInfo(
name = "Xbox Wireless Controller",
detail = "045E:0B13 · gamepad · joystick",
forwarded = true, controllerNumber = 2,
resolvedPref = Gamepad.PREF_XBOXONE, canRumble = true,
),
)
/**
* Publish the palette locals `App` would normally provide. A scene that calls a console screen
* directly gets the DEFAULT dark ink without this, and a pale-palette shot would then silently
* prove nothing at all.
*/
/**
* The game-library coverflow (the real [Coverflow] over the real console chrome) with a mock shelf.
* The library screen itself can't be shot — its state comes off the network — so the scene rebuilds
* the same shell [io.unom.punktfunk.LibraryScreen] draws around it: aurora, header, floating hint
* bar. Cover art is answered synchronously by coil-test's [FakeImageLoaderEngine] with generated
* posters, so the frozen animation clock never races an async load.
*/
@Composable
internal fun LibraryScene(paletteId: String = "violet") = ConsolePalette(paletteId) {
val context = LocalContext.current
val loader = remember { shotLibraryLoader(context) }
val games = remember { shotGames() }
val hazeState = remember { HazeState() }
val landscape =
LocalConfiguration.current.orientation == Configuration.ORIENTATION_LANDSCAPE
Box(Modifier.fillMaxSize()) {
Box(Modifier.fillMaxSize().hazeSource(hazeState)) {
GamepadAuroraBackground(Modifier.fillMaxSize())
Column(Modifier.fillMaxSize().consoleSafeArea()) {
ConsoleHeader("Living Room PC — Library")
Box(Modifier.weight(1f).fillMaxWidth(), contentAlignment = Alignment.Center) {
Coverflow(games, loader, navActive = false, onLaunch = {})
}
}
}
Box(
Modifier.align(Alignment.BottomStart)
.consoleLegendInsets(landscape)
.padding(ConsoleLegendInset),
) {
GamepadHintBar(
listOf(PadGlyph.hint('A', "Launch"), PadGlyph.hint('B', "Close")),
hazeState = hazeState,
)
}
}
}
/** A believable shelf: four titles with art plus the Steam launcher entry (brand-mark tile). */
private fun shotGames() = listOf(
GameEntry("custom:aurora", "custom", "Aurora Drift", Artwork("shot://art/aurora", null, null)),
GameEntry("steam:starfall", "steam", "Starfall Vale", Artwork("shot://art/starfall", null, null)),
GameEntry("heroic:neon", "heroic", "Neon Circuit", Artwork("shot://art/neon", null, null)),
GameEntry("gog:ember", "gog", "Ember Peaks", Artwork("shot://art/ember", null, null)),
GameEntry("steam:launcher", "steam", "Steam", Artwork(null, null, null), role = "launcher", icon = "steam"),
)
private fun shotLibraryLoader(context: Context): ImageLoader {
val engine = FakeImageLoaderEngine.Builder()
.intercept("shot://art/aurora", cover(context, 0xFF6656F2, 0xFF141040, "A"))
.intercept("shot://art/starfall", cover(context, 0xFFE86FA8, 0xFF3A1030, "S"))
.intercept("shot://art/neon", cover(context, 0xFF35D0C5, 0xFF0A2A33, "N"))
.intercept("shot://art/ember", cover(context, 0xFFEF8F4B, 0xFF3A1608, "E"))
.default(ColorDrawable(0xFF221E44.toInt()))
.build()
return ImageLoader.Builder(context).components { add(engine) }.build()
}
/** A generated 2:3 poster: vertical brand-adjacent gradient + a big monogram. */
private fun cover(context: Context, top: Long, bottom: Long, mark: String): Drawable {
val w = 600
val h = 900
val bmp = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888)
val canvas = Canvas(bmp)
canvas.drawRect(
0f, 0f, w.toFloat(), h.toFloat(),
Paint(Paint.ANTI_ALIAS_FLAG).apply {
shader = LinearGradient(
0f, 0f, 0f, h.toFloat(), top.toInt(), bottom.toInt(), Shader.TileMode.CLAMP,
)
},
)
canvas.drawText(
mark, w / 2f, h / 2f + 110f,
Paint(Paint.ANTI_ALIAS_FLAG).apply {
color = 0xD9FFFFFF.toInt()
textSize = 320f
typeface = Typeface.create(Typeface.DEFAULT, Typeface.BOLD)
textAlign = Paint.Align.CENTER
},
)
return BitmapDrawable(context.resources, bmp)
}
@Composable
private fun ConsolePalette(paletteId: String, content: @Composable () -> Unit) {
val palette = GamepadPalette.named(paletteId)
@@ -50,6 +50,10 @@ class TvScreenshotTest {
@Test
fun consoleControllers() = shootRoot("console-controllers") { ConsoleControllersScene() }
/** The library coverflow at TV geometry — the store's PICK & PLAY frame for the TV listing. */
@Test
fun library() = shootRoot("library") { LibraryScene() }
@Test
fun connectingConsole() = shootRoot("connecting-console") { ConnectConsoleScene() }
}
@@ -230,6 +230,46 @@ object Gamepad {
else -> 0
}
/**
* The BTN_* bit for one key event from a SOURCE_GAMEPAD device — [buttonBit] plus the
* Select-family button of every pad that carries no `BUTTON_SELECT` scancode at all.
*
* Plenty of controllers deliver that button as the plain `KEYCODE_BACK` a remote's Back uses,
* with no `BUTTON_SELECT` behind it: it is the Android-TV shape, where every input device is
* expected to offer Back, and a pad reaches it whether the vendor prints "Back" on the button
* (NVIDIA's SHIELD controller) or "Select"/"View" (most pads in an Android mode). Which one is
* on the couch cannot be told from here, and does not need to be — the keycode is what routes.
*
* Read through [buttonBit] alone that button mapped to nothing, so it fell out of the
* streaming branch unconsumed and reached the activity's back stack, which is the
* deliberate-quit exit: ONE press of Select dropped the session and the host logged a client
* quit. `KEYCODE_BACK` is in fact the ONLY keycode that can get there from a pad — a mapped
* button is consumed here, anything with a VK is consumed on the keycode path, volume/power go
* to the system, and a FLAG_FALLBACK BACK is swallowed — which is what identifies this as the
* cause of such a report without knowing the hardware.
*
* It also meant such a pad could not produce [BTN_BACK] at all, so every shortcut built on
* Select — the emergency exit chord this client's own start banner advertises, the mic mute,
* the stats tier — was unreachable on exactly the devices whose users have no keyboard.
*
* A pad that DOES carry `BUTTON_SELECT` is unaffected in both directions: it never had the
* bug, and this changes nothing for it.
*
* FLAG_FALLBACK events are excluded: those are the synthetic BACK the framework raises after
* an unconsumed `BUTTON_*` press (a pad reporting L2/R2 as keys, say), not a button anyone
* touched, and forwarding one would put a phantom Select on the wire. `MainActivity` drops
* them on the keycode path for the same reason.
*
* Callers must gate on `SOURCE_GAMEPAD` before asking, exactly as [buttonBit]'s `KEYCODE_DPAD_*`
* rows require: a remote's or keyboard's BACK shares this keycode and has to keep leaving the
* stream — for a device with no pad on it, Back IS the documented way out.
*/
fun padButtonBit(keyCode: Int, flags: Int): Int = when {
keyCode != KeyEvent.KEYCODE_BACK -> buttonBit(keyCode)
flags and KeyEvent.FLAG_FALLBACK != 0 -> 0
else -> BTN_BACK
}
/**
* Maps one controller's joystick MotionEvents to axis (+ HAT→dpad) sends on wire pad index [pad],
* **on change only**. Holds the previous axis/hat state so an unchanged frame emits nothing. One
@@ -0,0 +1,80 @@
package io.unom.punktfunk.kit
import android.view.KeyEvent
import org.junit.Assert.assertEquals
import org.junit.Test
/**
* Pure JVM test of [Gamepad.padButtonBit] — the streaming branch's gamepad keycode resolution
* (`KeyEvent`'s keycode/flag constants are compile-time-inlined ints, so no Android runtime is
* involved). Run: `./gradlew :kit:testDebugUnitTest`.
*
* The regression it pins is a field report: one press of Select disconnected the session. Plenty
* of pads deliver that button as the plain `KEYCODE_BACK` a remote uses, with no `BUTTON_SELECT`
* scancode behind it — so it mapped to nothing, fell out of the gamepad branch unconsumed, and
* reached the activity back stack, which is the deliberate-quit exit. The same gap made
* [Gamepad.BTN_BACK] unreachable on those pads, and with it every shortcut built on Select: the
* exit chord `StreamScreen`'s own start banner advertises, the mic mute, the stats tier.
*
* Which controller the report came from is not knowable from the logs and does not matter:
* `KEYCODE_BACK` is the only keycode that reaches the back stack from a SOURCE_GAMEPAD device, so
* a one-press quit identifies the button's keycode on its own.
*/
class PadButtonBitTest {
/** The report: Select on an Android-TV pad arrives as BACK and must be the Select bit. */
@Test
fun `a pad's BACK is its Select button`() {
assertEquals(Gamepad.BTN_BACK, Gamepad.padButtonBit(KeyEvent.KEYCODE_BACK, 0))
// Same bit either spelling reaches us by — a pad that DOES carry BUTTON_SELECT is unchanged.
assertEquals(
Gamepad.padButtonBit(KeyEvent.KEYCODE_BUTTON_SELECT, 0),
Gamepad.padButtonBit(KeyEvent.KEYCODE_BACK, 0),
)
}
/**
* With Select mapped, the three Select chords are reachable on a pad that has only a BACK
* keycode — which is the whole point of the mapping, not a side effect of it. Held-state
* assembly is [GamepadRouter]'s (see `GamepadChordTest`); what is pinned here is that the
* bits a SHIELD can actually produce cover each chord.
*/
@Test
fun `the Select chords are reachable from a BACK-only pad`() {
val select = Gamepad.padButtonBit(KeyEvent.KEYCODE_BACK, 0)
val start = Gamepad.padButtonBit(KeyEvent.KEYCODE_BUTTON_START, 0)
val l1 = Gamepad.padButtonBit(KeyEvent.KEYCODE_BUTTON_L1, 0)
val r1 = Gamepad.padButtonBit(KeyEvent.KEYCODE_BUTTON_R1, 0)
val x = Gamepad.padButtonBit(KeyEvent.KEYCODE_BUTTON_X, 0)
val y = Gamepad.padButtonBit(KeyEvent.KEYCODE_BUTTON_Y, 0)
assertEquals(GamepadRouter.EXIT_CHORD, select or start or l1 or r1)
assertEquals(GamepadRouter.STATS_CHORD, select or x)
assertEquals(GamepadRouter.MIC_CHORD, select or y)
}
/**
* The synthetic BACK the framework raises after an unconsumed `BUTTON_*` press is not a button
* anyone touched — forwarding it would put a phantom Select on the wire, and one of those
* landing while Start + L1 + R1 were held would complete the exit chord out of nowhere.
*/
@Test
fun `a fallback BACK is not a button press`() {
assertEquals(0, Gamepad.padButtonBit(KeyEvent.KEYCODE_BACK, KeyEvent.FLAG_FALLBACK))
// Only BACK is filtered on the flag; a real button keeps its bit whatever rides alongside.
assertEquals(
Gamepad.BTN_A,
Gamepad.padButtonBit(KeyEvent.KEYCODE_BUTTON_A, KeyEvent.FLAG_FALLBACK),
)
}
/** Everything else is [Gamepad.buttonBit] verbatim — BACK is the only row this adds. */
@Test
fun `every other keycode is unchanged`() {
for (code in 0..0x400) {
if (code == KeyEvent.KEYCODE_BACK) continue
assertEquals(Gamepad.buttonBit(code), Gamepad.padButtonBit(code, 0))
}
// And BACK is genuinely a new row, not one buttonBit already had.
assertEquals(0, Gamepad.buttonBit(KeyEvent.KEYCODE_BACK))
}
}
@@ -16,6 +16,7 @@
// can wait for layout instead of guessing with a fixed sleep.
#if DEBUG
import PunktfunkKit
import SwiftUI
#if os(macOS)
import AppKit
@@ -43,6 +44,17 @@ enum ScreenshotMode {
/// readiness ping for the capture script.
struct ScreenshotHostView: View {
let scene: ShotScene
init(scene: ShotScene) {
self.scene = scene
// Pin the palette for the capture. The aurora screens read the LIVE `uiPalette` default,
// and a reused Simulator (or a dev Mac) carries whatever was last picked there the
// Apple TV set once shipped out on a sunset palette that a test device had persisted.
// Idempotent, and only ever runs in shot mode (this view exists behind that gate).
UserDefaults.standard.set(
ProcessInfo.processInfo.environment["PUNKTFUNK_SHOT_PALETTE"] ?? "violet",
forKey: DefaultsKey.uiPalette)
}
#if os(iOS)
@Environment(\.horizontalSizeClass) private var hSizeClass
@Environment(\.verticalSizeClass) private var vSizeClass
@@ -35,6 +35,11 @@ enum ShotScenes {
ShotScene(name: "05-settings", orientation: .natural, colorScheme: .dark) {
AnyView(ShotSettings())
},
// 0610 are the iOS/macOS console-shell block below; the library is cross-platform
// (tvOS renders the same coverflow), hence the number above that range.
ShotScene(name: "11-library", orientation: .landscape, colorScheme: .dark) {
AnyView(ShotLibrary())
},
]
#if os(iOS) || os(macOS)
// The gamepad-mode console screens (no tvOS native focus engine there). Dev-only shots
@@ -68,6 +73,13 @@ enum ShotScenes {
ShotScene(name: "09f-wake-timed-out-modal", orientation: .natural, colorScheme: .dark) {
AnyView(ShotConnect(kind: .timedOut, gamepadUI: false))
},
// FEEL THE GAME the controller test panel with injected pads. Gated with the
// console block because ControllerTestView doesn't build on tvOS, not because it
// is a console screen. Landscape like the rest of the store set: the app is built
// for horizontal use, so the two pads sit as side-by-side columns (see the scene).
ShotScene(name: "12-controllers", orientation: .landscape, colorScheme: .dark) {
AnyView(ShotControllers())
},
]
#endif
scenes.append(ShotScene(name: "10-edithost", orientation: .natural, colorScheme: .dark) {
@@ -193,6 +205,24 @@ enum ShotMock {
#endif
}
/// A believable shelf for the library coverflow. Decoded rather than constructed:
/// `GameEntry`'s memberwise init is internal to PunktfunkKit, and Codable is its public
/// construction surface. No art URLs the posters render their deterministic fallback
/// (title tiles, the Steam entry its brand mark), which is also what keeps the shot offline.
static let games: [GameEntry] = {
let json = """
[
{"id": "custom:aurora", "store": "custom", "title": "Aurora Drift", "art": {}},
{"id": "steam:starfall", "store": "steam", "title": "Starfall Vale", "art": {}},
{"id": "heroic:neon", "store": "heroic", "title": "Neon Circuit", "art": {}},
{"id": "gog:ember", "store": "gog", "title": "Ember Peaks", "art": {}},
{"id": "steam:launcher", "store": "steam", "title": "Steam", "art": {},
"role": "launcher", "icon": "steam"}
]
"""
return (try? JSONDecoder().decode([GameEntry].self, from: Data(json.utf8))) ?? []
}()
/// A plausible-looking 32-byte SHA-256 for the trust card / pin lock glyphs.
static let fingerprint = hostFingerprint(0)
@@ -230,6 +260,19 @@ private struct ShotHome: View {
}
}
// MARK: - Library
/// The library coverflow with the mock shelf the store listing's PICK & PLAY frame. The real
/// `LibraryCoverflowView`, no network: artless entries settle to their deterministic fallback
/// posters, and the entrance's 700 ms backstop has long fired by the time the driver captures.
private struct ShotLibrary: View {
var body: some View {
LibraryCoverflowView(
games: ShotMock.games, artLoader: nil,
onLaunch: { _ in }, onDismiss: {}, controllerActive: false)
}
}
// MARK: - Gamepad-mode console screens (dev-only glass preview)
#if os(iOS) || os(macOS)
@@ -311,6 +354,61 @@ private struct ShotConnect: View {
}
}
}
// MARK: - Controllers (the pads the store listing names)
/// The FEEL THE GAME frame: the controller test panel rendering the two pads the listing talks
/// about. A GCController cannot be constructed, so the panel draws injected `ShotPad`s the
/// DualSense leads with the feedback surface (adaptive-trigger effects, rumble backend, lightbar
/// + player LEDs), the Xbox pad carries the input readout, frozen mid-game.
private struct ShotControllers: View {
var body: some View {
#if os(macOS)
// The panel is a window-modal sheet in the app float it at sheet width over the
// dimmed host grid, the way the other mac sheet shots read.
ZStack {
ShotHome().blur(radius: 24).overlay(Color.black.opacity(0.45))
ControllerTestView(shotPads: Self.pads)
.frame(width: 500, height: 840)
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 12))
.clipShape(RoundedRectangle(cornerRadius: 12))
.shadow(radius: 40, y: 16)
}
#else
// Landscape canvas: one column per pad, so neither story is cut by the short height
// the DualSense feedback surface left, the Xbox live-input readout right.
HStack(spacing: 0) {
ControllerTestView(shotPads: [Self.pads[0]])
ControllerTestView(shotPads: [Self.pads[1]])
}
#endif
}
/// Transport/battery/player ride in `detail` the panel has no dedicated battery row.
/// Each pad shows a different half of the panel: the DualSense skips the input card (the
/// effect grid is the marketing point), the Xbox pad skips rumble and shows the readout.
static let pads: [ControllerTestView.ShotPad] = [
.init(
name: "DualSense Wireless Controller",
detail: "Bluetooth · 85% · Player 1",
isDualSense: true, hasAdaptiveTriggers: true, hasLight: true,
rumbleBackend: "DualSense HID · Bluetooth"),
.init(
name: "Xbox Wireless Controller",
detail: "Bluetooth · 60% · Player 2",
isDualSense: false, hasAdaptiveTriggers: false, hasLight: false,
input: .init(
leftStick: .init(x: -0.31, y: 0.54),
rightStick: .init(x: 0.72, y: -0.16),
leftTrigger: 0.08, rightTrigger: 0.62,
buttons: [
("A", true), ("B", false), ("X", false), ("Y", false),
("LB", false), ("RB", true), ("L3", false), ("R3", false),
("Menu", false), ("Opts", false),
("", false), ("", false), ("", false), ("", false),
])),
]
}
#endif
// MARK: - Edit host (add/edit sheet with the Wake-on-LAN MAC field)
@@ -4,6 +4,11 @@
// physical pad (no host needed), so the rendering paths a session uses can be confirmed
// on-device. Driven by PunktfunkKit's `ControllerTester`, which reuses the real renderers.
//
// Every card renders a plain value model (`ShotPad` / `InputSnapshot`) that the live path samples
// out of the real pad each timeline tick. A GCController cannot be constructed, and the App Store
// screenshot harness needs this panel with pads the capture machine doesn't have ShotScenes
// injects them via `shotPads` (the same seam Android's ControllersScreen grew for its capture).
//
// tvOS is excluded for now (it has no segmented picker / the panel wants a pointer-style
// layout); macOS + iOS/iPadOS cover the validation need.
@@ -14,10 +19,63 @@ import SwiftUI
@MainActor
struct ControllerTestView: View {
/// What one panel section says about a pad, as plain values. The live path flattens the
/// active `DiscoveredController` into one; the screenshot harness hands the panel pads that
/// were never connected. `input`/`rumbleBackend` are the harness's section knobs (nil hides
/// that card) the live path always shows both, fed from the live pad and tester.
struct ShotPad: Identifiable {
let name: String
/// The header's second line. Production shows the GC product category; a shot packs
/// transport/battery/player facts into it (the panel has no dedicated battery row).
let detail: String
let isDualSense: Bool
let hasAdaptiveTriggers: Bool
let hasLight: Bool
var input: InputSnapshot? = nil
var rumbleBackend: String? = nil
var id: String { name }
}
/// One frame of the input readout. The live path samples the real `GCExtendedGamepad` into
/// one of these on every 30 Hz tick; the harness writes a mid-game frame by hand.
struct InputSnapshot {
struct Stick {
var x: Float
var y: Float
var pressed = false
}
struct Touch {
/// Finger position in GC's -1...1 axes; nil = lifted. (GC snaps a lifted finger to
/// exactly (0, 0), so a real (0, 0) contact is indistinguishable anyway.)
var primary: CGPoint?
var secondary: CGPoint?
var clicked = false
}
struct Motion {
var gyro: SIMD3<Double>
var accel: SIMD3<Double>
}
var leftStick: Stick
var rightStick: Stick
var leftTrigger: Float = 0
var rightTrigger: Float = 0
/// Grid order; label pressed.
var buttons: [(String, Bool)]
var touchpad: Touch?
var motion: Motion?
}
@Environment(\.dismiss) private var dismiss
@ObservedObject private var gamepads = GamepadManager.shared
@StateObject private var tester = ControllerTester()
/// Screenshot-harness injection nil (the app) renders the live active pad.
private let shotPads: [ShotPad]?
init(shotPads: [ShotPad]? = nil) {
self.shotPads = shotPads
}
@State private var heavyOn = false
@State private var lightOn = false
@State private var intensity = 0.75
@@ -62,12 +120,12 @@ struct ControllerTestView: View {
Divider()
ScrollView {
VStack(alignment: .leading, spacing: 16) {
if let active = gamepads.active {
header(active)
inputCard
rumbleCard()
triggerCard(active)
extrasCard(active)
if let shotPads {
ForEach(shotPads) { pad in
shotPanel(pad)
}
} else if let active = gamepads.active {
livePanel(active)
} else {
ContentUnavailableView(
"No controller",
@@ -81,9 +139,10 @@ struct ControllerTestView: View {
}
}
.frame(minWidth: 420, minHeight: 540)
.onAppear { tester.target(gamepads.active?.controller) }
.onDisappear { tester.stop() }
.onAppear { if shotPads == nil { tester.target(gamepads.active?.controller) } }
.onDisappear { if shotPads == nil { tester.stop() } }
.onChange(of: gamepads.active?.id) { _, _ in
guard shotPads == nil else { return }
heavyOn = false
lightOn = false
playerLED = -1
@@ -91,16 +150,53 @@ struct ControllerTestView: View {
}
}
// MARK: Panels
@ViewBuilder
private func livePanel(_ active: GamepadManager.DiscoveredController) -> some View {
let pad = Self.describe(active)
header(pad)
liveInputCard
rumbleCard(backend: tester.rumbleBackend, health: tester.rumbleHealth)
triggerCard(pad)
extrasCard(pad)
}
/// An injected pad's cards, in the live panel's order. The adaptive-trigger card is skipped
/// outright for a pad without them the live path's "needs a DualSense" hint is a diagnosis,
/// and a capture has nothing to diagnose.
@ViewBuilder
private func shotPanel(_ pad: ShotPad) -> some View {
header(pad)
if let input = pad.input {
card("Input") { inputReadout(input) }
}
if let backend = pad.rumbleBackend {
rumbleCard(backend: backend, health: nil)
}
if pad.hasAdaptiveTriggers {
triggerCard(pad)
}
extrasCard(pad)
}
/// The live pad, flattened to what the panel renders about it.
private static func describe(_ c: GamepadManager.DiscoveredController) -> ShotPad {
ShotPad(
name: c.name, detail: c.productCategory, isDualSense: c.isDualSense,
hasAdaptiveTriggers: c.hasAdaptiveTriggers, hasLight: c.hasLight)
}
// MARK: Header
private func header(_ c: GamepadManager.DiscoveredController) -> some View {
private func header(_ pad: ShotPad) -> some View {
HStack(spacing: 10) {
Image(systemName: c.isDualSense ? "playstation.logo" : "gamecontroller.fill")
Image(systemName: pad.isDualSense ? "playstation.logo" : "gamecontroller.fill")
.font(.title2)
.foregroundStyle(.secondary)
VStack(alignment: .leading, spacing: 2) {
Text(c.name).font(.geist(17, .semibold, relativeTo: .headline))
Text(c.productCategory).font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
Text(pad.name).font(.geist(17, .semibold, relativeTo: .headline))
Text(pad.detail).font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
Spacer()
}
@@ -108,13 +204,13 @@ struct ControllerTestView: View {
// MARK: Input
private var inputCard: some View {
private var liveInputCard: some View {
card("Input") {
// Poll the live controller at 30 Hz no handlers installed, so nothing else's
// capture is disturbed.
TimelineView(.periodic(from: .now, by: 1.0 / 30.0)) { _ in
if let gp = gamepads.active?.controller.extendedGamepad {
inputReadout(gp, controller: gamepads.active?.controller)
inputReadout(Self.snapshot(gp, controller: gamepads.active?.controller))
} else {
Text("Not an extended gamepad").foregroundStyle(.secondary)
}
@@ -122,40 +218,82 @@ struct ControllerTestView: View {
}
}
/// One readout frame off the live pad.
private static func snapshot(
_ g: GCExtendedGamepad, controller: GCController?
) -> InputSnapshot {
var buttons: [(String, Bool)] = [
("A", g.buttonA.isPressed), ("B", g.buttonB.isPressed),
("X", g.buttonX.isPressed), ("Y", g.buttonY.isPressed),
("LB", g.leftShoulder.isPressed), ("RB", g.rightShoulder.isPressed),
("L3", g.leftThumbstickButton?.isPressed ?? false),
("R3", g.rightThumbstickButton?.isPressed ?? false),
("Menu", g.buttonMenu.isPressed),
("Opts", g.buttonOptions?.isPressed ?? false),
("", g.dpad.up.isPressed), ("", g.dpad.down.isPressed),
("", g.dpad.left.isPressed), ("", g.dpad.right.isPressed),
]
let tp = touchpad(g)
if let tp { buttons.append(("Pad", tp.button.isPressed)) }
return InputSnapshot(
leftStick: .init(
x: g.leftThumbstick.xAxis.value, y: g.leftThumbstick.yAxis.value,
pressed: g.leftThumbstickButton?.isPressed ?? false),
rightStick: .init(
x: g.rightThumbstick.xAxis.value, y: g.rightThumbstick.yAxis.value,
pressed: g.rightThumbstickButton?.isPressed ?? false),
leftTrigger: g.leftTrigger.value, rightTrigger: g.rightTrigger.value,
buttons: buttons,
touchpad: tp.map {
.init(primary: finger($0.primary), secondary: finger($0.secondary),
clicked: $0.button.isPressed)
},
motion: controller?.motion.map { m -> InputSnapshot.Motion in
let a = totalAccel(m)
return .init(
gyro: .init(m.rotationRate.x, m.rotationRate.y, m.rotationRate.z),
accel: .init(a.0, a.1, a.2))
})
}
private static func finger(_ pad: GCControllerDirectionPad) -> CGPoint? {
let x = pad.xAxis.value, y = pad.yAxis.value
// GC snaps a lifted finger to exactly (0, 0).
return (x == 0 && y == 0) ? nil : CGPoint(x: CGFloat(x), y: CGFloat(y))
}
@ViewBuilder
private func inputReadout(_ g: GCExtendedGamepad, controller: GCController?) -> some View {
private func inputReadout(_ s: InputSnapshot) -> some View {
VStack(alignment: .leading, spacing: 14) {
HStack(alignment: .top, spacing: 20) {
stick("L", x: g.leftThumbstick.xAxis.value, y: g.leftThumbstick.yAxis.value,
pressed: g.leftThumbstickButton?.isPressed ?? false)
stick("R", x: g.rightThumbstick.xAxis.value, y: g.rightThumbstick.yAxis.value,
pressed: g.rightThumbstickButton?.isPressed ?? false)
stick("L", s.leftStick)
stick("R", s.rightStick)
VStack(spacing: 8) {
triggerBar("L2", value: g.leftTrigger.value)
triggerBar("R2", value: g.rightTrigger.value)
triggerBar("L2", value: s.leftTrigger)
triggerBar("R2", value: s.rightTrigger)
}
}
buttonGrid(g)
if let tp = Self.touchpad(g) {
buttonGrid(s.buttons)
if let tp = s.touchpad {
touchpadView(tp)
}
if let m = controller?.motion {
if let m = s.motion {
motionReadout(m)
}
}
}
private func stick(_ label: String, x: Float, y: Float, pressed: Bool) -> some View {
private func stick(_ label: String, _ s: InputSnapshot.Stick) -> some View {
VStack(spacing: 4) {
ZStack {
Circle().stroke(Color.secondary.opacity(0.3))
Circle()
.fill(pressed ? Color.accentColor : Color.secondary)
.fill(s.pressed ? Color.accentColor : Color.secondary)
.frame(width: 12, height: 12)
.offset(x: CGFloat(x) * 22, y: CGFloat(-y) * 22) // GC y is +up
.offset(x: CGFloat(s.x) * 22, y: CGFloat(-s.y) * 22) // GC y is +up
}
.frame(width: 56, height: 56)
Text("\(label) \(sgn(x)),\(sgn(y))").font(.caption2.monospaced()).foregroundStyle(.secondary)
Text("\(label) \(sgn(s.x)),\(sgn(s.y))").font(.caption2.monospaced()).foregroundStyle(.secondary)
}
}
@@ -175,20 +313,8 @@ struct ControllerTestView: View {
.frame(width: 150)
}
private func buttonGrid(_ g: GCExtendedGamepad) -> some View {
var items: [(String, Bool)] = [
("A", g.buttonA.isPressed), ("B", g.buttonB.isPressed),
("X", g.buttonX.isPressed), ("Y", g.buttonY.isPressed),
("LB", g.leftShoulder.isPressed), ("RB", g.rightShoulder.isPressed),
("L3", g.leftThumbstickButton?.isPressed ?? false),
("R3", g.rightThumbstickButton?.isPressed ?? false),
("Menu", g.buttonMenu.isPressed),
("Opts", g.buttonOptions?.isPressed ?? false),
("", g.dpad.up.isPressed), ("", g.dpad.down.isPressed),
("", g.dpad.left.isPressed), ("", g.dpad.right.isPressed),
]
if let tp = Self.touchpad(g) { items.append(("Pad", tp.button.isPressed)) }
return LazyVGrid(
private func buttonGrid(_ items: [(String, Bool)]) -> some View {
LazyVGrid(
columns: Array(repeating: GridItem(.flexible(), spacing: 6), count: 5), spacing: 6
) {
ForEach(items.indices, id: \.self) { i in
@@ -203,12 +329,9 @@ struct ControllerTestView: View {
}
}
private func touchpadView(
_ tp: (primary: GCControllerDirectionPad, secondary: GCControllerDirectionPad,
button: GCControllerButtonInput)
) -> some View {
private func touchpadView(_ tp: InputSnapshot.Touch) -> some View {
VStack(alignment: .leading, spacing: 4) {
Text("Touchpad\(tp.button.isPressed ? " — click" : "")")
Text("Touchpad\(tp.clicked ? " — click" : "")")
.font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
ZStack {
RoundedRectangle(cornerRadius: 8).stroke(Color.secondary.opacity(0.3))
@@ -219,29 +342,25 @@ struct ControllerTestView: View {
}
}
private func fingerDot(_ pad: GCControllerDirectionPad, color: Color) -> some View {
let x = pad.xAxis.value, y = pad.yAxis.value
let active = !(x == 0 && y == 0) // GC snaps a lifted finger to exactly (0, 0)
return Circle().fill(color).frame(width: 10, height: 10)
.offset(x: CGFloat(x) * 71, y: CGFloat(-y) * 33)
.opacity(active ? 1 : 0)
private func fingerDot(_ p: CGPoint?, color: Color) -> some View {
Circle().fill(color).frame(width: 10, height: 10)
.offset(x: (p?.x ?? 0) * 71, y: -(p?.y ?? 0) * 33)
.opacity(p == nil ? 0 : 1)
}
private func motionReadout(_ m: GCMotion) -> some View {
let a = Self.totalAccel(m)
return VStack(alignment: .leading, spacing: 2) {
private func motionReadout(_ m: InputSnapshot.Motion) -> some View {
VStack(alignment: .leading, spacing: 2) {
Text("Motion").font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
Text(String(format: "gyro %+.2f %+.2f %+.2f",
m.rotationRate.x, m.rotationRate.y, m.rotationRate.z))
Text(String(format: "gyro %+.2f %+.2f %+.2f", m.gyro.x, m.gyro.y, m.gyro.z))
.font(.caption2.monospaced())
Text(String(format: "accel %+.2f %+.2f %+.2f", a.0, a.1, a.2))
Text(String(format: "accel %+.2f %+.2f %+.2f", m.accel.x, m.accel.y, m.accel.z))
.font(.caption2.monospaced())
}
}
// MARK: Rumble
private func rumbleCard() -> some View {
private func rumbleCard(backend: String, health: String?) -> some View {
card("Rumble") {
VStack(alignment: .leading, spacing: 12) {
Picker("Strength", selection: $intensity) {
@@ -253,9 +372,9 @@ struct ControllerTestView: View {
.pickerStyle(.segmented)
Toggle("Heavy motor (left)", isOn: $heavyOn)
Toggle("Light motor (right)", isOn: $lightOn)
Label("Backend: \(tester.rumbleBackend)", systemImage: "waveform")
Label("Backend: \(backend)", systemImage: "waveform")
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
if let problem = tester.rumbleHealth {
if let problem = health {
Label(problem, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.orange)
}
@@ -276,9 +395,9 @@ struct ControllerTestView: View {
// MARK: Adaptive triggers
private func triggerCard(_ c: GamepadManager.DiscoveredController) -> some View {
private func triggerCard(_ pad: ShotPad) -> some View {
card("Adaptive triggers") {
if c.hasAdaptiveTriggers {
if pad.hasAdaptiveTriggers {
VStack(alignment: .leading, spacing: 12) {
Picker("Apply to", selection: $triggerTarget) {
ForEach(TriggerTarget.allCases) { Text($0.rawValue).tag($0) }
@@ -315,8 +434,8 @@ struct ControllerTestView: View {
// MARK: Lightbar + player LED
@ViewBuilder
private func extrasCard(_ c: GamepadManager.DiscoveredController) -> some View {
if c.hasLight {
private func extrasCard(_ pad: ShotPad) -> some View {
if pad.hasLight {
card("Lightbar & player LED") {
VStack(alignment: .leading, spacing: 12) {
HStack(spacing: 12) {
@@ -114,6 +114,10 @@ enum SettingsFields {
.init(name: "invert_scroll", key: DefaultsKey.invertScroll,
overlay: \.invertScroll, effective: \.invertScroll)
}
static var inhibitShortcuts: SettingsField<Bool> {
.init(name: "inhibit_shortcuts", key: DefaultsKey.inhibitShortcuts,
overlay: \.inhibitShortcuts, effective: \.inhibitShortcuts)
}
static var modifierLayout: SettingsField<String> {
.init(name: "modifier_layout", key: DefaultsKey.modifierLayout,
overlay: \.modifierLayout, effective: \.modifierLayout)
@@ -205,6 +209,7 @@ extension SettingsView {
#endif
#if os(macOS)
base.mouseMode = mouseMode
base.inhibitShortcuts = inhibitShortcuts
base.vsync = vsync
base.windowedSafePresent = windowedSafePresent
#endif
@@ -515,6 +515,9 @@ extension SettingsView {
Text("Desktop (absolute)").tag(MouseInputMode.desktop.rawValue)
}
}
described(inhibitShortcutsDescription, field: "inhibit_shortcuts") {
Toggle("Capture system shortcuts", isOn: scoped(SettingsFields.inhibitShortcuts))
}
#endif
described(
(ModifierLayout(rawValue: effective.modifierLayout) ?? .mac).detail,
@@ -534,6 +537,19 @@ extension SettingsView {
}
#if os(macOS)
/// Dynamic like the captions above, because the setting genuinely has no effect under the
/// desktop mouse model (system chords stay local there on every client) and a toggle that
/// silently does nothing should say so instead of leaving the user to find out.
private var inhibitShortcutsDescription: String {
if (MouseInputMode(rawValue: effective.mouseMode) ?? .capture) == .desktop {
return "⌘ shortcuts stay on this Mac under the desktop mouse model. Switch Mouse "
+ "input to Capture to send them to the host."
}
return "Sends ⌘ shortcuts to the host while input is captured, so ⌘Q and friends reach "
+ "the remote desktop instead of this app. ⌘⎋ always stays local — it is what "
+ "releases capture."
}
/// The SELECTED mouse model explained dynamic, like the touch-mode caption.
private var mouseModeDescription: String {
switch MouseInputMode(rawValue: effective.mouseMode) ?? .capture {
@@ -115,6 +115,10 @@ struct SettingsView: View {
#endif
#if os(macOS)
@AppStorage(DefaultsKey.mouseMode) var mouseMode = MouseInputMode.capture.rawValue
/// Cross-client `inhibit_shortcuts` here, the -chord passthrough (Q & co. reach the host
/// instead of the app menu while captured). macOS-only: it is the one platform whose window
/// system hands a plain app no keyboard grab, so the client has to claim the chords itself.
@AppStorage(DefaultsKey.inhibitShortcuts) var inhibitShortcuts = true
@AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0
@@ -86,6 +86,21 @@ public final class InputCapture {
/// its Esc suppression need it in both states).
private var cmdKeysDown: Set<UInt32> = []
#if os(macOS)
/// Windows VKs the -chord passthrough sent DOWN (see the keyDown monitor). macOS stops
/// delivering keyUp for ordinary keys while Command is held, so the release half of Q/W/
/// cannot be relied on to arrive through the responder chain at all: these are flushed when
/// the last comes up (`flushCommandChord`), which is what stands between the host and a
/// key held down for the rest of the session.
private var commandChordVKs: Set<UInt32> = []
/// Mirrors StreamLayerView's live mouse model M flips it mid-session, so it can't be
/// read from the settings. The -chord passthrough stays off under the desktop model, matching
/// what the SDL clients' keyboard grab does: a remote desktop is something you Tab away from,
/// not into.
public var desktopMouse = false
#endif
#if !os(macOS)
/// The key currently auto-repeating, and the timer driving it. iOS/tvOS only see
/// `startAutoRepeat`. Main-queue only, like every other field here.
@@ -244,19 +259,27 @@ public final class InputCapture {
) { [weak self] _ in
self?.releaseAll()
})
// the capture toggle is detected here so it works in both states. ONLY
// that one combo is intercepted: swallowing keys wholesale at the monitor level
// risks starving GC's own delivery, so the no-beep behavior lives in
// StreamLayerView (first responder consumes keyDown/keyUp while captured).
// (On iOS there is no NSEvent monitor the GC key handler detects the combo.)
// This monitor is the FIRST thing in the app to see a key: AppKit calls it before
// `sendEvent:`, so before any menu key equivalent and before StreamLayerView's keyDown.
// Returning nil discards the event outright which cuts BOTH of those off, and on macOS
// the second one is the host's only key path (the GCKeyboard send is iOS-only; see
// `attach(keyboard:)`). So the rule here is: anything swallowed must either be handled
// client-side or forwarded to the host from inside this block, because nothing downstream
// will get a second chance at it.
//
// (capture toggle) and M (mouse model) are client-side in BOTH states; Q/D/S/A
// and F are client-side only while forwarding (released, the events pass through and the
// menu's identical key equivalents handle them). Every OTHER chord is the HOST's while
// captured see `forwardsCommandChord`. (On iOS there is no NSEvent monitor the GC key
// handler detects the combos.)
#if os(macOS)
keyEventMonitor = NSEvent.addLocalMonitorForEvents(
matching: [.keyDown]
) { [weak self] event in
guard let self else { return event }
let flags = event.modifierFlags.intersection(.deviceIndependentFlagsMask)
let flags = Self.chordFlags(event)
if event.keyCode == 53 /* Esc */, flags == .command {
self.suppressedVK = 0x1B // the same physical Esc is en route via GC
self.suppressedVK = 0x1B // VK_ESC its keyUp still reaches the responder chain
self.onToggleCapture?()
return nil
}
@@ -266,7 +289,7 @@ public final class InputCapture {
// (latched like 's Esc) so it doesn't type into the host, and swallow the
// event so it doesn't beep.
if event.keyCode == 46 /* M */, flags == [.control, .option, .shift] {
self.suppressedVK = 0x4D // VK_M the same physical M is en route via GC
self.suppressedVK = 0x4D // VK_M its keyUp still reaches the responder chain
self.onToggleMouseMode?()
return nil
}
@@ -304,10 +327,34 @@ public final class InputCapture {
// captured stream view swallows the menu's identical equivalent); the F is latched so its
// keyUp can't type into the host. keyCode 3 = kVK_ANSI_F (layout-independent).
if self.forwarding, flags == [.control, .command], event.keyCode == 3 /* F */ {
self.suppressedVK = 0x46 // VK_F the same physical F is en route via GC
self.suppressedVK = 0x46 // VK_F its keyUp still reaches the responder chain
self.onToggleFullscreen?()
return nil
}
// Every OTHER chord belongs to the HOST while captured the cross-client "capture
// system shortcuts" setting, which the Apple client had no answer to because SDL's
// keyboard grab is what implements it everywhere else. Without this the app menu's key
// equivalents fire first, so Q quits the client instead of reaching the compositor as
// Super+Q one of the most-bound chords on a Linux desktop, and the reported break.
//
// It has to SEND from here: returning nil is what keeps the menu out, and it takes
// StreamLayerView's keyDown the host's only key path on macOS out with it.
// Chords with no host VK are swallowed but not sent: doing nothing beats a menu
// opening under a captured stream. The itself needs no handling modifiers arrive
// as flagsChanged, which this monitor never sees, so it was already forwarded as
// VK_LWIN/VK_RWIN (or Alt, under the Windows modifier layout) when it went down.
//
// The two cheap conditions are repeated in front of the call on purpose: off-session,
// `SessionSettings.current` re-reads the whole defaults suite, and this monitor sees
// every keystroke the app receives including the ones typed into the host list.
if self.forwarding, flags.contains(.command), Self.forwardsCommandChord(
keyCode: event.keyCode, flags: flags, forwarding: self.forwarding,
inhibitShortcuts: SessionSettings.current.inhibitShortcuts,
desktopMouse: self.desktopMouse
) {
if let vk = Self.keyCodeToVK[event.keyCode] { self.sendCommandChordKey(vk) }
return nil
}
return event
}
#endif
@@ -358,6 +405,9 @@ public final class InputCapture {
cmdKeysDown.removeAll()
chordModifiersDown.removeAll()
suppressedVK = nil
#if os(macOS)
commandChordVKs.removeAll() // their releases are in `pressedVKs`, flushed just below
#endif
for vk in pressedVKs {
emitKey(vk, down: false)
}
@@ -522,7 +572,15 @@ public final class InputCapture {
// Keep cmdKeysDown in step (the toggle + Esc suppression read it); sendKey
// adds the VK to pressedVKs so releaseAll/blur flushes a held modifier cleanly.
if vk == 0x5B || vk == 0x5C {
if down { cmdKeysDown.insert(vk) } else { cmdKeysDown.remove(vk) }
if down {
cmdKeysDown.insert(vk)
} else {
cmdKeysDown.remove(vk)
// Last up: release the chord keys whose own keyUp macOS never delivered. BEFORE
// the 's own release goes out, so the host never sees the letter outlive the
// modifier it was pressed with.
if cmdKeysDown.isEmpty { flushCommandChord() }
}
}
sendKey(vk, down: down)
}
@@ -552,6 +610,68 @@ public final class InputCapture {
}
return (mod.vk, down)
}
// MARK: - chord passthrough
/// The four modifiers a client chord is ever spelled with, isolated from the incidental bits
/// `deviceIndependentFlagsMask` also carries: Caps Lock, and the `.function`/`.numericPad`
/// pair every arrow and F-key sets. Equality against the raw masked flags meant a chord
/// stopped being recognized the moment Caps Lock was on and Q, both escape hatches,
/// included. That was survivable while the monitor claimed six chords; it is not, now that it
/// swallows every chord there is.
static let chordFlagMask: NSEvent.ModifierFlags = [.command, .control, .option, .shift]
/// One event's chord modifiers (see `chordFlagMask`).
static func chordFlags(_ event: NSEvent) -> NSEvent.ModifierFlags {
event.modifierFlags.intersection(chordFlagMask)
}
/// The chords the CLIENT keeps while captured, which is to say: the way out. releases
/// the mouse/keyboard and F leaves fullscreen hand either of those to the host and a
/// captured stream becomes a room with no door. (Q/D/S/A carry no and never reach here.)
static func isClientReservedChord(keyCode: UInt16, flags: NSEvent.ModifierFlags) -> Bool {
if keyCode == 53, flags == .command { return true } // capture toggle
if keyCode == 3, flags == [.control, .command] { return true } // F fullscreen
return false
}
/// Does this keyDown get taken off AppKit and forwarded to the host instead? Only while input
/// is actually captured, only with the cross-client `inhibit_shortcuts` on, and never under the
/// desktop mouse model (where the chords stay local by design) and never for the client's own
/// reserved chords, whatever the setting says.
static func forwardsCommandChord(
keyCode: UInt16, flags: NSEvent.ModifierFlags,
forwarding: Bool, inhibitShortcuts: Bool, desktopMouse: Bool
) -> Bool {
guard forwarding, inhibitShortcuts, !desktopMouse else { return false }
guard flags.contains(.command) else { return false }
return !isClientReservedChord(keyCode: keyCode, flags: flags)
}
/// Forward one key of a chord the monitor just took off AppKit, remembering it so its
/// release can be synthesized (see `commandChordVKs`).
private func sendCommandChordKey(_ vk: UInt32) {
commandChordVKs.insert(vk)
sendKey(vk, down: true)
}
/// Release whatever the -chord passthrough sent down and is still held called when the last
/// physical comes up. A keyUp that DID arrive has already taken its VK out of `pressedVKs`,
/// so this only fires for the ones macOS swallowed.
private func flushCommandChord() {
// Same cause, different victim: a one-shot latch whose key-up never arrived goes on to eat
// the NEXT press of that key (F's F, 's Esc). Once is up, a pending latch is stale.
suppressedVK = nil
guard !commandChordVKs.isEmpty else { return }
for vk in commandChordVKs where pressedVKs.contains(vk) {
pressedVKs.remove(vk)
emitKey(vk, down: false)
if inputDebug {
inputLog.debug("key \(vk, privacy: .public) up SYNTHESIZED (⌘ chord release)")
}
}
commandChordVKs.removeAll()
}
#endif
private func attach(mouse: GCMouse) {
@@ -410,8 +410,9 @@ public final class StreamLayerView: NSView {
// keycode) Windows VK and forward via InputCapture.sendKey, then CONSUME (return without
// super) to stop the responder chain's "unhandled keyDown" beep. Keys with no VK mapping
// are still consumed while captured so they don't beep either. The toggle's Esc is
// swallowed upstream by InputCapture's keyDown monitor (suppressedVK), so it never gets
// here as a send; -combos still arrive via performKeyEquivalent and stay functional (D).
// swallowed upstream by InputCapture's keyDown monitor (suppressedVK), so it never gets here
// as a send and so are combos generally while captured, which that monitor forwards to the
// host itself (`forwardsCommandChord`) rather than letting a menu key equivalent claim them.
// Modifier keys never fire keyDown/keyUp they come through flagsChanged below.
public override var acceptsFirstResponder: Bool { true }
// A click after the app was inactive (Cmd-Tab away and back) must reach mouseDown so the
@@ -570,6 +571,9 @@ public final class StreamLayerView: NSView {
let wasCaptured = captured
if wasCaptured { releaseCapture() }
desktopMouse = on
// The -chord passthrough is off under the desktop model (system chords stay local there,
// as on every other client) and the model moves live, so the capture is told, not asked.
inputCapture?.desktopMouse = on
if wasCaptured { engageCapture(fromClick: false) }
window?.invalidateCursorRects(for: self)
if on, let p = reappearAt, let sp = cgScreenPoint(forHostX: p.x, p.y) {
@@ -917,6 +921,7 @@ public final class StreamLayerView: NSView {
) ?? .capture
let absOK = connection.resolvedCompositor != .gamescope
desktopMouse = mode == .desktop && absOK
capture.desktopMouse = desktopMouse
if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
}
@@ -157,6 +157,16 @@ public enum DefaultsKey {
/// Read live at the wire boundary by `InputCapture`. Control/Shift never move (same position on
/// both keyboards).
public static let modifierLayout = "punktfunk.modifierLayout"
/// Send system chords to the host while input is captured the cross-client
/// `inhibit_shortcuts`, ON by default. On the SDL clients it is SDL's keyboard grab (Alt+Tab,
/// the Windows key); macOS has no such grab from a plain app, so `InputCapture`'s keyDown
/// monitor implements it by taking every chord off AppKit before a menu key equivalent can
/// fire and forwarding it instead which is what makes Q reach the host's compositor rather
/// than quitting the client. Off keeps the chords local (the second-screen/work profile).
/// The client's own reserved chords (, F, ) are never forwarded either way, and as
/// on the SDL clients the setting has no effect under the `desktop` mouse model, which is
/// something you Tab *away* from. macOS-only today; nothing reads it on iOS/tvOS.
public static let inhibitShortcuts = "punktfunk.inhibitShortcuts"
/// iPad: capture the mouse/trackpad pointer (pointer lock relative movement) for games,
/// rather than forwarding an absolute cursor position. On by default. Only meaningful on iPad
/// with a hardware mouse/trackpad; the system grants the lock only to a full-screen, frontmost
@@ -33,6 +33,9 @@ public struct EffectiveSettings: Equatable, Sendable {
public var touchMode = "trackpad"
public var mouseMode = "capture"
public var invertScroll = false
/// Cross-client `inhibit_shortcuts` (default on): system chords reach the host while input is
/// captured. See `DefaultsKey.inhibitShortcuts` on macOS this is the -chord passthrough.
public var inhibitShortcuts = true
public var gamepadType = 0
public var gamepadForwarding = true
/// Cross-client `system_buttons`: "auto" | "forward" | "local".
@@ -97,6 +100,7 @@ public struct EffectiveSettings: Equatable, Sendable {
touchMode = str(DefaultsKey.touchMode, touchMode)
mouseMode = str(DefaultsKey.mouseMode, mouseMode)
invertScroll = bool(DefaultsKey.invertScroll, invertScroll)
inhibitShortcuts = bool(DefaultsKey.inhibitShortcuts, inhibitShortcuts)
gamepadType = int(DefaultsKey.gamepadType, gamepadType)
gamepadForwarding = bool(DefaultsKey.gamepadForwarding, gamepadForwarding)
systemButtons = str(DefaultsKey.systemButtons, systemButtons)
@@ -177,6 +181,7 @@ public struct EffectiveSettings: Equatable, Sendable {
if let v = overlay.touchMode { s.touchMode = v }
if let v = overlay.mouseMode { s.mouseMode = v }
if let v = overlay.invertScroll { s.invertScroll = v }
if let v = overlay.inhibitShortcuts { s.inhibitShortcuts = v }
if let v = overlay.gamepadType { s.gamepadType = v }
if let v = overlay.gamepadForwarding { s.gamepadForwarding = v }
if let v = overlay.systemButtons { s.systemButtons = v }
@@ -109,6 +109,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
public var touchMode: String?
public var mouseMode: String?
public var invertScroll: Bool?
public var inhibitShortcuts: Bool?
public var gamepadType: Int?
public var gamepadForwarding: Bool?
public var systemButtons: String?
@@ -153,6 +154,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
case touchMode = "touch_mode"
case mouseMode = "mouse_mode"
case invertScroll = "invert_scroll"
case inhibitShortcuts = "inhibit_shortcuts"
case gamepadType = "gamepad"
case gamepadForwarding = "gamepad_forwarding"
case systemButtons = "system_buttons"
@@ -189,6 +191,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
touchMode = str(.touchMode)
mouseMode = str(.mouseMode)
invertScroll = bool(.invertScroll)
inhibitShortcuts = bool(.inhibitShortcuts)
gamepadType = int(.gamepadType)
gamepadForwarding = bool(.gamepadForwarding)
systemButtons = str(.systemButtons)
@@ -227,6 +230,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
try c.encodeIfPresent(touchMode, forKey: AnyKey(Key.touchMode.rawValue))
try c.encodeIfPresent(mouseMode, forKey: AnyKey(Key.mouseMode.rawValue))
try c.encodeIfPresent(invertScroll, forKey: AnyKey(Key.invertScroll.rawValue))
try c.encodeIfPresent(inhibitShortcuts, forKey: AnyKey(Key.inhibitShortcuts.rawValue))
try c.encodeIfPresent(gamepadType, forKey: AnyKey(Key.gamepadType.rawValue))
try c.encodeIfPresent(
gamepadForwarding, forKey: AnyKey(Key.gamepadForwarding.rawValue))
@@ -283,6 +287,7 @@ public enum OverlayField {
case "touch_mode": overlay.touchMode = nil
case "mouse_mode": overlay.mouseMode = nil
case "invert_scroll": overlay.invertScroll = nil
case "inhibit_shortcuts": overlay.inhibitShortcuts = nil
case "gamepad": overlay.gamepadType = nil
case "gamepad_forwarding": overlay.gamepadForwarding = nil
case "system_buttons": overlay.systemButtons = nil
@@ -321,6 +326,7 @@ public enum OverlayField {
case "touch_mode": return o.touchMode != nil
case "mouse_mode": return o.mouseMode != nil
case "invert_scroll": return o.invertScroll != nil
case "inhibit_shortcuts": return o.inhibitShortcuts != nil
case "gamepad": return o.gamepadType != nil
case "gamepad_forwarding": return o.gamepadForwarding != nil
case "system_buttons": return o.systemButtons != nil
@@ -0,0 +1,116 @@
#if os(macOS)
import AppKit
import XCTest
@testable import PunktfunkKit
/// Pins the macOS -chord passthrough the rule deciding which keyDowns `InputCapture`'s local
/// monitor takes off AppKit and forwards to the host instead of letting a menu key equivalent
/// claim them. Two things are worth a test rather than a comment:
///
/// * Q reaching the host at all. That is the whole point it is the compositor chord on
/// Hyprland/KDE/GNOME, and it used to quit the client.
/// * and F NOT reaching it, under every combination. They are the way out of a captured
/// stream; forward either and the user is locked in.
final class CommandChordTests: XCTestCase {
// kVK_ANSI_* physical positions, layout-independent (the same constants the monitor uses).
private let q: UInt16 = 12, w: UInt16 = 13, h: UInt16 = 4, m: UInt16 = 46
private let f: UInt16 = 3, esc: UInt16 = 53, leftArrow: UInt16 = 123
/// Captured, setting on, capture mouse model the shipping default.
private func forwards(
_ keyCode: UInt16, _ flags: NSEvent.ModifierFlags,
forwarding: Bool = true, inhibit: Bool = true, desktop: Bool = false
) -> Bool {
InputCapture.forwardsCommandChord(
keyCode: keyCode, flags: flags, forwarding: forwarding,
inhibitShortcuts: inhibit, desktopMouse: desktop)
}
func testCommandChordsGoToTheHostWhileCaptured() {
XCTAssertTrue(forwards(q, .command)) // Q the reported break
XCTAssertTrue(forwards(w, .command))
XCTAssertTrue(forwards(h, .command))
XCTAssertTrue(forwards(m, .command))
XCTAssertTrue(forwards(q, [.command, .shift])) // Q
XCTAssertTrue(forwards(m, [.command, .control, .option, .shift]))
}
func testTheEscapeHatchesAreNeverForwarded() {
// releases capture, F leaves fullscreen. Neither may ever reach the host.
XCTAssertFalse(forwards(esc, .command))
XCTAssertFalse(forwards(f, [.control, .command]))
XCTAssertTrue(InputCapture.isClientReservedChord(keyCode: esc, flags: .command))
XCTAssertTrue(
InputCapture.isClientReservedChord(keyCode: f, flags: [.control, .command]))
}
/// The reservation is exact: it is and F specifically, not "anything with Esc or F in
/// it". and F are the host's like any other chord.
func testNeighbouringChordsAreNotReserved() {
XCTAssertTrue(forwards(esc, [.command, .shift]))
XCTAssertTrue(forwards(f, .command))
XCTAssertFalse(InputCapture.isClientReservedChord(keyCode: f, flags: .command))
}
func testNothingWithoutCommandIsClaimedHere() {
// The family and bare keys reach the monitor's earlier blocks / the responder chain.
XCTAssertFalse(forwards(q, [.control, .option, .shift]))
XCTAssertFalse(forwards(q, []))
XCTAssertFalse(forwards(esc, []))
}
func testReleasedCaptureLeavesTheMenuAlone() {
// Not forwarding = the user is in the local UI: Q must quit the app, W close the window.
XCTAssertFalse(forwards(q, .command, forwarding: false))
XCTAssertFalse(forwards(w, .command, forwarding: false))
}
func testTheCrossClientSettingTurnsItOff() {
XCTAssertFalse(forwards(q, .command, inhibit: false))
}
func testTheDesktopMouseModelKeepsChordsLocal() {
// Matches the SDL clients' keyboard grab: a remote desktop is something you Tab away from.
XCTAssertFalse(forwards(q, .command, desktop: true))
XCTAssertFalse(forwards(q, .command, inhibit: true, desktop: true))
}
/// `deviceIndependentFlagsMask` also carries Caps Lock and the `.function`/`.numericPad` bits
/// every arrow key sets, so comparing it for equality made chords stop being recognized in
/// exactly the states a user does not connect to their keyboard: Caps Lock on, or the chord
/// spelled with an arrow. `chordFlags` isolates the four real modifiers.
func testCapsLockAndArrowBitsDoNotChangeAChord() throws {
let capsQ = try XCTUnwrap(keyEvent(q, [.command, .capsLock]))
XCTAssertEqual(InputCapture.chordFlags(capsQ), .command)
XCTAssertTrue(forwards(q, InputCapture.chordFlags(capsQ)))
// with Caps Lock on is still the escape hatch, not a chord for the host.
let capsEsc = try XCTUnwrap(keyEvent(esc, [.command, .capsLock]))
XCTAssertEqual(InputCapture.chordFlags(capsEsc), .command)
XCTAssertFalse(forwards(esc, InputCapture.chordFlags(capsEsc)))
// arrows set .function|.numericPad, which say nothing about the chord.
let cmdLeft = try XCTUnwrap(keyEvent(leftArrow, [.command, .function, .numericPad]))
XCTAssertEqual(InputCapture.chordFlags(cmdLeft), .command)
XCTAssertTrue(forwards(leftArrow, InputCapture.chordFlags(cmdLeft)))
}
/// A forwarded chord is only useful if the key has a host VK the monitor swallows either
/// way, so an unmapped one would silently do nothing. Spot-check the common letters.
func testTheCommonChordKeysMapToHostVKs() {
XCTAssertEqual(InputCapture.keyCodeToVK[q], 0x51) // VK 'Q'
XCTAssertEqual(InputCapture.keyCodeToVK[w], 0x57) // VK 'W'
XCTAssertEqual(InputCapture.keyCodeToVK[h], 0x48) // VK 'H'
XCTAssertEqual(InputCapture.keyCodeToVK[m], 0x4D) // VK 'M'
XCTAssertEqual(InputCapture.keyCodeToVK[leftArrow], 0x25) // VK_LEFT
}
private func keyEvent(_ keyCode: UInt16, _ flags: NSEvent.ModifierFlags) -> NSEvent? {
NSEvent.keyEvent(
with: .keyDown, location: .zero, modifierFlags: flags, timestamp: 0,
windowNumber: 0, context: nil, characters: "", charactersIgnoringModifiers: "",
isARepeat: false, keyCode: keyCode)
}
}
#endif
+16 -4
View File
@@ -45,7 +45,7 @@ BUNDLE_ID="io.unom.punktfunk"
# The App Store set, in listing order — the first three are what most people ever see, so they are
# the stream itself, the machines it found, and the couch/controller mode. Everything else in
# ShotScenes.all is a dev scene; capture those with `SCENES="06-gamepad-home 10-edithost" ...`.
SCENES=(${SCENES:-01-stream 02-hosts 06-gamepad-home 09e-waking-modal 05-settings 03-pair})
SCENES=(${SCENES:-01-stream 02-hosts 11-library 12-controllers 06-gamepad-home 09e-waking-modal 05-settings 03-pair})
SETTLE="${SETTLE:-4}" # seconds to let a scene lay out before capturing
mkdir -p "$OUT"
@@ -63,9 +63,13 @@ require_xcode() {
# ---------------------------------------------------------------------------- macOS
shoot_macos() {
log "macOS — building (swift build -c release)…"
swift build -c release >/dev/null
local bin=".build/release/PunktfunkClient"
# DEBUG build, deliberately: the whole shot harness lives behind `#if DEBUG`
# (ScreenshotHost/ScreenshotScenes), so a release binary launches as the NORMAL app, never
# prints PF_SHOT_WINDOW, and every scene "never reported a window". Debug renders the same
# pixels — SwiftUI has no release-only visuals.
log "macOS — building (swift build)…"
swift build >/dev/null
local bin=".build/debug/PunktfunkClient"
[ -x "$bin" ] || die "build produced no $bin"
for scene in "${SCENES[@]}"; do
@@ -142,6 +146,14 @@ shoot_sim() {
# incremental build instead of cold-building into a throwaway tmpdir — CI pins this
# (apple.yml); local runs keep the self-cleaning mktemp default.
local dd; dd="${PF_SHOT_DERIVED_DATA:-$(mktemp -d)}"; mkdir -p "$dd"
# tvOS-SIMULATOR trap (Xcode 26.6 and the 27 beta, local only so far): the build planner
# schedules the SwiftPM MACRO plugin targets that swiftui-navigation-transitions pulls in
# (OnceMacro/SwizzlingMacro/AssociationMacro) for the *tvOS* triple and never plans their
# swift-syntax dependencies at all — "unable to resolve module dependency: 'SwiftSyntax'".
# Device archives and iOS builds don't hit it (only the tvOS target links that package), and
# prebuilt-vs-source swift-syntax makes no difference. Until Xcode fixes the planner, the
# workaround is temporarily unlinking SwiftUINavigationTransitions from the tvOS target
# (HomeView's use is canImport-guarded — the push transition degrades to the crossfade).
xcodebuild -project Punktfunk.xcodeproj -scheme "$scheme" -configuration Debug \
-sdk "$sdk" -destination "id=$udid" -derivedDataPath "$dd" \
CODE_SIGNING_ALLOWED=NO build >/dev/null \
+10 -2
View File
@@ -224,8 +224,8 @@ regular pad). Automatic arms it only where the raw guide press can't reach the h
Gaming Mode, iPhone/iPad, Apple TV — because the gesture has a cost: a Select *tap* arrives a beat
late, and a game that expects a *held* Select would trigger it. Set **On** or **Off** to overrule.
**Capture system shortcuts** — *default: on.* Offered by the Linux and Windows apps and the console
home; Windows spells the row out as *Capture system shortcuts (Alt+Tab, Win, …)*. On a Deck it
**Capture system shortcuts** — *default: on.* Offered by the Linux, Windows and macOS apps and the
console home; Windows spells the row out as *Capture system shortcuts (Alt+Tab, Win, …)*. On a Deck it
matters only for a keyboard you attached yourself, for the reason the paragraph below gives: Gaming
Mode is gamescope, which has nothing to hold back. On, Alt+Tab and the Windows key
(Super on Linux) reach the host while the stream has input captured. Off, they act on this machine
@@ -234,6 +234,14 @@ back the moment you release capture with **Ctrl+Alt+Shift+Q**, the window loses
ends, and [Desktop mouse mode](/docs/input#mouse-modes) never takes them at all. Leaving this on does
mean **Ctrl+Alt+Shift+Q is your way out** of a captured stream, since Alt+Tab no longer is.
On macOS the chords in question are the **⌘** ones — ⌘Q above all, which reaches the host as Super+Q,
one of the most-bound chords on a Linux desktop. On, ⌘Q, ⌘W, ⌘H and the rest go to the host instead
of this app's menu bar while input is captured. Off, they act on the Mac as usual, which means ⌘Q
quits Punktfunk mid-stream. **⌘⎋ always stays local whichever way the toggle is set** — it is what
releases capture, as is ⌃⌥⇧Q, and ⌃⌘F keeps working on the window. A few chords never reach the host
either way, because macOS claims them before any app can see them: ⌘Tab, ⌘Space, and the Mission
Control keys.
On Linux this needs a compositor that supports keyboard-shortcuts-inhibit — KDE Plasma, GNOME and
the wlroots compositors all do, and X11 sessions grab the keyboard directly. Under
[gamescope](/docs/gamescope) there is nothing to inhibit: it hands the session everything already.
+6 -1
View File
@@ -68,6 +68,10 @@ to one readable line.
**⌃⌥⇧Q / M / D / S** — but not the microphone mute. **⌘⎋** also toggles capture,
**⌃⌘F** toggles fullscreen, and **⌃⌥⇧C** starts or stops [clipboard sharing](/docs/clipboard). The
**Stream** menu lists them all except the mouse-mode combo, which works but has no menu item.
Every *other* ⌘ chord goes to the host while input is captured — ⌘Q reaches the host's compositor
rather than quitting the app — unless you turn **Capture system shortcuts** off in
[client settings](/docs/client-settings#input). ⌘⎋ and ⌃⌘F are held back either way, so there is
always a way out.
- **iPhone and iPad** with a hardware keyboard: **⌃⌥⇧Q** releases input while it is captured, and
**⌘⎋** toggles capture in either direction. **⌃⌥⇧D** (disconnect) and **⌃⌥⇧S** (stats) come from
the app's Stream shortcuts rather than from the stream itself; if they don't respond while you're
@@ -154,7 +158,8 @@ There are two, and they are a per-client setting called **Mouse input**:
- **Capture (games)** — the pointer locks to the stream and only relative movement is sent. The only
cursor you see is the host's. This is what mouse-look in a game needs. The session window also
grabs the keyboard here, so Alt+Tab and the Windows key (Super on Linux) reach the host rather than
your own desktop — turn **Capture system shortcuts** off in
your own desktop — on macOS that is the ⌘ chords, ⌘Q included, with ⌘⎋ kept back as the way out.
Turn **Capture system shortcuts** off in
[client settings](/docs/client-settings#input) to keep them local.
- **Desktop (absolute)** — the pointer is not locked. It moves in and out of the stream freely and
its position is sent as an absolute point — what you want for remote desktop work. Your local