Compare commits

..

2 Commits

Author SHA1 Message Date
enricobuehler 6bd8c18b4d feat(clipboard): Linux + Windows host clipboard backends (Phase 1 host + Phase 3)
Host-side real-session-clipboard integration, reached through the portable
ClipCoordCmd seam so punktfunk1.rs stays platform-agnostic (design
clipboard-and-file-transfer.md §4). Advertised + started only when the operator
policy (PUNKTFUNK_CLIPBOARD, default off) allows it and the session mirrors a
live compositor.

clipboard/ module (gated cfg(any(linux, windows))):
- mod.rs — the unified HostClipboard enum + §3.5 wire<->platform normalization
  and the shared ClipEvent / PasteResponder.
- wayland.rs — ext-data-control-v1 (KWin / wlroots / Sway / Hyprland), the
  preferred Linux backend. On-glass verified on Hyprland.
- mutter.rs — GNOME. Mutter ships no wl/ext data-control at any version, so this
  talks to its DIRECT org.gnome.Mutter.RemoteDesktop clipboard (the xdg portal
  needs an interactive grant a headless host can't answer). On-glass on GNOME.
- windows.rs — the Win32 clipboard on a dedicated message-loop window:
  AddClipboardFormatListener -> WM_CLIPBOARDUPDATE for host copies, OLE delayed
  rendering (WM_RENDERFORMAT) for host pastes, per-window state via GWLP_USERDATA.
- winfmt.rs — pure Win32<->wire byte conversions (CF_HTML offset math, UTF-16
  text, RTF NUL trim), unit-tested on any host (cfg(any(windows, test))).
- session.rs — the backend-agnostic per-session coordinator owning all four data
  paths (host copy->client, client fetch, client copy->host, host paste).

Cargo.toml: windows crate gains Win32_System_DataExchange + Win32_System_Ole.

Verified: Linux builds + clippy + 282 host tests + fmt clean; the full host crate
compiles native on x86_64-pc-windows-msvc (checked on the RTX box).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 19:08:36 +02:00
enricobuehler af3a7d8cd5 feat(clipboard): wire protocol + client-core task for shared clipboard (Phase 0)
The portable shared-clipboard plane in punktfunk-core, all behind the `quic`
feature (design/clipboard-and-file-transfer.md §3):

- Control messages 0x40–0x44 (ClipControl / ClipOffer / ClipFetch...) and the
  HOST_CAP_CLIPBOARD capability bit, negotiated in the Welcome caps.
- Per-transfer QUIC bi-streams ("PKFs" magic) for lazy fetch of offered content,
  with ClipFetchHdr status/size framing (quic::clipstream).
- The §3.5 portable wire-MIME vocabulary (text/plain;utf-8, text/html, text/rtf,
  image/png) shared by both ends.
- Client-side clipboard task (client.rs) + C ABI surface bumped to v6 (abi.rs,
  regenerated include/punktfunk_core.h).
- Loopback transport tests (quic::tests).

No OS clipboard integration yet — that is the host backends (Phase 1/3) and the
macOS client (Phase 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 19:08:21 +02:00
968 changed files with 40956 additions and 225271 deletions
-18
View File
@@ -1,18 +0,0 @@
# Workspace-wide build flags.
#
# aes_armv8: RustCrypto's `aes` 0.8.x enables ARMv8-Crypto hardware AES on aarch64 only behind
# this cfg (x86_64 AES-NI is runtime-detected with no flag; the 0.9 line will make aarch64
# automatic too). Without it every aarch64 client (all Apple + virtually all Android) ran
# SOFTWARE AES on the per-packet decrypt path — measured 2026-07-14 on an M3 Ultra at
# ~240 MiB/s/core (~7 µs per 1.4 KB datagram), which single-handedly capped receive throughput
# at ~1.57 Gbps wire. The cfg still runtime-detects via `cpufeatures`, so a chip without the
# extensions falls back safely.
#
# NOTE: a RUSTFLAGS environment variable OVERRIDES config rustflags entirely — build scripts /
# CI lanes that set RUSTFLAGS for aarch64 targets (cargo-ndk, xcframework) must carry
# `--cfg aes_armv8` themselves.
# polyval_armv8: same story for GCM's other half — `polyval` 0.6.x gates its PMULL (carry-less
# multiply) GHASH path behind this cfg on aarch64. AES alone took open_in_place from 240 to
# ~790 MiB/s on the M3 Ultra; software GHASH still dominated until this flag joined it.
[target.'cfg(target_arch = "aarch64")']
rustflags = ["--cfg", "aes_armv8", "--cfg", "polyval_armv8"]
+1 -4
View File
@@ -94,10 +94,7 @@ jobs:
# Linux NVIDIA; design/linux-direct-nvenc.md). AMD/Intel-safe — NVENC/CUDA is dlopen'd at
# runtime (no link-time dep; identical DT_NEEDED to a plain build), and the encoder is only
# constructed for a CUDA capture frame + PUNKTFUNK_NVENC_DIRECT, never on VAAPI hosts.
# --features punktfunk-host/vulkan-encode: the AMD/Intel twin — raw VK_KHR_video_encode_h265
# with real RFI (design/linux-vulkan-video-encode.md). Pure Rust ash (no new lib / link dep);
# default on for HEVC (PUNKTFUNK_VULKAN_ENCODE=0 → libav VAAPI), failed open falls back to VAAPI.
cargo build --release --locked --features punktfunk-host/nvenc,punktfunk-host/vulkan-encode \
cargo build --release --locked --features punktfunk-host/nvenc \
-p punktfunk-host -p punktfunk-client-linux -p punktfunk-client-session
- name: Build + smoke-boot web console (bun preset)
+1 -8
View File
@@ -316,10 +316,6 @@ jobs:
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk iOS App Store Distribution"
# The embedded PunktfunkWidgetsExtension (bundle io.unom.punktfunk.widgets) is a second
# distribution artifact in the .ipa, so manual signing must map its App ID to its own
# App Store profile too — else exportArchive fails ("no profile for io.unom.punktfunk.widgets").
WIDGET_PROFILE="Punktfunk iOS Widgets App Store Distribution"
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-iOS \
-destination 'generic/platform=iOS' \
@@ -339,10 +335,7 @@ jobs:
<key>signingStyle</key><string>manual</string>
<key>signingCertificate</key><string>Apple Distribution</string>
<key>provisioningProfiles</key>
<dict>
<key>io.unom.punktfunk</key><string>$PROFILE</string>
<key>io.unom.punktfunk.widgets</key><string>$WIDGET_PROFILE</string>
</dict>
<dict><key>io.unom.punktfunk</key><string>$PROFILE</string></dict>
</dict>
</plist>
EOF
-60
View File
@@ -1,60 +0,0 @@
# Publish the TypeScript SDK (@punktfunk/host) to the Gitea npm registry
# (https://git.unom.io/api/packages/unom/npm/).
#
# Trigger: push a tag `sdk-vX.Y.Z` (must equal sdk/package.json "version"), or run manually.
# The SDK versions independently of the app's `v*` tags, so bumping the host doesn't republish it.
#
# Auth: REGISTRY_TOKEN — the same repo Actions secret docker.yml uses (a Gitea PAT with
# write:package scope). No new secret needed.
name: sdk-publish
on:
push:
tags: ['sdk-v*']
workflow_dispatch:
jobs:
publish:
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 15
defaults:
run:
working-directory: sdk
steps:
# oven/bun's slim base ships neither git, a CA bundle, nor node — actions/checkout's HTTPS
# fetch needs git + ca-certificates, and the version-guard step below uses node.
- name: Install git + node + CA certs
working-directory: /
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git nodejs
- uses: actions/checkout@v4
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
- name: Typecheck
run: bun run typecheck
- name: Test
run: bun test
- name: Build (dist/ JS + .d.ts)
run: bun run build
- name: Tag matches package version
if: startsWith(github.ref, 'refs/tags/')
run: |
TAG="${GITHUB_REF_NAME#sdk-v}"
PKG="$(node -p "require('./package.json').version")"
test "$TAG" = "$PKG" || { echo "tag $GITHUB_REF_NAME does not match package version $PKG"; exit 1; }
- name: Publish to Gitea registry
env:
NODE_AUTH_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
test -n "$NODE_AUTH_TOKEN" || { echo "REGISTRY_TOKEN secret is empty"; exit 1; }
# .npmrc already maps the @punktfunk scope to the registry; append the auth line.
printf '//git.unom.io/api/packages/unom/npm/:_authToken=%s\n' "$NODE_AUTH_TOKEN" >> .npmrc
bun publish
+3 -3
View File
@@ -153,9 +153,9 @@ jobs:
# `// SAFETY:` proof. Both invariants are lint-gated (`unsafe_op_in_unsafe_fn` +
# `undocumented_unsafe_blocks`); this step keeps them from regressing. (wdk-probe is a
# toolchain-only probe crate and is excluded.)
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p pf-mouse -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad/mouse drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense -p pf-mouse --check
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense --check
- name: Inspect /INTEGRITYCHECK (before) — expect FORCE_INTEGRITY set by wdk-build
run: |
# explicit --target (.cargo/config.toml) -> output under the triple subdir.
-5
View File
@@ -38,8 +38,3 @@ CLAUDE.md
# Local flatpak-builder output (build-flatpak.sh) — ostree repo + build dir at the repo root.
.flatpak-repo/
.flatpak-build/
# Nix build outputs (flake.nix) — `nix build` result symlinks + direnv cache. flake.lock IS tracked.
/result
/result-*
.direnv/
-10
View File
@@ -30,16 +30,6 @@ file with `scripts/gen-third-party-notices.sh` when the dependency tree changes.
## Before you push
Enable the repo git hooks once per clone — they run the exact rustfmt gates CI runs (main
workspace + the UMDF driver workspace) on every commit and push, so a push can never fail CI
on formatting alone:
```sh
git config core.hooksPath scripts/git-hooks
```
Then the usual full pass:
```sh
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
Generated
+32 -218
View File
@@ -870,6 +870,15 @@ dependencies = [
"itertools 0.10.5",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.6"
@@ -2145,7 +2154,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.13.0"
version = "0.9.2"
[[package]]
name = "lazy_static"
@@ -2277,7 +2286,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"punktfunk-core",
]
@@ -2754,32 +2763,11 @@ version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.13.0"
dependencies = [
"anyhow",
"ashpd",
"libc",
"pf-driver-proto",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-win-display",
"pf-zerocopy",
"pipewire",
"punktfunk-core",
"tokio",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-client-core"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"ash",
"async-channel",
"ffmpeg-next",
"mdns-sd",
@@ -2787,7 +2775,6 @@ dependencies = [
"pf-ffvk",
"pipewire",
"punktfunk-core",
"pyrowave-sys",
"rustls",
"sdl3",
"serde",
@@ -2798,26 +2785,9 @@ dependencies = [
"windows 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
name = "pf-clipboard"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"futures-util",
"libc",
"punktfunk-core",
"quinn",
"tokio",
"tracing",
"wayland-client",
"wayland-protocols",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-console-ui"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"ash",
@@ -2836,105 +2806,18 @@ dependencies = [
"bytemuck",
]
[[package]]
name = "pf-encode"
version = "0.13.0"
dependencies = [
"anyhow",
"ash",
"ffmpeg-next",
"libc",
"libloading",
"nvidia-video-codec-sdk",
"openh264",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-zerocopy",
"punktfunk-core",
"pyrowave-sys",
"tracing",
"tracing-subscriber",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-ffvk"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"ash",
"bindgen",
"pkg-config",
]
[[package]]
name = "pf-frame"
version = "0.13.0"
dependencies = [
"anyhow",
"libc",
"pf-zerocopy",
"punktfunk-core",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-gpu"
version = "0.13.0"
dependencies = [
"anyhow",
"pf-host-config",
"pf-paths",
"serde",
"serde_json",
"tempfile",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-host-config"
version = "0.13.0"
[[package]]
name = "pf-inject"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"futures-util",
"libc",
"parking_lot",
"pf-capture",
"pf-driver-proto",
"pf-host-config",
"pf-paths",
"punktfunk-core",
"reis",
"tokio",
"tracing",
"usbip-sim",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"wayland-protocols-misc",
"wayland-protocols-wlr",
"wayland-scanner",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"xkbcommon",
]
[[package]]
name = "pf-paths"
version = "0.13.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"ash",
@@ -2947,62 +2830,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "pf-vdisplay"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"bytemuck",
"futures-util",
"hex",
"libc",
"pf-driver-proto",
"pf-encode",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-paths",
"pf-win-display",
"punktfunk-core",
"serde",
"serde_json",
"sha2",
"tokio",
"tracing",
"utoipa",
"wayland-backend",
"wayland-client",
"wayland-scanner",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-win-display"
version = "0.13.0"
dependencies = [
"anyhow",
"pf-paths",
"punktfunk-core",
"serde_json",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-zerocopy"
version = "0.13.0"
dependencies = [
"anyhow",
"ash",
"khronos-egl",
"libc",
"libloading",
"serde",
"serde_json",
"tracing",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -3174,7 +3001,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"android_logger",
"jni",
@@ -3190,7 +3017,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"async-channel",
@@ -3206,7 +3033,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3221,17 +3048,22 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"async-channel",
"crossbeam-channel",
"ffmpeg-next",
"mdns-sd",
"opus",
"pf-client-core",
"punktfunk-core",
"sdl3",
"serde",
"serde_json",
"tracing",
"tracing-subscriber",
"wasapi",
"windows 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-reactor",
"windows-reactor-setup",
@@ -3240,7 +3072,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"aes-gcm",
"bytes",
@@ -3271,7 +3103,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"aes",
"aes-gcm",
@@ -3283,9 +3115,9 @@ dependencies = [
"base64",
"bytemuck",
"cbc",
"ffmpeg-next",
"futures-util",
"hex",
"hmac",
"http-body-util",
"hyper",
"hyper-util",
@@ -3296,20 +3128,10 @@ dependencies = [
"log",
"mac_address",
"mdns-sd",
"nvidia-video-codec-sdk",
"openh264",
"opus",
"parking_lot",
"pf-capture",
"pf-clipboard",
"pf-driver-proto",
"pf-encode",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-inject",
"pf-paths",
"pf-vdisplay",
"pf-win-display",
"pf-zerocopy",
"pipewire",
"punktfunk-core",
"quinn",
@@ -3353,7 +3175,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3367,29 +3189,21 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.13.0"
version = "0.9.2"
dependencies = [
"anyhow",
"ksni",
"libc",
"punktfunk-core",
"rustls",
"serde",
"serde_json",
"sha2",
"ureq",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service",
"winresource",
]
[[package]]
name = "pyrowave-sys"
version = "0.13.0"
dependencies = [
"bindgen",
"cmake",
]
[[package]]
name = "quick-error"
version = "1.2.3"
+1 -13
View File
@@ -6,22 +6,10 @@ members = [
"crates/punktfunk-host/vendor/usbip-sim",
"crates/punktfunk-tray",
"crates/pf-client-core",
"crates/pf-clipboard",
"crates/pf-presenter",
"crates/pf-console-ui",
"crates/pf-ffvk",
"crates/pf-driver-proto",
"crates/pf-paths",
"crates/pf-host-config",
"crates/pf-gpu",
"crates/pf-zerocopy",
"crates/pf-frame",
"crates/pf-win-display",
"crates/pf-encode",
"crates/pf-capture",
"crates/pf-inject",
"crates/pf-vdisplay",
"crates/pyrowave-sys",
"clients/probe",
"clients/linux",
"clients/session",
@@ -47,7 +35,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.13.0"
version = "0.9.2"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+297 -679
View File
File diff suppressed because it is too large Load Diff
+9 -1030
View File
File diff suppressed because it is too large Load Diff
@@ -27,27 +27,12 @@
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<!-- Gamepad rumble feedback. -->
<uses-permission android:name="android.permission.VIBRATE" />
<!-- Steam Controller 2 over direct BLE (Sc2BleLink talks Valve's vendor GATT service to the
bonded pad). A RUNTIME permission (NEARBY_DEVICES group); the capture engages only when
already granted — USB capture (wired / Puck dongle) needs no Bluetooth at all. -->
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<!-- We target phone + TV from day one: keep the app installable on TV (no touchscreen) and on
devices without a gamepad. -->
<uses-feature android:name="android.hardware.touchscreen" android:required="false" />
<uses-feature android:name="android.software.leanback" android:required="false" />
<uses-feature android:name="android.hardware.gamepad" android:required="false" />
<!-- Neutralize Play's IMPLIED hard requirements, which filtered real TVs as "not compatible"
(reported on a Philips OLED707): RECORD_AUDIO implies android.hardware.microphone and the
Wi-Fi state permissions imply android.hardware.wifi, both required=true unless declared
otherwise. Some TVs declare no microphone (mic uplink is optional and runtime-gated) and
ethernet-only boxes declare no wifi (discovery/WifiLock are best-effort hedges there). -->
<uses-feature android:name="android.hardware.microphone" android:required="false" />
<uses-feature android:name="android.hardware.wifi" android:required="false" />
<!-- Steam Controller 2 capture: USB host for the wired pad / Puck dongle, Bluetooth for the
direct-BLE pad — both optional (the feature quietly disengages without them). -->
<uses-feature android:name="android.hardware.usb.host" android:required="false" />
<uses-feature android:name="android.hardware.bluetooth_le" android:required="false" />
<!-- appCategory="game": a game-streaming client IS a game as far as the SoC is concerned.
On Snapdragon devices (and other OEMs with a Game Mode / Game Dashboard) this makes the app
@@ -73,16 +58,10 @@
android:name="android.game_mode_config"
android:resource="@xml/game_mode_config" />
<!-- configChanges includes `keyboard` (not just keyboardHidden): claiming a Steam
Controller 2's USB HID interface removes its lizard-mode keyboard/mouse input
devices, which flips CONFIG_KEYBOARD (QWERTY→NOKEYS) — without `keyboard` declared,
Android RECREATES the activity, disposing StreamScreen and killing the stream the
moment the capture engages (tester-diagnosed on-glass, 2026-07-15). Releasing the
interfaces at session end brings the devices back — same flip, same need. -->
<activity
android:name=".MainActivity"
android:exported="true"
android:configChanges="orientation|screenSize|keyboard|keyboardHidden|screenLayout|density|navigation"
android:configChanges="orientation|screenSize|keyboardHidden|screenLayout|density|navigation"
android:theme="@style/Theme.PunktfunkAndroid">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
@@ -303,8 +303,7 @@ internal fun PairPinDialog(
if (fp.isNotEmpty()) {
onPaired(fp) // verified host fp — caller saves + connects
} else {
// Cause-specific: wrong PIN vs not-armed vs unreachable.
err = ConnectErrors.pairMessage(NativeBridge.nativeTakeLastError())
err = "Pairing failed — wrong PIN, or the host isn't armed."
}
}
}
@@ -1,69 +0,0 @@
package io.unom.punktfunk
import io.unom.punktfunk.kit.NativeBridge
/**
* Cause-specific user-facing messages for failed pair/connect attempts, keyed on the stable
* machine token from [NativeBridge.nativeTakeLastError]. One vocabulary for both the PIN
* ceremony and the request-access (delegated approval) path, so a dead network path is never
* reported as "wrong PIN" and an operator denial is never reported as a timeout — the exact
* collapse behind more than one support thread.
*/
object ConnectErrors {
/** Message for a failed SPAKE2 PIN ceremony ([NativeBridge.nativePair] returned `""`). */
fun pairMessage(token: String): String = when (token) {
"crypto" -> "Wrong PIN — check the PIN on the host's Pairing page and try again."
else -> shared(token) ?: transport(token)
}
/**
* Message for a failed connect / request-access ([NativeBridge.nativeConnect] returned `0`).
* [requestAccess] tunes the fallback wording for the delegated-approval path.
*/
fun connectMessage(token: String, requestAccess: Boolean): String =
shared(token) ?: when (token) {
"crypto" ->
"The host's identity doesn't match the saved fingerprint — re-pair with this host."
"timeout", "io", "" ->
if (requestAccess) {
"The request never reached the host, or nobody approved it in time — " +
"check the network path (no VPN, no guest-Wi-Fi isolation) and the " +
"host's console."
} else {
transport(token)
}
else -> "Connection failed — check host/port and logcat."
}
/** The host's typed rejection reasons — identical wording across every punktfunk client. */
private fun shared(token: String): String? = when (token) {
"not-armed" ->
"Pairing isn't armed on the host — arm it on the host's Pairing page, then try again."
"bound-other" ->
"The host's pairing window is armed for a different device — arm it for this one."
"rate-limited" -> "Too many pairing attempts — wait a couple of seconds and try again."
"identity-required" ->
"The host requires pairing — pair this device (PIN or request access) first."
"denied" -> "The host declined this device's request."
"approval-timeout" ->
"Nobody approved the request on the host in time — approve this device in the " +
"host's console or web UI, then request access again."
"superseded" ->
"A newer request from this device replaced this one — approve the latest request " +
"on the host."
"wire-version" -> "Client and host versions don't match — update both to the same release."
"busy" -> "The host is busy with another session."
else -> null
}
/** Transport-level causes (nothing typed arrived from the host). */
private fun transport(token: String): String = when (token) {
"timeout" ->
"The host didn't answer — check that this device and the host are on the same " +
"network (no VPN on this device, no guest-Wi-Fi / AP isolation)."
"io" ->
"Couldn't reach the host — check that this device and the host are on the same " +
"network (no VPN on this device, no guest-Wi-Fi / AP isolation)."
else -> "Pairing failed — the host didn't answer or closed the connection (see logcat)."
}
}
@@ -305,17 +305,13 @@ fun ConnectScreen(
onConnected(handle)
} else {
discovery.start()
val token = NativeBridge.nativeTakeLastError()
val unreachable = token == "timeout" || token == "io" || token.isEmpty()
if (onFailure != null && unreachable) {
// Unreachable — hand off to the wake-and-wait flow — clearing `attempt` above
// and setting `waker.waking` here land in one recompose, so the overlay slides
if (onFailure != null) {
// Hand off to the wake-and-wait flow — clearing `attempt` above and setting
// `waker.waking` here land in one recompose, so the overlay slides
// Connecting → Waking without a blank frame.
onFailure()
} else {
// A typed host rejection (busy / versions differ / pairing required) means the
// host is awake — waking it would be nonsense; show the stated reason instead.
status = ConnectErrors.connectMessage(token, requestAccess = false)
status = "Connection failed — check host/port, PIN, and logcat"
}
}
}
@@ -420,12 +416,7 @@ fun ConnectScreen(
}
onConnected(handle)
} else {
// Cause-specific: an operator denial, an approval timeout, and a request that
// never reached the host are different problems with different fixes.
status = ConnectErrors.connectMessage(
NativeBridge.nativeTakeLastError(),
requestAccess = true,
)
status = "Request timed out — approve this device in the host's console, then retry."
discovery.start()
}
}
@@ -1,6 +1,5 @@
package io.unom.punktfunk
import android.content.Context
import android.hardware.input.InputManager
import android.os.Build
import android.os.CombinedVibration
@@ -45,7 +44,6 @@ import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.Sc2Capture
import kotlinx.coroutines.delay
/**
@@ -149,38 +147,8 @@ fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit) {
) {
Text("Controllers", style = MaterialTheme.typography.headlineMedium)
// Steam Controller 2 detection: never an InputDevice (lizard mode is kb/mouse; the
// capture claims even those away), so it's enumerated on the capture side — USB device
// list + bonded BLE — and re-checked on USB hot-plug.
var sc2Generation by remember { mutableIntStateOf(0) }
DisposableEffect(Unit) {
val receiver = object : android.content.BroadcastReceiver() {
override fun onReceive(c: Context?, i: android.content.Intent?) { sc2Generation++ }
}
val filter = android.content.IntentFilter().apply {
addAction(android.hardware.usb.UsbManager.ACTION_USB_DEVICE_ATTACHED)
addAction(android.hardware.usb.UsbManager.ACTION_USB_DEVICE_DETACHED)
}
if (Build.VERSION.SDK_INT >= 33) {
context.registerReceiver(receiver, filter, Context.RECEIVER_NOT_EXPORTED)
} else {
@Suppress("UnspecifiedRegisterReceiverFlag")
context.registerReceiver(receiver, filter)
}
onDispose { runCatching { context.unregisterReceiver(receiver) } }
}
val sc2Probe = remember { Sc2Capture(context) }
val sc2Usb = remember(sc2Generation) { sc2Probe.findUsbDevice() }
val sc2Ble = remember(sc2Generation) {
if (context.checkSelfPermission(android.Manifest.permission.BLUETOOTH_CONNECT) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
) sc2Probe.pairedBleAddress() else null
}
val sc2Present = sc2Usb != null || sc2Ble != null
Group("Gamepads") {
if (sc2Present) Sc2Row(sc2Usb, activity)
if (pads.isEmpty() && !sc2Present) {
if (pads.isEmpty()) {
Text(
"No controller detected. punktfunk can only forward devices Android " +
"classifies as a gamepad or joystick — a pad connected through an adapter " +
@@ -190,11 +158,8 @@ fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit) {
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
// Every real controller is forwarded now (Automatic forwards them all, each on its own
// wire pad index) — not just the first. A joystick-only device Android doesn't classify as
// a gamepad still can't be forwarded (the host wants a gamepad), so gate the badge on it.
pads.forEach { dev ->
PadRow(dev, forwarded = isForwarded(dev), gamepadSetting = gamepadSetting)
pads.forEachIndexed { i, dev ->
PadRow(dev, forwarded = i == 0, gamepadSetting = gamepadSetting)
}
}
@@ -246,79 +211,6 @@ fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit) {
}
}
/**
* The Steam Controller 2 card — capture-side state, since a (claimed or lizard-mode) SC2 never
* appears as a gamepad InputDevice. Shows the transport, whether the capture is live (driving
* these menus now; streamed as-is in a session), and a grant button when USB access is missing.
*/
@Composable
private fun Sc2Row(usbDev: android.hardware.usb.UsbDevice?, activity: MainActivity?) {
val context = LocalContext.current
val settingOn = remember { SettingsStore(context).load().sc2Capture }
val active = activity?.sc2MenuActive == true
val usbManager = context.getSystemService(Context.USB_SERVICE) as android.hardware.usb.UsbManager
val permitted = usbDev != null && usbManager.hasPermission(usbDev)
OutlinedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(6.dp),
) {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Text(
"Steam Controller 2",
style = MaterialTheme.typography.bodyLarge,
modifier = Modifier.weight(1f),
)
if (active) {
Text(
"navigating this UI",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary,
)
}
}
Text(
when {
usbDev == null -> "Paired via Bluetooth"
usbDev.productId == io.unom.punktfunk.kit.Sc2Device.PID_WIRED -> "Wired (USB)"
else -> "Puck dongle (USB)"
},
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
when {
!settingOn -> Text(
"Passthrough is disabled in Settings — enable \"Steam Controller 2 " +
"passthrough\" to capture it.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
active -> Text(
"Captured — streams as-is: the host presents a real Steam Controller 2 " +
"that its Steam drives directly (trackpads, gyro, haptics).",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
usbDev != null && !permitted -> {
Text(
"Needs USB access to be captured.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
OutlinedButton(onClick = { activity?.startSc2MenuNav(forceAsk = true) }) {
Text("Grant USB access")
}
}
else -> Text(
"Detected — capture engages automatically.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
}
/** One detected gamepad: identity, what it streams as, and a rumble test. */
@Composable
private fun PadRow(dev: InputDevice, forwarded: Boolean, gamepadSetting: Int) {
@@ -330,12 +222,8 @@ private fun PadRow(dev: InputDevice, forwarded: Boolean, gamepadSetting: Int) {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Text(dev.name, style = MaterialTheme.typography.bodyLarge, modifier = Modifier.weight(1f))
if (forwarded) {
// Android's own controller number (1-based; 0 = unassigned), shown so a multi-pad
// user can tell which physical pad is which. The stream's wire pad index is
// assigned separately (lowest-free per device) once streaming starts.
val number = dev.controllerNumber
Text(
if (number > 0) "forwarded · player $number" else "forwarded to host",
"forwarded to host",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary,
)
@@ -431,15 +319,6 @@ private fun Group(title: String, content: @Composable ColumnScope.() -> Unit) {
}
}
/**
* Whether this device is actually forwarded to the host — the same rule the stream's [GamepadRouter]
* applies: a real, non-virtual controller whose source classes include GAMEPAD. A joystick-only node
* (e.g. a DualSense motion-sensor sibling, or an adapter that enumerates as bare joystick) shows in
* the list but isn't forwarded.
*/
private fun isForwarded(dev: InputDevice): Boolean =
!dev.isVirtual && dev.sources and InputDevice.SOURCE_GAMEPAD == InputDevice.SOURCE_GAMEPAD
/** Whether the controller reports a rumble motor — via VibratorManager (API 31+) or the legacy Vibrator. */
private fun deviceHasVibrator(dev: InputDevice): Boolean =
if (Build.VERSION.SDK_INT >= 31) {
@@ -492,10 +371,6 @@ private fun prefLabel(pref: Int): String = when (pref) {
Gamepad.PREF_DUALSHOCK4 -> "DualShock 4"
Gamepad.PREF_STEAMCONTROLLER -> "Steam Controller"
Gamepad.PREF_STEAMDECK -> "Steam Deck"
Gamepad.PREF_DUALSENSEEDGE -> "DualSense Edge"
Gamepad.PREF_SWITCHPRO -> "Switch Pro"
Gamepad.PREF_STEAMCONTROLLER2 -> "Steam Controller 2"
Gamepad.PREF_STEAMCONTROLLER2_PUCK -> "Steam Controller 2 Puck"
else -> "Automatic"
}
@@ -351,12 +351,7 @@ fun GamepadPairPinDialog(pt: PendingTrust, identity: ClientIdentity?, onPaired:
NativeBridge.nativePair(pt.host, pt.port, id.certPem, id.privateKeyPem, pin, name)
}
pairing = false
if (fp.isNotEmpty()) {
onPaired(fp)
} else {
// Cause-specific: wrong PIN vs not-armed vs unreachable.
err = ConnectErrors.pairMessage(NativeBridge.nativeTakeLastError())
}
if (fp.isNotEmpty()) onPaired(fp) else err = "Pairing failed — wrong PIN, or the host isn't armed."
}
}
@@ -241,10 +241,7 @@ private fun resolveDir(s: NavInputState): NavDir? {
if (s.hatY >= 0.5f) return NavDir.DOWN
if (s.hatX <= -0.5f) return NavDir.LEFT
if (s.hatX >= 0.5f) return NavDir.RIGHT
// Horizontal wins an exact |x| == |y| diagonal tie (Y must be strictly greater to take the
// vertical branch), matching the SDL core and Apple nav so a perfect 45° push resolves the
// same on every client.
return if (abs(s.stickY) > abs(s.stickX)) {
return if (abs(s.stickY) >= abs(s.stickX)) {
when {
s.stickY <= -STICK_HIGH -> NavDir.UP
s.stickY >= STICK_HIGH -> NavDir.DOWN
@@ -49,14 +49,12 @@ import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.platform.LocalConfiguration
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import dev.chrisbanes.haze.HazeState
import dev.chrisbanes.haze.hazeSource
import io.unom.punktfunk.kit.deviceBodyVibrator
// The gamepad-driven settings screen — the Android mirror of the Apple client's GamepadSettingsView:
// the couch-relevant subset of the touch settings restyled as a console page and fully navigable with
@@ -84,10 +82,7 @@ fun GamepadSettingsScreen(
var s by remember { mutableStateOf(initial) }
fun update(next: Settings) { s = next; onChange(next) }
val context = LocalContext.current
// Gates the "Rumble on this phone" row — a TV box has no body vibrator to mirror onto.
val hasBodyVibrator = remember { deviceBodyVibrator(context) != null }
val rows = buildSettingsRows(s, hasBodyVibrator, ::update)
val rows = buildSettingsRows(s, ::update)
var focus by remember { mutableIntStateOf(0) }
if (focus > rows.lastIndex) focus = rows.lastIndex
// The direction the focused value last stepped (+1 forward / -1 back) — drives which way the
@@ -262,13 +257,8 @@ private fun SettingRowView(row: GpRow, focused: Boolean, adjustDir: Int, onClick
}
}
/** Build the console settings rows from the current [Settings], writing through [update].
* [hasBodyVibrator] gates the "Rumble on this phone" row (absent on TVs). */
private fun buildSettingsRows(
s: Settings,
hasBodyVibrator: Boolean,
update: (Settings) -> Unit,
): List<GpRow> {
/** Build the console settings rows from the current [Settings], writing through [update]. */
private fun buildSettingsRows(s: Settings, update: (Settings) -> Unit): List<GpRow> {
fun <T> choice(
id: String, header: String?, label: String, detail: String,
options: List<Pair<T, String>>, current: T, write: (T) -> Unit,
@@ -364,18 +354,7 @@ private fun buildSettingsRows(
"The virtual pad the host creates — Automatic matches this controller.",
GAMEPAD_OPTIONS.mapIndexed { i, lbl -> i to lbl }, s.gamepad,
) { update(s.copy(gamepad = it)) },
) + listOfNotNull(
if (hasBodyVibrator) {
toggle(
"phoneRumble", null, "Rumble on this phone",
"Also play controller 1's rumble on this phone's own vibration motor — " +
"for clip-on pads without rumble motors.",
s.rumbleOnPhone,
) { update(s.copy(rumbleOnPhone = it)) }
} else {
null
},
) + listOf(
choice(
"hud", "Interface", "Statistics overlay",
"How much the overlay shows: Compact (one line) → Normal → Detailed (full HUD). " +
@@ -10,7 +10,6 @@ import android.os.Looper
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.State
import androidx.compose.runtime.derivedStateOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.ui.platform.LocalContext
@@ -47,10 +46,6 @@ fun isTvDevice(context: Context): Boolean {
@Composable
fun rememberControllerConnected(): State<Boolean> {
val context = LocalContext.current
// A menu-captured Steam Controller 2 counts as connected: it drives the console UI through
// the capture link, but never surfaces as an Android InputDevice (lizard mode is kb/mouse,
// and the claim removes even those) — the InputManager path below can't see it.
val activity = context as? MainActivity
val connected = remember { mutableStateOf(Gamepad.firstPad() != null) }
DisposableEffect(Unit) {
val im = context.getSystemService(Context.INPUT_SERVICE) as InputManager
@@ -64,7 +59,5 @@ fun rememberControllerConnected(): State<Boolean> {
connected.value = Gamepad.firstPad() != null
onDispose { im.unregisterInputDeviceListener(listener) }
}
return remember {
derivedStateOf { connected.value || activity?.sc2MenuActive == true }
}
return connected
}
@@ -1,16 +1,8 @@
package io.unom.punktfunk
import android.app.PendingIntent
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.PackageManager
import android.hardware.usb.UsbManager
import android.os.Build
import android.os.Bundle
import android.view.InputDevice
import android.view.KeyCharacterMap
import android.view.KeyEvent
import android.view.MotionEvent
import androidx.activity.ComponentActivity
@@ -24,13 +16,9 @@ import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.GamepadRouter
import io.unom.punktfunk.kit.Keymap
import io.unom.punktfunk.kit.NativeBridge
/** Broadcast action for the menu-time SC2 USB-permission grant (see [MainActivity.startSc2MenuNav]). */
private const val SC2_MENU_PERMISSION = "io.unom.punktfunk.SC2_MENU_USB_PERMISSION"
class MainActivity : ComponentActivity() {
/**
* The active stream session handle (0 = not streaming). Set by [StreamScreen] while it's shown.
@@ -39,12 +27,8 @@ class MainActivity : ComponentActivity() {
*/
var streamHandle: Long = 0L
/**
* Multi-controller router for the active session (built/released by StreamScreen): assigns each
* connected pad a stable wire index, threads it onto every event, declares/removes pads on
* hot-plug, and routes rumble/HID feedback back by pad index. Null while not streaming.
*/
var gamepadRouter: GamepadRouter? = null
/** Joystick-axis state mapper for the active session (built/reset by StreamScreen). */
var axisMapper: Gamepad.AxisMapper? = null
/**
* Input observers for the Controllers debug screen (set while it is shown, like [streamHandle]).
@@ -60,6 +44,9 @@ class MainActivity : ComponentActivity() {
*/
var requestStreamExit: (() -> Unit)? = null
/** Currently-held forwarded pad buttons (bitmask of `Gamepad.BTN_*`), for chord detection. */
private var heldPadButtons = 0
/**
* Whether the last console input came from a real gamepad (face buttons / stick) vs. a TV D-pad
* remote (which has no A/B/X/Y). The console UI reads this to show glyphs the user recognises — pad
@@ -84,30 +71,6 @@ class MainActivity : ComponentActivity() {
/** The panel's highest-refresh display mode (0 = unknown/unsupported), resolved once at startup. */
private var highRefreshModeId = 0
/**
* Menu-time Steam Controller 2 capture (UI mode — no router): a captured SC2 never produces
* ordinary gamepad events (lizard mode is kb/mouse; the claim removes even those), so this
* drives the console UI directly from the parsed reports via [sc2NavKey]. Runs while the app
* is foreground and NOT streaming; StreamScreen pauses it around its own stream-mode capture.
* [sc2MenuActive] is observed by the console-UI gate ([rememberControllerConnected]) and the
* Controllers screen.
*/
private var sc2Menu: io.unom.punktfunk.kit.Sc2Capture? = null
var sc2MenuActive by mutableStateOf(false)
private set
private var sc2Receiver: BroadcastReceiver? = null
private var sc2PermissionAsked = false
/**
* Compose focus hook for the SC2's synthetic D-pad (set by [onCreate]'s composition). A
* synthetic KeyEvent dispatched from OUTSIDE the real input pipeline never reaches
* ViewRootImpl's focus-navigation stage — the one that grants initial focus for a real
* pad's first D-pad press — so on a phone in touch mode it lands on a focus-less window
* and does nothing (first on-glass run: only B worked, since it bypasses key events
* entirely). `FocusManager.moveFocus` is the public API for exactly this.
*/
private var sc2MoveFocus: ((androidx.compose.ui.focus.FocusDirection) -> Boolean)? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
lastPadIsGamepad = !isTvDevice(this)
@@ -125,166 +88,13 @@ class MainActivity : ComponentActivity() {
// UI without a physical pad — `adb shell am start -n io.unom.punktfunk/.MainActivity --ez
// pf_force_gamepad_ui true`. Never set in normal use; real activation is a connected pad / TV.
val forceGamepadUi = intent?.getBooleanExtra("pf_force_gamepad_ui", false) ?: false
// SC2 hot-plug + the menu-time USB-permission grant both (re)start the menu capture.
val receiver = object : BroadcastReceiver() {
override fun onReceive(c: Context?, intent: Intent?) {
when (intent?.action) {
UsbManager.ACTION_USB_DEVICE_ATTACHED -> {
sc2PermissionAsked = false // a fresh attach may ask once again
startSc2MenuNav()
}
SC2_MENU_PERMISSION -> {
if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)) {
startSc2MenuNav()
}
}
}
}
}
sc2Receiver = receiver
val filter = IntentFilter().apply {
addAction(UsbManager.ACTION_USB_DEVICE_ATTACHED)
addAction(SC2_MENU_PERMISSION)
}
if (Build.VERSION.SDK_INT >= 33) {
registerReceiver(receiver, filter, Context.RECEIVER_NOT_EXPORTED)
} else {
@Suppress("UnspecifiedRegisterReceiverFlag")
registerReceiver(receiver, filter)
}
setContent {
PunktfunkTheme {
// Focus hook for the SC2's synthetic navigation (see [sc2MoveFocus]). `Next` is
// the bootstrap: directional moves need an already-focused node, while one-
// dimensional traversal assigns initial focus when there is none.
val focusManager = androidx.compose.ui.platform.LocalFocusManager.current
androidx.compose.runtime.DisposableEffect(Unit) {
sc2MoveFocus = { dir ->
focusManager.moveFocus(dir) ||
focusManager.moveFocus(androidx.compose.ui.focus.FocusDirection.Next)
}
onDispose { sc2MoveFocus = null }
}
Surface(modifier = Modifier.fillMaxSize()) { App(forceGamepadUi = forceGamepadUi) }
}
}
}
override fun onResume() {
super.onResume()
startSc2MenuNav()
}
override fun onPause() {
// Release the claim while backgrounded so the OS (and other apps) get the pad back.
stopSc2MenuNav()
super.onPause()
}
override fun onDestroy() {
sc2Receiver?.let { runCatching { unregisterReceiver(it) } }
sc2Receiver = null
stopSc2MenuNav()
super.onDestroy()
}
/**
* Engage the menu-time SC2 capture if possible: setting on, not streaming, and a wired/Puck
* pad attached (asking for USB permission at most once per attach — [forceAsk] re-arms the
* dialog, for the Controllers screen's explicit grant button) — else an already-paired BLE
* controller when BLUETOOTH_CONNECT is granted. Safe to call repeatedly.
*/
fun startSc2MenuNav(forceAsk: Boolean = false) {
if (forceAsk) sc2PermissionAsked = false
if (streamHandle != 0L) return // StreamScreen owns the pad while streaming
if (sc2Menu?.isActive == true) return
if (!SettingsStore(this).load().sc2Capture) return
val cap = sc2Menu ?: io.unom.punktfunk.kit.Sc2Capture(this).also { c ->
c.onUiKey = { key, down -> runOnUiThread { sc2NavKey(key, down) } }
c.onActiveChanged = { on -> runOnUiThread { sc2MenuActive = on } }
sc2Menu = c
}
val usbManager = getSystemService(Context.USB_SERVICE) as UsbManager
val dev = cap.findUsbDevice()
when {
dev != null && usbManager.hasPermission(dev) -> cap.startUsb(dev)
dev != null && !sc2PermissionAsked -> {
sc2PermissionAsked = true
usbManager.requestPermission(
dev,
PendingIntent.getBroadcast(
this, 1,
Intent(SC2_MENU_PERMISSION).setPackage(packageName),
// MUTABLE: the USB stack appends the grant extras to this intent.
PendingIntent.FLAG_MUTABLE,
),
)
}
dev == null && checkSelfPermission(android.Manifest.permission.BLUETOOTH_CONNECT) ==
PackageManager.PERMISSION_GRANTED -> {
cap.pairedBleAddress()?.let { cap.startBle(it) }
}
}
}
/** Release the menu-time SC2 capture (backgrounded / stream taking over). Idempotent. */
fun stopSc2MenuNav() {
sc2Menu?.stop()
sc2MenuActive = false
}
/**
* One SC2 navigation key transition from the menu-time capture (main thread) — routed the
* same way [dispatchKeyEvent]'s not-streaming branch routes a real pad's buttons: B backs,
* A activates the focused element, everything else (D-pad, shoulders, Start/Select) goes to
* the framework's focus navigation. Also claims the console-UI glyphs for the pad.
*/
private fun sc2NavKey(keyCode: Int, down: Boolean) {
if (streamHandle != 0L) return // raced a stream start — the wire path owns input now
lastPadIsGamepad = true
lastPadStyle = Gamepad.PadStyle.XBOX // Valve pads carry A/B/X/Y in Xbox positions
val action = if (down) KeyEvent.ACTION_DOWN else KeyEvent.ACTION_UP
// The console UI navigates through padKeyProbe (GamepadNavEffect's held-state + repeat
// machinery — A/X/Y/D-pad/Select), NOT the focus system: synthesized events must be
// offered there first, exactly like real ones in dispatchKeyEvent (tester-diagnosed:
// routing everything via super.dispatchKeyEvent bypassed the probe, so only B — which
// never rides key events — did anything). The probes gate on keycode only, so a
// synthetic KeyEvent satisfies them.
padKeyProbe?.let { if (it(KeyEvent(action, keyCode))) return }
when (keyCode) {
// B → back, on release (same edge the real-pad path uses).
KeyEvent.KEYCODE_BUTTON_B -> if (!down) onBackPressedDispatcher.onBackPressed()
// A → activate the focused element (the focus system understands DPAD_CENTER; the
// Compose node focused via the moveFocus hook receives it once the ComposeView
// holds view-focus).
KeyEvent.KEYCODE_BUTTON_A ->
super.dispatchKeyEvent(KeyEvent(action, KeyEvent.KEYCODE_DPAD_CENTER))
// D-pad → Compose's own focus API (a synthetic DPAD KeyEvent can't grant initial
// focus — see [sc2MoveFocus]); one move per press edge.
KeyEvent.KEYCODE_DPAD_UP -> if (down) moveSc2Focus(androidx.compose.ui.focus.FocusDirection.Up)
KeyEvent.KEYCODE_DPAD_DOWN -> if (down) moveSc2Focus(androidx.compose.ui.focus.FocusDirection.Down)
KeyEvent.KEYCODE_DPAD_LEFT -> if (down) moveSc2Focus(androidx.compose.ui.focus.FocusDirection.Left)
KeyEvent.KEYCODE_DPAD_RIGHT -> if (down) moveSc2Focus(androidx.compose.ui.focus.FocusDirection.Right)
else -> super.dispatchKeyEvent(KeyEvent(action, keyCode))
}
}
private fun moveSc2Focus(dir: androidx.compose.ui.focus.FocusDirection) {
val hook = sc2MoveFocus
if (hook == null || !hook(dir)) {
// No composition hook (shouldn't happen) — fall back to the raw key dispatch.
super.dispatchKeyEvent(KeyEvent(KeyEvent.ACTION_DOWN, dirToKey(dir)))
super.dispatchKeyEvent(KeyEvent(KeyEvent.ACTION_UP, dirToKey(dir)))
}
}
private fun dirToKey(dir: androidx.compose.ui.focus.FocusDirection): Int = when (dir) {
androidx.compose.ui.focus.FocusDirection.Up -> KeyEvent.KEYCODE_DPAD_UP
androidx.compose.ui.focus.FocusDirection.Down -> KeyEvent.KEYCODE_DPAD_DOWN
androidx.compose.ui.focus.FocusDirection.Left -> KeyEvent.KEYCODE_DPAD_LEFT
else -> KeyEvent.KEYCODE_DPAD_RIGHT
}
/** Resolve the panel's highest-refresh mode (same resolution) once, for [setConsoleHighRefreshRate]. */
private fun resolveHighRefreshMode() {
@Suppress("DEPRECATION")
@@ -315,12 +125,23 @@ class MainActivity : ComponentActivity() {
if (event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
val bit = Gamepad.buttonBit(event.keyCode)
if (bit != 0) {
// The router forwards the bit on this device's own wire pad index and tracks held
// state per pad. The emergency-exit chord (Select + Start + L1 + R1) is handled
// inside the router: holding it briefly (~1 s, with an on-screen hint) fires
// router.onExitChord (wired in StreamScreen), so a couch user with no keyboard/Back
// can still leave — but an accidental brush of the four buttons no longer quits.
gamepadRouter?.onButton(event, bit)
when (event.action) {
// repeatCount guard: don't re-send a held button as auto-repeat.
KeyEvent.ACTION_DOWN -> {
if (event.repeatCount == 0) NativeBridge.nativeSendGamepadButton(handle, bit, true)
heldPadButtons = heldPadButtons or bit
// Emergency exit: Select + Start + L1 + R1 held together leaves the stream
// (a couch user has no keyboard/Back). Fired once per full chord.
if (heldPadButtons and STREAM_EXIT_CHORD == STREAM_EXIT_CHORD) {
heldPadButtons = 0
requestStreamExit?.let { exit -> window.decorView.post { exit() } }
}
}
KeyEvent.ACTION_UP -> {
NativeBridge.nativeSendGamepadButton(handle, bit, false)
heldPadButtons = heldPadButtons and bit.inv()
}
}
return true // consumed
}
}
@@ -341,18 +162,7 @@ class MainActivity : ComponentActivity() {
// physical-keyboard layout), keycode fallback — see Keymap docs.
val vk = Keymap.toVk(event)
if (vk != 0) {
// Soft-keyboard events (the IME's virtual device — the stream's
// KeyCaptureView path) carry Shift only as META state, where a real
// keyboard sends discrete Shift transitions — so mirror the meta bit as
// a VK_LSHIFT wrap or every IME capital/symbol lands unshifted on the
// host. Never applied to hardware events: their Shift already went over
// the wire, and a synthetic release here would un-hold a physical Shift
// the user is still pressing.
val imeShift = event.deviceId == KeyCharacterMap.VIRTUAL_KEYBOARD &&
event.isShiftPressed && vk != 0xA0 && vk != 0xA1
if (down && imeShift) NativeBridge.nativeSendKey(handle, 0xA0, true, 0)
NativeBridge.nativeSendKey(handle, vk, down, 0)
if (!down && imeShift) NativeBridge.nativeSendKey(handle, 0xA0, false, 0)
return true // consumed — don't let the system also act on it
}
}
@@ -393,7 +203,7 @@ class MainActivity : ComponentActivity() {
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L) {
if (gamepadRouter?.onMotion(event) == true) return true
if (axisMapper?.onMotion(event) == true) return true
return super.dispatchGenericMotionEvent(event)
}
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
@@ -438,4 +248,9 @@ class MainActivity : ComponentActivity() {
-> true
else -> KeyEvent.isGamepadButton(kc)
}
private companion object {
/** Emergency stream-exit chord: Select + Start + L1 + R1 held together. */
val STREAM_EXIT_CHORD = Gamepad.BTN_BACK or Gamepad.BTN_START or Gamepad.BTN_LB or Gamepad.BTN_RB
}
}
@@ -1,8 +1,6 @@
package io.unom.punktfunk
import android.content.Context
import android.os.Build
import android.util.Log
import android.view.Display
/**
@@ -84,23 +82,6 @@ data class Settings(
* otherwise misfire and wait out its timeout despite the host already being reachable.
*/
val autoWakeEnabled: Boolean = true,
/**
* Opt-in: ALSO play the rumble the host addresses to controller 1 (wire pad 0) on this
* phone's own vibration motor — for clip-on gamepads that ship without rumble motors, where
* the phone body is the only actuator in the player's hands. Off by default; read once per
* session by StreamScreen (it hands GamepadFeedback the device vibrator only when set). The
* toggle is hidden on devices without a vibrator (TVs), where this would be a silent no-op.
*/
val rumbleOnPhone: Boolean = false,
/**
* Capture a Steam Controller 2 (wired / Puck dongle over USB, or an already-paired BLE pad)
* and pass it through AS-IS: the host presents a real `28DE:1302` that its Steam drives
* directly (Linux hosts). ON by default — it engages only when such a controller is actually
* present at stream start, so it costs nothing otherwise; the toggle exists for the rare
* setup where the OS-level pad (lizard mode) is preferred.
*/
val sc2Capture: Boolean = true,
)
/** [Settings.touchMode] values; persisted by name. */
@@ -161,8 +142,6 @@ class SettingsStore(context: Context) {
libraryEnabled = prefs.getBoolean(K_LIBRARY, true),
lowLatencyMode = prefs.getBoolean(K_LOW_LATENCY, true),
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
)
fun save(s: Settings) {
@@ -183,8 +162,6 @@ class SettingsStore(context: Context) {
.putBoolean(K_LIBRARY, s.libraryEnabled)
.putBoolean(K_LOW_LATENCY, s.lowLatencyMode)
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.apply()
}
@@ -220,8 +197,6 @@ class SettingsStore(context: Context) {
*/
const val K_LOW_LATENCY = "low_latency_mode_v2"
const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode"
@@ -251,25 +226,11 @@ fun nativeDisplayMode(context: Context): Triple<Int, Int, Int> {
*/
fun displaySupportsHdr(context: Context): Boolean {
val display = runCatching { context.display }.getOrNull() ?: return false
val types = buildSet {
// API 34+: the sanctioned per-mode query (Display.Mode.getSupportedHdrTypes). The
// deprecated Display-level hdrCapabilities can return EMPTY on Android 14+ devices
// (Pixel-class panels included), which would make a genuinely HDR display advertise
// no-HDR and pin the whole session to 8-bit SDR.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
display.mode.supportedHdrTypes.forEach { add(it) }
}
// Union the legacy query defensively — the supported one on minSdk 31, and some vendors
// populate only this on newer APIs.
@Suppress("DEPRECATION")
display.hdrCapabilities?.supportedHdrTypes?.forEach { add(it) }
}
// HDR10/HDR10+ only: the stream is BT.2020 PQ — a Dolby-Vision/HLG-only panel can't present it.
val supported = types.any {
@Suppress("DEPRECATION") // hdrCapabilities is the supported query on minSdk 31
val caps = display.hdrCapabilities ?: return false
return caps.supportedHdrTypes.any {
it == Display.HdrCapabilities.HDR_TYPE_HDR10 || it == Display.HdrCapabilities.HDR_TYPE_HDR10_PLUS
}
Log.i("punktfunk", "display HDR types=$types → advertise HDR10=$supported")
return supported
}
/** Resolve [Settings] (with its 0=native placeholders) to the concrete mode to request. */
@@ -69,7 +69,6 @@ import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import io.unom.punktfunk.kit.VideoDecoders
import io.unom.punktfunk.kit.deviceBodyVibrator
/**
* Stream settings, organised as an iOS-Settings / Android-system-settings style list of category
@@ -415,26 +414,6 @@ private fun ControlsSettings(s: Settings, update: (Settings) -> Unit, onOpenCont
subtitle = "What the app detects, with a live input test",
onClick = onOpenControllers,
)
// Only where the device has a body vibrator to mirror onto (a TV box doesn't).
val context = LocalContext.current
val hasBodyVibrator = remember { deviceBodyVibrator(context) != null }
if (hasBodyVibrator) {
ToggleRow(
title = "Rumble on this phone",
subtitle = "Also play controller 1's rumble on this phone's own vibration " +
"motor — for clip-on pads without rumble motors",
checked = s.rumbleOnPhone,
onCheckedChange = { on -> update(s.copy(rumbleOnPhone = on)) },
)
ToggleRow(
title = "Steam Controller 2 passthrough",
subtitle = "Capture a Steam Controller 2 (wired, Puck dongle, or paired " +
"Bluetooth): it navigates these menus and streams as-is — Steam on the " +
"host drives it like the physical pad (trackpads, gyro, haptics)",
checked = s.sc2Capture,
onCheckedChange = { on -> update(s.copy(sc2Capture = on)) },
)
}
}
}
@@ -39,7 +39,6 @@ internal fun StatsOverlay(
s: DoubleArray,
verbosity: StatsVerbosity,
decoderLabel: String = "",
codecLabel: String = "",
modifier: Modifier = Modifier,
) {
if (verbosity == StatsVerbosity.OFF || s.size < 10) return
@@ -67,7 +66,7 @@ internal fun StatsOverlay(
statLine(decoderLabel, Color(0xFFB0D0FF))
}
if (detailed) {
videoFeedLine(s, codecLabel)?.let { statLine(it, Color.White) }
videoFeedLine(s)?.let { statLine(it, Color.White) }
}
if (latValid) {
// Display stage (s[22]s[25], from OnFrameRendered): when a render timestamp landed
@@ -152,15 +151,14 @@ private fun counterLine(s: DoubleArray, lostTotal: Long): String? {
}
/**
* Format the negotiated video-feed descriptor from [codecLabel] plus the trailing four stats
* doubles `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `AV1 · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4.
* [codecLabel] is the host-resolved codec (`nativeVideoCodecLabel`); a blank one falls back to
* `HEVC` (the pre-negotiation default) for the brief window before it's resolved.
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
*/
private fun videoFeedLine(s: DoubleArray, codecLabel: String): String? {
private fun videoFeedLine(s: DoubleArray): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
@@ -177,6 +175,5 @@ private fun videoFeedLine(s: DoubleArray, codecLabel: String): String? {
2 -> "4:2:2"
else -> "4:2:0"
}
val codec = codecLabel.ifEmpty { "HEVC" }
return "$codec · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -1,35 +1,20 @@
package io.unom.punktfunk
import android.Manifest
import android.app.PendingIntent
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.hardware.usb.UsbManager
import android.net.wifi.WifiManager
import android.os.Build
import android.text.InputType
import android.util.Log
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.View
import android.view.WindowManager
import android.view.inputmethod.BaseInputConnection
import android.view.inputmethod.EditorInfo
import android.view.inputmethod.InputConnection
import android.view.inputmethod.InputMethodManager
import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
@@ -39,21 +24,17 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.core.content.ContextCompat
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.GamepadRouter
import io.unom.punktfunk.kit.deviceBodyVibrator
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.Sc2Capture
import io.unom.punktfunk.kit.VideoDecoders
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.delay
@@ -84,7 +65,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var decoderLabel by remember { mutableStateOf("") }
var codecLabel by remember { mutableStateOf("") }
var statsVerbosity by remember { mutableStateOf(initialSettings.statsVerbosity) }
val statsOn = statsVerbosity != StatsVerbosity.OFF
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
@@ -100,9 +80,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
LaunchedEffect(handle, statsOn) {
NativeBridge.nativeSetVideoStatsEnabled(handle, statsOn)
if (statsOn) {
// Codec is resolved at the handshake (Welcome) — fixed for the session, so read its
// label once up front (before the first snapshot renders the video-feed line).
if (codecLabel.isEmpty()) codecLabel = NativeBridge.nativeVideoCodecLabel(handle)
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
@@ -172,10 +149,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
}.onEach { it.setReferenceCounted(false) }
}
// True while the gamepad exit chord (Select+Start+L1+R1) is held and counting down — drives the
// "hold to quit" hint overlay. Set from the router's onExitArmed (main thread).
var exitArming by remember { mutableStateOf(false) }
DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock ->
@@ -193,12 +166,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
it.systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
it.hide(WindowInsetsCompat.Type.systemBars())
}
// The soft keyboard (three-finger swipe up → KeyCaptureView below) must OVERLAY the
// stream, never pan/resize it — the video is a fixed-mode surface, not a document.
// Scoped to the stream; the app's other screens keep the default for their text fields.
val priorSoftInput = window?.attributes?.softInputMode
?: WindowManager.LayoutParams.SOFT_INPUT_ADJUST_UNSPECIFIED
window?.setSoftInputMode(WindowManager.LayoutParams.SOFT_INPUT_ADJUST_NOTHING)
// Lock to landscape while streaming — the host streams a landscape desktop, so pin the device
// there (either landscape direction is fine) and stop it rotating to portrait mid-session. The
// activity declares configChanges=orientation, so this re-lays out the surface in place without
@@ -207,99 +174,21 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val priorOrientation = activity?.requestedOrientation
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
activity?.streamHandle = handle // route hardware keys to this session
// Multi-controller router: a stable wire pad index per connected controller, per-device axis
// state, Arrival/Remove on hot-plug, and feedback routed back by pad index. Forwards every
// controller (Automatic). Built here, released on dispose.
val router = GamepadRouter(context, handle, initialSettings.gamepad)
activity?.gamepadRouter = router
activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes
// Select+Start+L1+R1 chord leaves the stream — a deliberate quit (signal it so the host skips
// the keep-alive linger), unlike a host-ended / backgrounded drop. The router debounces it
// (must be held ~1.5 s) and fires onExitChord on its main-thread timer, so leave the stream
// the same way the Back gesture does.
// the keep-alive linger), unlike a host-ended / backgrounded drop.
activity?.requestStreamExit = { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
router.onExitChord = { activity?.requestStreamExit?.invoke() }
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed }
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad
// index via the router; poll threads stopped + joined before the router is released and the
// session closed. "Rumble on this phone" (opt-in) additionally mirrors controller 1's
// rumble onto the device's own vibrator — for clip-on pads without rumble motors.
val feedback = GamepadFeedback(
handle,
router,
deviceVibrator = if (initialSettings.rumbleOnPhone) deviceBodyVibrator(context) else null,
).also { it.start() }
// Free a disconnected controller's rumble/lights bindings promptly (else the open lights
// session leaks until the session ends). The router owns hot-plug; the feedback owns the binds.
router.onSlotClosed = feedback::onDeviceRemoved
// Steam Controller 2 as-is passthrough (opt-out): capture a wired/Puck USB pad — or an
// already-paired BLE one — and forward its raw reports; the host mirrors a real
// 28DE:1302 that its Steam drives directly, and Steam's rumble/settings writes come back
// through feedback.onHidRaw onto the physical controller. Engages only when such a pad is
// actually present; the wire slot is claimed lazily on its first state report.
// The menu-time capture (UI navigation) must let go before the stream-mode capture can
// claim the interfaces; it resumes in onDispose once the stream releases them.
activity?.stopSc2MenuNav()
val sc2 = if (initialSettings.sc2Capture) Sc2Capture(context, router) else null
var sc2UsbReceiver: BroadcastReceiver? = null
if (sc2 != null) {
feedback.onHidRaw = sc2::onHidRaw
val usbManager = context.getSystemService(Context.USB_SERVICE) as UsbManager
val usbDev = sc2.findUsbDevice()
when {
usbDev != null && usbManager.hasPermission(usbDev) -> sc2.startUsb(usbDev)
usbDev != null -> {
// One-time system dialog; capture engages on grant (Android remembers the
// grant for as long as the device stays attached).
val action = "io.unom.punktfunk.SC2_USB_PERMISSION"
val receiver = object : BroadcastReceiver() {
override fun onReceive(c: Context?, intent: Intent?) {
if (intent?.action != action) return
val ok = intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)
if (ok) sc2.startUsb(usbDev) else Log.i("punktfunk", "SC2 USB permission denied")
}
}
sc2UsbReceiver = receiver
ContextCompat.registerReceiver(
context, receiver, IntentFilter(action), ContextCompat.RECEIVER_NOT_EXPORTED,
)
usbManager.requestPermission(
usbDev,
PendingIntent.getBroadcast(
context, 0,
Intent(action).setPackage(context.packageName),
// MUTABLE: the USB stack appends the grant extras to this intent.
PendingIntent.FLAG_MUTABLE,
),
)
}
ContextCompat.checkSelfPermission(context, Manifest.permission.BLUETOOTH_CONNECT) ==
PackageManager.PERMISSION_GRANTED -> {
sc2.pairedBleAddress()?.let { addr ->
Log.i("punktfunk", "SC2: no USB pad — using the paired BLE controller $addr")
sc2.startBle(addr)
}
}
}
}
// Host→client feedback (rumble + DualSense lightbar/LEDs); poll threads stopped before close.
val feedback = GamepadFeedback(handle).also { it.start() }
onDispose {
closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it
feedback.onHidRaw = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
sc2?.stop() // release the USB/BLE link + free the wire slot (host tears the pad down)
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null
feedback.stop() // stop + join the poll threads BEFORE nativeClose frees the handle
activity?.axisMapper?.reset() // release-all so nothing sticks on the host
activity?.axisMapper = null
activity?.streamHandle = 0L
activity?.requestStreamExit = null
// Back in the menus: the SC2 (if present) resumes driving the console UI.
activity?.startSc2MenuNav()
activity?.setConsoleHighRefreshRate(true) // back to the console UI's max refresh
controller?.hide(WindowInsetsCompat.Type.ime()) // drop any keyboard left showing
window?.setSoftInputMode(priorSoftInput)
controller?.show(WindowInsetsCompat.Type.systemBars())
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
if (lowLatencyMode && Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
@@ -320,9 +209,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Focus anchor the three-finger keyboard swipe summons the IME onto (see KeyCaptureView).
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
Box(modifier = Modifier.fillMaxSize()) {
AndroidView(
modifier = Modifier.fillMaxSize(),
@@ -370,25 +256,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// BEFORE the transparent gesture layer below, so it shows through and never eats touches.
if (statsOn) {
stats?.let {
StatsOverlay(it, statsVerbosity, decoderLabel, codecLabel, Modifier.align(Alignment.TopStart).padding(12.dp))
StatsOverlay(it, statsVerbosity, decoderLabel, Modifier.align(Alignment.TopStart).padding(12.dp))
}
}
// "Hold to quit" hint while the gamepad exit chord is armed — the exit debounces on a ~1 s
// hold, so without this cue a couch user reads the (deliberately no-longer-instant) chord as
// broken. Purely visual; it sits above the video and below the gesture layer.
if (exitArming) {
ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe
// up in the mouse modes) — it never draws or takes touches, it just owns IME focus.
AndroidView(
modifier = Modifier.size(1.dp),
factory = { ctx -> KeyCaptureView(ctx).also { keyCapture = it } },
)
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
// keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a
// host-OS gesture), so intercepting three fingers would corrupt real multi-touch.
// vocabulary) or real multi-touch passthrough — see TouchInput.kt.
Box(
Modifier.fillMaxSize().pointerInput(handle, touchMode) {
when (touchMode) {
@@ -397,63 +269,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
handle,
trackpad = touchMode == TouchMode.TRACKPAD,
onCycleStats = { statsVerbosity = statsVerbosity.next() },
onKeyboard = { show -> keyCapture?.setImeVisible(show) },
)
}
},
)
}
}
/**
* The "hold to quit" cue shown while the gamepad exit chord (Select + Start + L1 + R1) is held. The
* chord no longer quits on a quick press — the router debounces it on a ~1 s hold — so this confirms
* the press registered and tells the user to keep holding. Purely visual; [GamepadRouter.onExitArmed]
* toggles its visibility.
*/
@Composable
private fun ExitChordHint(modifier: Modifier = Modifier) {
Text(
"Hold to quit…",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/**
* Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME
* onto this view. `TYPE_NULL` puts the IME in "dumb keyboard" mode — it delivers raw [KeyEvent]s
* (no composing text, no autocorrect), which flow through `MainActivity.dispatchKeyEvent` →
* `Keymap.toVk` → the host, the exact path a hardware keyboard takes. Text an IME insists on
* committing instead still arrives: the non-editable [BaseInputConnection] synthesizes KeyEvents
* for it via `KeyCharacterMap` (with Shift carried as meta state — see the IME-shift wrap in
* `MainActivity.dispatchKeyEvent`).
*/
private class KeyCaptureView(context: Context) : View(context) {
init {
isFocusable = true
isFocusableInTouchMode = true
}
override fun onCheckIsTextEditor(): Boolean = true
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
outAttrs.inputType = InputType.TYPE_NULL
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN
return BaseInputConnection(this, false)
}
fun setImeVisible(show: Boolean) {
val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
?: return
if (show) {
requestFocus()
imm.showSoftInput(this, 0)
} else {
imm.hideSoftInputFromWindow(windowToken, 0)
}
}
}
@@ -19,10 +19,6 @@ private const val TAP_SLOP = 12f
private const val TAP_DRAG_MS = 250L
private const val SCROLL_DIV = 4f
// Three-finger vertical swipe: the fraction of the view height the centroid must travel to
// summon (up) / dismiss (down) the local soft keyboard.
private const val KB_SWIPE_FRACTION = 0.10f
// Trackpad-mode pointer ballistics (relative one-finger motion). POINTER_SENS: base finger-px →
// host-px gain (~1:1, never twitchy). The rest is mild acceleration so a flick crosses the screen
// while a slow drag stays precise: above ACCEL_SPEED_FLOOR px/ms the gain ramps by ACCEL_GAIN per
@@ -44,9 +40,7 @@ private const val ACCEL_MAX = 3.0f
*
* Both share the same gesture vocabulary: tap = left click; two-finger tap = right click;
* two-finger drag = scroll; tap-then-press-and-drag = left-drag (text selection / moving
* windows); three-finger tap = [onCycleStats] (cycle the stats-HUD verbosity tier);
* three-finger swipe up/down = [onKeyboard] (summon/dismiss the local soft keyboard, for
* typing on the host).
* windows); three-finger tap = [onCycleStats] (cycle the stats-HUD verbosity tier).
*/
/**
* Real multi-touch passthrough ([TouchMode.TOUCH]): every finger forwards as a host touchscreen
@@ -100,7 +94,6 @@ internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
trackpad: Boolean,
onCycleStats: () -> Unit,
onKeyboard: (show: Boolean) -> Unit,
) {
var lastTapUp = 0L
var lastTapX = 0f
@@ -135,12 +128,6 @@ internal suspend fun PointerInputScope.streamTouchInput(
var maxFingers = 1
var scrolling = false
var scrollCount = 0 // pointer count the scroll centroid is anchored at
// Keyboard-swipe state: the 3+-finger centroid anchor (per finger count, like the
// scroll anchor) and a once-per-gesture latch.
var kbCount = 0
var kbAnchorX = 0f
var kbAnchorY = 0f
var kbFired = false
var prevCx = startX
var prevCy = startY
var upTime = down.uptimeMillis
@@ -161,12 +148,9 @@ internal suspend fun PointerInputScope.streamTouchInput(
break
}
if (pressed.size > maxFingers) maxFingers = pressed.size
// Dropping below three fingers forgets the keyboard-swipe anchor, so a 3→2→3
// bounce re-anchors instead of reading the count change as swipe travel.
if (pressed.size < 3) kbCount = 0
if (pressed.size == 2) {
// Two fingers → scroll by the centroid delta; never move the cursor.
if (pressed.size >= 2) {
// Two+ fingers → scroll by the centroid delta; never move the cursor.
val cx = (pressed.sumOf { it.position.x.toDouble() } / pressed.size).toFloat()
val cy = (pressed.sumOf { it.position.y.toDouble() } / pressed.size).toFloat()
// (Re-)anchor whenever the finger COUNT changes, not just on scroll start: the
@@ -193,36 +177,6 @@ internal suspend fun PointerInputScope.streamTouchInput(
prevCx = cx
moved = true
}
} else if (pressed.size >= 3) {
// Three+ fingers → the keyboard swipe, never scroll (the documented
// vocabulary is TWO-finger scroll; 3+ only fell into the scroll path as an
// accident of its old `>= 2` bound). Anchor the centroid per finger count
// (same reasoning as the scroll anchor above) and fire once per gesture when
// the vertical travel crosses the threshold: up = show, down = hide.
val cx = (pressed.sumOf { it.position.x.toDouble() } / pressed.size).toFloat()
val cy = (pressed.sumOf { it.position.y.toDouble() } / pressed.size).toFloat()
if (pressed.size != kbCount) {
kbCount = pressed.size
kbAnchorX = cx
kbAnchorY = cy
} else {
val dy = cy - kbAnchorY
// Real centroid travel disqualifies the tap classification below (else a
// sub-threshold swipe would still fire the three-finger stats tap).
if (abs(dy) > TAP_SLOP || abs(cx - kbAnchorX) > TAP_SLOP) moved = true
if (!kbFired && abs(dy) >= size.height * KB_SWIPE_FRACTION) {
kbFired = true
onKeyboard(dy < 0) // finger up → show, finger down → hide
}
}
// Leaving the scroll state stale would read the 3→2 centroid jump as a wheel
// notch; clearing it makes a return to two fingers re-anchor fresh. Same for
// the trackpad's tracked finger: its prev position froze while 3+ fingers were
// down, so dropping straight back to one finger must re-anchor (zero delta),
// not replay the whole 3-finger phase as one cursor jump.
scrolling = false
scrollCount = 0
trackId = PointerId(Long.MIN_VALUE)
} else if (!scrolling) {
// One finger (skipped once a gesture turned into a scroll, so dropping
// back to one finger doesn't jerk the cursor).
@@ -214,7 +214,6 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
),
verbosity = verbosity,
decoderLabel = "c2.qti.hevc.decoder · low-latency",
codecLabel = "HEVC",
modifier = Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
@@ -36,16 +36,6 @@ object Gamepad {
const val BTN_X = 0x4000
const val BTN_Y = 0x8000
// Extended bits (Moonlight `buttonFlags2 << 16` namespace — `input.rs::gamepad`): the four
// back grips (Steam L4/L5/R4/R5 ≙ Elite P1P4), touchpad click, and the misc/QAM button.
// Android's standard InputDevice path never produces these; the SC2 capture link does.
const val BTN_PADDLE1 = 0x10000
const val BTN_PADDLE2 = 0x20000
const val BTN_PADDLE3 = 0x40000
const val BTN_PADDLE4 = 0x80000
const val BTN_TOUCHPAD = 0x100000
const val BTN_MISC1 = 0x200000
// Axis ids — must equal `input.rs::gamepad::AXIS_*`.
const val AXIS_LS_X = 0
const val AXIS_LS_Y = 1
@@ -62,10 +52,6 @@ object Gamepad {
const val PREF_DUALSHOCK4 = 4
const val PREF_STEAMCONTROLLER = 5
const val PREF_STEAMDECK = 6
const val PREF_DUALSENSEEDGE = 7
const val PREF_SWITCHPRO = 8
const val PREF_STEAMCONTROLLER2 = 9
const val PREF_STEAMCONTROLLER2_PUCK = 10
// USB vendor ids of the controllers we can identify by VID/PID.
private const val VID_SONY = 0x054C
@@ -73,19 +59,10 @@ object Gamepad {
private const val VID_VALVE = 0x28DE
private const val VID_NINTENDO = 0x057E
// Sony product ids. DualSense (PS5), DualSense Edge, and DualShock 4 (PS4) map to distinct
// host pad types — the Edge's back paddles get native slots on the virtual Edge (Android
// forwards no paddle input yet, but the identity + rich planes match the physical pad).
private val PID_DUALSENSE = setOf(0x0CE6)
private val PID_DUALSENSEEDGE = setOf(0x0DF2)
// Sony product ids. DualSense (PS5) and DualShock 4 (PS4) map to distinct host pad types.
private val PID_DUALSENSE = setOf(0x0CE6, 0x0DF2)
private val PID_DUALSHOCK4 = setOf(0x05C4, 0x09CC)
// Nintendo: Switch Pro Controller — the host builds the virtual hid-nintendo pad (correct
// glyphs + positional layout). The Switch 2 Pro Controller (0x2069) and a Joy-Con 2 pair
// (0x2068) are the same full pad surface and ride the same virtual pad (SDL folds them to
// its NINTENDO_SWITCH_PRO type too).
private val PID_SWITCHPRO = setOf(0x2009, 0x2069, 0x2068)
// Valve: Steam Deck built-in controller (0x1205); classic Steam Controller wired (0x1102) /
// dongle (0x1142). The host builds the virtual hid-steam pad; rich-input capture (paddles /
// trackpads / gyro) is out of scope on Android (no rich-input plane yet), so only the standard
@@ -93,12 +70,6 @@ object Gamepad {
private val PID_STEAMDECK = setOf(0x1205)
private val PID_STEAMCONTROLLER = setOf(0x1102, 0x1142)
// Steam Controller 2: wired (0x1302), BLE (0x1303), and Puck dongles (0x1304/0x1305).
// Sc2Capture normally claims these directly; the plain InputDevice path is only a degraded
// fallback. Keep Puck distinct so even that path requests the native multi-interface identity.
private val PID_STEAMCONTROLLER2 = setOf(0x1302, 0x1303)
private val PID_STEAMCONTROLLER2_PUCK = setOf(0x1304, 0x1305)
// Microsoft Xbox One / Series product ids (wired + the common Bluetooth/dongle revisions). All
// behave like Xbox 360 on the host minus the glyph identity, so they share one pref byte.
private val PID_XBOXONE = setOf(
@@ -120,15 +91,10 @@ object Gamepad {
val pid = dev.productId
return when {
vid == VID_SONY && pid in PID_DUALSENSE -> PREF_DUALSENSE
vid == VID_SONY && pid in PID_DUALSENSEEDGE -> PREF_DUALSENSEEDGE
vid == VID_SONY && pid in PID_DUALSHOCK4 -> PREF_DUALSHOCK4
vid == VID_MICROSOFT && pid in PID_XBOXONE -> PREF_XBOXONE
vid == VID_VALVE && pid in PID_STEAMDECK -> PREF_STEAMDECK
vid == VID_VALVE && pid in PID_STEAMCONTROLLER -> PREF_STEAMCONTROLLER
vid == VID_VALVE && pid in PID_STEAMCONTROLLER2_PUCK ->
PREF_STEAMCONTROLLER2_PUCK
vid == VID_VALVE && pid in PID_STEAMCONTROLLER2 -> PREF_STEAMCONTROLLER2
vid == VID_NINTENDO && pid in PID_SWITCHPRO -> PREF_SWITCHPRO
else -> PREF_XBOX360
}
}
@@ -205,26 +171,47 @@ object Gamepad {
}
/**
* Maps one controller's joystick MotionEvents to axis (+ HAT→dpad) sends on wire pad index [pad],
* **on change only**. Holds the previous axis/hat state so an unchanged frame emits nothing. One
* instance per forwarded controller (owned by [GamepadRouter], which routes each device's events
* to its own mapper so a second pad can't clobber the first); call [reset] on that slot closing
* (disconnect / session stop) so nothing sticks on the host (which has no client-side held-state
* knowledge).
* Maps joystick MotionEvents to axis (+ HAT→dpad) sends for one session, **on change only**.
* Holds the previous axis/hat state so an unchanged frame emits nothing. One instance per
* session; call [reset] on release-all (focus loss / disconnect / session stop) so nothing
* sticks on the host (which has no client-side held-state knowledge).
*
* The router only ever feeds this a qualifying event from the mapper's own device — a real
* gamepad (its source classes include GAMEPAD), never a controller's joystick-classified sibling
* node (DualSense/DS4 motion sensors), which reports every pad axis as 0. [onMotion] therefore
* folds the event straight in without re-qualifying it.
* Single-source: only ONE qualifying controller feeds pad 0. Events must come from a device
* whose source classes include GAMEPAD (see [onMotion]) and the mapper pins itself to the
* first such device — a controller's joystick-classified sibling nodes (DualSense/DS4 motion
* sensors) and any second pad report every axis as 0, and folding them into the same state
* flapped a held trigger/stick between its value and 0 on every event interleave.
*/
class AxisMapper(private val handle: Long, private val pad: Int) {
class AxisMapper(private val handle: Long) {
// Sentinel so the first real value (incl. 0) always sends once after attach (Linux parity).
private val last = IntArray(6) { Int.MIN_VALUE }
private var hatX = 0 // -1 / 0 / +1
private var hatY = 0
/** Fold one joystick ACTION_MOVE from this mapper's controller onto its pad index. */
fun onMotion(event: MotionEvent) {
/** deviceId of the controller pad 0 is pinned to; 1 until the first qualifying event. */
private var deviceId = -1
/** Returns true if this was a joystick ACTION_MOVE we consumed. */
fun onMotion(event: MotionEvent): Boolean {
if (!event.isFromSource(InputDevice.SOURCE_JOYSTICK)) return false
if (event.actionMasked != MotionEvent.ACTION_MOVE) return false
// Only a true gamepad drives pad 0. A joystick ACTION_MOVE's own source is plain
// JOYSTICK for every sender, so qualify by the DEVICE's source classes: a real pad
// carries the GAMEPAD (button) class too, its sensor/touchpad sibling nodes and
// joystick-class remotes don't — and those report every pad axis as 0 (see the
// class doc for the held-trigger flap this caused).
val dev = event.device ?: return false
if (dev.sources and InputDevice.SOURCE_GAMEPAD != InputDevice.SOURCE_GAMEPAD) return false
// Single-pad model: pin to the first qualifying controller so a second pad (or its
// stick drift) can't fight pad 0; re-adopt only once the pinned device is gone.
if (deviceId != event.deviceId) {
if (deviceId != -1) {
if (InputDevice.getDevice(deviceId) != null) return false
reset() // the pinned pad is gone — lift its held state before adopting
}
deviceId = event.deviceId
}
// Sticks: Android floats 1..1, +y = down → ±32767, negate Y for the wire's +y = up.
sendAxis(AXIS_LS_X, stick(event.getAxisValue(MotionEvent.AXIS_X)))
sendAxis(AXIS_LS_Y, stick(-event.getAxisValue(MotionEvent.AXIS_Y)))
@@ -253,39 +240,23 @@ object Gamepad {
),
)
// HAT → dpad button transitions. Android BATCHES joystick ACTION_MOVEs, so a rapid d-pad
// tap (press+release inside one batch window) lives only in the historical samples — the
// final getAxisValue would show the HAT already back at rest and miss the tap entirely.
// Feed every historical HAT sample (oldest→newest) through the same transition logic
// before the current one, so each edge is emitted. (Sticks/triggers stay latest-wins:
// only the final value matters for an analog axis.)
for (h in 0 until event.historySize) {
applyHat(
sign(event.getHistoricalAxisValue(MotionEvent.AXIS_HAT_X, h)),
sign(event.getHistoricalAxisValue(MotionEvent.AXIS_HAT_Y, h)),
)
}
applyHat(
sign(event.getAxisValue(MotionEvent.AXIS_HAT_X)),
sign(event.getAxisValue(MotionEvent.AXIS_HAT_Y)),
)
}
/** Emit dpad button deltas for one HAT sample (`hx`/`hy` each 1/0/+1), tracking held state. */
private fun applyHat(hx: Int, hy: Int) {
// HAT → dpad button transitions (track previous, emit only the deltas).
val hx = sign(event.getAxisValue(MotionEvent.AXIS_HAT_X))
if (hx != hatX) {
if (hatX < 0) btn(BTN_DPAD_LEFT, false) else if (hatX > 0) btn(BTN_DPAD_RIGHT, false)
if (hx < 0) btn(BTN_DPAD_LEFT, true) else if (hx > 0) btn(BTN_DPAD_RIGHT, true)
hatX = hx
}
val hy = sign(event.getAxisValue(MotionEvent.AXIS_HAT_Y))
if (hy != hatY) {
if (hatY < 0) btn(BTN_DPAD_UP, false) else if (hatY > 0) btn(BTN_DPAD_DOWN, false)
if (hy < 0) btn(BTN_DPAD_UP, true) else if (hy > 0) btn(BTN_DPAD_DOWN, true)
hatY = hy
}
return true
}
/** Release-all: zero every axis and clear the held dpad (all on this mapper's pad index). */
/** Release-all: zero every axis and clear the held dpad. */
fun reset() {
for (id in 0..5) sendAxis(id, 0)
if (hatX < 0) btn(BTN_DPAD_LEFT, false) else if (hatX > 0) btn(BTN_DPAD_RIGHT, false)
@@ -297,10 +268,10 @@ object Gamepad {
private fun sendAxis(id: Int, v: Int) {
if (last[id] == v) return
last[id] = v
NativeBridge.nativeSendGamepadAxis(handle, id, v, pad)
NativeBridge.nativeSendGamepadAxis(handle, id, v)
}
private fun btn(bit: Int, down: Boolean) = NativeBridge.nativeSendGamepadButton(handle, bit, down, pad)
private fun btn(bit: Int, down: Boolean) = NativeBridge.nativeSendGamepadButton(handle, bit, down)
// 1..1 float → ±32767 i16 (matches the Apple client's 32767 scale).
private fun stick(v: Float): Int = (v.coerceIn(-1f, 1f) * 32767f).toInt()
@@ -1,6 +1,5 @@
package io.unom.punktfunk.kit
import android.content.Context
import android.graphics.Color
import android.hardware.lights.Light
import android.hardware.lights.LightState
@@ -16,102 +15,76 @@ import android.view.InputDevice
import java.nio.ByteBuffer
/**
* Host→client gamepad feedback for one session, routed per controller by wire pad index. Two daemon
* poll threads drain the blocking native pulls and render in Kotlin: rumble → the addressed
* controller's `VibratorManager` (API 31+) or its single legacy `Vibrator` on API 2830; HID-output
* → that controller's lightbar / player-LED via `LightsManager` (API 33+); adaptive triggers are
* parse-validated and logged (Android has no public adaptive-trigger API).
*
* Each pull carries the wire pad index it is addressed to; [GamepadRouter.deviceForPad] resolves it
* to the physical controller currently holding that index — so a rumble the host aimed at pad 1
* drives pad 1's motors, and an update for an index with no live controller (a pad that just
* unplugged) is dropped. Per-controller rumble/light bindings are built lazily and cached by device
* id (bounded — at most 16 pads).
* Host→client gamepad feedback for one session (single-pad model — pad 0 only). Two daemon poll
* threads drain the blocking native pulls and render in Kotlin: rumble → the controller's
* `VibratorManager` (API 31+) or its single legacy `Vibrator` on API 2830; HID-output → lightbar /
* player-LED via `LightsManager` (API 33+); adaptive
* triggers are parse-validated and logged (Android has no public adaptive-trigger API).
*
* Mirrors `nativeStartAudio`'s lifecycle: [start]/[stop] driven by the StreamScreen. [stop] flips a
* flag; the ~100 ms native pull timeout lets the threads exit, then they're joined (bounded) — and
* this MUST run before the router is released and `nativeClose` frees the session handle.
* this MUST run before `nativeClose` frees the session handle.
*
* With no controller connected (emulator) rumble/lights become logged no-ops — exactly the
* verification path; the `Log.i` receipt lines fire regardless of rendering hardware.
*
* [deviceVibrator] is the opt-in phone mirror ("Rumble on this phone", off by default): when
* non-null, rumble the host addresses to wire pad 0 (controller 1) is ALSO played on this
* device's own vibration motor — for clip-on gamepads that ship without rumble motors, where the
* phone body is the only actuator in the player's hands. StreamScreen passes it only when the
* setting is on (see [deviceBodyVibrator]).
* The active pad is resolved from the connected input devices (first gamepad/joystick). With none
* connected (emulator) rumble/lights become logged no-ops — exactly the verification path; the
* `Log.i` receipt lines fire regardless of rendering hardware.
*/
class GamepadFeedback(
private val handle: Long,
private val router: GamepadRouter?,
private val deviceVibrator: Vibrator? = null,
) {
class GamepadFeedback(private val handle: Long) {
private companion object {
const val TAG = "pf.feedback"
const val TAG_LED: Byte = 0x01
const val TAG_PLAYER_LEDS: Byte = 0x02
const val TAG_TRIGGER: Byte = 0x03
const val TAG_HID_RAW: Byte = 0x05
// Fallback one-shot duration against a legacy host (no v2 TTL lease): the prior fixed value.
// A new host renews far below this, so it never actually holds this long there.
const val LEGACY_RUMBLE_MS = 60_000L
}
/** One controller's rumble binding — VibratorManager (API 31+) OR the legacy single Vibrator (API 2830). */
private class RumbleBind(
val vm: VibratorManager?,
val legacy: Vibrator?,
val ids: IntArray,
val amplitudeControlled: Boolean,
)
/** One controller's lights binding (API 33+): its open session + the RGB / player-id lights it exposes. */
private class LightBind(
val session: LightsManager.LightsSession,
val rgb: Light?,
val player: Light?,
)
@Volatile private var running = false
private var rumbleThread: Thread? = null
private var hidoutThread: Thread? = null
// Per-controller bindings, keyed by device id, built lazily. rumbleBinds is written by the rumble
// thread and lightBinds by the hidout thread while running; [onDeviceRemoved] also evicts+closes
// from the MAIN thread on a hot-unplug, and stop() clears both from the main thread after joining
// the threads. That main-vs-poll concurrency is why every access goes through `bindsLock` (a plain
// HashMap can corrupt under a concurrent structural write, and ConcurrentHashMap can't hold the
// null value that caches "this controller has no vibrator / no controllable lights"). The lock
// guards only the map ops — rendering runs on the returned reference outside it; a stale reference
// is harmless (a closed LightsSession's requestLights and a cancelled Vibrator are runCatching'd
// no-ops). A null value caches the negative result so a pad with no hardware isn't re-probed.
private val bindsLock = Any()
private val rumbleBinds = HashMap<Int, RumbleBind?>()
private val lightBinds = HashMap<Int, LightBind?>()
private var vm: VibratorManager? = null
// API 2830 fallback: the controller's single legacy Vibrator (no per-motor VibratorManager
// until API 31). Exactly one of [vm] / [legacy] is bound; rumble degrades to one blended motor.
private var legacy: Vibrator? = null
private var vibratorIds: IntArray = IntArray(0)
private var amplitudeControlled = false
private var lightsSession: LightsManager.LightsSession? = null
private var rgbLight: Light? = null
private var playerLight: Light? = null
fun start() {
val dev = resolvePad()
bindRumble(dev)
if (Build.VERSION.SDK_INT >= 33) {
bindLights(dev)
} else {
Log.i(TAG, "lights need API 33 (have ${Build.VERSION.SDK_INT}) — lightbar/playerLed no-op")
}
running = true
rumbleThread = Thread({
while (running) {
val ev = NativeBridge.nativeNextRumble(handle)
if (ev < 0L) continue // timeout / closed
// ev bits 49..52 = wire pad index; bits 32..47 = backstop duration (ms);
// 16..31 = low; 0..15 = high. These are EFFECTIVE commands from the core's shared
// rumble policy engine — it owns every lease/staleness/close decision (uniform
// across all clients; the old 60 s legacy-host exposure is gone) and emits
// explicit zeros, so apply verbatim: (0, 0) = cancel, non-zero = one-shot for
// the backstop (the hardware net under a stalled poll thread).
val pad = ((ev ushr 49) and 0xFL).toInt()
val backstopMs = ((ev ushr 32) and 0xFFFF)
// ev bit 48 = has a v2 lease; bits 32..47 = ttl_ms; 16..31 = low; 0..15 = high. The
// lease flag is out-of-band, so any ttl_ms (incl. 0xFFFF) is a real lease — no
// in-band sentinel. No lease (legacy host) → the prior long one-shot.
val hasLease = ((ev ushr 48) and 0x1L) == 0x1L
val ttl = ((ev ushr 32) and 0xFFFF).toInt()
val durationMs = if (hasLease) ttl.toLong() else LEGACY_RUMBLE_MS
renderRumble(
pad,
((ev ushr 16) and 0xFFFF).toInt(),
(ev and 0xFFFF).toInt(),
backstopMs,
durationMs,
)
}
}, "pf-rumble").apply { isDaemon = true; start() }
hidoutThread = Thread({
// 128: the raw as-is passthrough events are [pad][kind tag][report kind][≤64 bytes].
val buf = ByteBuffer.allocateDirect(128)
val buf = ByteBuffer.allocateDirect(64)
while (running) {
val n = NativeBridge.nativeNextHidout(handle, buf)
if (n < 0) continue // timeout / closed
@@ -120,137 +93,100 @@ class GamepadFeedback(
}, "pf-hidout").apply { isDaemon = true; start() }
}
/** Idempotent. Stops + joins the poll threads (must complete before the router is released / handle freed). */
/** Idempotent. Stops + joins the poll threads (must complete before the session handle is freed). */
fun stop() {
running = false
rumbleThread?.interrupt()
hidoutThread?.interrupt()
runCatching { vm?.cancel() } // drop any held rumble immediately
runCatching { legacy?.cancel() }
// Join WITHOUT a timeout. These poll threads dereference the native session handle on every
// pull (nativeNextRumble/nativeNextHidout) and read the router, so they MUST be dead before
// StreamScreen's onDispose reaches router.release() / nativeClose, which free that state. A
// *bounded* join that times out would let a thread survive into the freed handle → use-after-
// free SIGSEGV (the back-while-streaming crash, on the one path the main-thread `closed` guard
// can't cover). Safe to block unbounded: the native pulls are internally time-bounded
// (PULL_TIMEOUT ~100 ms) and rendering is a quick best-effort binder call, so each thread
// observes running=false and exits within ~one timeout — the join returns promptly.
// pull (nativeNextRumble/nativeNextHidout), so they MUST be dead before StreamScreen's
// onDispose reaches nativeClose, which frees that handle. A *bounded* join that times out
// would let a thread survive into the freed handle → use-after-free SIGSEGV (the
// back-while-streaming crash, on the one path the main-thread `closed` guard can't cover).
// Safe to block unbounded: the native pulls are internally time-bounded (PULL_TIMEOUT ~100 ms)
// and rendering is a quick best-effort binder call, so each thread observes running=false and
// exits within ~one timeout — the join returns promptly (well under any ANR threshold).
runCatching { rumbleThread?.join() }
runCatching { hidoutThread?.join() }
rumbleThread = null
hidoutThread = null
// Threads are dead — drop any held rumble (incl. the phone mirror's) and close every
// lights session.
runCatching { deviceVibrator?.cancel() }
synchronized(bindsLock) {
for (b in rumbleBinds.values) b?.let {
runCatching { it.vm?.cancel() }
runCatching { it.legacy?.cancel() }
}
for (b in lightBinds.values) b?.let { runCatching { it.session.close() } }
rumbleBinds.clear()
lightBinds.clear()
}
runCatching { lightsSession?.close() }
lightsSession = null
rgbLight = null
playerLight = null
vm = null
legacy = null
vibratorIds = IntArray(0)
}
/**
* Evict and release the bindings for a controller that just disconnected — invoked from
* [GamepadRouter]'s slot-close on the main thread (routed via `StreamScreen`). Closes its
* `LightsSession` and cancels any held rumble, so a hot-unplug mid-session frees the session
* immediately instead of leaking it until [stop]. A no-op for a device with no cached binding.
* The next feedback for that pad index rebinds against whatever controller now holds it.
*/
// Same runtime-guarded cleanup as [stop] (VIBRATE is app-declared; the light bind only exists
// under the SDK 33 guard) — suppress the module-isolation lint false positives it re-triggers.
@Suppress("MissingPermission", "NewApi")
fun onDeviceRemoved(deviceId: Int) {
synchronized(bindsLock) {
rumbleBinds.remove(deviceId)?.let {
runCatching { it.vm?.cancel() }
runCatching { it.legacy?.cancel() }
}
lightBinds.remove(deviceId)?.let { runCatching { it.session.close() } }
}
}
/** First connected gamepad/joystick InputDevice, or null (→ logged no-op on the emulator). */
private fun resolvePad(): InputDevice? = Gamepad.firstPad()
// ---- Rumble ----
/** The rumble binding for the controller on wire pad [pad], or null (no live pad / no vibrator). Cached by device id. */
private fun rumbleBindFor(pad: Int): RumbleBind? {
val dev = router?.deviceForPad(pad) ?: return null
synchronized(bindsLock) {
if (rumbleBinds.containsKey(dev.id)) return rumbleBinds[dev.id]
val bind = bindRumble(dev)
rumbleBinds[dev.id] = bind
return bind
private fun bindRumble(dev: InputDevice?) {
if (dev == null) {
Log.i(TAG, "rumble: no controller connected — rumble no-op (emulator path)")
return
}
}
private fun bindRumble(dev: InputDevice): RumbleBind? {
if (Build.VERSION.SDK_INT >= 31) {
val m = dev.vibratorManager
val ids = m.vibratorIds
if (ids.isEmpty()) {
Log.i(TAG, "rumble: controller '${dev.name}' has no vibrators — rumble no-op")
return null
return
}
val amp = ids.all { m.getVibrator(it).hasAmplitudeControl() }
Log.i(TAG, "rumble: bound ${ids.size} vibrators for '${dev.name}' amplitudeControl=$amp")
return RumbleBind(m, null, ids, amp)
vm = m
vibratorIds = ids
amplitudeControlled = ids.all { m.getVibrator(it).hasAmplitudeControl() }
Log.i(TAG, "rumble: bound ${ids.size} vibrators amplitudeControl=$amplitudeControlled")
} else {
// API 2830: no VibratorManager — fall back to the controller's single legacy Vibrator.
@Suppress("DEPRECATION")
val v = dev.vibrator
if (!v.hasVibrator()) {
Log.i(TAG, "rumble: controller '${dev.name}' has no vibrator — rumble no-op")
return
}
legacy = v
amplitudeControlled = v.hasAmplitudeControl()
Log.i(TAG, "rumble: bound legacy vibrator amplitudeControl=$amplitudeControlled")
}
// API 2830: no VibratorManager — fall back to the controller's single legacy Vibrator.
@Suppress("DEPRECATION")
val v = dev.vibrator
if (!v.hasVibrator()) {
Log.i(TAG, "rumble: controller '${dev.name}' has no vibrator — rumble no-op")
return null
}
Log.i(TAG, "rumble: bound legacy vibrator for '${dev.name}' amplitudeControl=${v.hasAmplitudeControl()}")
return RumbleBind(null, v, IntArray(0), v.hasAmplitudeControl())
}
/**
* low = heavy/left motor, high = light/right motor; both 0..0xFFFF (the host's u16 amplitudes),
* addressed to wire pad [pad]. `durationMs` is the engine command's backstop — the one-shot's
* self-termination net under a stalled poll thread; the engine emits explicit zero commands at
* every policy stop (lease expiry, legacy staleness, session close), so cancel-on-zero is the
* real stop mechanism.
* low = heavy/left motor, high = light/right motor; both 0..0xFFFF (the host's u16 amplitudes).
* `durationMs` is the host's v2 envelope TTL — the one-shot self-terminates after it unless the
* host renews, so a lost stop (or a dead host) silences at the lease instead of the old fixed
* 60 s. Against a legacy host it is [LEGACY_RUMBLE_MS] (the prior fixed duration).
*/
private fun renderRumble(pad: Int, low: Int, high: Int, durationMs: Long) {
Log.i(TAG, "rumble pad=$pad low=$low high=$high backstopMs=$durationMs") // verification line — BEFORE any no-op return
// Opt-in phone mirror, BEFORE the controller-bind early-return: the exact pads this
// serves have no vibrator of their own, so their bind below is null. It follows
// controller 1 unconditionally rather than only motor-less pads — capability probing
// already decided the bind, and the user opted in.
if (pad == 0) renderDeviceRumble(low, high, durationMs)
val bind = rumbleBindFor(pad) ?: return
private fun renderRumble(low: Int, high: Int, durationMs: Long) {
Log.i(TAG, "rumble low=$low high=$high ttlMs=$durationMs") // verification line — BEFORE any no-op return
val lo = toAmplitude(low)
val hi = toAmplitude(high)
val m = bind.vm
val m = vm
if (m != null) {
if (lo == 0 && hi == 0) {
m.cancel() // (0,0) = stop
return
}
val combo = CombinedVibration.startParallel()
if (bind.amplitudeControlled && bind.ids.size >= 2) {
// Two-motor split — ASSUMPTION: ids[0] = light/right, ids[1] = heavy/left
// (XInput/Moonlight convention). Android does not guarantee the order of
// VibratorManager.getVibratorIds(), so a pad that enumerates heavy-first would
// invert the feel: the stronger amplitude drives the physically-lighter motor.
// Failure mode is tactile only — both motors still fire, nothing silences or
// crashes — so this stays the default pending per-pad on-glass verification (G20).
// ids beyond the first two (rare) are left alone here.
if (hi != 0) combo.addVibrator(bind.ids[0], oneShot(hi, durationMs))
if (lo != 0) combo.addVibrator(bind.ids[1], oneShot(lo, durationMs))
if (amplitudeControlled && vibratorIds.size >= 2) {
// ids[0] = light/right, ids[1] = heavy/left (XInput/Moonlight convention).
if (hi != 0) combo.addVibrator(vibratorIds[0], oneShot(hi, durationMs))
if (lo != 0) combo.addVibrator(vibratorIds[1], oneShot(lo, durationMs))
} else {
// Single motor or no amplitude control: blend both into one effect.
val a = (lo * 0.8 + hi * 0.33).toInt().coerceIn(1, 255)
for (id in bind.ids) combo.addVibrator(id, oneShot(a, durationMs))
for (id in vibratorIds) combo.addVibrator(id, oneShot(a, durationMs))
}
runCatching { m.vibrate(combo.combine()) }
return
}
// API 2830 legacy single-motor path: blend both motors into one effect.
val lv = bind.legacy ?: return
val lv = legacy ?: return
if (lo == 0 && hi == 0) {
lv.cancel() // (0,0) = stop
return
@@ -258,30 +194,7 @@ class GamepadFeedback(
val a = (lo * 0.8 + hi * 0.33).toInt().coerceIn(1, 255)
runCatching {
lv.vibrate(
if (bind.amplitudeControlled) oneShot(a, durationMs)
else oneShot(VibrationEffect.DEFAULT_AMPLITUDE, durationMs)
)
}
}
/**
* The opt-in phone mirror: play a wire-pad-0 rumble on this device's own vibration motor —
* one physical actuator, so both wire motors blend into one effect (the same blend as the
* single-motor controller path). Same envelope semantics too: a one-shot held for the host's
* TTL, cancel on (0,0).
*/
private fun renderDeviceRumble(low: Int, high: Int, durationMs: Long) {
val v = deviceVibrator ?: return
val lo = toAmplitude(low)
val hi = toAmplitude(high)
if (lo == 0 && hi == 0) {
runCatching { v.cancel() } // (0,0) = stop
return
}
val a = (lo * 0.8 + hi * 0.33).toInt().coerceIn(1, 255)
runCatching {
v.vibrate(
if (v.hasAmplitudeControl()) oneShot(a, durationMs)
if (amplitudeControlled) oneShot(a, durationMs)
else oneShot(VibrationEffect.DEFAULT_AMPLITUDE, durationMs)
)
}
@@ -294,69 +207,42 @@ class GamepadFeedback(
}
// One-shot held for `durationMs` — the host's v2 TTL (renewed while the level holds), so it
// self-terminates on a lost stop; cancel on zero. Floor the duration at 1 ms: `createOneShot`
// throws IllegalArgumentException on a non-positive duration, and a lease can carry ttl_ms==0
// (e.g. the legacy-Deck ceiling) with a nonzero amplitude — which reaches here past the (0,0)
// stop guard. On the VibratorManager path the effect is built OUTSIDE the vibrate() runCatching,
// so an uncaught throw here would kill the whole rumble poll thread.
// self-terminates on a lost stop; cancel on zero.
private fun oneShot(amp: Int, durationMs: Long): VibrationEffect =
VibrationEffect.createOneShot(durationMs.coerceAtLeast(1), amp)
VibrationEffect.createOneShot(durationMs, amp)
// ---- HID output ----
private fun dispatchHidout(buf: ByteBuffer, n: Int) {
buf.rewind()
val pad = buf.get().toInt() and 0xFF // wire pad index the event is addressed to
when (buf.get()) { // kind tag
TAG_LED -> {
val r = buf.get().toInt() and 0xFF
val g = buf.get().toInt() and 0xFF
val b = buf.get().toInt() and 0xFF
Log.i(TAG, "hidout pad=$pad Led r=$r g=$g b=$b") // verification line
if (Build.VERSION.SDK_INT >= 33) setLightbar(pad, Color.rgb(r, g, b))
Log.i(TAG, "hidout Led r=$r g=$g b=$b") // verification line
if (Build.VERSION.SDK_INT >= 33) setLightbar(Color.rgb(r, g, b))
}
TAG_PLAYER_LEDS -> {
val bits = buf.get().toInt() and 0x1F
val player = playerIndexForBits(bits)
Log.i(TAG, "hidout pad=$pad PlayerLeds bits=$bits player=$player") // verification line
if (Build.VERSION.SDK_INT >= 33) setPlayerId(pad, player)
Log.i(TAG, "hidout PlayerLeds bits=$bits player=$player") // verification line
if (Build.VERSION.SDK_INT >= 33) setPlayerId(player)
}
TAG_TRIGGER -> {
val which = buf.get().toInt() and 0xFF // 0 = L2, 1 = R2
val effLen = n - 3 // [pad][kind][which] header, then the effect block
val effLen = n - 2
val mode = if (effLen > 0) buf.get().toInt() and 0xFF else 0
// No public adaptive-trigger API on Android — parse-validate the mode + log only.
Log.i(
TAG,
"hidout pad=$pad Trigger which=$which effLen=$effLen mode=0x%02x (adaptive triggers unsupported on Android)".format(mode),
"hidout Trigger which=$which effLen=$effLen mode=0x%02x (adaptive triggers unsupported on Android)".format(mode),
)
}
TAG_HID_RAW -> {
// As-is SC2 passthrough: a raw report the host's Steam wrote to the virtual pad —
// [kind: 0=output, 1=feature][report bytes, id first]. Handed to the capture link
// for verbatim replay on the physical controller; dropped when no link owns the pad.
val kind = buf.get().toInt() and 0xFF
val len = n - 3
if (len > 0) {
val data = ByteArray(len)
buf.get(data)
onHidRaw?.invoke(pad, kind, data)
}
}
else -> Log.d(TAG, "hidout: unknown kind, dropped")
}
}
/**
* Raw HID-report replay hook for the as-is Steam Controller 2 passthrough: invoked (on the
* hidout poll thread) with the wire pad index, the report kind (0 = output report, 1 =
* feature report), and the full report bytes (id first) the host's hidraw consumer wrote.
* `StreamScreen` wires this to the SC2 capture so Steam's rumble/settings land on the
* physical controller.
*/
@Volatile
var onHidRaw: ((pad: Int, kind: Int, data: ByteArray) -> Unit)? = null
/** hid-playstation 5-LED pattern → player index 1..4 (0 = off); falls back to a bit count. */
private fun playerIndexForBits(bits: Int): Int = when (bits and 0x1F) {
0b00000 -> 0
@@ -367,63 +253,37 @@ class GamepadFeedback(
else -> Integer.bitCount(bits and 0x1F).coerceIn(1, 4)
}
/** The lights binding for the controller on wire pad [pad], or null (no live pad / no lights / < API 33). Cached by device id. */
private fun lightBindFor(pad: Int): LightBind? {
if (Build.VERSION.SDK_INT < 33) return null
val dev = router?.deviceForPad(pad) ?: return null
synchronized(bindsLock) {
if (lightBinds.containsKey(dev.id)) return lightBinds[dev.id]
val bind = bindLights(dev)
lightBinds[dev.id] = bind
return bind
private fun bindLights(dev: InputDevice?) {
if (dev == null) {
Log.i(TAG, "lights: no controller connected — lightbar/playerLed no-op (emulator path)")
return
}
}
private fun bindLights(dev: InputDevice): LightBind? {
val lm = dev.lightsManager
var rgb: Light? = null
var player: Light? = null
for (l in lm.lights) {
if (rgb == null && l.hasRgbControl()) rgb = l
if (player == null && l.type == Light.LIGHT_TYPE_PLAYER_ID) player = l
if (rgbLight == null && l.hasRgbControl()) rgbLight = l
if (playerLight == null && l.type == Light.LIGHT_TYPE_PLAYER_ID) playerLight = l
}
if (rgb == null && player == null) {
if (rgbLight == null && playerLight == null) {
Log.i(TAG, "lights: controller '${dev.name}' exposes no controllable lights — no-op")
return null
return
}
val session = lm.openSession()
Log.i(TAG, "lights: bound rgb=${rgb != null} playerLed=${player != null} for '${dev.name}'")
return LightBind(session, rgb, player)
lightsSession = lm.openSession()
Log.i(TAG, "lights: bound rgb=${rgbLight != null} playerLed=${playerLight != null}")
}
private fun setLightbar(pad: Int, argb: Int) {
val bind = lightBindFor(pad) ?: return
val l = bind.rgb ?: return
private fun setLightbar(argb: Int) {
val s = lightsSession ?: return
val l = rgbLight ?: return
runCatching {
bind.session.requestLights(LightsRequest.Builder().addLight(l, LightState.Builder().setColor(argb).build()).build())
s.requestLights(LightsRequest.Builder().addLight(l, LightState.Builder().setColor(argb).build()).build())
}
}
private fun setPlayerId(pad: Int, player: Int) {
val bind = lightBindFor(pad) ?: return
val l = bind.player ?: return
private fun setPlayerId(player: Int) {
val s = lightsSession ?: return
val l = playerLight ?: return
runCatching {
bind.session.requestLights(LightsRequest.Builder().addLight(l, LightState.Builder().setPlayerId(player).build()).build())
s.requestLights(LightsRequest.Builder().addLight(l, LightState.Builder().setPlayerId(player).build()).build())
}
}
}
/**
* This device's own body vibrator (the phone, not a controller), or null where there is none
* (TVs) — gates the "Rumble on this phone" setting's visibility and feeds
* [GamepadFeedback.deviceVibrator] when it's on.
*/
fun deviceBodyVibrator(context: Context): Vibrator? {
val v = if (Build.VERSION.SDK_INT >= 31) {
context.getSystemService(VibratorManager::class.java)?.defaultVibrator
} else {
@Suppress("DEPRECATION")
context.getSystemService(Context.VIBRATOR_SERVICE) as? Vibrator
}
return v?.takeIf { it.hasVibrator() }
}
@@ -1,329 +0,0 @@
package io.unom.punktfunk.kit
import android.content.Context
import android.hardware.input.InputManager
import android.os.Handler
import android.os.Looper
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
import java.util.concurrent.ConcurrentHashMap
/**
* Multi-controller router for one stream session — the Android analogue of the Linux client's gamepad
* `Worker`/`Slot` model (`pf-client-core/src/gamepad.rs`) over the shared native-plane wire contract
* (`punktfunk-core/src/input.rs`). Each physical controller (Android `deviceId`) gets a STABLE
* lowest-free wire pad index (0..15) held for its lifetime and freed only on disconnect, so a pad
* dropping never renumbers the others (a game must not see its players shuffle). Every forwarded event
* carries that pad index; a [NativeBridge.nativeSendGamepadArrival] declaring the pad's type is sent
* once BEFORE its first input, a [NativeBridge.nativeSendGamepadRemove] on disconnect. Per-device axis
* state lives in each slot's [Gamepad.AxisMapper] so a second controller can't clobber the first.
* Feedback (rumble / HID) is routed BACK to the originating device by pad index via [deviceForPad].
*
* Selection: forward EVERY real controller (the Linux client's single-player pin has no Android UI
* surface yet — Automatic is the only mode). Lifetime matches the session: constructed on stream
* attach (opening a slot for every already-connected pad, so its Arrival lands before any input),
* released on detach.
*
* A single controller lands on wire index 0, so its per-transition button/axis wire is byte-identical
* to the old single-pad path (plus the Arrival/Remove declarations the contract requires — which an
* older host simply ignores).
*
* Threading: slot mutation + dispatch run on the main thread (Android input dispatch and the
* InputManager hot-plug callbacks both land there). [deviceForPad] is read from the feedback poll
* threads, so the slot table is a [ConcurrentHashMap].
*/
class GamepadRouter(context: Context, private val handle: Long, private val setting: Int) {
/** One forwarded controller: its stable wire pad index, per-device axis state, and held buttons. */
private class Slot(val index: Int, val mapper: Gamepad.AxisMapper) {
/** Forwarded button bits currently held (Gamepad.BTN_*) — for release-on-close + chord detection. */
var held = 0
}
/** deviceId → slot. Concurrent: the feedback poll threads read it via [deviceForPad]. */
private val slots = ConcurrentHashMap<Int, Slot>()
/**
* Invoked (main thread) with the deviceId whenever a slot closes — hot-unplug or session teardown.
* `StreamScreen` wires this to `GamepadFeedback.onDeviceRemoved` so a disconnected pad's rumble /
* lights bindings are released promptly instead of leaking until the feedback threads stop.
*/
var onSlotClosed: ((deviceId: Int) -> Unit)? = null
/**
* Invoked (main thread) when the emergency-exit chord has been HELD for [EXIT_HOLD_MS] — the caller
* leaves the stream. `StreamScreen` wires this to the deliberate-quit exit.
*/
var onExitChord: (() -> Unit)? = null
/**
* Invoked (main thread) with `true` the moment the exit chord completes and the hold countdown
* starts, and `false` when it's cancelled (a button lifted early) or the timer elapses. `StreamScreen`
* wires this to a "hold to quit" hint so the hold is discoverable — the chord no longer quits on a
* quick press, and without an on-screen cue that reads as the shortcut being broken.
*/
var onExitArmed: ((armed: Boolean) -> Unit)? = null
private val mainHandler = Handler(Looper.getMainLooper())
/** The pending exit-chord hold timer, or null when the chord isn't currently armed. */
private var pendingExit: Runnable? = null
private val inputManager = context.getSystemService(InputManager::class.java)
private val listener = object : InputManager.InputDeviceListener {
override fun onInputDeviceAdded(deviceId: Int) {
InputDevice.getDevice(deviceId)?.let { if (isForwardable(it)) openSlot(it) }
}
override fun onInputDeviceRemoved(deviceId: Int) = closeSlot(deviceId)
override fun onInputDeviceChanged(deviceId: Int) {}
}
init {
inputManager?.registerInputDeviceListener(listener, mainHandler)
// Open a slot for every controller already connected when the session starts — the pads that
// will never fire onInputDeviceAdded during this session; their Arrival lands before any input.
for (id in InputDevice.getDeviceIds()) {
InputDevice.getDevice(id)?.let { if (isForwardable(it)) openSlot(it) }
}
}
/**
* One gamepad button transition for the device that produced [event] (already resolved to BTN_*
* bit [bit]). Opens the device's slot (declaring its type) if unseen, forwards the bit on the
* slot's pad index, and tracks held state. Completing the emergency stream-exit chord (Select +
* Start + L1 + R1) on any one pad ARMS a [EXIT_HOLD_MS] hold timer rather than leaving instantly
* ([onExitArmed] fires so the UI can show a "hold to quit" hint); [onExitChord] fires only if the
* chord is still held at expiry (a brief accidental brush is ignored), matching `DISCONNECT_HOLD`
* on the SDL/Apple clients. Any controller can leave.
*/
fun onButton(event: KeyEvent, bit: Int) {
val slot = slotFor(event.device) ?: return
when (event.action) {
// repeatCount guard: don't re-send a held button as auto-repeat.
KeyEvent.ACTION_DOWN -> slotButton(slot, bit, down = true, send = event.repeatCount == 0)
KeyEvent.ACTION_UP -> slotButton(slot, bit, down = false, send = true)
}
}
/**
* One button transition on [slot] — the shared body behind [onButton] and an [ExternalPad]'s
* transitions: forward the wire event, track held state, and arm/disarm the exit chord.
*/
private fun slotButton(slot: Slot, bit: Int, down: Boolean, send: Boolean) {
if (down) {
if (send) NativeBridge.nativeSendGamepadButton(handle, bit, true, slot.index)
slot.held = slot.held or bit
// Full chord now held on this pad → start the hold countdown (idempotent while held).
if (slot.held and EXIT_CHORD == EXIT_CHORD) armExit()
} else {
if (send) NativeBridge.nativeSendGamepadButton(handle, bit, false, slot.index)
slot.held = slot.held and bit.inv()
// A chord button lifted before the hold elapsed → cancel, unless another pad still
// holds the full chord.
if (bit and EXIT_CHORD != 0 && slots.values.none { it.held and EXIT_CHORD == EXIT_CHORD }) {
disarmExit()
}
}
}
/** Arm the exit-chord hold timer (once); on expiry, if the chord is still held, flush + leave. */
private fun armExit() {
if (pendingExit != null) return // already counting down
val r = Runnable {
pendingExit = null
onExitArmed?.invoke(false) // countdown over — drop the hint whether or not we leave
// Fire only if the chord survived the full hold on some pad.
val held = slots.values.filter { it.held and EXIT_CHORD == EXIT_CHORD }
if (held.isNotEmpty()) {
// Release the held buttons + zero the axes on every triggering pad so nothing sticks
// host-side once we leave, then signal the deliberate exit.
for (s in held) releaseHeld(s)
onExitChord?.invoke()
}
}
pendingExit = r
mainHandler.postDelayed(r, EXIT_HOLD_MS)
onExitArmed?.invoke(true) // chord complete → show the "hold to quit" hint
}
/** Cancel a pending exit-chord hold timer. */
private fun disarmExit() {
val wasArmed = pendingExit != null
pendingExit?.let { mainHandler.removeCallbacks(it) }
pendingExit = null
if (wasArmed) onExitArmed?.invoke(false) // released early — drop the hint
}
/**
* One joystick MotionEvent — routed to the producing device's own [Gamepad.AxisMapper] (per-device
* state). Returns true if consumed. Only a real gamepad drives a pad: a DualSense/DS4 motion-sensor
* sibling node classifies as bare joystick (no GAMEPAD source class) and reports every pad axis as
* 0, so [isForwardable] filters it out before it can open a slot or clobber axes.
*/
fun onMotion(event: MotionEvent): Boolean {
if (!event.isFromSource(InputDevice.SOURCE_JOYSTICK)) return false
if (event.actionMasked != MotionEvent.ACTION_MOVE) return false
val dev = event.device ?: return false
if (!isForwardable(dev)) return false
val slot = slotFor(dev) ?: return false
slot.mapper.onMotion(event)
return true
}
/**
* The controller currently mapped to wire pad [pad], for feedback routing; null if that index
* holds no live slot (a pad that just unplugged — the update is then dropped) OR the slot is
* an [ExternalPad] (its synthetic id resolves to no InputDevice, so rumble binds naturally
* fall through to the capture link's own feedback path). Read from the feedback poll threads.
*/
fun deviceForPad(pad: Int): InputDevice? {
for ((deviceId, slot) in slots) {
if (slot.index == pad) return InputDevice.getDevice(deviceId)
}
return null
}
/**
* A capture-link pad occupying a wire slot without an Android [InputDevice] — the as-is Steam
* Controller 2 passthrough (USB/BLE claimed directly, invisible to the input stack). Shares
* the real slots' lifecycle: a stable lowest-free index, Arrival-before-input, held-state
* flush + Remove on [close], and full participation in the emergency exit chord.
*/
inner class ExternalPad internal constructor(private val syntheticId: Int, val index: Int) {
// Live lookup instead of a captured reference: after [close] (or a router release) the
// slot is gone from the table and every entry point below degrades to a safe no-op.
private val slot get() = slots[syntheticId]
/** One button transition (a wire [Gamepad].BTN_* bit). On-change only — the caller diffs. */
fun button(bit: Int, down: Boolean) {
slot?.let { slotButton(it, bit, down, send = true) }
}
/** One axis update ([Gamepad].AXIS_*: stick i16 +y=up / trigger 0..255). On-change only. */
fun axis(id: Int, value: Int) {
if (slot != null) NativeBridge.nativeSendGamepadAxis(handle, id, value, index)
}
/** One raw HID report, forwarded verbatim for the host's as-is virtual pad. */
fun hidReport(buf: java.nio.ByteBuffer, len: Int) {
if (slot != null) NativeBridge.nativeSendPadHidReport(handle, index, buf, len)
}
/** Flush held state, signal the removal, and free the wire index. Idempotent. */
fun close() = closeSlot(syntheticId)
}
/**
* Open a slot for a capture-link pad, declaring [pref] as its kind; null when all 16 wire
* indices are taken. Main thread (like the hot-plug callbacks).
*/
fun openExternal(pref: Int): ExternalPad? {
val index = lowestFreeIndex() ?: return null
// Synthetic ids live below any real InputDevice id (those are positive), so they can't
// collide and InputDevice.getDevice(id) resolves them to null for the feedback path.
val syntheticId = EXTERNAL_ID_BASE - index
NativeBridge.nativeSendGamepadArrival(handle, pref, index)
slots[syntheticId] = Slot(index, Gamepad.AxisMapper(handle, index))
return ExternalPad(syntheticId, index)
}
/**
* Flush + drop every slot and unregister the hot-plug listener. Call on session teardown, AFTER
* the feedback poll threads are joined (they read [deviceForPad]).
*/
fun release() {
inputManager?.unregisterInputDeviceListener(listener)
disarmExit() // drop any pending exit-chord timer so it can't fire after teardown
// Snapshot the ids first — closeSlot mutates the map.
for (id in slots.keys.toList()) closeSlot(id)
}
// ---- slots ----
/** A real, non-virtual controller we forward — its source classes include GAMEPAD (excludes a pad's bare-joystick sensor node). */
private fun isForwardable(dev: InputDevice): Boolean =
!dev.isVirtual && dev.sources and InputDevice.SOURCE_GAMEPAD == InputDevice.SOURCE_GAMEPAD
/**
* The slot for [dev], opening one (and declaring the pad) if this device is unseen; null when [dev]
* isn't a forwardable controller or every wire index is taken. The [isForwardable] gate lives here —
* the single lazy-open chokepoint both [onButton] and [onMotion] funnel through — so no entry point
* can open a phantom slot for a virtual/non-gamepad source (the hot-plug listener and init loop
* pre-filter and call [openSlot] directly).
*/
private fun slotFor(dev: InputDevice?): Slot? {
if (dev == null) return null
slots[dev.id]?.let { return it }
if (!isForwardable(dev)) return null
return openSlot(dev)
}
/**
* Open a slot for [dev] on the lowest free wire index, declaring its kind ([NativeBridge.nativeSendGamepadArrival])
* before any input so the host builds a matching virtual device (mixed types across pads).
* Idempotent; null when all 16 wire indices are already forwarded.
*/
private fun openSlot(dev: InputDevice): Slot? {
slots[dev.id]?.let { return it }
val index = lowestFreeIndex() ?: return null // 16 pads already forwarded — drop this one
// Automatic resolves the pad's type from its VID/PID; an explicit setting forces every pad
// to that type (a single global choice — matches the handshake's session-default pref).
val pref = if (setting == Gamepad.PREF_AUTO) Gamepad.prefFor(dev) else setting
NativeBridge.nativeSendGamepadArrival(handle, pref, index)
val slot = Slot(index, Gamepad.AxisMapper(handle, index))
slots[dev.id] = slot
return slot
}
/**
* Flush a slot's held wire state (so nothing sticks host-side), signal the removal, and free its
* index. Safe against an already-gone device — the flush emits wire events only, no device access.
*/
private fun closeSlot(deviceId: Int) {
val slot = slots.remove(deviceId) ?: return
releaseHeld(slot)
NativeBridge.nativeSendGamepadRemove(handle, slot.index)
// If this pad was mid-exit-chord, its removal may have left no pad holding it — drop the timer.
if (slots.values.none { it.held and EXIT_CHORD == EXIT_CHORD }) disarmExit()
// Release this controller's feedback bindings (close its lights session / cancel rumble).
onSlotClosed?.invoke(deviceId)
}
/** Lift every held button + zero the axes/HAT dpad for [slot] (wire events only, all on its index). */
private fun releaseHeld(slot: Slot) {
var bits = slot.held
while (bits != 0) {
val bit = bits and -bits // lowest set bit
NativeBridge.nativeSendGamepadButton(handle, bit, false, slot.index)
bits = bits and bit.inv()
}
slot.held = 0
slot.mapper.reset() // zero sticks/triggers + release the HAT dpad
}
/** Lowest wire index 0..[MAX_PADS) not held by a slot, or null when full — stable lowest-free keeps indices from shuffling on hot-plug. */
private fun lowestFreeIndex(): Int? {
val taken = slots.values.mapTo(HashSet()) { it.index }
for (i in 0 until MAX_PADS) if (i !in taken) return i
return null
}
private companion object {
/** Mirror of `punktfunk-core::input::MAX_PADS` — wire pad indices 0..15. */
const val MAX_PADS = 16
/** Emergency stream-exit chord: Select + Start + L1 + R1 held together (matches the legacy single-pad chord). */
const val EXIT_CHORD = Gamepad.BTN_BACK or Gamepad.BTN_START or Gamepad.BTN_LB or Gamepad.BTN_RB
/**
* How long the exit chord must be held before the stream leaves — long enough that an
* accidental brush of the four buttons doesn't quit, short enough to feel responsive (the
* on-screen hint covers the gap). Roughly matches SDL/Apple `DISCONNECT_HOLD`.
*/
const val EXIT_HOLD_MS = 1000L
/** Synthetic slot-key base for [ExternalPad]s — below every real (positive) InputDevice id. */
const val EXTERNAL_ID_BASE = -1000
}
}
@@ -85,16 +85,6 @@ object NativeBridge {
name: String,
): String
/**
* The machine token of the most recent failed [nativeConnect]/[nativePair], cleared on read
* (`""` when none) — call right after a `0` handle / `""` fingerprint. A typed host rejection
* yields its wire token ("not-armed", "denied", "approval-timeout", "superseded", "busy",
* "rate-limited", "bound-other", "identity-required", "wire-version"); transport-level causes
* yield "crypto" (wrong PIN / identity mismatch), "timeout", "io", or "error". Lets the UI say
* WHY instead of the old catch-all that blamed the PIN for dead network paths.
*/
external fun nativeTakeLastError(): String
/**
* Signal a **deliberate** user disconnect on [handle] before [nativeClose]: the session closes
* with `QUIT_CLOSE_CODE` so the host tears it down immediately instead of holding the keep-alive
@@ -161,14 +151,6 @@ object NativeBridge {
*/
external fun nativeVideoMime(handle: Long): String
/**
* A short human label for the codec the host resolved (`"H.264"` / `"HEVC"` / `"AV1"` /
* `"PyroWave"`), for the stats HUD's video-feed line, or `""` on a `0` handle. Distinct from
* [nativeVideoMime] because the MIME collapses PyroWave onto `video/hevc` and can't name it.
* Fixed for the session (resolved at the handshake); read once. Cheap; UI-safe.
*/
external fun nativeVideoCodecLabel(handle: Long): String
/**
* Start the decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. [decoderName] is the
@@ -287,52 +269,26 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
// ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ----
// The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0,
// so its wire is byte-identical to the old single-pad path. The core folds the per-transition
// events into seq'd GamepadState snapshots keyed on this index and owns the per-pad seq.
// ---- Gamepad: one pad forwarded as pad 0 (Rust hardcodes flags=0) ----
/** One gamepad button transition on wire pad [pad] (0..15). bit: a [Gamepad].BTN_* bit. down: press/release. */
external fun nativeSendGamepadButton(handle: Long, bit: Int, down: Boolean, pad: Int)
/** One gamepad button transition. bit: a [Gamepad].BTN_* bit. down: press/release. */
external fun nativeSendGamepadButton(handle: Long, bit: Int, down: Boolean)
/** One gamepad axis update on wire pad [pad] (0..15). axisId: [Gamepad].AXIS_* (0..5). value: stick i16 (+y=up) / trigger 0..255. */
external fun nativeSendGamepadAxis(handle: Long, axisId: Int, value: Int, pad: Int)
/**
* Declare the controller KIND presented on wire pad [pad] (0..15) so the host builds a matching
* virtual device (mixed types across pads). pref: a [Gamepad].PREF_* wire byte. Send ONCE when a
* pad opens, BEFORE any of its input; an older host ignores it (that pad then uses the handshake's
* session-default kind — the pre-existing single-pad behaviour on pad 0).
*/
external fun nativeSendGamepadArrival(handle: Long, pref: Int, pad: Int)
/** Signal wire pad [pad] (0..15) was unplugged so the host tears its virtual device down. The core stamps the seq + re-sends. */
external fun nativeSendGamepadRemove(handle: Long, pad: Int)
/**
* One raw HID input report from a client-captured controller (the as-is Steam Controller 2
* passthrough), forwarded verbatim on the rich-input plane. [buf] is a DIRECT ByteBuffer whose
* first [len] bytes are the report, id byte first (0x42/0x45/0x47 state, 0x43 battery, …);
* len is clamped to 64. Called from the capture thread at the controller's own report rate.
*/
external fun nativeSendPadHidReport(handle: Long, pad: Int, buf: java.nio.ByteBuffer, len: Int)
/** One gamepad axis update. axisId: [Gamepad].AXIS_* (0..5). value: stick i16 (+y=up) / trigger 0..255. */
external fun nativeSendGamepadAxis(handle: Long, axisId: Int, value: Int)
// ---- Host→client gamepad feedback: Rust pulls block ~100ms, Kotlin renders (see GamepadFeedback) ----
/**
* Block up to ~100 ms for the next rumble update. Returns a packed positive long: bits 49..52 =
* wire pad index (0..15), bit 48 = has a v2 lease, bits 32..47 = ttl_ms, bits 16..31 = low, bits
* 0..15 = high (each amplitude 0..0xFFFF; 0/0 = stop), or -1 on timeout / session closed. Kotlin
* routes the update to the controller holding that pad index. Call from a dedicated poll thread.
* Block up to ~100 ms for the next rumble update. Returns `(low shl 16) or high` (each
* 0..0xFFFF; 0 = stop), or -1 on timeout / session closed. Call from a dedicated poll thread.
*/
external fun nativeNextRumble(handle: Long): Long
/**
* Block up to ~100 ms for the next HID-output event, written into [buf] (a direct ByteBuffer,
* capacity >= 128) as `[pad][kind][fields…]` (leading pad = the wire pad index to route to):
* Led=pad 01 r g b, PlayerLeds=pad 02 bits, Trigger=pad 03 which effect…, raw as-is
* passthrough report=pad 05 kind report-bytes (kind 0 = output report, 1 = feature report).
* Returns the byte count, or -1 on timeout / session closed.
* Block up to ~100 ms for the next DualSense HID-output event, written into [buf] (a direct
* ByteBuffer, capacity >= 64) as `[kind][fields…]`: Led=01 r g b, PlayerLeds=02 bits,
* Trigger=03 which effect…. Returns the byte count, or -1 on timeout / session closed.
*/
external fun nativeNextHidout(handle: Long, buf: java.nio.ByteBuffer): Int
}
@@ -1,241 +0,0 @@
package io.unom.punktfunk.kit
import android.annotation.SuppressLint
import android.bluetooth.BluetoothDevice
import android.bluetooth.BluetoothGatt
import android.bluetooth.BluetoothGattCallback
import android.bluetooth.BluetoothGattCharacteristic
import android.bluetooth.BluetoothGattDescriptor
import android.bluetooth.BluetoothManager
import android.bluetooth.BluetoothProfile
import android.content.Context
import android.util.Log
import java.util.UUID
import java.util.concurrent.atomic.AtomicBoolean
/**
* BLE transport for a Steam Controller 2 paired directly with the device (no Puck). The standard
* HID service (0x1812) is claimed by the OS (and would feed the pad through the ordinary input
* stack in lizard-crippled form), so this talks Valve's vendor GATT service instead — the same
* approach Steam itself uses on hosts without a dongle.
*
* GATT operations are serialized by a small state machine (connect → MTU → discover → subscribe
* each notify char → lizard-off → ready); duplicate callbacks (the Android stack sometimes fires
* `onMtuChanged` twice) are ignored. Notified state reports arrive with the report-id byte
* stripped by the transport, so `0x45` (`ID_STATE_BLE`) is re-prepended for ≥40-byte payloads —
* the wire then carries the same id-first framing as USB.
*
* Requires BLUETOOTH_CONNECT (the caller gates on it); connection priority is bumped to HIGH to
* pull the connection interval from ~50 ms down to ~11 ms.
*/
@SuppressLint("MissingPermission")
class Sc2BleLink(
private val context: Context,
private val onReport: (report: ByteArray, len: Int) -> Unit,
private val onClosed: () -> Unit,
) {
private enum class State { IDLE, CONNECTING, MTU_REQUESTED, DISCOVERING, SUBSCRIBING, READY }
private val manager = context.getSystemService(Context.BLUETOOTH_SERVICE) as BluetoothManager
private var gatt: BluetoothGatt? = null
private var writeChar: BluetoothGattCharacteristic? = null
private val pendingSubs = mutableListOf<BluetoothGattCharacteristic>()
private var subsIndex = 0
private val writeBusy = AtomicBoolean(false)
private var lizardTicker: Thread? = null
@Volatile private var state = State.IDLE
/** Bonded devices that look like a Steam Controller (name heuristic — BLE exposes no PID here). */
fun pairedControllers(): List<BluetoothDevice> = runCatching {
manager.adapter?.bondedDevices.orEmpty().filter { dev ->
val n = runCatching { dev.name }.getOrNull() ?: return@filter false
NAME_HINTS.any { n.contains(it, ignoreCase = true) }
}
}.getOrDefault(emptyList())
/** Connect to the bonded controller at [address]. Reports start flowing once READY. */
fun start(address: String): Boolean {
val adapter = manager.adapter ?: return false
if (!adapter.isEnabled) return false
val device = runCatching { adapter.getRemoteDevice(address) }.getOrNull() ?: return false
state = State.CONNECTING
gatt = device.connectGatt(context, false, callback, BluetoothDevice.TRANSPORT_LE)
return true
}
/**
* Replay one raw report from the host: output reports (rumble) ride WRITE_NO_RESPONSE so they
* can't queue behind acks at the 25 Hz resend rate; feature reports (settings) use an acked
* write. The report-id byte stays in the payload (the firmware's vendor-channel framing).
*/
fun writeRaw(kind: Int, data: ByteArray) {
if (state != State.READY || data.isEmpty()) return
val g = gatt ?: return
val ch = writeChar ?: return
runCatching {
ch.value = data
ch.writeType = if (kind == 0) {
BluetoothGattCharacteristic.WRITE_TYPE_NO_RESPONSE
} else {
BluetoothGattCharacteristic.WRITE_TYPE_DEFAULT
}
g.writeCharacteristic(ch)
}
}
private fun sendLizardOff() {
if (state != State.READY) return
val g = gatt ?: return
val ch = writeChar ?: return
if (!writeBusy.compareAndSet(false, true)) return // previous acked write still in flight
runCatching {
ch.value = Sc2Device.DISABLE_LIZARD
ch.writeType = BluetoothGattCharacteristic.WRITE_TYPE_DEFAULT
if (!g.writeCharacteristic(ch)) writeBusy.set(false)
}.onFailure { writeBusy.set(false) }
}
/** Disconnect and stop the lizard ticker. Idempotent; does not fire [onClosed]. */
fun stop() {
lizardTicker?.interrupt()
lizardTicker = null
runCatching { gatt?.disconnect() }
runCatching { gatt?.close() }
gatt = null
writeChar = null
pendingSubs.clear()
subsIndex = 0
state = State.IDLE
}
private val callback = object : BluetoothGattCallback() {
override fun onConnectionStateChange(g: BluetoothGatt, status: Int, newState: Int) {
when (newState) {
BluetoothProfile.STATE_CONNECTED -> {
// ~11 ms connection interval instead of the ~50 ms default — input latency.
g.requestConnectionPriority(BluetoothGatt.CONNECTION_PRIORITY_HIGH)
if (state == State.CONNECTING) {
state = State.MTU_REQUESTED
if (!g.requestMtu(DESIRED_MTU)) {
state = State.DISCOVERING
g.discoverServices()
}
}
}
BluetoothProfile.STATE_DISCONNECTED -> {
val wasLive = state != State.IDLE
runCatching { g.close() }
gatt = null
writeChar = null
pendingSubs.clear()
subsIndex = 0
state = State.IDLE
if (wasLive) onClosed()
}
}
}
override fun onMtuChanged(g: BluetoothGatt, mtu: Int, status: Int) {
if (state != State.MTU_REQUESTED) return // fired twice on some stacks — act once
state = State.DISCOVERING
g.discoverServices()
}
override fun onServicesDiscovered(g: BluetoothGatt, status: Int) {
if (state != State.DISCOVERING || status != BluetoothGatt.GATT_SUCCESS) return
val valve = g.getService(VALVE_SERVICE) ?: run {
Log.e(TAG, "Valve vendor service missing — not an SC2?")
return
}
pendingSubs.clear()
writeChar = null
for (ch in valve.characteristics) {
val short = shortUuid(ch.uuid) ?: continue
val canNotify = ch.properties and BluetoothGattCharacteristic.PROPERTY_NOTIFY != 0
val canWrite = ch.properties and (
BluetoothGattCharacteristic.PROPERTY_WRITE or
BluetoothGattCharacteristic.PROPERTY_WRITE_NO_RESPONSE
) != 0
if (canNotify && short in NOTIFY_LOW..NOTIFY_HIGH) pendingSubs.add(ch)
if (canWrite && short in WRITE_LOW..WRITE_HIGH && writeChar == null) writeChar = ch
}
subsIndex = 0
state = State.SUBSCRIBING
subscribeNext(g)
}
override fun onDescriptorWrite(g: BluetoothGatt, d: BluetoothGattDescriptor, status: Int) {
if (state == State.SUBSCRIBING) subscribeNext(g)
}
override fun onCharacteristicWrite(g: BluetoothGatt, ch: BluetoothGattCharacteristic, status: Int) {
writeBusy.set(false)
}
override fun onCharacteristicChanged(g: BluetoothGatt, ch: BluetoothGattCharacteristic) {
val data = ch.value ?: return
// BLE strips the report-id prefix; restore 0x45 on state-sized payloads so the raw
// wire framing matches USB. Short payloads (battery/status) pass through as-is.
if (data.size >= 40) {
val framed = ByteArray(data.size + 1)
framed[0] = Sc2Device.ID_STATE_BLE.toByte()
System.arraycopy(data, 0, framed, 1, data.size)
onReport(framed, framed.size)
} else {
onReport(data, data.size)
}
}
}
private fun subscribeNext(g: BluetoothGatt) {
if (subsIndex >= pendingSubs.size) {
state = State.READY
Log.i(TAG, "SC2 BLE link up (${pendingSubs.size} notify chars)")
sendLizardOff()
// The firmware watchdog re-enables lizard mode; refresh on SDL's cadence until the
// host's Steam takes over via the raw plane (its writes land through writeRaw too).
lizardTicker = Thread({
while (state == State.READY) {
try {
Thread.sleep(Sc2Device.LIZARD_REFRESH_MS)
} catch (_: InterruptedException) {
return@Thread
}
sendLizardOff()
}
}, "pf-sc2-lizard").apply { isDaemon = true; start() }
return
}
val ch = pendingSubs[subsIndex++]
g.setCharacteristicNotification(ch, true)
val cccd = ch.getDescriptor(CCCD) ?: return subscribeNext(g)
cccd.value = BluetoothGattDescriptor.ENABLE_NOTIFICATION_VALUE
if (!g.writeDescriptor(cccd)) subscribeNext(g) // lose this one, try the rest
}
/** The 32-bit short id of a Valve vendor UUID, or null for foreign UUIDs. */
private fun shortUuid(uuid: UUID): Long? {
val s = uuid.toString()
if (!s.endsWith(VALVE_UUID_TAIL)) return null
return s.substring(0, 8).toLongOrNull(16)
}
private companion object {
const val TAG = "Sc2BleLink"
val VALVE_SERVICE: UUID = UUID.fromString("100f6c32-1735-4313-b402-38567131e5f3")
const val VALVE_UUID_TAIL = "-1735-4313-b402-38567131e5f3"
const val NOTIFY_LOW = 0x100f6c75L
const val NOTIFY_HIGH = 0x100f6c7aL
const val WRITE_LOW = 0x100f6cb5L
const val WRITE_HIGH = 0x100f6cbeL
val CCCD: UUID = UUID.fromString("00002902-0000-1000-8000-00805f9b34fb")
val NAME_HINTS = listOf("Steam Ctrl", "Steam Controller", "SteamController", "Valve")
/** Enough for a state payload (45 B) + ATT header with margin. */
const val DESIRED_MTU = 100
}
}
@@ -1,316 +0,0 @@
package io.unom.punktfunk.kit
import android.content.Context
import android.hardware.usb.UsbDevice
import android.util.Log
import java.nio.ByteBuffer
/**
* One captured Steam Controller 2 — the glue between a transport link ([Sc2UsbLink] /
* [Sc2BleLink]) and one of two consumers:
*
* **Stream mode** (`router != null`, owned by StreamScreen):
* - **Raw plane (the point):** every input report is forwarded verbatim
* ([GamepadRouter.ExternalPad.hidReport]) for the host's as-is virtual `28DE:1302` pad, which
* Steam Input drives like the physical controller.
* - **Typed mirror:** buttons/sticks/triggers are ALSO diffed onto the ordinary per-transition
* plane, so the emergency exit chord works, and a host that degraded the kind (no UHID → the
* Xbox 360 pad) still gets a playable controller.
* - **Raw return:** the host's hidraw writes (Steam's `0x80` rumble output reports, lizard/IMU
* feature settings) arrive via [GamepadFeedback.onHidRaw] → [onHidRaw] → the link, landing on
* the real controller's motors/firmware.
*
* **UI mode** (`router == null`, owned by MainActivity while NOT streaming): the lizard-mode
* kb/mouse never produces gamepad events, so an uncaptured SC2 can't drive the console UI at
* all. Here the parsed state is edge-detected into [onUiKey] navigation transitions instead
* (D-pad + face buttons + Start/Select; the left stick synthesizes one D-pad step per push,
* mirroring MainActivity's stick-to-focus behavior for ordinary pads).
*
* The wire slot is claimed lazily on the FIRST state report — a Puck with no controller powered
* on stays invisible to the host — and released (with a wireless-disconnect event or on [stop])
* so pad indices never leak. Report callbacks arrive on the link's own thread; the router's slot
* table and chord timer are thread-safe for this (same contract as the feedback poll threads),
* and UI-mode consumers hop to the main thread themselves.
*/
class Sc2Capture(
context: Context,
private val router: GamepadRouter? = null,
) {
private val usb = Sc2UsbLink(context, ::onReport, ::onLinkClosed)
private val ble = Sc2BleLink(context, ::onReport, ::onLinkClosed)
private var activeLink: Int = LINK_NONE
/** True when the USB link is a Puck dongle — the only transport whose wireless-status
* reports are authoritative. A WIRED pad also emits them, truthfully reporting "no radio
* link" — acting on that tore the slot down 255 ms after creation (first on-glass run). */
private var dongleLink = false
private var pad: GamepadRouter.ExternalPad? = null
private val rawBuf: ByteBuffer = ByteBuffer.allocateDirect(64)
/** Puck connect arrives before its first state report (and therefore before a wire pad exists).
* Preserve it so the native virtual Puck slot sees the same connect edge before state. */
private val pendingWireless = ByteArray(2)
private var pendingWirelessLen = 0
// Typed-mirror diff state (wire units).
private val state = Sc2Device.State()
private var wireButtons = 0
private val lastAxis = IntArray(6) { Int.MIN_VALUE }
/** Report ids seen so far — each logged once, for remote diagnosis of what the pad emits. */
private val seenIds = HashSet<Int>()
// UI-mode state (router == null): held navigation keys + the stick's current synth direction.
private var uiHeld = HashSet<Int>()
private var uiStickDir = 0
/**
* UI-mode sink: one navigation key transition (an Android `KeyEvent.KEYCODE_*`), invoked on
* the LINK thread — the consumer hops to the main thread. Set before [startUsb]/[startBle].
*/
@Volatile
var onUiKey: ((keyCode: Int, down: Boolean) -> Unit)? = null
/**
* Fired (link thread) when the capture engages or drops — lets the app surface "SC2
* connected" in the console-UI gate and the Controllers screen.
*/
@Volatile
var onActiveChanged: ((active: Boolean) -> Unit)? = null
val isActive: Boolean get() = activeLink != LINK_NONE
/** First attached SC2/Puck USB device, for the permission flow. */
fun findUsbDevice(): UsbDevice? = usb.findDevice()
/**
* The first already-bonded BLE Steam Controller's address, or null. The caller checks
* BLUETOOTH_CONNECT first (without it the bonded list reads as empty anyway).
*/
fun pairedBleAddress(): String? = ble.pairedControllers().firstOrNull()?.address
/** Start capturing [dev] over USB (permission already granted). */
fun startUsb(dev: UsbDevice): Boolean {
if (activeLink != LINK_NONE) return false
val ok = usb.start(dev)
if (ok) {
activeLink = LINK_USB
dongleLink = dev.productId != Sc2Device.PID_WIRED
onActiveChanged?.invoke(true)
}
return ok
}
/** Start capturing the bonded BLE controller at [address]. */
fun startBle(address: String): Boolean {
if (activeLink != LINK_NONE) return false
val ok = ble.start(address)
if (ok) {
activeLink = LINK_BLE
onActiveChanged?.invoke(true)
}
return ok
}
/** Replay a host raw write on the physical pad — wire to [GamepadFeedback.onHidRaw]. */
fun onHidRaw(padIndex: Int, kind: Int, data: ByteArray) {
if (padIndex != pad?.index) return // addressed to some other controller
when (activeLink) {
LINK_USB -> usb.writeRaw(kind, data)
LINK_BLE -> ble.writeRaw(kind, data)
}
}
/** Stop the link and free the wire slot (host tears the virtual pad down). Idempotent. */
fun stop() {
val wasActive = activeLink != LINK_NONE
when (activeLink) {
LINK_USB -> usb.stop()
LINK_BLE -> ble.stop()
}
activeLink = LINK_NONE
dongleLink = false
releaseSlot()
releaseUiKeys()
if (wasActive) onActiveChanged?.invoke(false)
}
// ---- link callbacks (link thread) ----
private fun onReport(report: ByteArray, len: Int) {
val id = report[0].toInt() and 0xFF
if (seenIds.add(id)) Log.i(TAG, "SC2 report id=0x%02x seen (len=%d)".format(id, len))
// Wireless status: authoritative ONLY through a Puck dongle (powering the pad off frees
// its wire index + the host's virtual device). A wired/BLE pad emits it too — truthfully
// saying "no radio link" — and must NOT tear the slot down (SDL's wired path likewise
// marks the controller connected unconditionally and reconnects on any state report).
if ((id == Sc2Device.ID_WIRELESS || id == Sc2Device.ID_WIRELESS_X) && len >= 2) {
if (dongleLink) {
when (report[1].toInt() and 0xFF) {
Sc2Device.WIRELESS_CONNECT -> {
pendingWireless[0] = report[0]
pendingWireless[1] = report[1]
pendingWirelessLen = 2
}
Sc2Device.WIRELESS_DISCONNECT -> {
pendingWirelessLen = 0
Log.i(TAG, "Puck reports controller powered off — releasing wire slot")
releaseSlot()
releaseUiKeys()
}
}
}
return
}
if (!Sc2Device.parseState(report, len, state)) {
// Battery/status and future report types still belong to the as-is stream.
forwardRaw(report, len)
return
}
if (router == null) {
mirrorUi()
return
}
val pref = if (dongleLink) {
Gamepad.PREF_STEAMCONTROLLER2_PUCK
} else {
Gamepad.PREF_STEAMCONTROLLER2
}
val p = pad ?: router.openExternal(pref)?.also {
pad = it
Log.i(
TAG,
"SC2 captured → wire pad ${it.index} (${if (dongleLink) "Puck" else "direct"} passthrough)",
)
if (pendingWirelessLen > 0) {
forwardRaw(pendingWireless, pendingWirelessLen)
pendingWirelessLen = 0
}
} ?: return // all 16 wire indices taken — drop until one frees
forwardRaw(report, len)
mirrorTyped(p)
}
private fun forwardRaw(report: ByteArray, len: Int) {
val p = pad ?: return
val n = len.coerceAtMost(rawBuf.capacity())
rawBuf.clear()
rawBuf.put(report, 0, n)
p.hidReport(rawBuf, n)
}
/** Diff the parsed state onto the per-transition plane (buttons + axes, on change only). */
private fun mirrorTyped(p: GamepadRouter.ExternalPad) {
val wired = Sc2Device.wireButtons(state.buttons)
var changed = wired xor wireButtons
while (changed != 0) {
val bit = changed and -changed // lowest changed bit
p.button(bit, wired and bit != 0)
changed = changed and bit.inv()
}
wireButtons = wired
axis(p, Gamepad.AXIS_LS_X, state.lsX)
axis(p, Gamepad.AXIS_LS_Y, state.lsY)
axis(p, Gamepad.AXIS_RS_X, state.rsX)
axis(p, Gamepad.AXIS_RS_Y, state.rsY)
axis(p, Gamepad.AXIS_LT, state.lt)
axis(p, Gamepad.AXIS_RT, state.rt)
}
private fun axis(p: GamepadRouter.ExternalPad, id: Int, v: Int) {
if (lastAxis[id] == v) return
lastAxis[id] = v
p.axis(id, v)
}
/**
* UI mode: edge-detect the parsed state into navigation key transitions. Buttons map to
* their Android keycodes (press AND release, so the focus system sees real holds); the left
* stick synthesizes ONE D-pad step per push past half deflection — the same single-move
* behavior MainActivity gives ordinary pads' sticks.
*/
private fun mirrorUi() {
val sink = onUiKey ?: return
val held = HashSet<Int>(8)
var i = 0
while (i < UI_KEY_MAP.size) {
if (state.buttons and UI_KEY_MAP[i] != 0) held.add(UI_KEY_MAP[i + 1])
i += 2
}
for (key in held) if (key !in uiHeld) sink(key, true)
for (key in uiHeld) if (key !in held) sink(key, false)
uiHeld = held
// Left stick → a HELD D-pad direction (device convention: +y = up): pressed while
// deflected, released on centre/direction change. The console UI's probe machinery
// turns a held direction into its own auto-repeat, exactly like a physical D-pad; the
// focus-hook path moves once per press edge either way.
val dir = when {
state.lsX <= -STICK_NAV -> android.view.KeyEvent.KEYCODE_DPAD_LEFT
state.lsX >= STICK_NAV -> android.view.KeyEvent.KEYCODE_DPAD_RIGHT
state.lsY >= STICK_NAV -> android.view.KeyEvent.KEYCODE_DPAD_UP
state.lsY <= -STICK_NAV -> android.view.KeyEvent.KEYCODE_DPAD_DOWN
else -> 0
}
if (dir != uiStickDir) {
// The D-pad bits share these keycodes; don't release a direction the physical
// D-pad itself still holds (uiHeld tracks the button-sourced state).
if (uiStickDir != 0 && uiStickDir !in uiHeld) sink(uiStickDir, false)
if (dir != 0 && dir !in uiHeld) sink(dir, true)
uiStickDir = dir
}
}
/** Release every held UI-mode key (link drop / stop) so nothing sticks in the focus system. */
private fun releaseUiKeys() {
val sink = onUiKey
if (sink != null) {
for (key in uiHeld) sink(key, false)
if (uiStickDir != 0 && uiStickDir !in uiHeld) sink(uiStickDir, false)
}
uiHeld = HashSet()
uiStickDir = 0
}
private fun onLinkClosed() {
Log.i(TAG, "SC2 link closed (unplug / power-off)")
activeLink = LINK_NONE
dongleLink = false
releaseSlot()
releaseUiKeys()
onActiveChanged?.invoke(false)
}
private fun releaseSlot() {
pad?.close()
pad = null
wireButtons = 0
lastAxis.fill(Int.MIN_VALUE)
pendingWirelessLen = 0
}
private companion object {
const val TAG = "Sc2Capture"
const val LINK_NONE = 0
const val LINK_USB = 1
const val LINK_BLE = 2
/** Half deflection (device i16 range) — the stick-to-focus threshold. */
const val STICK_NAV = 16384
/** UI-mode mapping: SC2 button bit → Android keycode, as (bit, key) pairs. */
val UI_KEY_MAP = intArrayOf(
Sc2Device.DPAD_UP, android.view.KeyEvent.KEYCODE_DPAD_UP,
Sc2Device.DPAD_DOWN, android.view.KeyEvent.KEYCODE_DPAD_DOWN,
Sc2Device.DPAD_LEFT, android.view.KeyEvent.KEYCODE_DPAD_LEFT,
Sc2Device.DPAD_RIGHT, android.view.KeyEvent.KEYCODE_DPAD_RIGHT,
Sc2Device.A, android.view.KeyEvent.KEYCODE_BUTTON_A,
Sc2Device.B, android.view.KeyEvent.KEYCODE_BUTTON_B,
Sc2Device.X, android.view.KeyEvent.KEYCODE_BUTTON_X,
Sc2Device.Y, android.view.KeyEvent.KEYCODE_BUTTON_Y,
Sc2Device.LB, android.view.KeyEvent.KEYCODE_BUTTON_L1,
Sc2Device.RB, android.view.KeyEvent.KEYCODE_BUTTON_R1,
Sc2Device.MENU, android.view.KeyEvent.KEYCODE_BUTTON_START,
Sc2Device.VIEW, android.view.KeyEvent.KEYCODE_BUTTON_SELECT,
)
}
}
@@ -1,165 +0,0 @@
package io.unom.punktfunk.kit
/**
* Steam Controller 2 (2026, Valve "Ibex" / SDL "Triton") protocol constants + the light state
* parser the CLIENT needs. The full report rides the wire verbatim (`nativeSendPadHidReport` →
* the host's as-is virtual pad); this parser only extracts what the client itself consumes: the
* button word for the typed mirror + exit chord, and sticks/triggers for the degrade path.
*
* Protocol ground truth: SDL's `SDL_hidapi_steam_triton.c` + `steam/controller_structs.h`
* (Valve-maintained), mirrored host-side in `punktfunk-host`'s `triton_proto.rs`.
*/
object Sc2Device {
const val VID_VALVE = 0x28DE
/** Wired controller. */
const val PID_WIRED = 0x1302
/** Direct BLE identity (transport handled by [Sc2BleLink], not USB). */
const val PID_BLE = 0x1303
/** The wireless Puck dongles (Proteus / Nereid) — controller on USB interfaces 2..5. */
const val PID_DONGLE_PROTEUS = 0x1304
const val PID_DONGLE_NEREID = 0x1305
val USB_PIDS = setOf(PID_WIRED, PID_DONGLE_PROTEUS, PID_DONGLE_NEREID)
/** Dongle interface range that carries controllers (SDL: "interfaces 2..5, currently"). */
val DONGLE_IFACES = 2..5
// Input report ids (`ETritonReportIDTypes`). State layouts share every offset the client
// reads (seq/buttons/triggers/sticks); 0x47 only diverges from byte 18 (trackpad timestamp).
const val ID_STATE = 0x42
const val ID_BATTERY = 0x43
const val ID_STATE_BLE = 0x45
const val ID_WIRELESS_X = 0x46
const val ID_STATE_TIMESTAMP = 0x47
const val ID_WIRELESS = 0x79
/** Wireless status payload byte: controller connected/disconnected through the Puck. */
const val WIRELESS_DISCONNECT = 1
const val WIRELESS_CONNECT = 2
// Button bits in the state report's u32 (SDL `TritonButtons`).
const val A = 0x00000001
const val B = 0x00000002
const val X = 0x00000004
const val Y = 0x00000008
const val QAM = 0x00000010
const val R3 = 0x00000020
const val VIEW = 0x00000040
const val R4 = 0x00000080
const val R5 = 0x00000100
const val RB = 0x00000200
const val DPAD_DOWN = 0x00000400
const val DPAD_RIGHT = 0x00000800
const val DPAD_LEFT = 0x00001000
const val DPAD_UP = 0x00002000
const val MENU = 0x00004000
const val L3 = 0x00008000
const val STEAM = 0x00010000
const val L4 = 0x00020000
const val L5 = 0x00040000
const val LB = 0x00080000
const val RPAD_CLICK = 0x00400000
/**
* The feature report that turns lizard mode (built-in keyboard/mouse emulation) off:
* `[report id 1][ID_SET_SETTINGS_VALUES 0x87][length 3][SETTING_LIZARD_MODE 9]
* [LIZARD_MODE_OFF u16]`, zero-padded to the 64-byte feature size. The firmware watchdog
* re-enables lizard mode after a few seconds of silence, so this is re-sent every
* [LIZARD_REFRESH_MS] (SDL's cadence) — and the host's Steam sends its own through the raw
* plane once it grabs the virtual pad, which lands here too.
*/
val DISABLE_LIZARD: ByteArray = ByteArray(64).also {
it[0] = 0x01 // feature report id
it[1] = 0x87.toByte() // ID_SET_SETTINGS_VALUES
it[2] = 3 // one ControllerSetting {u8 num, u16 value}
it[3] = 9 // SETTING_LIZARD_MODE
// [4..6] = LIZARD_MODE_OFF (0) — already zero
}
/**
* Force firmware-calibrated signed i16 stick coordinates. Steam sends this during physical
* controller initialization (`SETTING_ENABLE_RAW_JOYSTICK` = 0x2e, value 0); without it a
* controller previously opened in raw mode reports ADC coordinates around 0..3200, which a
* Triton consumer interprets as only a few percent of full travel.
*/
val NORMALIZE_JOYSTICKS: ByteArray = ByteArray(64).also {
it[0] = 0x01 // feature report id
it[1] = 0x87.toByte() // ID_SET_SETTINGS_VALUES
it[2] = 3 // one ControllerSetting {u8 num, u16 value}
it[3] = 0x2E // SETTING_ENABLE_RAW_JOYSTICK
// [4..6] = disabled (0) — firmware emits calibrated signed i16 values
}
const val LIZARD_REFRESH_MS = 3000L
/** Wire mapping: SC2 button bit → punktfunk `Gamepad.BTN_*`, the inverse of the host's
* typed-fallback mapping (`triton_proto::from_gamepad`): paddles R4/L4/R5/L5 =
* PADDLE1/2/3/4, QAM = MISC1, right-pad click = the touchpad wire bit. */
private val WIRE_MAP = intArrayOf(
A, Gamepad.BTN_A,
B, Gamepad.BTN_B,
X, Gamepad.BTN_X,
Y, Gamepad.BTN_Y,
LB, Gamepad.BTN_LB,
RB, Gamepad.BTN_RB,
VIEW, Gamepad.BTN_BACK,
MENU, Gamepad.BTN_START,
STEAM, Gamepad.BTN_GUIDE,
L3, Gamepad.BTN_LS_CLICK,
R3, Gamepad.BTN_RS_CLICK,
DPAD_UP, Gamepad.BTN_DPAD_UP,
DPAD_DOWN, Gamepad.BTN_DPAD_DOWN,
DPAD_LEFT, Gamepad.BTN_DPAD_LEFT,
DPAD_RIGHT, Gamepad.BTN_DPAD_RIGHT,
QAM, Gamepad.BTN_MISC1,
R4, Gamepad.BTN_PADDLE1,
L4, Gamepad.BTN_PADDLE2,
R5, Gamepad.BTN_PADDLE3,
L5, Gamepad.BTN_PADDLE4,
RPAD_CLICK, Gamepad.BTN_TOUCHPAD,
)
/** Translate an SC2 button word into the wire `Gamepad.BTN_*` bitmask. */
fun wireButtons(sc2: Int): Int {
var out = 0
var i = 0
while (i < WIRE_MAP.size) {
if (sc2 and WIRE_MAP[i] != 0) out = out or WIRE_MAP[i + 1]
i += 2
}
return out
}
/** The typed-mirror fields of one state report (buttons/sticks/triggers only). */
class State {
var buttons = 0 // SC2 bit layout
var lsX = 0; var lsY = 0 // i16, +y = up (device convention = wire convention)
var rsX = 0; var rsY = 0
var lt = 0; var rt = 0 // 0..255 (device 0..32767 scaled down)
}
/**
* Parse the client-consumed fields out of a state report (`0x42`/`0x45`/`0x47` — identical
* offsets for everything read here) into [out]. Returns false for non-state / short reports.
*/
fun parseState(report: ByteArray, len: Int, out: State): Boolean {
if (len < 18) return false
when (report[0].toInt() and 0xFF) {
ID_STATE, ID_STATE_BLE, ID_STATE_TIMESTAMP -> {}
else -> return false
}
fun i16(o: Int) = ((report[o + 1].toInt() shl 8) or (report[o].toInt() and 0xFF)).toShort().toInt()
out.buttons = (report[2].toInt() and 0xFF) or
((report[3].toInt() and 0xFF) shl 8) or
((report[4].toInt() and 0xFF) shl 16) or
((report[5].toInt() and 0xFF) shl 24)
out.lt = (i16(6).coerceIn(0, 32767)) shr 7
out.rt = (i16(8).coerceIn(0, 32767)) shr 7
out.lsX = i16(10); out.lsY = i16(12)
out.rsX = i16(14); out.rsY = i16(16)
return true
}
}
@@ -1,379 +0,0 @@
package io.unom.punktfunk.kit
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.hardware.usb.UsbConstants
import android.hardware.usb.UsbDevice
import android.hardware.usb.UsbDeviceConnection
import android.hardware.usb.UsbEndpoint
import android.hardware.usb.UsbInterface
import android.hardware.usb.UsbManager
import android.hardware.usb.UsbRequest
import android.os.Build
import android.util.Log
import java.nio.ByteBuffer
import java.util.concurrent.ConcurrentLinkedQueue
import java.util.concurrent.TimeoutException
/**
* USB transport for a Steam Controller 2 — wired (`28DE:1302`) or through the wireless Puck
* dongle (`1304`/`1305`). Claims the controller interface(s) — detaching the OS input stack, so
* the pad can't double-drive the ordinary InputDevice path — runs a multiplexed [UsbRequest]
* read loop, keeps lizard mode off on the firmware watchdog cadence, and replays the host's raw
* writes (Steam's rumble output reports / settings feature reports) back to the device.
*
* **The Puck claims ALL controller interfaces (2..5):** the dongle hosts up to four pads, one
* HID interface each, and there is no way to know which slot a controller bonded to — claiming
* only interface 2 read silence while Android's input stack kept the others (the round-2
* on-glass symptom: the pad surfaced as a generic InputDevice → Xbox360). Whichever interface
* streams state becomes the write target for rumble/settings.
*
* **Unplug is signalled, never inferred from silence:** a quiet controller is not a missing one
* (round 2's wired disconnect was the 5 s silence heuristic firing on an idle pad). The real
* signals are [UsbManager.ACTION_USB_DEVICE_DETACHED] for this device, or `requestWait`
* returning sustained hard errors (every transfer fails instantly once the fd is dead).
*/
class Sc2UsbLink(
private val context: Context,
private val onReport: (report: ByteArray, len: Int) -> Unit,
private val onClosed: () -> Unit,
) {
private val usb = context.getSystemService(Context.USB_SERVICE) as UsbManager
/** One claimed interface: its endpoints + the read state the reader thread owns. */
private class Claim(
val iface: UsbInterface,
val epIn: UsbEndpoint,
val epOut: UsbEndpoint?,
) {
val inBuf: ByteBuffer = ByteBuffer.allocate(64)
var inReq: UsbRequest? = null
var outReq: UsbRequest? = null
var outBusy = false
var reports = 0L
}
private var connection: UsbDeviceConnection? = null
private var device: UsbDevice? = null
private var claims: List<Claim> = emptyList()
/** The claim whose IN endpoint last produced data — where rumble/settings writes go.
* Written by the reader thread, read by the feedback thread (feature control transfers). */
@Volatile private var activeClaim: Claim? = null
/** Pending OUT reports (Steam's forwarded haptics), submitted by the reader thread — only
* one thread may drive a connection's [UsbRequest]s ([UsbDeviceConnection.requestWait]
* returns ANY completed request; a second waiter would steal the reader's completions). */
private val outQueue = ConcurrentLinkedQueue<ByteArray>()
private var reader: Thread? = null
private var detachReceiver: BroadcastReceiver? = null
@Volatile private var running = false
/** First attached SC2 (wired or Puck), or null. Does not need USB permission to enumerate. */
fun findDevice(): UsbDevice? = usb.deviceList.values.firstOrNull {
it.vendorId == Sc2Device.VID_VALVE && it.productId in Sc2Device.USB_PIDS
}
/**
* Claim [dev]'s controller interface(s) and start the read loop. The caller has already
* obtained USB permission. Returns false when nothing could be claimed.
*/
fun start(dev: UsbDevice): Boolean {
if (!usb.hasPermission(dev)) {
Log.e(TAG, "no USB permission for ${dev.deviceName}")
return false
}
val conn = usb.openDevice(dev) ?: run {
Log.e(TAG, "openDevice failed for ${dev.deviceName}")
return false
}
val claimed = claimControllerInterfaces(dev, conn)
if (claimed.isEmpty()) {
Log.e(TAG, "no claimable SC2 interface on ${dev.deviceName} (PID=0x%04x)".format(dev.productId))
conn.close()
return false
}
connection = conn
device = dev
claims = claimed
running = true
Log.i(
TAG,
"SC2 USB link up: PID=0x%04x ifaces=%s".format(
dev.productId,
claimed.joinToString {
"%d(in=0x%02x out=%s)".format(
it.iface.id, it.epIn.address,
it.epOut?.let { e -> "0x%02x".format(e.address) } ?: "-",
)
},
),
)
// The REAL unplug signal — silence never is (an idle pad may simply stop streaming).
val receiver = object : BroadcastReceiver() {
override fun onReceive(c: Context?, intent: Intent?) {
if (intent?.action != UsbManager.ACTION_USB_DEVICE_DETACHED) return
val gone: UsbDevice? = intent.getParcelableExtra(UsbManager.EXTRA_DEVICE)
if (gone?.deviceName == dev.deviceName) {
Log.i(TAG, "SC2 USB detached (${dev.deviceName})")
if (running) {
running = false
onClosed()
}
}
}
}
detachReceiver = receiver
val filter = IntentFilter(UsbManager.ACTION_USB_DEVICE_DETACHED)
if (Build.VERSION.SDK_INT >= 33) {
context.registerReceiver(receiver, filter, Context.RECEIVER_NOT_EXPORTED)
} else {
@Suppress("UnspecifiedRegisterReceiverFlag")
context.registerReceiver(receiver, filter)
}
claimed.forEach { configureInputMode(conn, it.iface.id) }
reader = Thread({ readLoop(conn, claimed) }, "pf-sc2-usb").apply {
isDaemon = true
start()
}
return true
}
/**
* Claim every candidate controller interface: the wired pad's single HID interface, or ALL
* of a Puck's controller slots (interfaces 2..5 — the controller may be bonded to any of
* them). `force = true` detaches the kernel/OS driver, so the pad also vanishes from
* Android's own input stack while captured.
*/
private fun claimControllerInterfaces(dev: UsbDevice, conn: UsbDeviceConnection): List<Claim> {
val dongle = dev.productId != Sc2Device.PID_WIRED
val out = mutableListOf<Claim>()
for (i in 0 until dev.interfaceCount) {
val iface = dev.getInterface(i)
if (dongle && iface.id !in Sc2Device.DONGLE_IFACES) continue
val hidOrVendor = iface.interfaceClass == UsbConstants.USB_CLASS_HID ||
iface.interfaceClass == 0xFF
if (!hidOrVendor) continue
var inEp: UsbEndpoint? = null
var outEp: UsbEndpoint? = null
for (e in 0 until iface.endpointCount) {
val ep = iface.getEndpoint(e)
val usable = ep.type == UsbConstants.USB_ENDPOINT_XFER_INT ||
ep.type == UsbConstants.USB_ENDPOINT_XFER_BULK
if (!usable) continue
if (ep.direction == UsbConstants.USB_DIR_IN && inEp == null) inEp = ep
if (ep.direction == UsbConstants.USB_DIR_OUT && outEp == null) outEp = ep
}
if (inEp == null) continue
if (conn.claimInterface(iface, true)) {
out.add(Claim(iface, inEp, outEp))
} else {
Log.w(TAG, "could not claim iface ${iface.id}")
}
}
return out
}
/**
* The multiplexed read loop: one IN request queued per claimed interface at all times, OUT
* writes submitted from [outQueue], completions routed via [UsbRequest.getClientData].
*/
private fun readLoop(conn: UsbDeviceConnection, claims: List<Claim>) {
val live = claims.filter { c ->
val req = UsbRequest()
if (!req.initialize(conn, c.epIn)) {
Log.w(TAG, "UsbRequest.initialize(IN, iface ${c.iface.id}) failed")
return@filter false
}
req.clientData = c
c.inReq = req
c.epOut?.let { ep ->
val o = UsbRequest()
if (o.initialize(conn, ep)) {
o.clientData = c
c.outReq = o
} else {
Log.w(TAG, "UsbRequest.initialize(OUT, iface ${c.iface.id}) failed — output reports via EP0")
}
}
c.inBuf.clear()
req.queue(c.inBuf)
}
if (live.isEmpty()) {
Log.e(TAG, "no IN request could be queued")
finishReader(claims)
return
}
val scratch = ByteArray(64)
var lastLizard = android.os.SystemClock.elapsedRealtime()
var errorsSince = 0L // elapsedRealtime of the first hard error in the current streak
try {
while (running) {
val now = android.os.SystemClock.elapsedRealtime()
if (now - lastLizard >= Sc2Device.LIZARD_REFRESH_MS) {
// Refresh both required firmware modes. The raw-joystick setting is normally
// persistent, but replaying it also repairs a host/driver that enabled ADC
// coordinates after capture started.
val target = activeClaim
if (target != null) configureInputMode(conn, target.iface.id)
else live.forEach { configureInputMode(conn, it.iface.id) }
lastLizard = now
}
// Submit the next pending OUT report on the active (else first) interface.
val outTarget = (activeClaim ?: live.first()).takeIf { it.outReq != null && !it.outBusy }
if (outTarget != null) {
outQueue.poll()?.let { data ->
if (outTarget.outReq!!.queue(ByteBuffer.wrap(data))) outTarget.outBusy = true
}
}
val done = try {
conn.requestWait(READ_TIMEOUT_MS)
} catch (_: TimeoutException) {
// A quiet controller is NOT an unplug — keep listening indefinitely; the
// detach broadcast is the real signal.
errorsSince = 0L
continue
}
if (done == null) {
// Hard error. On a real unplug these storm continuously (the detach
// broadcast usually beats us to it); tolerate transient ones.
if (errorsSince == 0L) errorsSince = now
if (now - errorsSince >= ERROR_UNPLUG_MS) {
Log.i(TAG, "SC2 USB request errors persisting ${now - errorsSince} ms — treating as unplug")
break
}
continue
}
errorsSince = 0L
val claim = done.clientData as? Claim ?: continue
if (done === claim.inReq) {
val n = claim.inBuf.position()
if (n > 0) {
claim.inBuf.flip()
claim.inBuf.get(scratch, 0, n)
if (claim.reports++ == 0L) {
Log.i(
TAG,
"SC2 first report on iface %d: id=0x%02x len=%d".format(
claim.iface.id, scratch[0].toInt() and 0xFF, n,
),
)
}
activeClaim = claim
onReport(scratch, n)
}
claim.inBuf.clear()
if (!claim.inReq!!.queue(claim.inBuf)) {
Log.i(TAG, "re-queue(IN, iface ${claim.iface.id}) failed — treating as unplug")
break
}
} else if (done === claim.outReq) {
claim.outBusy = false
}
}
} finally {
finishReader(claims)
}
if (running) {
running = false
onClosed()
}
}
private fun finishReader(claims: List<Claim>) {
for (c in claims) {
runCatching { c.inReq?.cancel(); c.inReq?.close() }
runCatching { c.outReq?.cancel(); c.outReq?.close() }
c.inReq = null
c.outReq = null
}
}
/**
* Replay one raw report from the host on the device: kind 0 = output report (Steam's `0x80`
* rumble & friends — the active interface's interrupt-OUT, else a `SET_REPORT(Output)`
* control transfer), kind 1 = feature report (`SET_REPORT(Feature)`). [data] is the full
* report, id byte first, exactly as hidapi framed it host-side.
*/
fun writeRaw(kind: Int, data: ByteArray) {
if (data.isEmpty()) return
when (kind) {
0 -> {
if ((activeClaim ?: claims.firstOrNull())?.outReq != null) {
// Interrupt-OUT rides UsbRequests submitted by the reader thread. Bounded,
// newest-wins: these are level-styled commands the host re-sends anyway.
while (outQueue.size >= 32) outQueue.poll()
outQueue.offer(data)
} else {
setReport(REPORT_TYPE_OUTPUT, data)
}
}
1 -> setReport(REPORT_TYPE_FEATURE, data)
}
}
private fun setReport(type: Int, data: ByteArray) {
val conn = connection ?: return
val ifId = (activeClaim ?: claims.firstOrNull())?.iface?.id ?: return
sendReport(conn, ifId, type, data)
}
private fun configureInputMode(conn: UsbDeviceConnection, ifaceId: Int) {
sendFeature(conn, ifaceId, Sc2Device.DISABLE_LIZARD)
sendFeature(conn, ifaceId, Sc2Device.NORMALIZE_JOYSTICKS)
}
private fun sendFeature(conn: UsbDeviceConnection, ifaceId: Int, data: ByteArray) {
sendReport(conn, ifaceId, REPORT_TYPE_FEATURE, data)
}
/**
* HID `SET_REPORT` control transfer with hidapi's report-id framing: a non-zero leading byte
* is the report id (sent in wValue AND kept in the payload); a zero leading byte means
* "unnumbered" (id 0 in wValue, id byte stripped from the payload). EP0 is independent of
* the interrupt endpoints, so this is safe alongside the reader thread's requestWait.
*/
private fun sendReport(conn: UsbDeviceConnection, ifaceId: Int, type: Int, data: ByteArray) {
val id = data[0].toInt() and 0xFF
val payload = if (id == 0) data.copyOfRange(1, data.size) else data
conn.controlTransfer(
0x21, // host→device, class, interface
0x09, // SET_REPORT
(type shl 8) or id,
ifaceId,
payload,
payload.size,
WRITE_TIMEOUT_MS,
)
}
/** Stop the read loop and release the interfaces. Idempotent; does not fire [onClosed]. */
fun stop() {
running = false
detachReceiver?.let { runCatching { context.unregisterReceiver(it) } }
detachReceiver = null
runCatching { reader?.join(1000) }
reader = null
outQueue.clear()
activeClaim = null
for (c in claims) runCatching { connection?.releaseInterface(c.iface) }
claims = emptyList()
runCatching { connection?.close() }
connection = null
device = null
}
private companion object {
const val TAG = "Sc2UsbLink"
const val READ_TIMEOUT_MS = 100L
const val WRITE_TIMEOUT_MS = 250
/** Hard `requestWait` ERRORS (not timeouts) persisting this long = the fd is dead. */
const val ERROR_UNPLUG_MS = 2000L
const val REPORT_TYPE_OUTPUT = 0x02
const val REPORT_TYPE_FEATURE = 0x03
}
}
File diff suppressed because it is too large Load Diff
@@ -1,626 +0,0 @@
//! The event-driven async MediaCodec decode loop (default) + its feeder/dispatch/present helpers.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{AsyncNotifyCallback, MediaCodec, MediaCodecDirection};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::reanchor::{GateVerdict, ReanchorGate};
use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, AtomicI64, Ordering};
use std::sync::{mpsc, Arc, Mutex};
use std::time::{Duration, Instant};
use super::display::{
apply_hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
configure_low_latency, create_codec, try_set_frame_rate,
};
use super::{DecodeOptions, FRAME_PARK_CAP, IN_FLIGHT_CAP, PENDING_SPLIT_CAP};
/// One decoded output buffer ready to release: its codec buffer index + the pts the codec echoed
/// (from the output callback's `BufferInfo`), used to pair the `decode` HUD stat, and the
/// wall-clock instant the output callback fired — the spec's `decoded` point ("decoder output
/// frame available"), stamped at the callback so the event-channel hop + coalescing wait in the
/// loop never inflates the decode stage.
struct OutputReady {
index: usize,
pts_us: u64,
decoded_ns: i128,
}
/// Events the async decode loop reacts to. The codec's async-notify callbacks (which run on its
/// internal looper thread) push the codec ones; the feeder thread pushes `Au`. Each carries only
/// owned/`Copy` data so the callback closures satisfy the `Send` bound and never touch the codec.
enum DecodeEvent {
/// A received access unit from the feeder, ready to queue into the decoder. The `bool` is the
/// feeder's [`NativeClient::note_frame_index`] verdict — `true` when this AU revealed a forward
/// frame-index gap, so the loop arms the freeze gate (the feeder already fired the RFI request).
Au(Frame, bool),
/// An input buffer slot freed (index) — we can queue an AU into it.
InputAvailable(usize),
/// A decoded frame is ready (buffer index + echoed pts + the callback-time `decoded` stamp).
OutputAvailable {
index: usize,
pts_us: u64,
decoded_ns: i128,
},
/// The output format changed — re-check the stream's colour signalling (HDR DataSpace).
FormatChanged,
/// The codec reported an error; `fatal` when neither recoverable nor transient.
Error { fatal: bool },
}
/// The event-driven async decode loop (default; see [`run`]/[`USE_ASYNC_DECODE`]). The codec drives
/// us: an async-notify callback fires the instant an input buffer frees or a frame finishes
/// decoding, so a decoded frame is presented immediately instead of waiting out a poll interval (the
/// latency the sync loop left on the table). The callbacks run on the codec's internal looper thread
/// and only *push events* — every `AMediaCodec` buffer op stays on this thread, which owns the codec,
/// sidestepping the self-reference that would arise from a callback calling back into the codec it's
/// stored in. A small `pf-decode-feed` thread blocks on the network so this loop never does.
pub(super) fn run_async(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
let mime = codec_mime(client.codec);
let mut codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (async, low-latency feature: {ll_feature})"
);
// The event channel: the callbacks + feeder push, this loop pulls. `Sender` is `Send`, so the
// callback closures (each capturing a clone) satisfy the async-notify `Send` bound.
let (ev_tx, ev_rx) = mpsc::channel::<DecodeEvent>();
// Install the callbacks BEFORE configure()/start() so we're in async mode from the first buffer.
// Each just forwards an index/flag — no codec access here (the codec owns these closures).
{
let out_tx = ev_tx.clone();
let in_tx = ev_tx.clone();
let fmt_tx = ev_tx.clone();
let err_tx = ev_tx.clone();
let cb = AsyncNotifyCallback {
on_input_available: Some(Box::new(move |idx| {
let _ = in_tx.send(DecodeEvent::InputAvailable(idx));
})),
on_output_available: Some(Box::new(move |idx, info| {
let _ = out_tx.send(DecodeEvent::OutputAvailable {
index: idx,
pts_us: info.presentation_time_us().max(0) as u64,
// The `decoded` HUD point: stamp HERE, on the codec's looper thread, so the
// decode stage ends when the frame actually became available — not after the
// channel hop + whatever work the loop coalesces in front of presenting it.
decoded_ns: now_realtime_ns(),
});
})),
on_format_changed: Some(Box::new(move |_fmt| {
let _ = fmt_tx.send(DecodeEvent::FormatChanged);
})),
on_error: Some(Box::new(move |e, code, _detail| {
let fatal = !code.is_recoverable() && !code.is_transient();
if fatal {
log::error!("decode: fatal codec error — stream will stop: {e:?}");
} else {
log::warn!("decode: codec error {e:?} (recoverable)");
}
let _ = err_tx.send(DecodeEvent::Error { fatal });
})),
};
if let Err(e) = codec.set_async_notify_callback(Some(cb)) {
log::error!("decode: set_async_notify_callback failed: {e}");
return;
}
}
// Build the low-latency format (identical keys to the sync path).
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
configure_low_latency(&mut format, &codec_name, low_latency_mode);
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: decoder started (async) at {}x{}",
mode.width,
mode.height
);
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// Skew-corrected latency stats (spec: design/stats-unification.md). Receipt stamps (keyed by the
// pts we queue) live in a shared map: the feeder writes them at receipt, this loop pairs decoded
// output back to them. Behind a `Mutex` since two threads touch it — only ever locked while the
// HUD is visible.
let clock_offset = client.clock_offset_shared();
// Whether the adaptive-bitrate controller wants the `decode` stage as its decoder-backlog
// signal (Automatic, non-PyroWave): then `in_flight` is fed regardless of the HUD.
let measure_decode = client.wants_decode_latency();
let in_flight = Arc::new(Mutex::new(VecDeque::<(u64, i128)>::new()));
// Display stage (spec `display` + the capture→displayed headline): the rendered frame is
// parked in the tracker at release; the OnFrameRendered callback pairs it with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
let tracker = DisplayTracker::new(stats.clone(), clock_offset.clone());
let render_cb = install_render_callback(&codec, &tracker);
// Feeder thread: block on the network so this loop doesn't (an AU's arrival becomes an event that
// wakes us immediately, with no input-side poll latency). It also records the `received` HUD stat.
let feeder = {
let client = client.clone();
let stats = stats.clone();
let in_flight = in_flight.clone();
let clock_offset = clock_offset.clone();
let shutdown = shutdown.clone();
let ev_tx = ev_tx.clone();
std::thread::Builder::new()
.name("pf-decode-feed".into())
.spawn(move || {
feeder_loop(
client,
stats,
measure_decode,
in_flight,
clock_offset,
shutdown,
ev_tx,
);
})
.ok()
};
drop(ev_tx); // only the feeder + callbacks keep the channel alive now
// ADPF: same as the sync path — register this thread now, create the session lazily on the first
// presented frame (by when the pump + audio + feeder threads have registered their tids too).
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread();
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
let mut free_inputs: VecDeque<usize> = VecDeque::new();
let mut pending_aus: VecDeque<Frame> = VecDeque::new();
let mut ready: Vec<OutputReady> = Vec::new();
let mut applied_ds: Option<DataSpace> = None;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
// AUs larger than the codec input buffer, dropped whole (see `feed`/`feed_ready`).
let mut oversized_dropped: u64 = 0;
// Freeze-until-reanchor gate (see the sync loop for the rationale). Armed on a frame-index gap
// (the feeder's Au verdict), a parked-AU overflow drop, a dropped-count climb, or a recoverable
// codec error; `recovery_flags` carries each AU's user_flags from `dispatch_event` (feed) to
// `present_ready` (present), keyed by the codec-echoed pts.
let mut gate = ReanchorGate::new(client.frames_dropped());
let mut recovery_flags: VecDeque<(u64, u32)> = VecDeque::new();
let mut last_kf_req: Option<Instant> = None;
// Productive (dispatch+feed+present) time between displayed frames; reported to ADPF once one is
// presented. The blocking event wait is excluded (idle, not work) — same accounting as the sync loop.
let mut work_accum_ns: i64 = 0;
let mut fatal = false;
while !shutdown.load(Ordering::Relaxed) && !fatal {
// Block for the next event (idle wait — excluded from the work tally). The short timeout
// drives loss-recovery housekeeping when the pipeline is momentarily quiet.
let ev0 = match ev_rx.recv_timeout(Duration::from_millis(5)) {
Ok(ev) => Some(ev),
Err(mpsc::RecvTimeoutError::Timeout) => None,
Err(mpsc::RecvTimeoutError::Disconnected) => break,
};
let work_t0 = Instant::now();
let mut fmt_dirty = false;
let mut aus_dropped: u64 = 0;
if let Some(ev) = ev0 {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
&mut gate,
&mut recovery_flags,
));
}
// Coalesce every other event already queued into this one work pass — correct newest-only
// presentation across a decode burst, and batched feeding.
while let Ok(ev) = ev_rx.try_recv() {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
&mut gate,
&mut recovery_flags,
));
}
stats.note_skipped(aus_dropped); // parked-AU overflow drops are client-side skips too
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
feed_ready(
&codec,
&client,
&mut pending_aus,
&mut free_inputs,
&mut fed,
&mut oversized_dropped,
);
let had_output = !ready.is_empty();
present_ready(
&codec,
&client,
measure_decode,
&mut ready,
&stats,
&in_flight,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut rendered,
&mut discarded,
&mut gate,
&mut recovery_flags,
);
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if had_output {
if !hint_tried {
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
if rendered > 0 && rendered % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
}
// Loss recovery + overdue backstop, folded through the gate. A parked-AU overflow drop is itself
// a loss, so it arms the freeze directly; the gate's `poll` then arms on a dropped-count climb
// and re-asks on an overdue freeze. All keyframe intents route through the shared 100 ms
// throttle so a multi-frame recovery gap can't flood the control stream.
let now = Instant::now();
if aus_dropped > 0 {
gate.arm(now);
}
if (gate.poll(client.frames_dropped(), now) || aus_dropped > 0)
&& last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = client.request_keyframe();
}
}
let _ = codec.stop();
shutdown.store(true, Ordering::SeqCst); // ensure the feeder wakes and exits, then join it
if let Some(j) = feeder {
let _ = j.join();
}
drop(codec); // AMediaCodec_delete — after this no render callback can fire
if let Some(ud) = render_cb {
// SAFETY: the codec was dropped above; this registration's single reclaim.
unsafe { release_render_callback(ud) };
}
log::info!("decode: stopped (async, fed={fed} rendered={rendered} discarded={discarded})");
}
/// The `pf-decode-feed` thread: block on the connector for the next access unit so the async loop
/// never has to. Records the `received` HUD stat (receipt point) — including the Phase-2 host/network
/// split from any matching 0xCF host timings — then hands the AU to the loop via the event channel.
/// Exits when `shutdown` is set, the session closes, or the loop's receiver is gone.
fn feeder_loop(
client: Arc<NativeClient>,
stats: Arc<crate::stats::VideoStats>,
measure_decode: bool,
in_flight: Arc<Mutex<VecDeque<(u64, i128)>>>,
clock_offset: Arc<AtomicI64>,
shutdown: Arc<AtomicBool>,
ev_tx: mpsc::Sender<DecodeEvent>,
) {
// Received AUs awaiting their 0xCF host timing (Phase-2 split), as (pts_ns, capture→received µs).
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
while !shutdown.load(Ordering::Relaxed) {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
// Loss recovery (RFI): a forward frame-index gap fires a throttled reference-frame-
// invalidation request so an RFI-capable host recovers with a cheap clean P-frame
// instead of a full IDR (the frames_dropped keyframe path is the backstop). The gap
// verdict rides the Au event so the decode loop arms its freeze gate on the same signal.
let gap = client.note_frame_index(frame.frame_index);
// Park the receipt stamp (keyed by the pts the codec echoes) whenever the `decode`
// stage is consumed: the HUD, or the ABR decode signal (`measure_decode`). The
// HUD-only `received` point + host/network split stay gated on the overlay.
if stats.enabled() || measure_decode {
let received_ns = now_realtime_ns();
{
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((frame.pts_ns / 1000, received_ns));
if g.len() > IN_FLIGHT_CAP {
g.pop_front(); // stale — codec never echoed it back
}
}
if stats.enabled() {
let clock_offset = clock_offset.load(Ordering::Relaxed) as i128;
let lat_ns = received_ns + clock_offset - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front();
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
}
if ev_tx.send(DecodeEvent::Au(frame, gap)).is_err() {
break; // the decode loop is gone
}
}
Err(PunktfunkError::NoFrame) => {} // timeout — re-check shutdown and poll again
Err(_) => break, // session closed
}
}
}
/// Route one [`DecodeEvent`] into the loop's working sets. Returns `true` only when a parked AU was
/// dropped on overflow (the caller then requests a keyframe).
#[allow(clippy::too_many_arguments)] // two call sites; the freeze gate + flag map are threaded in
fn dispatch_event(
ev: DecodeEvent,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
ready: &mut Vec<OutputReady>,
fmt_dirty: &mut bool,
fatal: &mut bool,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) -> bool {
match ev {
DecodeEvent::Au(f, gap) => {
// A forward frame-index gap arms the freeze; park this AU's flags for the present side to
// fold `on_decoded` (keyed by the pts the codec will echo).
if gap {
gate.arm(Instant::now());
}
recovery_flags.push_back((f.pts_ns / 1000, f.flags));
if recovery_flags.len() > IN_FLIGHT_CAP {
recovery_flags.pop_front();
}
pending_aus.push_back(f);
if pending_aus.len() > FRAME_PARK_CAP {
pending_aus.pop_front(); // sustained overflow — drop oldest, signal a keyframe request
return true;
}
}
DecodeEvent::InputAvailable(i) => free_inputs.push_back(i),
DecodeEvent::OutputAvailable {
index,
pts_us,
decoded_ns,
} => ready.push(OutputReady {
index,
pts_us,
decoded_ns,
}),
DecodeEvent::FormatChanged => *fmt_dirty = true,
DecodeEvent::Error { fatal: f } => {
if f {
*fatal = true;
} else {
// A recoverable/transient codec error is a decode hiccup on a broken reference chain —
// arm the freeze so the concealed output it recovers into is held off the screen.
gate.arm(Instant::now());
}
}
}
false
}
/// Queue as many parked AUs as there are free input buffer slots (async mode: the indices come from
/// `InputAvailable` callbacks, not a dequeue). Each AU is copied into its codec input buffer and
/// submitted; an AU larger than the buffer is DROPPED (+ a recovery keyframe requested) — a
/// truncated AU is corrupt input the decoder chews on silently, poisoning the reference chain.
fn feed_ready(
codec: &MediaCodec,
client: &NativeClient,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
fed: &mut u64,
oversized_dropped: &mut u64,
) {
while !pending_aus.is_empty() && !free_inputs.is_empty() {
let idx = free_inputs.pop_front().unwrap();
let frame = pending_aus.pop_front().unwrap();
let pts_us = frame.pts_ns / 1000;
let Some(dst) = codec.input_buffer(idx) else {
log::warn!("decode: input_buffer({idx}) returned None — dropping AU");
continue;
};
let au = &frame.data;
if au.len() > dst.len() {
// The slot was never queued, so it stays ours — recycle it for the next AU.
free_inputs.push_front(idx);
*oversized_dropped += 1;
log::warn!(
"decode: AU {} > input buffer {} — dropped ({} so far), requesting keyframe",
au.len(),
dst.len(),
*oversized_dropped
);
let _ = client.request_keyframe();
continue;
}
let n = au.len();
// SAFETY: `au` (wire AU) and `dst` (codec input buffer) are distinct allocations, both valid
// for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so this initializes dst[..n].
unsafe {
std::ptr::copy_nonoverlapping(au.as_ptr(), dst.as_mut_ptr().cast::<u8>(), n);
}
if let Err(e) = codec.queue_input_buffer_by_index(idx, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer_by_index: {e}");
} else {
*fed += 1;
}
}
}
/// Present only the NEWEST ready output (render = true) and release the rest without rendering — a
/// burst of stale frames on glass is worse than skipping to the freshest (the sync loop's newest-ready
/// policy, callback-driven). Every dequeued buffer, rendered or not, is the HUD's `decoded`
/// measurement point (it finished decoding either way); samples are recorded in pts order so the
/// receipt-map eviction stays monotonic. The presented frame's `(pts, decoded stamp)` is parked in
/// `tracker` for the OnFrameRendered callback — the `display` stage's other endpoint. `ready` is
/// drained.
#[allow(clippy::too_many_arguments)] // one call site; mirrors the sync loop's drain
fn present_ready(
codec: &MediaCodec,
client: &NativeClient,
measure_decode: bool,
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
clock_offset: i64,
tracker: &DisplayTracker,
rendered: &mut u64,
discarded: &mut u64,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) {
if ready.is_empty() {
return;
}
// Pair each output's decode stage (feeds the ABR decode signal always; the HUD histogram only
// while visible) — both consume the receipt map, so enter for either.
if stats.enabled() || measure_decode {
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
);
}
}
// Fold EVERY output through the gate in pts (== decode) order — even the ones newest-wins discards —
// so the two-mark re-anchor count stays correct; the newest's verdict decides whether it reaches
// glass (`false` = withheld concealment; the SurfaceView keeps the last rendered frame frozen on).
let now = Instant::now();
let last = ready.len() - 1;
let mut skipped: u64 = 0;
for (i, o) in ready.drain(..).enumerate() {
let flags = take_flags(recovery_flags, o.pts_us);
let present = gate.on_decoded(flags, false, now) == GateVerdict::Present;
let render = i == last && present;
match codec.release_output_buffer_by_index(o.index, render) {
Ok(()) if render => {
*rendered += 1;
if stats.enabled() {
tracker.note_rendered(o.pts_us, o.decoded_ns);
}
}
Ok(()) => {
*discarded += 1;
skipped += 1;
}
Err(e) => {
log::warn!(
"decode: release_output_buffer_by_index({}, {render}): {e}",
o.index
)
}
}
}
stats.note_skipped(skipped); // HUD `skipped` counter (newest-wins + held-off drops); no-op hidden
}
@@ -1,224 +0,0 @@
//! Display/frame-rendered tracking, render-callback registration, HDR dataspace mapping.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::MediaCodec;
use ndk::native_window::NativeWindow;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicI64, Ordering};
use std::sync::{Arc, Mutex};
use super::latency::now_realtime_ns;
use super::RENDERED_CAP;
/// `CLOCK_MONOTONIC` now in nanoseconds — the base of the `systemNano` render timestamp the
/// `OnFrameRendered` callback reports (Android's `System.nanoTime`), read only to re-base that
/// stamp onto `CLOCK_REALTIME` (see [`on_frame_rendered`]).
fn now_monotonic_ns() -> i128 {
let mut ts = libc::timespec {
tv_sec: 0,
tv_nsec: 0,
};
// SAFETY: `clock_gettime` with a valid out-pointer is an always-safe syscall.
unsafe { libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut ts) };
ts.tv_sec as i128 * 1_000_000_000 + ts.tv_nsec as i128
}
/// State shared between the decode loop and the `AMediaCodec` `OnFrameRendered` callback (which
/// fires on a codec-internal thread): rendered frames awaiting their render timestamp, so the HUD
/// gets the spec's `display` stage (decoded→displayed) and the `capture→displayed` end-to-end
/// headline (`design/stats-unification.md` — this replaces Android's v1 `capture→decoded`
/// endpoint whenever the platform delivers render callbacks).
pub(super) struct DisplayTracker {
stats: Arc<crate::stats::VideoStats>,
/// Live host-minus-client clock offset (ns) for the skew-corrected end-to-end sample —
/// loaded per callback so mid-stream re-syncs apply. Holding the handle (not the client)
/// keeps the leaked render-callback refcount from pinning the whole session alive.
clock_offset: Arc<AtomicI64>,
/// `(pts_us, decoded_real_ns)` of frames released with `render = true`, in release order,
/// awaiting their callback. Pushes are HUD-gated by the caller, so this stays empty (and the
/// callback early-outs) while the overlay is hidden.
rendered: Mutex<VecDeque<(u64, i128)>>,
}
impl DisplayTracker {
pub(super) fn new(
stats: Arc<crate::stats::VideoStats>,
clock_offset: Arc<AtomicI64>,
) -> Arc<DisplayTracker> {
Arc::new(DisplayTracker {
stats,
clock_offset,
rendered: Mutex::new(VecDeque::new()),
})
}
/// Park one just-rendered frame's `(pts, decoded stamp)` for the render callback to pair.
/// Caller gates on the HUD being visible.
pub(super) fn note_rendered(&self, pts_us: u64, decoded_ns: i128) {
let mut g = self
.rendered
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((pts_us, decoded_ns));
if g.len() > RENDERED_CAP {
g.pop_front(); // render callbacks stopped coming (allowed under load) — evict
}
}
}
/// Register [`on_frame_rendered`] on the codec (`AMediaCodec_setOnFrameRenderedCallback`,
/// **API 33** — "Available since Android T" per the NDK header; only the *Java* listener dates
/// back further). That sits above the API-28 floor, so the entry point is dlsym-resolved at
/// runtime like [`try_set_frame_rate`] — hard-linking it (as 0.9.0 shipped) made
/// `System.loadLibrary` fail on every pre-Android-13 device, taking down all of `NativeBridge`.
/// The `ndk` wrapper has no binding and the call needs the raw codec pointer, which is what the
/// vendored crate's public `as_ptr` patch is for. Returns the userdata pointer holding a leaked
/// `Arc<DisplayTracker>` refcount; the caller MUST reclaim it with [`release_render_callback`]
/// AFTER dropping the codec (`AMediaCodec_delete` is what guarantees no further callback can
/// fire). `None` (nothing to reclaim) if the symbol is absent (API < 33) or the platform refused —
/// the HUD then simply has no `display` stage, exactly the pre-callback behaviour.
pub(super) fn install_render_callback(
codec: &MediaCodec,
tracker: &Arc<DisplayTracker>,
) -> Option<*const DisplayTracker> {
// media_status_t AMediaCodec_setOnFrameRenderedCallback(
// AMediaCodec*, AMediaCodecOnFrameRendered, void*) (API 33)
type SetOnFrameRenderedFn = unsafe extern "C" fn(
*mut ndk_sys::AMediaCodec,
ndk_sys::AMediaCodecOnFrameRendered,
*mut c_void,
) -> ndk_sys::media_status_t;
// SAFETY: `dlopen` of `libmediandk.so`, which the `ndk` media wrapper already links — always
// mapped, so this only bumps its refcount (never closed — process-lifetime handle). `dlsym`
// returns null when the symbol is absent (device below API 33), checked before transmuting the
// non-null pointer to its fn-pointer type.
let set_on_frame_rendered = unsafe {
let lib = libc::dlopen(c"libmediandk.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return None;
}
let sym = libc::dlsym(lib, c"AMediaCodec_setOnFrameRenderedCallback".as_ptr());
if sym.is_null() {
log::info!("decode: no render callback on this API level (<33) — no display stage");
return None;
}
std::mem::transmute::<*mut c_void, SetOnFrameRenderedFn>(sym)
};
let ud = Arc::into_raw(tracker.clone());
// SAFETY: `codec.as_ptr()` is the live codec this thread owns; `ud` outlives the registration
// (reclaimed only after the codec is deleted, per this function's contract).
let status = unsafe {
set_on_frame_rendered(codec.as_ptr(), Some(on_frame_rendered), ud as *mut c_void)
};
if status == ndk_sys::media_status_t::AMEDIA_OK {
Some(ud)
} else {
log::warn!("decode: setOnFrameRenderedCallback failed ({status:?}) — no display stage");
// SAFETY: registration failed, so the codec never took the reference — reclaim it now.
unsafe { drop(Arc::from_raw(ud)) };
None
}
}
/// Reclaim [`install_render_callback`]'s leaked `Arc` refcount.
///
/// # Safety
/// Call exactly once, and only after the codec the callback was registered on has been dropped —
/// deleting the codec stops its internal threads, so no callback can still be running (or run
/// later) against this pointer.
pub(super) unsafe fn release_render_callback(ud: *const DisplayTracker) {
drop(Arc::from_raw(ud));
}
/// The `AMediaCodecOnFrameRendered` trampoline: fires (possibly batched) on a codec-internal
/// thread once per output frame actually placed on the output surface, with SurfaceFlinger's
/// render timestamp. That timestamp (`system_nano`) is on `CLOCK_MONOTONIC`, so it is re-based
/// onto `CLOCK_REALTIME` here — against monotonic-now at callback time, which also cancels any lag
/// between the frame rendering and the (batchable) callback delivery — to subtract against the
/// receipt/decode stamps and the host capture pts. Records the HUD's `displayed` point:
/// `end-to-end` = capture→displayed (skew-corrected) and `display` = decoded→displayed
/// (single-clock local). Panic-free by construction (poison-proof lock, saturating math) — an
/// unwind out of an `extern "C"` fn would abort the process.
unsafe extern "C" fn on_frame_rendered(
_codec: *mut ndk_sys::AMediaCodec,
userdata: *mut c_void,
media_time_us: i64,
system_nano: i64,
) {
let t = &*(userdata as *const DisplayTracker);
if !t.stats.enabled() {
return; // HUD hidden — the ring is empty too (pushes are caller-gated)
}
let displayed_ns = now_realtime_ns() - (now_monotonic_ns() - system_nano as i128);
let pts_us = media_time_us.max(0) as u64;
// Pair the frame back to its release record, evicting older entries (their callbacks were
// dropped by the platform, or the entry predates a HUD toggle) — same monotonic-eviction
// discipline as `note_decoded_pts`.
let mut decoded_ns = None;
{
let mut g = t
.rendered
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
while let Some(&(p, d)) = g.front() {
if p > pts_us {
break; // future frame — leave it for its own callback
}
g.pop_front();
if p == pts_us {
decoded_ns = Some(d);
break;
}
}
}
let e2e_ns =
displayed_ns + t.clock_offset.load(Ordering::Relaxed) as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
let display_us = decoded_ns.map(|d| ((displayed_ns - d).max(0) / 1000) as u64);
t.stats.note_displayed(e2e_us, display_us);
}
/// React to an output-format change by signalling the stream's HDR dataspace on the Surface (SDR
/// streams leave the default alone). The AMediaCodec analogue of the sync loop's `OutputFormatChanged`
/// handling; safe to call repeatedly (`applied_ds` dedups).
pub(super) fn apply_hdr_dataspace(
codec: &MediaCodec,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
) {
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => {
log::warn!("decode: set_buffers_data_space({ds}) failed (non-fatal): {e}")
}
}
}
}
}
/// Map the decoder's reported output colour to a BT.2020 HDR dataspace, or `None` for SDR. The
/// integer values are the Android MediaFormat colour constants the NDK shares: COLOR_TRANSFER
/// ST2084 = 6 (PQ/HDR10), HLG = 7; COLOR_RANGE FULL = 1, LIMITED = 2 (the host encodes limited).
pub(super) fn hdr_dataspace(codec: &MediaCodec) -> Option<DataSpace> {
let fmt = codec.output_format();
let full_range = fmt.i32("color-range") == Some(1);
match fmt.i32("color-transfer") {
Some(6) => Some(if full_range {
DataSpace::Bt2020Pq
} else {
DataSpace::Bt2020ItuPq
}),
Some(7) => Some(if full_range {
DataSpace::Bt2020Hlg
} else {
DataSpace::Bt2020ItuHlg
}),
_ => None, // SDR (BT.709 / SDR_VIDEO) or unspecified
}
}
@@ -1,83 +0,0 @@
//! Decode-latency bookkeeping: realtime clock + decoded-pts / user-flags stat recording.
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
/// Wall-clock now in nanoseconds (CLOCK_REALTIME basis), to compare against the host-stamped
/// capture `pts_ns` after the skew offset is applied.
pub(super) fn now_realtime_ns() -> i128 {
use std::time::{SystemTime, UNIX_EPOCH};
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos() as i128)
.unwrap_or(0)
}
/// HUD `decoded` point for one dequeued output frame, keyed by the echoed `presentationTimeUs`:
/// build the end-to-end (capture→decoded, skew-corrected, clamped to (0, 10 s)) and `decode`
/// (received→decoded, single-clock local, ≥ 0) samples and hand them to
/// [`crate::stats::VideoStats::note_decoded`]. The pts keys the receipt stamp in `in_flight`;
/// entries older than it are evicted (decode order == input order here — low-latency, no
/// B-frames — so anything before it was dropped inside the codec or stamped before a flush).
/// `decoded_ns` is the availability instant: the dequeue (sync loop) or the output callback's
/// stamp (async loop).
pub(super) fn note_decoded_pts(
client: &NativeClient,
measure_decode: bool,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
pts_us: u64,
decoded_ns: i128,
) {
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
in_flight.pop_front();
if p == pts_us {
received_ns = Some(r);
break;
}
}
let decode_us = received_ns.map(|r| ((decoded_ns - r).max(0) / 1000) as u64);
// Adaptive bitrate: the `decode` stage (received→decoded, single-clock local) IS the decoder-
// backlog signal — the only bottleneck the host-side network signals can't see (a fast LAN
// feeding a slower mobile decoder). Report it whenever the controller is armed, regardless of
// the HUD; `report_decode_us` is a cheap accumulate the pump windows.
if measure_decode {
if let Some(us) = decode_us {
client.report_decode_us(us.min(u32::MAX as u64) as u32);
}
}
// HUD histogram: only while the overlay is visible (a measure-only caller enters here for the
// ABR report alone). `end-to-end` = capture→decoded (skew-corrected) tiles the `decode` stage.
// pts_us is the truncated frame.pts_ns/1000 we queued, so ×1000 re-approximates capture time to
// < 1 µs — negligible against the ms-scale figures shown.
if stats.enabled() {
let e2e_ns = decoded_ns + clock_offset as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
stats.note_decoded(e2e_us, decode_us);
}
}
/// The AU `user_flags` for a decoded output, keyed by the echoed `presentationTimeUs`. Recovery
/// signalling (FLAG_SOF IDR marker / RECOVERY_ANCHOR / RECOVERY_POINT) rides the AU's flags, which are
/// only in scope at feed time — so the feed side parks `(pts_us, flags)` here and the present side
/// looks them up to fold [`ReanchorGate::on_decoded`]. Decode order == input order (low-latency, no
/// B-frames), so this evicts entries older than `pts_us` as it goes; a miss (probe filler, or an entry
/// aged past the cap) reads `0` — no recovery flags, decoded normally.
pub(super) fn take_flags(map: &mut VecDeque<(u64, u32)>, pts_us: u64) -> u32 {
while let Some(&(p, f)) = map.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
map.pop_front();
if p == pts_us {
return f;
}
}
0
}
-84
View File
@@ -1,84 +0,0 @@
//! Android video decode (android-only): pull HEVC access units from the connector and render them
//! to the SurfaceView via NDK `AMediaCodec` — hardware decode, zero per-frame JNI.
//!
//! One-in/one-out: the host opens every stream with an IDR carrying VPS/SPS/PPS **in-band**, so the
//! decoder needs no out-of-band codec-specific data — we configure with mime + the negotiated
//! WxH (from [`NativeClient::mode`]) and feed each access unit as it arrives. The decode thread owns
//! the codec + window for its whole life; [`crate::session`] signals it to stop via the shared flag.
mod async_loop;
mod display;
mod latency;
mod setup;
mod sync_loop;
use async_loop::run_async;
pub(crate) use setup::{codec_label, codec_mime};
use sync_loop::run_sync;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
/// Cap on AUs parked in the async loop awaiting a free codec input slot. Matches the connector's
/// own frame-channel depth; on sustained overflow the oldest is dropped and a keyframe requested
/// (same recovery as a reassembler drop). In steady state this stays near-empty.
const FRAME_PARK_CAP: usize = 16;
/// Cap on the pts→received-timestamp map below: MediaCodec holds only a handful of frames in
/// flight, so anything beyond this is stale (codec flushed / HUD toggled) and gets evicted.
const IN_FLIGHT_CAP: usize = 64;
/// Cap on received AUs awaiting their 0xCF host timing (Phase 2 host/network split): the timing
/// datagram trails its AU by at most the wire, so a match lands within a frame or two — anything
/// this deep is a lost datagram (or an old host that never sends any) and gets evicted.
const PENDING_SPLIT_CAP: usize = 256;
/// Cap on rendered frames parked in [`DisplayTracker`] awaiting their `OnFrameRendered` render
/// timestamp: the callback trails its release by at most a vsync or two, so anything this deep
/// means the platform stopped delivering render callbacks (allowed under load, per the docs) and
/// gets evicted.
const RENDERED_CAP: usize = 64;
/// Whether low-latency mode uses the event-driven async decode loop (default) or the synchronous
/// poll loop. Flip to `false` to A/B the two on the HUD (`design/…`); the async loop presents a
/// decoded frame the instant it's ready instead of waiting out a poll interval. Only consulted when
/// the user's "Low-latency mode" toggle is ON (now the default) — off, the sync loop always runs (the
/// original pipeline, kept as the per-device escape hatch).
const USE_ASYNC_DECODE: bool = true;
/// Per-session decode configuration, resolved by the JNI layer (`nativeStartVideo`) and passed to
/// the decode loop. Bundled so the loop entry points don't sprout a wide argument list.
pub(crate) struct DecodeOptions {
/// The decoder Kotlin ranked from `MediaCodecList` (`VideoDecoders.pickDecoder`). `None`/empty ⇒
/// let the platform resolve the default decoder for the MIME.
pub decoder_name: Option<String>,
/// Whether Kotlin found the chosen decoder advertises `FEATURE_LowLatency` (queryable only via
/// the Java `CodecCapabilities` API) — surfaced on the HUD next to the decoder name.
pub ll_feature: bool,
/// The user's "Low-latency mode" master toggle. On (default) ⇒ the full fast pipeline: async
/// decode loop, per-SoC vendor keys, pipeline thread boosts, ADPF max-performance, forced TV
/// mode switch. Off ⇒ the original synchronous pre-overhaul pipeline, kept as the per-device
/// escape hatch.
pub low_latency_mode: bool,
/// TV form factor (Kotlin's `UiModeManager`): actively drive the HDMI output into the stream's
/// refresh mode, vs. the softer seamless hint on a phone/tablet.
pub is_tv: bool,
}
/// The decode entry point on the `pf-decode` thread: dispatches to the async or synchronous loop.
/// Both run until `shutdown` is set or the session closes.
pub fn run(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
if opts.low_latency_mode && USE_ASYNC_DECODE {
run_async(client, window, shutdown, stats, opts);
} else {
run_sync(client, window, shutdown, stats, opts);
}
}
-254
View File
@@ -1,254 +0,0 @@
//! Codec creation, low-latency config, thread/frame-rate tuning, HDR static-info encode.
use ndk::media::media_codec::MediaCodec;
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use std::ffi::c_void;
/// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). Shared by the decode
/// thread and `nativeVideoMime` (which tells Kotlin what to rank decoders for). AV1 uses the
/// AOSP `video/av01` type; anything not H.264/AV1 is treated as HEVC (every pre-negotiation host
/// emitted HEVC).
pub(crate) fn codec_mime(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
punktfunk_core::quic::CODEC_AV1 => "video/av01",
_ => "video/hevc",
}
}
/// A short human label for the codec the host resolved, for the stats HUD's video-feed line
/// (`"H.264"` / `"HEVC"` / `"AV1"` / `"PyroWave"`). Mirrors [`codec_mime`]'s fallback: anything
/// not H.264/AV1/PyroWave is reported as HEVC (every pre-negotiation host emitted HEVC). Kept
/// beside [`codec_mime`] because the MIME collapses PyroWave onto `video/hevc` and so can't name it.
pub(crate) fn codec_label(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "H.264",
punktfunk_core::quic::CODEC_AV1 => "AV1",
punktfunk_core::quic::CODEC_PYROWAVE => "PyroWave",
_ => "HEVC",
}
}
/// Create the decoder: prefer the specific codec Kotlin ranked from `MediaCodecList`
/// (`from_codec_name`), falling back to the platform's default decoder for the MIME
/// (`from_decoder_type`) if that name can't be created (codec busy / renamed across an OS update).
pub(super) fn create_codec(mime: &str, preferred: Option<&str>) -> Option<MediaCodec> {
if let Some(name) = preferred.filter(|n| !n.is_empty()) {
if let Some(c) = MediaCodec::from_codec_name(name) {
return Some(c);
}
log::warn!(
"decode: from_codec_name({name}) failed — falling back to default {mime} decoder"
);
}
MediaCodec::from_decoder_type(mime)
}
/// Apply the low-latency MediaFormat keys for `codec_name`.
///
/// `aggressive` = the "Low-latency mode" master toggle. **Off** ⇒ the pre-overhaul key set,
/// byte-for-byte — the standard `low-latency` key, the blind Qualcomm vendor twin, `priority = 0` AND
/// `operating-rate = MAX` set together — kept as the per-device escape hatch (the profile every device
/// streamed with before the overhaul). **On** (default) ⇒ the Moonlight-parity
/// profile: MediaTek's `vdec-lowlatency` (unconditionally — ignored off MediaTek), the per-SoC
/// vendor extension keys (gated on the decoder-name prefix the way Moonlight-Android does, since a
/// key one vendor honours is meaningless on another), and one *mutually exclusive* clock hint.
///
/// Vendor keys mirror Moonlight's `MediaCodecHelper` (verified against current source): Qualcomm
/// picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon, MediaTek. NVIDIA
/// Tegra / Rockchip / Realtek expose no such key (nor does Moonlight) — they're covered by the
/// standard key + clock hint + being ranked first in `VideoDecoders`.
pub(super) fn configure_low_latency(format: &mut MediaFormat, codec_name: &str, aggressive: bool) {
// Standard key: request the no-reorder low-latency path where the platform decoder supports it.
format.set_i32("low-latency", 1);
if !aggressive {
// The original profile: the Qualcomm vendor twin set blind (unknown keys are ignored by
// other vendors' codecs), realtime priority, and the AOSP "unbounded" operating-rate
// sentinel — decode each frame at max clocks rather than pacing to the frame rate.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
format.set_i32("priority", 0); // 0 = realtime
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
return;
}
// MediaTek's low-latency key — very common (mid/budget phones + many Google TV / Fire TV boxes).
// Set unconditionally like the standard key: MediaTek decoders honour it, others ignore it, so it
// covers MediaTek whatever the exact decoder name (omx.mtk / c2.mtk / an OEM rename). Moonlight
// does the same, and also relies on it for Amazon's Amlogic fork.
format.set_i32("vdec-lowlatency", 1);
let name = codec_name.to_ascii_lowercase();
let is = |prefix: &str| name.starts_with(prefix);
// Qualcomm Snapdragon (the most common phone SoC): picture-order forces decode-order output
// (kills the reorder buffer on decoders that predate the standard key); low-latency is the older
// vendor twin.
if is("omx.qcom") || is("c2.qti") {
format.set_i32("vendor.qti-ext-dec-picture-order.enable", 1);
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
}
// Samsung Exynos — also covers Google Tensor (Pixel 6+), whose hardware decoder is `c2.exynos.*`.
if is("omx.exynos") || is("c2.exynos") {
format.set_i32("vendor.rtc-ext-dec-low-latency.enable", 1);
}
// Amlogic — the Android TV boxes (onn 4K, Chromecast w/ Google TV, Homatics).
if is("omx.amlogic") || is("c2.amlogic") {
format.set_i32("vendor.low-latency.enable", 1);
}
// HiSilicon / Kirin (older Huawei; paired req/rdy keys).
if is("omx.hisi") || is("c2.hisi") {
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-req",
1,
);
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-rdy",
-1,
);
}
// NVIDIA Tegra (Shield TV) and Rockchip/Realtek (budget TV boxes / smart TVs) expose no
// low-latency vendor key (Moonlight has none either) — their decoders are already low-latency
// oriented, so the standard `low-latency` key + the clock hint below + being ranked first
// (see `VideoDecoders`) is their treatment.
//
// Clock hint, mutually exclusive (matching Moonlight): the AOSP "unbounded" operating-rate
// sentinel (Short.MAX) tells the decoder to run each frame at max clocks and finish ASAP rather
// than pace to the frame rate — shaving per-frame decode latency at a power/heat cost. Only
// Qualcomm is known to handle the sentinel; every other vendor mis-paces on it, so they get the
// plain realtime `priority` hint instead.
if decoder_supports_max_operating_rate(&name) {
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
} else {
format.set_i32("priority", 0); // 0 = realtime
}
}
/// Whether a decoder tolerates `operating-rate = Short.MAX` rather than regressing on it. Follows
/// Moonlight's allowlist: Qualcomm decoders honour the sentinel (the Adreno 620 generation is the
/// known exception Moonlight excludes by GPU model — undetectable from native code here, so it
/// rides the master toggle as its escape hatch). Other vendors fall back to the plain `priority`
/// hint above.
fn decoder_supports_max_operating_rate(name_lower: &str) -> bool {
name_lower.starts_with("omx.qcom") || name_lower.starts_with("c2.qti")
}
/// Raise the pipeline's OTHER hot threads — the core's data-plane pump (UDP receive + FEC
/// reassembly) and the audio decode thread — toward the display band, matching this decode thread's
/// own boost. `setpriority(PRIO_PROCESS, tid)` targets any task in the process, so we do it from
/// here once their tids are known (the same set ADPF hints), without a per-platform priority hook
/// in the shared core. Slightly below the decode thread's -10 so the display path still wins.
/// Best-effort; skips this thread (already boosted) and is non-fatal if the platform refuses.
pub(super) fn boost_hot_threads(tids: &[i32]) {
// SAFETY: `gettid` is an always-safe syscall on the calling thread.
let self_tid = unsafe { libc::gettid() };
for &tid in tids {
if tid == self_tid {
continue;
}
// SAFETY: `setpriority` with PRIO_PROCESS + a live tid in our own process is an always-safe
// syscall; a refusal is reported via the return value, not UB.
unsafe {
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -8) != 0 {
log::debug!("decode: setpriority(-8) on hot tid {tid} failed (non-fatal)");
}
}
}
}
/// Best-effort: raise the decode thread toward Android's URGENT_DISPLAY band so background work
/// can't preempt it under load (which shows up as late/dropped frames). Non-fatal if the platform
/// refuses (foreground apps may set their own threads; the exact floor is policy-dependent).
pub(super) fn boost_thread_priority() {
// SAFETY: `gettid`/`setpriority` on the calling thread are always-safe syscalls. PRIO_PROCESS
// with a TID targets that one task on Linux — the same idiom `Process.setThreadPriority` uses.
unsafe {
let tid = libc::gettid();
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -10) != 0 {
log::warn!(
"decode: setpriority(-10) failed (non-fatal): {}",
std::io::Error::last_os_error()
);
}
}
}
/// Set the surface's frame-rate hint to the stream's refresh so SurfaceFlinger picks a matching
/// display mode and aligns vsync (no 60-in-120 judder). Both NDK entry points sit above our API-28
/// floor, so both are dlsym-resolved at runtime (a hard import of a >floor symbol makes
/// `dlopen`/`System.load` fail on every API-28/29 device, even where this path is never hit —
/// mirrors [`crate::adpf`]):
/// - On a **TV** (`is_tv`): `ANativeWindow_setFrameRateWithChangeStrategy` (**API 31**) with
/// `changeFrameRateStrategy = ALWAYS`, which actively drives the HDMI output into the matching
/// mode (e.g. 60↔120) instead of leaving the panel at its default and judder-matching. The
/// forced switch may blank the panel briefly — acceptable once at stream start, not wanted on a
/// phone. Falls through to the 2-arg hint on API 30.
/// - Otherwise: `ANativeWindow_setFrameRate` (**API 30**) with `compatibility = DEFAULT` — the
/// softer, seamless-preferred hint for phones/tablets and the universal fallback.
///
/// Returns `true` when the platform accepted a hint; `false` on API < 30 (symbols absent) or a
/// decline.
pub(super) fn try_set_frame_rate(window: &NativeWindow, frame_rate: f32, is_tv: bool) -> bool {
// int32_t ANativeWindow_setFrameRate(ANativeWindow*, float frameRate, int8_t compatibility)
type SetFrameRateFn = unsafe extern "C" fn(*mut c_void, f32, i8) -> i32;
// int32_t ANativeWindow_setFrameRateWithChangeStrategy(
// ANativeWindow*, float frameRate, int8_t compatibility, int8_t changeFrameRateStrategy)
type SetFrameRateStrategyFn = unsafe extern "C" fn(*mut c_void, f32, i8, i8) -> i32;
// SAFETY: `dlopen` of the always-mapped `libandroid.so` (only bumps its refcount; never closed —
// process-lifetime handle). Each `dlsym` returns null when the symbol is absent (device below the
// symbol's API level), checked before transmuting the non-null pointer to its fn-pointer type.
// `window.ptr()` is the live `ANativeWindow` this `NativeWindow` owns for the call's duration.
unsafe {
let lib = libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return false;
}
// TV: prefer the API-31 change-strategy form to force the mode switch (strategy 1 = ALWAYS,
// compatibility 0 = DEFAULT). Absent on API 30 ⇒ fall through to the 2-arg hint below.
if is_tv {
let sym = libc::dlsym(
lib,
c"ANativeWindow_setFrameRateWithChangeStrategy".as_ptr(),
);
if !sym.is_null() {
let set = std::mem::transmute::<*mut c_void, SetFrameRateStrategyFn>(sym);
return set(window.ptr().as_ptr().cast(), frame_rate, 0, 1) == 0;
}
}
let sym = libc::dlsym(lib, c"ANativeWindow_setFrameRate".as_ptr());
if sym.is_null() {
return false; // device API < 30 — no per-surface frame-rate hint
}
let set_frame_rate = std::mem::transmute::<*mut c_void, SetFrameRateFn>(sym);
set_frame_rate(window.ptr().as_ptr().cast(), frame_rate, 0) == 0
}
}
/// Serialize [`HdrMeta`](punktfunk_core::quic::HdrMeta) into Android's `KEY_HDR_STATIC_INFO`
/// (`hdr-static-info`) layout: a 25-byte CTA-861.3 / `HDRStaticInfo.Type1` blob — descriptor id 0,
/// then primaries in **R, G, B** order, white point, max/min display luminance, MaxCLL, MaxFALL, all
/// **little-endian** `u16`. Two conversions vs our wire form: HdrMeta stores primaries in ST.2086
/// **G, B, R** order (reorder to R, G, B), and `max_display_mastering_luminance` is in 0.0001-cd/m²
/// units while Android wants **whole nits** (min stays 0.0001-nit). Chromaticities (1/50000) and
/// MaxCLL/MaxFALL (nits) match 1:1.
pub(super) fn android_hdr_static_info(m: &punktfunk_core::quic::HdrMeta) -> [u8; 25] {
let [g, b_, r] = m.display_primaries; // ST.2086 G, B, R
let max_nits = (m.max_display_mastering_luminance / 10_000).min(u16::MAX as u32) as u16;
let min_units = m.min_display_mastering_luminance.min(u16::MAX as u32) as u16;
let fields: [u16; 12] = [
r[0],
r[1],
g[0],
g[1],
b_[0],
b_[1], // R, G, B primaries
m.white_point[0],
m.white_point[1], // white point
max_nits,
min_units, // max (nits) / min (0.0001-nit) display luminance
m.max_cll,
m.max_fall, // MaxCLL / MaxFALL (nits)
];
let mut out = [0u8; 25]; // out[0] = 0 (Type 1 descriptor id), already zero
for (i, v) in fields.iter().enumerate() {
out[1 + i * 2..3 + i * 2].copy_from_slice(&v.to_le_bytes());
}
out
}
@@ -1,547 +0,0 @@
//! The synchronous MediaCodec decode loop (the original poll path) + its feed/drain helpers.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{
DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection,
OutputBuffer,
};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::reanchor::{GateVerdict, ReanchorGate};
use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use super::display::{
hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
configure_low_latency, create_codec, try_set_frame_rate,
};
use super::{DecodeOptions, IN_FLIGHT_CAP, PENDING_SPLIT_CAP};
/// The synchronous poll loop — the original decode path: the only one when low-latency mode is off,
/// and the [`USE_ASYNC_DECODE`] A/B fallback when it's on. Feeds and drains on this one thread; the
/// only blocking wait is a short output dequeue while input is backed up.
pub(super) fn run_sync(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). AMediaCodec needs no
// out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = codec_mime(client.codec);
let codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
// The decoder's *actual* resolved name (Kotlin's pick, or the platform default when it fell
// back) drives both the HUD label and which vendor low-latency keys apply below.
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (low-latency feature: {ll_feature})"
);
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
// Generous input buffer so a large keyframe AU is never truncated.
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
// Standard + per-SoC vendor low-latency keys and the clock hints, gated on the resolved decoder
// name and the master toggle (see `configure_low_latency`).
configure_low_latency(&mut format, &codec_name, low_latency_mode);
// HDR static metadata (ST.2086 mastering + content light level): when an HDR session was
// negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade.
// MediaCodec wants it BEFORE configure(), and the host sends a 0xCE right after the handshake,
// so it's typically already queued; wait briefly otherwise. The Surface DataSpace (applied on
// OutputFormatChanged below) carries transfer/primaries regardless — this adds the luminance the
// tone-mapper needs. A non-HDR display still gets sensible SurfaceFlinger tone-mapping.
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: {mime} decoder started at {}x{}",
mode.width,
mode.height
);
// Tell the display the stream's refresh so Android can pick a matching display mode and align
// vsync (no 60-in-120 judder on high-refresh panels). `ANativeWindow_setFrameRate` is NDK API 30,
// above our API-28 floor, so we resolve it at runtime (see `try_set_frame_rate`) rather than link
// it — a hard import would stop `libpunktfunk_android.so` loading at all on API 28/29. Absent
// there ⇒ we simply skip the hint (non-fatal; the stream renders fine without it).
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// ADPF: hint the platform that the whole video pipeline — this pf-decode feed/drain/present
// loop, the core's data-plane pump (UDP receive + FEC reassembly), and the audio thread — runs a
// per-frame real-time workload, so the CPU governor keeps those threads on fast cores at high
// clocks instead of down-clocking between frames or parking them on a little core. Snapdragon's
// ADPF backend responds well to this. We register this thread now but create the session lazily
// on the first presented frame: by then the pump + audio threads have registered their ids too,
// and ADPF `createSession` rejects a set with any not-yet-live/dead tid. No-op below API 33.
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread(); // this decode thread → the pipeline's hot-thread set
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
// Accumulates the loop's productive (feed+drain) time between displayed frames; reported to ADPF
// once per rendered frame against the frame-period target.
let mut work_accum_ns: i64 = 0;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
// AUs larger than the codec input buffer, dropped whole (see `feed`/`feed_ready`).
let mut oversized_dropped: u64 = 0;
// The AU waiting for a free codec input buffer. `feed` is non-blocking; on transient input
// pressure the AU stays parked here instead of being dropped (a drop forces a keyframe
// round-trip) and we only pop the next one once it's queued.
let mut pending: Option<Frame> = None;
// Freeze-until-reanchor: the shared post-loss gate ([`punktfunk_core::reanchor::ReanchorGate`]).
// Armed on a frame-index gap or a dropped-count climb, it withholds the decoder's concealed output
// (released WITHOUT rendering — the SurfaceView keeps the last rendered frame on glass) until a
// proven clean re-anchor lifts it: an IDR (wire FLAG_SOF), an RFI anchor, or the 2nd recovery mark.
// `last_kf_req` throttles the keyframe intents it emits; `recovery_flags` carries each AU's
// user_flags from feed to present (keyed by the codec-echoed pts) so `on_decoded` reads the
// re-anchor signalling the platform decoder doesn't expose.
let mut gate = ReanchorGate::new(client.frames_dropped());
let mut recovery_flags: VecDeque<(u64, u32)> = VecDeque::new();
let mut last_kf_req: Option<Instant> = None;
// Skew-corrected latency stats (spec: design/stats-unification.md) use the negotiated
// host-minus-client clock offset (0 if the host didn't answer the skew handshake — then the
// HUD flags it "(same-host clock)").
let clock_offset = client.clock_offset_shared();
// Display stage (spec `display` + the capture→displayed headline): frames released with
// render = true are parked in the tracker; the OnFrameRendered callback pairs them with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
let tracker = DisplayTracker::new(stats.clone(), clock_offset.clone());
let render_cb = install_render_callback(&codec, &tracker);
// Receipt timestamps keyed by the pts we queue into the codec, so the decoded point (output-
// buffer dequeue — MediaCodec round-trips presentationTimeUs) can be paired back to its receipt
// for the `decode` stage. Fed while the HUD is visible OR the adaptive-bitrate controller wants
// the decode signal (`measure_decode`) — the decoder-backlog bottleneck the network can't see.
let measure_decode = client.wants_decode_latency();
let mut in_flight: VecDeque<(u64, i128)> = VecDeque::new();
// Phase-2 host/network split (design/stats-unification.md): received AUs awaiting their 0xCF
// host timing, as (pts_ns, capture→received µs). The timings are drained non-blockingly right
// where receipts are recorded and matched by pts; `network = hostnet host` (saturating).
// Only fed while the HUD is visible; an old host never sends a 0xCF, so entries just age out.
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
// The dataspace we've signalled on the Surface so far (None = default/SDR). Set reactively once
// the decoder reports an HDR stream (see `drain`); avoids re-applying every format event.
let mut applied_ds: Option<DataSpace> = None;
// One thread feeds AND drains: the NDK AMediaCodec wrapper isn't documented thread-safe for
// cross-thread feed/drain, so instead of splitting threads the loop decouples the two — input
// dequeue is non-blocking (never stalls presentation of already-decoded frames) and the only
// blocking wait is a short output dequeue while input is backed up (decoder progress is exactly
// what frees the next input buffer).
while !shutdown.load(Ordering::Relaxed) {
if pending.is_none() {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
// Loss recovery (RFI): feed the frame index so a forward gap fires a throttled
// reference-frame-invalidation request — an RFI-capable host (AMD LTR / NVENC)
// recovers with a cheap clean P-frame instead of a full IDR. The same forward gap
// arms the freeze gate so the decoder's concealment is held off the screen until the
// recovery re-anchors. The frames_dropped keyframe path below stays the backstop.
if client.note_frame_index(frame.frame_index) {
gate.arm(Instant::now());
}
// Park this AU's re-anchor flags for the present side (keyed by the pts the codec
// echoes on the output buffer) — unconditional, unlike the HUD's `in_flight` map.
recovery_flags.push_back((frame.pts_ns / 1000, frame.flags));
if recovery_flags.len() > IN_FLIGHT_CAP {
recovery_flags.pop_front();
}
if fed == 0 {
let p = &frame.data;
log::info!(
"decode: first AU {} bytes, head {:02x?}",
p.len(),
&p[..p.len().min(6)]
);
}
// Receipt stamp for the `decode` stage pairing, parked in `in_flight` (keyed by
// the pts the codec echoes on its output buffer) whenever it's needed: the HUD
// being visible, or the ABR decode signal (`measure_decode`). The HUD-only
// samplers (`received` point, host/network split) stay gated on the overlay so
// the hidden steady state adds only a wall-clock read + the receipt push.
if stats.enabled() || measure_decode {
let received_ns = now_realtime_ns();
in_flight.push_back((frame.pts_ns / 1000, received_ns));
if in_flight.len() > IN_FLIGHT_CAP {
in_flight.pop_front(); // stale — codec never echoed it back
}
// HUD stat, `received` point: host+network = client_now + (hostclient)
// capture_pts.
if stats.enabled() {
let clock_offset = clock_offset.load(Ordering::Relaxed);
let lat_ns = received_ns + clock_offset as i128 - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
// Phase-2 split: park this AU's capture→received sample, then match any
// 0xCF host timings that have arrived — host = the host's own
// capture→sent, network = our capture→received minus it (per-frame
// tiling; saturating in case of clock jitter).
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front(); // 0xCF lost / old host — evict
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) =
pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
}
pending = Some(frame);
}
Err(PunktfunkError::NoFrame) => {} // timeout — still drain output below
Err(_) => break, // session closed
}
}
// Time the productive work (feed + drain) only — the `next_frame` poll wait above is idle
// and excluded, so ADPF sees this thread's real per-frame CPU cost, not the poll timeout.
let work_t0 = Instant::now();
if let Some(frame) = pending.take() {
if feed(
&codec,
&client,
&frame.data,
frame.pts_ns / 1000,
&mut oversized_dropped,
) {
fed += 1;
if fed % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
} else {
// No input buffer free — transient back-pressure. Keep the AU and let `drain` block
// briefly below; a released output buffer is what recycles an input slot.
pending = Some(frame);
}
}
// Drain every iteration. When input is blocked, wait ~2 ms on output so the loop rides
// decoder progress instead of busy-spinning against a full input queue.
let wait = if pending.is_some() {
Duration::from_millis(2)
} else {
Duration::ZERO
};
let (r, d) = drain(
&codec,
&client,
measure_decode,
&window,
&mut applied_ds,
wait,
&stats,
&mut in_flight,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut gate,
&mut recovery_flags,
);
rendered += r;
discarded += d;
// ADPF: attribute this iteration's feed+drain time to the frame being produced, and report
// the accumulated per-frame work once one is actually presented (r > 0). Under back-pressure
// the short output-dequeue wait is included in the tally — for a latency-first client,
// biasing the governor toward "boost" is the desired behaviour. Cheap when `hint` is None
// (one `Instant` diff, no report).
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if r > 0 {
if !hint_tried {
// First presented frame: the pump + audio threads have registered their ids by now.
// Build one ADPF session over the whole pipeline's thread set (empty below API 33,
// or where the platform declines → `None`, and the loop runs unhinted).
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
}
// Loss recovery + overdue backstop, folded through the gate. Under infinite GOP the only
// recovery keyframe is one we request; the reassembler drops unrecoverable AUs (frames_dropped)
// and the decoder then conceals the reference-missing deltas and renders them without error, so
// a decode-error trigger rarely fires — the gate arms the freeze on the drop-count climb
// instead. An overdue freeze (held REANCHOR_FREEZE_MAX with no clean re-anchor) re-asks while it
// keeps holding: never resume to gray — a dead stream is the QUIC idle-timeout watchdog's job.
let now = Instant::now();
if gate.poll(client.frames_dropped(), now)
&& last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = client.request_keyframe();
log::debug!("decode: requested keyframe (loss recovery / overdue re-anchor)");
}
}
let _ = codec.stop();
drop(codec); // AMediaCodec_delete — after this no render callback can fire
if let Some(ud) = render_cb {
// SAFETY: the codec was dropped above; this registration's single reclaim.
unsafe { release_render_callback(ud) };
}
log::info!("decode: stopped (fed={fed} rendered={rendered} discarded={discarded})");
}
/// Try to copy one access unit into a codec input buffer and queue it, without blocking. Returns
/// `false` only on `TryAgainLater` (no input buffer free) — the caller keeps the AU pending and
/// retries; a hard dequeue/queue error counts as consumed (retrying can't salvage the AU, and
/// parking it forever would wedge the loop on a broken codec). An AU larger than the input
/// buffer is DROPPED (+ a recovery keyframe requested), never truncated — a truncated AU is
/// corrupt input the decoder chews on silently, poisoning the reference chain.
fn feed(
codec: &MediaCodec,
client: &NativeClient,
au: &[u8],
pts_us: u64,
oversized_dropped: &mut u64,
) -> bool {
match codec.dequeue_input_buffer(Duration::ZERO) {
Ok(DequeuedInputBufferResult::Buffer(mut buf)) => {
let n = {
let dst = buf.buffer_mut();
if au.len() > dst.len() {
*oversized_dropped += 1;
log::warn!(
"decode: AU {} > input buffer {} — dropped ({} so far), requesting keyframe",
au.len(),
dst.len(),
*oversized_dropped
);
let _ = client.request_keyframe();
0 // return the slot with zero valid bytes — a no-op input, not corrupt data
} else {
let n = au.len();
// SAFETY: `au` and `dst` are distinct allocations (wire AU vs. codec buffer),
// both valid for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so
// the cast write initializes exactly `dst[..n]`.
unsafe {
std::ptr::copy_nonoverlapping(
au.as_ptr(),
dst.as_mut_ptr().cast::<u8>(),
n,
);
}
n
}
};
if let Err(e) = codec.queue_input_buffer(buf, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer: {e}");
}
true
}
Ok(DequeuedInputBufferResult::TryAgainLater) => false, // caller keeps the AU pending
Err(e) => {
log::warn!("decode: dequeue_input_buffer: {e}");
true
}
}
}
/// Dequeue every ready output buffer and present only the NEWEST (render = true), discarding the
/// rest (render = false) — when decode falls behind, a back-to-back burst of stale frames on glass
/// is worse than skipping straight to the freshest one (the Apple client's 1-slot newest-ready
/// ring, ported). `first_wait` is the timeout for the first dequeue only: zero normally, ~2 ms when
/// the caller's input is blocked so the loop waits on decoder progress instead of busy-spinning.
/// Returns `(rendered, discarded)`. Also reacts to `OutputFormatChanged` (which can interleave
/// between buffers — handled without losing the held buffer) to signal HDR on the Surface.
///
/// Each dequeued buffer is also the HUD's `decoded` measurement point (rendered or not — the frame
/// finished decoding either way): end-to-end = decoded + clock_offset capture pts, and the
/// `decode` stage pairs the buffer's echoed presentationTimeUs back to the receipt stamp in
/// `in_flight` (single-clock local difference, no skew involved). The presented frame's
/// `(pts, decoded stamp)` is additionally parked in `tracker` for the OnFrameRendered callback —
/// the `display` stage's other endpoint.
#[allow(clippy::too_many_arguments)] // one call site; mirrors the async loop's present_ready
fn drain(
codec: &MediaCodec,
client: &NativeClient,
measure_decode: bool,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
first_wait: Duration,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
tracker: &DisplayTracker,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) -> (u64, u64) {
// Newest ready buffer so far (presented after the loop) with its HUD metadata —
// `Some((pts_us, decoded_ns))` only while the HUD is visible. `held_present` is the freeze gate's
// verdict for that newest buffer (`false` = a post-loss concealment to withhold).
let mut held: Option<(OutputBuffer<'_>, Option<(u64, i128)>)> = None;
let mut held_present = true;
let mut discarded: u64 = 0;
let mut wait = first_wait;
loop {
match codec.dequeue_output_buffer(wait) {
Ok(DequeuedOutputBufferInfoResult::Buffer(buf)) => {
// Only the first dequeue may block; later ones poll (wait == ZERO).
wait = Duration::ZERO;
// Fold every dequeued frame through the gate in pts (== decode) order — even the ones
// the newest-wins policy discards — so the two-mark re-anchor count stays correct; the
// verdict of the newest (last folded) buffer decides whether it reaches glass.
let pts_us = buf.info().presentation_time_us().max(0) as u64;
let flags = take_flags(recovery_flags, pts_us);
held_present =
gate.on_decoded(flags, false, Instant::now()) == GateVerdict::Present;
let meta = if stats.enabled() || measure_decode {
// The dequeue IS the sync loop's decoded-availability instant.
let decoded_ns = now_realtime_ns();
note_decoded_pts(
client,
measure_decode,
stats,
in_flight,
clock_offset,
pts_us,
decoded_ns,
);
// The tracker's `display` stage is a HUD concern — park only when visible.
stats.enabled().then_some((pts_us, decoded_ns))
} else {
None
};
if let Some((stale, _)) = held.replace((buf, meta)) {
// A newer frame is ready — drop the held one without rendering.
if let Err(e) = codec.release_output_buffer(stale, false) {
log::warn!("decode: release_output_buffer(discard): {e}");
}
discarded += 1;
stats.note_skipped(1); // HUD `skipped` counter; no-op while hidden
}
}
Ok(DequeuedOutputBufferInfoResult::OutputFormatChanged) => {
// The decoder has parsed the SPS and now reports the stream's real colour signalling
// (the AMediaCodec analogue of VideoToolbox's format description on the Apple client).
// If it's HDR (BT.2020 PQ/HLG), tell the Surface so the compositor/display switch to
// HDR; SDR streams leave the default dataspace alone. The decoder itself picks a
// Main10 path from the SPS — no profile override needed. Keep looping (buffers
// follow, and any held buffer stays held across this event).
wait = Duration::ZERO;
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => log::warn!(
"decode: set_buffers_data_space({ds}) failed (non-fatal): {e}"
),
}
}
}
}
// TryAgainLater / OutputBuffersChanged — nothing more to dequeue now.
Ok(_) => break,
Err(e) => {
log::warn!("decode: dequeue_output_buffer: {e}");
break;
}
}
}
// Present the newest ready frame — UNLESS the gate is withholding it as a post-loss concealment,
// in which case release it without rendering (the SurfaceView keeps the last rendered frame frozen
// on glass) and count it as a discard rather than a display.
let mut rendered = 0;
if let Some((buf, meta)) = held {
match codec.release_output_buffer(buf, held_present) {
Ok(()) if held_present => {
rendered = 1;
if let Some((pts_us, decoded_ns)) = meta {
tracker.note_rendered(pts_us, decoded_ns);
}
}
Ok(()) => discarded += 1, // held off the screen — awaiting a clean re-anchor
Err(e) => log::warn!("decode: release_output_buffer: {e}"),
}
}
(rendered, discarded)
}
+40 -65
View File
@@ -22,21 +22,14 @@ const PULL_TIMEOUT: Duration = Duration::from_millis(100);
const TAG_LED: u8 = 0x01;
const TAG_PLAYER_LEDS: u8 = 0x02;
const TAG_TRIGGER: u8 = 0x03;
const TAG_HID_RAW: u8 = 0x05;
/// `NativeBridge.nativeNextRumble(handle): Long` — block up to ~100 ms for the next EFFECTIVE
/// rumble command from the core's shared policy engine (`design/rumble-root-fix.md` §D). The
/// engine owns ALL rumble policy — v2 lease expiry, legacy-host staleness (a uniform 1 s, ending
/// the old 60 s Android exposure), connection-close drain zeros — so Kotlin applies commands
/// verbatim: `(0, 0)` = cancel now, non-zero = one-shot at this level.
///
/// Returns a packed positive long: bits 49..52 = wire `pad` index (0..15), bits 32..47 = the
/// command's `backstop_ms` (≤ 5000 — the one-shot duration, i.e. the hardware net under a stalled
/// poll thread; the engine emits explicit zeros at every policy stop, so it is never the stop
/// mechanism), bits 16..31 = `low`, bits 0..15 = `high` (0..=0xFFFF). `-1` on timeout / session
/// closed (all packed values are positive, so `-1` stays unambiguous). Kotlin routes the command
/// back to the controller holding that wire `pad` index (multi-pad rumble). Run from a Kotlin
/// poll thread.
/// `NativeBridge.nativeNextRumble(handle): Long` — block up to ~100 ms for the next rumble update.
/// Returns a packed positive long: bit 48 = "has a v2 lease", bits 32..47 = `ttl_ms`, bits 16..31 =
/// `low`, bits 0..15 = `high` (`low`/`high` 0..=0xFFFF, `0/0` = stop). The lease flag is
/// out-of-band so ANY 16-bit `ttl_ms` — including 0xFFFF — is unambiguous (no in-band sentinel to
/// collide with a real 65535 ms lease). No lease (legacy host) → bit 48 clear, and Kotlin falls
/// back to its long one-shot. `-1` on timeout / session closed (all packed values are positive, so
/// `-1` stays unambiguous). Pad index is dropped (single-pad model). Run from a Kotlin poll thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextRumble(
_env: JNIEnv,
@@ -48,17 +41,19 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextRumble(
if handle == 0 {
return -1;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; next_rumble_command is
// &self on the Sync connector — safe alongside the decode/audio/input threads. Kotlin
// stops these poll threads (and joins them — unbounded) before nativeClose frees the
// handle.
// SAFETY: live handle per the nativeConnect/nativeClose contract; next_rumble_ttl is &self on
// the Sync connector — safe alongside the decode/audio/input threads. Kotlin stops these poll
// threads (and joins them — unbounded) before nativeClose frees the handle.
let h = unsafe { &*(handle as *const SessionHandle) };
match h.client.next_rumble_command(PULL_TIMEOUT) {
Ok(cmd) => {
(jlong::from(cmd.pad & 0xF) << 49)
| (jlong::from(cmd.backstop_ms.min(0xFFFF) as u16) << 32)
| (jlong::from(cmd.low) << 16)
| jlong::from(cmd.high)
match h.client.next_rumble_ttl(PULL_TIMEOUT) {
Ok((_pad, low, high, ttl)) => {
// The reorder gate already ran in the core, so this update is fresh. Encode the
// Option out-of-band: a real lease sets bit 48 and carries ttl_ms verbatim.
let (lease_flag, ttl_bits) = match ttl {
Some(ms) => (1i64 << 48, jlong::from(ms) << 32),
None => (0, 0),
};
lease_flag | ttl_bits | (jlong::from(low) << 16) | jlong::from(high)
}
Err(_) => -1, // NoFrame (timeout) or Closed — Kotlin loops on its running flag
}
@@ -66,12 +61,10 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextRumble(
}
/// `NativeBridge.nativeNextHidout(handle, buf): Int` — block up to ~100 ms for the next DualSense
/// HID-output event, written into the caller's direct ByteBuffer as `[pad][kind][fields…]` (the
/// leading `pad` is the wire pad index the event is addressed to, so Kotlin routes it to that
/// controller — multi-pad HID feedback):
/// Led → `[pad][0x01][r][g][b]` (len 5)
/// PlayerLeds → `[pad][0x02][bits]` (len 3)
/// Trigger → `[pad][0x03][which][effect…]` (len 3 + effect.len())
/// HID-output event, written into the caller's direct ByteBuffer as `[kind][fields…]`:
/// Led → `[0x01][r][g][b]` (len 4)
/// PlayerLeds → `[0x02][bits]` (len 2)
/// Trigger → `[0x03][which][effect…]` (len 2 + effect.len())
/// Returns the byte count written, or `-1` on timeout / session closed / buffer too small.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextHidout(
@@ -104,37 +97,33 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextHidout(
// SAFETY: `ptr`/`cap` describe the direct ByteBuffer's backing store, valid for this call.
let out = unsafe { std::slice::from_raw_parts_mut(ptr, cap) };
// out[0] = wire pad index; out[1] = kind tag; the rest is the per-kind payload.
let n = match ev {
HidOutput::Led { pad, r, g, b } => {
if cap < 5 {
HidOutput::Led { r, g, b, .. } => {
if cap < 4 {
return -1;
}
out[0] = pad;
out[1] = TAG_LED;
out[2] = r;
out[3] = g;
out[4] = b;
5
out[0] = TAG_LED;
out[1] = r;
out[2] = g;
out[3] = b;
4
}
HidOutput::PlayerLeds { pad, bits } => {
if cap < 3 {
HidOutput::PlayerLeds { bits, .. } => {
if cap < 2 {
return -1;
}
out[0] = pad;
out[1] = TAG_PLAYER_LEDS;
out[2] = bits;
3
out[0] = TAG_PLAYER_LEDS;
out[1] = bits;
2
}
HidOutput::Trigger { pad, which, effect } => {
let n = 3 + effect.len();
HidOutput::Trigger { which, effect, .. } => {
let n = 2 + effect.len();
if cap < n {
return -1; // the raw DS5 trigger block is ~11 bytes; Kotlin allocates 64
}
out[0] = pad;
out[1] = TAG_TRIGGER;
out[2] = which;
out[3..n].copy_from_slice(&effect);
out[0] = TAG_TRIGGER;
out[1] = which;
out[2..n].copy_from_slice(&effect);
n
}
HidOutput::TrackpadHaptic { .. } => {
@@ -142,20 +131,6 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextHidout(
// rumble already rides the universal 0xCA plane).
return -1;
}
HidOutput::HidRaw { pad, kind, data } => {
// As-is SC2 passthrough: the host's hidraw consumer (Steam) wrote this report to
// the virtual pad; Kotlin replays it verbatim on the physical controller.
// `[pad][0x05][kind][report…]` — kind 0 = output report, 1 = feature report.
let n = 3 + data.len();
if cap < n {
return -1; // reports are ≤ 64 bytes; Kotlin allocates 128
}
out[0] = pad;
out[1] = TAG_HID_RAW;
out[2] = kind;
out[3..n].copy_from_slice(&data);
n
}
};
n as jint
})
@@ -11,43 +11,6 @@ use std::time::Duration;
use super::{hex32, jni_guard, parse_hex32, SessionHandle};
/// Machine token of the most recent `nativeConnect`/`nativePair` failure, taken (and cleared)
/// by `nativeTakeLastError` so Kotlin can render a cause-specific message instead of the old
/// catch-all "wrong PIN, or the host isn't armed" (which blamed the PIN for dead network paths
/// — the moko0878-class support threads). The app runs one attempt at a time, so one slot
/// suffices; a stale token is harmless (it is taken immediately after the failed call).
static LAST_ERROR: Mutex<String> = Mutex::new(String::new());
/// Stable token for a failed pair/connect cause, matched by Kotlin (`ConnectErrors.kt`):
/// a typed host rejection yields its `RejectReason::as_str()` token ("not-armed", "denied",
/// "approval-timeout", …); transport-level causes map to "crypto" / "timeout" / "io" / "error".
fn note_error(e: &punktfunk_core::error::PunktfunkError) {
use punktfunk_core::error::PunktfunkError as E;
let token = match e {
E::Rejected(r) => r.as_str(),
E::Crypto => "crypto",
E::Timeout => "timeout",
E::Io(_) => "io",
_ => "error",
};
*LAST_ERROR.lock().unwrap() = token.to_string();
}
/// `NativeBridge.nativeTakeLastError(): String` — the machine token of the most recent failed
/// `nativeConnect`/`nativePair`, cleared on read (`""` when none). Call right after a `0`
/// handle / `""` fingerprint.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTakeLastError<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
) -> jni::sys::jstring {
let token = std::mem::take(&mut *LAST_ERROR.lock().unwrap());
match env.new_string(token) {
Ok(s) => s.into_raw(),
Err(_) => JObject::null().into_raw(),
}
}
/// `NativeBridge.nativeGenerateIdentity(): String` — mint a fresh persistent self-signed identity.
/// Returns `"<certPem>\n-----PUNKTFUNK-KEY-----\n<keyPem>"`, or `""` on failure (logged). Kotlin
/// persists it (Keystore-wrapped) and only calls this again when the store is genuinely empty.
@@ -222,7 +185,6 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
}
Err(e) => {
log::error!("nativeConnect to {host}:{port} failed: {e}");
note_error(&e);
0
}
}
@@ -356,9 +318,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativePair<'local
Ok(host_fp) => hex32(&host_fp),
Err(e) => {
// Crypto error == wrong PIN / MITM; anything else == transport/host reject.
// The token lets Kotlin say WHICH (`nativeTakeLastError`).
log::error!("nativePair to {host}:{port} failed: {e}");
note_error(&e);
String::new()
}
}
+14 -106
View File
@@ -6,11 +6,10 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject};
use jni::objects::JObject;
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX};
use super::SessionHandle;
@@ -146,19 +145,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
// stable lowest-free index per Android device and threads it here. Buttons carry the gamepad::BTN_*
// bit in `code` and pressed/released in `x` (1/0); axes carry the gamepad::AXIS_* id in `code` and
// the value in `x` (sticks i16 32768..32767, +y = up; triggers 0..255). The host accumulates the
// incremental events per pad into a matching virtual device. The core input task folds these into
// the seq'd GamepadState snapshots (keyed on this same `flags` index) and owns the per-pad seq — so
// the only thing this layer must get right is the index. Wire contract: input.rs::gamepad. A single
// controller lands on index 0, so its wire is byte-identical to the old single-pad path.
// Single-pad model: exactly one controller, forwarded as pad 0 (flags = 0). Buttons carry the
// gamepad::BTN_* bit in `code` and pressed/released in `x` (1/0); axes carry the gamepad::AXIS_* id
// in `code` and the value in `x` (sticks i16 32768..32767, +y = up; triggers 0..255). The host
// accumulates the incremental events into its virtual xpad. Wire contract: input.rs::gamepad.
/// `NativeBridge.nativeSendGamepadButton(handle, bit, down, pad)` — one gamepad button transition on
/// wire pad index `pad`. `bit`: a `gamepad::BTN_*` bit (e.g. BTN_A = 0x1000). `down`: 1=press,
/// 0=release. `pad`: wire pad index 0..15 (rides `flags`).
/// `NativeBridge.nativeSendGamepadButton(handle, bit, down)` — one gamepad button transition.
/// `bit`: a `gamepad::BTN_*` bit (e.g. BTN_A = 0x1000). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadButton(
_env: JNIEnv,
@@ -166,21 +159,21 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepad
handle: jlong,
bit: jint,
down: jboolean,
pad: jint,
) {
// flags = 0: pad index 0 — single-pad model.
send_event(
handle,
InputKind::GamepadButton,
bit as u32,
i32::from(down != 0),
0,
pad as u32,
0,
);
}
/// `NativeBridge.nativeSendGamepadAxis(handle, axisId, value, pad)` — one gamepad axis update on wire
/// pad index `pad`. `axisId`: a `gamepad::AXIS_*` id (LS_X=0..RT=5). `value`: stick i16
/// (32768..32767, +y=up) or trigger 0..255. `pad`: wire pad index 0..15 (rides `flags`).
/// `NativeBridge.nativeSendGamepadAxis(handle, axisId, value)` — one gamepad axis update.
/// `axisId`: a `gamepad::AXIS_*` id (LS_X=0..RT=5). `value`: stick i16 (32768..32767, +y=up) or
/// trigger 0..255.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadAxis(
_env: JNIEnv,
@@ -188,92 +181,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepad
handle: jlong,
axis_id: jint,
value: jint,
pad: jint,
) {
send_event(
handle,
InputKind::GamepadAxis,
axis_id as u32,
value,
0,
pad as u32,
);
}
/// `NativeBridge.nativeSendGamepadArrival(handle, pref, pad)` — declare the controller KIND presented
/// on wire pad index `pad` so the host builds a matching virtual device (mixed types — pad 0 a
/// DualSense, pad 1 an Xbox pad). `pref`: the `GamepadPref` wire byte (rides `code`). `pad`: wire pad
/// index 0..15 (rides `flags`). Sent ONCE when a pad opens, BEFORE any of its input; the core re-sends
/// it a few times against datagram loss, and an older host ignores the unknown tag (that pad then uses
/// the session-default kind from the handshake — the pre-existing single-pad behaviour on pad 0).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadArrival(
_env: JNIEnv,
_this: JObject,
handle: jlong,
pref: jint,
pad: jint,
) {
send_event(
handle,
InputKind::GamepadArrival,
pref as u32,
0,
0,
pad as u32,
);
}
/// `NativeBridge.nativeSendGamepadRemove(handle, pad)` — signal that wire pad index `pad` was
/// unplugged so the host tears its virtual device down. `pad` (rides `flags`) is the only field; the
/// core stamps the per-pad seq (in the snapshot seq space, so a reordered snapshot can't resurrect the
/// pad) and arms a re-send burst against datagram loss. An older host ignores the unknown tag.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadRemove(
_env: JNIEnv,
_this: JObject,
handle: jlong,
pad: jint,
) {
send_event(handle, InputKind::GamepadRemove, 0, 0, 0, pad as u32);
}
/// `NativeBridge.nativeSendPadHidReport(handle, pad, buf, len)` — one raw HID input report from a
/// client-captured controller (the as-is Steam Controller 2 passthrough), forwarded verbatim on
/// the rich-input plane (`RichInput::HidReport`, 0xCC). `buf` is a DIRECT ByteBuffer whose first
/// `len` bytes are the report, id byte first (`0x42`/`0x45`/`0x47` state, `0x43` battery, …);
/// `len` is clamped to the 64-byte wire body. Called from the capture thread at the controller's
/// own report rate (~250500 Hz) — the direct-buffer read avoids a JNI array copy per report.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPadHidReport(
env: JNIEnv,
_this: JObject,
handle: jlong,
pad: jint,
buf: JByteBuffer,
len: jint,
) {
if handle == 0 || len <= 0 {
return;
}
let cap = match env.get_direct_buffer_capacity(&buf) {
Ok(c) => c,
Err(_) => return,
};
let ptr = match env.get_direct_buffer_address(&buf) {
Ok(p) if !p.is_null() => p,
_ => return,
};
let n = (len as usize).min(cap).min(HID_REPORT_MAX);
let mut data = [0u8; HID_REPORT_MAX];
// SAFETY: `ptr`/`cap` describe the direct ByteBuffer's backing store, valid for this call;
// `n` is bounded by both the buffer capacity and the fixed wire body.
data[..n].copy_from_slice(unsafe { std::slice::from_raw_parts(ptr, n) });
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_rich_input is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_rich_input(RichInput::HidReport {
pad: (pad as u32 & 0xF) as u8,
len: n as u8,
data,
});
// flags = 0: pad index 0 — single-pad model.
send_event(handle, InputKind::GamepadAxis, axis_id as u32, value, 0, 0);
}
@@ -102,31 +102,6 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime<'
})
}
/// `NativeBridge.nativeVideoCodecLabel(handle): String` — a short human label for the codec the
/// host resolved (`"H.264"` / `"HEVC"` / `"AV1"` / `"PyroWave"`), for the stats HUD's video-feed
/// line. Distinct from [`Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime`] because the MIME
/// collapses PyroWave onto `video/hevc` and can't name it. Empty string on a `0` handle. Cheap;
/// safe on the UI thread. Android-gated (reads `crate::decode`), matching `nativeVideoMime`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoCodecLabel<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jstring {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
match env.new_string(crate::decode::codec_label(h.client.codec)) {
Ok(s) => s.into_raw(),
Err(_) => std::ptr::null_mut(),
}
})
}
/// `NativeBridge.nativeVideoDecoderLabel(handle): String` — the resolved decoder identity for the
/// HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the decode thread has resolved one.
/// One-shot (the decoder is fixed for the session); poll once after the HUD appears. Not
+14 -39
View File
@@ -2,49 +2,24 @@
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>
<key>GCSupportedGameControllers</key>
<array>
<dict>
<key>ProfileName</key>
<string>ExtendedGamepad</string>
</dict>
<dict>
<key>ProfileName</key>
<string>MicroGamepad</string>
</dict>
</array>
<!-- Custom keys merged into the auto-generated Info.plist (GENERATE_INFOPLIST_FILE=YES
supplies the rest). NSBonjourServices is required for NWBrowser to browse this
service type on iOS/tvOS — without it the system blocks the browse and discovery
returns nothing. Kept OUT of the synchronized App/ + Sources/ groups so it isn't
auto-added as a bundle resource (which collides with Info.plist processing). -->
<key>NSBonjourServices</key>
<array>
<string>_punktfunk._udp</string>
</array>
<!-- Background keep-alive (opt-in, iOS/iPadOS): the ONLY sanctioned way to keep the long-lived
QUIC socket + pump-thread set alive while backgrounded is the audio background mode, backed
by the session's real, audible remote audio (AVAudioEngine keeps rendering). Video decode is
dropped; a bounded timer auto-disconnects. Never silence-as-keepalive (App Review 2.5.4).
tvOS ignores/tolerates the key; macOS is not gated by it. -->
<key>UIBackgroundModes</key>
<array>
<string>audio</string>
</array>
<!-- Live Activities (iOS/iPadOS): the Lock-Screen / Dynamic-Island session surface. Updated
locally (pushType nil) from the alive app process — no aps-environment. tvOS/macOS ignore it. -->
<key>NSSupportsLiveActivities</key>
<!-- Standard-algorithm crypto only (AES-GCM via the Rust core) — exempt from export
compliance, but the key must be declared or every TestFlight build stalls on the
compliance question. -->
<key>ITSAppUsesNonExemptEncryption</key>
<false/>
<!-- Allow CADisplayLink above 60 Hz on ProMotion iPhones: without this key the system
silently caps the link at 60 even when SessionPresenter asks for the stream's rate
via preferredFrameRateRange, so a 120 fps stream would present at half rate. -->
<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>
<!-- Deep links: punktfunk://connect/<host-uuid>[?launch=<GameEntry.id>]. Emitted by the
launcher widget and Siri/Shortcuts; routed by ContentView.onOpenURL into the existing
connect path. Shared across all three targets (tvOS/macOS accept it harmlessly). -->
<key>CFBundleURLTypes</key>
<array>
<dict>
<key>CFBundleURLName</key>
<string>io.unom.punktfunk.deeplink</string>
<key>CFBundleURLSchemes</key>
<array>
<string>punktfunk</string>
</array>
</dict>
</array>
</dict>
</plist>
@@ -73,15 +73,5 @@
<array>
<string>$(AppIdentifierPrefix)io.unom.punktfunk</string>
</array>
<!-- App Group: same shared UserDefaults suite as iOS (Config/Punktfunk.entitlements). Shared
here so a single HostStore code path (UserDefaults(suiteName:)) works on every platform;
macOS widgets that read it arrive with M5. macOS App Groups use the plain group id under
the App Store profile; a Developer-ID-signed build wants the team-prefixed form — the
Dev-ID codesign step in release.yml must verify this value against the Dev-ID profile. -->
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
@@ -20,14 +20,5 @@
is true on iOS/tvOS too. -->
<key>com.apple.developer.networking.multicast</key>
<true/>
<!-- App Group: the shared UserDefaults suite (group.io.unom.punktfunk) that both the app and
the Widget/Live-Activity extension read — the saved-host store moved there so a launcher
widget can see it (HostStore reads UserDefaults(suiteName:)). Must be registered on the
developer portal and enabled in the provisioning profile for BOTH app ids
(io.unom.punktfunk + io.unom.punktfunk.widgets). tvOS carries the key harmlessly. -->
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
+2 -19
View File
@@ -9,20 +9,13 @@ let package = Package(
platforms: [.macOS(.v14), .iOS(.v17), .tvOS(.v17)],
products: [
.library(name: "PunktfunkKit", targets: ["PunktfunkKit"]),
// Dependency-free foundation (stored-host model + JSON codec, settings keys, App-Group
// constant, deep-link grammar, Live Activity attributes). A separate PRODUCT so the widget
// extension which must never link PunktfunkKit (Rust staticlib + presentation layer)
// can link this and nothing else. PunktfunkKit re-exports it (see SharedReexport.swift).
.library(name: "PunktfunkShared", targets: ["PunktfunkShared"]),
.executable(name: "PunktfunkClient", targets: ["PunktfunkClient"]),
],
targets: [
.binaryTarget(name: "PunktfunkCore", path: "PunktfunkCore.xcframework"),
// No dependencies by design an extension process links this alone.
.target(name: "PunktfunkShared"),
.target(
name: "PunktfunkKit",
dependencies: ["PunktfunkCore", "PunktfunkShared"],
dependencies: ["PunktfunkCore"],
// OSS attribution shown by the app's Acknowledgements screen. Bundled here (not in the
// app target) so it rides along via Bundle.module in both `swift build` and the Xcode
// app, which links the PunktfunkKit product. Refresh with
@@ -47,16 +40,6 @@ let package = Package(
// its manifest breaks SwiftPM whole-graph validation on macOS, and only the
// Punktfunk-tvOS target links it; the #if os(tvOS) import never compiles here.)
.executableTarget(name: "PunktfunkClient", dependencies: ["PunktfunkKit"]),
// PunktfunkCore is a direct dep too so the wire tests can name the C ABI's
// `PunktfunkInputEvent` / `PUNKTFUNK_INPUT_KIND_*` when asserting the gamepad byte layout.
.testTarget(
name: "PunktfunkKitTests",
dependencies: ["PunktfunkKit", "PunktfunkShared", "PunktfunkCore"],
resources: [
// PyroWave golden fixtures: host-encoded AUs + upstream-decoded reference
// planes (regenerate with punktfunk-host's `pyrowave_dump_golden` on a
// Vulkan box see PyroWaveDecoderTests.swift).
.copy("PyroWaveFixtures")
]),
.testTarget(name: "PunktfunkKitTests", dependencies: ["PunktfunkKit"]),
]
)
@@ -11,56 +11,14 @@
BB0000000000000000000005 /* PunktfunkKit in Frameworks */ = {isa = PBXBuildFile; productRef = BB0000000000000000000006 /* PunktfunkKit */; };
CC0000000000000000000005 /* PunktfunkKit in Frameworks */ = {isa = PBXBuildFile; productRef = CC0000000000000000000006 /* PunktfunkKit */; };
DD0000000000000000000003 /* SwiftUINavigationTransitions in Frameworks */ = {isa = PBXBuildFile; productRef = DD0000000000000000000002 /* SwiftUINavigationTransitions */; };
E295569A300948B9009F939C /* WidgetKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = E2955699300948B9009F939C /* WidgetKit.framework */; };
E295569C300948B9009F939C /* SwiftUI.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = E295569B300948B9009F939C /* SwiftUI.framework */; };
E2CAFE000000000000000001 /* PunktfunkShared in Frameworks */ = {isa = PBXBuildFile; productRef = E2CAFE000000000000000002 /* PunktfunkShared */; };
E29556A9300948BA009F939C /* PunktfunkWidgetsExtension.appex in Embed Foundation Extensions */ = {isa = PBXBuildFile; fileRef = E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */; settings = {ATTRIBUTES = (RemoveHeadersOnCopy, ); }; };
/* End PBXBuildFile section */
/* Begin PBXContainerItemProxy section */
E29556A7300948BA009F939C /* PBXContainerItemProxy */ = {
isa = PBXContainerItemProxy;
containerPortal = AA000000000000000000000D /* Project object */;
proxyType = 1;
remoteGlobalIDString = E2955696300948B9009F939C;
remoteInfo = PunktfunkWidgetsExtension;
};
/* End PBXContainerItemProxy section */
/* Begin PBXCopyFilesBuildPhase section */
E29556AA300948BA009F939C /* Embed Foundation Extensions */ = {
isa = PBXCopyFilesBuildPhase;
buildActionMask = 2147483647;
dstPath = "";
dstSubfolderSpec = 13;
files = (
E29556A9300948BA009F939C /* PunktfunkWidgetsExtension.appex in Embed Foundation Extensions */,
);
name = "Embed Foundation Extensions";
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXCopyFilesBuildPhase section */
/* Begin PBXFileReference section */
AA0000000000000000000001 /* Punktfunk.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = Punktfunk.app; sourceTree = BUILT_PRODUCTS_DIR; };
BB0000000000000000000001 /* Punktfunk-iOS.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "Punktfunk-iOS.app"; sourceTree = BUILT_PRODUCTS_DIR; };
CC0000000000000000000001 /* Punktfunk-tvOS.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "Punktfunk-tvOS.app"; sourceTree = BUILT_PRODUCTS_DIR; };
E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */ = {isa = PBXFileReference; explicitFileType = "wrapper.app-extension"; includeInIndex = 0; path = PunktfunkWidgetsExtension.appex; sourceTree = BUILT_PRODUCTS_DIR; };
E2955699300948B9009F939C /* WidgetKit.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = WidgetKit.framework; path = System/Library/Frameworks/WidgetKit.framework; sourceTree = SDKROOT; };
E295569B300948B9009F939C /* SwiftUI.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = SwiftUI.framework; path = System/Library/Frameworks/SwiftUI.framework; sourceTree = SDKROOT; };
E295577B30094CE5009F939C /* PunktfunkWidgetsExtension.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.plist.entitlements; path = PunktfunkWidgetsExtension.entitlements; sourceTree = "<group>"; };
/* End PBXFileReference section */
/* Begin PBXFileSystemSynchronizedBuildFileExceptionSet section */
E29556AD300948BA009F939C /* Exceptions for "PunktfunkWidgets" folder in "PunktfunkWidgetsExtension" target */ = {
isa = PBXFileSystemSynchronizedBuildFileExceptionSet;
membershipExceptions = (
Info.plist,
);
target = E2955696300948B9009F939C /* PunktfunkWidgetsExtension */;
};
/* End PBXFileSystemSynchronizedBuildFileExceptionSet section */
/* Begin PBXFileSystemSynchronizedRootGroup section */
AA0000000000000000000002 /* App */ = {
isa = PBXFileSystemSynchronizedRootGroup;
@@ -72,14 +30,6 @@
path = Sources/PunktfunkClient;
sourceTree = "<group>";
};
E295569D300948B9009F939C /* PunktfunkWidgets */ = {
isa = PBXFileSystemSynchronizedRootGroup;
exceptions = (
E29556AD300948BA009F939C /* Exceptions for "PunktfunkWidgets" folder in "PunktfunkWidgetsExtension" target */,
);
path = PunktfunkWidgets;
sourceTree = "<group>";
};
/* End PBXFileSystemSynchronizedRootGroup section */
/* Begin PBXFrameworksBuildPhase section */
@@ -108,27 +58,14 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955694300948B9009F939C /* Frameworks */ = {
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
E2CAFE000000000000000001 /* PunktfunkShared in Frameworks */,
E295569C300948B9009F939C /* SwiftUI.framework in Frameworks */,
E295569A300948B9009F939C /* WidgetKit.framework in Frameworks */,
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXFrameworksBuildPhase section */
/* Begin PBXGroup section */
AA0000000000000000000007 = {
isa = PBXGroup;
children = (
E295577B30094CE5009F939C /* PunktfunkWidgetsExtension.entitlements */,
AA0000000000000000000002 /* App */,
AA0000000000000000000003 /* Sources/PunktfunkClient */,
E295569D300948B9009F939C /* PunktfunkWidgets */,
E2955698300948B9009F939C /* Frameworks */,
AA0000000000000000000008 /* Products */,
);
sourceTree = "<group>";
@@ -139,20 +76,10 @@
AA0000000000000000000001 /* Punktfunk.app */,
BB0000000000000000000001 /* Punktfunk-iOS.app */,
CC0000000000000000000001 /* Punktfunk-tvOS.app */,
E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */,
);
name = Products;
sourceTree = "<group>";
};
E2955698300948B9009F939C /* Frameworks */ = {
isa = PBXGroup;
children = (
E2955699300948B9009F939C /* WidgetKit.framework */,
E295569B300948B9009F939C /* SwiftUI.framework */,
);
name = Frameworks;
sourceTree = "<group>";
};
/* End PBXGroup section */
/* Begin PBXNativeTarget section */
@@ -187,12 +114,10 @@
BB000000000000000000000B /* Sources */,
BB0000000000000000000004 /* Frameworks */,
BB000000000000000000000C /* Resources */,
E29556AA300948BA009F939C /* Embed Foundation Extensions */,
);
buildRules = (
);
dependencies = (
E29556A8300948BA009F939C /* PBXTargetDependency */,
);
fileSystemSynchronizedGroups = (
AA0000000000000000000002 /* App */,
@@ -231,29 +156,6 @@
productReference = CC0000000000000000000001 /* Punktfunk-tvOS.app */;
productType = "com.apple.product-type.application";
};
E2955696300948B9009F939C /* PunktfunkWidgetsExtension */ = {
isa = PBXNativeTarget;
buildConfigurationList = E29556AE300948BA009F939C /* Build configuration list for PBXNativeTarget "PunktfunkWidgetsExtension" */;
buildPhases = (
E2955693300948B9009F939C /* Sources */,
E2955694300948B9009F939C /* Frameworks */,
E2955695300948B9009F939C /* Resources */,
);
buildRules = (
);
dependencies = (
);
fileSystemSynchronizedGroups = (
E295569D300948B9009F939C /* PunktfunkWidgets */,
);
name = PunktfunkWidgetsExtension;
packageProductDependencies = (
E2CAFE000000000000000002 /* PunktfunkShared */,
);
productName = PunktfunkWidgetsExtension;
productReference = E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */;
productType = "com.apple.product-type.app-extension";
};
/* End PBXNativeTarget section */
/* Begin PBXProject section */
@@ -261,15 +163,11 @@
isa = PBXProject;
attributes = {
BuildIndependentTargetsInParallel = 1;
LastSwiftUpdateCheck = 2700;
LastUpgradeCheck = 2700;
TargetAttributes = {
AA0000000000000000000009 = {
CreatedOnToolsVersion = 26.0;
};
E2955696300948B9009F939C = {
CreatedOnToolsVersion = 27.0;
};
};
};
buildConfigurationList = AA000000000000000000000E /* Build configuration list for PBXProject "Punktfunk" */;
@@ -292,7 +190,6 @@
AA0000000000000000000009 /* Punktfunk */,
BB0000000000000000000009 /* Punktfunk-iOS */,
CC0000000000000000000009 /* Punktfunk-tvOS */,
E2955696300948B9009F939C /* PunktfunkWidgetsExtension */,
);
};
/* End PBXProject section */
@@ -319,13 +216,6 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955695300948B9009F939C /* Resources */ = {
isa = PBXResourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXResourcesBuildPhase section */
/* Begin PBXSourcesBuildPhase section */
@@ -350,23 +240,8 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955693300948B9009F939C /* Sources */ = {
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXSourcesBuildPhase section */
/* Begin PBXTargetDependency section */
E29556A8300948BA009F939C /* PBXTargetDependency */ = {
isa = PBXTargetDependency;
target = E2955696300948B9009F939C /* PunktfunkWidgetsExtension */;
targetProxy = E29556A7300948BA009F939C /* PBXContainerItemProxy */;
};
/* End PBXTargetDependency section */
/* Begin XCBuildConfiguration section */
AA0000000000000000000010 /* Debug */ = {
isa = XCBuildConfiguration;
@@ -689,97 +564,6 @@
};
name = Release;
};
E29556AB300948BA009F939C /* Debug */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
ASSETCATALOG_COMPILER_WIDGET_BACKGROUND_COLOR_NAME = WidgetBackground;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CODE_SIGN_ENTITLEMENTS = PunktfunkWidgetsExtension.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = F4H37KF6WC;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = PunktfunkWidgets/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = PunktfunkWidgets;
INFOPLIST_KEY_NSHumanReadableCopyright = "";
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
"@executable_path/../../Frameworks",
);
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = io.unom.punktfunk.widgets;
PRODUCT_NAME = "$(TARGET_NAME)";
REGISTER_APP_GROUPS = YES;
SDKROOT = iphoneos;
SKIP_INSTALL = YES;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
name = Debug;
};
E29556AC300948BA009F939C /* Release */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
ASSETCATALOG_COMPILER_WIDGET_BACKGROUND_COLOR_NAME = WidgetBackground;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CODE_SIGN_ENTITLEMENTS = PunktfunkWidgetsExtension.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = F4H37KF6WC;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = PunktfunkWidgets/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = PunktfunkWidgets;
INFOPLIST_KEY_NSHumanReadableCopyright = "";
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
"@executable_path/../../Frameworks",
);
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = io.unom.punktfunk.widgets;
PRODUCT_NAME = "$(TARGET_NAME)";
REGISTER_APP_GROUPS = YES;
SDKROOT = iphoneos;
SKIP_INSTALL = YES;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
VALIDATE_PRODUCT = YES;
};
name = Release;
};
/* End XCBuildConfiguration section */
/* Begin XCConfigurationList section */
@@ -819,15 +603,6 @@
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
E29556AE300948BA009F939C /* Build configuration list for PBXNativeTarget "PunktfunkWidgetsExtension" */ = {
isa = XCConfigurationList;
buildConfigurations = (
E29556AB300948BA009F939C /* Debug */,
E29556AC300948BA009F939C /* Release */,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
/* End XCConfigurationList section */
/* Begin XCLocalSwiftPackageReference section */
@@ -861,10 +636,6 @@
isa = XCSwiftPackageProductDependency;
productName = PunktfunkKit;
};
E2CAFE000000000000000002 /* PunktfunkShared */ = {
isa = XCSwiftPackageProductDependency;
productName = PunktfunkShared;
};
DD0000000000000000000002 /* SwiftUINavigationTransitions */ = {
isa = XCSwiftPackageProductDependency;
package = DD0000000000000000000001 /* XCRemoteSwiftPackageReference "swiftui-navigation-transitions" */;
@@ -49,13 +49,6 @@
ReferencedContainer = "container:Punktfunk.xcodeproj">
</BuildableReference>
</BuildableProductRunnable>
<EnvironmentVariables>
<EnvironmentVariable
key = "PUNKTFUNK_BILINEAR_LUMA"
value = "1"
isEnabled = "YES">
</EnvironmentVariable>
</EnvironmentVariables>
</LaunchAction>
<ProfileAction
buildConfiguration = "Release"
@@ -1,11 +0,0 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -1,35 +0,0 @@
{
"images" : [
{
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "dark"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "tinted"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -1,6 +0,0 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -1,11 +0,0 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -1,186 +0,0 @@
// Home-Screen / Lock-Screen quick-launch widget (kind "PunktfunkHosts"). Reads the saved-host
// store from the shared App-Group suite, sorts most-recent-first, and deep-links each host into a
// session via `punktfunk://connect/<uuid>` the app's onOpenURL routes it through the normal
// connect path (trust policy / WoL / approval all apply).
//
// No reachability probing in v1 (a UDP check has no place in a timeline build; WoL handles offline
// hosts on tap). Timeline is a single `.never` entry the app pushes reloads on store changes
// (HostStore WidgetCenter.reloadTimelines).
import SwiftUI
import WidgetKit
import PunktfunkShared
// MARK: - Timeline
struct HostsEntry: TimelineEntry {
let date: Date
let hosts: [StoredHost]
}
struct HostsProvider: TimelineProvider {
func placeholder(in context: Context) -> HostsEntry {
HostsEntry(date: .now, hosts: [])
}
func getSnapshot(in context: Context, completion: @escaping (HostsEntry) -> Void) {
completion(HostsEntry(date: .now, hosts: Self.loadHosts()))
}
func getTimeline(in context: Context, completion: @escaping (Timeline<HostsEntry>) -> Void) {
// Single entry, never auto-refresh: the app reloads this timeline whenever the store
// changes (a new host, a fresh connect reordering by recency).
let entry = HostsEntry(date: .now, hosts: Self.loadHosts())
completion(Timeline(entries: [entry], policy: .never))
}
/// Decode the shared-suite host JSON (same wire format the app writes), most-recent first.
static func loadHosts() -> [StoredHost] {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return [] }
return hosts.sorted {
($0.lastConnected ?? .distantPast) > ($1.lastConnected ?? .distantPast)
}
}
}
// MARK: - Widget
struct HostsWidget: Widget {
var body: some WidgetConfiguration {
StaticConfiguration(kind: "PunktfunkHosts", provider: HostsProvider()) { entry in
HostsWidgetView(entry: entry)
.containerBackground(.fill.tertiary, for: .widget)
}
.configurationDisplayName("Punktfunk Hosts")
.description("Quick-launch your recent streaming hosts.")
.supportedFamilies([
.systemSmall, .systemMedium, .accessoryCircular, .accessoryRectangular,
])
}
}
// MARK: - Views
struct HostsWidgetView: View {
@Environment(\.widgetFamily) private var family
let entry: HostsEntry
var body: some View {
switch family {
case .systemMedium:
MediumHostsView(hosts: entry.hosts)
case .accessoryCircular:
CircularHostView(host: entry.hosts.first)
case .accessoryRectangular:
RectangularHostView(host: entry.hosts.first)
default: // systemSmall + fallback
SmallHostView(host: entry.hosts.first)
}
}
}
/// Deep link that connects to a stored host.
private func connectURL(_ host: StoredHost) -> URL {
DeepLink.connect(host: host.id, launchID: nil).url
}
private struct SmallHostView: View {
let host: StoredHost?
var body: some View {
if let host {
VStack(alignment: .leading, spacing: 6) {
Image(systemName: "play.tv.fill")
.font(.title2)
.foregroundStyle(.tint)
Spacer(minLength: 0)
Text(host.displayName)
.font(.headline)
.lineLimit(2)
if let last = host.lastConnected {
Text(last, format: .relative(presentation: .named))
.font(.caption2)
.foregroundStyle(.secondary)
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
.widgetURL(connectURL(host))
} else {
EmptyHostView()
}
}
}
private struct MediumHostsView: View {
let hosts: [StoredHost]
var body: some View {
if hosts.isEmpty {
EmptyHostView()
} else {
VStack(alignment: .leading, spacing: 8) {
Text("Punktfunk")
.font(.caption).bold()
.foregroundStyle(.tint)
ForEach(hosts.prefix(4)) { host in
Link(destination: connectURL(host)) {
HStack {
Image(systemName: "play.tv.fill")
.foregroundStyle(.tint)
Text(host.displayName)
.font(.subheadline)
.lineLimit(1)
Spacer()
if let last = host.lastConnected {
Text(last, format: .relative(presentation: .named))
.font(.caption2)
.foregroundStyle(.secondary)
}
}
}
}
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
}
}
}
private struct CircularHostView: View {
let host: StoredHost?
var body: some View {
ZStack {
AccessoryWidgetBackground()
Image(systemName: "play.tv.fill")
}
.widgetURL(host.map(connectURL))
}
}
private struct RectangularHostView: View {
let host: StoredHost?
var body: some View {
HStack {
Image(systemName: "play.tv.fill")
Text(host?.displayName ?? "Punktfunk")
.lineLimit(1)
}
.widgetURL(host.map(connectURL))
}
}
private struct EmptyHostView: View {
var body: some View {
VStack(spacing: 6) {
Image(systemName: "play.tv")
.font(.title2)
.foregroundStyle(.secondary)
Text("Open Punktfunk to add a host.")
.font(.caption)
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
}
-11
View File
@@ -1,11 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSExtension</key>
<dict>
<key>NSExtensionPointIdentifier</key>
<string>com.apple.widgetkit-extension</string>
</dict>
</dict>
</plist>
@@ -1,20 +0,0 @@
// The widget extension's entry point. ONE extension target (bundle id io.unom.punktfunk.widgets,
// iOS only) hosts both the launcher widgets and the Live Activity UI. It links PunktfunkShared and
// NOTHING else never PunktfunkKit (Rust staticlib + presentation layer would blow the widget
// process's ~30 MB budget).
//
// These files are NOT part of the SwiftPM package (Package.swift doesn't declare a PunktfunkWidgets
// target, so `swift build` ignores the directory). They compile only in the Xcode widget-extension
// target you add pointing at this folder see design/apple-live-activities-and-widgets.md §M1 and
// the GUI checklist.
import SwiftUI
import WidgetKit
@main
struct PunktfunkWidgetBundle: WidgetBundle {
var body: some Widget {
HostsWidget()
PunktfunkSessionLiveActivity()
}
}
@@ -1,140 +0,0 @@
// The Live Activity UI (Lock Screen banner + Dynamic Island) for a running session. The app owns
// the Activity's lifecycle (SessionActivityController); this is only its presentation, rendered in
// the widget-extension process from the shared `PunktfunkSessionAttributes`.
//
// The End button runs `EndStreamIntent` (a LiveActivityIntent) IN THE APP's process, which posts
// .punktfunkEndActiveSession the app disconnects. Elapsed time ticks client-side via
// Text(timerInterval:) no per-second push.
import ActivityKit
import AppIntents
import SwiftUI
import WidgetKit
import PunktfunkShared
struct PunktfunkSessionLiveActivity: Widget {
var body: some WidgetConfiguration {
ActivityConfiguration(for: PunktfunkSessionAttributes.self) { context in
LockScreenView(context: context)
.activitySystemActionForegroundColor(.white)
} dynamicIsland: { context in
DynamicIsland {
DynamicIslandExpandedRegion(.leading) {
Label {
Text(context.attributes.hostName).font(.caption).lineLimit(1)
} icon: {
Image(systemName: "play.tv.fill")
}
.foregroundStyle(.tint)
}
DynamicIslandExpandedRegion(.trailing) {
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.font(.caption).monospacedDigit()
.frame(maxWidth: 56)
.foregroundStyle(.secondary)
}
DynamicIslandExpandedRegion(.center) {
if let title = context.attributes.launchTitle {
Text(title).font(.caption2).lineLimit(1).foregroundStyle(.secondary)
}
}
DynamicIslandExpandedRegion(.bottom) {
VStack(spacing: 6) {
Text(context.state.modeLine)
.font(.caption2).foregroundStyle(.secondary).lineLimit(1)
StageLine(state: context.state)
EndButton()
}
}
} compactLeading: {
Image(systemName: "play.tv.fill").foregroundStyle(.tint)
} compactTrailing: {
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.monospacedDigit()
.frame(maxWidth: 44)
} minimal: {
Image(systemName: "play.tv.fill").foregroundStyle(.tint)
}
}
}
}
// MARK: - Lock Screen banner
private struct LockScreenView: View {
let context: ActivityViewContext<PunktfunkSessionAttributes>
var body: some View {
HStack(alignment: .top, spacing: 12) {
Image(systemName: "play.tv.fill")
.font(.title2)
.foregroundStyle(.tint)
VStack(alignment: .leading, spacing: 3) {
HStack {
Text(context.attributes.hostName).font(.headline).lineLimit(1)
Spacer()
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.font(.subheadline).monospacedDigit()
.foregroundStyle(.secondary)
}
if let title = context.attributes.launchTitle {
Text(title).font(.caption).foregroundStyle(.secondary).lineLimit(1)
}
Text(context.state.modeLine)
.font(.caption2).foregroundStyle(.secondary).lineLimit(1)
StageLine(state: context.state)
}
if context.state.stage == .background {
EndButton()
}
}
.padding()
}
}
// MARK: - Shared pieces
/// The stage badge + (while backgrounded) the auto-disconnect countdown.
private struct StageLine: View {
let state: PunktfunkSessionAttributes.ContentState
var body: some View {
switch state.stage {
case .streaming:
EmptyView()
case .background:
if let deadline = state.backgroundDeadline {
HStack(spacing: 3) {
Text("Keeps running for")
Text(timerInterval: Date()...deadline, countsDown: true)
.monospacedDigit()
}
.font(.caption2)
.foregroundStyle(.secondary)
} else {
badge("Running in background", .orange)
}
case .reconnecting:
badge("Reconnecting…", .yellow)
case .ending:
badge("Session ended", .secondary)
}
}
private func badge(_ text: String, _ color: Color) -> some View {
Text(text).font(.caption2).foregroundStyle(color)
}
}
/// End-stream button runs EndStreamIntent in the app process (LiveActivityIntent).
private struct EndButton: View {
var body: some View {
Button(intent: EndStreamIntent()) {
Label("End", systemImage: "stop.fill")
.font(.caption).bold()
}
.tint(.red)
.buttonStyle(.bordered)
}
}
@@ -1,10 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
@@ -1,10 +0,0 @@
import Foundation
import PunktfunkKit
/// A fresh `pair=required`/unknown host pending a trust decision: drives both the "request access
/// vs. pair with PIN" choice and the subsequent approval wait. `advertisedFingerprint` is the
/// discovered host's advertised cert (nil for a manually-typed host trust-on-first-use).
struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
@@ -24,7 +24,6 @@ struct ContentView: View {
@AppStorage(DefaultsKey.streamWidth) private var width = 1920
@AppStorage(DefaultsKey.streamHeight) private var height = 1080
@AppStorage(DefaultsKey.streamHz) private var hz = 60
@AppStorage(DefaultsKey.renderScale) private var renderScale = 1.0
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@@ -47,20 +46,10 @@ struct ContentView: View {
case "h264": return PunktfunkConnection.codecH264
case "hevc": return PunktfunkConnection.codecHEVC
case "av1": return PunktfunkConnection.codecAV1
case "pyrowave": return PunktfunkConnection.codecPyroWave
default: return 0
}
}
@State private var showAddHost = false
/// A `punktfunk://` deep link (widget / Siri / Shortcuts) couldn't be honored unknown host, or
/// a live session is already up. Surfaced as an informational alert (distinct from the
/// "Connection failed" one, which is for actual connect errors).
@State private var deepLinkNotice: String?
#if os(iOS)
/// Owns the Live Activity for the running session (Lock Screen / Dynamic Island). Driven from
/// the session model's published state below; iPhone/iPad only.
@State private var liveActivity = SessionActivityController()
#endif
@State private var pairingTarget: StoredHost?
/// A fresh `pair=required`/unknown host the user tapped: drives the choice between no-PIN
/// delegated approval ("Request Access") and the SPAKE2 PIN ceremony (rule 3b).
@@ -102,14 +91,6 @@ struct ContentView: View {
/// fires Wake-on-LAN up front and falls into the "Waking" wait if the dial fails. Off: connects
/// go straight through with no wake. The explicit "Wake Host" action is unaffected either way.
@AppStorage(DefaultsKey.autoWake) private var autoWakeEnabled = true
/// Background keep-alive (Settings General, iOS-only). Default OFF (today's freeze-on-background
/// is the default). When on, backgrounding a live session keeps audio + the connection alive and
/// drops video, auto-disconnecting after `backgroundTimeoutMinutes`.
@AppStorage(DefaultsKey.backgroundKeepAlive) private var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) private var backgroundTimeoutMinutes = 10
/// scenePhase drives the keep-alive: use THIS, not the willResignActive observers resign-active
/// also fires for Control Center / app-switcher peeks, where the disconnect timer must not start.
@Environment(\.scenePhase) private var scenePhase
private var gamepadUIActive: Bool {
GamepadUIEnvironment.isActive(
gamepadConnected: gamepadManager.active != nil, enabledSetting: gamepadUIEnabled)
@@ -131,62 +112,7 @@ struct ContentView: View {
.onAppear {
seedDefaultModeIfNeeded()
autoConnectIfAsked()
#if os(iOS)
SessionActivityController.sweepOrphans() // end any Activity a prior killed launch left
#endif
}
// Deep links (widget quick-launch, Siri/Shortcuts): route into the SAME connect path a card
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) }
#if os(iOS)
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
.onChange(of: scenePhase) { _, phase in
switch phase {
case .background:
if backgroundKeepAlive, model.phase == .streaming {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
}
case .active:
model.exitBackground()
default:
break
}
}
// Live Activity lifecycle, driven from the model's published state.
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
if let host = model.activeHost {
liveActivity.begin(
hostID: host.id, hostName: host.displayName,
launchTitle: nil, // no live foreground-app title mid-session (v1)
modeLine: currentModeLine(), startedAt: Date())
}
case .idle:
liveActivity.end()
default:
break
}
}
.onChange(of: model.isBackgrounded) { _, backgrounded in
liveActivity.update {
$0.stage = backgrounded ? .background : .streaming
$0.backgroundDeadline = model.backgroundDeadline
}
}
// The Live Activity's / Shortcuts' End button runs EndStreamIntent in-process, which posts
// this tear the session down deliberately (quit-close the host).
.onReceive(NotificationCenter.default.publisher(for: .punktfunkEndActiveSession)) { _ in
model.disconnect(deliberate: true)
}
// Connect App Intent (Siri/Shortcuts): route its punktfunk:// URL through the same handler
// as a widget tap.
.onReceive(NotificationCenter.default.publisher(for: .punktfunkOpenDeepLink)) { note in
if let url = note.object as? URL { handleDeepLink(url) }
}
#endif
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
@@ -224,9 +150,6 @@ struct ContentView: View {
#if !os(tvOS)
.focusedSceneValue(\.sessionFocus, SessionFocus(
isStreaming: model.connection != nil,
clipboardAvailable: model.connection?.hostSupportsClipboard == true,
clipboardOn: model.clipboardEnabled,
toggleClipboard: { model.toggleClipboardSync() },
disconnect: { model.disconnect() }))
#endif
#if os(macOS)
@@ -338,59 +261,6 @@ struct ContentView: View {
+ "console (port 3000 → Pairing). This device connects automatically once you "
+ "approve it — no need to reconnect.")
}
// Informational deep-link outcome (unknown host / already streaming). Not an error.
.alert("Can't open", isPresented: deepLinkNoticePresented) {
Button("OK", role: .cancel) {}
} message: {
Text(deepLinkNotice ?? "")
}
}
/// Presentation flag for the informational deep-link alert. Extracted from the `.alert` call so
/// the manual get/set Binding type-checks on its own instead of inflating the body chain's
/// budget (adding it inline tips SwiftUI's per-expression limit see the split sections idiom).
private var deepLinkNoticePresented: Binding<Bool> {
Binding(get: { deepLinkNotice != nil }, set: { if !$0 { deepLinkNotice = nil } })
}
#if os(iOS)
/// The Live Activity mode line, e.g. "2560×1440 @120 · HEVC · HDR", from the live connection.
private func currentModeLine() -> String {
guard let c = model.connection else { return "" }
let codec: String
switch c.videoCodec {
case .h264: codec = "H.264"
case .hevc: codec = "HEVC"
case .av1: codec = "AV1"
case .pyrowave: codec = "PyroWave"
}
var line = "\(c.width)×\(c.height)"
if c.refreshHz > 0 { line += " @\(c.refreshHz)" }
line += " · \(codec)"
if c.isHDR { line += " · HDR" }
return line
}
#endif
/// Route a `punktfunk://` deep link into the existing connect path. Rules (per design):
/// unknown host notice + no-op; a live session is up ignore if it's the same host, else
/// tell the user to end the current one first (NEVER tear down a live session on a background
/// tap); otherwise the normal `connect` trust policy, WoL and the approval sheet all apply.
private func handleDeepLink(_ url: URL) {
guard case let .connect(hostID, launchID)? = DeepLink(url) else { return }
guard let host = store.hosts.first(where: { $0.id == hostID }) else {
deepLinkNotice = "That host isn't saved on this device."
return
}
if model.phase != .idle {
guard model.activeHost?.id == hostID else {
let current = model.activeHost?.displayName ?? "a host"
deepLinkNotice = "Already streaming \(current). End that session first."
return
}
return // deep-linked to the host we're already on nothing to do
}
connect(host, launchID: launchID)
}
private var home: some View {
@@ -739,17 +609,6 @@ struct ContentView: View {
/// host is back online. `prepareWake` still runs here to LEARN/refresh the MAC now that the host
/// is advertising (and is a harmless no-op otherwise). `onUnreachable` hands a plain connect
/// failure back to the caller (the wake-wait fallback) instead of the error alert.
/// The stream mode to request = the chosen resolution × the render scale, aspect-preserved,
/// even, and clamped to the codec's max dimension. > 1 supersamples for sharpness (the presenter
/// downscales the larger decoded frame to this display); < 1 renders under native and upscales.
/// The match-window path applies the SAME scale to the live window size in `MatchWindowFollower`.
private func scaledMode() -> (width: UInt32, height: UInt32) {
RenderScale.apply(
baseWidth: width, baseHeight: height,
scale: renderScale,
maxDimension: RenderScale.maxDimension(codec: codec))
}
private func startSessionDirect(
_ host: StoredHost, launchID: String? = nil,
allowTofu: Bool, requestAccess: Bool = false, approvalReq: ApprovalRequest? = nil,
@@ -761,7 +620,7 @@ struct ContentView: View {
if let approvalReq { awaitingApproval = approvalReq }
model.connect(
to: host,
width: scaledMode().width, height: scaledMode().height,
width: UInt32(clamping: width), height: UInt32(clamping: height),
hz: UInt32(clamping: hz),
compositor: PunktfunkConnection.Compositor(
rawValue: UInt32(clamping: compositor)) ?? .auto,
@@ -944,7 +803,7 @@ struct ContentView: View {
}
model.connect(
to: host,
width: scaledMode().width, height: scaledMode().height,
width: UInt32(clamping: width), height: UInt32(clamping: height),
hz: UInt32(clamping: hz),
compositor: pref,
gamepad: pad,
@@ -955,3 +814,71 @@ struct ContentView: View {
autoTrust: true)
}
}
#if os(macOS)
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
private struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
if want != isFull { window.toggleFullScreen(nil) }
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
}
}
#endif
/// A fresh `pair=required`/unknown host pending a trust decision: drives both the "request access
/// vs. pair with PIN" choice and the subsequent approval wait. `advertisedFingerprint` is the
/// discovered host's advertised cert (nil for a manually-typed host trust-on-first-use).
private struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
@@ -1,83 +0,0 @@
import PunktfunkKit
import SwiftUI
#if os(macOS)
import AppKit
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
/// The last `active` value we DROVE the window to. We toggle only when `active` itself
/// changes (stream start/end) never to correct a mismatch so a deliberate mid-session
/// toggle (F / the green button) isn't snapped back on the next SwiftUI update.
var lastActive: Bool?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
// Drive the window only on an `active` EDGE (stream start/end), not to close a mismatch
// so a user's F / green-button toggle stays put. First pass (lastActive == nil) just
// records the state without toggling, so mounting never yanks a window into fullscreen.
if coordinator.lastActive != want {
coordinator.lastActive = want
if want != isFull { window.toggleFullScreen(nil) }
}
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
// The Stream menu's "Toggle Fullscreen" (F) and InputCapture's captured-state interception
// both post this; flip the KEY window only (posted app-wide, object nil). The transition
// observers above then mirror the real state back into the binding.
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: .punktfunkToggleFullscreen, object: nil, queue: .main
) { [weak window] _ in
guard let window, window.isKeyWindow else { return }
window.toggleFullScreen(nil)
})
}
}
#endif
@@ -20,12 +20,6 @@ struct AddHostSheet: View {
@State private var address: String
@State private var port: Int
@State private var mac: String
#if os(macOS)
/// Share the clipboard with this host (macOS sessions only; design
/// clipboard-and-file-transfer.md §5.3). Off by default; honored only when the host
/// advertises the capability at connect.
@State private var clipboardSync: Bool
#endif
#if os(tvOS)
private enum EditField: String, Identifiable {
case name, address, port, mac
@@ -47,9 +41,6 @@ struct AddHostSheet: View {
_port = State(initialValue: Int(existing?.port ?? 9777))
let stored = existing?.macAddresses ?? []
_mac = State(initialValue: (stored.isEmpty ? suggestedMacs : stored).joined(separator: ", "))
#if os(macOS)
_clipboardSync = State(initialValue: existing?.clipboardSync ?? false)
#endif
}
var body: some View {
@@ -105,9 +96,6 @@ struct AddHostSheet: View {
#if os(iOS)
.textInputAutocapitalization(.never)
#endif
#if os(macOS)
Toggle("Share clipboard with this host", isOn: $clipboardSync)
#endif
}
#if !os(tvOS)
.formStyle(.grouped)
@@ -159,11 +147,6 @@ struct AddHostSheet: View {
host.address = address.trimmingCharacters(in: .whitespaces)
host.port = UInt16(clamping: port)
host.macAddresses = Self.parseMacs(mac)
#if os(macOS)
// nil when off: the key stays absent from the saved JSON (forward-compat, and "never
// opted in" and "opted out" read the same off).
host.clipboardSync = clipboardSync ? true : nil
#endif
onSave(host)
dismiss()
}
@@ -1,102 +0,0 @@
// Siri / Shortcuts / Spotlight surface (design §M4). Deliberately thin: every action already has an
// internal entry point M0's deep-link router (connect / connect-and-launch), M3's in-process
// end-session hook, and the existing Wake-on-LAN path so these intents only wrap them.
//
// Gated os(iOS): the AppShortcutsProvider bundles `EndStreamIntent`, which is a LiveActivityIntent
// (iPhone/iPad only). Connect/Wake themselves are plain AppIntents; they live here with the
// provider rather than being split across platforms. `HostEntity` (the parameter type) is in
// PunktfunkShared so the widget's configuration intent can share it.
#if os(iOS)
import AppIntents
import Foundation
import PunktfunkKit
/// Load a full saved host (MACs, address) from the shared App-Group store by id HostEntity only
/// carries id + name.
private func loadStoredHost(_ id: UUID) -> StoredHost? {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return nil }
return hosts.first { $0.id == id }
}
/// Start a session with a stored host (optionally launching a title). Foregrounds the app and
/// routes through the SAME `.onOpenURL` path a widget tap uses trust policy, WoL and the approval
/// sheet all apply, and its guards (unknown host, already-streaming) hold.
struct ConnectToHostIntent: AppIntent {
static let title: LocalizedStringResource = "Connect to Host"
static let description = IntentDescription("Start a Punktfunk streaming session with a host.")
static let openAppWhenRun = true
@Parameter(title: "Host") var host: HostEntity
@Parameter(title: "Game ID", description: "Optional store id like steam:570")
var launchID: String?
func perform() async throws -> some IntentResult {
let url = DeepLink.connect(host: host.id, launchID: launchID).url
await MainActor.run {
NotificationCenter.default.post(name: .punktfunkOpenDeepLink, object: url)
}
return .result()
}
}
/// Wake a sleeping host (magic packet). No `openAppWhenRun` usable in automations ("when I get
/// home, wake the tower") without foregrounding the app.
struct WakeHostIntent: AppIntent {
static let title: LocalizedStringResource = "Wake Host"
static let description = IntentDescription("Send a Wake-on-LAN magic packet to a host.")
@Parameter(title: "Host") var host: HostEntity
func perform() async throws -> some IntentResult {
guard let stored = loadStoredHost(host.id), !stored.wakeMacs.isEmpty else {
throw IntentError.noWakeAddress
}
PunktfunkConnection.wakeOnLAN(macs: stored.wakeMacs, lastKnownIP: stored.address)
return .result()
}
}
/// Errors surfaced to Siri/Shortcuts. `CustomLocalizedStringResourceConvertible` makes the message
/// show as the intent's failure text.
enum IntentError: Error, CustomLocalizedStringResourceConvertible {
case noWakeAddress
var localizedStringResource: LocalizedStringResource {
switch self {
case .noWakeAddress:
// One string LITERAL LocalizedStringResource is ExpressibleByStringLiteral, but a
// `"" + ""` concatenation is a runtime String it can't convert.
return "That host has no saved Wake-on-LAN address yet. Connect to it once so Punktfunk can learn it."
}
}
}
/// Zero-setup Siri / Spotlight phrases. Parameterized phrases resolve a `HostEntity` by name; stays
/// well under the 10-shortcut cap.
struct PunktfunkShortcuts: AppShortcutsProvider {
static var appShortcuts: [AppShortcut] {
AppShortcut(
intent: ConnectToHostIntent(),
phrases: [
"Connect to \(\.$host) in \(.applicationName)",
"Stream \(\.$host) with \(.applicationName)",
],
shortTitle: "Connect", systemImageName: "play.tv.fill")
AppShortcut(
intent: WakeHostIntent(),
phrases: [
"Wake \(\.$host) with \(.applicationName)",
],
shortTitle: "Wake Host", systemImageName: "power")
AppShortcut(
intent: EndStreamIntent(),
phrases: [
"End the \(.applicationName) stream",
],
shortTitle: "End Stream", systemImageName: "stop.fill")
}
}
#endif
@@ -1,89 +0,0 @@
// Owns the ActivityKit Live Activity lifecycle for a streaming session (iPhone/iPad only). Driven
// by ContentView from the session model's published state (phase / isBackgrounded / deadline) so
// none of this leaks into the cross-platform SessionModel. Local updates only (`pushType: nil`)
// the app process is alive whenever there's a session to report, so there's no push token plumbing.
//
// Gated os(iOS): ActivityKit is iPhone/iPad only. Minimum deployment is iOS 17, so no @available
// guards are needed (Activity has existed since 16.1).
#if os(iOS)
import ActivityKit
import Foundation
// PunktfunkKit re-exports PunktfunkShared (@_exported), so the app target sees PunktfunkSessionAttributes
// without linking the Shared product directly same pattern as StoredHost in HostStore.
import PunktfunkKit
@MainActor
final class SessionActivityController {
private var activity: Activity<PunktfunkSessionAttributes>?
/// The last pushed state, so an update can mutate one field and keep the rest (notably
/// `startedAt`, which the Lock-Screen timer ticks from).
private var state: PunktfunkSessionAttributes.ContentState?
/// How far past the next expected update to mark the content stale a frozen opt-out session
/// then greys out instead of showing a lying clock.
private static let staleWindow: TimeInterval = 90
var isActive: Bool { activity != nil }
/// End any Activity left over from a previous launch that was killed mid-session. Call once at
/// app start (ContentView.onAppear).
static func sweepOrphans() {
Task {
for activity in Activity<PunktfunkSessionAttributes>.activities {
await activity.end(nil, dismissalPolicy: .immediate)
}
}
}
/// Start the Live Activity for a freshly-streaming session. No-op if the user disabled Live
/// Activities for the app, or one is already up.
func begin(hostID: UUID, hostName: String, launchTitle: String?, modeLine: String, startedAt: Date) {
guard ActivityAuthorizationInfo().areActivitiesEnabled, activity == nil else { return }
let attributes = PunktfunkSessionAttributes(
hostID: hostID, hostName: hostName, launchTitle: launchTitle)
let initial = PunktfunkSessionAttributes.ContentState(
stage: .streaming, startedAt: startedAt, modeLine: modeLine)
state = initial
do {
activity = try Activity.request(
attributes: attributes,
content: content(initial),
pushType: nil)
} catch {
activity = nil
state = nil
}
}
/// Coalesced update: mutate the running state in place (keeps `startedAt` etc.) and push once.
/// No-op when there's no live Activity.
func update(_ mutate: (inout PunktfunkSessionAttributes.ContentState) -> Void) {
guard let activity, var next = state else { return }
mutate(&next)
state = next
Task { await activity.update(content(next)) }
}
/// End with a final "ended" state, dismissed a few seconds later.
func end() {
guard let activity, var final = state else {
self.activity = nil
state = nil
return
}
self.activity = nil
state = nil
final.stage = .ending
final.backgroundDeadline = nil
Task {
await activity.end(content(final), dismissalPolicy: .after(.now + 4))
}
}
private func content(_ s: PunktfunkSessionAttributes.ContentState)
-> ActivityContent<PunktfunkSessionAttributes.ContentState> {
ActivityContent(state: s, staleDate: Date().addingTimeInterval(Self.staleWindow))
}
}
#endif
@@ -139,18 +139,6 @@ final class SessionModel: ObservableObject {
private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture?
private var gamepadFeedback: GamepadFeedback?
#if os(macOS)
/// The live session's clipboard bridge (design/clipboard-and-file-transfer.md §5) created
/// by `beginStreaming` when the per-host toggle is on and the host advertises
/// `HOST_CAP_CLIPBOARD`; stopped (off-main, drain joined) in `disconnect`.
private var clipboardSync: ClipboardSync?
#endif
/// Whether clipboard sync is live (host-acked `ClipState.enabled`) drives the Stream menu
/// item's title and the settings footnote. Always false off-macOS.
@Published private(set) var clipboardEnabled = false
/// The host's last `ClipState.reason` (`CLIP_REASON_*`) why an enable was refused
/// (backend unavailable / policy disabled / ); 0 = OK.
@Published private(set) var clipboardReason: UInt8 = 0
#if os(tvOS)
/// Siri Remote host pointer while streaming (touch surface moves, press = left click,
/// Play/Pause = right click) + the remote's deliberate exit (hold Back 1 s). See
@@ -160,16 +148,6 @@ final class SessionModel: ObservableObject {
var isBusy: Bool { phase != .idle }
/// True while a streaming session is running in the background under the opt-in keep-alive
/// (audio plays, video dropped, timeout armed). Drives the Live Activity's stage/countdown (M3)
/// and is cleared on foreground or teardown. iOS/iPadOS only in practice.
@Published private(set) var isBackgrounded = false
/// When the backgrounded keep-alive will auto-disconnect (nil unless backgrounded) drives the
/// Live Activity countdown. Set alongside `backgroundTimer`.
@Published private(set) var backgroundDeadline: Date?
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer?
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -261,18 +239,6 @@ final class SessionModel: ObservableObject {
// from these + the soft `preferredCodec`; `resolvedCodec` reflects what it chose.
var videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
if AV1.hardwareDecodeSupported { videoCodecs |= PunktfunkConnection.codecAV1 }
// PyroWave (wired LAN) is a pure opt-in: picking it in the codec setting both
// advertises the bit and prefers it the host never auto-selects it, and the
// picker only offers it when the Metal decode probe passed (simdgroup floor A13;
// every M-series Mac and the ATV 4K gen 3 pass). The codec is 8-bit 4:2:0 SDR
// BT.709 by contract, so the opt-in also drops the HDR/10-bit/4:4:4 caps for this
// session HDR sessions stay HEVC/AV1 (plan §4.7).
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
videoCodecs |= PunktfunkConnection.codecPyroWave
videoCaps &= ~(PunktfunkConnection.videoCap10Bit
| PunktfunkConnection.videoCapHDR
| PunktfunkConnection.videoCap444)
}
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
@@ -318,15 +284,10 @@ final class SessionModel: ObservableObject {
self.errorMessage = "\(host.displayName) is not paired yet. "
+ "Pair with its PIN before streaming."
}
case .failure(let error):
case .failure:
self.phase = .idle
self.activeHost = nil
if case PunktfunkClientError.rejected(let rejection) = error {
// The host answered and stated its reason (declined / approval timed
// out / busy / versions differ) show that, and never wake-retry a
// host that is demonstrably awake.
self.errorMessage = "\(host.displayName): \(rejection.userMessage)"
} else if let onUnreachable, !requestAccess {
if let onUnreachable, !requestAccess {
// The caller owns recovery (wake-and-retry) no error alert here; its
// own overlay explains what's happening.
onUnreachable()
@@ -354,48 +315,6 @@ final class SessionModel: ObservableObject {
}
}
// MARK: - Background keep-alive (opt-in, iOS)
/// Enter the backgrounded keep-alive state: keep audio playing, DROP video decode (no GPU work
/// off-screen), mute the mic (privacy), and arm a bounded auto-disconnect. The caller
/// (ContentView's scenePhase driver) gates this on the setting + `.streaming`; a no-op otherwise.
/// The video-drop seam is read by both pumps every iteration (`connection.isVideoDropped`).
func enterBackground(timeoutMinutes: Int) {
guard phase == .streaming, let conn = connection, !isBackgrounded else { return }
isBackgrounded = true
conn.setVideoDropped(true)
audio?.setMicMuted(true)
// Non-deliberate on fire (keep the host linger) so a user who returns late reconnects fast,
// exactly like today's network-drop path. min 1 minute guards a nonsense setting.
let minutes = max(1, timeoutMinutes)
backgroundDeadline = Date().addingTimeInterval(TimeInterval(minutes * 60))
let timer = DispatchSource.makeTimerSource(queue: .main)
timer.schedule(deadline: .now() + .seconds(minutes * 60))
timer.setEventHandler { [weak self] in
// The timer fires on `.main`, so the actor's executor is the main thread here.
MainActor.assumeIsolated { self?.disconnect(deliberate: false) }
}
backgroundTimer?.cancel()
backgroundTimer = timer
timer.resume()
}
/// Return to foreground: cancel the timeout, resume mic + video, and force a clean re-anchor
/// request a fresh IDR (infinite GOP: it won't come on its own) and let the pump's freeze gate
/// withhold the concealed frames until it lands (it auto-arms on the resumed frame-index gap).
func exitBackground() {
guard isBackgrounded else { return }
isBackgrounded = false
backgroundDeadline = nil
backgroundTimer?.cancel()
backgroundTimer = nil
audio?.setMicMuted(false)
if let conn = connection {
conn.setVideoDropped(false)
conn.requestKeyframe()
}
}
/// The user confirmed the fingerprint: returns it for pinning and enters streaming.
func confirmTrust() -> Data? {
guard case .awaitingTrust(let fingerprint) = phase else { return nil }
@@ -413,11 +332,6 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate()
statsTimer = nil
// Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel()
backgroundTimer = nil
isBackgrounded = false
backgroundDeadline = nil
let audio = self.audio
self.audio = nil
// Gamepad capture is main-actor (releases held buttons on the wire while the
@@ -430,12 +344,6 @@ final class SessionModel: ObservableObject {
#endif
let feedback = gamepadFeedback
gamepadFeedback = nil
#if os(macOS)
let clipboard = clipboardSync
clipboardSync = nil
#endif
clipboardEnabled = false
clipboardReason = 0
if let conn = connection {
// Drain-thread teardown waits the pullers out and close() waits out in-flight
// polls + joins the Rust worker threads keep all of it off the main actor,
@@ -443,9 +351,6 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
clipboard?.stop() // disables sync on the wire while the connection is still up
#endif
// Deliberate user quit tell the host to skip the keep-alive linger (must precede close).
if deliberate { conn.disconnectQuit() }
conn.close()
@@ -454,9 +359,6 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
clipboard?.stop()
#endif
}
}
connection = nil
@@ -517,10 +419,9 @@ final class SessionModel: ObservableObject {
micChannel: defaults.integer(forKey: DefaultsKey.micChannel),
micEnabled: defaults.object(forKey: DefaultsKey.micEnabled) as? Bool ?? true)
self.audio = audio
// Gamepads: forward every controller GamepadManager selected each on its own wire pad
// index (a pin forwards only one, Automatic forwards all) and render the host's feedback
// back to the pad it's addressed to (rumble always; lightbar/player-LEDs/adaptive-triggers
// when a pad's virtual device is a DualSense). Same trust gate as audio nothing is
// Gamepads: forward GamepadManager's active controller as pad 0 and render the
// host's feedback (rumble always; lightbar/player-LEDs/adaptive-triggers when the
// session's virtual pad is a DualSense). Same trust gate as audio nothing is
// forwarded during the trust prompt.
let capture = GamepadCapture(connection: conn, manager: .shared)
// The cross-client escape chord (hold L1+R1+Start+Select 1.5 s) on tvOS the only
@@ -531,14 +432,6 @@ final class SessionModel: ObservableObject {
let feedback = GamepadFeedback(connection: conn, manager: .shared)
feedback.start()
gamepadFeedback = feedback
#if os(macOS)
// Shared clipboard: opt-in per host AND host-advertised (older hosts / operator-disabled
// hosts never see a ClipControl). Same trust gate as audio nothing is announced
// during the trust prompt.
if activeHost?.clipboardSync == true, conn.hostSupportsClipboard {
startClipboardSync(conn)
}
#endif
#if os(tvOS)
let pointer = SiriRemotePointer(connection: conn)
pointer.onDisconnectRequest = { [weak self] in self?.disconnect() }
@@ -547,40 +440,6 @@ final class SessionModel: ObservableObject {
#endif
}
#if os(macOS)
/// Create + start the session's clipboard bridge and route its host acks into the published
/// UI state. `ClipboardSync.start()` sends the enable; the host's `.state` answer flips
/// `clipboardEnabled` (or leaves it false with a `clipboardReason` the UI can explain).
private func startClipboardSync(_ conn: PunktfunkConnection) {
let sync = ClipboardSync(connection: conn)
sync.onState = { [weak self] enabled, _, reason in
Task { @MainActor in
self?.clipboardEnabled = enabled
self?.clipboardReason = reason
}
}
sync.start()
clipboardSync = sync
}
#endif
/// Flip clipboard sync mid-session (the Stream menu). Off on requires the host cap; on
/// off tears the bridge down (off-main the drain join must not block the main actor) and
/// tells the host, which drops any selection we own there. No-op off-macOS or while idle.
func toggleClipboardSync() {
#if os(macOS)
guard let conn = connection, phase == .streaming else { return }
if let sync = clipboardSync {
clipboardSync = nil
clipboardEnabled = false
clipboardReason = 0
Task.detached { sync.stop() }
} else if conn.hostSupportsClipboard {
startClipboardSync(conn)
}
#endif
}
private func startStatsTimer() {
lastFramesDropped = 0 // a fresh connection's cumulative drop counter starts at 0
latencySplit.reset() // no stale receipts/samples from a previous session
@@ -21,12 +21,6 @@ import SwiftUI
/// `.focusedSceneValue` so the Scene-level commands can drive it.
struct SessionFocus {
var isStreaming: Bool
/// The connected host advertises `HOST_CAP_CLIPBOARD` (gates the Share Clipboard item
/// macOS-only UI, but the fact is platform-neutral).
var clipboardAvailable: Bool
/// Clipboard sync is live (host-acked) drives the item's Stop/Share title.
var clipboardOn: Bool
var toggleClipboard: () -> Void
var disconnect: () -> Void
}
@@ -64,15 +58,6 @@ struct StreamCommands: Commands {
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
#if os(macOS)
// Mid-session clipboard flip (design/clipboard-and-file-transfer.md §5.3). Greyed
// when the host doesn't advertise the cap (older host / operator policy off).
Button(session?.clipboardOn == true ? "Stop Sharing Clipboard" : "Share Clipboard") {
session?.toggleClipboard()
}
.keyboardShortcut("c", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true || session?.clipboardAvailable != true)
#endif
Divider()
Button("Disconnect") { session?.disconnect() }
.keyboardShortcut("d", modifiers: [.control, .option, .shift])
@@ -15,9 +15,6 @@ import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS) || os(tvOS)
import GameController
#if os(iOS)
import CoreHaptics
#endif
struct GamepadSettingsView: View {
@Environment(\.dismiss) private var dismiss
@@ -41,9 +38,6 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.gamepadUIEnabled) private var gamepadUIEnabled = true
@AppStorage(DefaultsKey.autoWake) private var autoWakeEnabled = true
@AppStorage(DefaultsKey.presenter) private var presenter = SettingsOptions.presenterDefault
#if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
#endif
@ObservedObject private var gamepads = GamepadManager.shared
#if os(iOS)
@@ -236,7 +230,7 @@ struct GamepadSettingsView: View {
.map { (label: "\($0) Hz", tag: $0) }
let bitrate = SettingsOptions.bitrateOptions(current: bitrateKbps)
let controllers = SettingsOptions.controllerOptions(gamepads)
var list: [Row] = [
return [
choiceRow(
id: "resolution", header: "Stream", icon: "aspectratio",
label: "Resolution",
@@ -335,23 +329,6 @@ struct GamepadSettingsView: View {
detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled),
]
#if os(iOS)
// The device-rumble mirror slots in after "Controller type" (staying inside the
// Controller group the next row carries the "Interface" header). iPhone only in
// practice: hidden where the device itself can't play haptics (iPad).
if CHHapticEngine.capabilitiesForHardware().supportsHaptics,
let at = list.firstIndex(where: { $0.id == "padType" }) {
list.insert(
toggleRow(
id: "deviceRumble", icon: "iphone.radiowaves.left.and.right",
label: "Rumble on this iPhone",
detail: "Also play player 1's rumble on the phone's own Taptic Engine — "
+ "for clip-on pads without rumble motors.",
value: $rumbleOnDevice),
at: at + 1)
}
#endif
return list
}
/// Resolution choices as "WxH" tags the current size is inserted when it's a custom mode
@@ -79,13 +79,6 @@ enum SettingsOptions {
if AV1.hardwareDecodeSupported {
options.insert(("AV1", "av1"), at: 2)
}
// PyroWave is the opt-in wired-LAN low-latency codec (100400 Mbps all-intra wavelet,
// 8-bit SDR): selecting it advertises + prefers it for the session. Offered only when
// the Metal decode probe passes (same gate SessionModel advertises by) elsewhere the
// host could never emit it.
if MetalWaveletDecoder.supported {
options.append(("PyroWave (wired LAN)", "pyrowave"))
}
return options
}()
@@ -1,9 +1,6 @@
// SettingsView's shared sections each setting's Section is defined exactly once here and
// composed by the per-platform bodies in SettingsView.swift.
#if os(iOS)
import CoreHaptics
#endif
import PunktfunkKit
import SwiftUI
@@ -440,34 +437,6 @@ extension SettingsView {
}
}
/// iOS/iPadOS only: keep a backgrounded session alive (audio background mode). Empty elsewhere
/// (tvOS backgrounding semantics differ; macOS isn't gated by the mode) so the shared `.general`
/// detail can reference it unconditionally.
@ViewBuilder var keepAliveSection: some View {
#if os(iOS)
Section {
Toggle("Keep streaming in background", isOn: $backgroundKeepAlive)
if backgroundKeepAlive {
Picker("Disconnect after", selection: $backgroundTimeoutMinutes) {
Text("1 minute").tag(1)
Text("5 minutes").tag(5)
Text("10 minutes").tag(10)
Text("30 minutes").tag(30)
}
}
} header: {
Text("Background")
} footer: {
Text("Off by default: backgrounding the app freezes the session. When on, audio keeps "
+ "playing and the connection stays live (video is dropped to save power) after you "
+ "switch away — and the session auto-disconnects after the time above so it can't "
+ "run down your battery. Returning to the app resumes video instantly.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder var experimentalSection: some View {
Section {
Toggle("Show game library", isOn: $libraryEnabled)
@@ -502,12 +471,6 @@ extension SettingsView {
Text(option.label).tag(option.tag)
}
}
#if os(iOS)
// iPhone only in practice: hidden where the device itself can't play haptics (iPad).
if CHHapticEngine.capabilitiesForHardware().supportsHaptics {
Toggle("Rumble on this iPhone", isOn: $rumbleOnDevice)
}
#endif
#if !os(tvOS)
Toggle("Gamepad-optimized browsing", isOn: $gamepadUIEnabled)
#endif
@@ -524,11 +487,6 @@ extension SettingsView {
// for its own footer and has no such toggle to describe.
VStack(alignment: .leading, spacing: 6) {
Text(Self.controllersFooter)
#if os(iOS)
if CHHapticEngine.capabilitiesForHardware().supportsHaptics {
Text(Self.deviceRumbleFooter)
}
#endif
#if !os(tvOS)
Text(Self.gamepadUIFooter)
#endif
@@ -88,13 +88,6 @@ extension SettingsView {
+ "controller (a DualSense keeps adaptive triggers, lightbar, touchpad and motion). "
+ "Applies from the next session."
#if os(iOS)
static let deviceRumbleFooter =
"Rumble on this iPhone plays player 1's rumble on the phone's own Taptic Engine as "
+ "well — for clip-on controllers that have no rumble motors of their own. Applies "
+ "from the next session."
#endif
#if !os(tvOS)
static let gamepadUIFooter =
"When a controller connects, the host list and library switch to a controller-"
@@ -140,10 +133,8 @@ extension SettingsView {
.foregroundStyle(.secondary)
}
Spacer()
// Every forwarded controller is surfaced (not just the primary `active`) with its
// wire pad index as a player number a pin forwards only one, Automatic forwards all.
if let pad = gamepads.padIndex(for: controller) {
Text("Player \(pad + 1)")
if gamepads.active?.id == controller.id {
Text("In use")
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 8)
.padding(.vertical, 3)
@@ -21,10 +21,10 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.streamWidth) var width = 1920
@AppStorage(DefaultsKey.streamHeight) var height = 1080
@AppStorage(DefaultsKey.streamHz) var hz = 60
// Opt-in (default OFF): the explicit mode below is used and never auto-resized. When ON, a
// windowed session instead streams at the window's native pixels (1:1, no scaling) so it stays
// pixel-exact rather than the presenter resampling a fixed-mode frame into the window.
@AppStorage(DefaultsKey.matchWindow) var matchWindow = false
// Default ON: a windowed session streams at the window's native pixels (1:1, no scaling) so it
// stays pixel-exact instead of the presenter resampling a fixed-mode frame into the window.
// Off falls back to the explicit mode below (fixed output, scaled to non-matching windows).
@AppStorage(DefaultsKey.matchWindow) var matchWindow = true
@AppStorage(DefaultsKey.compositor) var compositor = 0
@AppStorage(DefaultsKey.gamepadType) var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) var bitrateKbps = 0
@@ -49,15 +49,12 @@ struct SettingsView: View {
@ObservedObject var gamepads = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true
@AppStorage(DefaultsKey.autoWake) var autoWakeEnabled = true
@AppStorage(DefaultsKey.backgroundKeepAlive) var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) var backgroundTimeoutMinutes = 10
#if DEBUG && !os(tvOS)
@State var showControllerTest = false
#endif
#if os(iOS)
@AppStorage(DefaultsKey.pointerCapture) var pointerCapture = true
@AppStorage(DefaultsKey.touchMode) var touchMode = TouchInputMode.trackpad.rawValue
@AppStorage(DefaultsKey.rumbleOnDevice) var rumbleOnDevice = false
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
@@ -244,7 +241,6 @@ struct SettingsView: View {
pointerSection
compositorSection
wakeSection
keepAliveSection // iOS-only content; empty on tvOS
}
.formStyle(.grouped)
.navigationTitle("General")
@@ -11,13 +11,32 @@
import Foundation
import PunktfunkKit
import SwiftUI
#if canImport(WidgetKit)
import WidgetKit
#endif
// `StoredHost` (the model + its JSON codec) now lives in PunktfunkShared so the widget extension
// can read the same store; PunktfunkKit re-exports it. The discovery-join helpers below stay here
// because they reference PunktfunkKit's `DiscoveredHost`/`HostDiscovery`.
struct StoredHost: Identifiable, Codable, Hashable {
var id = UUID()
var name: String
var address: String
var port: UInt16 = 9777
/// SHA-256 of the host's certificate, set after the user explicitly trusted it.
var pinnedSHA256: Data?
/// Last time a streaming session actually started (nil until the first one).
var lastConnected: Date?
/// Management-API port for the library browser (distinct from the data-plane `port`). Optional
/// (NOT a defaulted non-optional) so older saved hosts whose JSON lacks this key still
/// decode: synthesized Decodable ignores property defaults but treats a missing Optional as
/// nil. Resolve via `effectiveMgmtPort`. (Auth is mTLS by the pinned identity no token.)
var mgmtPort: UInt16?
/// Wake-on-LAN MAC address(es) of the host's wake-capable NIC(s), each `aa:bb:cc:dd:ee:ff`.
/// Learned from the host's mDNS `mac` TXT record while it's awake and persisted here, so the
/// client can send a magic packet to wake the host later (when it's asleep and no longer
/// advertising). Optional (same forward-compat reason as `mgmtPort`); nil until first learned.
var macAddresses: [String]?
var displayName: String { name.isEmpty ? address : name }
var effectiveMgmtPort: UInt16 { mgmtPort ?? punktfunkDefaultMgmtPort }
/// Wake-capable, in a form the wake helper accepts (empty when none learned yet).
var wakeMacs: [String] { macAddresses ?? [] }
}
extension StoredHost {
/// True when a live mDNS advert (`DiscoveredHost`) describes THIS saved host drives the
@@ -67,14 +86,8 @@ final class HostStore: ObservableObject {
/// never advertises still reads Online. Not persisted (it's live reachability, not config).
@Published var probedOnline: Set<StoredHost.ID> = []
/// The App-Group suite shared with the Widget/Live-Activity extension so a launcher widget
/// sees the same saved hosts. Falls back to `.standard` in an un-entitled process (see
/// `AppGroup.defaults`).
private let defaults = AppGroup.defaults
init() {
Self.migrateToAppGroupIfNeeded()
if let data = defaults.data(forKey: Self.key),
if let data = UserDefaults.standard.data(forKey: Self.key),
let decoded = try? JSONDecoder().decode([StoredHost].self, from: data) {
hosts = decoded
} else {
@@ -82,20 +95,6 @@ final class HostStore: ObservableObject {
}
}
/// One-time move of the saved-host JSON from `UserDefaults.standard` (where every build before
/// the App Group wrote it) into the shared suite. Idempotent: only fires when the suite has no
/// hosts yet but standard does. The old value is LEFT in place during a staged TestFlight
/// rollout an older build still reads `.standard`, so tombstoning it now would hide hosts from
/// the not-yet-updated app. Remove the standard copy a release later.
private static func migrateToAppGroupIfNeeded() {
let suite = AppGroup.defaults
let standard = UserDefaults.standard
guard suite !== standard else { return } // un-entitled fallback: nothing to migrate
guard suite.data(forKey: key) == nil,
let legacy = standard.data(forKey: key) else { return }
suite.set(legacy, forKey: key)
}
func add(_ host: StoredHost) {
hosts.append(host)
}
@@ -113,7 +112,7 @@ final class HostStore: ObservableObject {
func markConnected(_ hostID: UUID) {
guard let i = hosts.firstIndex(where: { $0.id == hostID }) else { return }
hosts[i].lastConnected = Date() // didSet persist() writes the shared suite + reloads widget
hosts[i].lastConnected = Date()
}
/// One reachability sweep, driving `probedOnline`: probe every saved host NOT currently
@@ -159,17 +158,7 @@ final class HostStore: ObservableObject {
private func persist() {
if let data = try? JSONEncoder().encode(hosts) {
defaults.set(data, forKey: Self.key)
UserDefaults.standard.set(data, forKey: Self.key)
}
reloadHostsWidget() // the widget reads this store; any change refreshes its timeline
}
/// Ask WidgetKit to rebuild the hosts widget's timeline after any store change (add/remove/pin/
/// last-connected). iOS-only and a no-op where WidgetKit is absent; the widget uses
/// `.never`-refresh entries and relies on this push.
private func reloadHostsWidget() {
#if canImport(WidgetKit) && os(iOS)
WidgetCenter.shared.reloadTimelines(ofKind: "PunktfunkHosts")
#endif
}
}
@@ -212,18 +212,14 @@ struct PairSheet: View {
case .failure(PunktfunkClientError.wrongPIN):
errorText = "Wrong PIN — check the host's web console (port 3000) "
+ "and try again."
case .failure(PunktfunkClientError.rejected(let rejection)):
// The host answered and said why (not armed / rate-limited / armed for
// another device) show that instead of the guessing-game fallback.
errorText = rejection.userMessage
case .failure(is ClientIdentityStore.IdentityError):
errorText = "Can't store this Mac's identity in the Keychain, so the "
+ "pairing would not survive a relaunch. Unlock the login "
+ "keychain and try again."
case .failure:
errorText = "Pairing failed the host didn't answer. Is it running, "
+ "and is this device on the same network (no VPN, no guest-Wi-Fi "
+ "isolation)?"
errorText = "Pairing failed. Is the host reachable, pairing armed "
+ "(web console → Pairing), and not mid-session? Retries are "
+ "rate-limited to one per 2 seconds."
}
}
}
@@ -180,23 +180,6 @@ public final class SessionAudio {
}
}
/// Background keep-alive: silence the mic uplink while backgrounded (privacy no room audio
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
/// this stops the actual capture, which is the privacy-relevant part. Main thread.
public func setMicMuted(_ muted: Bool) {
stateLock.lock()
let capture = captureEngine
stateLock.unlock()
guard let capture else { return }
if muted {
capture.pause()
} else if !flag.isStopped {
try? capture.start()
}
}
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
@@ -1,361 +0,0 @@
// Shared clipboard, macOS client half (design/clipboard-and-file-transfer.md §5.2).
//
// Bridges NSPasteboard.general to the session's QUIC clipboard plane, both directions lazy:
//
// * **Local copy host**: a changeCount poll announces the *format list* (`clipOffer`); the
// bytes cross only when a host app pastes (a `.fetchRequest` event, answered from the live
// pasteboard by `clipServe`).
// * **Host copy local**: a `.remoteOffer` writes one NSPasteboardItem whose
// NSPasteboardItemDataProvider fires only when a Mac app actually pastes the provider then
// blocks (on its provider thread, never main) on a `clipFetch` round-trip.
//
// Password-manager respect: pasteboards marked `org.nspasteboard.ConcealedType` or
// `org.nspasteboard.TransientType` are never announced, never fetchable. Echo suppression: the
// changeCount of every write WE make is recorded so the announce poll skips it (§3.4).
//
// Phase 1 formats only (text / RTF / HTML / PNG). Files (NSFilePromiseProvider) ride Phase 2.
#if os(macOS)
import AppKit
import Foundation
/// One live session's clipboard bridge. Created by the session model when streaming begins on a
/// host that advertises `HOST_CAP_CLIPBOARD` and whose per-host toggle is on; `stop()` before the
/// connection closes. All pasteboard traffic runs on one dedicated drain thread plus the
/// AppKit-owned provider threads (paste fulfillment).
public final class ClipboardSync: NSObject {
/// Wire MIME NSPasteboard type for the Phase-1 vocabulary (§3.5), in announce order.
private static let wireToPasteboard: [(wire: String, type: NSPasteboard.PasteboardType)] = [
("text/plain;charset=utf-8", .string),
("text/rtf", .rtf),
("text/html", .html),
("image/png", .png),
]
/// Pasteboard marker types that must never cross the wire (password managers mark secrets
/// with these see nspasteboard.org).
private static let concealed = NSPasteboard.PasteboardType("org.nspasteboard.ConcealedType")
private static let transient = NSPasteboard.PasteboardType("org.nspasteboard.TransientType")
/// How long a blocked paste waits for the host's bytes before providing nothing (§5.2).
private static let fetchTimeout: TimeInterval = 10
/// Serve chunk size for host-side pastes of our data (bounds the per-call ABI copy).
private static let serveChunk = 4 << 20
private let connection: PunktfunkConnection
/// `CLIP_FLAG_*` sent with the enable (`CLIP_FLAG_FILES` when the session permits files
/// always 0 in Phase 1).
private let controlFlags: UInt8
/// Host `.state` updates, delivered on the main queue drives the toggle/footnote UI.
public var onState: ((_ enabled: Bool, _ policy: UInt8, _ reason: UInt8) -> Void)?
// Drain-thread state (touched only on the drain thread once started).
private var offerSeq: UInt32 = 0
private var lastSeenChangeCount = 0
/// The changeCount of the last pasteboard write WE made (echo suppression + "do we still
/// own the pasteboard" on teardown/clear).
private var ownedChangeCount = -1
/// The host offer currently installed on the local pasteboard (nil = none).
private var installedRemoteSeq: UInt32?
/// Outbound fetches a blocked paste is waiting on. Guarded by `fetchLock` appended by the
/// drain thread (`.data` events), consumed by AppKit's provider threads.
private struct PendingFetch {
var buffer = Data()
let done = DispatchSemaphore(value: 0)
var failed = false
}
private let fetchLock = NSLock()
private var pendingFetches: [UInt32: PendingFetch] = [:]
private final class StopFlag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
}
private let flag = StopFlag()
private let drainDone = DispatchSemaphore(value: 0)
private var started = false
/// Set by the app-activation observer, cleared by the drain loop: the user may have copied
/// elsewhere and is coming back to paste announce immediately instead of waiting out the
/// poll interval.
private final class OneShot: @unchecked Sendable {
private let lock = NSLock()
private var raised = false
func raise() {
lock.lock()
raised = true
lock.unlock()
}
func takeIfRaised() -> Bool {
lock.lock()
defer { lock.unlock() }
let was = raised
raised = false
return was
}
}
private let checkNow = OneShot()
private var activationObserver: NSObjectProtocol?
public init(connection: PunktfunkConnection, allowFiles: Bool = false) {
self.connection = connection
self.controlFlags = 0 // CLIP_FLAG_FILES rides Phase 2
_ = allowFiles
super.init()
}
deinit { flag.stop() }
/// Enable sync with the host and start the drain thread. The host answers the enable with a
/// `.state` event (surfaced via `onState`) `BACKEND_UNAVAILABLE` et al. arrive there.
public func start() {
guard !started else { return }
started = true
connection.clipControl(enabled: true, flags: controlFlags)
// Baseline: whatever is on the pasteboard when sync starts is announced immediately
// the "copy first, then connect and paste" flow must work.
lastSeenChangeCount = -1
activationObserver = NotificationCenter.default.addObserver(
forName: NSApplication.didBecomeActiveNotification, object: nil, queue: nil
) { [checkNow] _ in checkNow.raise() }
let connection = self.connection
let flag = self.flag
let thread = Thread { [weak self] in
var lastAnnounceCheck = Date.distantPast
while !flag.isStopped {
// Drain events (bounded burst so a chatty host can't starve the announce poll).
var drained = 0
while drained < 32, !flag.isStopped {
let ev: PunktfunkConnection.ClipEvent?
do {
ev = try connection.nextClipboard(timeoutMs: drained == 0 ? 200 : 0)
} catch {
flag.stop() // session closed
break
}
guard let ev else { break }
drained += 1
self?.handle(ev)
}
// Announce poll: every 500 ms, or immediately after app activation (§5.2).
let now = Date()
if now.timeIntervalSince(lastAnnounceCheck) >= 0.5
|| self?.checkNow.takeIfRaised() == true
{
lastAnnounceCheck = now
self?.announceIfChanged()
}
}
self?.drainDone.signal()
}
thread.name = "punktfunk-clipboard"
thread.qualityOfService = .utility
thread.start()
}
/// Disable sync and join the drain thread. Called off-main before `connection.close()`
/// (the same discipline as the audio/feedback drains). If the local pasteboard still holds
/// our remote-offer items, they are cleared their providers die with us.
public func stop() {
guard started else { return }
started = false
if let obs = activationObserver {
NotificationCenter.default.removeObserver(obs)
activationObserver = nil
}
connection.clipControl(enabled: false, flags: 0)
flag.stop()
drainDone.wait()
// Fail every paste still blocked on us so no provider thread waits out its timeout.
fetchLock.lock()
for (_, pending) in pendingFetches {
pending.done.signal()
}
pendingFetches.removeAll()
fetchLock.unlock()
let pb = NSPasteboard.general
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
}
}
// MARK: - Local copy host (announce)
/// Announce the local pasteboard's format list when it changed (skipping our own writes and
/// concealed/transient pasteboards). Runs on the drain thread.
private func announceIfChanged() {
let pb = NSPasteboard.general
let count = pb.changeCount
guard count != lastSeenChangeCount else { return }
lastSeenChangeCount = count
if count == ownedChangeCount { return } // our own write (a remote offer) never echo
installedRemoteSeq = nil // a local copy replaced the host's offer
let types = pb.types ?? []
if types.contains(Self.concealed) || types.contains(Self.transient) { return }
offerSeq &+= 1
let kinds = Self.wireToPasteboard
.filter { types.contains($0.type) }
.map { PunktfunkConnection.ClipKind(mime: $0.wire) }
// Empty = the pasteboard holds nothing we sync (or was cleared) clears the host side.
connection.clipOffer(seq: offerSeq, kinds: kinds)
}
// MARK: - Event handling (drain thread)
private func handle(_ ev: PunktfunkConnection.ClipEvent) {
switch ev {
case let .state(enabled, policy, reason):
if let onState {
DispatchQueue.main.async { onState(enabled, policy, reason) }
}
case let .remoteOffer(seq, kinds):
installRemoteOffer(seq: seq, kinds: kinds)
case let .fetchRequest(reqId, seq, _, mime):
serveFetch(reqId: reqId, seq: seq, mime: mime)
case let .data(xferId, chunk, last):
fetchLock.lock()
if var pending = pendingFetches[xferId] {
pending.buffer.append(chunk)
pendingFetches[xferId] = pending
if last {
pendingFetches[xferId]?.done.signal()
}
}
fetchLock.unlock()
case let .cancelled(id), let .error(id, _):
fetchLock.lock()
if var pending = pendingFetches[id] {
pending.failed = true
pendingFetches[id] = pending
pending.done.signal()
}
fetchLock.unlock()
}
}
// MARK: - Host copy local (lazy install + blocked-paste fetch)
/// Write one NSPasteboardItem advertising the host's formats, each backed by a lazy data
/// provider bytes cross only when a Mac app pastes. Empty `kinds` = the host cleared its
/// clipboard: drop our item if it's still current.
private func installRemoteOffer(seq: UInt32, kinds: [PunktfunkConnection.ClipKind]) {
let pb = NSPasteboard.general
let types = kinds.compactMap { kind in
Self.wireToPasteboard.first(where: { $0.wire == kind.mime })?.type
}
guard !types.isEmpty else {
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
}
installedRemoteSeq = nil
return
}
let item = NSPasteboardItem()
item.setDataProvider(RemoteOfferProvider(sync: self, seq: seq), forTypes: types)
pb.clearContents()
pb.writeObjects([item])
installedRemoteSeq = seq
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
}
/// Blocked-paste fulfillment: fetch one wire format of host offer `seq` and wait (provider
/// thread) for the drain thread to assemble the chunks. Nil on timeout/cancel/error the
/// paste then provides nothing rather than hanging (§3.4).
///
/// `fetchLock` is held ACROSS the `clipFetch` so the pending entry exists before the drain
/// thread can process the first `.data` event (its `handle` takes `fetchLock` after
/// releasing the connection's clipboard lock no cycle).
fileprivate func fetchBlocking(seq: UInt32, wireMime: String) -> Data? {
fetchLock.lock()
guard let xferId = connection.clipFetch(seq: seq, mime: wireMime) else {
fetchLock.unlock()
return nil
}
pendingFetches[xferId] = PendingFetch()
let done = pendingFetches[xferId]!.done
fetchLock.unlock()
let outcome = done.wait(timeout: .now() + Self.fetchTimeout)
fetchLock.lock()
let pending = pendingFetches.removeValue(forKey: xferId)
fetchLock.unlock()
if outcome == .timedOut {
connection.clipCancel(id: xferId)
return nil
}
guard let pending, !pending.failed else { return nil }
return pending.buffer
}
// MARK: - Host paste of our data (serve)
/// Answer a host paste of our offered data from the live pasteboard. A stale `seq` (the
/// local clipboard changed since that announce) is cancelled never serve mismatched bytes.
private func serveFetch(reqId: UInt32, seq: UInt32, mime: String) {
let pb = NSPasteboard.general
guard seq == offerSeq, pb.changeCount == lastSeenChangeCount,
let type = Self.wireToPasteboard.first(where: { $0.wire == mime })?.type,
let data = pb.data(forType: type)
else {
connection.clipCancel(id: reqId)
return
}
var offset = 0
while offset < data.count {
let end = min(offset + Self.serveChunk, data.count)
connection.clipServe(
reqId: reqId, data: data.subdata(in: offset..<end), last: end == data.count)
offset = end
}
if data.isEmpty {
connection.clipServe(reqId: reqId, data: Data(), last: true)
}
}
}
/// The lazy paste hook: AppKit calls `provideDataForType` only when a Mac app actually pastes;
/// the fetch then blocks this provider thread (never main) until the host's bytes arrive or the
/// timeout provides nothing. One provider per installed remote offer a dead sync (weak) or a
/// superseded offer provides nothing.
private final class RemoteOfferProvider: NSObject, NSPasteboardItemDataProvider {
private weak var sync: ClipboardSync?
private let seq: UInt32
init(sync: ClipboardSync, seq: UInt32) {
self.sync = sync
self.seq = seq
}
func pasteboard(
_ pasteboard: NSPasteboard?, item: NSPasteboardItem,
provideDataForType type: NSPasteboard.PasteboardType
) {
guard let sync,
let wire = wireMime(for: type),
let data = sync.fetchBlocking(seq: seq, wireMime: wire)
else { return }
item.setData(data, forType: type)
}
private func wireMime(for type: NSPasteboard.PasteboardType) -> String? {
switch type {
case .string: return "text/plain;charset=utf-8"
case .rtf: return "text/rtf"
case .html: return "text/html"
case .png: return "image/png"
default: return nil
}
}
}
#endif
@@ -54,12 +54,6 @@ public func pair(
switch rc {
case PUNKTFUNK_STATUS_OK.rawValue: return Data(observed)
case PUNKTFUNK_STATUS_CRYPTO.rawValue: throw PunktfunkClientError.wrongPIN
default:
// A typed host rejection (pairing not armed / rate-limited / armed for another
// device) carries its own reason never report it as a bad PIN or dead network.
if let rejection = HostRejection(status: rc) {
throw PunktfunkClientError.rejected(rejection)
}
throw PunktfunkClientError.status(rc)
default: throw PunktfunkClientError.status(rc)
}
}
@@ -59,26 +59,6 @@ public extension PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_GAMEPAD_AXIS.rawValue, code: axis, x: value, y: 0, flags: pad)
}
/// Declare a pad's controller KIND (`InputKind::GamepadArrival`): `pref` is the
/// `GamepadType` wire byte (Auto=0, Xbox360=1, DualSense=2, XboxOne=3, DualShock4=4,
/// SteamController=5, SteamDeck=6), `pad` the wire index. Sent once when a controller slot
/// opens BEFORE that pad's first input so the host builds a matching virtual device and a
/// session can mix types (pad 0 a DualSense, pad 1 an Xbox pad). The core re-sends it a few
/// times against datagram loss and folds per-pad state behind it; a host that predates the tag
/// ignores it and uses the session-default kind from the handshake. Idempotent on the host.
static func gamepadArrival(pref: UInt32, pad: UInt32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_GAMEPAD_ARRIVAL.rawValue, code: pref, x: 0, y: 0, flags: pad)
}
/// A pad disconnected (`InputKind::GamepadRemove`): `flags` = pad index. The client sends the
/// bare index; the core stamps the per-pad removal seq (`encode_gamepad_remove`) in the shared
/// snapshot seq space and arms a loss-resistant re-send burst, so the host tears the pad's
/// virtual device down and no reordered snapshot can resurrect it. A host that predates the tag
/// ignores it (the pad then lingers until session end the pre-existing behaviour).
static func gamepadRemove(pad: UInt32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_GAMEPAD_REMOVE.rawValue, code: 0, x: 0, y: 0, flags: pad)
}
// Touch (host-side: libei ei_touchscreen on the virtual output). `id` distinguishes
// fingers and is reusable after touchUp; coordinates are absolute pixels on the
// client's touch surface, whose size rides in `flags` so the host can rescale
@@ -11,9 +11,6 @@
// LaunchSpec schema in `crates/punktfunk-host/src/library.rs`.
import Foundation
// `punktfunkDefaultMgmtPort` (and StoredHost/DefaultsKey) now live in PunktfunkShared so the
// dependency-free widget extension can share them; PunktfunkKit re-exports the module.
import PunktfunkShared
/// Cover art URLs (the public Steam CDN for Steam titles, user-supplied for custom entries).
public struct Artwork: Codable, Hashable, Sendable {
@@ -67,6 +64,10 @@ public enum LibraryError: LocalizedError {
}
}
/// The management API's default port adjacent to the GameStream block; matches
/// `mgmt::DEFAULT_PORT` on the host.
public let punktfunkDefaultMgmtPort: UInt16 = 47990
/// Stateless fetcher for a host's library.
public enum LibraryClient {
/// `GET https://<address>:<port>/api/v1/library`, authenticated by **mTLS**: the client
@@ -59,68 +59,6 @@ public enum PunktfunkClientError: Error {
case wrongPIN
case closed
case status(Int32)
/// The host deliberately turned the attempt away and said why (its typed QUIC
/// application close) distinct from `.connectFailed` (unreachable/timeout) so the UI
/// can show the stated reason instead of blaming the network.
case rejected(HostRejection)
}
/// Why a host turned a connect/pair attempt away decoded from the
/// `PUNKTFUNK_STATUS_REJECTED_*` block. Lets the UI say "approve the request on the host"
/// or "pairing isn't armed" instead of a generic "could not connect".
public enum HostRejection: Sendable {
case pairingNotArmed
case pairingBoundToOtherDevice
case pairingRateLimited
case identityRequired
case denied
case approvalTimeout
case superseded
case wireVersionMismatch
case busy
init?(status: Int32) {
switch status {
case PUNKTFUNK_STATUS_REJECTED_NOT_ARMED.rawValue: self = .pairingNotArmed
case PUNKTFUNK_STATUS_REJECTED_BOUND_OTHER.rawValue: self = .pairingBoundToOtherDevice
case PUNKTFUNK_STATUS_REJECTED_RATE_LIMITED.rawValue: self = .pairingRateLimited
case PUNKTFUNK_STATUS_REJECTED_IDENTITY_REQUIRED.rawValue: self = .identityRequired
case PUNKTFUNK_STATUS_REJECTED_DENIED.rawValue: self = .denied
case PUNKTFUNK_STATUS_REJECTED_APPROVAL_TIMEOUT.rawValue: self = .approvalTimeout
case PUNKTFUNK_STATUS_REJECTED_SUPERSEDED.rawValue: self = .superseded
case PUNKTFUNK_STATUS_REJECTED_WIRE_VERSION.rawValue: self = .wireVersionMismatch
case PUNKTFUNK_STATUS_REJECTED_BUSY.rawValue: self = .busy
default: return nil
}
}
/// User-facing sentence wording shared with the desktop clients.
public var userMessage: String {
switch self {
case .pairingNotArmed:
return "Pairing isn't armed on the host — arm it on the host's Pairing page, "
+ "then try again."
case .pairingBoundToOtherDevice:
return "The host's pairing window is armed for a different device — arm it "
+ "for this one."
case .pairingRateLimited:
return "Too many pairing attempts — wait a couple of seconds and try again."
case .identityRequired:
return "The host requires pairing — pair this device (PIN or request access) first."
case .denied:
return "The host declined this device's request."
case .approvalTimeout:
return "Nobody approved the request on the host in time — approve this device "
+ "in the host's console or web UI, then request access again."
case .superseded:
return "A newer request from this device replaced this one — approve the "
+ "latest request on the host."
case .wireVersionMismatch:
return "Client and host versions don't match — update both to the same release."
case .busy:
return "The host is busy with another session."
}
}
}
/// `withCString` over an optional nil maps to a NULL C pointer.
@@ -196,10 +134,6 @@ public final class PunktfunkConnection {
/// Same role for the host-timing (0xCF) puller its own plane in the core, drained
/// non-blockingly by the app's 1 s stats tick (never contends with the blocking pullers).
private let statsLock = NSLock()
/// Same role for the shared-clipboard drain thread (`nextClipboard` its own plane in the
/// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too:
/// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple.
private let clipboardLock = NSLock()
/// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0
@@ -254,19 +188,6 @@ public final class PunktfunkConnection {
// exist so the resolved type round-trips and name parsing matches the host.
case steamController = 5
case steamDeck = 6
/// DualSense Edge (Linux UHID / Windows UMDF hosts): the DualSense plus native back/Fn
/// buttons. GameController exposes the Edge as a `GCDualSenseGamepad` with its own
/// product category; paddle CAPTURE is still gated on G22, but the declared identity +
/// rich planes match the physical pad.
case dualSenseEdge = 7
/// Nintendo Switch Pro Controller (Linux UHID hid-nintendo hosts): correct Nintendo
/// glyphs + positional layout on the host side.
case switchPro = 8
/// New Steam Controller (2026, `28DE:1302`), passed through as-is on Linux hosts (raw
/// report mirroring; Steam Input is the consumer). Parity only on Apple GameController
/// never surfaces the raw Valve device, so the client can't capture one; exists so the
/// resolved type round-trips and name parsing matches the host.
case steamController2 = 9
/// Loose name parsing for env/dev hooks, mirroring the host's
/// `GamepadPref::from_name`.
@@ -279,11 +200,6 @@ public final class PunktfunkConnection {
case "dualshock4", "dualshock", "ds4", "ps4": self = .dualShock4
case "steamdeck", "steam-deck", "deck": self = .steamDeck
case "steamcontroller", "steam-controller", "steamcon": self = .steamController
case "steamcontroller2", "steam-controller-2", "steamcon2", "sc2", "ibex":
self = .steamController2
case "dualsenseedge", "dualsense-edge", "edge", "dsedge": self = .dualSenseEdge
case "switchpro", "switch-pro", "switch", "procontroller", "pro-controller":
self = .switchPro
default: return nil
}
}
@@ -341,25 +257,9 @@ public final class PunktfunkConnection {
public private(set) var resolvedAudioChannels: UInt8 = 2
/// The video codec the host resolved for this session (`Welcome.codec`, `PUNKTFUNK_CODEC_*`):
/// `2` = HEVC (default / older host), `1` = H.264, `4` = AV1, `8` = PyroWave (only when this
/// client opted in). Build the decoder from THIS. The resolved value honors the client's
/// `preferredCodec` when the host could emit it.
/// `2` = HEVC (default / older host), `1` = H.264, `4` = AV1. Build the decoder from THIS. The
/// resolved value honors the client's `preferredCodec` when the host could emit it.
public private(set) var resolvedCodec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
/// The session's negotiated wire shard payload (`Welcome.shard_payload`, bytes) the
/// parse-window size for `USER_FLAG_CHUNK_ALIGNED` PyroWave AUs (plan §4.4). Other codecs
/// never need it.
public private(set) var shardPayload: UInt32 = 1408
/// The host capability bitfield (`Welcome.host_caps`): `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` /
/// `PUNKTFUNK_HOST_CAP_CLIPBOARD`. `0` for an older host that didn't say.
public private(set) var hostCaps: UInt8 = 0
/// Whether this host advertises the shared clipboard (`HOST_CAP_CLIPBOARD`) the gate for
/// offering the clipboard toggle. Absent on an older host, or one whose operator policy
/// (`PUNKTFUNK_CLIPBOARD=off`) keeps the feature dark.
public var hostSupportsClipboard: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0
}
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
@@ -401,10 +301,6 @@ public final class PunktfunkConnection {
) throws {
if let pin = pinSHA256, pin.count != 32 { throw PunktfunkClientError.invalidPin }
var observed = [UInt8](repeating: 0, count: 32)
// Why a failed connect failed (PunktfunkStatus): lets a typed host rejection
// ("denied in the console", "approval timed out", "host busy") surface as
// `.rejected` instead of the undifferentiated `.connectFailed`.
var connectStatus: Int32 = 0
// `videoCaps` advertises decode/present capability (PUNKTFUNK_VIDEO_CAP_10BIT | _HDR): the
// host upgrades to a 10-bit / BT.2020 PQ stream only when set. 0 = 8-bit BT.709 SDR.
// `launchID` (a host library id like "steam:570") asks the host to launch that title in
@@ -415,29 +311,24 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in
if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in
punktfunk_connect_ex8(
punktfunk_connect_ex7(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs, &connectStatus)
cert, key, timeoutMs)
}
}
return punktfunk_connect_ex8(
return punktfunk_connect_ex7(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch,
nil, &observed, cert, key, timeoutMs, &connectStatus)
nil, &observed, cert, key, timeoutMs)
}
}
}
}
guard handle != nil else {
if let rejection = HostRejection(status: connectStatus) {
throw PunktfunkClientError.rejected(rejection)
}
throw PunktfunkClientError.connectFailed
}
guard handle != nil else { throw PunktfunkClientError.connectFailed }
hostFingerprint = Data(observed)
var w: UInt32 = 0, h: UInt32 = 0, hz: UInt32 = 0
_ = punktfunk_connection_mode(handle, &w, &h, &hz)
@@ -472,12 +363,6 @@ public final class PunktfunkConnection {
var codec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
_ = punktfunk_connection_codec(handle, &codec)
resolvedCodec = codec
var shard: UInt32 = 1408
_ = punktfunk_connection_shard_payload(handle, &shard)
shardPayload = shard
var caps: UInt8 = 0
_ = punktfunk_connection_host_caps(handle, &caps)
hostCaps = caps
}
/// A bandwidth speed-test measurement (see `startSpeedTest`). Partial until `done`.
@@ -551,23 +436,6 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_request_keyframe(h)
}
/// Background-keep-alive video drop (opt-in). While true, both video pumps keep DRAINING
/// `nextAU()` (so QUIC flow control and host pacing stay healthy) but DISCARD each AU before any
/// VideoToolbox/Metal decode or render the crash/jetsam-safe way to hold a backgrounded
/// session (audio keeps rendering; no GPU work off-screen). Set on `SessionModel.enterBackground`,
/// cleared on `exitBackground` (which then requests a fresh IDR; the pump's re-anchor gate
/// auto-arms on the resumed frame-index gap). Its own tiny lock read on the pump thread every
/// iteration, written on the main actor; never contends the ABI/plane locks.
private let videoDropLock = NSLock()
private var videoDropped = false
public var isVideoDropped: Bool {
videoDropLock.lock(); defer { videoDropLock.unlock() }
return videoDropped
}
public func setVideoDropped(_ dropped: Bool) {
videoDropLock.lock(); videoDropped = dropped; videoDropLock.unlock()
}
/// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a
/// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap
/// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a
@@ -582,21 +450,6 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_note_frame_index(h, frameIndex, nil)
}
/// Like `noteFrameIndex`, but also reports whether the core saw a FORWARD frame-index gap the
/// signal that intervening frames were lost and the following AUs reference a picture that never
/// arrived. The post-loss re-anchor gate arms its display freeze on a gap (the earliest, most
/// precise loss trigger ahead of the `framesDropped` climb). Same core side effect as
/// `noteFrameIndex` (the throttled RFI request); call it for every received AU. Returns false
/// after close.
public func noteFrameIndexGap(_ frameIndex: UInt32) -> Bool {
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return false }
var gap = false
_ = punktfunk_connection_note_frame_index(h, frameIndex, &gap)
return gap
}
/// Cumulative access units the hostclient reassembler dropped as unrecoverable (FEC couldn't
/// rebuild them). The video pump polls this and calls `requestKeyframe()` when it climbs the
/// correct loss trigger under the host's infinite GOP, where unrecoverable loss yields
@@ -612,30 +465,6 @@ public final class PunktfunkConnection {
return out
}
/// Report one decoded frame's decode-stage latency, in microseconds (the AU leaving `nextAU`
/// through its VideoToolbox output). This feeds the Automatic bitrate controller's decode
/// signal the only one that sees this device's decoder so the rate is capped at the real
/// decode limit instead of climbing to the network link ceiling and choking the decoder. Cheap;
/// silently dropped after close. Only worth calling when `wantsDecodeLatency()` is true.
public func reportDecodeUs(_ us: UInt32) {
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
_ = punktfunk_connection_report_decode_us(h, us)
}
/// Whether `reportDecodeUs` is worth calling this session: true only when the adaptive-bitrate
/// controller is armed (Automatic bitrate, non-PyroWave). Query once constant for the session
/// and skip the per-frame decode measurement entirely when it's false. False after close.
public func wantsDecodeLatency() -> Bool {
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return false }
var out = false
_ = punktfunk_connection_wants_decode_latency(h, &out)
return out
}
/// The currently active session mode (updated by accepted `requestMode` switches).
public func currentMode() -> (width: UInt32, height: UInt32, refreshHz: UInt32) {
abiLock.lock()
@@ -794,34 +623,6 @@ public final class PunktfunkConnection {
}
}
/// Pull the next EFFECTIVE rumble command from the core's shared rumble policy engine the
/// uniform replacement for per-platform rumble policy. The engine owns every decision
/// (v2 lease expiry, legacy-host staleness at a uniform 1 s, connection-close drain zeros),
/// so apply commands verbatim: `(0, 0)` = stop now, non-zero = run at this level.
/// `backstopMs` is a safety-net duration for duration-parameterized platform APIs the
/// CoreHaptics renderer ignores it (its finite segment ceiling is the equivalent net).
/// Drain from the (single) feedback thread, alongside `nextHidOutput`.
public func nextRumbleCommand(timeoutMs: UInt32 = 0) throws
-> (pad: UInt16, low: UInt16, high: UInt16, backstopMs: UInt32)?
{
feedbackLock.lock()
defer { feedbackLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var pad: UInt16 = 0, low: UInt16 = 0, high: UInt16 = 0, backstop: UInt32 = 0
let rc = punktfunk_connection_next_rumble_cmd(h, &pad, &low, &high, &backstop, timeoutMs)
switch rc {
case statusOK:
return (pad, low, high, backstop)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// One DualSense feedback event a game wrote to the host's virtual pad replay it on
/// the real controller (GCDeviceLight, GCControllerPlayerIndex,
/// GCDualSenseAdaptiveTrigger). Only a `.dualSense` session emits these.
@@ -885,15 +686,6 @@ public final class PunktfunkConnection {
public static let codecH264: UInt8 = UInt8(PUNKTFUNK_CODEC_H264)
public static let codecHEVC: UInt8 = UInt8(PUNKTFUNK_CODEC_HEVC)
public static let codecAV1: UInt8 = UInt8(PUNKTFUNK_CODEC_AV1)
/// PyroWave (opt-in wired-LAN wavelet codec, 8-bit SDR): the host only ever resolves it
/// when the client both advertises the bit AND names it `preferredCodec` never
/// auto-selected. Decoded by the Metal wavelet decoder, not VideoToolbox.
public static let codecPyroWave: UInt8 = UInt8(PUNKTFUNK_CODEC_PYROWAVE)
/// `AccessUnit.flags` bit: the AU is shard-aligned self-delimiting chunks (the wire's
/// `USER_FLAG_CHUNK_ALIGNED`, PyroWave datagram-aligned mode §4.4) walk it
/// window-by-window at `shardPayload`. (The C `#define` doesn't import into Swift.)
public static let userFlagChunkAligned: UInt32 = 64
/// Static HDR mastering metadata (SMPTE ST.2086 + content light level) the host sent for an HDR
/// session. Mirrors the wire/ABI `PunktfunkHdrMeta`; primaries are in ST.2086 **G, B, R** order,
@@ -1032,12 +824,10 @@ public final class PunktfunkConnection {
audioLock.lock()
feedbackLock.lock()
statsLock.lock()
clipboardLock.lock()
abiLock.lock()
let h = handle
handle = nil
abiLock.unlock()
clipboardLock.unlock()
statsLock.unlock()
feedbackLock.unlock()
audioLock.unlock()
@@ -1099,163 +889,6 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_rich_input(h, &rich)
}
// MARK: - Shared clipboard (design/clipboard-and-file-transfer.md §5)
/// One advertised clipboard format in a lazy offer the format list crosses the wire,
/// the bytes only on a fetch.
public struct ClipKind: Sendable, Equatable {
public let mime: String
/// Best-effort size in bytes; `0` = unknown.
public let sizeHint: UInt64
public init(mime: String, sizeHint: UInt64 = 0) {
self.mime = mime
self.sizeHint = sizeHint
}
}
/// A shared-clipboard event from `nextClipboard`. The drain thread turns these into
/// NSPasteboard operations (`ClipboardSync`).
public enum ClipEvent: Sendable, Equatable {
/// The host copied: its lazy format list (empty = the host clipboard was cleared).
/// Fetch a format with `clipFetch(seq:mime:)` when a local app pastes.
case remoteOffer(seq: UInt32, kinds: [ClipKind])
/// Host ack / policy / backend update for `clipControl` (`CLIP_REASON_*`).
case state(enabled: Bool, policy: UInt8, reason: UInt8)
/// The host is pasting OUR offered data answer with `clipServe(reqId:...)`.
case fetchRequest(reqId: UInt32, seq: UInt32, fileIndex: UInt32, mime: String)
/// Bytes for a fetch we started (`last` = final chunk).
case data(xferId: UInt32, chunk: Data, last: Bool)
/// A transfer was cancelled (either side).
case cancelled(id: UInt32)
/// A transfer failed (`status` = a PunktfunkStatus code).
case error(id: UInt32, status: Int32)
}
/// Enable/disable the shared clipboard for this session. Opt-in: nothing is announced or
/// served until enabled. The host answers with a `.state` event carrying the resolved
/// outcome (its operator policy is authoritative). Best-effort a dropped call on a
/// closing session is fine.
public func clipControl(enabled: Bool, flags: UInt8 = 0) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_clipboard_control(h, enabled, flags)
}
/// Announce that the local pasteboard changed the lazy format-list offer (`seq` monotonic,
/// newest wins; empty `kinds` clears the host side). The bytes cross only if the host fetches.
public func clipOffer(seq: UInt32, kinds: [ClipKind]) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
guard !kinds.isEmpty else {
_ = punktfunk_connection_clipboard_offer(h, seq, nil, 0)
return
}
// The C array borrows NUL-terminated strings for the duration of the call only.
let cStrings = kinds.map { strdup($0.mime) }
defer { cStrings.forEach { free($0) } }
let arr = zip(cStrings, kinds).map {
PunktfunkClipKind(mime: $0.map { UnsafePointer($0) }, size_hint: $1.sizeHint)
}
_ = arr.withUnsafeBufferPointer {
punktfunk_connection_clipboard_offer(h, seq, $0.baseAddress, UInt(arr.count))
}
}
/// Start pulling one format of the host's offer `seq` (a local app is pasting). Returns the
/// transfer id echoed on the resulting `.data`/`.error`/`.cancelled` events, or nil when the
/// session is closing.
public func clipFetch(seq: UInt32, mime: String, fileIndex: UInt32 = UInt32.max) -> UInt32? {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return nil }
var xfer: UInt32 = 0
let rc = mime.withCString {
punktfunk_connection_clipboard_fetch(h, seq, $0, fileIndex, &xfer)
}
return rc == statusOK ? xfer : nil
}
/// Provide bytes answering a `.fetchRequest` (the host is pasting our offered data). Call
/// repeatedly to stream; `last = true` completes the transfer. An empty final chunk is fine.
public func clipServe(reqId: UInt32, data: Data, last: Bool) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
if data.isEmpty {
_ = punktfunk_connection_clipboard_serve(h, reqId, nil, 0, last)
} else {
data.withUnsafeBytes { p in
_ = punktfunk_connection_clipboard_serve(
h, reqId, p.bindMemory(to: UInt8.self).baseAddress, UInt(data.count), last)
}
}
}
/// Cancel a clipboard transfer by id an outbound fetch's `xferId` or an inbound
/// `.fetchRequest`'s `reqId`.
public func clipCancel(id: UInt32) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_clipboard_cancel(h, id)
}
/// Pull the next shared-clipboard event; nil on timeout, throws `.closed` once the session
/// ended. Drain from a single dedicated thread (`ClipboardSync`) the event's borrowed
/// payload is copied into the returned `ClipEvent` before the next poll can overwrite it.
public func nextClipboard(timeoutMs: UInt32) throws -> ClipEvent? {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var ev = PunktfunkClipEvent()
let rc = punktfunk_connection_next_clipboard(h, &ev, timeoutMs)
switch rc {
case statusOK:
return Self.decodeClipEvent(ev)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Copy a raw C clip event (whose `data` borrows a per-connection slot) into an owned Swift
/// value. Unknown kinds (a newer core) decode to nil and are skipped by the drain.
private static func decodeClipEvent(_ ev: PunktfunkClipEvent) -> ClipEvent? {
let payload = ev.data.map { Data(bytes: $0, count: Int(ev.len)) } ?? Data()
switch Int32(ev.kind) {
case PUNKTFUNK_CLIP_REMOTE_OFFER:
// One `mime\tsize_hint\n` line per advertised format.
let kinds = String(decoding: payload, as: UTF8.self)
.split(separator: "\n")
.compactMap { line -> ClipKind? in
let parts = line.split(separator: "\t", maxSplits: 1)
guard let mime = parts.first, !mime.isEmpty else { return nil }
let hint = parts.count > 1 ? UInt64(parts[1]) ?? 0 : 0
return ClipKind(mime: String(mime), sizeHint: hint)
}
return .remoteOffer(seq: ev.transfer_id, kinds: kinds)
case PUNKTFUNK_CLIP_STATE:
return .state(enabled: ev.enabled != 0, policy: ev.policy, reason: ev.reason)
case PUNKTFUNK_CLIP_FETCH_REQUEST:
return .fetchRequest(
reqId: ev.transfer_id, seq: ev.seq, fileIndex: ev.file_index,
mime: String(decoding: payload, as: UTF8.self))
case PUNKTFUNK_CLIP_DATA:
return .data(xferId: ev.transfer_id, chunk: payload, last: ev.last != 0)
case PUNKTFUNK_CLIP_CANCELLED:
return .cancelled(id: ev.transfer_id)
case PUNKTFUNK_CLIP_ERROR:
return .error(id: ev.transfer_id, status: ev.status)
default:
return nil
}
}
deinit { close() }
/// Snapshot the handle unless close is pending (callers hold their plane lock).
@@ -1,33 +1,24 @@
// Gamepad capture punktfunk/1 datagrams. Forwards EVERY controller GamepadManager selected
// each on its own stable wire pad index (pf-client-core's slot model) for the lifetime of a
// streaming session. One physical controller with no pin is player 0 (byte-identical to the old
// single-pad path); a pin forwards only that one, also as pad 0.
// Gamepad capture punktfunk/1 datagrams. Forwards exactly ONE controller whatever
// GamepadManager selected as pad 0, for the lifetime of a streaming session.
//
// Each forwarded controller gets a `Slot`: its open GC handlers plus the wire state (buttons,
// axes, touchpad fingers, motion throttle) for its pad index isolated per device so two
// controllers never clobber each other. On connect a slot opens (GamepadArrival declares its
// kind, then input flows); on disconnect / pin change / stop it closes (held state flushed to
// rest on the wire, then GamepadRemove tells the host to tear the pad's virtual device down).
//
// The wire is incremental (one button/axis transition per 18-byte event, accumulated host-side
// into the virtual pad see punktfunk_core::input::gamepad), so we snapshot the full
// GCExtendedGamepad state on every valueChanged and diff against the previous snapshot. Sticks
// are ±32767 with +y = up (GC already matches, no flip), triggers 0...255. The core folds these
// per-pad transitions into idempotent, sequence-numbered snapshots keyed on the same pad index,
// so all this layer must get right is the index one controller per slot, one slot per index.
// The wire is incremental (one button/axis transition per 18-byte event, accumulated
// host-side into the virtual pad see punktfunk_core::input::gamepad), so we snapshot the
// full GCExtendedGamepad state on every valueChanged and diff against the previous
// snapshot. Sticks are ±32767 with +y = up (GC already matches, no flip), triggers 0...255.
//
// PlayStation-pad extras ride the rich-input plane (0xCC): touchpad contacts normalized
// 0...65535 (origin top-left, +y down GC's ±1/+y-up is converted here) and motion samples in
// raw DualSense sensor units (gyro 20 LSB per deg/s, accel 10000 LSB per g derived from the
// host's fixed calibration blob; the conversion lives in ONE place, `Wire`, so a live sign/scale
// correction is a one-line change). The host ignores both unless a pad's virtual device is a
// DualSense or DualShock 4 both carry a touchpad and motion, so the capture below covers either
// (`GCDualShockGamepad` exposes the same `touchpad*` surface as `GCDualSenseGamepad`).
// 0...65535 (origin top-left, +y down GC's ±1/+y-up is converted here) and motion
// samples in raw DualSense sensor units (gyro 20 LSB per deg/s, accel 10000 LSB per g
// derived from the host's fixed calibration blob; the conversion lives in ONE place,
// `Wire`, so a live sign/scale correction is a one-line change). The host ignores both
// unless the session's virtual pad is a DualSense or DualShock 4 both carry a touchpad
// and motion, so the capture below covers either (`GCDualShockGamepad` exposes the same
// `touchpad*` surface as `GCDualSenseGamepad`).
//
// Unlike mouse/keyboard capture, gamepad forwarding is NOT gated on the mouse-capture toggle a
// controller can't click local UI, so it always drives the host while the app is active. On
// deactivation, controller switch, or stop, every held control is released on the wire (the host
// pad would otherwise stay stuck on the last state).
// Unlike mouse/keyboard capture, gamepad forwarding is NOT gated on the mouse-capture
// toggle a controller can't click local UI, so it always drives the host while the app
// is active. On deactivation, controller switch, or stop, every held control is released
// on the wire (the host pad would otherwise stay stuck on the last state).
#if os(macOS)
import AppKit
@@ -42,35 +33,17 @@ import GameController
public final class GamepadCapture {
private let connection: PunktfunkConnection
private let manager: GamepadManager
private var forwardedSub: AnyCancellable?
private var activeSub: AnyCancellable?
private var observers: [NSObjectProtocol] = []
private var bound: GCController?
/// App inactive GC stops delivering; everything is released and stays silent.
private var suspended = false
/// One forwarded controller: the open device plus the last wire state for its pad index (the
/// diff base also what `flush` unwinds). Held per Slot so two controllers never clobber each
/// other's held buttons/axes/fingers. Mirrors pf-client-core's `Slot`.
private final class Slot {
let controller: GCController
/// Wire pad index (GamepadManager's stable lowest-free assignment), threaded onto every
/// event this controller sends the low byte of `flags`.
let pad: UInt32
/// The controller KIND declared to the host (GamepadArrival) when the slot opened.
let pref: PunktfunkConnection.GamepadType
var buttons: UInt32 = 0
var axes: [Int32] = [0, 0, 0, 0, 0, 0]
var fingerActive: [Bool] = [false, false]
var lastMotionNs: UInt64 = 0
init(controller: GCController, pad: UInt32, pref: PunktfunkConnection.GamepadType) {
self.controller = controller
self.pad = pad
self.pref = pref
}
}
/// Open forwarded controllers, one Slot per physical pad on its own wire index. Reconciled
/// against `manager.forwarded` (empty until a session's `start`, cleared by `stop`).
private var slots: [Slot] = []
// Last wire state (the diff base also what releaseAll() unwinds).
private var buttons: UInt32 = 0
private var axes: [Int32] = [0, 0, 0, 0, 0, 0]
private var fingerActive: [Bool] = [false, false]
private var lastMotionNs: UInt64 = 0
/// Motion forwarding floor: 4 ms between samples ( 250 Hz, the DualSense's own rate).
private static let motionIntervalNs: UInt64 = 4_000_000
@@ -98,14 +71,10 @@ public final class GamepadCapture {
}
public func start() {
// Session-scoped index assignment: a controller pinned before the session forwards as
// pad 0 (pf-client-core assigns indices at slot-open time, not app-launch time).
manager.resetForwardingAssignment()
// Fires immediately with the current forwarded set, then on every change a connect,
// disconnect, or pin change reconciles the open slots against it (opening/closing devices
// and flushing wire state so nothing sticks down).
forwardedSub = manager.$forwarded.sink { [weak self] list in
MainActor.assumeIsolated { self?.reconcile(list) }
// Fires immediately with the current selection, then on every change a switch
// releases the old controller's wire state before the new one takes over.
activeSub = manager.$active.sink { [weak self] dc in
MainActor.assumeIsolated { self?.rebind(to: dc?.controller) }
}
#if os(macOS)
let resign = NSApplication.willResignActiveNotification
@@ -128,56 +97,53 @@ public final class GamepadCapture {
MainActor.assumeIsolated {
guard let self else { return }
self.suspended = false
// Re-send every open pad's current state (GC delivered nothing while inactive).
for slot in self.slots {
if let ext = slot.controller.extendedGamepad { self.sync(slot, ext) }
}
if let ext = self.bound?.extendedGamepad { self.sync(ext) }
}
})
}
public func stop() {
closeAllSlots()
forwardedSub = nil
releaseAll()
rebind(to: nil)
activeSub = nil
observers.forEach { NotificationCenter.default.removeObserver($0) }
observers.removeAll()
}
/// Bring `slots` in line with the forwarded set: close any slot no longer wanted (flushing its
/// held wire state and sending GamepadRemove first) and open any newly-forwarded controller into
/// its assigned wire index. A controller that stays forwarded keeps its slot untouched, so a
/// second pad connecting never disturbs the first. Mirrors pf-client-core's `reconcile_slots`.
private func reconcile(_ forwarded: [GamepadManager.DiscoveredController]) {
let wantIDs = Set(forwarded.map { ObjectIdentifier($0.controller) })
for slot in slots where !wantIDs.contains(ObjectIdentifier(slot.controller)) {
closeSlot(slot)
private func rebind(to controller: GCController?) {
guard controller !== bound else { return }
releaseAll()
if let ext = bound?.extendedGamepad {
ext.valueChangedHandler = nil
let tp = Self.touchpad(ext)
tp?.primary.valueChangedHandler = nil
tp?.secondary.valueChangedHandler = nil
}
for dc in forwarded where !slots.contains(where: { $0.controller === dc.controller }) {
openSlot(dc)
// Hand the system gestures back to the OS before letting the old pad go outside a
// stream the share button's screenshot and the Home overlay are the user's, not ours.
if let old = bound {
for element in old.physicalInputProfile.elements.values {
element.preferredSystemGestureState = .enabled
}
}
// A chord-holding pad may have just unplugged re-evaluate so a stale hold disarms.
updateEscapeChord()
}
if let motion = bound?.motion {
motion.valueChangedHandler = nil
// Power the sensors back down left active they keep the pad streaming
// gyro/accel over Bluetooth (battery drain) long after the session.
if motion.sensorsRequireManualActivation { motion.sensorsActive = false }
}
bound = controller
guard let c = controller, let ext = c.extendedGamepad else { return }
/// Open one forwarded controller on its assigned wire index: attach GC handlers, claim its
/// system gestures, declare its kind (GamepadArrival before any input), then wake the host
/// pad and send its initial state. Skipped when the pad has no wire index (every slot taken)
/// or exposes no extended profile.
private func openSlot(_ dc: GamepadManager.DiscoveredController) {
guard let pad = manager.padIndex(for: dc), let ext = dc.controller.extendedGamepad else { return }
let c = dc.controller
let slot = Slot(controller: c, pad: UInt32(pad), pref: dc.kind)
slots.append(slot)
ext.valueChangedHandler = { [weak self, weak slot] g, _ in
MainActor.assumeIsolated { if let self, let slot { self.sync(slot, g) } }
ext.valueChangedHandler = { [weak self] g, _ in
MainActor.assumeIsolated { self?.sync(g) }
}
// Claim EVERY element's system gesture while this pad drives a stream. The OS attaches
// gestures to several controller buttons share/create local screenshot/recording,
// Home Game Center overlay (iOS) / Launchpad's Games folder (macOS) and with a
// gesture attached the press is the system's, not the game's. During capture the remote
// session IS the game: the share button must reach the host (e.g. Steam screenshots),
// the PS button must open the host's Steam overlay. Restored to .enabled on close.
// the PS button must open the host's Steam overlay. Restored to .enabled on unbind.
for element in c.physicalInputProfile.elements.values {
element.preferredSystemGestureState = .disabled
}
@@ -187,114 +153,67 @@ public final class GamepadCapture {
// `extendedGamepad.buttonHome` is unreliable/often nil even when the physical element
// exists. On tvOS the element is absent (reserved) nil, the whole block no-ops.
if let home = c.physicalInputProfile.buttons[GCInputButtonHome] {
home.pressedChangedHandler = { [weak self, weak slot] _, _, pressed in
MainActor.assumeIsolated { if let self, let slot { self.sendGuide(slot, down: pressed) } }
home.pressedChangedHandler = { [weak self] _, _, pressed in
MainActor.assumeIsolated { self?.sendGuide(down: pressed) }
}
}
// Declare this pad's controller KIND before any of its input, so the host builds a
// matching virtual device (mixed types pad 0 a DualSense, pad 1 an Xbox pad). The core
// re-sends it a few times against datagram loss; an older host ignores it and uses the
// session-default kind. Then wake the host pad (pads are created lazily from the first
// event; a DualSense's UHID handshake + initial lightbar write only start then).
connection.send(.gamepadArrival(pref: slot.pref.rawValue, pad: slot.pad))
connection.send(.gamepadAxis(GamepadWire.axisLSX, value: 0, pad: slot.pad))
sync(slot, ext)
// Wake the host pad immediately (pads are created lazily from the first event;
// a DualSense's UHID handshake + initial lightbar write only start then).
connection.send(.gamepadAxis(GamepadWire.axisLSX, value: 0, pad: 0))
sync(ext)
if let tp = Self.touchpad(ext) {
tp.primary.valueChangedHandler = { [weak self, weak slot] _, x, y in
MainActor.assumeIsolated { if let self, let slot { self.touch(slot, finger: 0, x: x, y: y) } }
tp.primary.valueChangedHandler = { [weak self] _, x, y in
MainActor.assumeIsolated { self?.touch(finger: 0, x: x, y: y) }
}
tp.secondary.valueChangedHandler = { [weak self, weak slot] _, x, y in
MainActor.assumeIsolated { if let self, let slot { self.touch(slot, finger: 1, x: x, y: y) } }
tp.secondary.valueChangedHandler = { [weak self] _, x, y in
MainActor.assumeIsolated { self?.touch(finger: 1, x: x, y: y) }
}
}
if let motion = c.motion {
if motion.sensorsRequireManualActivation { motion.sensorsActive = true }
motion.valueChangedHandler = { [weak self, weak slot] m in
MainActor.assumeIsolated { if let self, let slot { self.forwardMotion(slot, m) } }
motion.valueChangedHandler = { [weak self] m in
MainActor.assumeIsolated { self?.forwardMotion(m) }
}
}
}
/// Flush a slot's held wire state (so nothing sticks down host-side) and signal the host to tear
/// its virtual device down (GamepadRemove), then detach GC handlers, hand the system gestures
/// back, and power the sensors down. Wire-only until the GC cleanup, so it is safe even when the
/// device already physically unplugged. Mirrors pf-client-core's `close_slot_at`.
private func closeSlot(_ slot: Slot) {
flush(slot)
// Sent after the flush so the core stamps it with a seq past the zeroing snapshots; the host
// seq-gates it, so a reordered snapshot can't resurrect the removed pad.
connection.send(.gamepadRemove(pad: slot.pad))
let c = slot.controller
if let ext = c.extendedGamepad {
ext.valueChangedHandler = nil
let tp = Self.touchpad(ext)
tp?.primary.valueChangedHandler = nil
tp?.secondary.valueChangedHandler = nil
}
c.physicalInputProfile.buttons[GCInputButtonHome]?.pressedChangedHandler = nil
// Hand the system gestures back to the OS before letting the pad go outside a stream the
// share button's screenshot and the Home overlay are the user's, not ours.
for element in c.physicalInputProfile.elements.values {
element.preferredSystemGestureState = .enabled
}
if let motion = c.motion {
motion.valueChangedHandler = nil
// Power the sensors back down left active they keep the pad streaming gyro/accel
// over Bluetooth (battery drain) long after the session.
if motion.sensorsRequireManualActivation { motion.sensorsActive = false }
}
slots.removeAll { $0 === slot }
}
private func closeAllSlots() {
while let slot = slots.first { closeSlot(slot) }
chordTimer?.invalidate()
chordTimer = nil
}
/// Snapshot the profile into a slot's wire state and send every transition since the last one,
/// tagged with the slot's wire pad index.
private func sync(_ slot: Slot, _ g: GCExtendedGamepad) {
/// Snapshot the profile into wire state and send every transition since the last one.
private func sync(_ g: GCExtendedGamepad) {
guard !suspended else { return }
// guide is driven separately (`sendGuide`, off the Home handler) and deliberately kept out
// of `buttonMask`. Preserve its current held state here so the XOR diff below never sees it
// as "changed" otherwise the first stick/button move after a guide press would emit a
// spurious guide-UP while the button is still physically held (and drop the bit from
// `slot.buttons`, swallowing the real release too). `flush`/`allButtons` still release it.
let newButtons = Self.buttonMask(g) | (slot.buttons & GamepadWire.guide)
let changed = newButtons ^ slot.buttons
let newButtons = Self.buttonMask(g)
updateEscapeChord(newButtons)
let changed = newButtons ^ buttons
if changed != 0 {
for bit in GamepadWire.allButtons where changed & bit != 0 {
connection.send(.gamepadButton(bit, down: newButtons & bit != 0, pad: slot.pad))
connection.send(.gamepadButton(bit, down: newButtons & bit != 0, pad: 0))
}
slot.buttons = newButtons
buttons = newButtons
}
let newAxes: [Int32] = [
Int32(g.leftThumbstick.xAxis.value * 32767),
Int32(g.leftThumbstick.yAxis.value * 32767),
Int32(g.rightThumbstick.xAxis.value * 32767),
Int32(g.rightThumbstick.yAxis.value * 32767),
Int32(g.leftTrigger.value * 255),
Int32(g.rightTrigger.value * 255),
Int32((g.leftThumbstick.xAxis.value * 32767).rounded()),
Int32((g.leftThumbstick.yAxis.value * 32767).rounded()),
Int32((g.rightThumbstick.xAxis.value * 32767).rounded()),
Int32((g.rightThumbstick.yAxis.value * 32767).rounded()),
Int32((g.leftTrigger.value * 255).rounded()),
Int32((g.rightTrigger.value * 255).rounded()),
]
for (i, v) in newAxes.enumerated() where v != slot.axes[i] {
connection.send(.gamepadAxis(UInt32(i), value: v, pad: slot.pad))
slot.axes[i] = v
for (i, v) in newAxes.enumerated() where v != axes[i] {
connection.send(.gamepadAxis(UInt32(i), value: v, pad: 0))
axes[i] = v
}
updateEscapeChord()
}
/// Forward the guide (Home/PS) transition directly it's kept out of `buttonMask` (the legacy
/// `buttonHome` element is unreliable). Folds into the slot's `buttons` so a held PS button is
/// released by `flush` on focus loss / close just like the others.
private func sendGuide(_ slot: Slot, down: Bool) {
/// `buttonHome` element is unreliable). Folds into `buttons` so a held PS button is released by
/// `releaseAll` on focus loss just like the others.
private func sendGuide(down: Bool) {
guard !suspended else { return }
let bit = GamepadWire.guide
let now = down ? (slot.buttons | bit) : (slot.buttons & ~bit)
guard now != slot.buttons else { return }
connection.send(.gamepadButton(bit, down: down, pad: slot.pad))
slot.buttons = now
let now = down ? (buttons | bit) : (buttons & ~bit)
guard now != buttons else { return }
connection.send(.gamepadButton(bit, down: down, pad: 0))
buttons = now
}
private static func buttonMask(_ g: GCExtendedGamepad) -> UInt32 {
@@ -305,21 +224,17 @@ public final class GamepadCapture {
if g.dpad.right.isPressed { b |= GamepadWire.dpadRight }
if g.buttonMenu.isPressed { b |= GamepadWire.start }
if g.buttonOptions?.isPressed == true { b |= GamepadWire.back }
// The dedicated share/create/capture element (Xbox-Series Share, DualSense Create, a clone
// pad's screenshot button e.g. the GameSir G8's, below its d-pad) the wire's capture
// bit, matching the Rust client's `Button::Misc1 => wire::BTN_MISC1`. On an Xbox-Series pad
// this is a button physically DISTINCT from View (buttonOptions, above), so it must not
// collapse onto back the host reads MISC1 as its own control (DualSense mute / Steam
// quick-access). Caveat: a pad that surfaces ONE physical button as both buttonOptions and
// this share element now emits back+misc1 for it harmless on a plain xpad session (no
// misc button) and rare otherwise. NOTE: on-glass verify on a real Xbox-Series pad.
if g.buttons[GCInputButtonShare]?.isPressed == true { b |= GamepadWire.misc1 }
// The share/create/capture element (Xbox Series share, a clone pad's screenshot button
// e.g. the GameSir G8's, below its d-pad) folds into back/select too. On pads that expose
// the create button BOTH as buttonOptions and as the share element this OR is harmless
// same wire bit.
if g.buttons[GCInputButtonShare]?.isPressed == true { b |= GamepadWire.back }
if g.leftThumbstickButton?.isPressed == true { b |= GamepadWire.leftStickClick }
if g.rightThumbstickButton?.isPressed == true { b |= GamepadWire.rightStickClick }
if g.leftShoulder.isPressed { b |= GamepadWire.leftShoulder }
if g.rightShoulder.isPressed { b |= GamepadWire.rightShoulder }
// guide (Home/PS) is NOT read here it's forwarded directly by the Home button's
// pressedChangedHandler (the legacy `buttonHome` element is unreliable). See `openSlot`.
// pressedChangedHandler (the legacy `buttonHome` element is unreliable). See `rebind`.
if g.buttonA.isPressed { b |= GamepadWire.a }
if g.buttonB.isPressed { b |= GamepadWire.b }
if g.buttonX.isPressed { b |= GamepadWire.x }
@@ -347,29 +262,29 @@ public final class GamepadCapture {
return nil
}
/// One touchpad finger moved on a slot's pad. GC reports ±1 positions and snaps to exactly
/// (0, 0) on lift treated as the lift signal (a real finger landing on the precise center
/// One touchpad finger moved. GC reports ±1 positions and snaps to exactly (0, 0) on
/// lift treated as the lift signal (a real finger landing on the precise center
/// momentarily reads as a lift; harmless for a 1-in-65k coincidence).
private func touch(_ slot: Slot, finger: Int, x: Float, y: Float) {
private func touch(finger: Int, x: Float, y: Float) {
guard !suspended else { return }
let lifted = x == 0 && y == 0
if lifted {
if slot.fingerActive[finger] {
slot.fingerActive[finger] = false
connection.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(finger), active: false, x: 0, y: 0)
if fingerActive[finger] {
fingerActive[finger] = false
connection.sendTouchpad(finger: UInt8(finger), active: false, x: 0, y: 0)
}
return
}
slot.fingerActive[finger] = true
fingerActive[finger] = true
let w = GamepadWire.touchpad(x: x, y: y)
connection.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(finger), active: true, x: w.x, y: w.y)
connection.sendTouchpad(finger: UInt8(finger), active: true, x: w.x, y: w.y)
}
private func forwardMotion(_ slot: Slot, _ m: GCMotion) {
private func forwardMotion(_ m: GCMotion) {
guard !suspended else { return }
let now = DispatchTime.now().uptimeNanoseconds
guard now &- slot.lastMotionNs >= Self.motionIntervalNs else { return }
slot.lastMotionNs = now
guard now &- lastMotionNs >= Self.motionIntervalNs else { return }
lastMotionNs = now
// Total acceleration in g: gravity + user when split, else the raw vector.
let ax: Float
let ay: Float
@@ -386,7 +301,6 @@ public final class GamepadCapture {
let gs = GamepadWire.gyroLSBPerRadS
let as_ = GamepadWire.accelLSBPerG
connection.sendMotion(
pad: UInt8(slot.pad),
gyro: (
GamepadWire.motionRaw(Float(m.rotationRate.x), scale: gs),
GamepadWire.motionRaw(Float(m.rotationRate.y), scale: gs),
@@ -399,12 +313,13 @@ public final class GamepadCapture {
))
}
/// Arm the disconnect timer when ANY forwarded pad holds the full escape chord, disarm the
/// moment none do a release, or the holding pad unplugged (pf-client-core's `chord_held` is
/// likewise any-slot). GC events only arrive on state CHANGES, so a held chord needs the timer:
/// the handler won't fire again until something moves.
private func updateEscapeChord() {
let held = slots.contains { $0.buttons & Self.escapeChord == Self.escapeChord }
/// Unwind everything held on the wire: button-ups, neutral axes, lifted fingers. The
/// host's virtual pad returns to rest instead of running with the last state.
/// Arm the disconnect timer when the full chord lands, disarm the moment any of the four
/// releases. Events only arrive on state CHANGES, so a held chord needs the timer the
/// handler won't fire again until something moves.
private func updateEscapeChord(_ newButtons: UInt32) {
let held = newButtons & Self.escapeChord == Self.escapeChord
if held, chordTimer == nil {
let timer = Timer(timeInterval: Self.disconnectHold, repeats: false) { [weak self] _ in
Task { @MainActor in self?.onDisconnectRequest?() }
@@ -417,31 +332,20 @@ public final class GamepadCapture {
}
}
/// Unwind everything a slot holds on the wire: button-ups, neutral axes, lifted fingers. The
/// host's virtual pad returns to rest instead of running with the last state. Wire events only
/// (no GC calls) safe against an already-removed device. Does NOT close the slot or send
/// GamepadRemove (that's `closeSlot`).
private func flush(_ slot: Slot) {
for bit in GamepadWire.allButtons where slot.buttons & bit != 0 {
connection.send(.gamepadButton(bit, down: false, pad: slot.pad))
}
slot.buttons = 0
for (i, v) in slot.axes.enumerated() where v != 0 {
connection.send(.gamepadAxis(UInt32(i), value: 0, pad: slot.pad))
slot.axes[i] = 0
}
for (f, active) in slot.fingerActive.enumerated() where active {
connection.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(f), active: false, x: 0, y: 0)
slot.fingerActive[f] = false
}
}
/// Flush every open slot's held state (app deactivation) keeps the slots open (GC just stops
/// delivering; resume re-syncs), disarms the escape chord. Distinct from `closeAllSlots`, which
/// also sends GamepadRemove and detaches handlers.
private func releaseAll() {
chordTimer?.invalidate()
chordTimer = nil
for slot in slots { flush(slot) }
for bit in GamepadWire.allButtons where buttons & bit != 0 {
connection.send(.gamepadButton(bit, down: false, pad: 0))
}
buttons = 0
for (i, v) in axes.enumerated() where v != 0 {
connection.send(.gamepadAxis(UInt32(i), value: 0, pad: 0))
axes[i] = 0
}
for (f, active) in fingerActive.enumerated() where active {
connection.sendTouchpad(finger: UInt8(f), active: false, x: 0, y: 0)
fingerActive[f] = false
}
}
}
@@ -1,83 +1,47 @@
// Hostclient gamepad feedback rendering: one drain thread polls the rumble (0xCA) and
// HID-output (0xCD) planes and replays each update on the forwarded physical controller it is
// ADDRESSED TO by wire pad index
// HID-output (0xCD) planes and replays them on the active physical controller
//
// rumble CHHapticEngine players (per-handle localities when the pad has them,
// one combined engine otherwise), a RumbleRenderer per pad,
// one combined engine otherwise),
// lightbar GCDeviceLight,
// player LEDs GCController.playerIndex (the DS bit patterns map to player 14),
// trigger FX DualSenseTriggerEffect.parse GCDualSenseAdaptiveTrigger.
//
// Every forwarded controller gets a per-pad feedback slot (its RumbleRenderer + last light /
// player-LED / trigger state) keyed on the same wire index GamepadCapture streams it on, so a
// rumble the host aimed at pad 1 drives pad 1's actuator and nothing else. An update for a pad
// with no live slot (one that just closed) is dropped. HID-output traffic exists only on
// PlayStation-pad sessions (a DualSense, or a DualShock 4 = lightbar only); the drain always
// polls both planes with short timeouts and never spins, so an Xbox pad just renders rumble.
// GameController profile mutation happens on main; CHHapticEngine work on the renderer's serial
// queue; the drain thread itself touches neither (it routes rumble to the pad's renderer under a
// lock and hops HID to main). When a controller leaves the forwarded set the old pad is reset
// (triggers off, player index unset) and its renderer silenced.
// Only pad 0 is rendered (exactly one controller is forwarded). HID-output traffic exists
// only on PlayStation-pad sessions (a DualSense, or a DualShock 4 = lightbar only) the
// drain always polls both planes with short timeouts and never spins, so an Xbox session
// just renders rumble. GameController profile mutation
// happens on main; CHHapticEngine work on its own serial queue; the drain thread itself
// touches neither. When GamepadManager switches the active controller mid-session, the
// old pad is reset (triggers off, player index unset) and the last known feedback state
// is replayed onto the new one.
import Combine
import CoreHaptics
import Foundation
import GameController
import PunktfunkShared
public final class GamepadFeedback {
private let connection: PunktfunkConnection
private let manager: GamepadManager
private let flag = StopFlag()
private let drainDone = DispatchSemaphore(value: 0)
private var drainStarted = false
private var forwardedSub: AnyCancellable?
private let rumble = RumbleRenderer(policy: .session)
private var activeSub: AnyCancellable?
/// One forwarded controller's non-rumble feedback state (main-actor) the GC target plus the
/// last applied lightbar / player-LED / trigger, replayed if the controller on this pad swaps.
@MainActor private final class Slot {
var controller: GCController?
var lastLight: (r: UInt8, g: UInt8, b: UInt8)?
var lastPlayerBits: UInt8?
var lastTrigger: [DualSenseTriggerEffect?] = [nil, nil]
init(controller: GCController?) { self.controller = controller }
}
/// HID / lightbar / player-LED slots, keyed by wire pad index. Main-actor only.
@MainActor private var slots: [UInt8: Slot] = [:]
/// Rumble renderers keyed by wire pad index, guarded by `routingLock` so the background drain
/// thread can route an incoming envelope to the right pad's renderer while the main actor
/// reconciles the set. RumbleRenderer serializes on its own queue, so calling `apply` from the
/// drain thread is safe only the map lookup needs the lock.
private let routingLock = NSLock()
private var rumbleByPad: [UInt8: RumbleRenderer] = [:]
/// Opt-in device mirror (`DefaultsKey.rumbleOnDevice`, iPhone only): rumble the host
/// addresses to controller 1 (wire pad 0) is ALSO rendered on this device's own Taptic
/// Engine for phone-clip pads that ship without rumble motors, where the phone body is the
/// only actuator in the player's hands. Session-scoped (the setting is read once here); nil
/// when off or where the device has no haptic actuator.
private let deviceRumble: RumbleRenderer?
// Last applied feedback (main-actor) replayed when the active controller changes.
@MainActor private var target: GCController?
@MainActor private var lastLight: (r: UInt8, g: UInt8, b: UInt8)?
@MainActor private var lastPlayerBits: UInt8?
@MainActor private var lastTrigger: [DualSenseTriggerEffect?] = [nil, nil]
public init(connection: PunktfunkConnection, manager: GamepadManager) {
self.connection = connection
self.manager = manager
#if os(iOS)
if UserDefaults.standard.bool(forKey: DefaultsKey.rumbleOnDevice),
CHHapticEngine.capabilitiesForHardware().supportsHaptics {
deviceRumble = RumbleRenderer(policy: .session, actuator: .device)
} else {
deviceRumble = nil
}
#else
deviceRumble = nil
#endif
// Capture self weakly in the hop too, so the inner sink's weak capture isn't shadowing
// an implicit strong one and the subscription (stored on self) never retain-cycles.
Task { @MainActor [weak self] in
guard let self else { return }
self.forwardedSub = manager.$forwarded.sink { [weak self] list in
MainActor.assumeIsolated { self?.reconcile(list) }
self.activeSub = manager.$active.sink { [weak self] dc in
MainActor.assumeIsolated { self?.retarget(dc?.controller) }
}
}
}
@@ -103,38 +67,6 @@ public final class GamepadFeedback {
}
}
/// Bring the per-pad feedback slots in line with the forwarded set: drop pads no longer
/// forwarded (silence + release their renderer, reset their controller), add a slot +
/// renderer for each new pad, and retarget a pad whose controller changed (a re-plug into the
/// same freed index) replaying its cached feedback onto the new device.
@MainActor
private func reconcile(_ forwarded: [GamepadManager.DiscoveredController]) {
var want: [UInt8: GCController] = [:]
for dc in forwarded {
if let pad = manager.padIndex(for: dc) { want[pad] = dc.controller }
}
for (pad, slot) in slots where want[pad] == nil {
reset(slot.controller)
slots[pad] = nil
let renderer = withRouting { rumbleByPad.removeValue(forKey: pad) }
renderer?.stop()
}
for (pad, controller) in want {
if let slot = slots[pad] {
guard slot.controller !== controller else { continue }
reset(slot.controller)
slot.controller = controller
withRouting { rumbleByPad[pad]?.retarget(controller) }
replay(slot)
} else {
slots[pad] = Slot(controller: controller)
let renderer = RumbleRenderer(policy: .session)
renderer.retarget(controller)
withRouting { rumbleByPad[pad] = renderer }
}
}
}
public func start() {
guard !drainStarted else { return }
drainStarted = true
@@ -155,18 +87,21 @@ public final class GamepadFeedback {
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
//
// Rumble arrives as EFFECTIVE commands from the core's shared policy engine
// (design/rumble-root-fix.md §D): the engine owns leases, legacy staleness,
// and close-drain zeros, and its per-pad mailbox already coalesces a
// stalled drain wakes to ONE current-level command per pad, and a stop can
// never be shed by a queue. Apply verbatim, in order.
// Rumble is idempotent state, so drain the plane DRY and apply only the newest
// level. The old one-datagram-per-cycle shape let a burst outpace the ~125 Hz
// drain: levels rendered up to ~130 ms late through the core's 16-deep queue,
// and its drop-newest overflow could shed a stop while stale nonzero states
// queued ahead of it buzzing until the host's next 500 ms refresh.
var newest: (low: UInt16, high: UInt16, ttl: UInt32)?
var rumbleBurst = 0
while rumbleBurst < 64, !flag.isStopped,
let c = try connection.nextRumbleCommand(timeoutMs: 0) {
self?.routeRumble(
pad: UInt8(truncatingIfNeeded: c.pad), low: c.low, high: c.high)
let r = try connection.nextRumble2(timeoutMs: 0) {
if r.pad == 0 { newest = (r.low, r.high, r.ttlMs) }
rumbleBurst += 1
}
if let n = newest {
self?.rumble.apply(low: n.low, high: n.high, ttlMs: n.ttl)
}
// Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
var burst = 0
@@ -191,7 +126,7 @@ public final class GamepadFeedback {
thread.start()
}
/// Stop the drain and silence every pad's motors. Blocks until the drain thread exits ( one
/// Stop the drain and silence the motors. Blocks until the drain thread exits ( one
/// poll cycle) call off the main actor, before `connection.close()`.
public func stop() {
flag.stop()
@@ -199,39 +134,19 @@ public final class GamepadFeedback {
drainDone.wait()
drainStarted = false
}
let renderers = withRouting { () -> [RumbleRenderer] in
let r = Array(rumbleByPad.values)
rumbleByPad.removeAll()
return r
}
for r in renderers { r.stop() }
deviceRumble?.stop()
// Drop the subscription and every dead pad's cached feedback a controller change after
// teardown must not replay this session's triggers/LEDs.
rumble.stop()
// Drop the retarget subscription and the dead session's cached feedback a
// controller change after teardown must not replay this session's triggers/LEDs.
Task { @MainActor in
self.forwardedSub = nil
for slot in self.slots.values { self.reset(slot.controller) }
self.slots.removeAll()
self.activeSub = nil
self.lastLight = nil
self.lastPlayerBits = nil
self.lastTrigger = [nil, nil]
self.reset(self.target)
self.target = nil
}
}
/// Route one engine command to its pad's renderer (drain thread). A command for a pad with no
/// live renderer one that just left the forwarded set is dropped.
private func routeRumble(pad: UInt8, low: UInt16, high: UInt16) {
let renderer = withRouting { rumbleByPad[pad] }
renderer?.apply(low: low, high: high)
// The opt-in device mirror follows controller 1 unconditionally the pads it exists for
// have no motors (their renderer above no-ops), and mirroring deliberately isn't gated on
// that: capability probing can't see a motor-less MFi pad, and the user opted in.
if pad == 0 { deviceRumble?.apply(low: low, high: high) }
}
private func withRouting<R>(_ body: () -> R) -> R {
routingLock.lock()
defer { routingLock.unlock() }
return body()
}
private func render(_ ev: PunktfunkConnection.HidOutputEvent) {
DispatchQueue.main.async {
MainActor.assumeIsolated { self.apply(ev) }
@@ -242,37 +157,40 @@ public final class GamepadFeedback {
private func apply(_ ev: PunktfunkConnection.HidOutputEvent) {
switch ev {
case let .led(pad, r, g, b):
guard let slot = slots[pad] else { return }
slot.lastLight = (r, g, b)
slot.controller?.light?.color = GCColor(
guard pad == 0 else { return }
lastLight = (r, g, b)
target?.light?.color = GCColor(
red: Float(r) / 255, green: Float(g) / 255, blue: Float(b) / 255)
case let .playerLEDs(pad, bits):
guard let slot = slots[pad] else { return }
slot.lastPlayerBits = bits
slot.controller?.playerIndex = Self.playerIndex(forBits: bits)
guard pad == 0 else { return }
lastPlayerBits = bits
target?.playerIndex = Self.playerIndex(forBits: bits)
case let .triggerEffect(pad, which, effect):
guard which < 2, let slot = slots[pad] else { return }
guard pad == 0, which < 2 else { return }
let parsed = DualSenseTriggerEffect.parse(effect)
slot.lastTrigger[Int(which)] = parsed
if let trigger = adaptiveTrigger(slot.controller, which) {
lastTrigger[Int(which)] = parsed
if let trigger = adaptiveTrigger(which) {
parsed.apply(to: trigger)
}
}
}
/// Replay a pad's cached feedback onto its (swapped-in) controller so a re-plug looks the same.
@MainActor
private func replay(_ slot: Slot) {
if let (r, g, b) = slot.lastLight {
slot.controller?.light?.color = GCColor(
private func retarget(_ controller: GCController?) {
guard controller !== target else { return }
reset(target)
target = controller
rumble.retarget(controller)
// Replay the session's feedback state so a swapped-in controller looks the same.
if let (r, g, b) = lastLight {
controller?.light?.color = GCColor(
red: Float(r) / 255, green: Float(g) / 255, blue: Float(b) / 255)
}
if let bits = slot.lastPlayerBits {
slot.controller?.playerIndex = Self.playerIndex(forBits: bits)
if let bits = lastPlayerBits {
controller?.playerIndex = Self.playerIndex(forBits: bits)
}
for which in 0..<2 {
if let effect = slot.lastTrigger[which],
let trigger = adaptiveTrigger(slot.controller, UInt8(which)) {
if let effect = lastTrigger[which], let trigger = adaptiveTrigger(UInt8(which)) {
effect.apply(to: trigger)
}
}
@@ -289,8 +207,8 @@ public final class GamepadFeedback {
}
@MainActor
private func adaptiveTrigger(_ controller: GCController?, _ which: UInt8) -> GCDualSenseAdaptiveTrigger? {
guard let ds = controller?.extendedGamepad as? GCDualSenseGamepad else { return nil }
private func adaptiveTrigger(_ which: UInt8) -> GCDualSenseAdaptiveTrigger? {
guard let ds = target?.extendedGamepad as? GCDualSenseGamepad else { return nil }
return which == 0 ? ds.leftTrigger : ds.rightTrigger
}
}
@@ -1,18 +1,14 @@
// Controller discovery + selection, app-lifetime. One GamepadManager (`.shared`) watches
// GCController connect/disconnect from launch, so the Settings page shows live controller
// state without a session, and the session components (GamepadCapture / GamepadFeedback)
// follow `forwarded` every forwarded controller is streamed to the host, each on its own
// wire pad index (pf-client-core parity; up to `GamepadWire.maxPads`).
// follow `active` exactly ONE physical controller is forwarded to the host, as pad 0.
//
// Selection (mirrors pf-client-core's `forwarded_ids` + slot model): with no pin, EVERY
// extended controller is forwarded each assigned a stable lowest-free pad index held for
// its forwarded lifetime, so a disconnect frees only its own index and never renumbers the
// others. A pin (Settings, persisted under DefaultsKey.gamepadID) forwards ONLY that one pad
// an explicit single-player choice. `active` stays the single "primary" pad (the pinned
// one, else the most recently connected extended gamepad) that the Settings / launcher / menu
// UI reads. GCController has no stable hardware serial, so the pin is a fingerprint of
// vendorName|productCategory (+ a connect-order suffix for twins); identical twin controllers
// may swap a pin across reconnects, which the Settings footer documents.
// Selection: the user can pin a controller in Settings (persisted under
// DefaultsKey.gamepadID); with no pin or the pinned one absent the most recently
// connected extended gamepad wins. GCController has no stable hardware serial, so the pin
// is a fingerprint of vendorName|productCategory (+ a connect-order suffix for twins);
// identical twin controllers may swap a pin across reconnects, which the Settings footer
// documents.
//
// A singleton (not a SwiftUI environment object) because macOS shows Settings in its own
// `Settings{}` scene there is no common ancestor view to inject from.
@@ -20,7 +16,6 @@
import Combine
import Foundation
import GameController
import PunktfunkShared
@MainActor
public final class GamepadManager: ObservableObject {
@@ -43,14 +38,13 @@ public final class GamepadManager: ObservableObject {
public let hasHaptics: Bool
public let hasMotion: Bool
public let hasAdaptiveTriggers: Bool
/// Specifically a DualSense (incl. the Edge same feedback surface) gates the
/// DualSense-only feedback (adaptive triggers, player LEDs) and the PlayStation glyph
/// in Settings.
public var isDualSense: Bool { kind == .dualSense || kind == .dualSenseEdge }
/// A PlayStation pad with a touchpad + motion (DualSense family OR DualShock 4) gates
/// Specifically a DualSense gates the DualSense-only feedback (adaptive triggers,
/// player LEDs) and the PlayStation glyph in Settings.
public var isDualSense: Bool { kind == .dualSense }
/// A PlayStation pad with a touchpad + motion (DualSense OR DualShock 4) gates
/// rich-input CAPTURE (touchpad contacts + gyro/accel on plane 0xCC).
public var hasTouchpadAndMotion: Bool {
kind == .dualSense || kind == .dualSenseEdge || kind == .dualShock4
kind == .dualSense || kind == .dualShock4
}
/// 0...1, nil when the controller doesn't report a battery (e.g. wired).
public let batteryLevel: Float?
@@ -66,23 +60,9 @@ public final class GamepadManager: ObservableObject {
/// Every detected controller, in connect order (Settings lists these).
@Published public private(set) var controllers: [DiscoveredController] = []
/// The single "primary" controller the pinned one, else the most recently connected
/// extended gamepad; nil when none qualifies. The Settings / launcher / menu UI and the
/// connect-time `resolveType` read this; the streaming input path uses `forwarded`.
/// The one controller forwarded to the host (pad 0); nil when none qualifies.
@Published public private(set) var active: DiscoveredController?
/// The controllers forwarded to the host this session, in wire-pad-index preference order
/// (pf-client-core's `forwarded_ids`): a pin forwards ONLY the pinned pad; Automatic forwards
/// every extended controller. GamepadCapture opens a slot per entry and GamepadFeedback routes
/// feedback back to it, each on the index from `padIndex(for:)`.
@Published public private(set) var forwarded: [DiscoveredController] = []
/// Stable wire pad index (0..<`GamepadWire.maxPads`) per forwarded controller, keyed by
/// GCController identity. Lowest-free, held while the controller stays forwarded a
/// disconnect frees only its own index so the others never renumber (pf-client-core's
/// `lowest_free_index`). Recomputed by `assignPadIndices` whenever `forwarded` changes.
private var padIndexByController: [ObjectIdentifier: UInt8] = [:]
/// The user's pinned controller fingerprint ("" = automatic). Persisted; updating it
/// reselects immediately, so a Settings Picker can bind straight to this.
@Published public var preferredID: String {
@@ -179,57 +159,12 @@ public final class GamepadManager: ObservableObject {
let candidates = controllers.filter(\.isExtended)
// The pin wins when present; otherwise the most recently connected extended pad
// (list is in connect order). A stale pin falls back to automatic.
let pinned = candidates.last { $0.id == preferredID }
active = pinned ?? candidates.last
// Forwarded set (pf-client-core's `forwarded_ids`): a pin forwards ONLY the pinned pad
// (explicit single-player); Automatic forwards every extended controller in connect order
// (oldestnewest), so a game's player numbers are stable across hot-plug churn.
let next = pinned.map { [$0] } ?? candidates
// Update the pad-index assignment BEFORE publishing `forwarded`: @Published emits in
// `willSet`, so GamepadCapture/GamepadFeedback reconcile against `padIndex(for:)` the
// instant this assignment lands a stale map here would skip a newly-forwarded pad.
assignPadIndices(for: next)
forwarded = next
}
/// Assign each forwarded controller a stable wire pad index (lowest-free, held while it stays
/// forwarded) mirrors pf-client-core's slot model, where a disconnect frees only its own
/// index and the others keep theirs. A controller already holding an index keeps it across the
/// churn; a slot beyond `GamepadWire.maxPads` goes unassigned (that pad is not forwarded).
private func assignPadIndices(for next: [DiscoveredController]) {
let live = Set(next.map { ObjectIdentifier($0.controller) })
padIndexByController = padIndexByController.filter { live.contains($0.key) }
for dc in next {
let key = ObjectIdentifier(dc.controller)
guard padIndexByController[key] == nil,
let free = Self.lowestFreeIndex(Set(padIndexByController.values)) else { continue }
padIndexByController[key] = free
}
}
/// The lowest wire pad index not already taken, or nil when all `GamepadWire.maxPads` are in
/// use (pf-client-core's `lowest_free_index`).
private static func lowestFreeIndex(_ taken: Set<UInt8>) -> UInt8? {
(0..<UInt8(GamepadWire.maxPads)).first { !taken.contains($0) }
}
/// The wire pad index a forwarded controller streams on, or nil when it isn't forwarded.
public func padIndex(for controller: DiscoveredController) -> UInt8? {
padIndexByController[ObjectIdentifier(controller.controller)]
}
/// Drop every pad-index assignment and recompute from the current forwarded set called when
/// a streaming session begins so the assignment starts fresh (a controller pinned before the
/// session forwards as pad 0, not whatever index it held for the Settings list). pf-client-core
/// assigns indices at slot-open time; this reproduces that session-scoped start.
public func resetForwardingAssignment() {
padIndexByController.removeAll()
reselect()
active = candidates.last { $0.id == preferredID } ?? candidates.last
}
private static func describe(_ c: GCController, id: String) -> DiscoveredController {
let extended = c.extendedGamepad
let kind = padKind(extended, productCategory: c.productCategory)
let kind = padKind(extended)
return DiscoveredController(
id: id,
name: c.vendorName ?? c.productCategory,
@@ -239,40 +174,28 @@ public final class GamepadManager: ObservableObject {
hasLight: c.light != nil,
hasHaptics: c.haptics != nil,
hasMotion: c.motion != nil,
// GCDualSenseGamepad's triggers are GCDualSenseAdaptiveTrigger by declaration (the
// Edge included); the DualShock 4 has none.
hasAdaptiveTriggers: kind == .dualSense || kind == .dualSenseEdge,
// GCDualSenseGamepad's triggers are GCDualSenseAdaptiveTrigger by declaration; the
// DualShock 4 has none.
hasAdaptiveTriggers: kind == .dualSense,
batteryLevel: c.battery.flatMap { $0.batteryLevel >= 0 ? $0.batteryLevel : nil },
isCharging: c.battery?.batteryState == .charging,
controller: c)
}
/// Resolve a physical controller's matching virtual-pad type from its GameController
/// subclass (+ the product-category string where the subclass is shared). Detection order
/// (all are `: GCExtendedGamepad`): DualSense family first (the Edge is a
/// `GCDualSenseGamepad` too its distinct product category splits it out), then
/// DualShock 4, any Xbox pad, then Nintendo Switch pads by category (GameController has no
/// dedicated subclass for them). A non-extended / absent profile falls back to `.xbox360`
/// (it's never forwarded anyway).
/// subclass. Detection order (all are `: GCExtendedGamepad`): DualSense first, then
/// DualShock 4, then any Xbox pad, else fall back to Xbox 360. A non-extended / absent
/// profile also falls back to `.xbox360` (it's never forwarded anyway).
private static func padKind(
_ extended: GCExtendedGamepad?,
productCategory: String
_ extended: GCExtendedGamepad?
) -> PunktfunkConnection.GamepadType {
guard let extended else { return .xbox360 }
let category = productCategory.lowercased()
// Deployment floor (macOS 14 / iOS 17 / tvOS 17) clears every introduction version
// here, so no `@available` guard is needed matching the unguarded
// `GCDualSenseGamepad` use elsewhere in the package.
if extended is GCDualSenseGamepad {
return category.contains("edge") ? .dualSenseEdge : .dualSense
}
if extended is GCDualSenseGamepad { return .dualSense }
if extended is GCDualShockGamepad { return .dualShock4 }
if extended is GCXboxGamepad { return .xboxOne }
// Nintendo Switch Pro Controller / a paired Joy-Con set (a full pad surface). Single
// Joy-Cons ("Joy-Con (L)" / "(R)") stay on the Xbox 360 fallback half a pad.
if category.contains("switch pro") || category.contains("joy-con (l/r)") {
return .switchPro
}
return .xbox360
}
}
@@ -140,9 +140,7 @@ public final class GamepadMenuInput {
let stick = gamepad.leftThumbstick
let x = stick.xAxis.value
let y = stick.yAxis.value
// Horizontal wins an exact |x| == |y| diagonal tie (>=), matching the SDL core and Android
// nav so a perfect 45° push resolves to the same direction on every client.
if abs(x) >= abs(y), abs(x) > deadzone {
if abs(x) > abs(y), abs(x) > deadzone {
return x > 0 ? .right : .left
} else if abs(y) > deadzone {
return y > 0 ? .up : .down
@@ -6,7 +6,6 @@
// the two combine without adding a second ObservableObject or an environment key nobody else needs.
import Foundation
import PunktfunkShared
public enum GamepadUIEnvironment {
/// `enabledSetting` is the user's Settings toggle (`DefaultsKey.gamepadUIEnabled`);
@@ -1,14 +1,10 @@
// The gamepad wire contract shared by capture (GamepadCapture), feedback (GamepadFeedback),
// and the tests the pad count, button bits, axis ids, and the touchpad/motion unit conversions.
// and the tests button bits, axis ids, and the touchpad/motion unit conversions.
import Foundation
/// The gamepad wire contract (mirrors `punktfunk_core::input::gamepad`).
public enum GamepadWire {
/// Gamepads addressable on the wire the pad index rides the low byte of `flags` on every
/// per-pad event, 0...15 (`punktfunk_core::input::MAX_PADS`).
public static let maxPads: Int = 16
public static let dpadUp: UInt32 = 0x0001
public static let dpadDown: UInt32 = 0x0002
public static let dpadLeft: UInt32 = 0x0004
@@ -26,27 +22,11 @@ public enum GamepadWire {
public static let y: UInt32 = 0x8000
/// DualSense touchpad click (Moonlight's extended-button bit position).
public static let touchpadClick: UInt32 = 0x10_0000
/// Misc / capture button Xbox-Series Share, DualSense Create, Steam-Deck quick-access
/// (Moonlight's extended-button namespace; `input::gamepad::BTN_MISC1`). The host routes it to
/// the DualSense mute / Steam quick-access menu; a plain virtual xpad has no such button.
public static let misc1: UInt32 = 0x0020_0000
/// Back-grip paddles (Xbox Elite P1P4 / DualSense Edge / Steam-Deck L4-L5-R4-R5), in
/// Moonlight's extended-button namespace (`input::gamepad::BTN_PADDLE1..4`, R4/L4/R5/L5).
/// Defined for wire completeness and pinned by the tests; `GamepadCapture.buttonMask` does not
/// read them yet the GameController `paddleButton1..4` BTN_PADDLE physical correspondence
/// needs confirming on a real Elite pad first (see the gamepad-review-cleanup plan, G22), so
/// they are intentionally absent from `allButtons` until that forwarding lands.
public static let paddle1: UInt32 = 0x0001_0000
public static let paddle2: UInt32 = 0x0002_0000
public static let paddle3: UInt32 = 0x0004_0000
public static let paddle4: UInt32 = 0x0008_0000
/// Every button `buttonMask`/`sendGuide` can set walked by `sync`'s transition diff and by
/// `flush` on release. Paddles are excluded until their capture lands (see above).
public static let allButtons: [UInt32] = [
dpadUp, dpadDown, dpadLeft, dpadRight, start, back,
leftStickClick, rightStickClick, leftShoulder, rightShoulder, guide,
a, b, x, y, touchpadClick, misc1,
a, b, x, y, touchpadClick,
]
public static let axisLSX: UInt32 = 0
@@ -23,6 +23,23 @@ enum RumbleTuning {
/// the churn that lost stops inside CoreHaptics. Newest level wins when the window opens;
/// zero is never throttled.
static let minRebakeSeconds: TimeInterval = 0.025
/// Session watchdog: silence the motors when no wire command arrived for this long. This is
/// the **legacy-host fallback only** an old host sends no self-termination lease, so its
/// periodic re-send (every 500 ms) is the sole liveness signal and 3 vanished refreshes means
/// the channel or host died while audible. A v2 host instead supplies a per-command TTL (see
/// [`leaseSeconds`]); that deadline supersedes this watchdog.
static let sessionStaleSeconds: TimeInterval = 1.6
/// The legacy no-lease sentinel a v2 `ttl_ms` carries for an old host (mirrors the C ABI's
/// `PUNKTFUNK_RUMBLE_NO_TTL`). `UInt32.max` by construction.
static let noTTL: UInt32 = .max
/// Interpret a wire TTL (ms) from a rumble update: `nil` for the legacy no-lease sentinel
/// ([`noTTL`]) the renderer falls back to [`sessionStaleSeconds`] else the self-termination
/// lease in seconds (render the level for at most this long unless the host renews it).
static func leaseSeconds(ttlMs: UInt32) -> TimeInterval? {
ttlMs == noTTL ? nil : TimeInterval(ttlMs) / 1000
}
/// Levels closer than this (0.4 % of full scale) are the same level an identical host
/// refresh must never rebuild a player.
static let levelEpsilon: Float = 1.0 / 256.0
@@ -93,30 +110,17 @@ enum RumbleTuning {
/// `@unchecked Sendable` is sound because every property is read and written only inside
/// `queue` closures the serial queue is the synchronization.
final class RumbleRenderer: @unchecked Sendable {
/// Who ends an un-refreshed nonzero target. Session mode applies the core policy engine's
/// commands verbatim the engine (punktfunk-core `client/rumble.rs`) owns every lease,
/// staleness, and close decision and emits explicit zeros, so the renderer keeps NO
/// staleness policy of its own anymore. The controller test panel (`manual`) holds a slider
/// level indefinitely; both are identical renderer-side today, the distinction is kept for
/// the call sites' intent.
/// What an un-refreshed nonzero target means. A live session ties motor life to wire
/// liveness (the host refreshes state every 500 ms); the controller test panel holds a
/// slider level indefinitely.
struct Policy {
static let session = Policy()
static let manual = Policy()
}
/// Which physical actuator this renderer drives: the forwarded controller's haptics engine
/// (the default), or THIS device's own Taptic Engine (`CHHapticEngine()`) the opt-in
/// "rumble on this device" mirror for phone-clip pads that ship without rumble motors.
/// Device mode ignores `retarget`'s controller and always renders one combined motor
/// (a phone body has a single actuator).
enum Actuator {
case controller
case device
let staleAfter: TimeInterval?
static let session = Policy(staleAfter: RumbleTuning.sessionStaleSeconds)
static let manual = Policy(staleAfter: nil)
}
private let queue = DispatchQueue(label: "io.unom.punktfunk.haptics", qos: .userInteractive)
private let policy: Policy
private let actuator: Actuator
/// One finite haptic play on a motor: the player plus when (engine timeline) it expires.
/// A PLAIN pattern player on purpose: the controller haptics server (gamecontrollerd)
@@ -145,9 +149,13 @@ final class RumbleRenderer: @unchecked Sendable {
private var controller: GCController?
private var low: Motor?
private var high: Motor?
/// Wire-truth target (raw wire units) the engine command's level, applied verbatim; the
/// core policy engine owns when it ends (explicit zero commands), so no deadline lives here.
/// Wire-truth target (raw wire units) and when it was last confirmed by any command.
private var target: (low: UInt16, high: UInt16) = (0, 0)
private var lastCommand = DispatchTime(uptimeNanoseconds: 0)
/// The v2 envelope lease: the active level is authorized until here unless the host renews it
/// (`tick` silences at the deadline). `nil` against a legacy host (no lease the
/// `sessionStaleSeconds` watchdog is the backstop) and while silent.
private var envelopeDeadline: DispatchTime?
/// Runs while anything is (or should be) audible: staleness watchdog, segment re-arm,
/// throttled-level catch-up, engine rebuild after a reset, HID keepalive. Nil while silent,
/// so an idle controller costs no timer wakeups and no radio traffic.
@@ -190,9 +198,8 @@ final class RumbleRenderer: @unchecked Sendable {
((0, 0), DispatchTime(uptimeNanoseconds: 0))
#endif
init(policy: Policy = .session, actuator: Actuator = .controller) {
init(policy: Policy = .session) {
self.policy = policy
self.actuator = actuator
}
/// `onBackend`, if given, is invoked (on the internal queue) with a human-readable name of the
@@ -228,9 +235,17 @@ final class RumbleRenderer: @unchecked Sendable {
/// against a legacy host (no lease the staleness watchdog is the backstop). Renewals at an
/// unchanged level extend the deadline before the idempotence guard, so a held rumble never
/// lapses mid-effect.
func apply(low lowAmp: UInt16, high highAmp: UInt16) {
func apply(low lowAmp: UInt16, high highAmp: UInt16, ttlMs: UInt32 = RumbleTuning.noTTL) {
queue.async {
self.lastCommand = .now()
let active = lowAmp != 0 || highAmp != 0
// v2 lease: a nonzero level gets an explicit deadline; a stop or a legacy update clears
// it. Set BEFORE the idempotence guard so an identical renewal still extends the lease.
if let lease = RumbleTuning.leaseSeconds(ttlMs: ttlMs), active {
self.envelopeDeadline = .now() + lease
} else {
self.envelopeDeadline = nil
}
if active != self.wasActive {
self.wasActive = active
log.debug(
@@ -248,6 +263,7 @@ final class RumbleRenderer: @unchecked Sendable {
self.ticker?.cancel()
self.ticker = nil
self.target = (0, 0)
self.envelopeDeadline = nil
self.wasActive = false
self.teardown()
self.closeHID()
@@ -303,11 +319,25 @@ final class RumbleRenderer: @unchecked Sendable {
healthSink?(problem)
}
/// Housekeeping heartbeat while audible: segment re-arm, HID keepalive, backoff retries.
/// Every liveness decision (lease expiry, legacy-host staleness, session close) lives in the
/// core policy engine now it emits explicit zero commands, so the renderer never guesses
/// when a level should end.
/// Watchdog + housekeeping heartbeat while audible.
private func tick() {
if let deadline = envelopeDeadline {
// v2 host lease: silence the moment it lapses unrenewed. This firing in the wild is the
// observable signature of a host that stopped renewing (a dropped stop, or a dead host)
// the whole point of the envelope model: the motor can't outlive the host's intent.
if target != (0, 0), DispatchTime.now() >= deadline {
log.warning("rumble: envelope expired unrenewed — silencing")
target = (0, 0)
envelopeDeadline = nil
}
} else if let after = policy.staleAfter, target != (0, 0), seconds(since: lastCommand) > after {
// Legacy host (no lease): it re-sends state every 500 ms, so this much silence means the
// channel (or host) died while a motor was on. A direct-connected pad would have been
// stopped by its game long ago force the same outcome.
log.warning(
"rumble: no wire refresh for \(after, format: .fixed(precision: 1), privacy: .public)s — auto-silencing")
target = (0, 0)
}
render()
}
@@ -438,10 +468,6 @@ final class RumbleRenderer: @unchecked Sendable {
/// high = right/light the Xbox/XInput convention the wire carries); one combined
/// engine otherwise, driven by whichever amplitude is stronger.
private func setup() {
if actuator == .device {
setupDevice()
return
}
guard let haptics = controller?.haptics else {
// No haptics engine at all an Xbox controller on an OS/firmware that doesn't expose
// rumble through GameController (works on Android via the standard Vibrator path, but
@@ -491,41 +517,10 @@ final class RumbleRenderer: @unchecked Sendable {
}
}
/// Device-actuator mode: one combined motor on this device's own Taptic Engine. Only an
/// iPhone has one everything else (iPad, Mac, TV) reports no haptic hardware and latches
/// off (nothing to retry; the settings toggle is hidden there anyway, this is the backstop).
private func setupDevice() {
#if os(iOS)
guard CHHapticEngine.capabilitiesForHardware().supportsHaptics else {
log.info("rumble: this device has no haptic actuator — device rumble unavailable")
broken = true
reportHealth("This device has no haptic actuator.")
return
}
do {
low = startMotor(try CHHapticEngine(), sharpness: RumbleTuning.sharpnessCombined)
} catch {
log.warning("rumble: device haptic engine creation failed: \(error, privacy: .public)")
}
if low == nil {
// Same shape as the controller path: haptics exist but the engine couldn't be built
// right now back off and retry, don't latch off.
scheduleRetryBackoff()
}
#else
broken = true
#endif
}
private func makeMotor(
_ haptics: GCDeviceHaptics, _ locality: GCHapticsLocality, sharpness: Float
) -> Motor? {
guard let engine = haptics.createEngine(withLocality: locality) else { return nil }
return startMotor(engine, sharpness: sharpness)
}
/// Configure + start an engine (controller-locality or the device's own) into a [`Motor`].
private func startMotor(_ engine: CHHapticEngine, sharpness: Float) -> Motor? {
// A controller's motors carry no audio, so keep this engine OUT of the app's audio session
// (the default is to join it). Streaming keeps an AVAudioSession active the whole time;
// letting a haptics-only engine join it is a needless coupling that can get its
@@ -551,7 +546,7 @@ final class RumbleRenderer: @unchecked Sendable {
try engine.start()
return Motor(engine: engine, sharpness: sharpness)
} catch {
log.warning("haptic engine setup failed: \(error, privacy: .public)")
log.warning("haptic engine setup failed (\(locality.rawValue, privacy: .public)): \(error, privacy: .public)")
return nil
}
}
@@ -85,12 +85,6 @@ public final class InputCapture {
/// its Esc suppression need it in both states).
private var cmdKeysDown: Set<UInt32> = []
/// Physical Control/Option/Shift keys currently held (Windows VKs, both L/R sides). iPad only:
/// the Q release chord is recognized from the HID stream here (iOS has no NSEvent monitor,
/// like the toggle), so it needs the live modifier state tracked in both forwarding states,
/// exactly like `cmdKeysDown`, and flushed by `releaseAll` when GC delivery stops.
private var chordModifiersDown: Set<UInt32> = []
/// While true, mouse/keyboard flow to the host and key NSEvents are swallowed
/// locally; while false the user is interacting with the local UI (dragging the
/// window, clicking the HUD) and nothing is forwarded. Main-queue only.
@@ -125,21 +119,6 @@ public final class InputCapture {
public var onDisconnect: (() -> Void)?
public var onCycleStats: (() -> Void)?
#if os(iOS)
/// Windows VKs of the three modifier classes in the Q release chord, both L/R sides:
/// control (0xA2/0xA3), option (0xA4/0xA5), shift (0xA0/0xA1). Used to sift the HID key stream.
private static let chordModifierVKs: Set<UInt32> = [0xA2, 0xA3, 0xA4, 0xA5, 0xA0, 0xA1]
/// Whether Control AND Option AND Shift are all currently held (either side of each counts)
/// the modifier precondition for the iPad Q release chord.
private var hasReleaseChordModifiers: Bool {
let m = chordModifiersDown
return (m.contains(0xA2) || m.contains(0xA3)) // control
&& (m.contains(0xA4) || m.contains(0xA5)) // option
&& (m.contains(0xA0) || m.contains(0xA1)) // shift
}
#endif
/// Fired when a newer InputCapture takes the process-global GC handler slots (the
/// singletons hold ONE handler each): the preempted owner must drop its capture
/// state its handlers are gone, so it would otherwise sit "captured" with dead
@@ -315,7 +294,6 @@ public final class InputCapture {
/// in another app would otherwise stay "held" here forever hijacking Esc).
private func releaseAll() {
cmdKeysDown.removeAll()
chordModifiersDown.removeAll()
suppressedVK = nil
for vk in pressedVKs {
connection.send(.key(vk, down: false))
@@ -598,13 +576,6 @@ public final class InputCapture {
self.cmdKeysDown.remove(vk)
}
}
#if os(iOS)
// Track Control/Option/Shift for the Q release chord below in both forwarding
// states (like `cmdKeysDown`) so a modifier held before capture engaged still counts.
if Self.chordModifierVKs.contains(vk) {
if pressed { self.chordModifiersDown.insert(vk) } else { self.chordModifiersDown.remove(vk) }
}
#endif
// The toggle's Esc checked before the forwarding gate, because in the
// engage direction forwarding is already true when this fires.
if vk == self.suppressedVK {
@@ -621,18 +592,6 @@ public final class InputCapture {
}
#endif
guard self.forwarding else { return }
#if os(iOS)
// Q releases the captured mouse/keyboard (cross-client parity the same combo the
// macOS keyDown monitor handles). Recognized only while forwarding (nothing to release
// otherwise). The Q is latched (`suppressedVK`) so its keyUp can't type into the host;
// the modifiers were forwarded as they went down and are flushed by the release
// path (setCaptured(false) releaseAll). VK 0x51 is layout-independent (physical Q).
if pressed, vk == 0x51, self.hasReleaseChordModifiers {
self.suppressedVK = 0x51
self.onReleaseCapture?()
return
}
#endif
// Release direction of the toggle: GC's Esc-down can beat the NSEvent
// monitor never type Esc into the host while is held ( is reserved).
if vk == 0x1B, !self.cmdKeysDown.isEmpty {
@@ -118,44 +118,3 @@ extension InputCapture {
]
#endif
}
#if os(iOS)
/// US-layout character Windows VK for the on-screen keyboard (`StreamLayerUIView`'s
/// UIKeyInput). Unlike every other key source, `insertText` delivers CHARACTERS, not key
/// positions, so this is the inverse of a US layout: `shift` means "wrap in VK_LSHIFT so the
/// host types the shifted symbol". Same contract as `hidToVK`: emit only VKs the host's
/// vk_to_evdev knows; anything unmapped is dropped by the caller.
enum SoftKeyMap {
static func vk(for ch: Character) -> (vk: UInt32, shift: Bool)? {
guard let ascii = ch.asciiValue else { return nil }
switch ascii {
case UInt8(ascii: "a")...UInt8(ascii: "z"): return (UInt32(ascii) - 0x20, false)
case UInt8(ascii: "A")...UInt8(ascii: "Z"): return (UInt32(ascii), true)
case UInt8(ascii: "0")...UInt8(ascii: "9"): return (UInt32(ascii), false)
case 0x0A, 0x0D: return (0x0D, false) // return
case 0x09: return (0x09, false) // tab
case 0x20: return (0x20, false) // space
default: return symbols[ch]
}
}
/// US punctuation, plain and shifted, on the OEM VKs (mirrors `hidToVK`'s OEM block) plus
/// the shifted digit row.
private static let symbols: [Character: (vk: UInt32, shift: Bool)] = [
"-": (0xBD, false), "_": (0xBD, true),
"=": (0xBB, false), "+": (0xBB, true),
"[": (0xDB, false), "{": (0xDB, true),
"]": (0xDD, false), "}": (0xDD, true),
"\\": (0xDC, false), "|": (0xDC, true),
";": (0xBA, false), ":": (0xBA, true),
"'": (0xDE, false), "\"": (0xDE, true),
"`": (0xC0, false), "~": (0xC0, true),
",": (0xBC, false), "<": (0xBC, true),
".": (0xBE, false), ">": (0xBE, true),
"/": (0xBF, false), "?": (0xBF, true),
"!": (0x31, true), "@": (0x32, true), "#": (0x33, true), "$": (0x34, true),
"%": (0x35, true), "^": (0x36, true), "&": (0x37, true), "*": (0x38, true),
"(": (0x39, true), ")": (0x30, true),
]
}
#endif
@@ -3,8 +3,7 @@
// identical. Two mouse modes share one gesture vocabulary tap = left click · two-finger
// tap = right click · two-finger drag = scroll · tap-then-press-and-drag = held left drag
// (text selection / window moves) · three-finger tap = cycles the stats overlay tiers
// (off compact normal detailed, matching Android) · three-finger swipe up/down =
// summon/dismiss the local soft keyboard for typing on the host (`onKeyboardGesture`):
// (off compact normal detailed, matching Android):
//
// * trackpad (default): the cursor STAYS PUT on touch-down and moves by the finger's
// relative delta with mild acceleration swipe to nudge, lift and re-swipe to walk it
@@ -19,7 +18,6 @@
#if os(iOS)
import Foundation
import PunktfunkCore
import PunktfunkShared
import UIKit
/// How touchscreen fingers drive the host persisted under `DefaultsKey.touchMode`, latched
@@ -63,9 +61,6 @@ final class TouchMouse {
static let accelGain: CGFloat = 0.6
static let accelSpeedFloor: CGFloat = 0.3
static let accelMax: CGFloat = 3.0
/// Three-finger vertical swipe: the fraction of the view height the centroid must
/// travel to summon (up) / dismiss (down) the local soft keyboard.
static let keyboardSwipeFraction: CGFloat = 0.10
/// Acceleration multiplier for a finger speed in physical px per ms.
static func accel(forSpeed speed: CGFloat) -> CGFloat {
@@ -77,9 +72,6 @@ final class TouchMouse {
var send: ((PunktfunkInputEvent) -> Void)?
/// View-space point host-mode pixels through the letterbox (pointer mode's moves).
var hostPoint: ((CGPoint) -> StreamLayerUIView.HostPoint?)?
/// Three-finger vertical swipe crossed the threshold: `true` = show the local soft
/// keyboard (swipe up), `false` = dismiss it (swipe down). Fires at most once per gesture.
var onKeyboardGesture: ((Bool) -> Void)?
/// No gesture in flight (all fingers up) the view uses this to release its mode latch.
var isIdle: Bool { !sessionActive && lastPos.isEmpty }
@@ -103,11 +95,6 @@ final class TouchMouse {
private var carryY: CGFloat = 0
/// Scroll anchor (centroid) re-anchored every time a notch fires.
private var scrollAnchor = CGPoint.zero
// Keyboard-swipe state: the 3+-finger centroid anchor (per finger count, like the scroll
// anchor) and a once-per-gesture latch.
private var kbCount = 0
private var kbAnchor = CGPoint.zero
private var kbFired = false
// Tap-drag arming: a quick tap leaves a window in which the next nearby touch drags.
private var lastTapUp: TimeInterval = 0
private var lastTapPoint = CGPoint.zero
@@ -127,8 +114,6 @@ final class TouchMouse {
maxFingers = 0
moved = false
scrolling = false
kbCount = 0
kbFired = false
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left
// button for this whole gesture (laptop-trackpad convention).
dragHeld = first.timestamp - lastTapUp < Tuning.tapDragWindow
@@ -155,13 +140,8 @@ final class TouchMouse {
for touch in touches where lastPos[ObjectIdentifier(touch)] != nil {
lastPos[ObjectIdentifier(touch)] = touch.location(in: view)
}
// Dropping below three fingers forgets the keyboard-swipe anchor, so a 323 bounce
// re-anchors instead of reading the count change as swipe travel.
if lastPos.count < 3 { kbCount = 0 }
if lastPos.count == 2 {
if lastPos.count >= 2 {
scrollByCentroid()
} else if lastPos.count >= 3 {
keyboardSwipe(in: view)
} else if !scrolling, let touch = touches.first(where: {
lastPos[ObjectIdentifier($0)] != nil
}) {
@@ -228,9 +208,9 @@ final class TouchMouse {
// MARK: - Per-event work
/// Two fingers scroll by the centroid delta; never move the cursor. Fires a notch per
/// `scrollNotchPt` of pan and re-anchors on fire; finger up scrolls up, finger right
/// scrolls right (the host WHEEL(120) convention).
/// Two fingers (or more) scroll by the centroid delta; never move the cursor. Fires a
/// notch per `scrollNotchPt` of pan and re-anchors on fire; finger up scrolls up, finger
/// right scrolls right (the host WHEEL(120) convention).
private func scrollByCentroid() {
let n = CGFloat(lastPos.count)
let cx = lastPos.values.reduce(0) { $0 + $1.x } / n
@@ -253,38 +233,6 @@ final class TouchMouse {
}
}
/// Three+ fingers the keyboard swipe, never scroll (the documented vocabulary is
/// TWO-finger scroll; 3+ only fell into the scroll path as an accident of its old `>= 2`
/// bound). The centroid is anchored per finger count real fingers never land or lift in
/// the same event, so a count change must re-anchor rather than read as travel and the
/// gesture fires at most once, when the vertical travel crosses the threshold: up = show
/// the local soft keyboard, down = dismiss it.
private func keyboardSwipe(in view: UIView) {
let n = CGFloat(lastPos.count)
let cx = lastPos.values.reduce(0) { $0 + $1.x } / n
let cy = lastPos.values.reduce(0) { $0 + $1.y } / n
if lastPos.count != kbCount {
kbCount = lastPos.count
kbAnchor = CGPoint(x: cx, y: cy)
} else {
let dy = cy - kbAnchor.y
// Real centroid travel disqualifies the tap classification in `ended` (else a
// sub-threshold swipe would still fire the three-finger stats tap).
if abs(dy) > Tuning.tapSlop || abs(cx - kbAnchor.x) > Tuning.tapSlop { moved = true }
if !kbFired, abs(dy) >= view.bounds.height * Tuning.keyboardSwipeFraction {
kbFired = true
onKeyboardGesture?(dy < 0) // finger up show, finger down dismiss
}
}
// Leaving the scroll state stale would read the 32 centroid jump as a wheel notch;
// clearing it makes a return to two fingers re-anchor fresh. Same for the trackpad's
// tracked finger: its prev position froze while 3+ fingers were down, so dropping
// straight back to one finger must re-anchor (zero delta), not replay the whole
// 3-finger phase as one cursor jump.
scrolling = false
trackKey = nil
}
/// One finger (and the gesture never became a scroll dropping back from two fingers to
/// one must not jerk the cursor).
private func singleFinger(_ touch: UITouch, in view: UIView) {
@@ -1,9 +0,0 @@
// PunktfunkShared holds what the app AND the widget extension both need the stored-host model,
// the settings-key names, the App-Group constant, the deep-link grammar, and the Live Activity
// attributes in a module that links neither the Rust core nor the presentation layer.
//
// Re-export it so every existing consumer of PunktfunkKit (`import PunktfunkKit`) keeps seeing
// `StoredHost`, `DefaultsKey`, `punktfunkDefaultMgmtPort`, `DeepLink`, etc. with no call-site churn.
// (Files INSIDE PunktfunkKit still `import PunktfunkShared` explicitly Swift imports are
// file-scoped; the re-export only reaches downstream modules.)
@_exported import PunktfunkShared
@@ -1,8 +1,7 @@
// One source of truth for the client's UserDefaults / @AppStorage keys. A magic-string key
// duplicated across a setting's writer (a Settings @AppStorage) and reader (e.g. a stream view
// reading UserDefaults) splits silently on a typo the setting just stops taking effect. These
// live in the dependency-free PunktfunkShared module (re-exported by PunktfunkKit) because the app,
// the kit's views, AND the widget extension all read them the widget needs `DefaultsKey.hosts`.
// live in PunktfunkKit because both the app and the kit's views read them.
import Foundation
@@ -21,14 +20,6 @@ public enum DefaultsKey {
/// is native either way, so this degenerates to Auto-native there). Read per session by the
/// stream views' `MatchWindowFollower`.
public static let matchWindow = "punktfunk.matchWindow"
/// Render-resolution multiplier (a `RenderScale` value, default 1.0): the client asks the host
/// to render/encode at `chosen resolution × scale`, then the presenter downscales the larger
/// decoded frame to this display in one Catmull-Rom pass. > 1 supersamples (sharper, at the cost
/// of more bandwidth AND client decode both grow scale²); < 1 renders below native for a
/// weak host GPU / constrained link (the presenter upscales). Purely client-side the host just
/// sees a normal (larger/smaller) `Mode`, and Automatic bitrate scales with it. Clamped even +
/// to the codec's max dimension at connect. Applies to the fixed mode and the match-window path.
public static let renderScale = "punktfunk.renderScale"
public static let compositor = "punktfunk.compositor"
public static let gamepadType = "punktfunk.gamepadType"
public static let gamepadID = "punktfunk.gamepadID"
@@ -36,10 +27,8 @@ public enum DefaultsKey {
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the in-core decode + AVAudioEngine layout.
public static let audioChannels = "punktfunk.audioChannels"
/// Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, `"av1"`, or
/// `"pyrowave"` (the opt-in wired-LAN wavelet codec picking it advertises AND prefers it,
/// and forces the session SDR). A soft preference the host emits it when it can, else
/// falls back. Drives the decoder via `Welcome.codec`.
/// Preferred video codec: `"auto"` (host decides), `"hevc"`, or `"h264"`. A soft preference
/// the host emits it when it can, else falls back. Drives the decoder via `Welcome.codec`.
public static let codec = "punktfunk.codec"
public static let micEnabled = "punktfunk.micEnabled"
public static let speakerUID = "punktfunk.speakerUID"
@@ -77,21 +66,6 @@ public enum DefaultsKey {
public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never".
public static let cursorMode = "punktfunk.cursorMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
/// the macOS wheel, the iOS trackpad pan, and a GCMouse wheel. For users whose host expects the
/// opposite convention from their local OS preference.
public static let invertScroll = "punktfunk.invertScroll"
/// Location-based modifier mapping (a `ModifierLayout` value, default `.mac`): which Windows VK
/// each PHYSICAL modifier position forwards to the host. `.mac` keeps Option Alt and
/// Command Super/Win (the Apple positions). `.windows` swaps the Alt/Super ROLE between the
/// Option and Command keys preserving side (L/R) so the key nearest the space bar acts as
/// Alt and the next one as the Windows key, matching a Windows keyboard's `Ctrl / / Alt` row.
/// Only what's FORWARDED changes; client-local shortcuts ( &co.) stay on the physical key.
/// Read live at the wire boundary by `InputCapture`. Control/Shift never move (same position on
/// both keyboards).
public static let modifierLayout = "punktfunk.modifierLayout"
/// iPad: capture the mouse/trackpad pointer (pointer lock relative movement) for games,
/// rather than forwarding an absolute cursor position. On by default. Only meaningful on iPad
/// with a hardware mouse/trackpad; the system grants the lock only to a full-screen, frontmost
@@ -123,28 +97,12 @@ public enum DefaultsKey {
/// layout (the console launcher, gamepad-navigable settings, a coverflow-style library)
/// whenever a gamepad is connected. On by default; see `GamepadUIEnvironment.isActive`.
public static let gamepadUIEnabled = "punktfunk.gamepadUIEnabled"
/// iPhone: ALSO play the rumble the host addresses to controller 1 (wire pad 0) on this
/// device's own Taptic Engine for phone-clip pads that ship without rumble motors, where
/// the phone body is the only actuator in the player's hands. Off by default (opt-in); read
/// once per session by `GamepadFeedback`. The toggle is shown only where the device actually
/// has a haptic actuator (no iPad/Mac/TV).
public static let rumbleOnDevice = "punktfunk.rumbleOnDevice"
/// Auto-wake on connect: when connecting to a saved host that isn't advertising on mDNS, fire
/// Wake-on-LAN and, if the dial fails, wait for it to come back before retrying (the "Waking"
/// overlay). On by default. Turn off if a host that's already on just isn't seen on mDNS (a
/// routed/VPN host), so connects go straight through instead of waiting out the wake timeout.
/// The explicit "Wake Host" action stays available regardless. Read by ContentView.startSession.
public static let autoWake = "punktfunk.autoWake"
/// iOS/iPadOS: keep a streaming session ALIVE when the app is backgrounded (audio background
/// mode). Off by default (today's freeze-on-background is the default). When on, backgrounding a
/// live session keeps audio playing and the QUIC/pump live while DROPPING video decode, and a
/// bounded timer (`backgroundTimeoutMinutes`) auto-disconnects if the user doesn't return. Read
/// by ContentView's scenePhase driver. Hidden on tvOS/macOS.
public static let backgroundKeepAlive = "punktfunk.backgroundKeepAlive"
/// iOS/iPadOS: minutes a backgrounded keep-alive session runs before auto-disconnecting (a
/// battery/thermal/bandwidth backstop). Default 10; the UI offers 1/5/10/30. The auto-disconnect
/// is non-deliberate (host linger kept), so a late return reconnects fast. Read on enterBackground.
public static let backgroundTimeoutMinutes = "punktfunk.backgroundTimeoutMinutes"
}
extension Notification.Name {
@@ -154,21 +112,4 @@ extension Notification.Name {
/// menus) it exists so the menu item is honest whenever it CAN fire, and as the shortcut's
/// discoverable menu-bar surface.
public static let punktfunkReleaseCapture = Notification.Name("io.unom.punktfunk.release-capture")
/// Posted by the app's Stream menu ("Toggle Fullscreen", F) and by InputCapture's monitor
/// when the same combo fires while input is captured (the menu key-equivalent never reaches a
/// captured stream view). The key window's `FullscreenController` flips the window's fullscreen
/// state. macOS only.
public static let punktfunkToggleFullscreen = Notification.Name("io.unom.punktfunk.toggle-fullscreen")
/// Posted by the Live Activity's / Shortcuts' End-stream intent (`EndStreamIntent.perform`,
/// which runs in the app's process): the app tears the active session down deliberately
/// (quit-close the host). Same cross-process-signal pattern as `punktfunkReleaseCapture`
/// the intent lives in PunktfunkShared and can't reach the app's `SessionModel` directly.
public static let punktfunkEndActiveSession = Notification.Name("io.unom.punktfunk.end-active-session")
/// Posted by the Connect App Intent (Siri/Shortcuts) with a `punktfunk://` URL as `object`:
/// the app routes it through the SAME `.onOpenURL` handler a widget tap uses (one router, one
/// set of guards). The intent uses `openAppWhenRun`, so the app is foregrounded to receive it.
public static let punktfunkOpenDeepLink = Notification.Name("io.unom.punktfunk.open-deep-link")
}
@@ -8,7 +8,6 @@
// tap, InputCapture's captured-state S) cycle it directly.
import Foundation
import PunktfunkShared
/// How much of the streaming statistics overlay to show. The raw values are stable on disk
/// rename the cases freely, never the strings.
@@ -543,24 +543,19 @@ public enum AV1 {
extension VideoCodec {
/// Codec-dispatching format-description refresh: the AV1 path keys on an in-band sequence
/// header, the NAL codecs on in-band parameter sets one call site in each pump. PyroWave
/// has no CoreMedia representation at all (its pump feeds the Metal wavelet decoder raw).
/// header, the NAL codecs on in-band parameter sets one call site in each pump.
public func formatDescription(fromKeyframe au: Data) -> CMVideoFormatDescription? {
switch self {
case .av1: return AV1.formatDescription(fromKeyframe: au)
case .pyrowave: return nil
default: return AnnexB.formatDescription(fromIDR: au, codec: self)
}
self == .av1
? AV1.formatDescription(fromKeyframe: au)
: AnnexB.formatDescription(fromIDR: au, codec: self)
}
/// Codec-dispatching sample wrap (see `formatDescription(fromKeyframe:)`).
public func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
switch self {
case .av1: return AV1.sampleBuffer(au: au, format: format)
case .pyrowave: return nil
default: return AnnexB.sampleBuffer(au: au, format: format, codec: self)
}
self == .av1
? AV1.sampleBuffer(au: au, format: format)
: AnnexB.sampleBuffer(au: au, format: format, codec: self)
}
}
@@ -26,18 +26,12 @@ public enum VideoCodec: Equatable {
case h264
case hevc
case av1
/// PyroWave wavelet (opt-in wired-LAN low-latency codec): not a NAL/OBU codec and not
/// VideoToolbox-decoded at all the Metal wavelet decoder consumes the raw AUs
/// (Stage2Pipeline's PyroWave pump). Only ever resolved when this client both advertised
/// and preferred it.
case pyrowave
/// Resolve from the wire `Welcome.codec` byte (`PUNKTFUNK_CODEC_*`; unknown HEVC).
public init(wire: UInt8) {
switch wire {
case 0x01: self = .h264 // PUNKTFUNK_CODEC_H264
case 0x04: self = .av1 // PUNKTFUNK_CODEC_AV1
case 0x08: self = .pyrowave // PUNKTFUNK_CODEC_PYROWAVE
default: self = .hevc // PUNKTFUNK_CODEC_HEVC the default / older-host codec
}
}
@@ -153,8 +147,8 @@ public enum AnnexB {
sets = [vps, sps, pps]
case .h264:
sets = [sps, pps]
case .av1, .pyrowave:
return nil // no parameter-set NALs dispatched in AV1.swift, never reaches here
case .av1:
return nil // OBU stream, no parameter-set NALs handled in AV1.swift, never here
}
var format: CMVideoFormatDescription?
@@ -190,8 +184,8 @@ public enum AnnexB {
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
formatDescriptionOut: &format)
case .av1, .pyrowave:
break // unreachable the arm above already returned
case .av1:
break // unreachable the .av1 arm above already returned
}
}
return status == noErr ? format : nil
@@ -124,16 +124,7 @@ float2 chromaUV(texture2d<float> lumaTex, texture2d<float> chromaTex, float2 uv)
float3 sampleRgb(texture2d<float> lumaTex, texture2d<float> chromaTex, float2 uv,
constant CscUniform& csc) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
#ifdef PF_BILINEAR_LUMA
// DEBUG (PUNKTFUNK_BILINEAR_LUMA=1): plain bilinear luma Catmull-Rom OFF. A/B lever to see if
// the bicubic overshoot contributes to edge fringing. NOTE: at a true 1:1 present both paths
// reduce to the identity texel, so if this toggle VISIBLY changes the picture, the present is
// NOT 1:1 (there's a resample); if it looks identical, the fringing is upstream (codec/source/OS).
float lumaY = lumaTex.sample(s, uv).r;
#else
float lumaY = catmullRomLuma(lumaTex, s, uv);
#endif
float3 yuv = float3(lumaY,
float3 yuv = float3(catmullRomLuma(lumaTex, s, uv),
chromaTex.sample(s, chromaUV(lumaTex, chromaTex, uv)).rg);
return saturate(float3(dot(csc.r0.xyz, yuv) + csc.r0.w,
dot(csc.r1.xyz, yuv) + csc.r1.w,
@@ -149,28 +140,6 @@ fragment float4 pf_frag(VOut in [[stage_in]],
return float4(sampleRgb(lumaTex, chromaTex, in.uv, csc), 1.0);
}
// PyroWave planar SDR: three separate R8 planes (Y full-res, Cb/Cr half-res 4:2:0) from the
// Metal wavelet decoder the Metal twin of pf-presenter's planar_csc.frag. Same bicubic luma
// and left-cosited chroma correction as the biplanar path (chromaUV self-disables at 4:4:4).
fragment float4 pf_frag_planar(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> cbTex [[texture(1)]],
texture2d<float> crTex [[texture(2)]],
constant CscUniform& csc [[buffer(0)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
#ifdef PF_BILINEAR_LUMA
float lumaY = lumaTex.sample(s, in.uv).r;
#else
float lumaY = catmullRomLuma(lumaTex, s, in.uv);
#endif
float2 cuv = chromaUV(lumaTex, cbTex, in.uv);
float3 yuv = float3(lumaY, cbTex.sample(s, cuv).r, crTex.sample(s, cuv).r);
float3 rgb = saturate(float3(dot(csc.r0.xyz, yuv) + csc.r0.w,
dot(csc.r1.xyz, yuv) + csc.r1.w,
dot(csc.r2.xyz, yuv) + csc.r2.w));
return float4(rgb, 1.0);
}
// HDR: 10-bit P010 / 4:4:4 (BT.2020, PQ-encoded YCbCr) full-range PQ RGB, output as-is
// the CAMetalLayer's itur_2100_PQ colour space + edrMetadata tell the compositor the samples are
// PQ, so it does the PQdisplay tone-map. No EOTF here. The rows fold in the exact 10-bit
@@ -237,16 +206,8 @@ public final class MetalVideoPresenter {
/// tvOS only: the in-shader PQSDR tone-map fallback (pf_frag_hdr_tv bgra8), used whenever
/// the display is composited without HDR headroom see `setDisplayHeadroom`. nil elsewhere.
private let pipelineHDRToneMap: MTLRenderPipelineState?
/// PyroWave's 3-plane SDR path (pf_frag_planar bgra8) see `renderPlanar`.
private let pipelinePlanar: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache?
/// The PyroWave Metal decoder records on the presenter's device + queue: one device means
/// decode, CSC and present share textures with zero interop, and one queue means Metal's
/// hazard tracking orders a ring-slot rewrite after the render still sampling it.
var metalDevice: MTLDevice { device }
var metalQueue: MTLCommandQueue { queue }
/// Current layer configuration switched in `configure(hdr:)` when a frame's HDR-ness differs.
/// Render-thread confined once the pipeline runs (Stage2Pipeline.start's one pre-thread
/// `configure` call is ordered before the thread starts, so it doesn't race).
@@ -288,18 +249,8 @@ public final class MetalVideoPresenter {
let pipelineSDR: MTLRenderPipelineState
let pipelineHDR: MTLRenderPipelineState
let pipelineHDRToneMap: MTLRenderPipelineState?
let pipelinePlanar: MTLRenderPipelineState
do {
// DEBUG A/B lever: PUNKTFUNK_BILINEAR_LUMA=1 compiles the shader with Catmull-Rom OFF
// (plain bilinear luma) by prepending a #define ahead of the source. Default (unset) is
// the normal bicubic path. Read at presenter creation set it in the environment and
// relaunch to flip; the log line confirms which path built.
let bilinearLuma = ProcessInfo.processInfo.environment["PUNKTFUNK_BILINEAR_LUMA"] == "1"
let source = (bilinearLuma ? "#define PF_BILINEAR_LUMA 1\n" : "") + shaderSource
if bilinearLuma {
presenterLog.info("stage2: PUNKTFUNK_BILINEAR_LUMA=1 — Catmull-Rom luma DISABLED (bilinear)")
}
let library = try device.makeLibrary(source: source, options: nil)
let library = try device.makeLibrary(source: shaderSource, options: nil)
let vtx = library.makeFunction(name: "pf_vtx")
let sdr = MTLRenderPipelineDescriptor()
sdr.vertexFunction = vtx
@@ -323,11 +274,6 @@ public final class MetalVideoPresenter {
#else
pipelineHDRToneMap = nil
#endif
let planar = MTLRenderPipelineDescriptor()
planar.vertexFunction = vtx
planar.fragmentFunction = library.makeFunction(name: "pf_frag_planar")
planar.colorAttachments[0].pixelFormat = .bgra8Unorm // PyroWave is 8-bit SDR
pipelinePlanar = try device.makeRenderPipelineState(descriptor: planar)
} catch {
return nil
}
@@ -367,14 +313,12 @@ public final class MetalVideoPresenter {
return MetalVideoPresenter(
device: device, queue: queue, pipelineSDR: pipelineSDR, pipelineHDR: pipelineHDR,
pipelineHDRToneMap: pipelineHDRToneMap, pipelinePlanar: pipelinePlanar,
textureCache: textureCache, layer: layer)
pipelineHDRToneMap: pipelineHDRToneMap, textureCache: textureCache, layer: layer)
}
private init(
device: MTLDevice, queue: MTLCommandQueue, pipelineSDR: MTLRenderPipelineState,
pipelineHDR: MTLRenderPipelineState, pipelineHDRToneMap: MTLRenderPipelineState?,
pipelinePlanar: MTLRenderPipelineState,
textureCache: CVMetalTextureCache, layer: CAMetalLayer
) {
self.device = device
@@ -382,7 +326,6 @@ public final class MetalVideoPresenter {
self.pipelineSDR = pipelineSDR
self.pipelineHDR = pipelineHDR
self.pipelineHDRToneMap = pipelineHDRToneMap
self.pipelinePlanar = pipelinePlanar
self.textureCache = textureCache
self.layer = layer
}
@@ -553,67 +496,6 @@ public final class MetalVideoPresenter {
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false }
#if os(tvOS)
// HDR splits by the display's headroom (kept in step with the layer by `configure` above):
// PQ passthrough into an HDR-composited display, the tone-map shader otherwise.
let hdrPipeline = hdrPassthroughActive ? pipelineHDR : (pipelineHDRToneMap ?? pipelineHDR)
let pipeline = hdrActive ? hdrPipeline : pipelineSDR
#else
let pipeline = hdrActive ? pipelineHDR : pipelineSDR
#endif
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
return encodePresent(
decodedSize: decodedSize, targetFromLayout: targetFromLayout, pipeline: pipeline,
presentAtMediaTime: presentAtMediaTime, onPresented: onPresented,
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU
// finishes sampling releasing them at scope exit could free the backing mid-read.
keepAlive: [luma, chroma, pixelBuffer]
) { encoder in
encoder.setFragmentTexture(CVMetalTextureGetTexture(luma), index: 0)
encoder.setFragmentTexture(CVMetalTextureGetTexture(chroma), index: 1)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
}
}
/// Draw one PyroWave planar frame (three R8 planes off the Metal wavelet decoder) and
/// present it. RENDER THREAD, same contract as `render` PyroWave is 8-bit SDR, so the
/// layer always takes the plain SDR config, and the CSC rows arrive precomputed from the
/// stream's own sequence-header signaling (no CVPixelBuffer to inspect).
@discardableResult
func renderPlanar(
_ planes: WaveletPlanes,
presentAtMediaTime: CFTimeInterval? = nil,
onPresented: ((Int64?) -> Void)? = nil
) -> Bool {
stagingLock.lock()
let targetFromLayout = drawableTarget
stagingLock.unlock()
configure(hdr: false)
var csc = planes.csc
return encodePresent(
decodedSize: CGSize(width: planes.width, height: planes.height),
targetFromLayout: targetFromLayout, pipeline: pipelinePlanar,
presentAtMediaTime: presentAtMediaTime, onPresented: onPresented,
// The ring textures stay valid by ring depth; retaining them here also pins the
// slot's set until the sample completes (mirrors the biplanar keep-alive).
keepAlive: [planes.y, planes.cb, planes.cr]
) { encoder in
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
}
}
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback.
private func encodePresent(
decodedSize: CGSize, targetFromLayout: CGSize, pipeline: MTLRenderPipelineState,
presentAtMediaTime: CFTimeInterval?, onPresented: ((Int64?) -> Void)?,
keepAlive: [Any], bind: (MTLRenderCommandEncoder) -> Void
) -> Bool {
// Size the drawable to the LAYER's pixels (its laid-out frame × contentsScale, pushed here by
// SessionPresenter.layout via `setDrawableTarget` not read off the layer, whose geometry the
// main thread owns) so the Catmull-Rom shader performs the decodedon-screen scale in one pass:
@@ -622,6 +504,8 @@ public final class MetalVideoPresenter {
// Before the first layout (zero target) fall back to the decoded size. drawableSize does NOT
// track bounds (defaults to 0), so set it BEFORE nextDrawable; re-set only on a change
// (layout / Reconfigure / HDR flip and every frame of a live resize, which is fine).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
if layer.drawableSize != targetSize { layer.drawableSize = targetSize }
@@ -640,8 +524,17 @@ public final class MetalVideoPresenter {
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipeline)
bind(encoder)
#if os(tvOS)
// HDR splits by the display's headroom (kept in step with the layer by `configure` above):
// PQ passthrough into an HDR-composited display, the tone-map shader otherwise.
let hdrPipeline = hdrPassthroughActive ? pipelineHDR : (pipelineHDRToneMap ?? pipelineHDR)
encoder.setRenderPipelineState(hdrActive ? hdrPipeline : pipelineSDR)
#else
encoder.setRenderPipelineState(hdrActive ? pipelineHDR : pipelineSDR)
#endif
encoder.setFragmentTexture(CVMetalTextureGetTexture(luma), index: 0)
encoder.setFragmentTexture(CVMetalTextureGetTexture(chroma), index: 1)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
if let onPresented {
@@ -669,8 +562,9 @@ public final class MetalVideoPresenter {
} else {
commandBuffer.present(drawable)
}
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// sampling releasing them at scope exit could free the backing mid-read.
commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) }
commandBuffer.commit()
return true
}
@@ -696,17 +590,8 @@ public final class MetalVideoPresenter {
let sig = "\(Int(decoded.width))x\(Int(decoded.height))\(Int(drawable.width))x\(Int(drawable.height))|hdr\(hdrActive ? 1 : 0)"
if sig != lastSizeSig {
lastSizeSig = sig
// Explicit verdict: is the shader presenting 1:1 (decoded == drawable) or resampling? The
// scale ratio makes a residual match-window mismatch obvious. If this says 1:1 but the
// picture is still soft, the resample is downstream of us (macOS compositor a scaled
// display mode, or a fractional-pixel window position), not the shader.
let sx = decoded.width > 0 ? drawable.width / decoded.width : 0
let sy = decoded.height > 0 ? drawable.height / decoded.height : 0
let verdict = decoded == drawable
? "1:1 (no resample)"
: String(format: "RESAMPLE scale=%.4fx%.4f", sx, sy)
let msg =
"stage2: decoded \(Int(decoded.width))x\(Int(decoded.height)) → drawable \(Int(drawable.width))x\(Int(drawable.height)) [\(verdict)] hdr=\(hdrActive)"
"stage2: decoded \(Int(decoded.width))x\(Int(decoded.height)) → drawable \(Int(drawable.width))x\(Int(drawable.height)) hdr=\(hdrActive)"
presenterLog.info("\(msg, privacy: .public)")
}
}
@@ -1,604 +0,0 @@
// PyroWave native Metal decoder the Apple twin of pf-client-core's Vulkan decoder
// (crates/pf-client-core/src/video_pyrowave.rs), reimplemented on the presenter's own MTLDevice
// so decode + CSC + present share one device with zero interop (design/pyrowave-codec-plan.md
// §4.7). No upstream C/C++ ships in the app: the bitstream parse below reimplements
// pyrowave_decoder.cpp's push_packet/decode_packet walk, and the two compute kernels
// (MetalWaveletShaders.swift) are hand-ported from the vendored GLSL. The §4.2 upstream pin
// covers this hand-port: a vendored bump means re-diffing two decode shaders and the two 8-byte
// header structs, and it is already a protocol-version event.
//
// Wire shape (all fixed by the host encoder, punktfunk-host encode/linux/pyrowave.rs):
// One AU = one frame = a self-delimiting stream of packets. Each packet is one 32x32
// coefficient block for one (component, level, band), self-sized by its 8-byte
// BitstreamHeader; a per-frame START_OF_FRAME sequence header carries dims + total block
// count + the VUI bits (chroma 4:2:0, BT.709/BT.2020, limited/full).
// With `USER_FLAG_CHUNK_ALIGNED` (Phase 4) the AU is a whole number of `shard_payload`-sized
// windows, each 4-byte-prefixed (used-len u16 LE + kind u16 LE): kind 0 = whole packets,
// 1/2/3 = FRAG chain for a packet bigger than one window. A missing shard of a partial frame
// arrives as an all-zero window (used = 0) skipped, its blocks reconstruct as zeros
// (localized blur, the Phase-4 design intent). The reassembler enables partial delivery
// core-side automatically for PyroWave sessions.
// Decode acceptance mirrors upstream decode_is_ready(allow_partial=true): a frame with no
// SOF or with no more than half its blocks is dropped rather than decoded to garbage.
//
// GPU structure per frame (mirroring pyrowave_decoder.cpp's barriers): one concurrent compute
// encoder with all ~42 dequant dispatches (each writes a distinct band layer no intra-stage
// hazards), then one concurrent encoder per iDWT level (5) encoder boundaries provide the
// writesampled-read synchronization the Vulkan version expresses as pipeline barriers. The
// output is a ring of 4 plane sets (Y full-res + Cb/Cr half-res R8Unorm); ring depth plus
// same-queue hazard tracking keeps a set alive while the presenter still samples it (the same
// scheme as the Vulkan client's ring).
#if canImport(Metal)
import Foundation
import Metal
import os
private let waveletLog = Logger(subsystem: "io.unom.punktfunk", category: "pyrowave")
/// The per-(component, level, band) 32x32-block table the exact Swift port of
/// `WaveletBuffers::init_block_meta` (pyrowave_common.cpp): the walk order (level 40,
/// component 02 skipping level-0 chroma in 4:2:0, band (level==4 ? 0 : 1)3) DEFINES the
/// global `block_index` space the wire packets address, so it must match the encoder exactly.
struct WaveletLayout {
static let decompositionLevels = 5
static let alignment = 32
static let minimumImageSize = 128
let width: Int
let height: Int
let alignedWidth: Int
let alignedHeight: Int
/// blockMeta[component][level][band] = (blockOffset32x32, blockStride32x32); -1 offset =
/// band not coded (level-0 chroma in 4:2:0).
let blockMeta: [[[(offset: Int, stride: Int)]]]
let blockCount32: Int
/// Band-image extent at `level` mip `level` of the (aligned/2)-sized coefficient image.
/// Exact halving: the aligned dims are 32-aligned, so /2 is 16-aligned and survives 4 shifts.
func levelWidth(_ level: Int) -> Int { (alignedWidth / 2) >> level }
func levelHeight(_ level: Int) -> Int { (alignedHeight / 2) >> level }
init(width: Int, height: Int) {
self.width = width
self.height = height
let align = { (v: Int) in
max((v + Self.alignment - 1) & ~(Self.alignment - 1), Self.minimumImageSize)
}
alignedWidth = align(width)
alignedHeight = align(height)
var meta = [[[(offset: Int, stride: Int)]]](
repeating: [[(offset: Int, stride: Int)]](
repeating: [(offset: Int, stride: Int)](repeating: (-1, 0), count: 4),
count: Self.decompositionLevels),
count: 3)
var count32 = 0
let aw = alignedWidth
let ah = alignedHeight
for level in stride(from: Self.decompositionLevels - 1, through: 0, by: -1) {
for component in 0..<3 {
if level == 0 && component != 0 { continue } // 4:2:0: no top-level chroma
for band in (level == Self.decompositionLevels - 1 ? 0 : 1)..<4 {
let levelW = (aw / 2) >> level
let levelH = (ah / 2) >> level
let blocksX8 = (levelW + 7) / 8
let blocksY8 = (levelH + 7) / 8
let blocksX32 = (levelW + 31) / 32
meta[component][level][band] = (count32, blocksX32)
// accumulate_block_mapping's 32x32 count.
count32 += ((blocksX8 + 3) / 4) * ((blocksY8 + 3) / 4)
}
}
}
blockMeta = meta
blockCount32 = count32
}
}
/// One parsed frame, CPU side: the per-block payload offset table + the flat payload words the
/// dequant kernel consumes (packet words INCLUDING each 8-byte header, as upstream uploads
/// them), plus the sequence header's facts.
struct ParsedWaveletFrame {
var layout: WaveletLayout
/// Per 32x32 block: u32 word offset into `payload`, or UInt32.max = block missing.
var offsets: [UInt32]
var payload: [UInt32]
var totalBlocks: Int
var decodedBlocks: Int
/// VUI bits from the sequence header (BitstreamSequenceHeader).
var bt2020: Bool
var fullRange: Bool
/// The frame's YCbCrRGB signal for the presenter's planar CSC. PyroWave today is always
/// BT.709 limited (the host's fixed contract), but the sequence header signals it, so honor
/// what it says.
var cscSignal: CscRows.Signal {
CscRows.Signal(matrix: bt2020 ? 9 : 1, fullRange: fullRange)
}
}
enum WaveletBitstream {
/// Window kinds of the chunk-aligned framing (host WIN_* constants).
private static let winPacked: UInt16 = 0
private static let winFragFirst: UInt16 = 1
private static let winFragCont: UInt16 = 2
private static let winFragLast: UInt16 = 3
/// Parse one AU into the dequant kernel's inputs. `windowSize` > 0 with `chunkAligned`
/// walks the Phase-4 shard-window framing first; otherwise the AU is one packet stream.
/// nil = drop the frame (malformed, no SOF, or not enough blocks survived loss to be worth
/// decoding upstream's `decoded_blocks > total/2` partial rule).
static func parse(au: Data, chunkAligned: Bool, windowSize: Int) -> ParsedWaveletFrame? {
var state = ParseState()
let ok = au.withUnsafeBytes { (raw: UnsafeRawBufferPointer) -> Bool in
guard let base = raw.baseAddress?.assumingMemoryBound(to: UInt8.self) else {
return false
}
let count = raw.count
if chunkAligned, windowSize >= 8 {
// Whole windows only; a trailing partial window would be a framing bug.
guard count % windowSize == 0 else { return false }
var frag: [UInt8] = []
var fragLive = false
var pos = 0
while pos < count {
let win = UnsafeBufferPointer(start: base + pos, count: windowSize)
pos += windowSize
let used = Int(win[0]) | (Int(win[1]) << 8)
let kind = UInt16(win[2]) | (UInt16(win[3]) << 8)
// A zeroed (missing) shard or an overrun drops the window AND breaks any
// fragment chain riding across it (mirrors video_pyrowave.rs push_window).
guard used > 0, 4 + used <= windowSize else {
frag.removeAll(keepingCapacity: true)
fragLive = false
continue
}
let body = UnsafeBufferPointer(start: win.baseAddress! + 4, count: used)
switch kind {
case winPacked:
frag.removeAll(keepingCapacity: true)
fragLive = false
guard state.pushPackets(body) else { return false }
case winFragFirst:
frag.removeAll(keepingCapacity: true)
frag.append(contentsOf: body)
fragLive = true
case winFragCont:
if fragLive { frag.append(contentsOf: body) }
case winFragLast:
if fragLive {
frag.append(contentsOf: body)
let ok = frag.withUnsafeBufferPointer { state.pushPackets($0) }
guard ok else { return false }
}
frag.removeAll(keepingCapacity: true)
fragLive = false
default:
frag.removeAll(keepingCapacity: true)
fragLive = false
}
}
return true
}
return state.pushPackets(UnsafeBufferPointer(start: base, count: count))
}
guard ok, let frame = state.finish() else { return nil }
// Upstream decode_is_ready(allow_partial=true): with no SOF the frame is undecodable;
// at half the blocks or fewer it is presumed garbage.
guard frame.totalBlocks > 0, frame.decodedBlocks > frame.totalBlocks / 2 else {
return nil
}
return frame
}
/// Streaming packet-walk state (pyrowave_decoder.cpp push_packet + decode_packet). The
/// SOF sequence header arrives first in every host AU, which fixes the dims layout
/// offset-table size before any coefficient packet lands; a coefficient packet before the
/// SOF (its window was lost) is skipped its block just stays missing.
private struct ParseState {
var layout: WaveletLayout?
var offsets: [UInt32] = []
var payload: [UInt32] = []
var totalBlocks = 0
var decodedBlocks = 0
var bt2020 = false
var fullRange = false
var sawSOF = false
mutating func pushPackets(_ buf: UnsafeBufferPointer<UInt8>) -> Bool {
guard let base = buf.baseAddress else { return true }
var pos = 0
let count = buf.count
while count - pos >= 8 {
let word0 = loadWord(base, pos)
let word1 = loadWord(base, pos + 4)
let extended = (word0 >> 31) & 1
if extended != 0 {
// BitstreamSequenceHeader: w-1[0:14] h-1[14:28] seq[28:31] ext[31];
// total[0:24] code[24:26] chroma[26] prim[27] trc[28] mtx[29] range[30]
// siting[31].
let code = (word1 >> 24) & 0x3
guard code == 0 else { return false } // only START_OF_FRAME is defined
let chromaRes = (word1 >> 26) & 1
guard chromaRes == 0 else { return false } // host contract: 4:2:0
let w = Int(word0 & 0x3fff) + 1
let h = Int((word0 >> 14) & 0x3fff) + 1
guard w >= 2, h >= 2, w % 2 == 0, h % 2 == 0 else { return false }
if sawSOF {
// One frame, one geometry a second SOF must agree.
guard layout?.width == w, layout?.height == h else { return false }
} else {
sawSOF = true
let l = WaveletLayout(width: w, height: h)
layout = l
offsets = [UInt32](repeating: .max, count: l.blockCount32)
payload.reserveCapacity(64 * 1024 / 4)
totalBlocks = Int(word1 & 0xff_ffff)
bt2020 = (word1 >> 29) & 1 != 0
fullRange = (word1 >> 30) & 1 == 0 // YCBCR_RANGE_FULL = 0
}
pos += 8
continue
}
// BitstreamHeader: ballot[0:16] payload_words[16:28] seq[28:31] ext[31];
// quant_code[0:8] block_index[8:32]. payload_words counts u32s INCLUDING the
// 8-byte header.
let payloadWords = Int((word0 >> 16) & 0xfff)
guard payloadWords >= 2, pos + payloadWords * 4 <= count else { return false }
let blockIndex = Int(word1 >> 8)
if let layout, blockIndex < layout.blockCount32 {
// First write wins (duplicate packets are ignored, like upstream).
if offsets[blockIndex] == .max {
offsets[blockIndex] = UInt32(payload.count)
decodedBlocks += 1
payload.reserveCapacity(payload.count + payloadWords)
for w in 0..<payloadWords {
payload.append(loadWord(base, pos + w * 4))
}
}
} else if layout != nil {
return false // out-of-bounds block index corrupt stream
}
// No layout yet (SOF lost): skip the packet, the block stays missing.
pos += payloadWords * 4
}
// In the windowed framing, `used` delimits exactly; dense AUs must also consume
// fully (upstream errors on trailing bytes).
return pos == count
}
private func loadWord(_ base: UnsafePointer<UInt8>, _ offset: Int) -> UInt32 {
UInt32(base[offset])
| (UInt32(base[offset + 1]) << 8)
| (UInt32(base[offset + 2]) << 16)
| (UInt32(base[offset + 3]) << 24)
}
func finish() -> ParsedWaveletFrame? {
guard let layout else { return nil }
return ParsedWaveletFrame(
layout: layout, offsets: offsets, payload: payload,
totalBlocks: totalBlocks, decodedBlocks: decodedBlocks,
bt2020: bt2020, fullRange: fullRange)
}
}
}
/// One decoded frame's output planes, handed to the presenter's planar render path. The
/// textures belong to the decoder's ring ring depth (4) plus same-queue hazard tracking keep
/// them valid while referenced. Public because it rides inside `ReadyImage`.
public struct WaveletPlanes: @unchecked Sendable {
public let y: MTLTexture
public let cb: MTLTexture
public let cr: MTLTexture
public let csc: CscUniform
public var width: Int { y.width }
public var height: Int { y.height }
}
public final class MetalWaveletDecoder {
/// Matches the Vulkan client's ring: deep enough that a slot is never rewritten while the
/// presenter still samples it in practice; same-queue hazard tracking is the hard backstop.
private static let ringDepth = 4
/// Device-capability gate for advertisement (SessionModel) and the settings picker: the
/// dequant kernel needs simdgroup prefix sums with its 16 header lanes inside one
/// simdgroup, so compile the real kernels once and check the pipeline facts. Apple6 (A13)
/// and every Mac2 device pass the family check; the compile probe is authoritative.
public static let supported: Bool = {
guard let device = MTLCreateSystemDefaultDevice() else { return false }
guard device.supportsFamily(.apple6) || device.supportsFamily(.mac2) else { return false }
do {
let lib = try device.makeLibrary(source: waveletShaderSource, options: nil)
guard let dequant = lib.makeFunction(name: "wavelet_dequant") else { return false }
let p = try device.makeComputePipelineState(function: dequant)
var shift = false
let fc = MTLFunctionConstantValues()
fc.setConstantValue(&shift, type: .bool, index: 0)
_ = try lib.makeFunction(name: "idwt", constantValues: fc)
return p.threadExecutionWidth >= 16 && p.maxTotalThreadsPerThreadgroup >= 128
} catch {
waveletLog.info("pyrowave probe: kernels rejected (\(error, privacy: .public))")
return false
}
}()
private let device: MTLDevice
private let queue: MTLCommandQueue
private let dequantPipeline: MTLComputePipelineState
private let idwtPipeline: MTLComputePipelineState
private let idwtShiftPipeline: MTLComputePipelineState
private let mirrorSampler: MTLSamplerState
// Size-dependent state, rebuilt when the SOF dims change (this is also the mid-stream
// Reconfigure/resize path the wavelet decoder is fixed-size per geometry).
private var layout: WaveletLayout?
/// coefficients[component][level]: 4-slice R16Float (levels 01) / R32Float (levels 24)
/// texture2d_array the band images (precision-1 split, see MetalWaveletShaders).
private var coefficients: [[MTLTexture]] = []
/// llViews[component][level]: slice-0 (LL band) 2D write view of `coefficients` the iDWT
/// output target chaining level L+1 into level L.
private var llViews: [[MTLTexture]] = []
private struct Slot {
var y: MTLTexture
var cb: MTLTexture
var cr: MTLTexture
var offsets: MTLBuffer
var payload: MTLBuffer
}
private var slots: [Slot] = []
private var nextSlot = 0
/// The current geometry (from the last SOF that built the resources) the pump reports
/// decoded-size changes to the resize overlay from this. PUMP THREAD.
var decodedSize: (width: Int, height: Int)? {
layout.map { ($0.width, $0.height) }
}
/// The pump thread owns `decode`; everything mutable is confined to it.
init?(device: MTLDevice, queue: MTLCommandQueue) {
self.device = device
self.queue = queue
do {
let lib = try device.makeLibrary(source: waveletShaderSource, options: nil)
guard let dequantFn = lib.makeFunction(name: "wavelet_dequant") else { return nil }
dequantPipeline = try device.makeComputePipelineState(function: dequantFn)
var shift = false
let fcOff = MTLFunctionConstantValues()
fcOff.setConstantValue(&shift, type: .bool, index: 0)
idwtPipeline = try device.makeComputePipelineState(
function: try lib.makeFunction(name: "idwt", constantValues: fcOff))
shift = true
let fcOn = MTLFunctionConstantValues()
fcOn.setConstantValue(&shift, type: .bool, index: 0)
idwtShiftPipeline = try device.makeComputePipelineState(
function: try lib.makeFunction(name: "idwt", constantValues: fcOn))
} catch {
waveletLog.error("pyrowave: pipeline build failed (\(error, privacy: .public))")
return nil
}
guard dequantPipeline.threadExecutionWidth >= 16,
dequantPipeline.maxTotalThreadsPerThreadgroup >= 128
else { return nil }
// Upstream's mirror_repeat_sampler: mirrored repeat, NEAREST everything, normalized
// coords the idwt gather footprint + coordinate nudge depend on exactly this.
let samp = MTLSamplerDescriptor()
samp.sAddressMode = .mirrorRepeat
samp.tAddressMode = .mirrorRepeat
samp.minFilter = .nearest
samp.magFilter = .nearest
samp.mipFilter = .notMipmapped
samp.normalizedCoordinates = true
guard let sampler = device.makeSamplerState(descriptor: samp) else { return nil }
mirrorSampler = sampler
}
/// Decode one AU. Synchronous CPU parse + async GPU decode: returns false when the frame
/// was dropped (malformed / SOF lost / not enough blocks); on true, `completion` fires on a
/// Metal callback thread once the planes are decoded (nil = the GPU pass errored).
/// PUMP THREAD only.
func decode(
au: Data, chunkAligned: Bool, windowSize: Int,
completion: @escaping @Sendable (WaveletPlanes?) -> Void
) -> Bool {
guard
let frame = WaveletBitstream.parse(
au: au, chunkAligned: chunkAligned, windowSize: windowSize)
else { return false }
if layout?.width != frame.layout.width || layout?.height != frame.layout.height {
guard rebuild(layout: frame.layout) else { return false }
}
guard let layout, !slots.isEmpty else { return false }
var slot = slots[nextSlot]
// Grow the payload buffer to the frame (+16-byte zeroed guard: the kernel's 64-bit
// sign-window load and eager plane-byte prefetch may read past the payload end
// upstream pads its Vulkan buffer for exactly this).
let payloadBytes = frame.payload.count * 4
if slot.payload.length < payloadBytes + 16 {
guard
let grown = device.makeBuffer(
length: max(64 * 1024, (payloadBytes + 16) * 2), options: .storageModeShared)
else { return false }
slot.payload = grown
slots[nextSlot] = slot
}
frame.offsets.withUnsafeBytes { src in
slot.offsets.contents().copyMemory(
from: src.baseAddress!, byteCount: min(src.count, slot.offsets.length))
}
frame.payload.withUnsafeBytes { src in
slot.payload.contents().copyMemory(from: src.baseAddress!, byteCount: src.count)
}
memset(slot.payload.contents() + payloadBytes, 0, 16)
guard let cmd = queue.makeCommandBuffer() else { return false }
// Stage 1: dequant every (component, level, band) block grid in one concurrent
// encoder (each dispatch writes its own band layer; no intra-stage hazards, exactly
// like the barrier-free Vulkan dispatch loop).
guard let dequant = cmd.makeComputeCommandEncoder(dispatchType: .concurrent) else {
return false
}
dequant.label = "pyrowave dequant"
dequant.setComputePipelineState(dequantPipeline)
dequant.setBuffer(slot.offsets, offset: 0, index: 0)
dequant.setBuffer(slot.payload, offset: 0, index: 1)
for level in 0..<WaveletLayout.decompositionLevels {
for component in 0..<3 {
if level == 0 && component != 0 { continue } // 4:2:0
for band in (level == WaveletLayout.decompositionLevels - 1 ? 0 : 1)..<4 {
let meta = layout.blockMeta[component][level][band]
let w = layout.levelWidth(level)
let h = layout.levelHeight(level)
var regs = DequantRegisters(
resolution: SIMD2(Int32(w), Int32(h)),
outputLayer: Int32(band),
blockOffset32x32: Int32(meta.offset),
blockStride32x32: Int32(meta.stride))
dequant.setTexture(coefficients[component][level], index: 0)
dequant.setBytes(
&regs, length: MemoryLayout<DequantRegisters>.stride, index: 2)
dequant.dispatchThreadgroups(
MTLSize(width: (w + 31) / 32, height: (h + 31) / 32, depth: 1),
threadsPerThreadgroup: MTLSize(width: 128, height: 1, depth: 1))
}
}
}
dequant.endEncoding()
// Stage 2: iDWT, coarsest level in one encoder per level; the encoder boundary is
// the writesampled-read barrier chaining each level's LL into the next.
for inputLevel in stride(from: WaveletLayout.decompositionLevels - 1, through: 0, by: -1) {
guard let idwt = cmd.makeComputeCommandEncoder(dispatchType: .concurrent) else {
return false
}
idwt.label = "pyrowave idwt L\(inputLevel)"
idwt.setSamplerState(mirrorSampler, index: 0)
// Resolution rides TRANSPOSED (the kernel transposes on load and store).
let rx = layout.levelHeight(inputLevel)
let ry = layout.levelWidth(inputLevel)
var regs = IdwtRegisters(
resolution: SIMD2(Int32(rx), Int32(ry)),
invResolution: SIMD2(1.0 / Float(rx), 1.0 / Float(ry)))
idwt.setBytes(&regs, length: MemoryLayout<IdwtRegisters>.stride, index: 0)
let grid = MTLSize(width: (rx + 15) / 16, height: (ry + 15) / 16, depth: 1)
let group = MTLSize(width: 64, height: 1, depth: 1)
if inputLevel == 0 {
// 4:2:0: the final full-res pass is luma only (chroma finished at level 1).
idwt.setComputePipelineState(idwtShiftPipeline)
idwt.setTexture(coefficients[0][0], index: 0)
idwt.setTexture(slot.y, index: 1)
idwt.dispatchThreadgroups(grid, threadsPerThreadgroup: group)
} else {
for component in 0..<3 {
idwt.setTexture(coefficients[component][inputLevel], index: 0)
if component != 0 && inputLevel == 1 {
// 4:2:0 chroma emits its final half-res plane one level early.
idwt.setComputePipelineState(idwtShiftPipeline)
idwt.setTexture(component == 1 ? slot.cb : slot.cr, index: 1)
} else {
idwt.setComputePipelineState(idwtPipeline)
idwt.setTexture(llViews[component][inputLevel - 1], index: 1)
}
idwt.dispatchThreadgroups(grid, threadsPerThreadgroup: group)
}
}
idwt.endEncoding()
}
let planes = WaveletPlanes(
y: slot.y, cb: slot.cb, cr: slot.cr,
csc: CscRows.rows(frame.cscSignal, depth: 8, msbPacked: false))
cmd.addCompletedHandler { buffer in
completion(buffer.error == nil ? planes : nil)
}
cmd.commit()
nextSlot = (nextSlot + 1) % Self.ringDepth
return true
}
/// (Re)allocate every size-dependent resource for `layout`'s geometry. Also the mid-stream
/// resize path: a Reconfigure shows up here as new SOF dims.
private func rebuild(layout newLayout: WaveletLayout) -> Bool {
waveletLog.info(
"pyrowave: building decoder \(newLayout.width)x\(newLayout.height) (aligned \(newLayout.alignedWidth)x\(newLayout.alignedHeight), \(newLayout.blockCount32) blocks)")
var coeff: [[MTLTexture]] = []
var lls: [[MTLTexture]] = []
for component in 0..<3 {
var perLevel: [MTLTexture] = []
var perLevelLL: [MTLTexture] = []
for level in 0..<WaveletLayout.decompositionLevels {
let desc = MTLTextureDescriptor()
desc.textureType = .type2DArray
desc.arrayLength = 4
// Upstream precision 1: fp16 storage for the two finest levels, fp32 for the
// coarse levels whose values feed every later reconstruction step.
desc.pixelFormat = level < 2 ? .r16Float : .r32Float
desc.width = newLayout.levelWidth(level)
desc.height = newLayout.levelHeight(level)
desc.usage = [.shaderRead, .shaderWrite]
desc.storageMode = .private
guard let tex = device.makeTexture(descriptor: desc) else { return false }
tex.label = "pyrowave coeff c\(component) L\(level)"
guard
let ll = tex.makeTextureView(
pixelFormat: desc.pixelFormat, textureType: .type2D,
levels: 0..<1, slices: 0..<1)
else { return false }
ll.label = "pyrowave LL c\(component) L\(level)"
perLevel.append(tex)
perLevelLL.append(ll)
}
coeff.append(perLevel)
lls.append(perLevelLL)
}
var newSlots: [Slot] = []
for i in 0..<Self.ringDepth {
let plane = { (w: Int, h: Int, name: String) -> MTLTexture? in
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .r8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.shaderRead, .shaderWrite]
desc.storageMode = .private
let t = self.device.makeTexture(descriptor: desc)
t?.label = name
return t
}
guard
let y = plane(newLayout.width, newLayout.height, "pyrowave Y[\(i)]"),
let cb = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cb[\(i)]"),
let cr = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cr[\(i)]"),
let offsets = device.makeBuffer(
length: max(newLayout.blockCount32 * 4, 4), options: .storageModeShared),
let payload = device.makeBuffer(length: 64 * 1024, options: .storageModeShared)
else { return false }
newSlots.append(Slot(y: y, cb: cb, cr: cr, offsets: offsets, payload: payload))
}
coefficients = coeff
llViews = lls
slots = newSlots
nextSlot = 0
layout = newLayout
return true
}
// MSL-side layouts (MetalWaveletShaders.swift) keep in lockstep.
private struct DequantRegisters {
var resolution: SIMD2<Int32>
var outputLayer: Int32
var blockOffset32x32: Int32
var blockStride32x32: Int32
}
private struct IdwtRegisters {
var resolution: SIMD2<Int32>
var invResolution: SIMD2<Float>
}
}
#endif
@@ -1,551 +0,0 @@
// PyroWave decode compute kernels the Metal port of the vendored Vulkan shaders
// (crates/pyrowave-sys/vendor/pyrowave/shaders/wavelet_dequant.comp + idwt.comp, upstream pin
// 509e4f88, MIT © 2025 Hans-Kristian Arntzen). Runtime-compiled Swift strings per client
// convention (no metallib build step see GamepadChrome.swift's rationale); these are the
// client's first compute pipelines.
//
// Port notes (design/pyrowave-codec-plan.md §4.7):
// Only the STORAGE_MODE 0 path exists: MSL device pointers replace the 8/16-bit-storage SSBO
// aliases; the texel-buffer (mode 1) and linear-image (mode 2) fallbacks are non-Apple IHV
// workarounds and are dropped, as is the fragment-iDWT path (Mali/Adreno only).
// Subgroup ops map 1:1: subgroupInclusiveAdd simd_prefix_inclusive_sum, and the fixed
// 32-wide Apple simdgroups take the GLSL's `SubgroupSize <= 32` scan branch; the shuffle-up
// and LDS fallbacks for exotic wave sizes are dead code here. The dequant kernel needs the
// 16 header lanes inside ONE simdgroup MetalWaveletDecoder's probe enforces
// threadExecutionWidth >= 16.
// Precision matches upstream's desktop default (PYROWAVE_PRECISION=1): float arithmetic,
// half2 threadgroup storage; the coefficient textures are R16Float for DWT levels 01 and
// R32Float for levels 24 (the low-res levels feed long reconstruction chains upstream
// keeps them fp32 for exactly that reason).
// The gather + mirrored-repeat addressing in idwt is the precision-sensitive spot (upstream
// fought a Mali compiler bug there); the golden-frame PSNR fixtures are the guard.
import Foundation
let waveletShaderSource = """
#include <metal_stdlib>
using namespace metal;
// ---------------------------------------------------------------------------------------------
// Shared helpers (dwt_swizzle.h / constants.h / dwt_quant_scale.h)
// ---------------------------------------------------------------------------------------------
static inline int2 unswizzle8x8(uint index)
{
uint y = extract_bits(index, 0, 1);
uint x = extract_bits(index, 1, 2);
y |= extract_bits(index, 3, 2) << 1;
x |= extract_bits(index, 5, 1) << 2;
return int2(int(x), int(y));
}
// GLSL bitfieldExtract(x, 0, n) where n may be 0; MSL extract_bits(bits=0) is not guaranteed
// to return 0, so mask explicitly.
static inline uint mask_lo(uint x, int n)
{
return (n <= 0) ? 0u : (x & (0xffffffffu >> (32 - n)));
}
// pyrowave_common.hpp decode_quant: custom FP formulation, MaxScaleExp = 4.
static inline float decode_quant(uint quant_code)
{
int e = 4 - int(quant_code >> 3);
int m = int(quant_code) & 0x7;
return (1.0f / (8.0f * 1024.0f * 1024.0f)) * float((8 + m) * (1 << (20 + e)));
}
// dwt_quant_scale.h: per-8x8 quant scale, min 0.25, max ~2.2.
static inline float decode_quant_scale(uint code)
{
return float(code) / 8.0f + 0.25f;
}
// constants.h
constant int QUANT_SCALE_OFFSET = 20;
constant int QUANT_SCALE_BITS = 4;
// ---------------------------------------------------------------------------------------------
// wavelet_dequant one 128-thread threadgroup decodes one 32x32 coefficient block
// ---------------------------------------------------------------------------------------------
struct DequantRegisters {
int2 resolution;
int output_layer;
int block_offset_32x32;
int block_stride_32x32;
};
struct DecodedPair { float4 col0; float4 col1; }; // GLSL mat2x4: m[j][i] -> colJ[i]
// Bit-plane magnitude decode for one thread's 4x2 coefficient group (decode_payload in the
// GLSL). `code_word` is the 8x8 block's 16-bit control word (2 bits of extra planes per 4x2
// group), `q_bits` the base plane count, `offset` the block's plane-payload start byte,
// `block_index` this thread's group (0..7). Nonzero magnitudes get the +0.5 deadzone
// reconstruction bias.
static DecodedPair decode_payload(const device uchar *payload_u8,
uint code_word, uint q_bits, uint offset, uint block_index)
{
DecodedPair m;
m.col0 = float4(0.0f);
m.col1 = float4(0.0f);
if (code_word == 0)
return m;
int bit_offset = 2 * int(block_index);
uint lsbs = code_word & 0x5555u;
uint msbs = code_word & 0xaaaau;
uint msbs_shift = msbs >> 1;
msbs |= msbs_shift;
uint byte_offset =
popcount(mask_lo(lsbs, bit_offset)) +
popcount(mask_lo(msbs, bit_offset)) +
q_bits * block_index + offset;
uint payload = uint(payload_u8[byte_offset]);
uint local_control_word = extract_bits(code_word, uint(bit_offset), 2);
int decoded_abs[8] = {0, 0, 0, 0, 0, 0, 0, 0};
int plane_iterations = int(q_bits + local_control_word);
for (int q = plane_iterations - 1; q >= 0; q--)
{
for (int b = 0; b < 8; b++)
{
int decoded = int(extract_bits(payload, uint(b), 1));
decoded_abs[b] = insert_bits(decoded_abs[b], decoded, uint(q), 1);
}
byte_offset++;
payload = uint(payload_u8[byte_offset]);
}
for (int i = 0; i < 4; i++)
{
for (int j = 0; j < 2; j++)
{
float v = float(decoded_abs[i * 2 + j]);
if (v != 0.0f)
v += 0.5f;
if (j == 0) m.col0[i] = v; else m.col1[i] = v;
}
}
return m;
}
kernel void wavelet_dequant(
texture2d_array<float, access::write> uDequantImg [[texture(0)]],
const device uint *payload_offsets [[buffer(0)]],
const device uint *payload_u32 [[buffer(1)]],
constant DequantRegisters &registers [[buffer(2)]],
uint3 wg_id [[threadgroup_position_in_grid]],
uint local_index [[thread_index_in_threadgroup]],
uint simd_lane [[thread_index_in_simdgroup]],
uint simd_group [[simdgroup_index_in_threadgroup]],
uint simd_size [[threads_per_simdgroup]])
{
// STORAGE_MODE 0's three aliased SSBO views over one buffer, as typed pointers.
const device ushort *payload_u16 = reinterpret_cast<const device ushort *>(payload_u32);
const device uchar *payload_u8 = reinterpret_cast<const device uchar *>(payload_u32);
threadgroup uint shared_sign_offset;
threadgroup uint shared_plane_byte_offsets[16];
threadgroup uint shared_sign_scan[128 / 4];
int block_index_32x32 = int(uint(registers.block_offset_32x32) +
wg_id.y * uint(registers.block_stride_32x32) +
wg_id.x);
uint block_local_index = extract_bits(local_index, 0, 3);
uint block_x = extract_bits(local_index, 3, 2);
uint block_y = extract_bits(local_index, 5, 2);
uint linear_block = block_y * 4 + block_x;
// Each thread individually decodes 8 values (a 4x2 group of its 8x8 block).
int2 local_coord = unswizzle8x8(block_local_index << 3);
int2 coord = int2(wg_id.xy) * 32;
coord += 8 * int2(int(block_x), int(block_y));
coord += local_coord;
uint offset_u32 = payload_offsets[block_index_32x32];
// Missing / lost block: zero coefficients (this is how a partial frame's holes decode).
if (offset_u32 == ~0u)
{
for (int j = 0; j < 2; j++)
for (int i = 0; i < 4; i++)
uDequantImg.write(float4(0.0f), uint2(coord + int2(i, j)), uint(registers.output_layer));
return;
}
uint ballot = payload_u32[offset_u32] & 0xffffu;
uint q_code = payload_u32[offset_u32 + 1] & 0xffu;
// Threads 0..15 (one per 8x8 block, all inside simdgroup 0) prefix-scan the per-block
// plane-payload byte costs into shared_plane_byte_offsets, and lane 15 records where the
// sign bitstream starts.
if (local_index < 16)
{
uint control_word = 0;
uint q_bits = 0;
if (extract_bits(ballot, local_index, 1) != 0)
{
uint local_code_offset = popcount(mask_lo(ballot, int(local_index)));
control_word = uint(payload_u16[offset_u32 * 2 + 4 + local_code_offset]);
q_bits = uint(payload_u8[offset_u32 * 4 + 8 + popcount(ballot) * 2 + local_code_offset]) & 0xfu;
}
uint lsbs = control_word & 0x5555u;
uint msbs = control_word & 0xaaaau;
uint msbs_shift = msbs >> 1;
msbs |= msbs_shift;
uint byte_cost = popcount(lsbs) + popcount(msbs) + q_bits * 8;
uint byte_scan = offset_u32 * 4 + 8 + 3 * popcount(ballot) + simd_prefix_inclusive_sum(byte_cost);
if (local_index == 15)
shared_sign_offset = 8 * byte_scan;
shared_plane_byte_offsets[local_index] = byte_scan - byte_cost;
}
threadgroup_barrier(mem_flags::mem_threadgroup);
DecodedPair v;
int significant_count;
if (extract_bits(ballot, linear_block, 1) != 0)
{
uint local_code_offset = popcount(mask_lo(ballot, int(linear_block)));
uint control_word = uint(payload_u16[offset_u32 * 2 + 4 + local_code_offset]);
uint control_word2 = uint(payload_u8[offset_u32 * 4 + 8 + popcount(ballot) * 2 + local_code_offset]);
v = decode_payload(payload_u8, control_word, control_word2 & 0xfu,
shared_plane_byte_offsets[linear_block], block_local_index);
significant_count = 0;
for (int j = 0; j < 2; j++)
for (int i = 0; i < 4; i++)
significant_count += int(((j == 0) ? v.col0[i] : v.col1[i]) != 0.0f);
float q = decode_quant(q_code);
float inv_scale = q * decode_quant_scale(extract_bits(control_word2, uint(QUANT_SCALE_OFFSET - 16), uint(QUANT_SCALE_BITS)));
v.col0 *= inv_scale;
v.col1 *= inv_scale;
}
else
{
v.col0 = float4(0.0f);
v.col1 = float4(0.0f);
significant_count = 0;
}
// Cross-threadgroup scan of significant-coefficient counts each thread's first sign-bit
// position. Apple simdgroups are >= 16 wide, so this is the GLSL's `SubgroupSize <= 32`
// branch; the shuffle/LDS fallbacks are unnecessary.
int significant_scan = int(simd_prefix_inclusive_sum(uint(significant_count)));
if (simd_lane == simd_size - 1)
shared_sign_scan[simd_group] = uint(significant_scan);
threadgroup_barrier(mem_flags::mem_threadgroup);
uint num_simdgroups = (128 + simd_size - 1) / simd_size;
if (local_index < num_simdgroups)
shared_sign_scan[local_index] = simd_prefix_inclusive_sum(shared_sign_scan[local_index]);
threadgroup_barrier(mem_flags::mem_threadgroup);
uint sign_offset = shared_sign_offset + uint(significant_scan - significant_count);
if (simd_group != 0)
sign_offset += shared_sign_scan[simd_group - 1];
// Load 64 bits of sign stream and bit-align (may read one word past the payload the
// buffer carries a 16-byte zeroed guard tail for exactly this).
uint sign_word = payload_u32[sign_offset / 32 + 0];
uint sign_word_upper = payload_u32[sign_offset / 32 + 1];
uint masked_sign_offset = sign_offset & 31u;
if (masked_sign_offset != 0)
{
sign_word >>= masked_sign_offset;
sign_word |= sign_word_upper << (32 - masked_sign_offset);
}
int sign_counter = 0;
for (int i = 0; i < 4; i++)
{
for (int j = 0; j < 2; j++)
{
float val = (j == 0) ? v.col0[i] : v.col1[i];
if (val != 0.0f)
{
val *= 1.0f - 2.0f * float(extract_bits(sign_word, uint(sign_counter), 1));
sign_counter++;
if (j == 0) v.col0[i] = val; else v.col1[i] = val;
}
}
}
for (int j = 0; j < 2; j++)
for (int i = 0; i < 4; i++)
uDequantImg.write(float4((j == 0) ? v.col0[i] : v.col1[i]),
uint2(coord + int2(i, j)), uint(registers.output_layer));
}
// ---------------------------------------------------------------------------------------------
// idwt inverse CDF 9/7; one 64-thread threadgroup reconstructs one 32x32 output tile from the
// four half-res band layers (LL/HL/LH/HH), with a 4-sample mirror apron. The caller passes the
// band-image resolution TRANSPOSED (the kernel transposes on load and store, so one kernel does
// both the horizontal and vertical passes).
// ---------------------------------------------------------------------------------------------
constant bool DCShift [[function_constant(0)]];
struct IdwtRegisters {
int2 resolution;
float2 inv_resolution;
};
constant int APRON = 4;
constant int APRON_HALF = APRON / 2;
constant int BLOCK_SIZE = 32;
constant int BLOCK_SIZE_HALF = BLOCK_SIZE >> 1;
// CDF 9/7 lifting constants (dwt_common.h).
constant float ALPHA = -1.586134342059924f;
constant float BETA = -0.052980118572961f;
constant float GAMMA = 0.882911075530934f;
constant float DELTA = 0.443506852043971f;
constant float K = 1.230174104914001f;
constant float inv_K = 1.0f / 1.230174104914001f;
constant int SHARED_ROWS = (BLOCK_SIZE + 2 * APRON) / 2; // 20
constant int SHARED_COLS = (BLOCK_SIZE + 2 * APRON) + 1; // 41 (+1 avoids bank conflicts)
static inline float2 load_shared(threadgroup half2 (&blk)[SHARED_ROWS][SHARED_COLS], int y, int x)
{
return float2(blk[y][x]);
}
static inline void store_shared(threadgroup half2 (&blk)[SHARED_ROWS][SHARED_COLS], int y, int x, float2 v)
{
blk[y][x] = half2(v);
}
// Even/odd-phase coordinate nudge so mirrored-repeat gather reproduces JPEG2000 whole-sample
// mirroring at the image borders, then transpose (uv.yx) on load.
static inline float2 generate_mirror_uv(int2 coord, bool even_x, bool even_y,
int2 resolution, float2 inv_resolution)
{
coord.x -= int(even_x && coord.x < 0);
coord.y -= int(even_y && coord.y < 0);
coord += 1;
coord.x += int(!even_x && coord.x >= resolution.x);
coord.y += int(!even_y && coord.y >= resolution.y);
float2 uv = float2(coord) * inv_resolution;
return uv.yx;
}
static inline void write_shared_4x4(threadgroup half2 (&blk)[SHARED_ROWS][SHARED_COLS],
int2 coord, float4 t0, float4 t1, float4 t2, float4 t3)
{
store_shared(blk, coord.y + 0, 2 * coord.x + 0, float2(t0.x, t2.x));
store_shared(blk, coord.y + 0, 2 * coord.x + 1, float2(t1.x, t3.x));
store_shared(blk, coord.y + 0, 2 * coord.x + 2, float2(t0.y, t2.y));
store_shared(blk, coord.y + 0, 2 * coord.x + 3, float2(t1.y, t3.y));
store_shared(blk, coord.y + 1, 2 * coord.x + 0, float2(t0.z, t2.z));
store_shared(blk, coord.y + 1, 2 * coord.x + 1, float2(t1.z, t3.z));
store_shared(blk, coord.y + 1, 2 * coord.x + 2, float2(t0.w, t2.w));
store_shared(blk, coord.y + 1, 2 * coord.x + 3, float2(t1.w, t3.w));
}
// textureGather(...).wxzy Metal's gather returns the same counter-clockwise-from-(i0,j1)
// component order as Vulkan, so the reorder is identical.
static inline float4 gather_layer(texture2d_array<float, access::sample> tex, sampler smp,
float2 uv, uint layer)
{
float4 g = tex.gather(smp, uv, layer);
return float4(g.w, g.x, g.z, g.y);
}
static void load_image_with_apron(texture2d_array<float, access::sample> tex, sampler smp,
threadgroup half2 (&blk)[SHARED_ROWS][SHARED_COLS],
uint local_index, uint2 wg_id,
int2 resolution, float2 inv_resolution)
{
int2 base_coord = int2(wg_id) * BLOCK_SIZE_HALF - APRON_HALF;
int2 local_coord0 = 2 * unswizzle8x8(local_index);
int2 coord0 = base_coord + local_coord0;
// Band layers gathered in 0/2/1/3 order (LL/LH/HL/HH interleave for the 2x2 scatter).
float4 texels0 = gather_layer(tex, smp, generate_mirror_uv(coord0, true, true, resolution, inv_resolution), 0);
float4 texels1 = gather_layer(tex, smp, generate_mirror_uv(coord0, false, true, resolution, inv_resolution), 2);
float4 texels2 = gather_layer(tex, smp, generate_mirror_uv(coord0, true, false, resolution, inv_resolution), 1);
float4 texels3 = gather_layer(tex, smp, generate_mirror_uv(coord0, false, false, resolution, inv_resolution), 3);
write_shared_4x4(blk, local_coord0, texels0, texels1, texels2, texels3);
int2 local_coord_horiz = int2(BLOCK_SIZE_HALF + 2 * int(local_index % 2u), 2 * int(local_index / 2u));
if (local_coord_horiz.y < BLOCK_SIZE_HALF + 2 * APRON_HALF)
{
int2 c = base_coord + local_coord_horiz;
texels0 = gather_layer(tex, smp, generate_mirror_uv(c, true, true, resolution, inv_resolution), 0);
texels1 = gather_layer(tex, smp, generate_mirror_uv(c, false, true, resolution, inv_resolution), 2);
texels2 = gather_layer(tex, smp, generate_mirror_uv(c, true, false, resolution, inv_resolution), 1);
texels3 = gather_layer(tex, smp, generate_mirror_uv(c, false, false, resolution, inv_resolution), 3);
write_shared_4x4(blk, local_coord_horiz, texels0, texels1, texels2, texels3);
}
int2 local_coord_vert = local_coord_horiz.yx;
if (local_coord_vert.x < BLOCK_SIZE_HALF)
{
int2 c = base_coord + local_coord_vert;
texels0 = gather_layer(tex, smp, generate_mirror_uv(c, true, true, resolution, inv_resolution), 0);
texels1 = gather_layer(tex, smp, generate_mirror_uv(c, false, true, resolution, inv_resolution), 2);
texels2 = gather_layer(tex, smp, generate_mirror_uv(c, true, false, resolution, inv_resolution), 1);
texels3 = gather_layer(tex, smp, generate_mirror_uv(c, false, false, resolution, inv_resolution), 3);
write_shared_4x4(blk, local_coord_vert, texels0, texels1, texels2, texels3);
}
threadgroup_barrier(mem_flags::mem_threadgroup);
}
static void inverse_transform8x2(threadgroup half2 (&blk)[SHARED_ROWS][SHARED_COLS], uint local_index)
{
const int SIZE = 8;
const int PADDED_SIZE = SIZE + 2 * APRON;
const int PADDED_SIZE_HALF = PADDED_SIZE / 2;
float2 values[PADDED_SIZE];
int2 local_coord = int2(8 * int(local_index % 4u), int(local_index / 4u));
for (int i = 0; i < PADDED_SIZE; i += 2)
{
float2 v0 = load_shared(blk, local_coord.y, local_coord.x + i + 0);
float2 v1 = load_shared(blk, local_coord.y, local_coord.x + i + 1);
values[i + 0] = v0 * K;
values[i + 1] = v1 * inv_K;
}
// CDF 9/7 inverse lifting steps.
for (int i = 2; i < PADDED_SIZE - 1; i += 2)
values[i] -= DELTA * (values[i - 1] + values[i + 1]);
for (int i = 3; i < PADDED_SIZE - 2; i += 2)
values[i] -= GAMMA * (values[i - 1] + values[i + 1]);
for (int i = 4; i < PADDED_SIZE - 3; i += 2)
values[i] -= BETA * (values[i - 1] + values[i + 1]);
for (int i = 5; i < PADDED_SIZE - 4; i += 2)
values[i] -= ALPHA * (values[i - 1] + values[i + 1]);
// Avoid WAR hazard.
threadgroup_barrier(mem_flags::mem_threadgroup);
for (int i = APRON_HALF; i < PADDED_SIZE_HALF - APRON_HALF; i++)
{
float2 a = values[2 * i + 0];
float2 b = values[2 * i + 1];
// Transpose the 2x2 block, transpose write.
float2 t0 = float2(a.x, b.x);
float2 t1 = float2(a.y, b.y);
int y_coord = (local_coord.x >> 1) + (i - APRON_HALF);
store_shared(blk, y_coord, 2 * local_coord.y + 0, t0);
store_shared(blk, y_coord, 2 * local_coord.y + 1, t1);
}
}
static void inverse_transform4x2(threadgroup half2 (&blk)[SHARED_ROWS][SHARED_COLS],
uint local_index, bool active_lane, int y_offset)
{
const int SIZE = 4;
const int PADDED_SIZE = SIZE + 2 * APRON;
const int PADDED_SIZE_HALF = PADDED_SIZE / 2;
float2 values[PADDED_SIZE];
int2 local_coord = int2(4 * int(local_index % 8u), int(local_index / 8u) + y_offset);
if (active_lane)
{
for (int i = 0; i < PADDED_SIZE; i += 2)
{
float2 v0 = load_shared(blk, local_coord.y, local_coord.x + i + 0);
float2 v1 = load_shared(blk, local_coord.y, local_coord.x + i + 1);
values[i + 0] = v0 * K;
values[i + 1] = v1 * inv_K;
}
for (int i = 2; i < PADDED_SIZE - 1; i += 2)
values[i] -= DELTA * (values[i - 1] + values[i + 1]);
for (int i = 3; i < PADDED_SIZE - 2; i += 2)
values[i] -= GAMMA * (values[i - 1] + values[i + 1]);
for (int i = 4; i < PADDED_SIZE - 3; i += 2)
values[i] -= BETA * (values[i - 1] + values[i + 1]);
for (int i = 5; i < PADDED_SIZE - 4; i += 2)
values[i] -= ALPHA * (values[i - 1] + values[i + 1]);
}
threadgroup_barrier(mem_flags::mem_threadgroup);
if (active_lane)
{
for (int i = APRON_HALF; i < PADDED_SIZE_HALF - APRON_HALF; i++)
{
float2 a = values[2 * i + 0];
float2 b = values[2 * i + 1];
float2 t0 = float2(a.x, b.x);
float2 t1 = float2(a.y, b.y);
int y_coord = (local_coord.x >> 1) + (i - APRON_HALF);
store_shared(blk, y_coord, 2 * local_coord.y + 0, t0);
store_shared(blk, y_coord, 2 * local_coord.y + 1, t1);
}
}
}
kernel void idwt(
texture2d_array<float, access::sample> uTexture [[texture(0)]],
texture2d<float, access::write> uOutput [[texture(1)]],
sampler uSampler [[sampler(0)]],
constant IdwtRegisters &registers [[buffer(0)]],
uint3 wg_id [[threadgroup_position_in_grid]],
uint local_index [[thread_index_in_threadgroup]])
{
threadgroup half2 shared_block[SHARED_ROWS][SHARED_COLS];
load_image_with_apron(uTexture, uSampler, shared_block, local_index, wg_id.xy,
registers.resolution, registers.inv_resolution);
// Horizontal transform.
inverse_transform8x2(shared_block, local_index);
// Also need to transform the apron.
inverse_transform4x2(shared_block, local_index, local_index < 32, BLOCK_SIZE_HALF);
threadgroup_barrier(mem_flags::mem_threadgroup);
// Vertical transform.
inverse_transform8x2(shared_block, local_index);
threadgroup_barrier(mem_flags::mem_threadgroup);
int2 local_coord = unswizzle8x8(local_index);
for (int y = local_coord.y; y < BLOCK_SIZE_HALF; y += 8)
{
for (int x = local_coord.x; x < BLOCK_SIZE; x += 8)
{
float2 v = load_shared(shared_block, y, x);
if (DCShift)
v += 0.5f;
// Transposed store (wg_id.yx) undoes the transpose-on-load; out-of-range writes
// at the aligned-size overhang are dropped by Metal (matching the Vulkan behavior).
int2 out0 = int2(2 * y + 0, x) + BLOCK_SIZE * int2(int(wg_id.y), int(wg_id.x));
int2 out1 = int2(2 * y + 1, x) + BLOCK_SIZE * int2(int(wg_id.y), int(wg_id.x));
uOutput.write(float4(v.x), uint2(out0));
uOutput.write(float4(v.y), uint2(out1));
}
}
}
"""

Some files were not shown because too many files have changed in this diff Show More