Compare commits

...
Author SHA1 Message Date
enricobuehler 8c94e2517e fix(plugin-kit): the bun.lock I committed was corrupt, and it blocked the release
audit / cargo-audit (push) Canceled after 0s
audit / bun-audit (plugin-kit) (push) Canceled after 0s
audit / bun-audit (sdk) (push) Canceled after 0s
audit / bun-audit (web) (push) Canceled after 0s
audit / docs-site-audit (push) Canceled after 0s
audit / pnpm-audit (push) Canceled after 0s
audit / license-gate (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
ci / bun-nix (push) Canceled after 0s
ci / rust (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 14s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 27s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 20s
plugin-kit-publish / publish (push) Successful in 55s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 41s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 13s
docker / deploy-docs (push) Successful in 34s
docker / builders-arm64cross (push) Successful in 3m18s
nix / flake (push) Failing after 12m55s
`plugin-kit-v0.3.2` failed at its very first real step:

    error: Duplicate package path
        at bun.lock:71:5
    InvalidPackageKey: failed to parse lockfile: 'bun.lock'
    warn: Ignoring lockfile
    error: lockfile had changes, but lockfile is frozen

`@punktfunk/host` was listed TWICE, byte-identically, at lines 69 and 71. I
introduced it: the lock had exactly one entry before 10a0ef32 and two after.
Running `bun install` to add the biome devDependency duplicated the `file:../sdk`
entry — the same `file:`-dependency lock corruption already recorded against the
web workspace's overrides.

Nothing else in the lock is wrong, so this removes the duplicate entry rather than
regenerating (a regenerate risks reproducing it, since the `file:` dep is the
cause).

Verified with the exact commands the publish workflow runs, in order:
`bun install --frozen-lockfile --ignore-scripts` (the step that failed) now
succeeds, then the file:-dep repair, `bun run check`, `bun run typecheck`,
`bun test` 67/67, `bun run build` — all clean.

No source change; 0.3.2 is unpublished, so the tag moves to this commit.
2026-08-08 02:51:52 +02:00
enricobuehler 08525e618e Merge pull request 'chore(release): bump workspace version to 0.25.0' (#56) from worktree-release-0250 into main
audit / cargo-audit (push) Canceled after 0s
audit / bun-audit (plugin-kit) (push) Canceled after 0s
audit / bun-audit (sdk) (push) Canceled after 0s
audit / bun-audit (web) (push) Canceled after 0s
audit / docs-site-audit (push) Canceled after 0s
audit / pnpm-audit (push) Canceled after 0s
audit / license-gate (push) Canceled after 0s
ci / rust (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
ci / bun-nix (push) Canceled after 0s
apple / swift (push) Successful in 1m38s
nix / flake (push) Canceled after 0s
android-screenshots / screenshots (push) Successful in 2m1s
android / android (push) Successful in 6m3s
decky / build-publish (push) Successful in 42s
apple / screenshots (push) Successful in 5m54s
arch / build-publish (push) Successful in 10m4s
release / apple (push) Successful in 11m30s
sbom / sbom (push) Successful in 30s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
web-screenshots / screenshots (push) Successful in 5m28s
linux-client-screenshots / screenshots (push) Successful in 2m15s
deb / build-publish-client-arm64 (push) Successful in 1m39s
deb / build-publish (push) Successful in 24m28s
deb / build-publish-host (push) Successful in 6m52s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m12s
windows-host / package (push) Successful in 14m39s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Successful in 36s
docker / deploy-docs (push) Failing after 1m11s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 4m32s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 5m52s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 2m34s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5m43s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m28s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8m19s
docker / builders-arm64cross (push) Successful in 3m26s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m48s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m52s
flatpak / build-publish (push) Successful in 12m16s
Reviewed-on: #56
2026-08-08 00:45:46 +00:00
enricobuehler fa2bcd9dbb docs(release): the last three PRs — FFmpeg 9, the Arch soname trap, and the Linux buffer ceiling that was defeating A/V sync
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m9s
apple / swift (pull_request) Successful in 1m38s
apple / screenshots (pull_request) Skipped
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m12s
android / android (pull_request) Successful in 4m47s
ci / web (pull_request) Successful in 2m27s
ci / bun-nix (pull_request) Successful in 40s
ci / rust-arm64 (pull_request) Successful in 5m18s
ci / docs-site (pull_request) Successful in 1m26s
nix / flake (pull_request) Failing after 17m20s
ci / rust (pull_request) Successful in 23m58s
2026-08-08 02:43:46 +02:00
enricobuehler 353091270d Merge remote-tracking branch 'origin/main' into worktree-release-0250 2026-08-08 02:42:00 +02:00
enricobuehler 86bb09e2cf Merge pull request 'Arch could upgrade FFmpeg out from under the host and brick it — and the host now builds against FFmpeg 9' (#108) from worktree-ffmpeg9-support into main
apple / swift (push) Successful in 1m28s
android / android (push) Canceled after 0s
apple / screenshots (push) Canceled after 0s
arch / build-publish (push) Canceled after 0s
audit / cargo-audit (push) Canceled after 0s
audit / bun-audit (plugin-kit) (push) Canceled after 0s
audit / bun-audit (sdk) (push) Canceled after 0s
audit / bun-audit (web) (push) Canceled after 0s
audit / docs-site-audit (push) Canceled after 0s
audit / pnpm-audit (push) Canceled after 0s
audit / license-gate (push) Canceled after 0s
ci / rust (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
ci / bun-nix (push) Canceled after 0s
deb / build-publish (push) Canceled after 0s
deb / build-publish-host (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
flatpak / build-publish (push) Canceled after 0s
nix / flake (push) Canceled after 0s
release / apple (push) Canceled after 2m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 0s
windows-host / package (push) Canceled after 0s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 4s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
decky / build-publish (push) Successful in 1m6s
Reviewed-on: #108
2026-08-08 00:41:33 +00:00
enricobuehler deeb8b6700 feat(pf-encode): build against FFmpeg 9
apple / swift (pull_request) Successful in 1m53s
apple / screenshots (pull_request) Skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 2m34s
ci / web (pull_request) Successful in 2m32s
ci / docs-site (pull_request) Successful in 1m25s
ci / bun-nix (pull_request) Successful in 26s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 3m23s
android / android (pull_request) Successful in 6m47s
ci / rust-arm64 (pull_request) Successful in 8m49s
nix / flake (pull_request) Failing after 16m7s
ci / rust (pull_request) Successful in 23m39s
ffmpeg-next 8.1.0 could not accept FFmpeg 9 at all: ffmpeg-sys-next's version probe
covered avcodec majors 56..62 (the range is exclusive of its end), so libavcodec 63 fell
outside what it knew how to bind. 9.0.0 widens that to 56..63, which is what actually
unblocks Arch. Bump both pins — the unconditional Linux dep and the optional Windows
amf-qsv one — and the lock with them.

No API drift to fix. The crate major is a CEILING, not a target: one source tree still
spans FFmpeg 7.x/libavcodec 61, 8.x/62 and 9.x/63 via per-version cfgs, and every wrapper
symbol the NVENC-libav, VAAPI and amf-qsv backends name survives 8.1.0 -> 9.0.0
unchanged. The three hand-written #[repr(C)] hwcontext mirrors are the parts no compiler
checks, so they were re-read against the real headers rather than trusted:
AVCUDADeviceContext and AVD3D11VAFramesContext are byte-identical across 7.1/8/9, and
AVD3D11VADeviceContext gained two trailing UINTs in 8 that 7.1 lacks — which is why that
mirror deliberately stops at the common prefix, and why its assertions now say what they
do and do not buy you. They pin our layout, not libav's; a green build is not evidence.

The CI image is the step that makes this reach users. arch.yml deliberately runs no -Syu
("the image's snapshot IS the build environment"), so the builder stayed frozen on ffmpeg
8 no matter what Arch shipped, and a canary built from that snapshot could not satisfy the
soname dep the PKGBUILD now derives. Re-keying ci/ rebuilds it against ffmpeg 9.

Ubuntu and Windows deliberately stay put: the noble .deb bundles its own FFmpeg 8 behind
an rpath and strips the libav sonames from its Depends, and Windows bundles BtbN DLLs into
the signed installer — neither is exposed to the break, BtbN publishes no FFmpeg 9 build,
and moving either would re-qualify an encode stack to buy nothing.

Verified end to end on 192.168.1.21 (CachyOS, system ffmpeg 2:9.0-5, RTX 5070 Ti): host
builds clean and links libavcodec.so.63/libavutil.so.61/libavfilter.so.12/libswscale.so.10
with no unresolved sonames; the ffmpeg-8 compat shim is gone and the service runs with
NRestarts=0 and answers 401 on :47990; pf-encode's 67 tests pass; and a live synthetic
encode drives real NVENC hardware through FFmpeg 9's libavcodec to a decodable 1080p HEVC
stream (180/180 frames, FEC loopback 0 mismatches) with libavcodec.so.63 and
libnvidia-encode both mapped into the encoding process.
2026-08-08 02:35:20 +02:00
enricobuehler b1e0525872 fix(packaging/arch): pacman could upgrade FFmpeg out from under the host and brick it
`depends=('ffmpeg' ...)` carried no version bound, and pacman is the only one of our
packaging formats that does not derive dependencies from ELF DT_NEEDED — rpm
auto-generates `libavcodec.so.62()(64bit)`, dpkg-shlibdeps emits `libavcodec62`, nix
pins the closure. So when Arch shipped ffmpeg 2:9.0-5 on 2026-08-08 and every soname
moved (libavutil .60->.61, libavcodec .62->.63, libavfilter .11->.12, libavdevice
.62->.63, libswscale .9->.10), a plain `pacman -Syu` walked every Arch/CachyOS install
straight across the break. The result is not a crash we can log: the dynamic loader
cannot start the binary at all, so it is exit 127 *before* main() in a systemd restart
loop, and because punktfunk-web is a separate bun service with no libav linkage it keeps
serving happily while :47990 has nothing listening — which reads as "the mgmt API is
broken" rather than "the host is not running". `ldd /usr/bin/punktfunk-host | grep
"not found"` is the one-line diagnosis.

Depend on the sonames instead of the package. Arch's ffmpeg declares the matching
`provides=(libavcodec.so=63-64 ...)`, and makepkg rewrites each bare `libfoo.so` listed
in depends into `libfoo.so=<soname>-<arch>` by reading the built binary's DT_NEEDED, so
the bound tracks whatever FFmpeg the builder linked against with nothing to hand-maintain
across the next bump. pacman now refuses the ffmpeg upgrade rather than bricking the
install. A hand-written `ffmpeg<2:9` would have gone stale on the very next major; not
bundling FFmpeg the way the .deb does, because that exists only because Ubuntu 24.04 LTS
is frozen on 6.1 and can never satisfy the dep, while rolling Arch always ships a current
one.

Verified on a real ffmpeg-9 box (192.168.1.21): the built package records
libavcodec.so=63-64, libavutil.so=61-64, libavfilter.so=12-64, libavdevice.so=63-64 and
libswscale.so=10-64, exactly matching DT_NEEDED, with the two libs --as-needed drops left
bare and satisfied by any ffmpeg.

The new arch.yml step asserts that expansion actually happened. If it ever stops — Arch
dropping the soname provides, someone tidying the entries out of depends — the dep
silently degrades to an unversioned name that any ffmpeg satisfies, which is exactly the
state that caused this, and it is invisible in a green build until a box bricks weeks later.
2026-08-08 02:34:59 +02:00
enricobuehler 4adea10557 Merge pull request 'fix(plugin-kit): regSubKeys could never return a subkey, and adopt biome' (#107) from worktree-kit-regsubkeys into main
audit / cargo-audit (push) Successful in 32s
audit / bun-audit (plugin-kit) (push) Failing after 23s
audit / bun-audit (sdk) (push) Successful in 20s
audit / docs-site-audit (push) Successful in 25s
audit / bun-audit (web) (push) Failing after 26s
audit / pnpm-audit (push) Successful in 12s
ci / rust-arm64 (push) Successful in 1m37s
ci / bun-nix (push) Successful in 28s
ci / web (push) Successful in 1m21s
ci / docs-site (push) Successful in 1m29s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 11s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 19s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 12s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 22s
audit / license-gate (push) Successful in 6m4s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m0s
plugin-kit-publish / publish (push) Failing after 34s
docker / builders-arm64cross (push) Successful in 9s
docker / deploy-docs (push) Successful in 35s
ci / rust (push) Canceled after 8m5s
nix / flake (push) Canceled after 3m25s
Reviewed-on: #107
2026-08-08 00:23:46 +00:00
enricobuehler 4edb662b63 fix(plugin-kit): regSubKeys could never return a subkey
ci / rust-arm64 (pull_request) Successful in 2m40s
ci / web (pull_request) Successful in 1m13s
ci / bun-nix (pull_request) Successful in 22s
ci / docs-site (pull_request) Successful in 1m20s
ci / rust (pull_request) Successful in 11m48s
nix / flake (pull_request) Successful in 16m39s
Found on hardware by the GOG plugin's own parity gate, on a box with exactly one
GOG game installed:

    HKLM\SOFTWARE\WOW6432Node\GOG.com\Games -> 1 subkey (IRON NEST ...)
    host's built-in scanner:  1 entry
    plugin:                   detect: absent, 0 games
    parity FAILED - 1 missing, exit 1

`reg.exe` ALWAYS echoes the full hive name in its output rows, never the
abbreviation it was given: query `HKLM\SOFTWARE\...` and every line comes back
`HKEY_LOCAL_MACHINE\SOFTWARE\...`. regSubKeys built its match prefix from the
`HKLM\...` string it was handed, so no line ever matched and it returned `[]` —
on every machine, for every key, always. Measured verbatim on .173:

    reg.exe:     [HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\GOG.com\Games\2013434102]
    regSubKeys:  []

Its only consumer is the GOG plugin, so the symptom was "GOG reports no games
installed" rather than an error — the same shape as the SQLite reader in 0.3.1:
a total failure that every layer degrades into an empty library.

The contract was wrong too, and the hive bug hid it. regSubKeys returned whole
key PATHS while the GOG plugin uses each result as a bare NAME
(`const key = \`${GAMES_KEY}\\${id}\``, and the subkey name IS the product id
that becomes `external_id`). Even with the prefix fixed, paths would have
composed nonsense keys. It now returns names, which is what the sole consumer
and its own comment always assumed.

Parsing is split into an exported `parseRegSubKeys(stdout, key)` for the same
reason `parseRegQuery` is exported — this is a text format that breaks quietly,
and it had NO test coverage at all. Six added, using the verbatim .173 output:
names not paths, multiple subkeys, grandchildren ignored, the queried key is not
its own subkey, case-insensitivity, and empty/error input. Four of the six FAIL
against the old behaviour.

0.3.1 -> 0.3.2. Gates: biome clean, tsc clean, 67/67 tests, build clean.
2026-08-08 02:19:30 +02:00
enricobuehler 10a0ef3283 style(plugin-kit): adopt the biome config its own plugins already use
The kit had NO biome config and no lint script, while every plugin repo that
consumes it has both. So its source quietly drifted — unused imports, unsorted
imports, formatting — with nothing to catch any of it. Running biome here for
the first time reported 20 findings across 8 files.

Adds `plugin-kit/biome.json` mirroring the plugin repos' (tab indent, double
quotes, recommended lint preset, organizeImports), a `check` script, and
`@biomejs/biome` pinned to the same `^2.5.2` the plugins pin — without that pin
`bunx biome` resolved 2.4.6, which rejects the 2.5 `rules.preset` key.

Two deliberate differences from the plugin repos' copy:

  * no `vcs.useIgnoreFile` — those are standalone repos with a .gitignore beside
    the config; plugin-kit is a directory inside this one, and biome errors with
    "couldn't find an ignore file". The `files.includes` exclusions cover it.
  * `!examples/**/dist` instead of `!ui/dist` — the kit has examples, not a UI.

`css.parser.tailwindDirectives` is carried over and is load-bearing: without it
biome cannot parse `@theme` in src/theme.css and reports three parse errors on
CSS that is perfectly valid Tailwind v4.

Everything here is formatter/import churn except two real findings, both fixed:

  * `Layer` (library/define.ts) and `Cause` (sync-engine.ts) were imported and
    never used;
  * test/spike-httpapi.test.ts read `(reg?.body as …).ui.secret` one line after
    `expect(reg).toBeDefined()`. The optional chain undoes the assertion: had
    `reg` been undefined the `.ui` access would throw a TypeError instead of
    failing the test readably. Now asserted to the type system too.

Wired into plugin-kit-publish.yml as a `Lint & format` step ahead of Typecheck,
so this cannot rot again.

Gates after: biome clean (42 files), tsc clean, 67/67 tests, build clean.
2026-08-08 02:19:06 +02:00
enricobuehler cabd011f1d Merge pull request 'The 272 ms audio buffer was legal: the PipeWire callback filled the buffer ceiling, not the graph's request' (#106) from fix/pw-playback-requested into main
apple / swift (push) Successful in 1m34s
ci / web (push) Successful in 1m3s
ci / bun-nix (push) Successful in 17s
ci / docs-site (push) Successful in 1m15s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m37s
ci / rust-arm64 (push) Successful in 3m1s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 9s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 13s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 27s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 23s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
android / android (push) Successful in 5m6s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m43s
apple / screenshots (push) Successful in 6m5s
deb / build-publish (push) Successful in 4m8s
deb / build-publish-host (push) Successful in 3m54s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m9s
deb / build-publish-client-arm64 (push) Successful in 4m36s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m11s
ci / rust (push) Canceled after 2m31s
docker / builders-arm64cross (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
arch / build-publish (push) Successful in 8m34s
flatpak / build-publish (push) Successful in 18m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 22m4s
Reviewed-on: #106
2026-08-08 00:12:02 +00:00
enricobuehler d939c7c14e Merge pull request 'fix(pf-inject): the DualShock 4 Windows backend never imported OFF_INPUT' (#105) from worktree-ds4-off-input into main
ci / web (push) Successful in 1m12s
ci / docs-site (push) Successful in 1m18s
android / android (push) Canceled after 1m46s
apple / swift (push) Canceled after 1m53s
apple / screenshots (push) Canceled after 0s
ci / bun-nix (push) Successful in 24s
ci / rust-arm64 (push) Successful in 1m47s
arch / build-publish (push) Canceled after 2m6s
ci / rust (push) Canceled after 37s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 14s
deb / build-publish (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Canceled after 29s
deb / build-publish-host (push) Canceled after 30s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 10s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 4s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 0s
windows-host / package (push) Successful in 15m9s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 27s
Reviewed-on: #105
2026-08-08 00:10:18 +00:00
enricobuehler be86cfcdc0 fix(client/audio): the PipeWire callback stops filling the buffer ceiling every cycle
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m12s
apple / swift (pull_request) Successful in 1m38s
apple / screenshots (pull_request) Skipped
ci / bun-nix (pull_request) Successful in 23s
ci / web (pull_request) Successful in 1m0s
ci / docs-site (pull_request) Successful in 1m8s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m18s
ci / rust-arm64 (pull_request) Successful in 1m40s
android / android (pull_request) Successful in 5m24s
ci / rust (pull_request) Successful in 12m51s
The playback process callback sized its writes from the mapped buffer's
capacity — PipeWire's quantum-limit, 8192 frames ≈ 170 ms — instead of
the graph's per-cycle ask (pw_buffer.requested). Every cycle therefore
queued up to 170 ms of PCM downstream of the ring, and, worse, taught
JitterPolicy that the device drains 170 ms per callback: the underrun
floor (want + one frame) rose above any depth the A/V sync loop may
request, so sync measured audio ~280 ms late and was forbidden — by its
own continuity rule — from draining it. The first on-glass run of the
latency overhaul showed exactly that: audio buffer 272 ms, a/v +284 ms,
stable.

Honor requested (capacity remains both the ceiling and the fallback for
requested == 0), and log requested-vs-capacity once per stream in the
shape of the host's per-capture-open quantum line, so the next on-glass
report can say which one is sizing the writes.

Needs libpipewire >= 0.3.49 (2022-03) for the requested field; every
ship target clears that.

Verified on .21: cargo clippy -p pf-client-core --all-targets -D
warnings clean, 167 tests pass, fmt clean.
2026-08-08 02:04:15 +02:00
enricobuehler e9e1ec7dc5 fix(pf-inject): the DualShock 4 Windows backend never imported OFF_INPUT
ci / web (pull_request) Successful in 1m22s
apple / swift (pull_request) Successful in 1m32s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 2m58s
ci / docs-site (pull_request) Successful in 1m25s
ci / bun-nix (pull_request) Successful in 30s
android / android (pull_request) Successful in 5m2s
ci / rust (pull_request) Successful in 8m2s
The Windows host does not build:

  error[E0425]: cannot find value `OFF_INPUT` in this scope
    --> crates\pf-inject\src\inject\windows\dualshock4_windows.rs:65:48
  error: could not compile `pf-inject` (lib) due to 1 previous error

`dualshock4_windows.rs` writes the neutral report straight to `OFF_INPUT` in its
bootstrap path — correctly, and exactly as the DualSense and Steam Deck backends
do: the devnode does not exist yet at that point, so there is no reader to race
and no seqlock to take. Its steady-state path already goes through
`publish_input`, which is the v2.3 seqlock.

But the import list only names `publish_input`. `steam_deck_windows.rs` imports
`OFF_INPUT` explicitly for the same bootstrap write; this one was missed when the
list was edited to add `publish_input`.

One word in a `use`. No behaviour.

WHY CI DID NOT CATCH IT: `pf-inject`'s Windows backends compile only for
`*-pc-windows-msvc`, and the crate is host-side, so the client Windows workflow
never touches it. A cargo check from a Mac cannot stand in either — pf-inject
pulls punktfunk-core and therefore ring, whose C build wants MSVC headers, so the
cross-check dies in cc-rs long before it reaches this file.

FOUND BY: running windows-host.yml's own build line on the CI runner (.133)
against the v0.25.0 release tree before tagging —
`cargo build --release -p punktfunk-host --features nvenc,amf-qsv,qsv`. It fails
at `pf-inject`, which is step 1 of the host job, so a v0.25.0 tag would have
produced no Windows host binary, no installer, and no host asset on the release.
2026-08-08 01:43:11 +02:00
enricobuehler cacfe04a93 docs(release): the commit count catches up with main (327 -> 395)
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m26s
apple / swift (pull_request) Successful in 1m36s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 4m30s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m33s
ci / web (pull_request) Successful in 1m24s
ci / bun-nix (pull_request) Successful in 22s
ci / rust-arm64 (pull_request) Successful in 2m34s
ci / docs-site (pull_request) Successful in 1m27s
ci / rust (pull_request) Successful in 14m47s
nix / flake (pull_request) Failing after 13m47s
2026-08-08 01:41:09 +02:00
enricobuehler 5fda4d4805 Merge remote-tracking branch 'origin/main' into worktree-release-0250
android / android (pull_request) Canceled after 9s
apple / swift (pull_request) Canceled after 0s
apple / screenshots (pull_request) Canceled after 0s
ci / rust (pull_request) Canceled after 3s
ci / rust-arm64 (pull_request) Canceled after 2s
ci / web (pull_request) Canceled after 0s
ci / docs-site (pull_request) Canceled after 0s
ci / bun-nix (pull_request) Canceled after 0s
nix / flake (pull_request) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (pull_request) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (pull_request) Canceled after 0s
2026-08-08 01:40:04 +02:00
enricobuehler 959d3eb604 docs(release): A/V sync takes a TL;DR slot, and the old audio claim was wrong
android / android (pull_request) Canceled after 0s
apple / swift (pull_request) Canceled after 0s
apple / screenshots (pull_request) Canceled after 0s
ci / rust (pull_request) Canceled after 0s
ci / rust-arm64 (pull_request) Canceled after 0s
ci / web (pull_request) Canceled after 0s
ci / docs-site (pull_request) Canceled after 0s
ci / bun-nix (pull_request) Canceled after 0s
nix / flake (pull_request) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (pull_request) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (pull_request) Canceled after 0s
main moved another 62 commits (a8a4b11f -> fca9f42c), taking 0.25.0 to 391 since
v0.24.0. Five PRs: decode aliasing (#102), A/V sync (#101), gyro correctness
(#99), web console sweep (#100), Apple ATS (#103).

THE CORRECTION THAT MATTERED. The notes carried "Audio that falls behind the
picture pulls itself back … Android was worst, with no correction at all",
describing the jitter ring's buffer-shedding as if it were sync. It never was.
The host has stamped `pts_ns` on every audio datagram since long before v0.24.0
and EVERY CLIENT DECODED IT AND NEVER READ IT — verified in the v0.24.0 tree
(`crates/punktfunk-host/src/native/audio.rs:162` stamps it; the client audio
paths ignore it). Lip-sync was an emergent property of buffer depth, and it got
WORSE as video got faster, which is why shaving milliseconds off the audio budget
had never helped. That bullet is rewritten to say what is actually true, and A/V
sync takes a TL;DR slot.

It displaces the settings-BOM bullet, which was the weakest of the six as a
HEADLINE: conditional (only if the file was ever saved by PowerShell), partly
duplicated by the Windows non-C: entry, and it survives verbatim in Fixed. A/V
sync affects every user, every session, every client, with sound on — and unlike
most of this release it shipped broken in EVERY release we have ever made.

GYRO NEEDS AN UPGRADE NOTE, so it got one. The pipeline was wrong end to end and
is now measured against a real controller, which MOVES AIM SENSITIVITY: a pad
presented as a DualShock 4 reported gyro 40x fast (host-side), and a PlayStation
pad on Android reported ~30% short (client-side). At 40x nobody could have
compensated — gyro aim was unusable, not miscalibrated — but the Android ~1.4x
change is exactly the size a real person tunes around, so `## Before you update`
names it specifically.

DELIBERATELY NOT PROMOTED. The decode-aliasing program (#102) reads like a
catastrophe — H.264 decoding into a surface it predicted from on 297 of every 300
access units of every stream we emit, on both rungs — but it NEVER SHIPPED:
`git ls-tree v0.24.0 crates/` has no pf-vkdecode/pf-dxvadec/pf-vaadec/pf-bitstream.
It is a ship-blocker that was cleared, and writing "your picture was subtly wrong"
would be false for every reader. It contributes one clause to the decode entry
(every path is now checked frame-by-frame against a reference decoder; Windows +
Intel AV1 routes through Direct3D) and a full section in the changelog. Same
reasoning already applied to #96 and the rav1d abort.

Changelog gains the A/V sync mechanism (including that video is the master and
continuity outranks sync — the ring refuses a sync request that would break audio
on a jittery link) and the aliasing section, with the four independent reasons
four gates missed it: a structurally-blind conformance vector, a test that had
encoded the bug AS CORRECT, a vacuous assertion that could not fail, and the fact
that it streamed clean on glass. gpu_parity is 11 legs, not the 9 an earlier note
claimed.

Verified after the merge: lock diff versions-only 35/35, `cargo metadata --locked`
resolves (39 members), `cargo fmt --all --check` clean in both workspaces, notes
body 0 internal-vocabulary hits, Play notes 497/500 by android.yml's own gate.
Wire 2, C ABI 17, no new capability bits in this range.
2026-08-08 01:36:53 +02:00
enricobuehler b2f08b1a73 Merge pull request 'fix(pf-dxvadec): a wrapped sentence turned "6." into an ordered list' (#104) from worktree-dxva-doclint into main
ci / web (push) Successful in 1m35s
ci / docs-site (push) Successful in 1m35s
ci / bun-nix (push) Successful in 1m16s
ci / rust-arm64 (push) Successful in 2m8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 14s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 18s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 1m1s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m2s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 1m11s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 19s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 27s
android / android (push) Successful in 5m46s
deb / build-publish-client-arm64 (push) Successful in 3m56s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m52s
apple / swift (push) Successful in 1m35s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m32s
deb / build-publish-host (push) Successful in 5m26s
arch / build-publish (push) Successful in 8m58s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders-arm64cross (push) Successful in 9s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m57s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m39s
apple / screenshots (push) Successful in 6m16s
docker / deploy-docs (push) Failing after 4m26s
ci / rust (push) Successful in 7m45s
deb / build-publish (push) Successful in 7m51s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m48s
Reviewed-on: #104
2026-08-07 23:26:52 +00:00
enricobuehler 7f82bca9c0 fix(pf-dxvadec): a wrapped sentence turned "6." into an ordered list
ci / bun-nix (pull_request) Successful in 23s
ci / web (pull_request) Successful in 1m10s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m17s
ci / docs-site (pull_request) Successful in 2m3s
ci / rust-arm64 (pull_request) Successful in 2m16s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m10s
apple / swift (pull_request) Successful in 1m34s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 4m59s
ci / rust (pull_request) Successful in 6m3s
A doc paragraph in `pic_av1.rs` wrapped so that "first at frame / 6. Releasing…"
put `6.` at the start of a line. rustdoc reads that as an ordered-list item
starting at 6, which makes the following unindented `///` line a lazy
continuation — `clippy::doc_lazy_continuation`, denied by `-D warnings`.

Reflowed so the number cannot begin a line. Prose is byte-identical in content;
only the wrap points move. No code, no behaviour.

WHY THIS MATTERS FOR THE TAG. `pf-dxvadec` is Windows-only, and no Windows leg
runs on a push to main — so main being green proves nothing about this. The
failure surfaces for the first time in a release tag's fan-out, which is exactly
what happened to the FIRST v0.23.0 tag: it went red on Windows clippy for this
same lint, and the cure was a tag re-point.

Caught pre-tag by re-running the lazy-continuation scanner over the tree while
preparing v0.25.0 (0 hits before this commit's parent merged the new decode
crates, 1 after). Cannot be verified by compiling here — the crate does not build
on macOS — so the evidence is the scanner plus the lint's own rule, not a clippy
run.
2026-08-08 01:25:59 +02:00
enricobuehler 9043332002 Merge remote-tracking branch 'origin/main' into worktree-release-0250 2026-08-08 01:24:23 +02:00
enricobuehler fca9f42c44 Merge pull request 'Worktree apple mgmt ats bypass' (#103) from worktree-apple-mgmt-ats-bypass into main
apple / swift (push) Successful in 1m39s
ci / rust-arm64 (push) Successful in 2m3s
ci / web (push) Successful in 1m4s
ci / bun-nix (push) Successful in 31s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 14s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 12s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 13s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
ci / docs-site (push) Successful in 1m58s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 39s
apple / screenshots (push) Canceled after 0s
ci / rust (push) Canceled after 49s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 3s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
release / apple (push) Successful in 9m42s
Reviewed-on: #103
2026-08-07 23:20:19 +00:00
enricobuehler 3d20f2c0e5 Merge pull request 'Three decode rungs were decoding into a surface they were predicting from' (#102) from integration/decode-aliasing-program into main
ci / bun-nix (push) Successful in 30s
ci / web (push) Successful in 1m22s
ci / docs-site (push) Successful in 1m24s
apple / swift (push) Successful in 1m35s
ci / rust-arm64 (push) Successful in 1m56s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 16s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 16s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 16s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
deb / build-publish-client-arm64 (push) Successful in 3m6s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m11s
android / android (push) Successful in 5m57s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m21s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m37s
apple / screenshots (push) Successful in 5m52s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m24s
deb / build-publish-host (push) Successful in 6m20s
ci / rust (push) Canceled after 6m56s
docker / builders-arm64cross (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
windows-host / package (push) Failing after 1m37s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
deb / build-publish (push) Successful in 5m39s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m47s
arch / build-publish (push) Successful in 11m19s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m19s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 6m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 10m20s
flatpak / build-publish (push) Successful in 8m54s
Reviewed-on: #102
2026-08-07 23:11:28 +00:00
enricobuehler 7798401f06 perf(apple): cache posters on disk and pool the mgmt connections
ci / rust-arm64 (pull_request) Successful in 1m36s
ci / bun-nix (pull_request) Successful in 24s
ci / web (pull_request) Successful in 1m16s
apple / swift (pull_request) Successful in 1m38s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m12s
ci / rust (pull_request) Successful in 13m3s
Moving the management API onto Network.framework left one request per
connection, so a library grid paid a TLS handshake per poster where the pooled
URLSession had shared one. And the Apple client -- unlike Windows -- never
cached art at all, so it re-fetched every poster on every visit.

ArtCache: a size- and age-bounded blob cache in the CACHES directory (every byte
is re-derivable from the host, so the system is welcome to evict it). Keyed by
the SHA-256 of the absolute URL, so host-proxy paths and store CDN URLs share
one cache without colliding. Reads touch the entry, so eviction is by last USE,
not last write. Empty bodies and data: URLs are refused -- neither is worth a
file. Defaults: 128 MB, 30 days.

Connection pooling: MgmtConnectionPool keeps up to four keep-alive connections
per host and makes further callers wait rather than opening more, which is the
part that matters -- a grid can ask for dozens of posters at once. A connection
the host dropped since we last used it is indistinguishable from a live one
until we write, so a REUSED connection that fails is retried once on a fresh
one; a fresh failure is a real failure.

Keep-alive means a response can no longer be delimited by the peer hanging up,
so HTTPResponseParser.messageLength finds the end from the framing itself --
Content-Length or the chunked terminal chunk plus trailers. Getting that wrong
would truncate a response or bleed one into the next, silently, so it carries
the bulk of the new tests. A connection with bytes left over after a response is
dropped rather than reused: we never pipeline, so anything trailing means we are
out of sync.

LibraryView closes the loader's pooled connections on disappear instead of
leaving sockets open on a screen the user has left.

16 new tests: message framing (both encodings, partial reads, back-to-back
responses, close detection) and the cache (binary round trip, key separation,
refusals, expiry, LRU eviction).
2026-08-08 01:09:25 +02:00
enricobuehler 244cafe005 refactor(apple): move the mgmt API off URLSession so ATS can stay on
The previous commit bought the library back on VPN/remote hosts by declaring
NSAllowsArbitraryLoads, which works but is blunt: it drops ATS for ALL of the
app's URLSession traffic, and the only other traffic is third-party cover-art
CDN fetches -- the one surface we never wanted to open. It cost the TLS-version
floor, forward secrecy, and the cleartext-HTTP block on URLs the host supplies
at runtime (custom entries and scanner plugins carry arbitrary ones).

So take the host out of the URL loading system instead. MgmtTransport speaks
HTTPS over Network.framework, which ATS does not govern, and states the trust
rule we actually mean in a verify block: the leaf must hash to the fingerprint
pinned during PIN pairing. That is the same rule punktfunk-core has always
applied on the QUIC stream plane -- which is exactly why streaming kept working
over Tailscale while the library did not.

With that, the ATS dict is gone and ATS is fully enforced again. Cover-art CDN
fetches keep ordinary URLSession with full system trust evaluation and no client
certificate. LibraryTLSDelegate is deleted; nothing pins through URLSession now.

Also here:
- HTTPResponse: just enough HTTP/1.1 to read one GET -- status, headers,
  Content-Length and chunked framing (hyper streams the art proxy chunked). A
  body shorter than Content-Length throws instead of returning partial JSON,
  which would otherwise read as "this host has no games".
- LibraryError.pinMismatch, so a re-keyed host says "pair again" rather than
  sending someone to debug their network.
- 403 joins 401 as "unauthorized": both are the host declining the certificate.
- baseURL brackets IPv6 literals; the old string interpolation did not.
- 11 tests covering the framings hyper emits and the failure modes that would
  otherwise be silent.

Known trade-off: no connection reuse yet, so each poster costs its own
handshake where the pooled URLSession shared one. Fine on a LAN, worth revisiting
for large libraries over a high-latency link.
2026-08-08 01:09:04 +02:00
enricobuehler bae8742e48 fix(apple): the library's unreachable message pointed at the wrong layer
It led with `--mgmt-bind 127.0.0.1`, a rare cause, and never mentioned the one
fact that actually explains the symptom: the library rides the management API on
a different port (47990) than the QUIC stream plane (9777), so it can fail while
streaming to the same host works. Field triage of exactly that case spent hours
on the stream path before anyone questioned the port.

Leads with that now, and names browser-testing the port as the fast split between
"unreachable" and anything client-side.
2026-08-08 01:07:26 +02:00
enricobuehler 78a02bc994 fix(apple): ATS blocked the library on every non-RFC1918 host
The game library rides the management REST API over HTTPS (TCP 47990) through
URLSession, authenticated by mTLS and pinned by SHA-256 fingerprint. The app
declared no App Transport Security policy at all, so it ran under default ATS --
which exempts only "local" destinations (.local, unqualified names, RFC1918 and
link-local literals) and applies the full policy everywhere else. The host
certificate is self-signed (and carries no SubjectAltName), so it cannot satisfy
that policy: the library loaded at 192.168.x and failed at the TLS layer on any
other address.

Field-reported against a Tailscale host. 100.64.0.0/10 is CGNAT, not RFC1918, so
the library failed there while streaming to the very same address worked -- the
QUIC stream plane is raw UDP and never enters the URL loading system. A WireGuard
peer or a public-IP host breaks identically.

Declares NSAllowsArbitraryLoads, which must stay the dict's only key: iOS 10+ and
macOS 10.12+ ignore it whenever a more granular ATS key sits alongside it. Trust
is unchanged -- LibraryTLSDelegate still pins the host by fingerprint and hands
every other origin (cover-art CDNs) to full system trust evaluation.
2026-08-08 01:07:26 +02:00
enricobuehler 2b167595aa docs(client): the VAAPI rung has parity now — say what is actually left
ci / bun-nix (pull_request) Successful in 28s
windows / build (aarch64-pc-windows-msvc) (pull_request) Failing after 30s
apple / swift (pull_request) Successful in 1m36s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m19s
ci / web (pull_request) Successful in 1m30s
ci / rust-arm64 (pull_request) Successful in 2m39s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m26s
android / android (pull_request) Successful in 4m31s
ci / rust (pull_request) Successful in 10m52s
Its rows still read "never frame-hash parity-checked: the rung exports a tiled
dmabuf with no CPU-readable image, so parity needs a readback path that does not
exist yet". That readback now exists, and all SEVEN legs came back bit-identical
to libavcodec on RDNA3: vendored H.264 250/250, our host's low-delay H.264
120/120, vendored H.265 250/250, host low-delay H.265 120/120, HEVC Main 10
50/50 as P010, vendored AV1 250/250 of 274 decoded, and host low-delay 4K
two-tile AV1 60/60.

The two arms collapse into one, because the thing that split them — AV1 having
evidence the other legs lacked — is gone. Every leg now has the same evidence.

It stays `verified = false`, and the note says why in the words the
unproven-rung test requires: it has NEVER run on a second vendor and has never
been soaked. That is a real limit rather than a formality — every other verified
pair in this table earned it on more than one part, and the D3D11VA AV1 row two
entries up is a rung that passed on one vendor's driver while failing on
another's.

The second reason is not about evidence at all, and it belongs in the record
rather than in a commit nobody reads later: flipping this flag is a ROUTING
change. `native_rung_admitted` is `verified || !below.verified`, so a verified
VAAPI outranks Vulkan Video on every Linux AMD and Intel client — the Steam Deck
included. The parity result justifies that change; it should still be made on
purpose, by someone who wants it, rather than arriving as a side effect of
writing down a test result.
2026-08-08 01:00:06 +02:00
enricobuehler a9e7c033c3 test(client/vaapi): the last rung of the ladder, finally checked in pixels — 7 legs, all bit-identical
Every other decode rung earns `verified` with frame-hash parity against
libavcodec. VAAPI could not: it hands out a DRM-PRIME dmabuf whose memory the
driver tiles, so nothing could read its decoded pixels back, and all four of its
legs sat at "never frame-hash parity-checked".

That was never bookkeeping. The D3D11VA AV1 rung decoded 250 frames, streamed
4K60 through a clean five-minute soak, and produced WRONG PIXELS for 186 of 250
frames on NVIDIA and 245 of 250 on Intel. It looked perfect on glass; only the
goldens caught it, and the same defect turned out to be in H.264 on two other
rungs. VAAPI was the one rung where that class of bug could still be sitting
with nothing able to see it.

It is not. Measured on .25 (Radeon 780M, RDNA3, radeonsi, Mesa 26.0.3, VA-API
1.23) on 2026-08-08, against the SAME golden files the Vulkan and D3D11VA rungs
are held to, read across the crate boundary rather than copied:

  H.264 vendored vector            250/250 bit-identical  (7 from the flush)
  H.264 our host, low-delay 640x480 120/120 bit-identical  (3 from the flush)
  H.265 vendored vector            250/250 bit-identical  (2 from the flush)
  H.265 our host, low-delay 640x480 120/120 bit-identical  (0 from the flush)
  HEVC Main 10, P010                 50/50 bit-identical  (2 from the flush)
  AV1 vendored vector              250/250 delivered of 274 decoded, and
                                   display frame 0 byte-identical to
                                   libavcodec's own PIXELS
  AV1 our host, 4K two-tile          60/60 bit-identical

⚠ ONE vendor. AMD/radeonsi only; no Intel iHD box has run these legs.

The readback that made it possible:

* `pf-vaadec`'s `va` module gains `VAImage` and `VAImageFormat`, hand-declared
  with every size and offset measured off libva 2.23.0's real headers by
  `layout-probe.c` and pinned as compile-time assertions — the same discipline
  the decode buffers already keep. The trap: `VAImage::width`/`height` are
  16-bit, so `data_size` sits at 60 and not at the 64 counting 32-bit fields
  gives, and every field after them is two bytes earlier than it looks.
* `pack_two_plane` is the pure geometry — the crop to the picture, the padding
  columns dropped per row, and the chroma plane taken from the driver's OWN
  `offsets[1]` rather than from `pitch * display_height`, which is the 1088-row
  smear this program has already paid for once. It needs no device, so ten CPU
  tests cover it on macOS and in the container.
* `video_vaapi_native::parity` drives the seven streams above through the
  production entry point and hashes what the rung DELIVERS, in delivery order,
  tail included — so the delivery path is under test as well as the decode, and
  a frame's surface comes from its own release token rather than from an
  inference about which pool entry holds which picture.

THE READBACK CANNOT REACH THE PRODUCTION PATH, and that is structural rather
than a promise. `vaDeriveImage`, `vaCreateImage`, `vaGetImage`, `vaMapBuffer`
and the rest are resolved by a `#[cfg(test)]` type that dlopens libva itself;
the production `Libva` gains no field; `sha2` is a dev dependency. A CPU test
scans this file's own source and fails if any of those symbols is dlsym'd
outside the harness, so a refactor cannot quietly undo it.

Derive is not guaranteed, so both routes are implemented and neither is
optional: `vaDeriveImage` first, `vaCreateImage` + `vaGetImage` as the fallback
(which also detiles), and if neither yields the pool's own fourcc the leg FAILS
naming what the driver gave it. There is no skip path — a parity test that
passes because it could not read anything is the failure mode this program has
been bitten by three times. Both answer on radeonsi, the first frame of every
leg is read through BOTH and they must agree, and `PF_VAAPI_READBACK=getimage`
reproduces the H.264 leg's 250/250 through the copying route alone, so the
fallback is exercised rather than merely written.

And it can fail — proven, not asserted. Planting the real geometry defect this
driver's layout makes visible (rows read contiguously, ignoring the 512-byte
pitch behind a 320-wide picture) fails at display frame 0 with the full
localisation: 68312 luma and 14998 chroma samples differing, max |delta| 255,
luma bounding box (0,1)..(319,239) — and with the goldens forced through one
route, 250/250 diverging with "suspect the readback geometry". `compare` and
`localise` also have CPU counterfactuals, and a hardware leg proves the readback
reads real and DISTINCT pixels and localises a one-byte flip to the exact pixel.

⚠ One thing the hardware legs do NOT cover, found by planting the other defect
and watching it do nothing: radeonsi's decode surfaces for every fixture here
have no VERTICAL padding — `offsets[1]` is exactly `pitch * height` — so the
chroma-plane trap is untested on this driver, and `pf-vaadec`'s
`reading_chroma_at_the_display_height_would_have_been_caught` is the only place
it is checked at all. `probe_this_machines_readback_routes` now prints the
derived layout and says which of the two it is, so the next driver answers for
itself instead of being assumed.
2026-08-08 00:55:03 +02:00
enricobuehler 167e35f720 Merge pull request 'Audio was late because nothing ever placed it against the picture' (#101) from audio/latency-overhaul into main
ci / web (push) Successful in 1m17s
apple / swift (push) Successful in 1m33s
ci / rust-arm64 (push) Successful in 1m35s
ci / bun-nix (push) Successful in 26s
ci / docs-site (push) Successful in 1m21s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m24s
deb / build-publish-client-arm64 (push) Successful in 2m52s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 17s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 8s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m49s
deb / build-publish-host (push) Successful in 4m41s
apple / screenshots (push) Successful in 5m50s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m36s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m31s
docker / builders-arm64cross (push) Successful in 11s
arch / build-publish (push) Successful in 8m51s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 3m33s
android / android (push) Successful in 11m39s
ci / rust (push) Successful in 11m32s
docker / deploy-docs (push) Failing after 1m43s
windows-host / package (push) Failing after 2m16s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
deb / build-publish (push) Successful in 5m3s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m12s
flatpak / build-publish (push) Successful in 8m42s
release / apple (push) Successful in 9m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m12s
Reviewed-on: #101
2026-08-07 22:32:08 +00:00
enricobuehler bfed711921 Merge remote-tracking branch 'origin/main' into audio/latency-overhaul
ci / bun-nix (pull_request) Successful in 33s
ci / web (pull_request) Successful in 1m9s
ci / docs-site (pull_request) Successful in 1m21s
apple / swift (pull_request) Successful in 1m33s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 2m13s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m36s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m9s
android / android (pull_request) Successful in 7m53s
ci / rust (pull_request) Successful in 12m23s
2026-08-08 00:16:02 +02:00
enricobuehler c43769282a fix(apple): place audio with the picture instead of wherever the ring settles
ci / bun-nix (pull_request) Successful in 41s
ci / web (pull_request) Successful in 1m25s
apple / swift (pull_request) Successful in 1m35s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m51s
ci / rust-arm64 (pull_request) Successful in 2m24s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m55s
ci / rust (pull_request) Successful in 3m31s
android / android (pull_request) Successful in 3m54s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m18s
The Apple half of the A/V sync overhaul; the Rust half is 12a53183 and this
mirrors its policy rather than re-deriving one.

The host stamps `pts_ns` on every audio datagram and the client decoded it into
`AudioPCM` — and then never read it. Video's `pts_ns` is used end to end (the
end-to-end meter computes a true glass-to-glass `displayed + clockOffset − pts`
per presented frame), so audio free-ran at whatever depth its jitter ring
happened to reach, video was presented on an independent path, and nothing ever
compared them. The A/V offset was an accident of buffer depths: it moved
whenever the ring ratcheted under underrun pressure, and it got WORSE every time
video got faster, because a quicker decoder lowers the video leg and leaves
audio's exactly where it was.

Video is the master:

  audio_e2e = (now + buffered_ahead + clock_offset) − pts_ns
  av_offset = audio_e2e − video_e2e        (> 0 ⇒ audio behind the picture)

`AvSync` smooths that with an EWMA, ignores what sits inside a deadband no
listener can detect, refuses the implausible outright rather than clamping it (a
wall-clock step must not steer the ring), and proposes a depth. Swift refuses
one thing Rust does not have to: the arithmetic itself. The Rust controller
works in i128, while Swift has no Int128 at this tools version, so the terms are
combined with overflow-REPORTING arithmetic instead of the `&-` the latency
meters use. That is not defensive padding — `ptsNs = 1 << 63` reads as
`Int64.min`, the difference lands on exactly `Int64.min`, and `abs()` of that
has no representable result, so checking the overflow flags AFTER the sanity
limit does not mis-measure the stream, it aborts the process from the audio
drain thread. The guard's short-circuit ordering is what makes the sanity check
safe to run at all.

Continuity outranks sync, always. `AudioRing.setSyncTarget` only ever takes a
REQUEST, clamped between the existing underrun-driven floor and the hard cap. A
link whose jitter genuinely needs more buffer than the picture is away keeps its
buffer and the residual is reported. `nil` is the default and reproduces the
previous behaviour exactly. The clamp raises its ceiling to the floor rather
than using it as-is: a device whose callback quantum alone exceeds the hard cap
makes floor > cap, and a plain `min(max(s, floor), cap)` would then hand back
the CAP — quietly below the continuity floor, inverting the exact ordering this
exists to guarantee, on the awkward hardware it exists to survive. (Rust's
`Ord::clamp` announces that condition by panicking; Swift would just get it
wrong, which is worse.)

The reference is the other half, and without it the loop is inert — which is why
this was split out rather than shipped alongside the Rust side. `LatencyMeter`
now publishes its most recent sample as a LEVEL, so the end-to-end meter the
presenter already writes per presented frame becomes the video figure the audio
plane reads. Both present paths (arrival and deadline) feed it without either
knowing audio exists, and the stage-1 fallback presenter — which stamps no
present at all — offers nothing, so the loop correctly declines to correct. The
level EXPIRES, unlike the Rust atomic: this client has a backgrounded keep-alive
that keeps audio playing and drops video decode entirely, and a reference with
no expiry would go on steering the ring against a figure minutes old and frozen.

And the reason none of this was visible: `bufferedMS`/`targetMS` existed only in
a periodic log line, absent from anything a surface could render. The HUD's
detailed tier now carries `audio buffer N ms · a/v ±N ms` and the 1 Hz stats log
gains the same pair, appended last so existing parsers are unaffected — both
numbers, because a deep ring on a jittery link is correct and only the offset
separates that from audio held late.

`PUNKTFUNK_NO_AV_SYNC=1` disarms the loop without a rebuild, as on the Rust
clients.

Verified: swift build + 225 tests (5 skipped) green. Every new gate was proven
non-vacuous by planting its own defect and confirming the gate caught it — the
continuity invariant, the clamp inversion, the deadband, both refusal paths, the
evidence threshold, the sync-pressure relax, the reference's staleness and its
survival of a drain, and `setSyncTarget` being live at all rather than dead
code, which is how the previous pass in this area shipped a correction that was
structurally unreachable with a green test. Two gates came back VACUOUS on the
first sweep and are the reason their inputs look so specific: the overflow test
was being caught by the sanity limit instead of the overflow guard, and the
refused-reference test was being caught by `latestSample`'s own `> 0` check
rather than by where the publish sits.
2026-08-08 00:10:48 +02:00
enricobuehler f926bab9f7 fix(client): the native VAAPI rung stopped dropping decoded frames on the floor
`finish` showed `outputs.last()` and retired every other picture an access unit
bumped out of the DPB without ever displaying it, and nothing flushed the DPB at
end of stream. Measured on .25 against the vendored vectors: 225 of 250 frames
for H.264, 204 of 250 for H.265, 45 of 50 for HEVC Main 10. D3D11VA and Vulkan
deliver every frame, so this was the rung's alone. All four legs now deliver
250 / 250 / 50 / 250.

The same function carried a second defect. `DmabufFrame::keyframe` was stamped
with the CURRENT access unit's `is_idr`, not the flag of the picture it was
about to display, and on a reordering stream those are different pictures: the
IDR is bumped out several units after it decodes and arrived flagged `false` on
all three legs' first frame, while a later AU draining the DPB flagged some old
trailing picture as a keyframe. That field is `DecodedImage::is_keyframe`, the
pump's post-loss re-anchor signal, so a mislabel re-anchors on the wrong frame.

Three changes, all inside this rung:

* **A deliverable queue**, the same shape as `video_vk_native`'s — extend, ship
  the front, trim the oldest past the bound, count and rate-limit the drops into
  `DecodeHealth::dropped`. Its DEPTH is derived differently and the divergence is
  documented: the Vulkan rung's bound is `HOLD_HEADROOM - PIPELINE_HOLD` = 1
  because a queued frame there counts against the pool ON TOP of the DPB's own
  residency. Here the three claims are disjoint and a bumped picture MOVES from
  `pending`/slot to `held`, so the queue inherits the claim rather than adding
  one. The bound is the DPB's depth — the deepest carry-over a bump can leave —
  and the measured cost is at most one surface (zero on H.264, whose three
  seven-picture IDR drains are the deepest bursts these vectors have). A bound of
  1 would have left 235 of 250 on H.264, most of the defect still in place.

* **An end-of-stream flush.** This rung has no EOS signal and cannot have one:
  the pump feeds access units until the session ends and then drops the decoder.
  So `flush` has the two honest callers — `Drop`, where nothing can be presented
  and the job is to release the queue's surfaces and the DPB's before the pool
  goes, and a caller that KNOWS the stream ended, which today is the conformance
  harness. One walk, not a production path and an untested teardown path. AV1
  needs none: it shows at most one frame per temporal unit and buffers nothing,
  which its 250/250 says out loud.

* **`PictureFacts` recorded when a picture decodes**, and read back when it is
  displayed. `keyframe` was the defect; `color` and `display` are the same
  mistake one field along — an in-band HDR switch changes the VUI mid-stream and
  AV1's render region is per-frame, so a queued frame shown two units later would
  have been drawn with the newest picture's signalling.

Concealment answers `Ok(None)` and deliberately does NOT drain the queue, which
is the Vulkan rung's order and is load-bearing: `clears_demotion_streak` is
`delivered || !concealed`, so shipping a queued frame on a concealed AU would
zero the streak and take away the escape hatch that stops a rung concealing
forever from holding a frozen picture.

The three delivered-count assertions moved with the fix, and so did the CPU
derivation that reproduces them without a GPU — it now simulates the whole
delivery model (ledger, queue, one-per-AU hand-off, flush) in the order `decode`
does it, and carries the old behaviour beside the new one as a counterfactual:
a queue bound of 0 with no flush still reproduces 225/204/45 exactly, and the
test fails if it ever stops being SHORT. `settle` was split out as the pure half
of `finish` so the claim walk, the display ordering and the picture facts are
all assertable with no device; `the_queue_never_needs_a_surface_the_pool_does_not_have`
runs the surface-lifetime arithmetic over the real vectors and pins the peak
claims (9 of a 16-surface pool on H.264, 8 of 14 on both HEVC vectors), with an
unbounded queue as the counterfactual that shows the bound doing its job.

Gates run: `cargo fmt --all -- --check`, `cargo clippy -p pf-client-core
-p pf-vaadec --all-targets --features sdl3/build-from-source -- -D warnings`,
`cargo test -p pf-client-core --lib --features sdl3/build-from-source` (176
pass), the same filtered to `video_vaapi_native -- --include-ignored` (23 pass,
0 ignored) and `cargo test -p pf-vaadec` (48 pass) — all on .25 (Radeon 780M,
RDNA3, radeonsi, Mesa 26.0.3, VA-API 1.23); plus `cargo fmt --all -- --check`
and `cargo clippy --workspace --all-targets -- -D warnings` in pf-lxcheck2.
2026-08-08 00:10:35 +02:00
enricobuehler 74270109dd ci(android): lint the Android target, which nothing had ever done
`ci.yml` runs `cargo clippy --workspace` on the HOST, where
`clients/android/native` and every `#[cfg(target_os = "android")]` module
elsewhere compile out, and `android.yml` only ever built. So the Android target
was never linted at all — not once. Five lints were sitting in
clients/android/native when this was noticed, in code no gate had ever read.

The gate is a Gradle task rather than a YAML step because cargo-ndk needs a
specific discovery environment (NDK sysroot, SDK cmake 3.22.1 for libopus,
`LIBOPUS_STATIC`, Ninja) and duplicating it into the workflow would let the lint
drift from the build — a lint that ran against a different toolchain is a lint
about a different program. `registerCargoNdkClippy` reuses the build task's
environment verbatim via the extracted `cargoNdkEnvironment`, so local and CI
runs are the same invocation.

It lints BOTH pointer widths, and that is load-bearing rather than thorough:
arm64-v8a is 64-bit and armeabi-v7a is 32-bit, so a cast that is redundant on
one can be required on the other. Linting only the primary ABI would license
"fixes" that break the 32-bit build — the shipping ABI for the many 32-bit
Google TV / Android TV boxes this client targets. x86_64 is skipped: it is
emulator-only and shares its width with arm64, so it costs lint time for no
signal the other two do not already carry.

The five resident lints:

* `audio.rs` / `mic.rs` `type_complexity` — the open-attempt closures now return
  named `OpenedPlayback` / `OpenedCapture` aliases. The two tuples are mirror
  images of each other (playback sends, capture receives), which the aliases now
  say out loud.
* `vsync.rs` ×2 `unnecessary_cast` — **not** taken. `timespec`'s fields are
  32-bit on armv7 and 64-bit on arm64, so the casts are REQUIRED on one shipping
  ABI and redundant on the other; following the suggestion would break the
  32-bit build. `i64::from`/`.into()` do not escape it either, they trade
  `unnecessary_cast` for `useless_conversion` on the 64-bit side. Answered with
  a documented `#[allow]` at the expression instead of in whichever build breaks
  first.
* `pad_audio.rs` `needless_range_loop` — iterator form, preserving the
  `channels < 2` no-op the range had.

Verified: `:kit:cargoNdkClippy` green on both ABIs, host-lane clippy for the
crate still clean, `cargo fmt --all --check` clean. The gate was proven
non-vacuous by planting `1i32 as i32` in an android-only module and confirming
it fails the task, then reverting.
2026-08-08 00:01:39 +02:00
enricobuehler 70e6b80200 fix(client/android): place audio with the picture on Android too
The core, Linux, Windows and host halves of the audio latency overhaul landed
with Android deliberately left inert: `JitterPolicy`'s sync target defaults to
`None`, so this ring kept behaving exactly as it always had. What was missing
was not the loop but its REFERENCE — nothing here published where a frame
actually reached glass, and a controller with no reference is the mechanism you
can prove is present but that cannot act. This wires both halves.

The decode thread now reads the host capture `pts_ns` that every `AudioPacket`
has always carried and that this client, like every other, dropped on the floor.
Against the ring depth (published by the AAudio callback through the shared
`AudioSyncCell`) and the video plane's end-to-end figure it computes

  audio_e2e = (now + buffered_ahead + clock_offset) − pts_ns
  av_offset = audio_e2e − video_e2e        (> 0 ⇒ audio behind the picture)

and asks the ring for a depth that closes it. Only ASKS: `set_sync_target` is
clamped between the underrun-driven adaptive floor and the hard cap, so a link
whose jitter genuinely needs more buffer than the picture is away keeps its
buffer and the residual is reported instead of being taken out of the listener's
stream. Continuity outranks sync, on this ring as on the others.

The reference comes from `DisplayTracker`'s `OnFrameRendered` callback — the one
place in the client that knows a frame truly latched — and it is computed ABOVE
the HUD gate now. A sync loop that only ran while the overlay was up would be
off on exactly the devices that report latency; the stats LOCK stays gated,
which is what that early-return was really protecting. Both decode loops feed
it, so sync works with "Low-latency mode" off as well.

Two deliberate refusals:

* The figure is published RAW. The HUD shaves the OS present floor off its shown
  display/end-to-end numbers — metrics report what Punktfunk controls — but sound
  has to reach the ear when the light reaches the eye, and a floor-shaved
  reference would place audio a whole latch period early on every device.
* Below API 33 there is no render callback, so there is no confirmed present and
  the loop stays inert (target `None` ⇒ today's behaviour exactly). The release
  instant is NOT substituted for it: a release targets a FUTURE vsync and runs a
  whole latch period (8-21 ms measured) ahead of glass, well outside the loop's
  deadband — it would place audio early on every frame while looking like it was
  working.

The plane is also no longer invisible. Ring depth and the smoothed offset ride
the stats array at 33/34 and the Detailed HUD carries `audio buffer N ms · a/v
±N ms`, the same wording the desktop HUD uses — both numbers, because a deep ring
on a jittery link is correct behaviour and only the offset separates that from
audio simply held late. The 1 Hz logcat line gains `av_ms` beside its depth, and
the depth itself now has ONE publisher: the counter copy is gone in favour of the
sync cell both readers already share.

The escape hatch is two levers. `PUNKTFUNK_NO_AV_SYNC=1` keeps the contract the
desktop clients document, but an app launched from the launcher inherits no
environment, so the one a field tester can actually reach is
`adb shell setprop debug.punktfunk.no_av_sync 1` — no rebuild, exactly like
`debug.punktfunk.presenter`. A loop that steers playback has to be bisectable on
the device that reports the regression.

Verified: `cargo ndk -t arm64-v8a check` clean; `cargo clippy -p
punktfunk-client-android --all-targets -- -D warnings` clean on the host lane CI
lints, and the Android target introduces no new findings (5 pre-existing lints in
audio/mic/pad_audio/vsync are unchanged — the android-gated modules are never
linted by the host workspace); `cargo fmt --all --check` clean;
`./gradlew :app:testDebugUnitTest` green. The new HUD test was proven
non-vacuous by planting the defect first — dropping the render call fails its
three positive assertions and leaves the three absence assertions passing, which
is the shape a test that "passes for the wrong reason" would not have.

design/audio-latency-overhaul.md W4. Apple (W6) still keeps today's behaviour.
2026-08-07 23:51:15 +02:00
enricobuehler 12a5318397 fix(audio): place audio with the picture instead of wherever the ring settles
The host stamps `pts_ns` on every audio datagram and the client decoded it
into `AudioPacket` — and then never read it. Video's `pts_ns` is used end to
end (the presenter computes a true glass-to-glass `displayed + clock_offset −
pts`), so audio free-ran at whatever depth its jitter ring happened to reach,
video was presented on an independent path, and nothing ever compared them.
The A/V offset was an accident of buffer depths: it moved whenever the ring
ratcheted under underrun pressure, and it got WORSE every time video got
faster, because a quicker decoder lowers the video leg and leaves audio's
exactly where it was. That is what a field report on the Steam Deck heard as
"the audio delay is way too high", and it is why shaving milliseconds off the
audio budget had not helped.

Video is the master. In a game streamer the video leg is the input-feel budget
and must never be inflated to satisfy the audio clock, while audio tolerates
small crossfaded corrections that are inaudible — and `crossfade_drop` already
applies them. So audio moves:

  audio_e2e = (now + buffered_ahead + clock_offset) − pts_ns
  av_offset = audio_e2e − video_e2e        (> 0 ⇒ audio behind the picture)

`AvSync` smooths that with an EWMA, ignores what sits inside a deadband no
listener can detect, refuses the implausible outright rather than clamping it
(a wall-clock step must not steer the ring), and proposes a depth.

Continuity outranks sync, always. `JitterPolicy::set_sync_target` only ever
takes a REQUEST, clamped between the existing underrun-driven floor and the
hard cap. A link whose jitter genuinely needs more buffer than the picture is
away keeps its buffer and the residual is reported — sync can never starve the
ring into dropouts. `None` is the default and reproduces the previous behaviour
exactly, so the four client rings can adopt this one at a time without
diverging.

Two upstream defects found on the way, both prerequisites:

* The host stamped `pts_ns` at ENCODE time, inside the loop draining an
  already-accumulated chunk, so every frame of a chunk carried near-identical
  timestamps describing when we got round to encoding. Harmless while nothing
  consumed it; a sync loop regulating against it would regulate against a
  fiction. It now comes off the capture clock.
* The host did not pace. One capture callback hands over a whole quantum — 5 ms
  when the graph honours our ask, 21.3 ms on a VM, where stock PipeWire raises
  `min-quantum` to 1024 — and the loop drained all of it into back-to-back
  `send_datagram` calls. The wire carried a 4-5 frame burst then ~21 ms of
  nothing, and a ring can only absorb that by standing a burst period deep.
  Frames now leave on the audio clock, which costs no average latency.

And the reason none of this was visible: `buffer_ms`/`target_ms` existed only
as a `tracing::debug!` line, absent from `Stats`. On a Deck the client runs
under Steam's `reaper` with stdout on a pipe nobody can read, so the one number
identifying a deep ring was unobtainable on the device reporting the latency.
The HUD now carries `audio buffer N ms · a/v ±N ms` — both, because a deep ring
on a jittery link is correct and only the offset separates that from audio held
late. The host also reports its negotiated quantum against the one it asked
for, per capture open rather than once per process.

Verified: 364 core + 40 presenter tests on Linux, clippy -D warnings clean on
punktfunk-{core,host} + pf-{client-core,presenter}, fmt clean. New tests pin
the safety invariant (sync cannot pull the target below the continuity floor on
any preset), that `None` leaves the policy bit-identical, and that a device
quantum exceeding the hard cap does not panic `Ord::clamp` inside a realtime
callback.

Android and Apple keep today's behaviour (the `None` default) until their
presenters publish a video figure to align against; design/audio-latency-
overhaul.md carries the plan.
2026-08-07 23:33:45 +02:00
enricobuehler 1482e6b373 docs(client): all four VAAPI legs have decoded — the evidence table said two never had
The H.264 and H.265 rows still read "NEVER decoded a frame on any hardware".
That stopped being true on 2026-08-07, in the same session that proved AV1:
every access unit of the vendored H.264 (250), H.265 (250) and HEVC Main 10
(50) vectors was accepted on .25 (Radeon 780M, RDNA3, Mesa 26.0.3) with no
decode error — NV12 for the 8-bit legs, P010 for Main 10, all on the same tiled
AMD modifier — and probe_this_machines_libva reports VLD decode for all three
profiles.

The row records the delivered counts honestly rather than rounding them up:
225/204/45 against 250/250/50 access units, because `finish` shows
`outputs.last()` and drops the other pictures an AU bumps, and nothing flushes
the DPB at end of stream. That is this rung's own behaviour — D3D11VA delivers
all 250 — and it is invisible on punktfunk's zero-reorder host output. It is
recorded and asserted rather than fixed: changing the one-frame-per-AU contract
touches the pump's deliverable queue, an end-of-stream flush, and the
`keyframe`-labels-the-access-unit defect in the same function, so it belongs in
a commit that moves all three.

Still `verified = false` for all four, and the note says why in the words the
unproven-rung test requires: never frame-hash parity-checked. That is not
pedantry — the D3D11VA AV1 row two lines above is a rung that decoded 250
frames and produced wrong pixels for every one of them. Parity is what
distinguishes them, and this rung exports a tiled dmabuf with no CPU-readable
image, so it needs a readback path nothing has written yet.
2026-08-07 23:13:17 +02:00
enricobuehler 79afa9ce79 Merge branch 'fix/hevc-lowdelay-parity-gate' into integration/decode-aliasing-program 2026-08-07 23:12:22 +02:00
enricobuehler bb9f482b2c Merge branch 'fix/vaapi-decode-target-aliasing' into integration/decode-aliasing-program 2026-08-07 23:12:06 +02:00
enricobuehler dc116d28ca Merge branch 'fix/vaapi-h264-h265-hardware-proof' into integration/decode-aliasing-program 2026-08-07 23:12:02 +02:00
enricobuehler d25a20a233 feat(vkdecode): the AV1 rungs meet a second tile for the first time
Every AV1 frame either decode rung has ever been measured against is `tile_cols =
tile_rows = 1`. The vendored vector is single-tile on all 274 of its frames, so every
tile array the conversions fill — `tiles.widths`, `tiles.heights`, the per-tile records
— had only ever been written at index 0, and a conversion that wrote tile 0 and left
the rest zero would pass the whole suite. Our encoder splits 4K into TWO TILE ROWS.

**The fixture.** `lowdelay-3840x2160.ivf.av1`, 261 KB, 60 frames — `punktfunk-host
spike --source synthetic --codec av1 --width 3840 --height 2160 --fps 60 --seconds 1
--bitrate 1` on .21 (NVENC, RTX 5070 Ti), wrapped to IVF with `ffmpeg -f obu … -c copy`
so `common::split_av1_aus` (the vendored parser's own `IvfIterator`) frames it exactly
as it frames the vector, with no second splitter that could disagree.

**4K is not a size choice, it is the only shape with the property.** Measured on the
same box with the same command: 1280x720, 1920x1080 and 2560x1440 all give `tile_cols =
tile_rows = 1`; 3840x2160 gives `tile_cols = 1, tile_rows = 2` with
`width_in_sbs_minus_1 = [59]`, `height_in_sbs_minus_1 = [16, 16]`, and both tiles in ONE
Tile Group OBU. 60 frames instead of 120 pays for the resolution: 261 KB, under both the
282 KB H.264 and 270 KB H.265 low-delay fixtures.

Goldens are libavcodec's software decode, cross-checked between ffmpeg n8.1.2 (Arch
x86_64, libdav1d) and 8.1.1 (Homebrew, macOS arm64, libdav1d) whose 746,496,000-byte raw
outputs are BYTE-IDENTICAL, not merely equal per frame. 60 of 60 digests distinct.

**AV1's frame accounting is asserted, never derived.** The vendored vector is 250
temporal units carrying 274 coded frames of which 24 are hidden; this stream is 60 units,
60 coded, 60 shown, 0 hidden, 0 `show_existing_frame`, 1 key frame. Neither is the
general case, so both parity harnesses now take units / decoded / shown as three
independent parameters instead of computing one from another, and the CPU guard states
all six numbers.

**A CPU gate that needed no hardware at all.** `pic_av1`'s new
`a_two_tile_frame_fills_both_row_entries_and_leaves_the_rest_zero` pins the second row
entry against its OWN `height_in_sbs_minus_1`, requires the two rows to tile the frame
exactly, and requires TWO tile RECORDS out of ONE tile group with rows (0,0) and (1,0) —
the transposition a square grid could never reveal — each spanning real bytes. The
existing one-tile test asserts index 0 is right and `1..` are zero, which a broken
multi-tile conversion also satisfies.

⚠⚠ **This is a file, and on AV1 that distinction has already cost a release.** "250/250
delivered frames bit-identical to libavcodec" was true for the entire period the host was
shipping only the FIRST TILE of every 4K frame: the verification ran against a vendored
file while the truncation lived in packetisation, and the suite stayed green throughout.
This fixture closes the multi-tile gap on the DECODE rungs and closes nothing about
fragmentation, reassembly, loss or AU boundaries — the golden header, both module docs
and the leg docs all say so, at length, so the next reader does not inherit the same
false confidence.

Legs: `low_delay_host_av1_every_frame_hashes_bit_identical_to_libavcodec` on the Vulkan
rung (11 ignored legs now) and on the D3D11VA rung, plus two non-ignored CPU tests.

Verified: 11/11 Vulkan parity legs on .21 (RTX 5070 Ti, 610.57.04), the new one 60/60
bit-identical; workspace clippy `-D warnings` and `cargo fmt --all --check` clean on .21.
2026-08-07 23:09:12 +02:00
enricobuehler de1bfdc0c5 Merge pull request 'The console stops borrowing the browser's controls — and its preset tiles stop landing all at once' (#100) from worktree-web-console-sweep into main
ci / bun-nix (push) Successful in 32s
ci / docs-site (push) Successful in 1m12s
ci / web (push) Successful in 1m14s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 13s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 8s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
ci / rust-arm64 (push) Successful in 2m28s
windows-host / package (push) Failing after 1m31s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m37s
deb / build-publish-client-arm64 (push) Successful in 4m16s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
deb / build-publish-host (push) Successful in 5m27s
docker / builders-arm64cross (push) Successful in 6s
deb / build-publish (push) Successful in 5m37s
ci / rust (push) Successful in 7m27s
arch / build-publish (push) Successful in 8m48s
docker / deploy-docs (push) Failing after 6m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m44s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m22s
Reviewed-on: #100
2026-08-07 21:08:42 +00:00
enricobuehler 3be7d1d4f8 feat(web): the console asks its own questions
ci / bun-nix (pull_request) Successful in 30s
ci / web (pull_request) Successful in 1m7s
ci / docs-site (pull_request) Successful in 1m12s
ci / rust-arm64 (pull_request) Successful in 1m36s
ci / rust (pull_request) Successful in 6m32s
Follow-up to a85e8452, closing the three items that sweep flagged and left.

SIXTEEN BROWSER DIALOGS, GONE. Every destructive action in an otherwise fully
branded console handed off to `window.confirm` — a grey OS box with the page's
URL in it, no brand, no red on a delete, and untouchable by any story or
screenshot, which is part of why it survived this long.

They are replaced by one promise-based surface (components/dialogs.tsx) rather
than a dialog per call site. The native calls were EXPRESSIONS — `if
(!confirm(…)) return;` — threaded through mutation handlers; rewriting each into
"hold the pending action in state, render a dialog, run it from onConfirm" would
have put dialog machinery in every section file and turned each linear handler
inside out. Returning a promise keeps them the shape they already were, and it
is what let the navigation guard come along too: TanStack's `shouldBlockFn`
accepts `Promise<boolean>`. `beforeunload` necessarily stays native — a reload
is the browser's dialog to draw, and it will not wait on ours.

No warning copy was rewritten. Each message was SPLIT at its existing sentence
boundary: the question becomes the dialog's title, the consequence its body,
and "Continue?" is dropped where the affirmative button now carries the verb
("Delete", "Uninstall", "Unpair", "Stop every session"). 16 new keys, en and de
in parity at 629.

Verified by driving the real dialogs in a headless browser — all seven contract
checks pass, including the two that would be invisible until they bit: Escape
SETTLES the promise (an unsettled one would hang a mutation handler forever with
no error), and a cancelled prompt resolves null rather than "", so a caller can
still tell "backed out" from "cleared the field".

FOUR OF THE SEVEN NUMERIC FIELDS became InputNumber; three deliberately did not,
and now say why in place. The layout X/Y pair had a real defect: a screen left
of the origin has a negative coordinate, and `Number("-") || 0` rewrote the lone
minus sign to "0" before the digits could be typed. Measured on the built page:
the field can now be emptied to retype instead of snapping to its floor, and 900
in a 1..=16 field clamps to 16. The three left alone cannot take it — the grace
seconds field writes to the HOST on blur (InputNumber commits while typing, so
its clamp would race the apply), and the library's year/players are OPTIONAL,
where `value: number` has no way to say "unset" and would invent a year for
every entry without one.

The select's highlighted row moves off @unom/ui's neutral grey onto the brand
wash the nav and the preset cards already use.

The Displays story earned its keep immediately: adding `useDialogs` to that page
broke it in Storybook, because the provider was mounted in __root and nowhere
else. It belongs beside the other app-level providers in .storybook/preview.
2026-08-07 22:56:46 +02:00
enricobuehler e8a7a1e6af fix(client/vaapi): the third rung does NOT alias — and now it cannot start to
The D3D11VA and Vulkan rungs both decoded into a surface they were predicting
from, on 117 of 120 access units of our own host's low-delay H.264 (`1c54d099`
for AV1, `834b2443` for H.264). `pf-vaadec` feeds `reference_frames` from the
same `plan.dpb_refs` snapshot, releases its whole `removed` list inline exactly
as the two broken conversions did, and neither fix commit touched it. It is
still exempt — this is the evidence, and the thing that keeps it true.

**Measured on the CPU, no GPU needed.** `walk_for_aliasing` drives the planner
and `plan_to_va` over both streams and counts four shapes. On
`lowdelay-640x480.h264` the aliasing PRECONDITION is fully present: 117 of 120
access units remove a picture their own `dpb_refs` still names, and on the same
117 the setup picture is handed the slot of a picture that access unit READS —
the D3D11VA/Vulkan defect verbatim, in this conversion, today. On the vendored
conformance vector both counts are 0, which is why that vector proved nothing
on two other backends for two milestones. Aliased submissions: **0 on both**.

**Why.** A slot is not a surface here. `plan_to_va` never invents one — every
reference it can name is read out of the `surfaces` table it is handed — and
the decode target is a separate parameter the caller takes from OUTSIDE that
table. `setup_surface` reaches the submission at exactly one field per codec
(H.264/H.265 `curr_pic.picture_id`, AV1 `current_frame` and
`current_display_picture`); HEVC is doubly safe, because its per-slice
`RefPicList` stores an INDEX into `reference_frames` rather than a surface.
AV1's documented substitution fallback is the one place the target can be named
as a reference, and only where the store resolved nothing at all to prefer.

**The exemption was incidental; it is structural now.** It needs the reference
table and the decode target to come from ONE snapshot of the bindings, and the
rung had that only by writing `free_surface()` and `surface_table()` adjacently
at three call sites. Split them and this rung acquires the defect exactly: the
table must be the PRE-removal one (that is where the references are), while a
free list consulted after the removals offers precisely the displaced picture's
surface. `Session::acquire_target` now returns the index, the surface and the
table together from `&self`, so a later edit cannot move one call and not the
other. No behaviour change: same order, same values, same refusal message.

Tests. `no_submission_names_its_decode_target_as_one_of_its_own_references`
(both streams, 0) with
`taking_the_decode_target_from_the_slot_table_aliases_on_the_low_delay_stream`
as the counterfactual that reproduces the defect on 117 of 120 — so the walk
demonstrably CAN see it when it is there.
`the_low_delay_stream_reassigns_slots_whose_pictures_it_still_reads` pins 0/250
and 117/120 so neither can drift silently.
`the_decode_target_can_never_be_a_surface_the_reference_table_names` sweeps
every binding state a 4-surface/3-slot pool can hold, and
`taking_the_free_surface_after_the_removals_would_hand_out_a_referenced_surface`
is the ordering counterfactual.

⚠ One existing test lost a VACUOUS half.
`the_setup_picture_routinely_inherits_a_just_freed_slot` asserted the decode
target was never also a reference while handing every picture its own
never-reused surface id — distinct integers cannot collide, so that assertion
could not fail whatever the conversion did. Its real measurement (225 of 250
access units reuse a just-freed slot, which is why the target is a parameter)
is kept; the collision half is gone, and the doc says where the question is
actually answered and why a recycling pool is what it takes to answer it.

Gates, run on `.25` (Radeon 780M, radeonsi, Mesa 26.0.3, VA-API 1.23), this
rung being Linux-only: `cargo fmt --all -- --check`; `cargo clippy -p
pf-client-core -p pf-vaadec --all-targets --features sdl3/build-from-source --
-D warnings`; `cargo test -p pf-client-core --lib --features
sdl3/build-from-source` (171 passed); the same filtered to `video_vaapi_native`
with `--include-ignored` (18 passed); `cargo test -p pf-vaadec` (48 passed).
Plus the pf-lxcheck2 container for the cross-platform half — fmt, clippy and
`cargo test -p pf-vaadec`, all clean.

All four VAAPI legs still decode with the refactor in place, not one access
unit refused: H.264 225 of 250 access units delivering a frame, H.265 204 of
250, HEVC Main 10 45 of 50 (P010), AV1 250 of 250 — the same counts and the
same tiled modifier 0x200000010401b04 those legs recorded before it. ⚠ The
H.26x legs live on `fix/vaapi-h264-h265-hardware-proof`, not on this branch, so
they were run by overlaying that commit's test module onto the scratch tree;
only the AV1 leg and the libva probe are reachable from here. This is a decode
measurement, not frame-hash parity — the rung exports a driver-tiled DRM-PRIME
dmabuf, so there is no CPU-readable image to hash. The alias assertions above
are the real evidence and they need no device.

⚠ NOT taken: `finish`'s `outputs.last()`, which ships one frame per access unit
and drops the rest of what a bump displaces (225/204/45 against 250/250/50),
with no end-of-stream flush. It cannot bite punktfunk — hosts emit zero-reorder
output, so `outputs` never holds more than one picture — and fixing it changes
`decode()`'s one-frame-per-access-unit contract with the pump (it wants a
deliverable queue, which `video_vk_native` already keeps) plus an end-of-stream
flush and the `keyframe`-labels-the-access-unit defect in the same function.
It is recorded and asserted on that other branch, whose three delivered-count
assertions any fix has to move in the same commit; doing that from here, blind
to them, would be worse than leaving it.
2026-08-07 22:55:25 +02:00
enricobuehler f0702f3e06 feat(vkdecode): HEVC's exemption stops being an argument and becomes a vendored stream
`fd6241a2` made HEVC's freedom from the release-ordering defect falsifiable on CPU and
recorded what was still missing: no low-delay HEVC stream was vendored, so the exemption
rested on a structural argument plus one throwaway measurement. This vendors the stream,
and the exemption HELD.

**The fixture.** `lowdelay-640x480.h265`, 270 KB, 120 pictures — `punktfunk-host spike
--source synthetic --codec h265 --width 640 --height 480 --fps 60 --seconds 2 --bitrate 1`
on .21 (NVENC, RTX 5070 Ti, driver 610.57.04). Deliberately the H.264 sibling's resolution
and frame count: the two are then directly comparable, 640 and 480 are both multiples of
MinCbSizeY so there is no conformance window and a hash mismatch can only be decode rather
than readback geometry, and 270 KB sits alongside the 282 KB already accepted for H.264.
Goldens are libavcodec's software decode, cross-checked BIT-IDENTICAL across ffmpeg n8.1.2
(Arch, x86_64) and 8.1.1 (Homebrew, macOS arm64), 120 of 120 digests distinct.

**The exemption held, measured rather than argued.** `sps_max_dec_pic_buffering_minus1 = 4`
against the four pictures 8.3.2 keeps marked in steady state, `sps_max_num_reorder_pics = 0`,
`numRefL0 = 1` — a five-picture DPB filled exactly by four references plus the current
picture. 115 of the 120 access units retire a picture, and `removed ∩ dpb_refs` is **0 of
120**. A 300-picture 1080p stream from the same host reports the same shape: 295
retirements, 0 intersections. It is the encoder and not the resolution, exactly as for
H.264.

**A zero proves nothing on its own, so the fixture is pinned by its counterfactual.**
`test-25fps.h264` reported zero for two milestones while every stream we ship aliased on
99% of its frames. So the guarantee here is not "we looked and it was fine": hand
`plan_to_dxva_h265` the marked DPB as it stood BEFORE `decode_rps` — the mutation a
snapshot move would cause, reconstructed exactly as `dpb_refs(N-1) ∪ {stored(N-1)}` — and
the alias appears on **115 of 120** access units, driven through the real conversion rather
than through planner arithmetic. If a regeneration ever produced a stream that reordered,
or a DPB deeper than its reference count, that 115 collapses to 0 and the tests say so
instead of continuing to pass.

**The two rungs are exempt for different reasons, and the asymmetry is now a gate.** DXVA
binds the whole marked DPB — `RefPicList` is spec-defined that way, and an RFI long-term
anchor has to survive in it — so its exemption really is `H265Planner`'s snapshot ordering,
one call away from being untrue. `plan_to_vk_h265` never reads `dpb_refs` at all:
`pReferenceSlots` is the slots the operation uses, so it binds the current RPS sets, which
`decode_rps` itself derives and which therefore cannot name a picture that same RPS just
dropped. A new test feeds that conversion the identical widened snapshot and asserts
nothing changes, so a future change making the Vulkan rung bind the marked DPB — a
legitimate thing to want, since a *Foll* anchor invisible to the hardware is the RFI
failure shape — fails loudly instead of silently acquiring the defect.

What the Vulkan pixel leg adds is therefore NOT aliasing coverage, and its docs say so:
it is the first HEVC frame either rung has decoded from our own encoder, under a DPB that
retires and reissues a slot on 115 of 120 access units back to back, where the vendored
vector's reordering keeps that eviction slack.

Legs: `low_delay_host_h265_every_frame_hashes_bit_identical_to_libavcodec` on the Vulkan
rung (10 ignored legs now, up from 9) and on the D3D11VA rung, plus three non-ignored CPU
guards that run in ordinary CI.

Verified: 10/10 Vulkan parity legs on .21 (RTX 5070 Ti, 610.57.04), the new one 120/120
bit-identical; workspace clippy `-D warnings` and `cargo fmt --all --check` clean on .21.
2026-08-07 22:53:22 +02:00
enricobuehler a85e845255 fix(web): the console stops falling out of its own design system
A pre-release sweep of the management console for two things that no type check
and no diff can catch: primitives that were never @unom/ui's, and animation
that a nested motion parent quietly cancelled.

THE PRESET TILES ALL LANDED ON THE SAME FRAME. @unom/ui's <Section> sets
`delayChildren: stagger(...)`, so a page whose cards are direct descendants of
it staggers for free — which is why every page but one looked right. An
<AnimatedCard> is ALSO a motion element and sets no `delayChildren`, and the
Virtual displays preset tiles are cards nested INSIDE that page's config card,
so that card became their timing group. Measured in a headless browser: the
opacity spread between the first and last tile was 0.00 across the whole
animation (six tiles in lockstep), and is 0.98 now — a ~100 ms cascade matching
the rest of the console. The four hand-rolled copies of the stagger container
collapse into one `<Stagger>` that carries the explanation.

FIVE FILES IMPORTED THE WRONG BUTTON. `@unom/ui/button` exports both a plain
`Button` and the `AnimatedButton` that this console's wrapper re-exports under
the same name — so `import { Button } from "@unom/ui/button"` compiles, renders,
and silently opts out of the mount animation and the hover/tap response.
Displays, SessionGame, GPU, Update and PendingDevices had dead buttons sitting
next to live ones.

THREE PRIMITIVES HAD NO WRAPPER, SO NOBODY REACHED FOR THEM. @unom/ui ships
form/select, form/textarea and form/checkbox; components/ui did not, and the
gap was filled with browser-chrome `<select>`, `<textarea>` and
`<input type="checkbox">` in the add-hook modal and both library forms. Select
needs the same token correction Tabs needed — upstream `text-secondary` is a
text colour, but here `--secondary` is a SURFACE, so the trigger's chevron and
placeholder rendered at near-zero contrast on the card behind them.

The hook timeout also stops accepting a value the host rejects: `min`/`max` on
a controlled `<input type="number">` are decoration (no form validation ever
runs), so 900 went into a field capped at 600 and failed later, at run time.
@unom/ui's InputNumber clamps on blur and lets the field be empty while you
retype instead of snapping to the fallback.

Storybook gains the page that had no story at all — the console's largest
config surface, and the reason this shipped unseen. Its <Card> wrapper is load
bearing: it reproduces the motion nesting that IS the bug.
2026-08-07 22:34:24 +02:00
enricobuehler fd6241a24f fix(dxvadec): the review round — a doc that had become false, a warn-storm on renegotiation, and HEVC's exemption made falsifiable
Four findings, all real.

**`SlotMap`'s own docs had become false.** "feed it every `DpbUpdate` in decode order
(via `Self::apply` or `plan_to_vk`, which applies internally)" — `plan_to_vk` no longer
applies internally, which is the entire point of the change, and `release`'s docs named
it as one of the two things that may free a slot. A reader following those docs would
build the next caller wrong in exactly the way this commit's parent fixed. Both now say
which conversions defer, which one does not, and why H.265 is the one that does not.

**The deferred release warned on a legitimate event.** `release_deferred` warned per id
when a deferred release found no slot — but a renegotiation replaces the whole
`Session`, and with it the slot map, INSIDE `plan`, while the planner's own drain
reports every drained picture in that same access unit's `removed`. Every one of those
ids then misses, and nothing is wrong. `debug!`, with the legitimate cause named so the
illegitimate one stays diagnosable.

**HEVC's exemption was asserted only in its consequence.** `the_current_picture_is_
named_by_curr_pic_and_never_aliases_a_reference` checked that no reference shares the
decode target's slot — which on the vendored vector holds whether or not the reasoning
behind it does. That is precisely how the H.264 leg passed for two milestones. The test
now also asserts the PLANNER property the exemption rests on (`removed ∩ dpb_refs = ∅`,
falsified by moving `dpb_snapshot()` above `decode_rps`), and records that the low-delay
measurement was 0 of 300 against H.264's 297 of 300 from the same host and the same run.
It also records what is still missing: no low-delay HEVC stream is vendored, so HEVC's
freedom is a re-derivable argument plus one measurement, not a standing hardware leg.

**Two stale cross-references.** Both AV1 conversions told the reader the H.264/H.265
zero was "measured on reordering vectors and not a proof" — the open question this
commit's parent closed. They now say what the answer was.
2026-08-07 22:19:12 +02:00
enricobuehler 834b244301 fix(client): the H.264 twin was real — every low-delay picture decoded into a surface it predicted from
The AV1 review round flagged the H.264 leg as "plausibly the same defect, traced in
source, not reproduced" and deliberately did not touch it. It is reproduced now, and
it is worse than the AV1 one: it fires on 297 of 300 access units of every stream a
punktfunk host emits, at 720p, 1080p and 2160p alike, on BOTH the DXVA rung and the
Vulkan one.

**Decided on the CPU, no GPU needed.** `H264Planner` snapshots `dpb_refs` in
`begin_picture`, BEFORE `finish_picture` runs 8.2.5's marking and C.4.5.3's bump, so a
picture the sliding window unmarks and the bump then evicts lands in both `dpb_refs`
(which `RefFrameList` is built from) and `dpb.removed`. The conversion released the
whole `removed` list and then assigned the decode target a slot; `SlotMap::assign`
takes the lowest free slot, which is the one just vacated. `CurrPic = N` and
`RefFrameList[k] = N`, in one submission.

The two conditions have to coincide in ONE access unit, and low-delay H.264 is exactly
what makes them: `max_num_reorder_frames = 0` means the evicted picture has already
been output, which is what makes it evictable at all. NVENC seals it by writing
`max_num_ref_frames = 3` ALONGSIDE `max_dec_frame_buffering = 3` — a DPB exactly as
deep as its reference count — so the window unmarks the oldest reference in the very
unit whose bump drops it. The aliased picture is `ref_idx 2` of a three-entry
`num_ref_idx_l0_active` list: addressable by any macroblock, not a spare.

**Why two hardware-proven codecs and four GPUs never saw it.** `test-25fps.h264` is
level 1.3 with no VUI `bitstream_restriction`, so `dpb_limit` falls back to A.3.1's
level ceiling and gives a 7-frame DPB against 2 reference frames — the window unmarks
two units before the bump can evict — and it REORDERS, which keeps an unmarked picture
alive past the unit that unmarked it. Two independent reasons, both properties of that
vector rather than of H.264. It measured zero and passed 250/250 throughout.
`data/lowdelay-640x480.h264` is vendored to close exactly that: our own host's output,
120 pictures, goldens from libavcodec cross-checked bit-identical across two ffmpeg
builds on two architectures.

**The fix is the AV1 fix.** `DecodePlanDxva` and `DecodePlanVk` grow
`release_after_decode`, the conversions hand the removals back instead of applying
them, and the callers release them once the decode op is issued. It costs no slot the
map does not have: `SlotMap::new` allocates `max_dpb_frames + 1` and the DPB never
exceeds `max_dpb_frames`, so a free slot always exists with the whole `removed` list
still held — measured, peak 4 of 4 on the stream that defers on 117 of 120 units.

The Vulkan rung breaks on it in both DPB modes and neither loudly: DISTINCT hands the
aliased reference the same array layer the setup writes; COINCIDE clears
`slot_image[setup]` in the binding sync and the reference then resolves to no bound
image, dropping out of `pReferenceSlots` with a `trace!`. Its deferred release runs on
the FAILURE paths too — the fallible region's Result is held rather than `?`-ed,
because seven exits sat between the conversion and the release and each would have
leaked a slot.

`a_full_dpb_bump_reuses_the_slot_but_the_pool_model_binds_a_fresh_image` asserted the
aliasing as "the planner's normal behaviour": an authored depth-1 stream whose AU1
references the picture it evicts. It now asserts the opposite, which is the defect in
two lines.

New evidence, all of it runnable: the CPU proof pins BOTH numbers (0 on the vector,
117 of 120 on the low-delay stream) so neither can drift silently; the ledger-pressure
test measures the peak; and a low-delay parity leg is added to `pf-vkdecode`'s
`gpu_parity` and `pf-client-core`'s `video_d3d11_native::parity` so both rungs are held
to what they stream rather than only to what they conform to.
2026-08-07 22:09:44 +02:00
enricobuehler 42e4d47743 Merge pull request 'Gyro: the pipeline was wrong end to end — measured against a real controller, and fixed' (#99) from worktree-gyro-p0-correctness into main
ci / bun-nix (push) Successful in 30s
windows-drivers / probe-and-proto (push) Successful in 24s
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 1m23s
ci / rust-arm64 (push) Successful in 1m31s
apple / swift (push) Successful in 1m30s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 11s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 10s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 11s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
windows-drivers / driver-build (push) Successful in 1m37s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
deb / build-publish-client-arm64 (push) Successful in 2m34s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m3s
windows-host / package (push) Failing after 1m37s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m18s
deb / build-publish-host (push) Successful in 4m14s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m27s
deb / build-publish (push) Successful in 6m52s
docker / builders-arm64cross (push) Successful in 14s
android / android (push) Successful in 9m13s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m40s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m15s
ci / rust (push) Successful in 11m20s
release / apple (push) Successful in 9m47s
flatpak / build-publish (push) Successful in 7m44s
arch / build-publish (push) Successful in 12m13s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m8s
docker / deploy-docs (push) Failing after 6m20s
apple / screenshots (push) Successful in 5m49s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m45s
Reviewed-on: #99
2026-08-07 19:51:17 +00:00
enricobuehler 5aeb8d2552 fix(client): a failed AV1 decode left the surface's facts saying it holds the last picture
The `damaged` path has cleared `Session::held[setup_slot]` since M7, for a reason
that now applies to the failure path too: the slot map says the surface holds THIS
picture while the surface still carries whatever the previous occupant decoded, so
a later `show_existing_frame` naming it blits the old picture's pixels with the old
picture's geometry and colour. The failure path never reached that far before —
`decode_into`'s error returned straight out of `frame_av1` — and the previous commit
made it continue so the slot releases could run.
2026-08-07 21:36:16 +02:00
enricobuehler 3a4c94ad79 fix(dxvadec): the review round — a vacuous predicate, an overstated claim, and the H.264 twin of this defect
Five findings from the adversarial pass, all real.

**The deferral predicate was vacuous.** `plan.dpb.removed` is ALWAYS a subset of
`plan.dpb_refs`: `Av1Planner::plan_frame` snapshots `dpb_refs` before any mutation
and `refresh_slots` can only report a picture that was in `self.slots` at that
moment. So `filter(|id| dpb_refs.contains(id))` was a condition that is never
false, the eager-release loop beside it could never release anything, and the test
assertion "only a picture the submission points at earns the reprieve" could never
fire. Now: defer every removal, say why in terms of the planner, and assert the
PLANNER's property (`removed ⊆ dpb_refs`) — which is falsifiable, and whose failure
would mean the conversion is releasing a surface `ref_frame_map` points at.

**The failure-path claim was overstated.** Holding the decode's `Result` closes
this frame's leak, not the unit's: `decode_av1` returns on the first failing frame
and abandons the rest of the temporal unit's plans, so their removals are never
released. 24 of 250 units carry a second frame. Named rather than fixed — what to
do with the frames after a failure is the pump's question.

**⚠⚠ The H.264 leg plausibly has the same defect, and the comment this change added
said it could not.** `pic.rs` builds `RefFrameList` from `plan.dpb_refs`, and
`H264Planner` snapshots that in `begin_picture` — BEFORE 8.2.5 marking and the DPB
bump. The vendored bump drops a picture the sliding window just unmarked once it
has been output, so a picture can land in both `RefFrameList` and `dpb.removed`:
the AV1 aliasing shape exactly. Measured zero on the vendored vector — but that
vector REORDERS, which is precisely what keeps an unmarked picture alive past the
AU that unmarked it. A punktfunk host emits LOW-DELAY H.264, where output happens
as each picture is decoded, which is the condition that makes eviction and
unmarking land in the same access unit. Traced end to end in source, not
reproduced (no low-delay vector). NOT fixed: changing a hardware-proven codec on an
unreproduced suspicion is the worse risk two commits before a release. Instead
`no_au_removes_a_picture_its_own_reference_list_names` makes the assumption
falsifiable, and its message says what to do when it fires. HEVC is structurally
safe and now says why: `H265Planner` snapshots `dpb_refs` AFTER `decode_rps`.

**Four more stale promotion sites**, past the four already fixed: `Backend::
NativeD3d11va`'s variant doc, `Decoder::new`'s Windows rung comment, `lib.rs`'s
module note and `clients/session/README.md`. Two sites that used the AV1 leg as
the live EXAMPLE of an unproven rung are marked as expired rather than deleted —
the reasoning is what the next bad-evidence leg will need.

**The AV1 dump was missing.** `PF_DXVA_DUMP` wrote h264 and hevc only, for the one
codec whose libavcodec capture has never been taken and where the dump is
therefore the only tool.
2026-08-07 21:34:59 +02:00
enricobuehler ee61e8c9ba fix(clients/pads): the phone mirror never needed the controller path's frame change
ci / bun-nix (pull_request) Successful in 30s
ci / docs-site (pull_request) Successful in 1m16s
ci / web (pull_request) Successful in 1m22s
apple / swift (pull_request) Successful in 1m33s
apple / screenshots (pull_request) Skipped
windows-drivers / driver-build (pull_request) Successful in 1m38s
ci / rust-arm64 (pull_request) Successful in 2m12s
windows-drivers / probe-and-proto (pull_request) Successful in 21s
android / android (pull_request) Successful in 3m39s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m11s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m14s
ci / rust (pull_request) Successful in 6m32s
Reverts half of 1eab4b66 and closes G10's open frame question, both settled by the same
measurement.

1eab4b66 made two corrections to the Apple phone-gyro mirror. The negation was right and
stays: Apple reports the gravity VECTOR, pointing down, while an accelerometer measures
proper acceleration, pointing up at rest, and the wire carries the latter. The frame
change was wrong, and this removes it.

The mistake was a name collision. Two different frames are both called "the controller
frame". GCMotion reports a CONTROLLER in (Right, Forward, Up) — measured on a real
DualSense — which is not the wire's frame, which is why `GamepadCapture.forwardMotion`
converts. The mirror's orientation remap resolves THIS DEVICE into the frame its header
describes, x right, y up, z out of the screen. For the pose that mirror exists to serve —
a phone clipped upright with the screen facing the player — "out of the screen" points at
the player, so that frame is (Right, Up, Backward), which IS the wire's. It was already
correct. Applying the controller path's conversion on top rotated it out of true: a phone
sitting still would have reported gravity as −1 g on the roll axis rather than +1 g up,
i.e. claimed to be lying on its edge.

Reasoning by analogy is what produced it — "the mirror says controller frame, the capture
path says controller frame, so the same fix applies". Both files say it; they mean
different things.

What caught it was measuring the Android twin, which does the same thing straight through.
On glass: a DualSense on Bluetooth to a phone, streaming to a Linux host, reads +1 g on the
up axis end to end. Had the Apple mirror needed a conversion, the Android one would have
needed the same one and would have been visibly wrong. It is not.

The same run settles G10's frame, which shipped straight-through and explicitly unverified
because nobody had put a Bluetooth pad in front of the platform sensor framework. Now
somebody has. `PadSensors`' own first-sample log read `accel 0, 10000, 0` — exactly 1 g on
slot 1 — and at the far end hid-playstation published gravity as +0.991 g on ABS_Y, with
every rotation driving its correctly-named axis and the signs agreeing with gravity's
independent witness on 95 of 100 rotating samples. Android hands a controller's sensors
over in the pad's own frame, as documented. No remap, and the comment now says measured
instead of assumed.

Worth recording why the earlier suspicion was wrong, since it is the same trap in the other
direction: Android's DEVICE sensor frame really does put +z out of the screen, so a flat
phone puts gravity on z — but a CONTROLLER's sensors are reported in the controller's
frame, not the phone's. One platform, two conventions, chosen by what the sensor is
attached to.

Gate: Apple macOS `swift build` + full suite (215 tests, 5 skipped, 0 failures) and the
iOS-triple typecheck, which is what actually compiles `DeviceGyro.swift`; Android
`:kit:compileDebugKotlin`, `:kit:testDebugUnitTest`, `:app:compileDebugKotlin`. Green.

Still owed: `DeviceGyroRemap`'s four orientation matrices remain derived — this run used a
controller's own sensors, not the mirror, so it says nothing about them. They need a
gyro-less pad on wire index 0 and a phone turned through all four orientations.
2026-08-07 21:28:13 +02:00
enricobuehler af4d265168 fix(client): the fourth site that swore the DXVA AV1 leg fails parity, and a clippy lint
`the_evidence_table_says_exactly_which_rungs_have_run_on_hardware` asserts the
same fact a third way — a proven list and a NOT-proven list, both spelled out —
so promoting the rung in the three places the handoff named still left a test
saying "the DXVA AV1 leg FAILS parity on two GPUs — claiming otherwise is the
dishonesty this program must not ship". It was right to fail; the pair moves
lists here.

Three prose sites that still described the leg as decoding wrong pixels move
with it: `native_supports_av1`'s device-facts note, `log_rung`'s honesty-surface
docs, and the OPEN question in the Windows Intel arm of `pick_native` — that last
one is marked CLOSED rather than deleted, because the question it raised (the
evidence filter asks "any evidence", and has no answer for BAD evidence) is a
real gap in the rule that outlived this particular leg.
2026-08-07 21:10:03 +02:00
enricobuehler a29e366b3e feat(vaapi): VAAPI decodes H.264, H.265 and Main 10 — their first frames on any hardware
The evidence table said these legs "have still never decoded a frame anywhere",
and VAAPI is the rung every Linux AMD/Intel client lands on. They have now
decoded, on `.25` (Radeon 780M / Phoenix1, RDNA3, radeonsi, Mesa 26.0.3, VA-API
1.23, /dev/dri/renderD128):

  H.264         250/250 access units accepted, 225 frames delivered, NV12
  H.265         250/250 accepted, 204 delivered, NV12
  HEVC Main 10   50/50 accepted,  45 delivered, P010
  (AV1, unchanged: 250/250 accepted, 250 delivered, NV12)

all on the same tiled AMD modifier (0x200000010401b04). Not one access unit of
any vector was refused.

Three `#[ignore]`d legs modelled on the AV1 one, plus the Annex-B access-unit
splitters they need — ported verbatim from `video_d3d11_native`'s test module so
the two platform rungs are driven over the same access units rather than over two
splitters free to disagree. Main 10 earns a third leg rather than a variation on
the second: ten bits is a different VAAPI profile, a different render-target
format and a different surface fourcc, and that leg's fourcc assertion is the
only thing that would catch a driver quietly handing back NV12 for a ten-bit
stream.

This is NOT frame-hash parity, and the doc comments say so rather than letting
the test names imply it. The Vulkan and D3D11VA legs hash every frame against
libavcodec because both can read their decoded surface back; this rung exports a
DRM-PRIME dmabuf whose memory the driver tiles, so there is no CPU-readable image
to hash without a `vaDeriveImage`/`vaGetImage` path production neither uses nor
wants. What these legs prove is that every access unit is accepted, that the
expected number of frames comes back, and that each one is a real exported
surface of the right shape and fourcc — enough to turn "never decoded a frame
anywhere" into a measurement, not enough to promote the rung to `verified`.

Two findings the run surfaced, neither of which bites punktfunk's own streams:

* The delivered counts are 225/204/45, not 250/250/50, and that is the RUNG, not
  the driver. `finish` shows `outputs.last()` and never more, so an access unit
  whose plan bumps several pictures out of the DPB displays the last and drops
  the rest — 18 dropped at the H.264 vector's three draining IDRs, 45 on the
  H.265 vector's 45 two-picture bumps — and there is no end-of-stream flush.
  Hosts emit zero-reorder low-delay output with no B pictures, so `outputs` never
  holds more than one picture in the field. A CPU-only test derives all three
  counts from the planner alone, on any Linux box with no GPU, so they stay
  explanations rather than recordings.

* `DmabufFrame::keyframe` labels the ACCESS UNIT, not the picture delivered:
  `finish` is handed the current AU's `is_idr`. On a reordering stream the IDR is
  bumped out several access units after it decoded and arrives flagged `false`,
  while the access unit that drains the DPB at a later IDR flags whichever old
  picture it displays as a keyframe. That flag is `DecodedImage::is_keyframe`,
  the pump's post-loss re-anchor signal. Asserted so that fixing it is noticed,
  not so that it is preserved.

Gates, all run on `.25` (this rung only compiles on Linux): `cargo fmt --all --
--check`; `cargo clippy -p pf-client-core --all-targets --features
sdl3/build-from-source -- -D warnings`; `cargo test -p pf-client-core --lib
--features sdl3/build-from-source` (169 passed); the same filtered to
video_vaapi_native with `--include-ignored` (16 passed). Plus the pf-lxcheck2
container's workspace-wide `cargo fmt --all -- --check` and `cargo clippy
--workspace --all-targets -- -D warnings`, both clean.

The evidence table in `video.rs` still says these legs have never decoded a
frame. It is being edited concurrently, so its replacement row is handed over
rather than raced for here.
2026-08-07 21:07:48 +02:00
enricobuehler f4dda9074b feat(dxvadec): the AV1 picparams harness AV1 forgot, and the D3D11VA AV1 rung is promoted
Two halves.

**The harness.** `libav_picparams_parity` covered H.264 and HEVC only, which is
exactly the gap that let a wrong AV1 submission ship. It now plans, converts and
packs all 274 frames of the vendored AV1 vector and checks what needs no capture:
the three-buffer descriptor set with no quantization matrix (AV1's matrices are
selected by index, so `dxva2_av1_end_frame` passes NULL/0 and there is no buffer
to submit), no macroblock count anywhere, the 912-byte picture-parameter buffer,
and the tile records — which unlike H.264/HEVC slice records do NOT abut, because
a `DXVA_Tile_AV1` addresses a tile PAYLOAD and consecutive payloads are separated
by their `tile_size_minus_1` fields.

The one that matters most is `no_av1_submission_names_its_decode_surface_in_the_
reference_store`: the invariant the previous commit fixed, over the submitted
BYTES rather than over the plan. libavcodec cannot produce that shape — it fills
`RefFrameMapTextureIndex` from the pre-refresh store and takes
`CurrPicTextureIndex` from a frame the reference update has not run on — which is
the argument for calling it a defect rather than a convention.

`AV1_FIELDS` reaches into the eight nested blocks (`tiles.widths`,
`segmentation.feature_data`, …) so a future capture reports a field and not "260
bytes of tiles differ"; `field_table!` grew nested-path support for it. The
`#[ignore]`d `our_av1_picture_parameters_match_libavcodecs` and the capture recipe
are in place, and `the_dump_and_the_parser_agree…` now self-compares AV1 too.

⚠ NO libavcodec AV1 capture was taken and the module docs say so rather than
leaving an absent result to be read as a pass: `.221` has no MSYS2, no gcc and no
make, so a patched FFmpeg there is a toolchain bring-up, not a build. Everything
this file claims about libavcodec's AV1 side is READ out of `dxva2_av1.c` (n8.1).
That reading did turn up one live divergence, recorded at `pic_av1.rs`'s
`pp.width` and deliberately NOT changed: libavcodec sends `avctx->width`, which is
FrameWidth (pre-superres), where this crate sends UpscaledWidth. The two are equal
whenever superres is off, which is every stream that exists here, so the 250/250
result says nothing either way and a blind change would be unmeasured.

**The promotion.** `(D3d11va, CODEC_AV1)` is `verified` — 250/250 delivered frames
bit-identical to libavcodec on an RTX 3500 Ada AND an Intel Arc. All three places
move together: the evidence arm, the module table and
`every_rung_runs_and_the_unproven_ones_are_named`, whose `unproven` array loses the
pair and whose proven list gains it.

⚠ This changes rung SELECTION, not just a label. `verified` is what lets `auto`
pick D3D11VA ahead of Vulkan Video, so Windows Intel and unknown-vendor boxes —
where the ladder is `native-d3d11va → native-vk → sw` — now decode AV1 on D3D11VA
where they previously fell to Vulkan. Taken deliberately: ~10x the Vulkan leg's
speed, and the parity that promoted it was measured on an Intel Arc, which is the
vendor family the change moves. Still no soak on the goldens, and the notes say so.

Also: `frame_av1` holds the decode's `Result` instead of `?`-ing it, so both slot
releases run on the failure path. `decode_av1` notes an error and keeps the
session rather than rebuilding the slot map, so an early return leaked a surface
per failed frame and hit `SlotError::Full` after nine.
2026-08-07 21:07:23 +02:00
enricobuehler 1c54d0999b fix(client): the D3D11VA AV1 rung decoded every inter frame into a surface it was predicting from
AV1 applies `refresh_frame_flags` AFTER the frame is decoded (7.20), so a frame
that reads a reference slot and then overwrites it is the ORDINARY case, not an
exotic one: 268 of the vendored vector's 274 frames do it, first at frame 6.

`plan_to_dxva_av1` released every displaced picture inside the conversion — which
is what the H.264 and H.265 siblings do with their whole `removed` list — and then
assigned the decode target a slot. `SlotMap::assign` takes the lowest free slot,
and the lowest free slot is the one just vacated. So the submission said
`CurrPicTextureIndex = N` and `RefFrameMapTextureIndex[k] = N` in the same breath,
on 268 of 274 frames: decode into the surface you predict from.

Neither vendored H.264 nor H.265 vector ever produces that shape (measured: zero
on 250 AUs), which is why an eager release survived two hardware-proven codecs and
opened on the first AV1 frame past the key frame's neighbourhood. HEVC even has
the invariant under test already — `the_current_picture_is_named_by_curr_pic_and_
never_aliases_a_reference` — and AV1 had nothing.

The Vulkan rung already carries the fix; this is the same contract, and the DXVA
constraint is the STRICTER of the two: Vulkan binds only the references a frame
names, while `RefFrameMapTextureIndex` declares the whole store, so every picture
the store still names has to survive the conversion. `DecodePlanDxvaAv1` grows
`release_after_decode` and `frame_av1` applies it once the decode op is issued —
next to the `refresh_frame_flags == 0` release that already waits for the same
reason. Peak surfaces held goes 7 of the 9 the pool allocates, so the spare slot
`SlotMap::new` adds is doing exactly the job it exists for.

Measured on hardware before the fix: Intel Arc got 245 of 250 delivered frames
wrong — 47% of luma at the first bad frame, max |delta| 242, chroma wrong too, a
frame predicted from the wrong picture — and the only late frame it got right was
the one intra frame, which names no reference and so could not alias. That reads
as a `primary_ref_frame` defect and is not one: PRIMARY_REF_NONE and "has no
references to alias" are the same frames.
2026-08-07 20:52:00 +02:00
enricobuehler 4c587ef531 docs(release): split the technical half into CHANGELOG.md, and lead with a TL;DR
ci / docs-site (pull_request) Failing after 34s
ci / bun-nix (pull_request) Successful in 41s
ci / web (pull_request) Successful in 1m9s
apple / swift (pull_request) Successful in 1m34s
apple / screenshots (pull_request) Skipped
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m6s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m5s
ci / rust-arm64 (pull_request) Successful in 5m4s
android / android (pull_request) Successful in 5m17s
nix / flake (pull_request) Failing after 10m14s
ci / rust (pull_request) Successful in 15m52s
main went from 35ba64ca to a8a4b11f while this branch sat open — 190 more commits,
taking 0.25.0 to 327 since v0.24.0 and making it the largest release so far. That
scale is what forced the restructure.

THE SPLIT. Through v0.24.0 the engineering detail lived in an `## Under the hood
(for developers)` section at the bottom of the notes. It had grown to 21 dense
bullets sitting under the user-facing half — the exact burying the voice rules
exist to prevent, and it would only have got worse here. So:

  * `CHANGELOG.md` at the repo root is now the technical half, newest release
    first. It opens with a version table that lists every protocol number
    INCLUDING the unchanged ones, because "did the driver protocol move?" is the
    question an embedder most often needs answered and "no" is a real answer.
    Then breaking changes, capability bits, wire planes, env vars, and the rest.
  * `docs/releases/vX.Y.Z.md` keeps a short `## For developers` pointer and
    otherwise contains no internal names at all.
  * The link targets the file AT THE TAG, not at main. A release's notes are
    frozen; a main link would silently start describing a later release.
  * README.md and TEMPLATE.md now document this as the ritual rather than
    leaving it a one-off, including a new rule 7 requiring the TL;DR.

The notes are SHORTER than before the merge — 83 lines against 108 — while
covering three times as much. That is the point.

TL;DR, six bullets: FFmpeg gone from the desktop clients; AV1 at 4K shipping half
of every frame; HDR/10-bit leaving half the encoder idle; Windows hosts minting
their own audio devices; the controller sweep; and settings silently resetting.
The last bullet points at `## Before you update` rather than restating it.

WHAT I DELIBERATELY DID NOT PROMOTE. #96 (HEVC DPB) and the rav1d half of #97 are
ship-blockers for the NEW decode stack, not live field bugs — verified, `git
ls-tree v0.24.0 -- crates/` has none of pf-vkdecode/pf-bitstream/pf-dxvadec/
pf-vaadec and `dpb_limit` did not exist at the tag. Nobody on a released build
has ever hit them. They are in Fixed and in the changelog, not the TL;DR, which
is reserved for things a reader is living with today. (A memory note claiming
shipped 0.24.x clients carry the HEVC bug was wrong and has been corrected — it
was about to drive a release decision.)

Also kept out of user-facing Fixed: the plugin-UI empty panel (fixes the origin
split from earlier in THIS release, so it folds into that change — but its 47993
firewall consequence IS in `## Before you update`, because an upgraded host keeps
a 47992-only rule and every plugin interface goes blank), and the pad-audio
WASAPI path fix (repairs a Windows build break in never-shipped code).

HONESTY CARRIED FORWARD rather than smoothed. The notes say plainly that the
Android overlay change did not make the stream faster, that nothing is
recoverable from the Windows non-C: settings loss, that Deck HDR still needs
Steam's own display setting, and that VB-CABLE should be left installed. The
changelog's verification section says the FFmpeg-deletion milestone never
executed on a GPU, pf-vaadec has never decoded a frame anywhere, openh264 has
never run on glass, and controller audio has never touched a real DualSense.

Play's "What's new" refreshed and re-fitted: swapped the TV-profiles line for the
safe-area/notch fix, which is visible to every modern phone user. First draft came
out at 525 chars; trimmed to 497/500, verified with android.yml's gate logic.

Re-verified after the merge (Cargo.lock conflicted — took main's and regenerated
the bump): lock diff versions-only 35/35 against origin/main, `cargo metadata
--locked` resolves (39 members; fec-rs, pf-driver-proto, usbip-sim and the newly
vendored cros-codecs keep their own versions), `cargo fmt --all --check` clean in
both workspaces, doc lazy-continuation scanner 0 hits over 579 files, notes body 0
internal-vocabulary hits above `## For developers`.

Wire 2. C ABI 14 -> 17 (15 rumble floor, 16 pad audio, 17 session end reason).
Driver protocol 6 and gamepad channel 3 untouched. host_caps is down to its last
free bit (0x80) and video_caps has been full since 0.23.0 — both now stated in the
changelog rather than left to be rediscovered.
2026-08-07 20:24:08 +02:00
enricobuehler 6d0a389dd2 fix(client): the D3D11VA AV1 rung decodes wrong pixels — the parity harness existed all along
The follow-up was framed as "build the frame-hash parity harness the D3D11VA AV1
rung is missing, then flip hardware_verified to true". Both halves were wrong.

The harness was never missing. `video_d3d11_native`'s `parity` module has carried
`av1_every_delivered_frame_hashes_bit_identical_to_libavcodec` since M7 wired the
rung — wired to the SAME libavcodec goldens the Vulkan AV1 leg passes against,
with the display-order model that handles the vector's 24 hidden frames, sitting
`#[ignore]`d beside the H.264/H.265/Main10 legs. It had simply never been run on a
device; .173 was powered off the day it was written. What the old evidence note
called a missing harness is real about pf-dxvadec the CRATE, which cannot host one
— it links no D3D11 — but the device half lives here and was already done.

Run on .221, it FAILS, on both GPUs, deterministically (three runs each, identical
first-divergent frame and identical hashes): 186/250 diverging display frames on an
RTX 3500 Ada, 245/250 on an Intel Arc.

It is the decode that is wrong, not the measurement, and three independent checks
say so. H.264 and H.265 pass 250/250 and HEVC Main 10 50/50 through the same
harness, the same readback geometry, the same crop and the same slot map on those
same two GPUs. pf-vkdecode's Vulkan AV1 leg reproduces the same golden file 250/250
on the same box. And the goldens regenerate byte-for-byte from the ffmpeg build
their own header names.

Two signatures, and they are not one defect wearing two faces. NVIDIA is bit-exact
for display frames 0..=63 and then loses ONE 16x24 luma block — 174 pixels, max
|delta| 8, chroma untouched — on the frame whose order_hint first reaches 64, after
which every remaining frame is downstream of it through prediction. The stream
parks the key frame (order_hint 0) in BWDREF and ALTREF2 for its whole length, so
64 is where the distance to it reaches the edge of what get_relative_dist can
represent at OrderHintBits = 7. Intel is structurally wrong from display frame 4 —
47% of luma, max |delta| 242, chroma wrong too, a frame predicted from the wrong
picture — and the only later frame it gets right is the one whose primary_ref_frame
is PRIMARY_REF_NONE.

None of this is visible on glass, which is the whole argument for goldens: the rung
streams 4K60 on both parts with a clean five-minute soak at roughly ten times the
Vulkan leg's decode time. The 2026-08-07 field sessions that looked clean were
looking at wrong pixels.

So hardware_verified stays false, and the note now says why in the strongest
available terms — it prints at warn on every session that lands here, and "decodes
AV1 to wrong pixels" is what a support engineer needs to read. The pair stays in
`every_rung_runs_and_the_unproven_ones_are_named`'s unproven array; its note still
contains NEVER, because the pair has never PASSED parity, which is now a measured
statement rather than an absence.

Left deliberately unchanged: `auto` on Windows can still reach this rung for AV1,
and on Intel it is the arm that fires, because that vendor advertises no SAMPLED
usage on any decode profile so zero-copy Vulkan Video cannot run there. Barring it
trades visibly-wrong AV1 for the software rung, which cannot keep up at 4K and is
itself unproven. Which way that trade goes is a product call, so it is recorded at
the admission site rather than made silently here.

`av1_divergence_map` is kept, cleaned up and documented: it is what turned "186
frames differ" into a lead — one line per display frame, its verdict beside the
plan facts that could explain it, and an opt-in raw-NV12 dump. At a frame where one
vendor hashes correctly, that vendor's bytes ARE libavcodec's bytes and so a valid
reference for the other's, which is how "how badly" was answered without new
goldens. The tool that would localise the rest does not exist: pf-dxvadec's
libav_picparams_parity covers H.264 and HEVC only, so the AV1 conversion has never
been compared against libavcodec at the picture-parameter level either. That is the
next step, not another session.

Also in this file, since it is the same table and the same day: the VAAPI rung's
AV1 leg has now decoded 250/250 of the vendored vector on RDNA3 and its arm is
split from the H.264/H.265 ones, which genuinely have still never decoded anything.
It is unverified for the same reason as ever — no parity — and the D3D11VA row
above is exactly why that distinction is worth keeping: a rung can decode 250
frames and still be wrong.
2026-08-07 20:22:33 +02:00
enricobuehler 7a4cdac5b7 fix(client/android): a Bluetooth pad's gyro obeys the same reachability gate as the rest
ci / bun-nix (pull_request) Successful in 26s
ci / web (pull_request) Successful in 1m8s
ci / docs-site (pull_request) Successful in 1m9s
ci / rust-arm64 (pull_request) Successful in 1m34s
windows-drivers / driver-build (pull_request) Successful in 1m34s
android / android (pull_request) Successful in 3m30s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m15s
windows-drivers / probe-and-proto (pull_request) Successful in 18s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m6s
ci / rust (pull_request) Successful in 6m34s
apple / swift (pull_request) Failing after 11m43s
apple / screenshots (pull_request) Skipped
Follow-up to the G10 merge. The new sensor path was written against main, which does not
carry this branch's G8 work, so it forwarded motion unconditionally — the one thing G8
exists to stop.

`deviceMotion` checked `forwarding` and nothing else. A Bluetooth DualSense in a session
that resolved to an X-Box backend would stream ~200 Hz of samples the host parses and
discards, for the whole session, exactly as the USB capture path did before G8. Not a
regression against shipped behaviour — the path is new — but it would have shipped the
defect back into a client that had just been taught not to have it.

`Slot` now carries `motionReaches`, asked once at open off the kind that pad DECLARED, in
the same shape `ExternalPad` already used. Per pad, not per session: under Automatic the
handshake carries the active pad's kind, so a couch with an X-Box pad on slot 0 and a
DualSense on slot 1 must not have slot 1's working gyro suppressed by slot 0's answer.

The notice moved to where the truth is known. `openSlot` knows only what kind a pad
declared, not whether it physically has a gyro — that is discovered later, when
`PadSensors` finds a gyroscope and calls `setDeviceHasSensorMotion`. Raising it there is
the only placement that both tells a player whose gyro is being dropped and stays silent
for the pads that never had one.

Also unified the last duplicate scale in the module. G10 hoisted the wire units into
`Gamepad` and pointed `DeviceGyro` at them, but `DsDevice` kept its own `20L` / `10000L`
— and `Gamepad`'s new comment claims every sender goes through one place, which was not
yet true. Two copies of a unit constant in one module is precisely the defect this program
opened with (a DualShock 4 blob 40× hot because a second copy had drifted), so the claim
and the code now agree. `val` rather than `const val` only because widening to Long is not
a constant expression; Long is deliberate, since the calibration arithmetic overflows an
Int before it divides.

Proven non-vacuous rather than assumed: changing `Gamepad.MOTION_GYRO_LSB_PER_DEG_S` from
20 to 16 now fails four named cases across three classes —
`DsDeviceTest.calibrationRescalesRawCountsOntoTheWireUnits`,
`.theHostsOwnBlobIsAPassthrough`, `.parseStateAppliesTheCalibration` and
`DeviceGyroTest.wireUnitConstants`. Before this change `DsDevice` would not have noticed.

The gate itself has no test, for the reason the surrounding code already documents:
`GamepadRouter` needs Android plus a live JNI handle, there is no Robolectric in this
module, and a mock would test the mock. It is argued at the call sites instead.

Gate: `:kit:compileDebugKotlin`, `:kit:testDebugUnitTest`, `:app:compileDebugKotlin`,
`:app:testDebugUnitTest` — kit 75 / app 67, 0 failures, counts read out of the JUnit XML.
The merge reconciles: 62 on this branch, plus 6 from main's DeviceGyroTest, plus G10's 7.
2026-08-07 20:18:39 +02:00
enricobuehler e81ab1ff2b Merge branch 'worktree-agent-a6f74e8ea7824fb56' into worktree-gyro-p0-correctness 2026-08-07 20:14:41 +02:00
enricobuehler 8f1081719f feat(client/android): a Bluetooth controller's gyro stops going nowhere
Android had two motion sources and both of them are USB claims. DsCapture
takes a Sony pad's HID interface away from the kernel; Sc2Capture does the
same for a Steam Controller 2. Everything else — a DualSense, a DualShock 4,
a Switch Pro, an 8BitDo, paired over Bluetooth — arrives as an ordinary
InputDevice. Its buttons worked, its sticks worked, and its gyro was dead,
silently, with no log line and nothing in the UI to suggest the pad had a
sensor at all. That is not one controller, it is the whole class of
controllers people actually pair to a phone.

The platform has had the answer since Android 12: InputDevice.getSensorManager
hands back a SensorManager scoped to that one controller, carrying its
TYPE_GYROSCOPE and TYPE_ACCELEROMETER. PadSensors registers a listener per
forwarded pad that has a gyroscope and sends the samples on that pad's wire
index. Below API 31 it registers nothing and the pads behave exactly as they
did.

It is built on DeviceGyro's shape, because the phone mirror had already paid
for these lessons. One dedicated HandlerThread, never the main one. Batching
off (maxReportLatencyUs = 0) — batching would trade away precisely the latency
gyro aim exists to avoid. 200 Hz requested, which is also the ceiling the
framework grants an app without HIGH_SAMPLING_RATE_SENSORS, so asking for more
would only be capped. And a feed that lets go of a pad still alive parks its
rotation at zero first: the host holds motion as state and re-emits it in every
virtual-pad report, so an angular velocity left behind is a pad that rotates
forever.

Two writers on one pad's motion is the failure this program has spent the day
unpicking, so the coordination is explicit in three places. A USB capture wins:
DsCapture.startUsb already calls releaseDevice at claim time, that closes the
slot, and the close now also takes the sensor listeners off — the claim makes
the InputDevice vanish anyway, but going through the explicit teardown is what
makes the ordering deterministic instead of a race against the platform's own
removal callback. The phone-gyro mirror stands down: registering flips a bit
the router reports through padHasOwnMotion, which DeviceGyro re-reads on every
sample and answers with its own zero park. And a pad with an accelerometer but
no gyroscope is deliberately NOT taken — it could only send gravity while
pinning rotation at zero, on a pad the mirror is otherwise entitled to speak
for, which is the same fight in a quieter costume.

The wire units are measured fact (punktfunk_core::input::gamepad: 20 LSB/deg·s,
10000 LSB/g), and they now live in exactly one place on this client:
Gamepad.motionGyroWire / motionAccelWire, which DeviceGyro was hand-inlining a
second copy of. The gyro program's first finding was a client sending 40x hot
because a second copy of a number had drifted, and the merge that followed
found a sender nobody remembered to correct. One function, both callers.

THE AXIS FRAME ON THIS PATH IS NOT VERIFIED, and the mapping is deliberately
straight through rather than guessed at. What is known: the wire is a unit
passthrough into a virtual DualSense report, and that report's frame was
measured over raw HID on 2026-08-07 as (Right, Up, Backward-toward-the-player)
carrying (pitch, yaw, roll), right-handed — which is why the USB path forwards
the pad's own order un-remapped and is correct to. Android documents its sensor
frame for a handheld device as +x right, +y up, +z out of the face, the same
frame once "the face" is read as the one the player looks at. So straight
through is what the documentation implies. What nobody has done is put a
Bluetooth DualSense in front of the platform sensor framework and compare —
those numbers come through a HID driver and InputFlinger's sensor mapper,
either of which could permute or negate without saying so. A plausible-looking
wrong remap is exactly the bug this program keeps finding, so the code says
unverified and names the measurement that settles it, and each feed logs its
first converted sample so the cheapest half of that measurement — which slot
gravity lands on with the pad flat and still — costs a logcat line.

PadSensorsTest pins the scale, the clamp, the rounding and the straight-through
order, mutation-checked four ways: 20 to 16 fails gyroScaleFromRadiansPerSecond
and straightThroughFrame, reversing the axis order fails straightThroughFrame,
truncating instead of rounding fails roundsToNearestNotTowardZero, and negating
the accel fails restingPadIsTheHostNeutral. Its frame expectations are written
to change together with any remap that lands, not to be edited around one.
GamepadRouter needs Android and a live JNI handle and there is no Robolectric
here, so its half is argued in comments beside the code, as DsCapture's claim
ordering already is.

Gates: kit 65 tests (58 before, plus 7), app 67 unchanged, 0 failures, read out
of the JUnit XML rather than off a green build.
2026-08-07 20:12:39 +02:00
enricobuehler 979ed8f426 docs(clients/pads): say that an X-Box virtual pad has nowhere to put motion
G17's motion half. The docs described what the CLIENT sends and stopped there, which
made a promise the host does not always keep.

The support matrix said a desktop client forwards motion from any pad SDL exposes a gyro
on "and the host injects it into the matching virtual pad". The first clause is true; the
second is only true when the virtual pad has a motion plane. The X-Box 360 and One
backends do not — no gyro in their HID contract — so the host parses every sample and
discards it. That is where *Automatic* lands anything it does not recognise as Sony or
Valve, an 8BitDo with a perfectly good gyro included, and where a Switch Pro lands on a
Windows host with no `hid-nintendo` backend to fold it into.

A reader following the old text would conclude their gyro was broken. The failure has no
other symptom: motion just does nothing.

So both pages now say what to do about it — pick a DualSense-class type — and the
client-settings page says it where the choice is actually made, next to the degrade
paragraph that explains why a session ends up on an X-Box pad in the first place.

The Deck's Steam-Input requirement moves out of Decky's settings blurb, which is the one
place a Deck user streaming FROM the Deck would never look. With Steam Input on, Steam
hands the app its own virtual X-Box pad, so no controller-type choice can help: there is
no gyro on the pad the client can see.

The picker help text now mentions motion on GTK and Android, which is where it was
missing — Windows already said it and Apple says it in its own words. One sentence, the
same sentence, so the four clients answer the question the same way.

This is the doc side of the on-screen notice that shipped earlier in this branch. The two
exist for the same reason and now agree: the client says it when it detects the case, the
docs say it when someone goes looking.

Not covered: the preset COUNTS in note 1 ("Android and the console home offer six …
Windows and Apple offer five") are still unverified against the four pickers, and the
Apple picker's missing Steam Deck entry is a code gap rather than a doc one. Both are
noted in the plan and left for their own change rather than guessed at here.

Gate: Linux CI image fmt + `clippy --locked --all-targets -D warnings` on
punktfunk-client-linux (the GTK string is compiled) plus the core crates and their tests;
Android `:app:compileDebugKotlin` + `:app:testDebugUnitTest`. Green.
2026-08-07 20:11:28 +02:00
enricobuehler 4ddc0e311d Merge remote-tracking branch 'origin/main' into worktree-release-0250
# Conflicts:
#	Cargo.lock
2026-08-07 20:10:42 +02:00
enricobuehler f351eb01e9 feat(vaapi): VAAPI decodes AV1 — the rung's first frame on any hardware
The evidence table has said "native VAAPI: has never decoded a frame anywhere
(M6/M7)" since the rung was written. That is no longer true. Measured on `.25`
(Radeon 780M / Phoenix1 RDNA3, radeonsi, Mesa 26.0.3, VA-API 1.23, Ubuntu
26.04 — headless, no display server needed):

    VAAPI AV1 rung constructed: native-vaapi av1
    VAAPI AV1: 250 frames delivered, first 320x240 fourcc="NV12"
               modifier=0x200000010401b04

250 of 250 displayed frames, first try, on the same vendored vector the Vulkan
and D3D11VA AV1 legs walk. The count matters as more than a smoke test: the
vector carries 274 coded frames in 250 temporal units — 24 units carry two, and
those extras are HIDDEN (decoded, referenced, never shown) — so 250 delivered is
this rung agreeing with the other two about which frames are output. A tiled AMD
DRM modifier rather than a linear one says the surface is a real decode target,
not a fallback.

Two changes, both in the rung's own file.

**The probe never asked about AV1.** `probe_this_machines_libva` walked H.264
High, HEVC Main and HEVC Main 10 and stopped there, which is part of why "never
decoded a frame" could stand so long without anyone noticing what had not been
asked. It now covers both AV1 profiles, and this box answers:

    H.264 High: VLD decode        AV1 Profile 0: VLD decode
    HEVC Main: VLD decode         AV1 Profile 1: no (VAProfile not supported)

Profile 1 being refused is correct — 4:4:4 AV1, which radeonsi does not do — and
it is the negative case that proves the probe reports rather than assumes.

**`av1_decodes_the_vendored_vector_on_this_machines_vaapi`** is the decode
itself, `#[ignore]`d beside the probe.

It is deliberately WEAKER than the Vulkan and D3D11VA AV1 legs, and the docs say
so rather than letting the name imply parity: those two hash every frame against
libavcodec's goldens because both can read their decoded surface back. This rung
hands out a DRM-PRIME dmabuf whose memory the driver tiles, so there is no
CPU-readable image to hash without adding a vaDeriveImage/vaGetImage path that
production neither uses nor wants. So it asserts what can be asserted honestly —
every temporal unit accepted, the right number of frames back, each a real
exported surface of the right shape, the first flagged as a keyframe — and it is
NOT frame-hash parity. Promoting this rung to `verified` still wants parity, and
parity wants a readback path first.

It fails loudly rather than skipping when the device has no AV1 entry point. It
is `#[ignore]`d, so it only runs when someone points it at a box that is supposed
to have one, and a silent pass there is exactly the invisible-failure mode this
program exists to end.

Gates: on `.25`, fmt clean, `clippy -p pf-client-core --all-targets -D warnings`
green under the Linux cfg where this rung actually compiles, the whole lib suite
167/167, and all 11 VAAPI tests green with `--include-ignored`. Workspace fmt +
clippy + lib suite also green in the Linux container.

⚠ Not touched here on purpose: the evidence table in `video.rs`. Its VAAPI row
still reads "never decoded a frame anywhere" and now understates what is known —
but a parallel agent is editing that same file for the D3D11VA AV1 row, so the
row is left for whoever lands second to update once, rather than conflicting.

Note for anyone reproducing on `.25`: it has no system SDL3 and no passwordless
sudo, so the test binary links only with `--features sdl3/build-from-source`
(SDL3 is gamepads, irrelevant to decode; production Linux still links the system
one). Its disk sits at ~99% full, and the tree there is a `git archive` export
with no `.git`, so `git apply`/`git checkout --` silently do nothing.
2026-08-07 20:04:33 +02:00
enricobuehler 1eab4b6626 fix(client/apple): the phone-gyro mirror was left on the old motion convention
ci / docs-site (pull_request) Successful in 1m3s
apple / swift (pull_request) Successful in 1m37s
apple / screenshots (pull_request) Skipped
ci / bun-nix (pull_request) Successful in 1m30s
windows-drivers / driver-build (pull_request) Successful in 1m39s
ci / web (pull_request) Successful in 2m5s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m10s
ci / rust-arm64 (pull_request) Successful in 3m21s
android / android (pull_request) Successful in 4m54s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m6s
ci / rust (pull_request) Successful in 9m4s
windows-drivers / probe-and-proto (pull_request) Failing after 10m10s
Surfaced by the merge. `DeviceGyro`'s header states the contract plainly — "units and
axis semantics match `GamepadCapture.forwardMotion` exactly … the same convention, so a
future sign/scale correction lands in one place for both sources" — and this branch made
two such corrections in only one of the two places. That is a promise the code stopped
keeping the moment the controller path was fixed.

Both were true parity when #88 was written; both broke here.

**The negation.** `GamepadCapture` sends `-(gravity + userAcceleration)` because Apple
reports the gravity VECTOR, pointing down, while an accelerometer measures proper
acceleration, pointing up at rest — and the wire carries the latter. The mirror sent it
un-negated, so a phone lying still told the host it was accelerating downward at 1 g.
The comment above that line even claimed the convention matched.

**The frame.** The mirror's remap targets the controller frame its own header describes —
x right, y up, z out of the screen — which is exactly GameController's frame, and that is
not the DualSense report frame the wire is defined in. So the same change of basis the
controller path now takes applies here, after the orientation remap rather than instead
of it: the remap resolves which way the phone is being held, and the basis change
translates the result into the pad's language. Two different jobs that happen to compose.

Order matters for the closing sample too. `stop` replays `lastAccel` beside a zero gyro
so "rotation stopped" does not also read as free fall; `lastAccel` is recorded after both
conversions, so what gets parked is what was actually sent.

Left alone deliberately: `DeviceGyroRemap` itself and `DeviceGyroRemapTests`. The
orientation matrices answer a different question — which way is the phone being held —
and nothing measured this evening bears on them. They remain derived-not-verified, as
their own doc says, and the on-glass pass that owes the controller path a check owes them
one too, in all four orientations.

Gate: macOS `swift build` + full suite (215 tests, 5 skipped, 0 failures) and the
iOS-triple typecheck green — the latter is what actually compiles this file, since the
whole thing is `#if os(iOS)`.
2026-08-07 19:34:51 +02:00
enricobuehler 5a4305c072 merge: bring current main into the gyro correctness branch
main moved ~60 commits while this branch was in progress, and one of them matters
here: PR #88 (the phone-gyro mirror) landed, touching the same motion path.

One conflicted file, `GamepadCapture.swift`, in three places — all of them the two
changes meeting rather than disagreeing:

- **Slot fields.** #88 added `motionSent` + `lastAccel` for its flush-parks-motion fix;
  this branch removed `lastMotionNs` with the 4 ms drop-throttle. Kept both decisions:
  the parking state stays, the throttle field goes.
- **forwardMotion's head.** #88 added the mirror stand-down (`pad 0` yields while the
  phone speaks for it); this branch deleted the throttle guard. Kept the stand-down,
  dropped the guard.
- **The send.** This branch converts into the DualSense report frame; #88 records what
  went out so `flush` can replay it beside a zero gyro. Both, with the recording placed
  AFTER the conversion — `flush` replays `lastAccel`, so it has to be the vector that
  actually went on the wire, or a still pad's gravity gets parked in the wrong axis.

The two features compose exactly, which is worth stating because it is not luck: this
branch gates motion capture on `hasRotationRate`, and #88 engages the phone mirror when
`hasRotationRate != true`. They are complements — a pad either drives its own gyro or the
phone mirrors for it, never both and never neither.

Everything else auto-merged. Note `DeviceGyroRemapTests` is `#if os(iOS)`, so the macOS
suite reports the same 215 as before the merge rather than gaining #88's six — checked,
not assumed.

Gates re-run against the merged tree rather than trusting either side's: Linux fmt +
build + `clippy --locked --all-targets -D warnings` + punktfunk-core and pf-inject
suites; Apple 215 tests and the iOS-triple typecheck; Android kit + app compile and
tests. All green.
2026-08-07 19:34:22 +02:00
enricobuehler 19c9165d4b docs(client): the D3D11VA AV1 rung has two vendors and a soak now — and still no parity
Re-measured against a host carrying #95, from .21 (RTX 5070 Ti, av1_nvenc) to
.221, on glass:

  Intel Arc,   auto -> native-d3d11va   4K60, decode 1.4 ms, e2e 16.7 ms p50
  RTX 3500 Ada, pinned native-d3d11va   4K60, decode 1.0 ms
  RTX 3500 Ada, pinned native-vulkan    4K60, decode 11.6-16.7 ms

Plus a 5-minute Arc soak: 297 stats lines, 60 fps, decode 1.3 ms, e2e 10.9/14.8 ms
p50, and exactly one WARN in the whole run — the hardware_verified=false notice
itself. No refusals, no demotions, no concealed runs.

Three things that follow.

The rung is no longer a one-session curiosity: it decodes 4K60 AV1 on TWO
vendors and survives a soak. The Arc leg matters twice over, because the Arc
advertises no SAMPLED usage on any decode profile — zero-copy Vulkan Video
cannot work there — so `auto` demoting to D3D11VA and then decoding is the
whole demotion path working as designed.

It is roughly 10x faster than the Vulkan AV1 leg on the SAME NVIDIA GPU. That
is the strongest argument yet for eventually letting `auto` pick it ahead of
Vulkan Video, which is exactly what `verified` gates.

And it stays `verified = false` anyway, because the missing piece is specific:
there is no frame-hash parity against libavcodec. Every other verified pair in
that table earned it with one, and pf-dxvadec has no harness that could produce
one — `libav_picparams_parity` compares picture parameters on the CPU and never
decodes a frame. Building that harness is the work that promotes this rung; a
fourth session is not. The evidence string now says so, so the next reader does
not have to rediscover which half is missing.

The VAAPI row is corrected in the same spirit rather than left as a bare "NO":
the reachable VAAPI box (.25, RDNA3) reports VAProfileAV1Profile0 /
VAEntrypointVLD and advertises no Vulkan AV1 decode at all, which makes it the
right box to prove that rung on and an unambiguous oracle when it happens. What
stopped it is recorded too — no punktfunk checkout there and 4 GB of usable RAM.

Documentation only — no behaviour change, and no flag flipped.
2026-08-07 19:31:52 +02:00
enricobuehler c64cdc4ef7 docs(encode): close out the tile-aware AV1 sub-frame reader — measured, not worth it
#95 disarmed sub-frame readback for AV1, which means AV1 forgoes the latency
win HEVC gets from shipping slice 1 while slice 2 encodes. The follow-up was to
teach the reader AV1's units: cut on OBU boundaries rather than byte counts and
arm from the driver's reported unit count. Measured on .21 (RTX 5070 Ti,
av1_nvenc) before writing any of it, and the measurement closes it rather than
scoping it.

Reading the frame headers av1_nvenc actually emits at 4K:

  width_in_sbs_minus_1[0]      = 59        one tile column, the full 3840
  height_in_sbs_minus_1[0..1]  = 16, 16    two tile rows
  tile_start_and_end_present_flag = 0      BOTH TILES IN ONE TILE GROUP OBU

That last flag is the finding. "Cut on OBU boundaries" presumes the tiles are
separate OBUs and they are not — there is no boundary between them to cut on.
Shipping tile 1 early would need the HOST to re-author AV1 syntax per chunk,
synthesising a fresh Tile Group OBU header with tile_start_and_end_present_flag
= 1 and its own tg_start/tg_end. That is bitstream surgery on the encode path,
not the reader change it was assumed to be.

And the prize would be small even then, because split encode already spent it.
The two tile rows go to two split-encode engines that run CONCURRENTLY, so they
complete at nearly the same moment — the win is bounded by the skew between
engines, not by half a frame. Whole-frame encode measures 3.3-3.6 ms at 4K60
against a 16.7 ms p50 end-to-end, so even the sequential-tiles fantasy caps near
1.7 ms and the real number is a fraction of it. HEVC's win is bigger for a
structural reason that does not transfer: forced split and sub-frame are
mutually unsupported, so HEVC's slices genuinely are produced one after another.

1080p settles it further: tile_cols_log2 = tile_rows_log2 = 0, a single tile, so
there is nothing to pipeline at the commonest streaming resolution at all.

Recorded next to the disarm with the reopen condition named — NVENC emitting one
OBU per tile, or setting tile_start_and_end_present_flag = 1 — so this is closed
on evidence rather than left as an open maybe.

Documentation only — no behaviour change.
2026-08-07 19:26:32 +02:00
enricobuehler 6b4be28d24 docs(client): write down why the CPU rung is not process-isolated
#97's frame-context floor closes the one rav1d abort we hit and can prove. It
does not make the rung panic-proof and nothing at that call site can, because
rav1d's public surface is dav1d's C ABI: any reachable panic crosses
`extern "C"` as `panic_cannot_unwind` and becomes `abort()`, past every
`catch_unwind`, rung demotion and typed refusal we have.

Counted across rav1d 1.1.0's 60 source files: 285 `unwrap()`, 214 `assert!`,
19 `unreachable!`, 11 `expect()`, 10 `panic!`. 539 sites that end the client if
a stream can reach them. #97 fixed one of them.

Process isolation is the only defence that actually works, and this records the
decision NOT to build it, with the reasoning, so it is not re-argued from
scratch each time someone reads that number:

  * the defect is upstream's and is one line (memorysafety/rav1d#1497, filed
    2026-08-07 with the fix and a reproducer; still open, no PR, as of today);
  * 539 is an unbounded number, not a risk estimate — none of those sites is
    known reachable from a punktfunk stream, and the honest next step is to
    fuzz the rung and find out, which is cheap, rather than buy insurance,
    which is not;
  * the cost lands on the video path across Linux, Windows and Android (the
    Apple clients decode through VideoToolbox and never reach this code), each
    needing its own shared-memory frame transport, child lifecycle and
    backpressure, and it adds a scheduling boundary to the slowest rung on the
    ladder while zero-copy is a hard requirement;
  * an abort here costs a session that was already degraded — this rung exists
    because the GPU rungs failed first.

The trigger to revisit is named as an event rather than a feeling: a SECOND
distinct abort in the field, or a fuzzer finding a reachable panic. Either
makes it a class of bugs instead of one, and a class is what would justify the
architecture.

Documentation only — no behaviour change.
2026-08-07 19:25:22 +02:00
enricobuehler 669176982d fix(h264): name the DPB cliff #96 left standing in the other codec
H.264 derives its DPB size the same way HEVC did before #96 — from a level
ceiling that says what a stream MAY use, not what it needs — and the ceiling
saturates at 16 frames, which is 17 hardware slots with the picture in flight.
That is the exact arithmetic that cost 720p and 1080p their HEVC.

Measured on real encoders (2026-08-07) rather than assumed: H.264 escapes it
twice over, and both escapes belong to the encoders, not to the format.

  encoder                              level picked        VUI restriction
  NVENC (RTX 5070 Ti, 610.57.04)       3.2/4.2/5.1/5.2     present, buffering 3
  VAAPI via libavcodec (RDNA3, 26.0.3) 4.1/4.2/5.1/5.2     present, buffering 1
  openh264 (the software rung)         3.2/4.2/5.1/5.2     present, buffering 1

Every one picks a level proportionate to the picture AND states its real need
in the VUI bitstream restriction, so the ceiling is never reached and never
consulted. Nothing is broken today, and clamping would be wrong: with the
restriction present the number IS the stream's own statement, and a stream that
genuinely asked for a deep DPB would decode wrong if we shrank it.

So this does not change what any stream decodes. It gives the arithmetic one
named home (`dpb_limit`, the twin of `h265::dpb_limit`) carrying the evidence
and the reasoning, and it adds the signal that was missing: when an SPS carries
no restriction AND its level ceiling would demand more slots than mainstream
hardware provides, the plan now says so with `PlanWarning::LevelDerivedDpb`
instead of a user silently losing the codec the way #96's users silently lost
HEVC. It is not an integrity warning — the picture is intact; what fails is
opening a session — so `is_integrity_warning` classifies it false.

One thing the sweep corrects about how the follow-up was framed: it is SMALL
pictures that saturate the ceiling most easily, not 720p specifically. 640x360
at level 3.1 computes 16 as readily as 720p at level 5.0, because the ceiling
is MaxDpbMbs divided by the picture's macroblocks. The authored 64x64 test
fixtures land there too, which is why they now assert through `picture_warnings`.

Guards, as the missing consumer-end half of pf-encode's
`rfi_dpb_fits_a_mainstream_vulkan_decoder`:

  * every_reachable_h264_stream_fits_a_mainstream_slot_pool — the measured
    (picture, level, declaration) pairs, asserting slots <= 16
  * the_level_ceiling_alone_would_reproduce_96_and_is_warned_about — the same
    resolutions at levels that saturate, pinned WITH the warning
  * a_proportionate_level_fits_even_without_a_vui_restriction — so neither
    escape looks like it is doing all the work alone

Gates: fmt + clippy -D warnings clean; pf-client-core 167/167; pf-bitstream
84/84; and gpu_parity 8/8 bit-identical to libavcodec on the RTX 5070 Ti, which
is the gate that matters for anything touching the bitstream layer.
2026-08-07 19:15:53 +02:00
enricobuehler 0170da2a5f fix(client/apple): stop dropping rotation, and stop inventing it
ci / bun-nix (pull_request) Successful in 46s
ci / web (pull_request) Successful in 1m3s
ci / docs-site (pull_request) Successful in 1m38s
apple / swift (pull_request) Successful in 1m37s
apple / screenshots (pull_request) Skipped
windows-drivers / driver-build (pull_request) Successful in 1m42s
ci / rust-arm64 (pull_request) Successful in 2m19s
windows-drivers / probe-and-proto (pull_request) Successful in 33s
android / android (pull_request) Successful in 3m33s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m15s
ci / rust (pull_request) Successful in 4m50s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m9s
G13 — the three capture-fidelity findings from the gyro sweep, two fixed and one
argued.

**The 4 ms floor was a DROP, and it was shedding real rotation.** A sample arriving
3.9 ms after the last one was discarded outright. That is the wrong shape for this
signal: buttons and sticks are absolute state, so a dropped frame costs nothing — the
next one says everything it would have. Angular velocity is a RATE, and a consumer
integrates it into an angle, so a dropped sample is rotation that happened and can never
be recovered. GameController's delivery jitters around the pad's own ~250 Hz, so a floor
set AT that rate does not shed a rare extra sample; it sheds a steady fraction of every
turn. And the error is one-signed, so it accumulates — aim drifting short, which reads
as bad sensitivity rather than as a bug.

Nothing needed the ceiling. GC delivers at the sensor's rate rather than faster, the SDL
client has always forwarded every sample, and the host's idle watchdog is a 100 ms
timeout this cannot outpace. The throttle's two fields went with it: `lastMotionNs` was
left set-but-never-read once the guard was gone, and `motionIntervalNs` had no other
consumer. (Notes elsewhere say `flush` parks motion and reads it — that is PR #88's
branch, not this one. Checked rather than assumed.)

**An X-Box pad was streaming gyro it does not have.** Capture attached to any `GCMotion`,
and an X-Box controller exposes one that reports gravity and NOTHING else. So the client
sent a permanently-zero `rotationRate` to the host as authoritative gyro, under a
declaration saying this pad has one. That is worse than having no motion plane at all: a
game sees a controller being held perfectly still forever, and there is nothing to fall
back to and nothing to notice. Now gated on `hasRotationRate`, which is GameController's
own answer to the question we actually mean.

The settings badge had the same bug from the same cause — `hasMotion` was
`motion != nil`, so an X-Box pad got a gyroscope icon. It now reads `hasRotationRate`
too. One wrong predicate was driving both the UI promise and the wire behaviour, which is
why they were wrong together.

That also simplifies G8's "your gyro can't reach this session" notice, which had to test
`hasRotationRate` itself to avoid nagging about a gyro the pad never had. With the attach
gated on it, the notice is just the else-branch.

**Motion stays on the main queue, and this is the argument for why.** GameController's
`handlerQueue` is a property of the CONTROLLER, not of an element, so moving motion off
main moves buttons, sticks, the touchpad and the escape chord with it. This class is
`@MainActor` throughout — eight `assumeIsolated` sites, the slot table, the gesture
timers — so that is a rewrite of the isolation model rather than a queue assignment, and
it would put the tvOS escape chord (the only controller way out of a stream there) on a
background queue. That is a real risk for a speculative gain. The comment says so at the
call site, and names the measurement to make first if it ever does bite: the host's
per-pad motion inter-arrival histogram already reports exactly this and would say whether
the delay is client-side or on the wire.

Gate: macOS `swift build` + the full suite (215 tests, 5 skipped, 0 failures) and the
iOS-triple typecheck green. No test pins the throttle removal or the capability gate:
both are properties of live `GCMotion` delivery, which this module cannot fake — there is
no injectable seam, and inventing one to assert "we called sendMotion twice" would test
the mock. They are argued at the call sites instead, in the same spirit as the parts of
`DsCapture` that are not unit-testable in their module either. On-glass verification is
owed with the two already outstanding on that rig.
2026-08-07 19:12:02 +02:00
enricobuehler d996449a82 fix(host/pads): a virtual pad at rest said it was in free fall
G14, unblocked by the frame measurement in efb7f991 — the plan deliberately left this
one alone until the up axis was known, on the grounds that a confidently wrong constant
would be worse than an obviously wrong zero. It is known now.

A virtual DualSense, DualShock 4 or Steam Deck that had received no motion reported
acceleration `[0, 0, 0]`. That is not "no data": zero proper acceleration means free
fall, which is a definite claim about the physical world and one that is never true of
a controller sitting on a desk or held in someone's hands — both read 1 g up. Anything
that interprets the accelerometer gets a confident wrong answer rather than a boring
right one.

It is worst exactly where it is least visible. A pad with no gyro at all — an X-Box
controller forwarded as a DualSense, which is what "Automatic" does for anything not
Sony or Valve — never sends motion, so it sits on that neutral for the entire session,
telling every game that reads it that the controller is falling. `switch_proto` has
always done this correctly on its own up axis, which is what made the gap visible in the
first place.

Which axis, and why it took a measurement. The wire is a unit passthrough into the
virtual pad's report, so the wire's up axis is the pad's own, and on 2026-08-07 a real
DualSense read over raw HID put `+0.997 g` on report axis 1 at rest, in a frame pinned
the same session as (Right, Up, Backward). So `MOTION_NEUTRAL_ACCEL` is `[0, 10000, 0]`
— NOT the z-up the notes had assumed from `switch_proto`'s documentation, which is why
guessing would have shipped a backend confidently disagreeing with the hardware.

The constant lives in punktfunk-core beside the units it is expressed in, and every
backend derives from it rather than restating it. The Deck's neutral in particular goes
through `steam_remap::motion_wire_to_deck`, the same rescale a real sample takes, so the
neutral and the live path can never end up with two opinions about what 1 g is — its
`hid-steam` resolution stays in exactly one place. The DS4 needs no separate change: it
reuses `DsState`.

`switch_proto` is deliberately NOT touched, and the test says so. It is a different
device on a different driver, its up axis is its own, and nobody has measured its frame
— aligning it to the DualSense for consistency would be the same unmeasured guess this
commit exists to avoid, just in the other direction.

Non-vacuity proven both ways rather than assumed. Moving the up axis to slot 2 (the old
z-up assumption) fails on the wire constant itself, which is what makes the measurement
load-bearing rather than decorative; reverting both neutrals to `[0, 0, 0]` fails on the
DualSense assertion with the message naming the defect. Each backend is checked in ITS
OWN units, because hard-coding "1 g" three times is how the halves of a unit contract
drift apart.

Gate (Linux CI image): fmt, build, `clippy --locked --all-targets -D warnings` across
punktfunk-core / pf-inject / pf-client-core, and both test suites — green, with
`Running tests/motion_contract.rs` and the new case's own `... ok` line observed in the
log rather than inferred from a green exit (`cargo test` stops after the first failing
binary, so a green-looking run can mean the contract test never executed at all).
2026-08-07 19:07:30 +02:00
enricobuehler efb7f99129 fix(client/apple): motion arrived in the wrong frame — measured against a real pad
G16 step 1, and the second half of what 9e9bb9f4 started. That commit fixed the SIGN
of acceleration (Apple reports the gravity vector, pointing down; a pad reports proper
acceleration, pointing up). This fixes the FRAME, which is a separate defect and was
never going to show up as an inverted axis — it shows up as roll where the game reads
yaw.

The wire is a unit passthrough. `dualsense_proto::write_report` puts gyro[0..3] and
accel[0..3] straight into the virtual pad's report bytes 16.. and 22.., in order, with
no permutation — the same slots a real DualSense fills. So the frame the wire is
DEFINED in is the pad's own report frame, and forwarding GameController's x/y/z
unconverted was speaking a different language with the same vocabulary.

Both frames measured 2026-08-07 from ONE physical DualSense on one desk, read twice —
over raw HID and through GameController — so this is two readings of the same
controller in the same orientations rather than two documents:

  DualSense report frame: (Right, Up, Backward)   axis 0 pitch, 1 yaw, 2 roll
  GameController frame:   (Right, Forward, Up)

Right is already slot 0; Up is GC's z and moves to slot 1; slot 2 wants Backward, which
is GC's y negated. Hence (x, z, -y), applied to gyro AND acceleration because it is a
change of basis and both live in that basis.

Notable: the wire's documented naming was right all along — gyro[0]=pitch, [1]=yaw,
[2]=roll is exactly what the hardware does. And Android needs no remap at all: it
forwards the pad's own axis order un-remapped, which is correct. Its old reading was
purely the scale bug f6de620f fixed. Only Apple was converting nothing.

How the hardware frame was established, since a wrong frame here is invisible. Gravity
at rest put +0.997 g on axis 1. Yaw clockwise-from-above drove axis 1 negative (98% of
the rotation), pitch nose-down drove axis 0 negative (100%), roll right-side-down drove
axis 2 negative (95%) — plain right-hand rule, and (a0 x a1 = a2) confirms the triad is
right-handed. The accelerometer then corroborated the gyro's assignment independently:
under pitch-down axis 2 rose 0.160 -> +0.339 (nose down raises the back, so world-up
gains a Backward component) and under roll-right-down axis 0 went +0.021 -> -0.197,
while yaw left acceleration untouched. Two different physical quantities agreeing on
one triad.

Apple's frame took four attempts, and the failures are worth recording because each was
a different way to be confidently wrong:
  - peak |w| over a window containing BOTH the tip-down and the return stroke can record
    the return, with the opposite sign. Yaw (a continuous one-way spin) was unaffected;
    pitch and roll were exactly the two that disagreed with everything else.
  - reading `gravity + userAcceleration` when `hasGravityAndUserAcceleration` is FALSE
    yields a constant (0,0,1) in every orientation. It looks like data. The tell is that
    it never moves. The client's own else-branch on `m.acceleration` is the correct read
    and is what the instrument now mirrors.
  - `da/dt = -w x a` holds only for gravity, so testing it during vigorous waving — when
    `m.acceleration` carries inseparable linear acceleration — fits nothing.
The frame that survived all of that: static poses, three of them, three repetitions
each. Nose-down moved axis 1 by -0.635 (so axis 1 is Forward), right-side-down moved
axis 0 by -0.686 (so axis 0 is Right), flat put +0.99 on axis 2 (Up). That conclusion
holds whether or not the acceleration negation is right, because negating flips the
measured vector and the physical direction it represents together.

Confidence, stated honestly. The accelerometer half is solid: nine pose measurements,
and mapping the flat pose through gives (+0.005, +0.992, +0.192) against the hardware's
own (+0.021, +0.997, +0.160) — all three components, including the small tilt term that
is what distinguishes this mapping from the five other permutations that also put
gravity on slot 1. That the gyro shares the frame unmodified rests on a weaker
measurement: a gravity-dominated consistency test that preferred (+x,+y,+z) by 1.22x,
which is a margin, not a landslide. It is corroborated by the yaw reading (the one
rotation measured without the return-stroke ambiguity) agreeing with right-hand rule in
that frame, and by the peak-vs-return mechanism explaining the two that did not. A
device-side confirmation is still owed and is listed below.

The tests carry the measurements, not just the conclusion. Resting gravity is asserted
against BOTH readings of that pose; each rotation is asserted to reach the slot the wire
reads it from; and two properties guard the shape rather than the numbers — that the
conversion is an isometry (a basis change may not stretch anything) and that it
preserves handedness. That last one matters most: a permutation with the wrong number of
sign flips is a REFLECTION, which looks plausible axis by axis and inverts every
rotation. Mutation-checked: dropping only the negation fails 6 assertions across 4 of
the 5 cases, the handedness test among them.

Owed, and not claimed done: on-glass re-verification through a real iOS device, together
with the two already owed on that rig (the 9e9bb9f4 sign fix and the Android
calibration read) — one pass covers all three. G14's DualSense neutral acceleration is
now unblocked by this measurement (1 g on slot 1, not the z-up the notes assumed) but is
deliberately left to its own change; and that constant must NOT be propagated to
switch_proto, which is a different device whose frame nobody has measured.

Gate: macOS `swift build` + the full suite (215 tests, 5 skipped, 0 failures) with the
five new cases observed in the run's own output, and the iOS-triple typecheck green.
2026-08-07 18:41:54 +02:00
enricobuehler a8a4b11f5c Merge pull request 'HEVC died at 720p and 1080p because we sized the DPB from the level's ceiling, not the stream's need' (#96) from fix/hevc-dpb-level-ceiling into main
ci / web (push) Successful in 1m6s
ci / bun-nix (push) Successful in 50s
apple / swift (push) Successful in 1m32s
ci / docs-site (push) Successful in 1m15s
ci / rust-arm64 (push) Successful in 2m46s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 10s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 8s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
ci / rust (push) Failing after 4m2s
deb / build-publish-client-arm64 (push) Successful in 3m0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m4s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m18s
deb / build-publish (push) Successful in 5m30s
android / android (push) Successful in 7m28s
docker / builders-arm64cross (push) Successful in 10s
deb / build-publish-host (push) Successful in 5m58s
apple / screenshots (push) Successful in 6m1s
docker / deploy-docs (push) Successful in 32s
arch / build-publish (push) Successful in 10m30s
flatpak / build-publish (push) Successful in 7m11s
windows-host / package (push) Successful in 14m24s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 22s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m37s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m49s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m2s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m25s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m30s
Reviewed-on: #96
2026-08-07 16:21:31 +00:00
enricobuehler 8805a18375 Merge pull request 'A damaged AV1 frame stops killing the whole client (and a BOM stops erasing every setting)' (#97) from worktree-rav1d-single-frame-context-abort into main
android / android (push) Canceled after 0s
apple / swift (push) Canceled after 16s
apple / screenshots (push) Canceled after 0s
arch / build-publish (push) Canceled after 0s
ci / rust (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
ci / bun-nix (push) Canceled after 0s
deb / build-publish (push) Canceled after 0s
deb / build-publish-host (push) Canceled after 4s
deb / build-publish-client-arm64 (push) Canceled after 4s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
flatpak / build-publish (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 0s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 4s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
Reviewed-on: #97
2026-08-07 16:20:54 +00:00
enricobuehler e9ddc3c561 Merge pull request 'AV1 stops shipping half a frame' (#95) from worktree-av1-subframe-truncation into main
android / android (push) Canceled after 22s
apple / swift (push) Canceled after 57s
apple / screenshots (push) Canceled after 0s
arch / build-publish (push) Canceled after 46s
ci / rust (push) Canceled after 54s
ci / rust-arm64 (push) Canceled after 28s
ci / docs-site (push) Canceled after 14s
ci / web (push) Canceled after 21s
ci / bun-nix (push) Canceled after 15s
deb / build-publish (push) Canceled after 0s
deb / build-publish-host (push) Canceled after 39s
deb / build-publish-client-arm64 (push) Canceled after 35s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 17s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 16s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 17s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 9s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 10s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 9s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 8s
docker / deploy-docs (push) Canceled after 0s
flatpak / build-publish (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 32s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
windows-host / package (push) Canceled after 2m28s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
Reviewed-on: #95
2026-08-07 16:19:28 +00:00
enricobuehler a1b286245c Merge pull request 'Windows audio substrate: minted Punktfunk endpoints retire VB-Cable' (#98) from worktree-audio-substrate into main
android / android (push) Canceled after 32s
ci / bun-nix (push) Successful in 22s
apple / swift (push) Canceled after 33s
apple / screenshots (push) Canceled after 0s
arch / build-publish (push) Canceled after 36s
ci / rust (push) Canceled after 35s
ci / rust-arm64 (push) Canceled after 45s
ci / web (push) Canceled after 45s
ci / docs-site (push) Canceled after 44s
deb / build-publish (push) Canceled after 31s
deb / build-publish-host (push) Canceled after 27s
deb / build-publish-client-arm64 (push) Canceled after 12s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 21s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 17s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 17s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 20s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 3s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 6s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 4s
windows-host / package (push) Canceled after 59s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
decky / build-publish (push) Failing after 10m34s
Reviewed-on: #98
2026-08-07 16:18:56 +00:00
enricobuehler 8551e88fcb merge: bring current main into the audio-substrate branch
ci / bun-nix (pull_request) Successful in 47s
ci / web (pull_request) Successful in 1m19s
apple / swift (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m27s
ci / rust-arm64 (pull_request) Successful in 4m25s
android / android (pull_request) Successful in 5m59s
ci / rust (pull_request) Successful in 7m19s
Two conflicts, both unions of independent removals/fixes: main fixed the
same three install.rs SAFETY comments this branch fixed (main's phrasing
kept), and the runner provisioning drops BOTH env lines — main removed
PF_FFVK_VULKAN_INCLUDE (pf-ffvk is gone since the FFmpeg replacement),
this branch removed VBCABLE_DIR (the retirement).
2026-08-07 17:49:50 +02:00
enricobuehler bca63cd9ff docs(client/video): name the upstream rav1d issue next to the workaround
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m13s
apple / swift (pull_request) Successful in 1m30s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m53s
ci / bun-nix (pull_request) Successful in 25s
ci / docs-site (pull_request) Successful in 1m23s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m36s
ci / web (pull_request) Successful in 2m22s
android / android (pull_request) Successful in 3m41s
ci / rust (pull_request) Successful in 6m21s
memorysafety/rav1d#1497, filed with the one-line fix and a reproducer that
needs no capture — any AV1 stream with one temporal unit removed. Written down
where the setting is, because the next person to read `av1_settings` and
wonder whether the floor is still needed should be able to check rather than
re-derive it.
2026-08-07 17:48:54 +02:00
enricobuehler eb9203b115 fix(client/settings): a byte order mark stops silently erasing every setting
ci / bun-nix (pull_request) Successful in 40s
ci / web (pull_request) Successful in 1m14s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m12s
ci / docs-site (pull_request) Successful in 1m27s
apple / swift (pull_request) Successful in 1m34s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m56s
android / android (pull_request) Canceled after 2m30s
ci / rust (pull_request) Canceled after 2m17s
windows / build (x86_64-pc-windows-msvc) (pull_request) Canceled after 1m8s
`Set-Content -Encoding UTF8` writes a UTF-8 BOM, and every Windows how-to
reaches for it, so `%APPDATA%\punktfunk\client-windows-settings.json` edited
from a shell arrives with `EF BB BF` in front of the `{`. serde_json rejects
that at byte 0 — correctly, JSON has no BOM — and
`.and_then(|s| serde_json::from_str(&s).ok())` turned the refusal into
`Default`. Every setting in the file, gone, with the file plainly correct on
screen and not one word anywhere about why.

Cost an hour on 08-07: a `codec: "av1"` edit was ignored and the client
negotiated HEVC. The obvious suspects — the negotiation, the caps, the host —
were all working exactly as designed.

So the mark is stripped, which is what every other JSON consumer on Windows
does. But the BOM is only the instance; the bug is the `.ok()`, which hides a
trailing comma, a truncated write and a hand-edit typo just as completely.
Those now cost one `warn!` naming the file and serde's own line and column. A
file that cannot be READ at all is reported too, and for the same reason: PowerShell's
`-Encoding Unicode` writes UTF-16LE, `read_to_string` rejects it as invalid
UTF-8, and that lands in exactly the same hole.

The RESULT is deliberately unchanged — `Default`, never an error. Nothing about
streaming may hinge on a settings file being readable, and refusing to start
because one is malformed would be a worse failure than the one being fixed. A
missing file stays silent, because that is just first run.

All three of this client's JSON stores share the loader, because all three had
the identical line: the settings file, the known-hosts store (where a BOM
silently unpairs every host) and the profiles catalog.
2026-08-07 17:45:38 +02:00
enricobuehler 32ff93199f fix(client/video): a damaged AV1 frame stops killing the whole client
The software rung aborted the process — not the session, the process — the
first time a 4K AV1 stream lost a frame. Reproduced on .21 twice on 08-07,
`SIGABRT` a few hundred milliseconds after "first frame decoded".

It was never about 4K, and it was never our bitstream.

rav1d 1.1.0 kills the process on ANY decode error while it holds a single
frame context. `rav1d_submit_frame`'s `c.fc.len() == 1` branch calls
`rav1d_decode_frame` inline; that always finishes in
`rav1d_decode_frame_exit`, which does an unconditional
`mem::take(&mut f.frame_hdr)` (decode.rs:4873); and then, only if the decode
returned `Err`, the same branch re-enters a local `on_error` whose first act is
`f.frame_hdr.as_ref().unwrap()` (decode.rs:4997) — on the `None` the teardown
just left. The panic unwinds into `dav1d_send_data`, which is `extern "C"`, so
it is `panic_cannot_unwind` → `abort()`: no `catch_unwind` at our call site, no
rung demotion and no `NoSoftwareRung` refusal can catch it. The same code is in
upstream `main` today, and 1.1.0 is the newest release, so there is no version
to bump to.

4K was only where an error first HAPPENED. The CPU rung cannot keep up at
3840x2160 (35-39 fps against a 60 fps stream), so the receive backlog stopped
draining, `pump::data` flushed it and jumped to live, and the next AU
referenced frames nobody had decoded. libdav1d gives the identical verdict on
the identical capture — 13 frames, then "Invalid data found when processing
input" — and simply carries on. At 1080p the rung keeps up, nothing is ever
flushed, no AU is ever damaged, and the same code ran for years without
anybody seeing this.

So the fix is to stop asking rav1d for the configuration whose error path is
broken. `c.fc.len() > 1` never calls `rav1d_decode_frame` at all: it hands the
frame to `rav1d_task_frame_init` and errors come back through `cached_error` /
`task_thread.retval` as ordinary `EINVAL`s, which the pump already answers with
a keyframe request. Measured, against the captured 4K stream:

  n_threads=8 max_frame_delay=1  -> n_fc=1 -> ABORT
  n_threads=1 max_frame_delay=1  -> n_fc=1 -> ABORT
  n_threads=1 max_frame_delay=2  -> n_fc=1 -> ABORT   <- proves the rule
  n_threads=8 max_frame_delay=2  -> n_fc=2 -> 13 pictures, EINVAL, survives
  n_threads=8 max_frame_delay=0  -> n_fc=3 -> survives

The third row is why `n_threads` grows a floor of two as well as the delay:
`n_fc` is `min(max_frame_delay, n_threads)`, so one decode thread silently puts
the whole thing back on the aborting path. That row is also what rules out the
theory this investigation started with — pinning threads to 1 was the suspected
trigger, and it makes things WORSE, so the tile workers are innocent and the
single frame context is the entire defect.

Two frame contexts would normally cost a frame of latency, and this does not,
because `decode` now drains PAST the first `EAGAIN`. `rav1d_get_picture` only
reaches its blocking `drain_picture` on a call whose own `drain` flag is already
set, and that flag is set by the PREVIOUS `get_picture` and cleared by every
`send_data` that carried bytes — so the first `EAGAIN` after a send does not
mean "no picture for this AU", it means "ask again", and this AU's frame comes
out of the second call. Stopping at the first `None` is what a
single-frame-context reading of dav1d's API teaches, and it would have put the
pipeline two frames behind while looking perfectly healthy. Measured over 14
temporal units at `n_fc = 2`: stopping at the first `None` produces nothing at
all for units 0 and 1; draining past it produces one frame per unit from unit 0,
at 20-42 ms per unit against `n_fc = 1`'s 21-53 ms. Not a trade — same cadence,
slightly faster, because the tile workers overlap the drain.

`Av1Software::new` then asks rav1d itself, through `dav1d_get_frame_delay`,
what those settings actually bought, and refuses to open a decoder that would
run with one frame context. That is not a restatement of the arithmetic: it is
`get_num_threads`' own answer, so it stays right if rav1d's derivation changes.
It is there because the failure it guards is uniquely quiet — an edit that
reinstates `n_fc = 1` costs nothing at build time, nothing in the tests and
nothing on a clean link, and then kills the client the first time a frame
arrives damaged. Losing the rung is recoverable; `abort()` is not.

On glass, .21, 35-second sessions, `PUNKTFUNK_DECODER=software`:

  4K60 AV1   before: SIGABRT on the second frame, every run
             after:  exit 0, 0 panics, 35-39 fps, 1204 frames, decode_failed=0,
                     and 13 decode errors recovered from across 17 backlog
                     flushes — the exact condition that used to abort, survived
                     thirteen times in one session
  1080p AV1  after:  40 fps, decode p50 2.2 ms (2.1 ms before the change)

What this does NOT buy: rav1d has other `unwrap()`s, and because its whole
public surface is dav1d's `extern "C"` ABI — every internal `rav1d_*` entry
point is `pub(crate)` — no in-process guard can turn one of them into anything
but an abort. This removes the one we hit and can prove; it does not make the
CPU rung panic-proof, and the evidence table says so.

Reported upstream with a self-contained reproducer: the in-tree
`test-25fps.ivf.av1` vector with one temporal unit dropped aborts rav1d at
`n_fc = 1`, survives at `n_fc = 2`, and libdav1d decodes it with 145 error
reports and no crash.
2026-08-07 17:45:21 +02:00
enricobuehler 0ee690f8bc fix(client/hevc): the DPB we demanded was the level's ceiling, not the stream's need
ci / bun-nix (pull_request) Successful in 34s
ci / web (pull_request) Successful in 1m12s
ci / docs-site (pull_request) Successful in 1m12s
apple / swift (pull_request) Successful in 1m32s
apple / screenshots (pull_request) Skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m18s
ci / rust-arm64 (pull_request) Successful in 2m30s
android / android (pull_request) Successful in 3m21s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m21s
ci / rust (pull_request) Successful in 5m42s
A punktfunk client streaming HEVC from .21 (RTX 5070 Ti) refused every access
unit with "stream needs 17 DPB slots, device caps at 16", flushed, waited for an
IRAP, got a fresh IDR that needed 17 too, exhausted the decode ladder and
reconnected with HEVC excluded. On a build with no software HEVC decoder — there
is no permissively licensed one — that is not a slower path, it is losing the
codec.

The host was blameless. Reading the SPS it actually emitted: general_level_idc
153 (L5.1 High, which NVENC autoselects at hevcConfig.level = 0 because a
130 Mbps target does not fit L5.0's 100 Mbps ceiling) and
sps_max_dec_pic_buffering_minus1 = 5 — six pictures, RFI_DPB references plus the
current one. Six, at every resolution. That is already the minimum the encoder
can honestly declare, and the only host-side lever, the level, cannot be lowered
without signalling a bitrate the stream exceeds. There was nothing to fix there.

dpb_limit was reading equation A-2 instead. A-2 is a CEILING on what an SPS may
signal — 7.4.3.2.1 constrains sps_max_dec_pic_buffering_minus1 to
0..=MaxDpbSize-1 — not a statement of what a stream needs, and it branches on
picture size against the LEVEL's MaxLumaPs. At 1080p the coded 1920x1088 =
2 088 960 luma samples fall under MaxLumaPs(L5.1) >> 2 = 2 228 224, taking the
first branch for min(4 * MaxDpbPicBuf, 16) = 16. max(A-2, buffering) then
reported 16 where the stream had asked for 6, the backends added one slot for the
picture in flight, and 17 is one more than NVIDIA's maxDpbSlots.

A resolution sweep on the box drew A-2's branch table exactly, and it is the two
commonest streaming resolutions that lost the codec:

  720p  1280x720  = 921 600     branch 1 -> 16 frames, 17 slots  82 refusals, HEVC dropped
  1080p 1920x1088 = 2 088 960   branch 1 -> 16 frames, 17 slots  41 refusals, HEVC dropped
  1440p 2560x1440 = 3 686 400   branch 2 -> 12 frames, 13 slots  clean
  4K    3840x2176 = 8 355 840   else     ->  6 frames,  7 slots  clean, decode 1.9 ms

One host, one level, one six-picture requirement. Only which branch the picture
size landed in decided whether HEVC worked. That is also why this hid for so
long: 4K was the resolution it was exercised at, and 4K is the one size that
falls through to the honest answer. H.264 escaped for an unrelated reason — its
own level-derived ceiling happened to land at 13 for 1080p L5.0 and 5 for 4K
L5.2 — but it is the same shape of derivation and would fail the same way if
NVENC ever picked a higher level for a smaller picture.

So dpb_limit now returns the stream's own sps_max_dec_pic_buffering_minus1 + 1,
capped at 16. That is not a workaround, it is what the number means: it is
exactly the bound C.5.2.2's fullness clause bumps against, and A.4.1 bounds the
total RPS entries by the same value, so `buffering` pictures hold `buffering - 1`
references plus the current one with nothing left over.

The max() that produced the 16 was written to be generous to malformed streams —
"storing their pictures beats erroring the AU" — but it never did that either.
Dpb::needs_bumping (C.5.2.2) already keys on the signalled buffering, not on
max_num_pics, so a stream referencing more pictures than it declared was ALREADY
being bumped below its own declared depth before every store. The widened limit
bought no tolerance at all; all it ever did was over-allocate hardware surfaces,
by ten pictures per session at 1080p, and on NVIDIA take HEVC away entirely.

The fix moves 720p and 1080p onto the pool shape 4K has been running in the field
all along (7 slots, 6 references), so it is not a new operating point — it is the
one already proven. max_active_references drops from 15 to 6, still above the 5
an RFI_DPB stream can name. The per-AU level gate in pf-vkdecode reads
plan.picture.level_idc directly, so dropping A-2 out of NegotiationInfo costs no
sensitivity to a mid-stream level change.

Two regression tests pin the arithmetic from both ends, because either end
drifting back reproduces this:

  - h265: the field SPS synthesized byte for byte on the fields that matter must
    plan 6 frames / 7 slots, all four resolutions must agree because the stream
    does, and every depth the envelope gate admits must leave room for the picture
    in flight. The one honest residue is pinned too and deliberately left
    refusing: A.4 does let a conforming stream declare a full 16-picture DPB, and
    17 slots genuinely do not fit 16, so that stream is still refused rather than
    decoded with too few slots and silently corrupted references.

  - pf-encode: RFI_DPB + 2 <= 16, guarding the producer end. RFI is a real
    latency win and this does not cap it at today's value — there are nine slots
    of headroom — it just stops it being raised past the point where clients can
    no longer decode us at all.
2026-08-07 17:34:17 +02:00
enricobuehler 6348334eff docs(client/video): the evidence table stops saying AV1 never decoded
ci / bun-nix (pull_request) Successful in 27s
ci / web (pull_request) Successful in 1m15s
ci / docs-site (pull_request) Successful in 1m19s
ci / rust-arm64 (pull_request) Successful in 1m40s
apple / swift (pull_request) Successful in 1m34s
apple / screenshots (pull_request) Skipped
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m17s
android / android (pull_request) Successful in 3m33s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m4s
ci / rust (pull_request) Successful in 5m32s
Two of its notes became false the moment the host stopped truncating AV1.

native D3D11VA / AV1 said "NEVER decoded a frame on any hardware". It has
now decoded 4K60 on an RTX 3500 Ada — and the same run is why the note
matters: its warn line named the rung as unproven moments before it
failed 72 access units running with "reference picture N holds no DPB
slot". That was the host shipping half of every frame, not the rung, so
the M7 wiring was right all along.

It stays UNVERIFIED regardless. `verified` gates `native_rung_admitted` —
whether `auto` may pick this rung ahead of Vulkan Video — and one
25-second session with no frame-hash parity and no soak does not buy
that. Promoting it wants a deliberate gpu_parity-style run. The note now
says what is true instead of what is convenient.

software / AV1 said rav1d had "CPU unit tests only". rav1d has now run on
glass: 1080p AV1 decodes, and 4K ABORTS THE PROCESS. It takes an internal
error path and panics inside its own on_error (rav1d 1.1.0
decode.rs:4997, unwrap on a None frame header); the panic crosses the
extern "C" boundary in dav1d_send_data, so it is panic_cannot_unwind and
no rung demotion or NoSoftwareRung refusal can catch it. libdav1d decodes
the same 4K stream 715/715, so this is rav1d's own defect and is recorded
where the next person to reach that rung will see it.
2026-08-07 17:06:35 +02:00
enricobuehler 7cab7ae6bc feat(client/android): say when a captured pad's gyro can't reach the session
G8's Android half, and the last of the three clients. Same failure as the other
two: a controller with a gyro, in a session whose virtual pad has no motion
plane, does nothing when tilted — silently, with no way from the couch to tell
that apart from a broken sensor. The fix is the Controller type setting, so the
notice names it.

Android read neither the requested nor the resolved backend, so this needed a
plumb. What it did NOT need was a third copy of the rule. `nativePadMotionReaches`
takes the kind a pad declared and answers off `pad_motion_reaches` in
punktfunk-core, where the argument and the tests already live. The rule is
subtler than it looks — the host builds each pad from its OWN declaration and
folds what it cannot build, so neither the declaration nor the session echo
answers it alone — and every way of getting it wrong is silent. A Kotlin
transcription would have been a third thing to keep in step with the host, which
is exactly how the SDL half got it wrong the first time.

Asked once per pad, at claim, in `openExternal` — where the pad's kind is already
being declared to the host — and the answer held for the pad's lifetime on the
`ExternalPad`. Not per sample: this runs at a DualSense's full report rate.

`hasGyro` gates only the NOTICE, and defaults to false. `DsCapture` passes true —
every pad it captures is a Sony one whose IMU is a headline feature, forwarded on
the rich plane. `Sc2Capture` keeps the default, because the Steam Controller 2's
motion rides inside the opaque passthrough report that `hidReport` carries, which
nothing here may second-guess: warning about motion for a pad that never calls
`motion()` would be a notice about a feature the player never lost. The
suppression itself is on `motion()` regardless, where it costs a dead pad nothing
and stops a live one paying to send samples the host will decode and discard.

The notice sits at the BOTTOM of the stream overlay, unlike the mic-chord
confirmation at the top. The two can coincide — a pad is claimed at roughly the
moment someone might be muting — and one landing on the other would cost the user
both. It holds 6 s rather than the mic chord's 1.6: that one confirms something
the user just did, this one explains something they did not, in a sentence they
have to read. Nulled at teardown beside `onExitArmed`/`onMicChord`, for the same
reason those are — a slot closing during release must not poke Compose state on
the way out.

Not covered by tests, and this is a limit of the module rather than a choice:
`GamepadRouter` needs Android plus a live JNI handle, there is no Robolectric
here, and the predicate it defers to is pure Rust that already has its table. So
the parts that carry the reasoning are argued in comments, as `DsCapture`'s
claim/teardown ordering already is. What IS mechanically verified is the piece
that a compiler cannot catch and a device would fail on: the JNI symbol
`Java_io_unom_punktfunk_kit_NativeBridge_nativePadMotionReaches` is present and
global in the built arm64-v8a `.so`, so the `external fun` resolves rather than
throwing `UnsatisfiedLinkError` at the first pad.

Gate: `:kit:compileDebugKotlin`, `:kit:testDebugUnitTest` (62 cases, 0 failed,
read out of the JUnit XML rather than inferred from a green build — unchanged
from this branch's previous count), `:app:compileDebugKotlin` and
`:app:testDebugUnitTest` (67 cases, 0 failed), with `:kit:cargoNdkRelease`
rebuilding the JNI crate clean across all three ABIs, plus `cargo fmt --check` on
it. On-glass verification is owed on the rig the earlier legs used, and is worth
doing as one pass with the two already owed there.
2026-08-07 17:02:04 +02:00
enricobuehler aaa58ad817 feat(client/apple): say when a pad's gyro can't reach the session, and stop powering it
G8's Apple half — the UI hint 77797a9e left owed, plus the suppression, which on
this client is worth more than it was on the SDL one.

The failure being fixed is entirely silent. A controller with a gyro, in a session
whose virtual pad has no motion plane, simply does nothing when tilted: nothing
in the app says so, and from the couch a session that resolved an X-Box backend
is indistinguishable from a broken sensor. The fix is the Controller type setting,
so the hint has to name it — a badge that only said "motion unavailable" would
leave the player exactly as stuck.

Asked per pad, off what the slot declared, via the predicate punktfunk-core now
carries. `GamepadCapture` is the one client where this is naturally per pad
already: `openSlot` computes `manager.declaredKind(for:)` and puts it in
`slot.pref`, so the question is answered where the pad is opened rather than on
every sample. `GamepadType.motionReaches(declared:asked:resolved:)` is static and
pure so it can be tested without a live session; the connection's instance method
fills in the two halves it owns, and `requestedGamepad` is stored beside
`resolvedGamepad` for the same reason it exists in the Rust client — the echo is
only this pad's answer when the pad declared what we asked for.

Where Apple differs from the SDL client, and better: it never powers the IMU. The
existing code already declined to activate sensors when forwarding was off,
reasoning that with nothing to forward there is no reason to make the pad stream
gyro over Bluetooth and burn its battery — `closeSlot` is careful to power them
back down for exactly that reason. A host that built this pad a backend without a
motion plane is the same situation, so it takes the same branch. No per-sample
check, no handler attached, and a DualSense in an X-Box-class session stops paying
for a sensor nobody reads.

The hint fires only for a pad that really has a gyro (`motion.hasRotationRate`).
A gravity-only GCMotion — what an X-Box controller exposes — would otherwise
produce a notice about a feature the player never had. That is a narrower
condition than the capture path itself uses, deliberately: making the capture
gate agree is G13's job and its own change.

The badge sits in the bottom-centre stack with the muted-mic badge and the
start-of-stream banner, at every stats tier and with the overlay off, because
this is not a statistic. Unlike the mic badge it is not a control: the setting is
not reachable mid-stream on every platform and applies from the next session
anyway. So it states the fact, names the setting, and leaves after the banner's
same 6 s. Every platform including tvOS — a DualSense on an Apple TV is an
ordinary way to play, and is exactly the pad this happens to. The model owns the
expiry rather than the view, so a second pad's hint replaces the first cleanly
instead of stacking, and ending the session cancels a pending clear rather than
carrying a stale hint into the next stream.

Non-vacuity proven by mutation, not assumed: collapsing the predicate to
`resolved.hasMotion` fails 4 assertions, including the mixed-pad row that is the
whole reason it is not a session-level check. The table mirrors the Rust one row
for row — a client that disagrees with the host here either kills a working gyro
or streams ~250 Hz into a void, and both are silent.

Gate: macOS `swift build` + the FULL suite (210 tests, 5 skipped, 0 failures) with
the two new cases observed in the run's own output, and the iOS-triple typecheck
green (`arm64-apple-ios17.0`, iOS slices + hand-assembled xcframework per the
memory recipe) — the badge and the overlay it joins are on every platform, so the
macOS build alone would not have covered them. tvOS remains unverifiable from
this Mac; the badge deliberately reuses the neighbouring banner's shape rather
than introducing anything tvOS-specific.
2026-08-07 16:55:03 +02:00
enricobuehler ebc2f1cf92 feat(host/audio): the minted microphone returns to tier-0 — pitch-true
ci / web (pull_request) Successful in 1m30s
apple / swift (pull_request) Successful in 1m26s
apple / screenshots (pull_request) Skipped
ci / bun-nix (pull_request) Successful in 21s
ci / docs-site (pull_request) Successful in 2m18s
android / android (pull_request) Successful in 3m27s
ci / rust-arm64 (pull_request) Successful in 4m24s
ci / rust (pull_request) Canceled after 7m47s
The revert un-reverts, on measurement: with the per-direction stamp sets
(render = the pad-proven PCM16-device/float-mix stereo split, capture =
device-format only), micpitch reads 440 Hz in as 440 Hz out at exact
peak. The octave-low voice was the driver DEFAULT endpoints disagreeing
(stereo render vs mono capture), never a raw-crossing design. The user
called the wrong verdict — the pad program 4ch success was the
counter-evidence that reopened the case.
2026-08-07 16:53:48 +02:00
enricobuehler c16e07d746 fix(encode/nvenc): AV1 stops shipping half a frame
Every 4K AV1 frame this host encoded reached the wire truncated to its
first tile, and had since AV1 was wired up. Measured on .21 (RTX 5070 Ti,
4K60, split AUTO): each access unit carried a frame header declaring two
tile rows and a single Tile Group OBU with tg_start = tg_end = 0, so
libdav1d rejected 835 of 836 AUs with "Error parsing frame header".
NVIDIA's hardware decoder accepts the truncated stream, which is why
native Vulkan Video looked healthy at 60 fps while both conformant
software decoders — rav1d in-tree and libdav1d out-of-tree — refused
every frame and clients fell to a black screen.

The two halves of sub-frame readback are armed by different conditions.
build_init_params arms the WRITER (enableSubFrameWrite +
reportSliceOffsets) from subframe_on alone; the chunked READER
additionally requires slices >= 2, and resolve_slices returns 1 for AV1
unconditionally — before the PUNKTFUNK_NVENC_SLICES override is even
read, because AV1 partitions via tiles rather than slices. So an AV1
session asked the driver to publish its output tile by tile and then took
only the first tile with one blocking lock_bitstream.

resolve_split_subframe — the one arbitration point both direct-SDK
backends already call — now disarms sub-frame for AV1 and returns
split_mode untouched, so AV1 keeps every engine split encode gives it.
Arming the reader instead is not a drop-in alternative: poll_chunk cuts
at bitstreamSizeInBytes on the reasoning that "slices are contiguous
Annex-B", which AV1's OBUs are not.

With sub-frame disarmed and split still AUTO, the same session decodes
654/654 frames clean through libdav1d.

The test that pinned this as correct (av1_untouched, "both features are
legal together") is replaced by one that pins the disarm, and by one that
checks the reader's gate against the writer's — the comparison nothing
made. The Linux latch comment claiming the two "can't disagree" is
corrected; that claim is what made this invisible.
2026-08-07 16:50:17 +02:00
enricobuehler 52cd42e911 fix(host/audio): capture endpoints get the device format only
Live bisect on a fresh endpoint: the mix/host format keys are
RENDER-engine properties — stamped onto a capture endpoint they broke its
shared-mode graph (IsFormatSupported reported 2ch/48k OK while Initialize
failed 0x88890008 on a once-stamped fresh endpoint; unstamped it opened
fine, S3). The capture now gets ONLY the device-format key — the knob
mmsys.cpl itself writes — declaring the stereo the pins actually accept.
2026-08-07 16:49:15 +02:00
enricobuehler 5c4969fd6b fix(client/pads): the gyro cut-off asked about the session, not the pad
Supersedes the check 77797a9e shipped an hour ago. The suppression, the
log-once, and the "unknown must not suppress" rule all stand; the field it reads
does not.

77797a9e read `Welcome.gamepad` — the backend the host resolved for the SESSION
— and stopped sending motion when it had no motion plane. But the host does not
build pads from that. It builds each virtual device from that pad's own
`GamepadArrival` (`Pads::set_kind`) and falls back to the session default only
for a pad that never declares one, which is precisely why `declared_kind` exists
and why its doc comment says an explicit setting has to be re-declared per pad.

So the check had a false negative, and it is an ordinary living-room setup. Under
"Automatic" the Hello carries the ACTIVE pad's kind (`auto_pref`), so a couch
with an X-Box pad on slot 0 and a DualSense on slot 1 echoes Xbox360 — while the
host, reading pad 1's arrival, builds it a DualSense with a working motion plane.
The old check read the echo, saw no motion plane, and killed pad 1's gyro. That
is the exact failure 77797a9e's own commit message names as the worse of the two
("a false negative kills working motion"), introduced by the fix for the other
one.

The question is per pad, so the slot now carries what it declared, beside the
physical `pref` it already held. The two are deliberately separate fields
answering different questions: `pref` is the controller in the user's hands, which
is what the local feedback paths must keep reading, and `declared` is the one the
host is pretending to have.

Three facts decide the predicate, and they are written out in
`pad_motion_reaches` rather than at the call site because all three clients need
the same reasoning:

- the echo is not this pad's answer when the pad declared something else;
- the host FOLDS what it cannot build — a Switch Pro on Windows, any UHID backend
  on a host whose /dev/uhid is unusable — and nothing client-side can predict it;
- but the echo IS one observed sample of that fold, for the kind the Hello asked
  about, so it is authoritative for a pad that declared exactly that.

Hence: trust the echo when declared == asked, else fall back to the declaration.
That keeps both motivating cases — a generic pad under Automatic (declares X-Box
360, suppressed, the sweep's H5c) and an explicit Switch Pro folded to X-Box 360
by a Windows host (declared == asked, so the echo catches it, H5d) — where either
field alone gets one of them wrong. `requested_gamepad` is kept on the client
next to `resolved_gamepad` for this: the pair is what makes the echo usable per
pad, and a lone field would only tempt the next reader back into the session-level
question.

The residual gap is a pad whose declared kind differs from the session's AND gets
folded: we keep sending and the host keeps dropping. That is the direction to be
wrong in, and it is what the session-level check was worth in the first place —
wasted datagrams, not a dead gyro.

Non-vacuity proven both directions rather than assumed. Reverting to
`resolved.has_motion()` fails on the mixed-pad row; reverting to
`declared.has_motion()` (no echo at all) fails on the Switch-Pro-on-Windows row.
Each case in the table is a session someone can actually sit down to, and the
comment on each says which of the three inputs decides it.

Gate (Linux CI image, pf-lxcheck2): fmt, `build -p punktfunk-core`, `build -p
pf-client-core`, `clippy --locked --all-targets -D warnings`, and both test
suites — green, with the new case observed in the run's own `... ok` line rather
than inferred from a green gate, and pf-client-core's 163 unchanged.
2026-08-07 16:46:06 +02:00
enricobuehler 8c274d6256 fix(host/devtest): the probe asks stereo — its mono ask WAS the unopenable endpoint
Measured resolution of the 0x88890008 mystery: IsFormatSupported said the
capture accepts 2ch/48k shared while Initialize kept failing — because
the probe itself had switched to a MONO ask for frequency counting, and
this stack does not bridge channel counts on capture even under
autoconvert. Every unopenable-endpoint verdict after that switch was the
instrument, not the endpoint. Stereo ask restored; crossings counted on
channel 0.
2026-08-07 16:44:01 +02:00
enricobuehler 16e506f943 feat(host/devtest): audio-probe micpins — the driver-capability map
Exclusive+shared IsFormatSupported across {1,2}ch x {16,32}bit x
{44.1,48,96}kHz on both minted mic pins. Interrogates the DRIVER,
bypassing every endpoint-store stamping question: what the pins truly
accept decides whether the mic leg has any coherent configuration, and
whether an exclusive-mode mono open is an escape hatch. (The pad program
made its own breakthrough with exactly this instrument on the sibling
SSS driver.)
2026-08-07 16:39:48 +02:00
enricobuehler 7f822d9c41 Merge pull request 'Apple mic uplink: the microphone was never in the audio graph, so nothing ever pulled it' (#94) from fix/apple-mic-uplink into main
apple / swift (push) Successful in 1m31s
ci / web (push) Successful in 2m15s
ci / bun-nix (push) Successful in 21s
ci / docs-site (push) Successful in 1m14s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 13s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 12s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 37s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 24s
ci / rust-arm64 (push) Successful in 3m41s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 22s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 32s
docker / builders-arm64cross (push) Successful in 11s
docker / deploy-docs (push) Successful in 36s
ci / rust (push) Successful in 6m49s
release / apple (push) Successful in 9m41s
apple / screenshots (push) Successful in 5m53s
Reviewed-on: #94
2026-08-07 14:33:44 +00:00
enricobuehler 2eed9823e5 fix(host/audio): the mic pair gets the pad-proven coherent stereo stamp set
The user challenged the format-locked-pins verdict, and the pad program
is the counter-evidence: it hit the SAME 0x88890008 unopenable-endpoint
signature and cured it with a COHERENT stamp set, after which the same
driver family served 4ch happily. This branch previous attempts were
contaminated twice over — a float device-format (the pad bisect proved
the split must be PCM16 device / float mix+host) and no
AudioEndpointBuilder restart (Restart-Service Audiosrv never touches its
dependency, so endpoint configs were never rebuilt). Both mic endpoints
now get one identical coherent stereo set; the octave-low hypothesis
shifts from "raw crossing by design" to "the two endpoint stores
disagreed (stereo render default vs mono capture default)".
2026-08-07 16:33:20 +02:00
enricobuehler 9db2c06140 Merge pull request 'Gamepad UI (iOS): the field pass — a linear drum, no close chrome, a heading that leads, and strips that assemble themselves' (#93) from worktree-gamepad-ios-polish-2 into main
apple / swift (push) Canceled after 28s
apple / screenshots (push) Canceled after 0s
ci / rust (push) Canceled after 34s
ci / rust-arm64 (push) Canceled after 33s
ci / web (push) Canceled after 33s
ci / docs-site (push) Canceled after 0s
ci / bun-nix (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
release / apple (push) Canceled after 0s
Reviewed-on: #93
2026-08-07 14:33:18 +00:00
enricobuehler d2d5058d0b fix(client/apple): the mic was never in the graph, so nothing ever pulled it
ci / web (pull_request) Successful in 1m1s
ci / docs-site (pull_request) Successful in 1m15s
ci / bun-nix (pull_request) Successful in 20s
apple / swift (pull_request) Successful in 1m30s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m45s
ci / rust (pull_request) Successful in 5m43s
ROOT CAUSE, from the reporter's device log:

  16:25:49.093  mic capture: 48000 Hz, 1 ch          <- tap installed, format fine
  16:25:49.235  audio engines joined - voice processing active
  ... 13 s of session, no errors, and the 10 s silence verdict NEVER fires

The engine started clean and the tap was installed against a valid
format - so neither the format timing nor the encoder was the fault. The
tripwire fires after ten seconds of CAPTURED frames and never fired
across a 13-second session: the tap received nothing at all.

Because the capture side must be pulled, and only the render graph pulls
anything. On the combined engine the input node carried a tap and no
connection, so it was not in the graph and nobody drove it: the IO unit
came up (the recording indicator lit for a beat, then went out as the
input went idle) and not one buffer ever reached the tap. No error, no
failed start - a session that quietly sent no microphone.

The input now runs through a silent sink into the main mixer, which is
what Apple's own voice-processing sample does. outputVolume = 0 because
the mic must reach the graph and never the speaker. The split path never
needed this - a capture-only engine has the input node AS its graph - so
this broke exactly when the combined topology became the default.

Verified: swift build (macOS), swift build --triple arm64-apple-ios17.0,
swift test 208 passed. Awaiting the reporter's on-device confirmation.
2026-08-07 16:30:29 +02:00
enricobuehler 8e8d30202c fix(client/android): a Sony pad's buttons no longer wait on its calibration
Supersedes the parse gate in 26b0819f. The off-thread read, the claim token, the
teardown ordering and its bounded wait all stand — only what happens in the gap
changes.

26b0819f held every report back until the calibration read came home, so a pad
that stalled on EP0 could feel dead for up to the link's 250 ms timeout: no
buttons, no sticks, nothing. Reports are now forwarded immediately and their
motion scaled by the nominal calibration until the real one lands.

That gap is exactly the behaviour that shipped before f6de620f — acceleration
~18% short, gyro unscaled — for about a millisecond. Nobody can feel that. A
controller that ignores a button press for a quarter of a second is not in the
same category, and it is the only one of the two a user would ever report.

It is also the safer of the two conservatisms available here. The rejected third
option, forwarding motion as zeroes until the real numbers arrive, would have the
host read a still pad as being in free fall — a lie about the physical world
rather than an imprecision about it. The nominal constants are merely a slightly
wrong scale.

The token is more load-bearing under this, not less. With a gate, an unpublished
calibration meant "parse nothing"; now it means "scale nominally", so begin()
clearing the previous pad's value is the whole reason a re-claim falls back to
the nominal constants instead of silently inheriting factory numbers belonging to
a different unit — which are, in general, further off than nominal. The fallback
therefore lives in the hand-off itself (MotionCalHandoff.effective) rather than
as an elvis at the call site: restoring the gate now means changing the type's
API, not deleting three characters in onReport.

The tests moved with the contract. They assert the nominal calibration is what is
in effect during the gap, rather than merely that the slot is empty — an empty
slot is now compatible with either behaviour, so asserting on it would have let
a regression pass. Added the case the change exists for: the same raw report,
parsed either side of publication, forwards identical buttons and sticks while
its gyro and acceleration convert differently. Mutation-checked three ways —
dropping the nominal fallback fails all five cases, dropping begin's clear fails
the inheritance case, dropping the token check fails three.

Gate: `:kit:compileDebugKotlin`, `:kit:testDebugUnitTest` and
`:app:compileDebugKotlin` green on a forced clean rerun, 62 cases across the
module, 0 failed, with the five hand-off cases read back out of the JUnit XML.
The on-glass re-verification f6de620f owes is still owed and unchanged.
2026-08-07 16:23:48 +02:00
enricobuehler 428377eaa1 Merge pull request 'Intel Arc never used Vulkan Video because our pNext order fed the decode caps to the wrong struct' (#92) from worktree-probe-codec-names into main
audit / bun-audit (plugin-kit) (push) Successful in 19s
audit / bun-audit (sdk) (push) Successful in 21s
audit / bun-audit (web) (push) Failing after 20s
audit / cargo-audit (push) Successful in 32s
audit / docs-site-audit (push) Successful in 15s
audit / pnpm-audit (push) Failing after 39s
apple / swift (push) Successful in 1m28s
ci / rust-arm64 (push) Successful in 1m25s
ci / web (push) Successful in 1m37s
ci / bun-nix (push) Successful in 27s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 4m47s
audit / license-gate (push) Successful in 5m7s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 10s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 11s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
deb / build-publish-client-arm64 (push) Successful in 2m33s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 49s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m21s
android / android (push) Successful in 8m3s
deb / build-publish (push) Successful in 7m40s
deb / build-publish-host (push) Successful in 7m22s
docker / builders-arm64cross (push) Successful in 5s
release / apple (push) Successful in 9m44s
windows-host / package (push) Successful in 11m14s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 16s
docker / deploy-docs (push) Failing after 1m41s
arch / build-publish (push) Successful in 13m59s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m27s
flatpak / build-publish (push) Successful in 8m26s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m37s
apple / screenshots (push) Successful in 5m47s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m6s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m18s
nix / flake (push) Successful in 15m19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m14s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 24m25s
Reviewed-on: #92
2026-08-07 14:13:14 +00:00
enricobuehler 26b0819f5c fix(client/android): plugging in a Sony pad could hitch the interface
Supersedes the synchronous calibration read f6de620f shipped an hour ago. The
ordering it protected is kept; the blocking it cost is not.

f6de620f read the pad's calibration inline in DsCapture.startUsb, which runs on
the main thread — the stream's setup path, and the USB-permission broadcast. The
read is a blocking EP0 control transfer: a pad that is there answers in about a
millisecond, but a pad that is stalling takes the link's whole 250 ms write
timeout, and either way the interface was waiting on a controller. That is the
wrong thread for it.

It now runs on its own daemon thread, one per claim, named pf-ds-cal — the same
shape HidUsbLink already uses for its reader rather than a second style. A
pathological stall now delays the pad's motion by a moment instead of freezing
the UI.

What kept the ordering honest before was "assign the calibration before `model`",
since `model` is what lets the link thread into the parse. That reasoning stands,
so the gate simply moved: MotionCalHandoff holds the claim's calibration, starts
null, and onReport parses nothing until it lands. No report is ever scaled by the
last pad's numbers — those are per unit — nor by the nominal fallback the real
read is about to replace. Dropping the first millisecond of a capture costs
nothing: the reports carry absolute state, so the next one says everything the
dropped one would have.

The calibration is what got deferred, not `model`, and that is deliberate.
Keeping `model` synchronous keeps isActive, the teardown writes, the feedback
sinks and the active-changed true/false pairing meaning exactly what they meant
yesterday — and, more to the point, it makes a late completion structurally
unable to resurrect a dead capture. A straggler can only ever publish a
calibration, and nothing is parsed while `model` is null.

Teardown, which is where this sort of change actually bites. Both stop() and the
unplug path end the claim before they close anything: ending burns the token, so
a read that lands afterwards publishes nothing and says so in the log. They then
wait, bounded at 500 ms and normally already over, for the read to let go of the
connection they are about to close — closing a descriptor with a transfer in
flight pulls it out from under the kernel, the same rule the pad-audio borrow
follows. It cannot deadlock: the reading thread blocks on the EP0 transfer and on
the hand-off's own monitor, never on anything a teardown holds. If a pad has
stopped answering entirely the wait elapses and teardown proceeds regardless,
which is the same exposure the feedback writes already carry and better than an
interface that never comes back.

Tested where it is testable. MotionCalHandoff is the piece that carries the
hazard and it is pure, so it has its own test: nothing is visible until the read
lands, a read that outlived its claim publishes nothing, a re-claim never
inherits the previous pad's calibration, and a doubled end still refuses every
outstanding token. Mutation-checked both ways — deleting the token check fails 3
of them, deleting begin's clear fails the fourth.

Not covered: DsCapture's own claim/teardown ordering is not unit-testable in this
module — there is no Robolectric, and the class builds a main-Looper Handler and
needs a UsbManager — so it is argued in comments rather than pinned. The on-glass
re-verification f6de620f owes is unchanged and still owed.

Gate: `:kit:compileDebugKotlin`, `:kit:testDebugUnitTest` (61 cases across the
module, 0 failed) and `:app:compileDebugKotlin` green, with the four new cases
confirmed present in the JUnit XML rather than assumed from a green build.
2026-08-07 16:10:22 +02:00
enricobuehler 7e34ca3d5e fix(client/apple): a session that can't voice-process still sends its microphone
ci / bun-nix (pull_request) Successful in 25s
ci / docs-site (pull_request) Successful in 1m7s
ci / web (pull_request) Successful in 1m8s
apple / swift (pull_request) Successful in 1m33s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m29s
ci / rust (pull_request) Successful in 5m44s
Field report: mic uplink dead on iOS, iPadOS and macOS alike, while
Android on the same host works - so the host and the wire are fine.

Two defects in the combined (voice-processing) engine, which became the
default on all three Apple platforms a week ago and has never run on a
device - CI only runs swift test on macOS, and the loopback test counts
datagrams without decoding them.

- The tap read the input format before the engine was prepared. Enabling
  voice processing swaps the engine's IO unit for the VPIO one and
  renegotiates its formats; until prepare() the input node can still
  report the pre-swap state, 0 Hz / 0 channels included, which
  installMicTap correctly refuses as 'no usable input device'. Both
  topologies now prepare first, so the chain is built against what the
  voice processor actually emits.

- A mic chain that failed on the voice-processed engine took the whole
  uplink down for the session: that arm fell back to playback ONLY. The
  sibling failure a few lines above - the voice processor refusing to
  engage at all - already falls back to the split path, which is a
  working mic without echo cancellation. Both arms do that now. The mic
  outranks the AEC.

Not reproduced locally (no Punktfunk entries in this Mac's log store,
and collecting the device's log needs root), so this is a strong
inference plus one proven logic defect rather than a confirmed fix. If
it persists, Console filtered to subsystem io.unom.punktfunk / category
audio names the stage: 'mic capture: N Hz' then 'audio engines joined'
then, 10 s in, either 'mic uplink OK - peak ...' or the SILENCE warning.

Follow-up worth doing separately: nothing reports whether the uplink
actually opened, so the HUD offers a Mute Microphone button over a
session sending nothing. Android gates that on a real micRunning signal.
2026-08-07 16:09:33 +02:00
enricobuehler cc53b3d6b0 fix(host/audio): unwire the minted microphone — the driver mic path is unusable
Final pitch-probe verdict on the SSM driver pair: the render pin is
stereo-only, the capture pin mono-only (stamping either differently makes
the endpoint unopenable), and the crossing between them is a RAW byte
pass — so voice fed through the render endpoint reads back an octave low
and no format stamp can fix it. S3 peak-based PASS = false pass; per the
design doc revert clause the mic falls back to the name ladder (a virtual
cable), pending the user re-decision. The SPEAKERS substrate keeps tier-0
(no driver crossing — a plain engine loopback tap, measured clean).

minted_ids() publishes speakers only; the mic endpoints stay minted and
recorded (provisioned()) for the micpitch probe and a possible future
non-render transport, and their format stamps now pin each side to its
pin one true format — healing the endpoints this branch earlier
mis-stamped.
2026-08-07 16:06:56 +02:00
enricobuehler 5922cbe325 fix(host/audio): the minted mic pair declares stereo on BOTH sides
Second measurement round: the driver render pin is STEREO-ONLY — the
mono render stamp turned the endpoint unopenable (0x88890008 on every
open, the incoherent-stamp signature the pad program documented). Since
the crossing is raw, the coherent choice inverts: the CAPTURE side now
declares the stereo float stream that actually crosses (fixing the
octave-low voice), and the render has its stereo float default stamped
explicitly — pinning the pair AND healing any endpoint a previous build
left mono-stamped.
2026-08-07 16:01:39 +02:00
enricobuehler ed98814145 fix(host/audio): the minted microphone renders MONO — voice was an octave low
Measured with the new pitch probe: 440 Hz into the minted mic render came
back as 220 Hz off its capture side. The driver forwards the render
stream RAW into its mono capture, so a stereo-declared render (the
driver-default we inherited) turns every stereo frame into two mono
samples — half speed, octave down, exactly the field report. The mic
render now gets a coherent MONO 48 kHz format set stamped alongside its
name (PCM16 device format + float mix/host formats), making the engine
downmix before the driver crossing. The mic pump keeps pushing stereo;
shared-mode autoconvert handles the rest.
2026-08-07 15:57:19 +02:00
enricobuehler 0cf76af0c2 feat(host/devtest): audio-probe measures pitch, not just peaks
Field report through the minted microphone: voice plays back an octave
low. Peaks are pitch-blind — S3 passed while a potential half-rate link
hid in the numbers (288k samples fits both the honest and the half-speed
story). Every probe measurement now estimates the dominant frequency by
zero crossings over the signal span, and `audio-probe micpitch` runs the
decisive experiment against the LIVE minted pair: 440 Hz in, frequency
out — ~440 = pair innocent, ~220 = the stereo render stream is forwarded
raw into the mono capture.
2026-08-07 15:53:00 +02:00
enricobuehler adf92939ec fix(client/apple): the stats card's corner stops eating its own text
ci / bun-nix (pull_request) Successful in 28s
ci / web (pull_request) Successful in 1m21s
apple / swift (pull_request) Successful in 1m28s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m33s
ci / rust-arm64 (pull_request) Successful in 1m53s
ci / rust (pull_request) Successful in 4m39s
Making the HUD concentric with the physical display corner had no upper
bound, so a modern phone (~62 pt of display radius) asked for a 48 pt
corner on a card whose lines sit 10 pt from the edge. A corner of radius
r pulls the edge inward by r - sqrt(r^2 - (r-y)^2) at distance y below
the top: at the first line that is ~19 pt, so the top and bottom lines
rendered INSIDE the arc.

Concentricity is only a virtue while the radius is small next to the
card. The radius is now capped at 28 (devices asking for less still get
a truly concentric corner) and the iOS content padding scales with it at
0.45*r, which leaves ~4.6 pt of arc against 12.6 pt of padding at the
cap. The card grows by under 3 pt a side; the compact pill is unchanged.
2026-08-07 15:49:37 +02:00
enricobuehler f6de620f34 fix(client/android): a captured Sony pad's gyro turned the wrong amount
G14/G16 leg 3. This supersedes the nominal constant 0e40b374 shipped, which was
always labelled a stopgap.

Measured on glass 2026-08-07: a DualSense over USB into an Android phone,
streaming to a Linux host, flat and face up, arrived as |accel| = 0.811 g where
1.000 was owed. The parse forwarded the pad's raw i16s verbatim, and raw device
units are not wire units. 0e40b374 rescaled acceleration by the nominal
10000/8192 and deliberately left gyro alone, because a constant provably cannot
fix gyro: the same still average showed this unit's accel calibration is
near-identity (~1% off) while its gyro's emphatically is not — a near-identity
gyro calibration would imply 1024 LSB per deg/s, i.e. ±32 deg/s full scale, which
no controller has. That scale is per unit, and the only thing that knows it is
the pad.

So the client now asks. HidUsbLink grows a GET_REPORT path — EP0, the exact
mirror of the SET_REPORT it already had — and DsCapture reads the pad's IMU
calibration feature report ONCE, while claiming it: 0x05 / 41 B on a DualSense or
Edge, 0x02 / 37 B on a USB DualShock 4. DsDevice.MotionCal then applies
hid-playstation's own arithmetic per axis, which is the same math the host's
contract test (crates/pf-inject/tests/motion_contract.rs, SonyImuCalibration)
reads from the other end: gyro raw × speed_2x × 20 / (|plus−bias| + |minus−bias|),
accel (raw − (plus − range/2)) × 20000 / range. Long arithmetic, because the gyro
multiplier overflows an Int, and clamped, because both are >1 multipliers and a
full-scale flick would otherwise wrap the i16 into a motion in the opposite
direction. Reading the blob also removes acceleration's residual ~1% factory bias
that the nominal constant left behind.

Once at claim and never per report. EP0 is independent of the interrupt endpoints
so the read is safe alongside the reader thread, but a blocking control transfer
in the report path would wreck capture latency, and the calibration is fixed for
the life of the connection anyway. The capture logs the derived resolutions, which
is the discriminator for whether a blob was read at all: a real pad declares ≈16
LSB per deg/s, the fallback reads back as exactly 20.

A pad that refuses, answers short, or declares zeroes (a clone, a broken unit)
keeps today's behaviour per axis — nominal accel, gyro straight through. Nothing
here ever zeroes motion: slightly mis-scaled beats silent.

Not covered. The axis frame is still untouched: this leg puts gravity on Y where
the Apple leg put it on Z, so at least one client's frame is wrong, and settling it
needs the bare-metal Linux reference reading G16 step 1 calls for. Rescaling is
frame-independent, so it stands however that resolves — remapping is not, so it
stays out. Bluetooth's grouped plus/minus layout is not implemented either: this
path is USB-only by construction (Android exposes no raw path to a Classic pad),
and a half-used generalisation would be a latent bug rather than a feature.

Gate: `:kit:compileDebugKotlin` + `:kit:testDebugUnitTest` green, 16 DsDeviceTest
cases run 0 failed, and the five new ones were confirmed present in the JUnit XML
rather than merely compiled. Non-vacuity checked by mutation — perturbing the gyro
conversion fails 6 tests, including all four new ones that assert a number.

On-glass re-verification owed, on the rig that measured the defect (DualSense →
USB → phone → 192.168.1.21): at rest |a| = 1.00 g exactly via ~/gyroscope.py, and
a nominal 90 deg yaw integrating to ~90 deg via ~/integrate.py — the same 90 deg
that read ~62.7 deg before this change.
2026-08-07 15:47:33 +02:00
enricobuehler b166c53cc2 fix(vkdecode): a refused device says what the refusal costs, and about which format
ci / web (pull_request) Successful in 1m8s
apple / swift (pull_request) Successful in 1m33s
apple / screenshots (pull_request) Skipped
ci / bun-nix (pull_request) Successful in 1m33s
ci / docs-site (pull_request) Successful in 2m2s
ci / rust-arm64 (pull_request) Successful in 2m25s
android / android (pull_request) Successful in 3m1s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m13s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m3s
ci / rust (pull_request) Successful in 11m15s
nix / flake (pull_request) Successful in 12m49s
Three things the Intel Arc measurement showed were wrong or unhelpful in the refusal
path.

The message named NV12 whatever the stream was. A Main 10 session refused over P010
was told about NV12, which sends the reader to look up the wrong format's support.
Both variants now carry the format the driver's own entry reported.

A missing SAMPLED now says what it costs. "does not advertise usage SAMPLED" is
accurate and tells a field reporter nothing: the consequence is that no shader can
read this device's decoded pictures, so the zero-copy path cannot exist on it at all
— which is a different conversation from a device that is merely slower. The line
points at --probe-decode for the driver's own words.

And the probe's second opinion no longer claims to be one. Measured on both vendors,
vkGetPhysicalDeviceImageFormatProperties2 answers "creatable" for combinations the
video-format query rejects — on NVIDIA too, for SAMPLED alone, which is not a legal
video image usage at all. So it does not honour the chained profile list and must not
be read as permission; it is still printed, because otherwise everyone who reads a
refusal asks the question again, but it is labelled as not authority.

Also names the three video ENCODE usage bits, which NVIDIA advertises on decode
pictures and the probe was printing as "unrecognised 0xC000".
2026-08-07 15:44:07 +02:00
enricobuehler 9582da07b4 fix(client/apple): the entrance goes underneath the scroll transition
apple / swift (pull_request) Successful in 1m31s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m15s
ci / rust-arm64 (pull_request) Successful in 1m29s
ci / bun-nix (pull_request) Successful in 19s
ci / web (pull_request) Successful in 1m41s
ci / rust (pull_request) Canceled after 10m43s
Structural, because tuning the transform values was treating a symptom.
A scroll transition derives its phase from the geometry of the view it
wraps, and the entrance was wrapping each card on the OUTSIDE - so it
moved the very thing the transition measures. Every card read as far
from centre for the whole travel, phase pinned at fully receded, and the
centred card only collapsed into its focused look as the entrance ended.
That collapse was the jump; shrinking the offset last round only made it
smaller.

The card builder now hands each caller its own CardEntrance and both the
launcher and the coverflow apply it BENEATH their .scrollTransition. The
transition measures a card that never moves and composes its scale and
rotation on top of the entrance's, so the two can no longer fight - and
the fuller travel is back (34 pt rise) now that the geometry constraint
that forced it down to 16 is gone.
2026-08-07 15:38:38 +02:00
enricobuehler 48339326ae fix(client/apple): the entrance keeps every card inside its own footprint
apple / swift (pull_request) Successful in 1m25s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m7s
ci / rust-arm64 (pull_request) Successful in 1m29s
ci / rust (pull_request) Canceled after 3m34s
ci / docs-site (pull_request) Canceled after 41s
ci / bun-nix (pull_request) Canceled after 29s
The focused card jumping into its correct state at the end of the
entrance was the entrance's own geometry. The caller's .scrollTransition
reads the geometry of the view underneath the entrance's transforms, so
a card shoved 58 pt down and hinged on its leading edge spent the whole
travel reported as far from centre - phase pinned at fully receded - and
only collapsed to identity as the card came home. That collapse IS the
jump, and it explains why it looked timing-dependent rather than simply
broken.

Now the rotation is about the card's centre (it turns in place instead
of swinging sideways out of position) and the rise is 16 pt, inside the
strip's own vertical slack, so nothing the entrance does moves a card
away from where the scroll view thinks it is. The entrance also waits a
couple of frames for real layout - the GeometryReader's first pass can
report no width, so there is nothing to centre on yet - and the
transaction override from the previous round is gone: it was not the
cause, and nil-ing inherited animation could have made navigation
snappier than intended.
2026-08-07 15:34:56 +02:00
enricobuehler c06ee55b61 diag(vkdecode): --probe-decode reports what the driver says about video images
ci / bun-nix (pull_request) Successful in 38s
apple / swift (pull_request) Successful in 1m31s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 2m11s
ci / docs-site (pull_request) Successful in 2m21s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m44s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m10s
ci / rust-arm64 (pull_request) Successful in 6m29s
android / android (pull_request) Successful in 7m44s
ci / rust (pull_request) Canceled after 9m40s
nix / flake (pull_request) Canceled after 9m38s
The Intel Arc refusal moved one step down the caps query and stopped again: the
coincide NV12 entry does not advertise SAMPLED. That sentence is punktfunk's, not
the driver's, and the last two times a conclusion was drawn from a sentence of ours
the conclusion was wrong.

So --probe-decode now prints the driver's own answers instead. For every profile the
client can negotiate (H.264 High, H.265 Main and Main 10, AV1 Main 8- and 10-bit) it
asks vkGetPhysicalDeviceVideoFormatPropertiesKHR in six usage combinations — the
three the image pools really create with, plus DPB|DST without sampling, SAMPLED
alone and DST alone, which are what localise a refusal to a half. Each answer is
printed as the driver gave it: format, usage and create flags named AND in hex with
unrecognised bits called out, image type, tiling. A failed query prints its VkResult
rather than vanishing into an empty list.

It goes through pf-vkdecode's own query rather than a copy of it, which meant
splitting query_formats into a physical-device form — the call never needed the
VkDevice the old signature demanded. VideoFormat gains imageType and imageTiling to
carry the whole record; VUID-VkImageCreateInfo-pNext-06811 compares both for
equality, so they were being assumed rather than read.

And because a driver that under-reports usage would be indistinguishable from one
that genuinely lacks it, the probe asks a second, independent question —
vkGetPhysicalDeviceImageFormatProperties2 over the same profile list — and prints it
only where the two disagree. A disagreement is the finding.

No behaviour change to any decode path: derivation reads the same fields it did.
2026-08-07 15:34:12 +02:00
enricobuehler 7993386900 feat(host/audio): the minted endpoints get their names — 'Punktfunk Speakers/Microphone'
Field-measured necessity, not cosmetics: unstamped, the minted instances
read 'Lautsprecher (2- Steam Streaming Microphone)' and even the box's
owner picked the wrong device out of the Sound settings zoo (as did the
S1 probe's name match before it). The provider now stamps device-desc +
device-name through the pad program's proven machinery — write_stamps/
stamps_served, extracted from the pad-only stamp functions — with the
same store-first/registry-fallback routes and settle/re-pass discipline.
Names only: a wider stamp set makes AudioEndpointBuilder re-mint the
endpoint under a new GUID (measured on pads). Stamping is best-effort
(SYSTEM ACL route); the wiring never depends on names — identity stays
the recorded id.
2026-08-07 15:33:52 +02:00
enricobuehler eaa87ca0cc fix(client/apple): the entrance stops dragging the scroll transition along with it
ci / bun-nix (pull_request) Successful in 23s
ci / web (pull_request) Successful in 1m13s
apple / swift (pull_request) Successful in 1m32s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m59s
ci / docs-site (pull_request) Successful in 2m6s
ci / rust (pull_request) Successful in 5m50s
The strip entrance is one animated progress value now, not a Bool behind
per-card .animation modifiers. Those modifiers wrap the caller's card -
INCLUDING its .scrollTransition - so a delayed spring flipping while the
scroll view was still settling captured the transition's own per-frame
phase updates and stranded the centred card half-receded until the next
scroll re-drove it. That was the 'only navigating fixes it' report, and
the race with load speed was the same thing.

CardEntrance is now a ViewModifier + Animatable: it slices its own
window out of one master clock the carousel animates 0 -> 1, so every
transform is a pure function of an interpolated Double and no animation
modifier wraps a card at all. Benign failure mode too - progress
reaching 1 without animating leaves each card at exact identity rather
than stranded. The entrance also moved inside .frame(width:) so a scroll
target's geometry never depends on what its card is doing, and the
non-tvOS branch states its .id explicitly.

One leak remained after that: withAnimation sets its animation on the
whole TRANSACTION, so the scroll view's initial centring still inherited
the 1 s linear clock and the focused card only reached its correct look
as that clock ran out - arriving as a jump. The card subtree now clears
the inherited animation, so its phase lands per frame while the
entrance's own transforms (driven by animatableData, not by the
transaction) keep running.
2026-08-07 15:28:09 +02:00
enricobuehler 0e40b374e7 fix(client/android): DualSense acceleration arrived ~18% short
G16 leg 2. A DualSense over USB to an Android phone, streaming to a Linux host,
flat and face up: |accel| = 0.811 g where 1.000 is owed. Magnitude is
frame-invariant, so this is unambiguous regardless of the separate axis question
below, and it came from a 27-second static average — no sampling error in it.

`DsDevice` said so plainly: "Gyro/accel stay in raw device units". It read the
i16s out of the pad's report and forwarded them verbatim. But raw device units
are not wire units — the wire is fixed at 10000 LSB/g and the pads' native
resolution is the 8192 that hid-playstation calls DS_ACC_RES_PER_G. 8192/10000 =
0.819 predicted against 0.811 measured. Acceleration is now rescaled on both the
DualSense and DualShock 4 parse paths, clamped because the multiplier is >1 and
a real near-full-scale slam would otherwise wrap the i16 into an impossible
acceleration in the opposite direction.

Two things deliberately NOT done.

Gyro is left alone. It is almost certainly low by the same mechanism, but it
cannot be corrected with a nominal constant the way acceleration can: the still
average shows this pad's accel calibration is near-identity (~1% off), while the
gyro's emphatically is not — a near-identity gyro calibration would imply
1024 LSB per deg/s, i.e. ±32 deg/s full scale, which no controller has. Fixing
gyro means reading the pad's calibration feature report and applying its own
numbers, which also removes acceleration's residual 1% bias. `HidUsbLink` can
SET_REPORT but has no GET_REPORT path yet, so that is a real change rather than
a constant, and it is owed.

I tried to pin the gyro factor by integrating the on-glass rotations instead: a
nominal 90 deg yaw integrated to ~88.5 deg through the Apple client (correct)
and ~62.7 deg through Android. Directionally consistent, but the readout samples
at 5 Hz and a ~1 s rotation is badly undersampled, so that ratio is not a
constant anyone should ship. Recorded, not used.

The axis frame is also left alone. This leg puts gravity on Y where the Apple
leg put it on Z, so at least one client's frame is wrong — but Android forwards
the pad's own axis order un-remapped, which makes its reading evidence about the
hardware rather than about us, and resolving it needs the bare-metal reference
reading G16 step 1 calls for. Every bare-metal Linux box was unreachable
(Deck down, HTPC down, .25 is another KVM guest). Rescaling does not touch axis
order, so this fix stands however that resolves.

Gate: `:kit:compileDebugKotlin` and `:kit:testDebugUnitTest` green, JNI libs
built clean at the API-28 floor across 3 ABIs. On-glass re-verification owed:
re-run the at-rest reading and expect 0.99-1.00 g.
2026-08-07 15:23:10 +02:00
enricobuehler 769a8c049d fix(client/apple): the entrance waits for the art, and every card swings
ci / bun-nix (pull_request) Successful in 29s
ci / web (pull_request) Successful in 59s
ci / docs-site (pull_request) Successful in 1m4s
ci / rust-arm64 (pull_request) Successful in 1m25s
apple / swift (pull_request) Successful in 1m26s
apple / screenshots (pull_request) Skipped
ci / rust (pull_request) Successful in 4m21s
Three defects behind an entrance that read as a card sliding up:

- The centred card never rotated. The stagger fans out from an anchor,
  and the anchor was given side 0 = no rotation - but the anchor IS the
  card the eye is on, so the single most visible card only rose. Side is
  never 0 now; every card turns.

- The swing happened while the card was invisible. Opacity shared the
  transform's spring, so the card spent its whole rotation at near-zero
  alpha and only the last few degrees showed. The fade now runs on its
  own 0.22 s curve (a second .animation governs only the modifiers above
  it) while the transform springs over ~0.6 s. The travel is deeper too
  - 0.74 scale, 64 degrees, 58 pt - and the rotation sign now matches the
  coverflow's own recede, so a card unwinds INTO its resting angle
  instead of swinging against it.

- It fired before the art existed. Cards swung in as grey placeholders
  and filled with artwork afterwards. PosterImage reports when a cover
  settles (art loaded, or candidates exhausted), the coverflow counts
  the first few, and GamepadCarousel holds its entrance on a
  contentReady gate - with a 700 ms backstop so a slow or artless
  library still animates.
2026-08-07 15:16:49 +02:00
enricobuehler 2fb80073e6 fix(client/apple): the library actually plays its entrance, and it swings in 3D
ci / bun-nix (pull_request) Successful in 54s
ci / docs-site (pull_request) Successful in 1m12s
ci / rust-arm64 (pull_request) Successful in 1m28s
apple / swift (pull_request) Successful in 1m28s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m56s
ci / rust (pull_request) Successful in 4m21s
The strip entrance never ran in the library, for two reasons:

- The trigger was lost. Flipping the state inside onAppear puts the
  change in the SAME transaction as the view's insertion, where SwiftUI
  runs with animations disabled. The launcher got away with it; the
  library's strip mounts late - only once the fetch lands - and lost
  every time. The flip now defers one runloop turn, so it is an ordinary
  animated state change.

- The art snapped in behind it. Covers hard-swapped from grey
  placeholder to image, so even a working entrance was followed by a run
  of cards popping to artwork after the strip had settled. PosterImage
  cross-fades now (the touch grid inherits it).

And the entrance is 3D: a card starts turned away on the drum, small,
low and invisible, then swings flat, grows and rises on an overshooting
spring. Cards left of the anchor hinge on their trailing edge and cards
right of it on their leading one, so the strip FANS OPEN from the cursor
instead of sweeping past it - the same hinge-and-perspective language
the coverflow's own recede speaks, so arriving and scrolling read as one
object. Reduce Motion still drops every bit of travel.
2026-08-07 15:10:33 +02:00
enricobuehler ca667cb79a fix(vkdecode): the pNext order decided which struct got the decode caps
ci / web (pull_request) Successful in 1m6s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m4s
ci / docs-site (pull_request) Successful in 1m14s
apple / swift (pull_request) Successful in 1m26s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m40s
ci / bun-nix (pull_request) Successful in 2m5s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m10s
android / android (pull_request) Successful in 5m17s
ci / rust (pull_request) Successful in 6m44s
Intel Arc never used Vulkan Video decode on Windows. The rung refused every
session with "driver advertises neither DPB_AND_OUTPUT_COINCIDE nor DISTINCT"
and fell back to D3D11VA — and that refusal was ours.

vkGetPhysicalDeviceVideoCapabilitiesKHR was called with the codec capability
struct chained BEFORE VkVideoDecodeCapabilitiesKHR (push_next prepends, so the
chain was caps -> h265_caps -> decode_caps). On Arc/Windows 101.8724 the driver
fills those two by POSITION, not by sType, and returned them SWAPPED. Measured,
on glass, both ways:

  before:  decode_flags_raw=12   max_level_idc=1
  after:   decode_flags_raw=1    max_level_idc=12

12 is STD_VIDEO_H265_LEVEL_IDC_6_2 and 1 is DPB_AND_OUTPUT_COINCIDE. We were
reading an H.265 level as a decode-capability bitmask; 12 contains neither 0x1
nor 0x2, so the check concluded the device had no DPB mode. It had one all along.
The base struct was fully populated throughout — 15 DPB slots, 8192x8192 max
extent — which is what gave the lie away: a driver that answers in that much
detail is not declining.

NVIDIA and RADV dispatch by sType and do not care about the order, which is
exactly why the fleet stayed green and this reached the field. Both orders are
spec-legal for us to write; only one survives a driver that assumes the
conventional one, and the conventional one — decode caps first, as every Vulkan
sample writes it — is now what all three codecs use.

⚠ This does NOT yet give the Arc Vulkan Video. It moves the refusal one step
down the same function: the device advertises only COINCIDE (no DISTINCT), and
its NV12 coincide entry does not advertise SAMPLED usage, which the zero-copy
presenter path needs. Whether that is a second bug of ours or a real Intel
constraint is not yet established, and this commit does not claim it either way.

Found because the user disbelieved my "Intel driver bug" conclusion. He was
right: I had reasoned from our own error message, which is the same circularity
the caps logging added in fb1a0a61/a183cac8 now exists to break.

Gates: fmt clean; clippy -D warnings; 187 pf-vkdecode tests. The GPU parity legs
that cover this code cannot run here (no GPU on the build host) — the evidence
is the on-glass A/B above.
2026-08-07 15:07:48 +02:00
enricobuehler cf68d33e2c feat(client/apple): the strips assemble themselves, and the store chip stops flashing
ci / bun-nix (pull_request) Successful in 38s
ci / web (pull_request) Successful in 1m7s
ci / rust-arm64 (pull_request) Successful in 1m23s
apple / swift (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m48s
ci / rust (pull_request) Canceled after 6m16s
Two more from the on-glass pass:

- The coverflow's store/source chip only showed its background on the
  centred cover. Same mechanism as the tray blur: a card rides a
  scrollTransition that composites it with opacity < 1 and a 3D
  rotation, and a material cannot sample a backdrop through an offscreen
  composite - so the frost stayed blank everywhere except the one card
  sitting at exactly full opacity. The coverflow's chip is a flat wash
  now (StoreBadge gains `solid`), which has no backdrop to sample and
  is therefore simply always there. The touch grid keeps its material -
  its cards carry no transform, so its frost samples fine.

- Host cards and library covers now arrive with the strip instead of
  being there: each card rises out of a fade on a lightly overshooting
  spring, delayed by its distance from the cursor, so the strip
  assembles outward from where the eye already is. Implemented once in
  GamepadCarousel, so the launcher and the coverflow inherit it
  together. Transforms only - snapping, the callers' own
  scrollTransition and the tvOS focus engine are untouched - and Reduce
  Motion drops the travel for a plain unstaggered cross-fade.
2026-08-07 15:03:54 +02:00
enricobuehler 9e9bb9f466 fix(client/apple): acceleration was upside down — measured on glass
G16, first result. A DualSense paired to an iPhone, streaming to a Linux host,
lying flat and face up: hid-playstation decoded z = −0.99 g where a DualSense
owes +1.00. Vector magnitude was 1.006 g, so the scale was already correct —
this is purely direction, and it was wrong for every accelerometer sample the
Apple client has ever sent.

The cause is a convention mismatch, not a sign typo. Apple reports acceleration
as the gravity VECTOR, which points down: a device face-up on a table reads
z = −1. An accelerometer physically measures proper acceleration, and at rest
that is the +1 g normal force pushing UP — which is what a DualSense's report,
and therefore our wire, carries. The two are exact negatives. Both branches were
affected, because `m.acceleration` follows the same Apple convention as the
gravity/userAcceleration split, so reading the "raw vector" was not an escape
from it.

`rotationRate` is a true angular rate and needs no flip. The same session
confirmed that independently: rotating the pad clockwise seen from above
produced a negative yaw, which is correct under the right-hand rule about an
up-pointing Z. That asymmetry — accel wrong, gyro right — is itself evidence for
this diagnosis rather than a blanket frame error, and it is why the fix is three
negations at one site instead of a remap.

The sweep predicted this ("Apple accel plausibly INVERTED — CoreMotion gravity
-1 g vs DS +1 g up at rest") but could not confirm it without hardware. It is
now measured, and the mechanism is confirmed in the code rather than inferred
from the number.

Method, for whoever repeats it: the readout is python-evdev on the host reading
the virtual pad's own motion node, dividing by the axis `resolution` the kernel
publishes, so it prints deg/s and g. That is downstream of the calibration blob
— the same layer a game reads — which is what makes a sign error visible to a
human at all.

Two things this does NOT establish. The host was a KVM guest, so the DualSense
could not be attached natively for a side-by-side reference reading; the test
stands on the DualSense convention being a fixed property of the hardware, which
is decisive for the at-rest sign but weaker for the gyro axis ORDER. And the fix
itself is unverified on glass: confirming it needs a rebuilt client on the
device, so someone should re-run the same at-rest reading and see +1.00.

Gate: `swiftc -parse` clean. A full typecheck needs the gitignored
PunktfunkCore.xcframework assembled first and has not been run.
2026-08-07 14:56:27 +02:00
enricobuehler a183cac8aa diag(vkdecode): log maxLevelIdc beside the decode flags
ci / bun-nix (pull_request) Successful in 44s
ci / docs-site (pull_request) Successful in 1m21s
apple / swift (pull_request) Successful in 1m33s
ci / web (pull_request) Successful in 1m28s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 2m15s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m33s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m11s
ci / rust (pull_request) Successful in 5m23s
android / android (pull_request) Canceled after 6m14s
The Arc returned decode_flags=0b1100 = 12 with a fully populated base struct (15
DPB slots, 8192x8192 max extent). Neither COINCIDE (0x1) nor DISTINCT (0x2) is
set, and 0x4|0x8 are not defined for that field at all — but 12 IS
STD_VIDEO_H265_LEVEL_IDC_6_2, and VkVideoDecodeCapabilitiesKHR and
VkVideoDecodeH265CapabilitiesKHR have identical layouts (sType, pNext, one u32).

So the suspicion is that we are reading H.265's maxLevelIdc where the decode
flags belong. Logging both settles it: if max_level_idc comes back as 1 or 2 the
two structs are crossed, and the refusal is ours rather than the driver's.
2026-08-07 14:55:27 +02:00
enricobuehler fb1a0a61e9 diag(vkdecode): log the driver's video capabilities verbatim
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m9s
ci / bun-nix (pull_request) Successful in 30s
ci / web (pull_request) Successful in 1m20s
apple / swift (pull_request) Successful in 1m28s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m51s
ci / rust-arm64 (pull_request) Successful in 2m44s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m46s
android / android (pull_request) Successful in 4m16s
ci / rust (pull_request) Successful in 4m52s
Nothing in the caps module logged anything, so when a device refused with
"advertises neither DPB_AND_OUTPUT_COINCIDE nor DISTINCT" there was no way to
separate two very different situations that present identically as a zero: the
driver filling the chain and genuinely declaring no DPB mode, versus our own
pNext chain never reaching VkVideoDecodeCapabilitiesKHR at all.

Printing the BASE VkVideoCapabilitiesKHR beside the decode flags is the
discriminator. A populated max_dpb_slots next to decode_flags: 0 means the
driver traversed the chain and answered; zeros across both mean the query never
landed and the refusal is ours, not the driver's.

Raised by the Intel Arc result on .221, where I concluded "driver bug" on the
strength of our own code's report — which is precisely the circular reasoning
this line exists to break.
2026-08-07 14:50:08 +02:00
enricobuehler 70de58f446 fix(client/apple): the tray blur goes full-bleed by layout, and its frost sinks to black
ci / bun-nix (pull_request) Successful in 30s
ci / web (pull_request) Successful in 1m6s
apple / swift (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m23s
ci / rust-arm64 (pull_request) Successful in 1m53s
ci / rust (pull_request) Successful in 4m54s
Round-3 field findings: the pop-in had retreated to the X axis alone -
the one growth still left to safe-area resolution (the landscape side
insets), which settles a beat after insertion, outside any geometry
group and outside the view's own transaction. The scrim now reaches
full-bleed purely by layout: a fixed 80 pt negative-padding overhang on
the outer edge and both sides replaces ignoresSafeArea entirely, so
every axis is deterministic from the first frame (and orientation no
longer changes the gradient's proportions). The mask's strong region
moves to 0.65 to account for the overhang leading the gradient.

And the frost reads black now, not grey: an ink.shade(0.35) wash inside
the mask sinks the material's luminance lift toward the palette's shade
- black on a dark field, palette-honest on a pale one.
2026-08-07 14:50:01 +02:00
enricobuehler f674a06a08 fix(client/apple): the tray blur keeps its shape, and the tab pill turns to glass
ci / bun-nix (pull_request) Successful in 27s
ci / docs-site (pull_request) Successful in 1m4s
ci / web (pull_request) Successful in 1m9s
ci / rust-arm64 (pull_request) Successful in 1m30s
apple / swift (pull_request) Successful in 1m35s
apple / screenshots (pull_request) Skipped
ci / rust (pull_request) Canceled after 4m39s
Two follow-ups from the second on-glass pass:

- The tray blur's pop-in survived the geometryGroup: the full-bleed
  growth (negative padding + safe-area expansion) rode the push's
  transaction, and safe-area resolution sits outside a geometry group.
  The scrim now pins its own geometry out of any animation - the layer
  fade/slide still carries it, only its SHAPE can never animate. The
  bottom overshoot grows 32 -> 72 pt (the tray sits over scrolling rows
  plus the detail line; the blur influence starts well above the legend
  now) and the mask holds strength longer before dissolving.

- The selected tab pill is a Liquid Glass surface (accent-tinted
  through consoleGlass, material fallback pre-26/tvOS) - the strip
  wears the same material language as the rows below it. The
  matched-geometry travel between pills is unchanged.
2026-08-07 14:45:15 +02:00
enricobuehler 49fa344c92 fix(client/apple): the gamepad UI answers its first field test
ci / rust-arm64 (pull_request) Successful in 1m29s
ci / bun-nix (pull_request) Successful in 19s
apple / swift (pull_request) Successful in 1m22s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m48s
ci / docs-site (pull_request) Successful in 1m53s
ci / rust (pull_request) Successful in 4m25s
Six findings from the on-device pass over #91, all iOS-facing:

- The tray blurs no longer grow into place on a push: the screen layer
  resolves its internal layout (safe-area trays, the scrims' full-bleed)
  in a geometryGroup BEFORE the insertion animates.

- The option band is LINEAR now, not a ring. A ring showed the first
  option waiting to the right of the last one - unreachable, since
  left/right clamps - and on a 2-option ring the unselected item flipped
  sides with every step (the 60/120 Hz row). Positions are fixed, the
  ends are the ends, and A's wrap travels back across the list. Options
  other than the facing one exist only while the drum is moving, so a
  long label never sits under a resting neighbour as overlapping text.

- Toggles (and the pin rows) ride the band too: Off left of On,
  matching the left-off/right-on step semantics.

- The close X is gone from settings, add-host and the library - a
  gamepad UI exits with B. A chromeless cancel button keeps hardware
  Esc and the macOS sheet working, and the library's loading/error/empty
  states gain a zero-size B listener so a controller-only user is never
  trapped where the coverflow (and its B) doesn't exist yet.

- The heading is a real heading: leading-aligned with the 24 pt content
  inset, 24/34 pt (was 20/30), top margin 18/28 (was 10/18) - launcher,
  settings, add-host and library alike. The launcher's hidden-mirror
  chip trick died with the centred title that needed it.

Verified: swift build (macOS), swift build --triple arm64-apple-ios17.0,
swift test 208 passed / 0 failed.
2026-08-07 14:35:47 +02:00
enricobuehler c34e1412fb fix(client): a decoder pin with a stray space was silently ignored
ci / bun-nix (pull_request) Successful in 20s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m7s
apple / swift (pull_request) Successful in 1m26s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m32s
ci / docs-site (pull_request) Successful in 1m44s
ci / rust-arm64 (pull_request) Successful in 2m23s
android / android (pull_request) Successful in 3m9s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m7s
ci / rust (pull_request) Successful in 6m0s
Found on glass, and it cost a whole session to find. PUNKTFUNK_DECODER was read
untrimmed, so "native-vulkan " — ONE trailing space — matched no arm of
native_vulkan_gate, fell through to `auto`, and on an Intel box `auto` takes
d3d11va first. The operator's pin never ran and NOTHING said so. Read against a
log, that is indistinguishable from the rung being refused for a hardware
reason, which is precisely the ambiguity the rest of this module's logging was
just rewritten to remove.

The space is not exotic. A Windows .cmd produces it for free: `echo x>> file`
keeps the space before the redirect, so every line written that way carries one.
PUNKTFUNK_VK_ADAPTER already trimmed; this did not, and the inconsistency is what
made it invisible — the GPU override obeyed while the decoder override did not.

The rule now lives in one pure function, resolve_decoder_pref, called by BOTH
readers. decode_pinned_to_software had the identical untrimmed expression, and
its own doc comment says a second reading of the same two inputs is a second
place for them to drift — fixing one and not the other would have proved it
right. Whitespace-only counts as ABSENT rather than as a pin to "", because an
exported-but-empty variable means "no override" and "" is a value the gate
happens to accept.

Tested as a pure rule (no process environment), including the end-to-end leg
that matters: the trimmed pin reaches native_vulkan_gate and is admitted. Like
the create-array tests in dee97e89 its before-state is a compile error rather
than a failing assertion, because the function is new — what it guards going
forward is real, and an editor who drops the trim fails it.

Gates: fmt clean; clippy -D warnings over pf-client-core,
punktfunk-client-session and pf-presenter in the Linux container; 164
pf-client-core tests.
2026-08-07 14:34:59 +02:00
enricobuehler 6c4df043fe Merge pull request 'Gamepad UI (iOS): palette-washed glass, room to breathe, in-place screens, and a real option drum' (#91) from worktree-gamepad-ios-polish into main
ci / bun-nix (push) Successful in 31s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 16s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 16s
ci / web (push) Successful in 1m12s
ci / docs-site (push) Successful in 1m20s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 14s
apple / swift (push) Successful in 1m32s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
ci / rust-arm64 (push) Successful in 1m39s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 17s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / builders-arm64cross (push) Successful in 14s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m19s
apple / screenshots (push) Successful in 5m42s
docker / deploy-docs (push) Successful in 6m35s
ci / rust (push) Failing after 12m11s
release / apple (push) Successful in 9m15s
Reviewed-on: #91
2026-08-07 12:07:54 +00:00
enricobuehler cbfa03b7ad docs(host/pads): the SC2's bInterval is already 1 kHz — don't "fix" it to 250 Hz
Working G14/G18 turned up two sweep findings that do not survive contact with
the code. Neither is implemented; one is now guarded.

The 2026-08-07 sweep read the Triton (Steam Controller 2) usbip endpoint's
`bInterval: 1` as 125 µs — an 8 kHz duplicate storm — and the plan's G14 says to
raise it to 4 "like the Deck". That reading assumes a high-speed device, where
bInterval is the 2^(n-1) × 125 µs exponent. Both Triton devices declare
`UsbSpeed::Full`, and on a full-speed device the field is a plain frame count in
milliseconds: 1 means 1 ms, which is the 1 kHz the existing comment claims.
Raising it to 4 would mean 4 ms — a 4× cut to the motion rate a passed-through
SC2 delivers, in the name of fixing a problem it doesn't have. The endpoint now
carries the reasoning so the next reader doesn't repeat it.

G18's first bullet ("bound/rate-cap the host's rich-input channel; motion is
unbounded") is stale rather than wrong — it was true of the tree the sweep read.
Current main already routes rich input, motion included, through a 1024-deep
`sync_channel` whose `offer()` helper `try_send`s and drops on full, ending the
loop only on Disconnected. That is the same bounded-queue pattern the mic plane
adopted for security-review S6. Nothing owed.

G14's remaining bullet — DS/Deck neutral accel should read 1 g on the up axis
instead of 0 g free-fall — is deliberately NOT done here. Which axis is up is
precisely what G16's on-glass session measures: `switch_proto` documents the
wire as z-up and its neutral ships +Z, but the Deck's kernel negates Z/RZ, so
guessing would leave one backend confidently disagreeing with another. A wrong
constant is worse than the current obviously-unset 0.

Gate: fmt, build, clippy --all-targets -D warnings, and the test suites — green.
2026-08-07 14:00:30 +02:00
enricobuehler c010139e6e feat(client/apple): the gamepad UI moves and colours like the console it mirrors
ci / bun-nix (pull_request) Successful in 49s
ci / web (pull_request) Successful in 1m3s
ci / docs-site (pull_request) Successful in 1m16s
apple / swift (pull_request) Successful in 1m33s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 4m38s
ci / rust (pull_request) Successful in 16m9s
Four reworks from the first palette-era on-glass review, all iOS-facing:

- Surfaces carry the palette now, not just the text on them: ConsoleGlass
  washes every tier (Liquid Glass tint, pre-26 material, tvOS material)
  with ink.glass — the same colour the desktop console fills its panels
  with — and the close buttons move to an ink-aware consoleGlassBackground.
  The pre-26 branch also gains the focus tint it had silently dropped.
  Stray literals follow: ConnectOverlay text rides ink in the console
  takeover, card shadows soften on pale fields, the focused keycap reads
  onAccent. The online pip stays status-green on purpose.

- The header breathes: title top padding 4/10 -> 10/18 plus shared
  header-spacing and title-bottom helpers mapped from the console shell's
  rhythm, applied to the launcher, settings and add-host alike, with the
  add-host close X re-anchored to the title row.

- Settings, Add Host and the Library present IN PLACE on iOS: one
  persistent aurora whose calm is chased (the console's bg_mix), screens
  as transparent layers with the console's 0.26 s ease-out-cubic push/pop,
  an input drop for the transition, and the controller handed off through
  isActive — no more opaque bottom-up covers, no backdrop teardown.
  macOS keeps its sheets, tvOS its focus-engine covers.

- The settings select is a real band: choice rows mount GamepadOptionBand,
  a spring-driven drum (Animatable body, ring-distance wrap, neighbours
  gated by focus and flight) whose retargeting spring accumulates rapid
  steps into one continuous spin. Reduce Motion falls back to a plain
  crossfade; toggles keep the quiet 14 pt slip.

Verified: swift build (macOS), swift build --triple arm64-apple-ios17.0,
swift test 208 passed / 0 failed. On-glass QA still owed: palette sweep on
a pale palette, transition compositing over materials, drum feel on device.
2026-08-07 13:59:40 +02:00
enricobuehler 77797a9e20 feat(client/pads): stop streaming gyro into a session that cannot receive it
G8 of the gyro program, SDL-client half.

The `Welcome` has always carried the backend the host actually RESOLVED, which
is not necessarily the one the client asked for — Auto lands on Xbox 360 for
anything not Sony/Valve/Xbox, and a Switch Pro on a Windows host folds to X360
too. No client read the field. So a player with an 8BitDo, or a Switch Pro on
Windows, got a controller whose gyro did nothing, with nothing anywhere saying
why: the client shipped ~250 Hz of Motion datagrams and the host parsed and
discarded every one.

`GamepadPref::has_motion()` answers whether a backend has a motion plane at all.
The SDL client checks it on the first gyro sample: it logs one line naming the
resolved backend and pointing at the fix (pick a DualSense-class controller
type), then stops sending. Once per slot, not per sample — this path runs at the
pad's sensor rate.

`Auto` deliberately answers true. It means "unknown" — an old host that omitted
the echo, which may well have resolved a DualSense — and suppressing motion on
unknown would silently break working gyro, a worse failure than sending
datagrams nobody reads. The predicate is an exhaustive match so a new backend
has to state its answer rather than inherit one, and a table test pins both
halves: a false negative kills working motion, a false positive keeps the void
open, and both are silent.

Owed: the plan wants this surfaced as a one-line UI hint, not just a log line.
Apple already stores `resolvedGamepad` and Android needs the plumb; neither is
done here, and both want their own gate.

Gate (Linux CI image): fmt, build, clippy --all-targets -D warnings, and the
test suites — green, with the new capability test observed running.
2026-08-07 13:52:27 +02:00
enricobuehler c0f8f051c3 fix(client): the probe printed a device index the env var does not take
--probe-decode printed its DISPLAY position and called it the
PUNKTFUNK_VK_DEVICE value. It is not. pick_device resolves that variable against
the RAW vkEnumeratePhysicalDevices order (setup.rs, `devices.get(i)`) BEFORE any
ranking runs, while the probe sorts discrete-first for readability.

Those two orders disagree precisely on the hardware this flag exists to
diagnose. pick_device's own comment records why the ranking is there: "enumeration
order puts the iGPU FIRST on some hybrids (observed: Ryzen iGPU ahead of an RTX
dGPU)". So on a hybrid laptop the number the probe printed for the iGPU could
well be the number for the dGPU — a diagnostic handing out an actionable value
that selects the other GPU, which is worse than printing none.

Measured on the Arc + RTX 3500 Ada laptop, which is also where the first output
went out with the wrong claim in it: three adapters, and the same Arc iGPU
enumerated TWICE. So AdapterDecode now carries the raw enumeration index,
captured before the sort, and the printer uses it; the "default presenter"
marker stays on the first LISTED entry, because sorted-first is what pick_device
lands on when nothing overrides.

The duplicate is why the trailing hint names PUNKTFUNK_VK_ADAPTER as the safer
knob and admits its limit: two adapters sharing a marketing name cannot be told
apart by it, and a name match resolves to whichever enumerates first. The hint
also states the thing this whole output invites a reader to get wrong — that a
capable GPU in the list does not mean the decoder will use it, because Vulkan
Video decodes on the presenter's device and PUNKTFUNK_DECODER does not move the
presenter.

Gates: fmt clean; clippy -D warnings on punktfunk-client-session and pf-presenter.
2026-08-07 13:52:23 +02:00
enricobuehler 43e713ecca fix(client): the probe now accounts for every bit it prints
First hardware run of --probe-decode, on the RTX 5070 Ti:

    driver decode ops:   H.264, H.265, AV1 (0xF)

Three names, four bits. 0xF is H.264|H.265|AV1|VP9 — bit 3 is
VK_VIDEO_CODEC_OPERATION_DECODE_VP9_BIT_KHR, a real decode operation this
client has no rung for, so the name table stopped short of it and the line
looked complete while silently dropping a codec the driver had advertised.

That is the exact failure this flag exists to prevent. The whole point of
--probe-decode is that a reader can trust the words to cover the number; a mask
with an unexplained bit asks them to trust it instead. VP9 is now named (marked
as having no punktfunk rung, because advertising it as decodable would be its
own lie), and any bit beyond the four we know prints as "unrecognised bits
0x…" rather than vanishing — so the next codec Khronos adds shows up as an
unknown rather than as nothing at all.

Gates: fmt clean; clippy -D warnings on punktfunk-client-session.
2026-08-07 13:46:28 +02:00
enricobuehler ce5047f3ad fix(host/pads): the Windows driver stops halving motion and stops serving torn reports
G6 + G15 of the gyro program.

G6 — the UMDF gamepad driver's input path. Its timer ran at 8 ms and completed
one pended READ_REPORT per tick, so a game could observe at most ~125 Hz while
clients stream motion at ~250 Hz: every other sample was overwritten in the slot
before anything read it, and the ones that survived carried up to 8 ms of extra
latency. For gyro, a dropped sample is not a dropped frame — it is rotation that
never reaches the game.

The timer now ticks at 2 ms (about a real DualShock 4's Bluetooth cadence). Only
the cheap half runs on every tick: read the input slot, complete one pended
read. The channel handshake and the health marks stay on their historical ~8 ms,
because they cost more, nothing wants them faster, and `driver_heartbeat`'s
documented "+1 per ~8 ms tick" is what the host reads as liveness.

The same slot is a single unqueued buffer that both sides touch without a lock,
so a driver read landing mid-copy handed the game a report that was half the
previous frame and half the next. For a button that is a one-tick glitch; for
motion it is a spike in angular velocity, which an integrator turns into aim
movement. `PadShm` gains an `input_gen` seqlock (v2.3, carved from reserved
space inside the v2 legacy region): the host takes it odd, fences, writes the 64
bytes, and stores it even; the driver samples it either side of its read and
retries once. The old code's own comment called this out as a known residual —
it is now closed rather than documented.

Version posture matches the ring's, with one simplification: no capability stamp
is needed, because an old host never writes the field and a constant 0 is
indistinguishable from "no write in flight", so a new driver against an old host
behaves exactly as it does today, and an old driver ignores the field entirely.

The Steam Deck write path had neither the seqlock nor even the trailing Release
its DualSense sibling carried; all three Windows backends now publish through
one `publish_input`.

G15 — motion-cadence observability. The host already computed the measurement a
"gyro feels floaty" report needs (client inter-arrival percentiles), but kept
ONE global accumulator, so two motion-capable pads in a session interleaved into
each other's gaps and produced a number describing neither. It also sat at
`debug` behind a `tracing::enabled!` check, so a field log arrived with nothing
in it and the only way to get the measurement was to ask for a re-run.

Now per-pad and always on, summarized at `info` when the session ends — the
moment a field report is being written. It costs one subtraction and one array
increment per sample: percentiles come from a fixed log2 histogram instead of a
growing sorted Vec, so there is no allocation, no per-window sort, and no way
for a client streaming as fast as the link allows to make the instrument
expensive. Percentiles are reported as bucket upper bounds (`_le`), which is a
factor-of-two answer to a question whose answers are orders of magnitude apart.
Gaps of 500 ms or more are counted as stalls rather than folded into the
percentiles — an interruption is not a cadence, and averaging it in would report
a healthy feed as a terrible one.

Gates. Windows CI runner .133, the drivers workspace on the real WDK: cargo
build, clippy -D warnings (which enforces the unsafe-audit lints), and fmt —
all green, against a source whose SHA-256 matches this commit's. Linux CI image:
fmt, build, clippy --all-targets -D warnings over pf-inject / punktfunk-core /
punktfunk-probe / pf-client-core / pf-driver-proto / punktfunk-host, and the
test suites including the 5 new motion-cadence tests — all green.

Not measured on glass. G6's stated gate is a sensor-rate reading (SDL
testcontroller or Steam's calibration screen) that matches the client's send
rate; that is still owed, and a driver change only a compile has seen deserves
it before anyone trusts the number.
2026-08-07 13:44:54 +02:00
enricobuehler 31bb3ca7f7 Merge pull request 'The console answers a mouse and a finger, and host cards get a menu' (#90) from worktree-console-tabs-pointer into main
ci / bun-nix (push) Successful in 24s
ci / docs-site (push) Successful in 1m17s
apple / swift (push) Successful in 1m32s
ci / web (push) Successful in 1m36s
deb / build-publish-client-arm64 (push) Successful in 2m42s
ci / rust-arm64 (push) Successful in 3m18s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m34s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 1m38s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 14s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 17s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 17s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m15s
docker / builders-arm64cross (push) Successful in 11s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m34s
deb / build-publish (push) Successful in 4m51s
docker / deploy-docs (push) Successful in 34s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m51s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m2s
apple / screenshots (push) Successful in 5m53s
deb / build-publish-host (push) Successful in 7m16s
arch / build-publish (push) Successful in 7m59s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m20s
android / android (push) Failing after 11m58s
flatpak / build-publish (push) Failing after 12m26s
ci / rust (push) Successful in 17m57s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m39s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 21m14s
Reviewed-on: #90
2026-08-07 11:38:41 +00:00
enricobuehler d49f955171 fix(console/host menu): the actions get their explainer line and room to breathe
ci / bun-nix (pull_request) Successful in 31s
ci / web (pull_request) Successful in 1m11s
ci / docs-site (pull_request) Successful in 1m39s
ci / rust-arm64 (pull_request) Successful in 1m41s
apple / swift (pull_request) Successful in 1m36s
apple / screenshots (pull_request) Skipped
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 3m2s
android / android (pull_request) Successful in 3m37s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m11s
ci / rust (pull_request) Successful in 5m6s
The rows sat hard against the pinned title — the menu is the one form screen
with no subtitle, so the list started at the very top of the content band. It
now wears Add Host's explainer, which both says what the menu is for and is the
air that keeps the first row off the title. A pinned card explains what unpinning
does and does not touch, the same wording the Android dialog uses.

Caught by the screenshot pass, not by a test.
2026-08-07 13:29:59 +02:00
enricobuehler 507ea58da8 fix(host/audio): the silent-sink check recognizes the minted Speakers
Observed in the first real session on the substrate: the loopback ran on
the minted 'Punktfunk Speakers' (silent on the host by construction), but
have_silent name-matches only the Streaming Microphone — so the capture
open logged 'desktop audio will also play on the host' (false) and
re-attempted the Steam-pair install it doesn't need. The minted sink is
recognized by id; its name honestly says Speakers, which the name rule
must keep refusing for FOREIGN instances.
2026-08-07 12:59:15 +02:00
enricobuehler 2830cbe912 Merge pull request 'Say which GPU can do Vulkan Video, and why not when it can't' (#89) from worktree-vulkan-decode-probe into main
ci / rust-arm64 (push) Successful in 3m35s
ci / bun-nix (push) Successful in 29s
ci / web (push) Successful in 1m6s
ci / docs-site (push) Successful in 1m18s
apple / swift (push) Successful in 1m29s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m20s
deb / build-publish-host (push) Failing after 2m7s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 34s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 32s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 26s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 15s
deb / build-publish-client-arm64 (push) Successful in 3m7s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 14s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 24s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m33s
android / android (push) Successful in 6m25s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m2s
docker / builders-arm64cross (push) Successful in 10s
docker / deploy-docs (push) Successful in 31s
apple / screenshots (push) Successful in 6m4s
arch / build-publish (push) Successful in 7m45s
ci / rust (push) Successful in 8m13s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m7s
deb / build-publish (push) Successful in 8m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m30s
flatpak / build-publish (push) Successful in 17m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m4s
Reviewed-on: #89
2026-08-07 10:50:26 +00:00
enricobuehler 4834c2ee51 fix(host/pads): DualShock 4 gyro ran 40× fast, and no pad ever stopped turning
Phase 1 of the gyro program (design/gyro-program.md, G1-G5) — the five
correctness fixes under it. Gyro aim integrates angular velocity over time, so
each of these is not a cosmetic wrongness: a wrong scale is every rotation being
the wrong size, a wrong clock is every rotation being integrated against a
fictional dt, and a stale sample is rotation that never happened.

G1 — the DualShock 4 calibration blob. A Sony pad does not assume a motion
scale, it reads one out of a fixed calibration feature report. Ours declared
0.5 LSB per °/s and 8192 LSB/g while the wire delivers 20 and 10000, so every
DS4-type session decoded gyro 40× too fast and acceleration 1.22× hot — since
the backend shipped. The blob now states the wire's own units (the DualSense
blob's numbers, deliberately: both pads consume the identical wire sample). Its
interleaved per-axis order is NOT a bug and stays: the virtual pad declares
BUS_USB, where interleaved is the correct layout; grouped is Bluetooth's.

The same blob lives a second time in the UMDF driver, which is a separate WDK
workspace that cannot depend on pf-inject — one wrong table in two files, where
fixing one reads as fixing it. Both are fixed, and the DS4 feature reports now
live in dualshock4_proto beside the DualSense's rather than in the Linux
backend, so there is one canonical copy to point at.

Field hosts keep the old blob until they update the host package.

G2 — the gate that would have caught it. Nothing pinned any backend's
declaration against the wire, so tests/motion_contract.rs now applies the
CONSUMER's arithmetic (the kernel's, and SDL's, which differ) to each backend
and asserts the result lands back on the wire constants — for the DualSense and
DS4 blobs, and for the Deck and Switch Pro rescales. It also parses the driver's
Rust source and re-derives the units from THAT, so the two copies cannot drift.
Verified non-vacuous both ways: re-introducing the old blob fails with "declares
a fractional 32/64 LSB per °/s", and reverting only the driver's copy fails with
"the UMDF driver's DS4_FEATURE_CALIBRATION has drifted from pf-inject's".

The wire units themselves move to punktfunk_core::input::gamepad, referenced by
the client's capture scale, the Deck/Switch rescales, and the probe — whose
at-rest vector said 16384 (a driver's number, not the wire's) and now says 1 g.

G3 — real sensor clocks. The DualSense advanced its sensor timestamp by +1 raw
unit per report (0.33 µs — a frozen clock) and the DS4 by a flat +188 (~1 ms)
regardless of the real 4-8 ms cadence. Anything integrating rate × dt off that
field got nonsense. All four backends now stamp elapsed monotonic time in their
own units via a shared SensorClock, anchored to the pad's first report so an
irregular publish loop cannot make it drift, and truncated to the field width —
which reproduces the wrap real hardware does.

G4 — motion is level-triggered and had no watchdog. merge_frame preserves the
last sample and the heartbeat re-emits it, so a feed that stops leaves the pad
rotating forever — and with G3's honest clock, at a dt that keeps growing.
Rumble and the pen plane each have an idle timeout; motion now has one too, at
100 ms. Angular velocity only: acceleration is kept, because gravity is
legitimately persistent and blanking it reads as free-fall. The SDL client
parks its gyro at zero when a slot closes, which is the case we can flush
rather than wait out. (The Apple half of this rides in PR #88.)

G5 — a pad returning inside the 300 ms replug grace keeps the same device and
skips the create path, so a different controller inherits the previous one's
touch contact and rotation — and a pad with no gyro never sends a sample to
correct it. sweep() now reports re-claims separately from drops, and the manager
clears the rich plane on one. Rich fields only: rumble and hidout dedup
deliberately survive a removal.

Gates (Linux, CI image): fmt, build, clippy --all-targets -D warnings over
pf-inject/punktfunk-core/punktfunk-probe/pf-client-core, and the test suites —
110 pf-inject unit + 6 contract + 29 pf-client-core gamepad, all green.
Not yet verified on glass; the on-glass sign/scale session is G16.
2026-08-07 12:47:26 +02:00
enricobuehler 413a3e3e76 feat(client): say which GPU can do Vulkan Video, and why not when it can't
ci / bun-nix (pull_request) Successful in 29s
ci / rust-arm64 (pull_request) Successful in 1m44s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m5s
apple / swift (pull_request) Successful in 1m35s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 3m21s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m1s
ci / rust (pull_request) Successful in 4m27s
ci / docs-site (pull_request) Failing after 16m24s
ci / web (pull_request) Failing after 16m25s
Field report from an Intel Arc + NVIDIA laptop: pinning the Vulkan rung on the Arc
iGPU silently produced D3D11VA, and there was no way to tell whether the build had
tried at all. That ambiguity was ours, in three places.

The "unavailable" log printed three of the FIVE conjuncts that gate Vulkan Video.
A device with 1.3, the features and a decode queue family — but no codec extension
— logged dev_is_13=true features_ok=true decode_family=true next to the word
"unavailable" and named nothing actionable. It now prints all five, plus which
base extensions are missing, which codec extensions are present, the decode
family's own advertised codec operations, and the device name and vendor. It also
no longer says "VAAPI/software" on Windows, where the rung below is D3D11VA.

The native-vulkan PIN refusal logged `video_decode` alone. On a device that
decodes something but not THIS codec, that reads as a contradiction: refused, yet
video_decode=true. It now carries the caps mask and the codec bit that was wanted,
so "your GPU can't" is distinguishable from "we asked for the wrong thing" — only
the second is our bug.

And `--probe-decode` is new: per-adapter Vulkan Video capability with no session,
no surface and no logical device. For each GPU it answers usable yes/no, the
driver's own decode ops, the extensions, and — when the answer is no — which
conjunct failed, in words. Separate from --list-adapters, which the desktop shells
parse line-by-line for their GPU picker and which therefore keeps printing bare
names.

The listing is ordered like pick_device (discrete first) and marks entry 0 as the
default presenter, because that ordering is very likely the reporter's actual
answer: pick_device ranks DISCRETE_GPU above INTEGRATED_GPU, Vulkan Video decodes
on the PRESENTER's device by design (that is what makes it zero-copy), and
PUNKTFUNK_DECODER does not move the presenter. So on a hybrid laptop, pinning the
decoder while the dGPU presents probes the wrong GPU entirely —
PUNKTFUNK_VK_DEVICE=<index> is the knob that moves it, and the index printed is
that value.

To keep the probe honest, VIDEO_BASE and VIDEO_CODECS moved to module scope and
the five-way AND became video_decode_gate(), called by both the probe and device
creation. A probe holding its own copy of the rule is one that eventually reports
a capability the session then refuses — which reads to everyone as a decoder bug
rather than a probe bug.

Gates: fmt clean; clippy -D warnings over punktfunk-client-session and
pf-presenter. The Linux container was unavailable (the host's disk filled and took
the docker daemon with it), so this ran on the macOS host target only — the
container leg is owed, and CI covers it on the PR.
2026-08-07 12:42:33 +02:00
enricobuehler f06b3d9d04 feat(console): the console answers a mouse and a finger, and host cards get a menu
Two gaps, both found on the shared Linux/Windows console UI.

**The settings tabs only moved for a gamepad.** They were bound to the shoulder
buttons and to PgUp/PgDn, and the legend spells PgUp/PgDn out only when NO pad is
attached — so with a controller plugged in a keyboard user had nothing to find,
and a mouse or a touchscreen could not change section at all.

The root cause was wider than the strip: `SkiaOverlay::handle_event` matched only
`KeyDown` and `TextInput`, so every mouse button, wheel and touch contact fell
past the console into the run loop, which routes pointer input exclusively at
`stream.capture` — `None` while you are browsing. Nothing in the console had ever
been clickable. Making just the pills answer would not have helped either: the
settings screen is opened with X from home, so a mouse could not reach it.

So the console gets a real pointer path:

- `Overlay::handle_pointer` carries mouse/touch in SWAPCHAIN PIXELS. The run loop
  converts (it owns the window, hence the display scale, and mouse coordinates are
  logical while fingers are normalised); the console then hit-tests the very rects
  it drew last frame. Only DIRECT touch devices are offered — an indirect trackpad
  already drives the mouse.
- Widgets act on the PRESS, not the release. The list and both carousels scroll the
  focused item toward the centre, so what you pressed has slid out from under your
  finger by the time it lifts; press-to-act has no such race and there is no drag
  gesture to compete with.
- The hint bar became the pointer's button bar. It is already the console's only
  on-screen statement of what the face buttons do, and a pointer has none — so its
  Confirm/Back/Secondary/Tertiary pills are clickable on every screen, which is what
  puts Settings and Library within reach of a mouse at all.
- Tab / Shift+Tab change section; PgUp/PgDn still do, and the keyboard legend now
  reads "Tab".
- Right-click is Back everywhere, EXCEPT at the root: B there quits the launcher and
  a right-click is far easier to fire by accident. Quitting stays explicit.

**Host cards had no menu.** Every other client hangs Wake / Copy link / Edit /
Forget off a host card; the console could add a host and connect to one, and that
was all — so a renamed machine or a fat-fingered address stayed wrong forever
unless you opened a desktop shell. UP on a saved tile now opens that host's menu,
the same gesture the Android console uses, on the one direction a horizontal
carousel leaves free.

- `ConsoleCmd::UpdateHost` edits the stored host IN PLACE. Removing and re-adding
  would silently drop the fingerprint, the learned MAC, the pinned cards and the
  profile binding — that is a rename, not a re-pair.
- `ConsoleCmd::ForgetHost` drops it; if it is still advertising it returns as a
  discovered, unpaired row, which is the honest state.
- Forget arms on the first press and fires on the second. The other clients forget
  outright; a console is driven by a thumbstick from across a room.
- A pinned profile card offers only Unpin. It is a shortcut, not a second host, and
  offering to forget the host from it would blur exactly the distinction a pin draws.
- "Edit…" REPLACES the menu on the stack rather than stacking over it, so Back from
  the editor doesn't land on a menu describing the host as it was before the edit.

Verified in the pf-lxcheck2 container (this crate compiles to nothing on macOS —
a bare `cargo check` there is vacuous): plain build and `clippy --all-targets`
clean under `-D warnings`, 72 tests pass. Seven are new, and cover the reported
bug directly — a press on a pill selects that tab, and each tab still keeps its
own cursor when a pointer is what switched it.
2026-08-07 12:36:22 +02:00
enricobuehler 99a59fb5c7 fix(host/audio): the mic pump's first resolve waits for the minted endpoints
Measured on the target box: the pump wired 2 s before the provisioning
worker latched, took the cable as its write target, and the next wiring
pass would then have paired the default recording with the minted
microphone — which nothing writes into: dead mic-air until a pump reopen.
resolve_target now provisions synchronously (instant once latched; the
opt-out env is honoured), so the pump's held device and the plan's verdict
can never disagree.
2026-08-07 12:02:48 +02:00
enricobuehler 15dddf7ec0 feat(web): the Dashboard shows the audio wiring verdict
An Audio wiring card (Windows hosts) below the status tiles: a readiness
badge (Ready / No microphone / No game audio / Not wired), the friendly
names carrying each role, and the degradation notes that were previously
visible only in the host log — mic withheld for game audio, the known-
degraded last resort, a narrowing endpoint. api/openapi.json regenerated
from the host build (AudioWiring + RuntimeStatus.audio); en+de messages.
2026-08-07 12:00:13 +02:00
enricobuehler 3870cdd1da feat(host/mgmt): the audio wiring verdict joins /status
RuntimeStatus gains an 'audio' object (Windows hosts): readiness
(full/audio_only/mic_only/none), the friendly names carrying each role,
and the three degradation flags (mic_withheld, last_resort, narrowing) —
the verdicts that previously lived only in tracing logs. Snapshot of the
last wiring pass (the mic pump wires at host start and on every reopen);
a status poll never triggers COM work or IPolicyConfig writes.
2026-08-07 11:53:15 +02:00
enricobuehler f7ef41b45b Merge pull request 'feat(clients): the phone's gyro can speak for a gyro-less pad' (#88) from worktree-gyro-phone-mirror into main
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 1m9s
ci / bun-nix (push) Successful in 53s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 14s
ci / docs-site (push) Successful in 1m35s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 11s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 1m2s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m23s
ci / rust-arm64 (push) Successful in 6m21s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 25s
android / android (push) Successful in 8m5s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 5m18s
release / apple (push) Successful in 10m33s
docker / builders-arm64cross (push) Successful in 1m46s
docker / deploy-docs (push) Failing after 3m52s
ci / rust (push) Successful in 11m25s
apple / screenshots (push) Successful in 5m48s
Reviewed-on: #88
2026-08-07 09:47:04 +00:00
enricobuehler 55aba3e936 feat(client/android): the phone's gyro can speak for a gyro-less pad
apple / swift (pull_request) Successful in 1m30s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m37s
ci / bun-nix (pull_request) Successful in 30s
android / android (pull_request) Successful in 4m5s
ci / rust-arm64 (pull_request) Successful in 4m1s
ci / docs-site (pull_request) Successful in 1m26s
ci / rust (pull_request) Successful in 10m54s
Opt-in "Gyro from this phone" (gyro_on_phone, off by default): this
device's IMU sources wire pad 0's motion while that pad is a controller
with no motion source of its own. On Android that gate is exact — the only
pads that forward motion are the capture links (USB DualSense / SC2,
claimed as ExternalPads), so the mirror stands down per sample whenever
GamepadRouter.padHasOwnMotion(0) says a capture link holds the index, and
sends nothing while pad 0 has no slot at all (motion never creates a host
pad). "Rumble on this phone"'s sibling, data flowing the other way: same
read-once-at-attach settings plumbing, same hardware-gated rows in the
touch and controller settings (a TV box has no gyroscope to mirror from).

DeviceGyro registers TYPE_GYROSCOPE + TYPE_ACCELEROMETER at ~200 Hz on a
dedicated HandlerThread with batching disabled (maxReportLatencyUs = 0 —
batching is poison for gyro aim), converts with the wire contract shared
with pf-client-core (rad/s → 20 LSB/°·s, m/s² → g → 10000 LSB/g; Android's
accelerometer already reads specific force, the DualSense report's own
convention), and rotates each sample from the natural-portrait sensor
frame into the controller frame by display rotation — a phone clipped
landscape yaws when the player yaws instead of rolling. The remap matrix
and unit constants are pinned by DeviceGyroTest.

A stand-down edge (capture link claims pad 0, or session teardown) sends
one zero-gyro sample so the host's virtual pad never keeps integrating an
angular velocity this device stopped producing — the gyro sweep's
stale-rotation latch, avoided by construction here.
2026-08-07 11:38:31 +02:00
enricobuehler 1f54b75c5c feat(client/apple): the phone's gyro can speak for a gyro-less pad
Opt-in "Gyro from this device" (DefaultsKey.gyroFromDevice, off by default,
iOS only): while player 1's forwarded controller reports no rotation rate of
its own — no GCMotion, or the gravity-only motion an Xbox pad exposes — this
device's IMU sources pad 0's wire motion instead. The rumble-on-device
mirror's sibling, data flowing the other way: same session-scoped
UserDefaults read, same hardware-gated settings rows, same pad-0 rule.

DeviceGyro wraps CMDeviceMotion at the ~100 Hz CoreMotion ceiling on a
dedicated serial queue (not main — the controller path's main-queue delivery
is a known jitter source), converts with the shared GamepadWire constants,
and rotates each sample from the device's portrait frame into the controller
frame by interface orientation, so a phone clipped landscape yaws when the
player yaws instead of rolling. The remap matrix is derived and pinned by
DeviceGyroRemapTests.

GamepadCapture owns engage/stand-down (reconcile, suspend/resume, stop), and
suppresses pad 0's controller-motion forwarding while the mirror runs — two
writers on one pad's motion state would fight, and the accel-only stream
would stomp the mirror's gyro with zeros.

Also fixes the stale-motion latch from the gyro sweep on the controller
path: flush now parks motion at zero (keeping the last accel, so gravity
doesn't become free-fall), and the mirror's stop sends the same closing
zero. The host holds motion as state and re-emits it — a nonzero angular
velocity left behind read as endless rotation for as long as an overlay
(Control Center pull-down) kept the app inactive.
2026-08-07 11:38:13 +02:00
enricobuehler 79c72fa64d fix(host/audio): capture endpoints carry the {0.0.1.…} id prefix
The capture-direction lookup built its endpoint id with the RENDER prefix
{0.0.0.00000000}., but WASAPI's enumeration returns capture ids as
{0.0.1.00000000}.{guid} — so the minted microphone's capture side never
string-matched the enumeration and the wiring plan paired no recording
device (audio-probe plan on the target box: mic_capture = '-'). Measured;
IMMDeviceEnumerator::GetDevice tolerated the wrong prefix, which is why
the S3 spike's direct open still passed.
2026-08-07 11:37:39 +02:00
enricobuehler 64655c5275 fix(host/devtest): audio-probe plan provisions synchronously
A fresh CLI process has no startup worker to have finished, so the plan
devtest raced its own background provisioning thread and printed the name
ladder instead of tier-0. ensure_blocking() re-resolves existing marker
devnodes in milliseconds before the wiring pass runs.
2026-08-07 11:32:05 +02:00
enricobuehler c6b183450a Merge pull request 'docs: the 4:4:4 story catches up with the code that ships it' (#87) from worktree-docs-support-matrix-444 into main
ci / rust-arm64 (push) Successful in 2m14s
ci / web (push) Successful in 2m17s
ci / bun-nix (push) Successful in 24s
ci / docs-site (push) Successful in 1m59s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Failing after 5s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 2m54s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 4m31s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 3m25s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m50s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m14s
ci / rust (push) Successful in 13m30s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m29s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 4m41s
docker / builders-arm64cross (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
Reviewed-on: #87
2026-08-07 09:27:32 +00:00
enricobuehler 4a621de6b1 chore(packaging): retire VB-Cable — audio's substrate is Steam's drivers
The other half of the audio-substrate decision (spikes S2+S3 green, minted
endpoints landed in the previous commit): stop bundling a third-party
kernel driver the host no longer needs.

  installer      the VB-CABLE task, payload, silent-install run and the
                 donationware notice are gone; a suppressible notice tells
                 a Steam-less box that audio needs Steam INSTALLED (never
                 running) and that installing it later just works. A cable
                 from an older install is still deliberately not removed.
  packer + CI    -VbCableDir/VBCABLE_DIR, the staged-payload check and the
                 runner provisioning download are gone; SBOM drops the
                 redistributed-driver component.
  winget         the VB-Audio bundling-grant agreement becomes the honest
                 Steam requirement (surfaced on the unattended path where
                 no wizard is on screen).
  docs           windows-host/uninstall/security/echo say what actually
                 ships: no kernel-mode driver of our own, endpoints minted
                 from Valve's vendor-signed drivers, VB-CABLE mentioned
                 only as the historical fallback that keeps working.
  host wording   the mic-open guidance and module headers lead with Steam;
                 the NAME ladder itself is untouched — demoting 'cable
                 input' was considered and rejected (on a box where minting
                 transiently fails, the SSM would outrank an installed
                 cable, steal the silent sink, and make audio host-audible).
2026-08-07 11:27:29 +02:00
enricobuehler 3a35773b70 Merge origin/main; 4:4:4 has no software floor, and the ABI is 17
ci / rust-arm64 (pull_request) Successful in 1m37s
ci / web (pull_request) Successful in 1m8s
ci / bun-nix (pull_request) Successful in 16s
ci / docs-site (pull_request) Successful in 2m45s
ci / rust (pull_request) Successful in 8m28s
Reconciling with #85 (FFmpeg is gone from the client). Two of my claims
were true against the pre-merge tree and false against this one.

Note 4 said the desktop clients need no 4:4:4 decode probe "because every
rung can display full chroma — swscale converts for the software rung".
There is no swscale any more. The CPU floor is openh264 + rav1d, it is
4:2:0 8-bit by contract and has no HEVC at all, so it refuses a 4:4:4
stream rather than converting one. The client still advertises the bit
unprobed, which was the point of the original fix, but the honest reason
is different: full chroma is a hardware path (Vulkan RExt, NVIDIA today),
and what catches a box whose hardware 4:4:4 fails is note 2's codec
reconnect, not a downgraded picture. Note 13 repeated the same wrong
premise and now names both halves of its warning.

The C ABI is 17, not the 14 I read before the merge.

Textual side of the conflict: #85 rewrote the Codecs column and notes 1-3
of the same table while leaving note 4's stale 4:4:4 text alone. Theirs
kept in full; only the 4:4:4 column and note 4 are mine.
2026-08-07 11:25:17 +02:00
enricobuehler 5aa1ca392e Merge pull request 'fix(host/encode): the Windows host build stops failing on unused split-encode helpers' (#86) from worktree-fix-winhost-clippy-dead-code into main
ci / web (push) Successful in 1m22s
ci / rust (push) Canceled after 1m39s
ci / rust-arm64 (push) Canceled after 1m38s
ci / docs-site (push) Canceled after 9s
ci / bun-nix (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Successful in 1m37s
android / android (push) Successful in 4m58s
deb / build-publish-host (push) Successful in 5m28s
apple / swift (push) Successful in 1m30s
arch / build-publish (push) Successful in 11m59s
apple / screenshots (push) Successful in 5m59s
deb / build-publish (push) Successful in 8m56s
windows-host / package (push) Successful in 15m28s
windows-host / winget-source (push) Skipped
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 24m1s
windows-host / canary-manifest (push) Successful in 23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m43s
Reviewed-on: #86
2026-08-07 09:22:57 +00:00
enricobuehler 0b160a4e22 feat(host/audio): minted Punktfunk endpoints become the wiring plan's tier-0
The audio-substrate program's Phase 2 (spikes S2+S3 measured green on the
target box): the host mints its OWN instances of Valve's streaming-audio
drivers and wires by IDENTITY instead of borrowing Steam's primaries —

  minted.rs      the provider: one devnode per role ('Punktfunk Speakers'
                 from SteamStreamingSpeakers.inf, 'Punktfunk Microphone'
                 from SteamStreamingMicrophone.inf), marker-matched across
                 restarts (PunktfunkAudioRole in Device Parameters — names
                 are NOT identity, a minted instance is name-identical to
                 the primaries), provisioned on a startup worker like pad
                 audio, retried with a 60 s cool-down from wiring passes,
                 defaults restored when a fresh endpoint grabs them.
  wiring_plan    MintedIds tier-0: the mic takes its minted device outright
                 (capture side paired by the provider's id — a name search
                 cannot tell it from the primary), the loopback prefers the
                 minted sink at the head of the silent tier, an operator
                 override still beats everything, a narrowing minted sink
                 demotes below real hardware, and stale ids fall back to
                 the ladder unchanged. Plus AudioReadiness — the
                 full/audio-only/mic-only/nothing classification, logged
                 with every plan change (§C4's seed).
  audio-probe    'mint' runs the provider synchronously; 'plan' prints one
                 real wiring pass + readiness — the field-triage command.

Without Steam's drivers nothing changes: provisioning degrades to absent
ids and the plan keeps the name-based ladder (primaries → cable → real
hardware) exactly as before.
2026-08-07 11:22:06 +02:00
enricobuehler b245cb6b29 Merge pull request 'FFmpeg is gone from the client — native decode M0–M10' (#85) from worktree-native-decode-m0 into main
android / android (push) Canceled after 0s
apple / swift (push) Canceled after 0s
apple / screenshots (push) Canceled after 0s
arch / build-publish (push) Canceled after 0s
ci / rust (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
ci / bun-nix (push) Canceled after 0s
deb / build-publish (push) Canceled after 0s
deb / build-publish-host (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 0s
windows-host / package (push) Canceled after 33s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
audit / cargo-audit (push) Successful in 29s
audit / bun-audit (sdk) (push) Successful in 20s
audit / bun-audit (plugin-kit) (push) Successful in 23s
audit / bun-audit (web) (push) Failing after 25s
audit / docs-site-audit (push) Successful in 21s
audit / pnpm-audit (push) Successful in 24s
windows-msix / package (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m53s
audit / license-gate (push) Successful in 4m48s
windows-msix / package (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m5s
decky / build-publish (push) Successful in 23s
release / apple (push) Successful in 9m25s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m4s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m5s
flatpak / build-publish (push) Failing after 14m52s
nix / flake (push) Successful in 16m42s
Reviewed-on: #85
2026-08-07 09:21:45 +00:00
enricobuehler f49f22a292 fix(host/encode): the Windows host build stops failing on unused split-encode helpers
ci / bun-nix (pull_request) Successful in 23s
ci / web (pull_request) Successful in 1m26s
ci / docs-site (pull_request) Successful in 1m28s
apple / swift (pull_request) Successful in 1m37s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 2m31s
android / android (pull_request) Successful in 3m29s
ci / rust (pull_request) Successful in 9m51s
The Windows host job died in its clippy step: eight items in pf-encode's
split-encode policy (`SPLIT_AUTO`..`SPLIT_DISABLE`, `resolve_split_mode`,
`max_forced_split_mode`, `clamp_to_engines`) were reported as never used, and
`-D warnings` turns that into a build failure. Nothing about the encoder was
wrong — the items simply have no reader in one particular build of the crate,
and nothing was telling the compiler that.

`codec.rs` compiles on every platform, but the split policy only ever has a
caller on Linux (the libav NVENC path reads it unconditionally) or on Windows
with the `nvenc` feature (the direct-SDK backend). A featureless Windows build
of pf-encode has neither, so every item in the cluster is genuinely dead there.
Gate them on the union of their callers' cfgs, the way `forced_split_width`
next door already is.

The step lints pf-encode itself WITH `--features nvenc,amf-qsv,qsv`, where the
items are live, which is why this was invisible there; the failure came from the
next command in the same step, `clippy -p pf-vdisplay`, which pulls pf-encode in
as a plain default-features dependency. Same item-level `dead_code` trap this
crate has now hit five times.

Verified: default-features pf-encode reproduces all eight errors before the
change and none after (macOS default-features exercises the identical
"cluster has no caller" arm as featureless Windows — the two remaining errors
there, `vbv_frames_env` and a redundant closure call, are pre-existing and
macOS-only; both items have real Windows callers). Linux default-features and
Linux + nvenc `--all-targets` both stay clean, so the callers still see the
policy. `cargo fmt` clean.
2026-08-07 11:11:15 +02:00
enricobuehler f1e7ec3535 docs: the 4:4:4 story catches up with the code that ships it
ci / bun-nix (pull_request) Successful in 19s
ci / docs-site (pull_request) Successful in 2m12s
ci / web (pull_request) Successful in 2m19s
ci / rust (pull_request) Canceled after 3m49s
ci / rust-arm64 (pull_request) Canceled after 3m29s
The support matrix said the desktop clients' Full chroma switch "has no
effect today" and that only the Apple client asks for 4:4:4. Both stopped
being true in July: `clients/session/src/main.rs` advertises VIDEO_CAP_444
whenever the setting is on, deliberately with no client-side probe, because
every desktop decode rung can display full chroma — the Vulkan presenter
samples the 2-plane 4:4:4 pool formats and swscale converts for the software
rung. So Linux, Windows and Apple all ask; Android is the one that genuinely
doesn't implement it.

The other half was HDR. `9f72a3b6` gave the Windows IDD-push capturer a
packed 10-bit BT.2020 PQ RGB output, so NVENC encodes HEVC Main 4:4:4 10 and
the two compose — the matrix still said "4:4:4 and HDR together is refused",
and hdr.md still called PyroWave the only exception. Linux is the side that
keeps the trade: handshake.rs resolves the depth back to 8 for a 4:4:4
session, so full chroma wins and the stream is SDR.

Three cells move ⚠️ rather than  on purpose. The client half is
unconditional, but the host half is not: HEVC 4:4:4 means an NVIDIA host, or
PyroWave on any vendor. The notes say which, and point at the stats overlay's
`4:4:4→4:2:0` tag — this negotiation is the one that fails loudly.

Also: C ABI version 13 → 14; PyroWave's ≈8K 4:4:4 block-index ceiling now
has a note; and the roadmap no longer calls Intel 4:4:4 a hardware limit,
which the matrix and vaapi.rs both contradict — VCN can't, VAAPI hasn't.

Spot-checked and left alone as still accurate: the Linux client clipboard
stub, VAAPI declining 4:4:4, Android having no 4:4:4 at all, and the wire /
driver / gamepad-channel versions.
2026-08-07 10:58:58 +02:00
enricobuehler 03ee3e55d7 feat(host/devtest): audio-probe — the audio-substrate spike measurements
The S1-S3 spikes from windows-audio-endpoints-and-vbcable.md as one
runnable devtest (no game, no client, ssh-drivable):

  audio-probe ssm          S3, the decision gate: mint a SECOND devnode of
                           Valve's Steam Streaming Microphone driver and
                           prove the pair end to end (tone into its render
                           endpoint must come back out of its capture
                           endpoint). Pass = a punktfunk-owned virtual mic
                           needs no VB-Cable wherever Steam is installed.
  audio-probe sink         S2: mint a Speakers instance, park the DEFAULT
                           playback on it, tone through the default device,
                           WASAPI-loopback the instance - the desktop-audio
                           capture path minus the game.
  audio-probe sss-primary  S1: the primary Speakers' known-silent loopback,
                           re-measured, with mix format + steam.exe state.
  audio-probe cleanup      remove every probe-minted devnode (marker value
                           in Device Parameters, never name-guessing).

pad_endpoint grows the first slice of the design's §C1 shared minting
surface: create_media_devnode(desc, hwid, mark), bind_driver(hwid, inf),
find_capture_endpoint_for_devnode — the pad provisioner now calls the
same functions. The probe restores whatever default devices the minting
disturbed before it exits.
2026-08-07 10:47:00 +02:00
enricobuehler 152047051c fix(host/windows): three SID unsafe blocks get their safety proofs
install.rs (landed 2026-08-05 with the security-review remediation, while
the Windows CI runner was down) fails windows-host.yml's clippy gate:
#![deny(clippy::undocumented_unsafe_blocks)] wants the SAFETY comment on
the line preceding EACH unsafe block, and three blocks didn't have one —
two sat behind a comment anchored to the enclosing closure/neighbouring
statement, and EqualSid had none at all. Comments only; no behavior
change.
2026-08-07 10:38:03 +02:00
enricobuehler ef8214415f fix(host/audio): game audio outranks the mic on cable-less boxes
The wiring plan reserved the mic target unconditionally first, so on a box
without VB-Cable the mic took the Steam Streaming Microphone — the only
working client-only loopback sink — and desktop audio fell to the
known-silent Speakers last resort: a headless Steam-only host streamed
SILENCE (the 2026-08 field case), and the installer's 'optional (mic
passthrough)' wording never warned anyone.

The mic may now hold the Streaming Microphone only while the loopback still
gets a preferred (non-last-resort) pick without it — another silent sink or
real hardware. Otherwise the loopback takes the endpoint and the mic falls
to a lesser candidate or is honestly withheld (Wiring::mic_withheld), with
the open error naming the trade and the remedy. An operator
PUNKTFUNK_MIC_DEVICE override is exempt: an explicit choice may still
strand the loopback on the last resort.

Also: the Steam-pair auto-install latch is now once per INF-state instead
of once per process — an attempt made while Steam was absent re-arms when
its driver INFs later appear (files are invisible to the endpoint-set
fingerprint, so nothing else would ever retry), and a withheld mic skips
the pointless reinstall (the pair exists; the plan gave it to the loopback).
2026-08-07 10:29:37 +02:00
enricobuehler 1535d69852 docs(release): the 0.25.0 notes cover the 37 commits since the bump
ci / docs-site (pull_request) Successful in 1m13s
ci / web (pull_request) Successful in 1m15s
apple / swift (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 3m14s
android / android (pull_request) Successful in 4m34s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 6m45s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 4m27s
ci / rust (pull_request) Successful in 14m5s
main moved from 8983ec04 to 35ba64ca while this branch sat open, taking the
release from 98 commits to 135. Merged in and folded the new work into the notes.

The largest addition is a new `## Before you update` section, because this batch
carries changes that need the reader to DO something and they were not going to
survive being buried in a Fixed bullet:

  * Linux users of the virtual Steam Deck pad must `usermod -aG punktfunk` and
    log back in, or it stops attaching — the capability moved off the `input`
    group (which every gamepad guide tells you to join) onto its own, because it
    can emulate arbitrary USB hardware.
  * Plugin UIs moved to their own origin on PORT+1, so a self-signed console
    needs the new port trusted once, and custom firewalls/proxies need it opened.
  * Saving a custom launch command re-confirms the console password, and add-ons
    may no longer set launch/pre-launch commands at all — a real break for any
    third-party add-on that populated them.
  * A fresh install now runs the plugin runner by default (upgrades untouched).
  * The Deck setup script used to leave the generated console password
    world-readable, so rotating it is worth a sentence.

The library-sources work is written as GROUNDWORK, deliberately. All six built-in
scanners still ship, still on by default, and nothing is removed — and none of
the replacement add-ons are published yet, so the migration banner only appears
as they arrive. Promising a user they can move Steam to an add-on today would be
the v0.22.3 mistake again: notes describing a build nobody is getting.

Two other honesty items. The Android HUD entry says outright that the stream did
not get faster and the headline number only got smaller because it stopped
counting the compositor's wait — otherwise every reader takes it for a speed-up.
The Windows non-C: settings entry says plainly that nothing is recoverable,
because the writes never reached disk, so there is no orphaned copy to restore
and the reader has to re-enter their preferences once.

`56adb470` (pad-audio WASAPI module path) is deliberately NOT a user-facing Fixed
entry: verified it is not an ancestor of v0.24.0, so it repairs a Windows build
break in code that has never shipped. It folds into the pad-audio feature. Same
for `19f637ea`, which is CI-only.

Under the hood gained the origin-isolation mechanism, the allowlist authorization
gate that fails the build on an unclassified route, store claims and the v2
library.json shape, the registry auth work, the config-writer fallback, send
pacing, and the vendored Deck WSI layer. The unverified list grew too: the origin
split has not been in a real browser, the packaging default-on changes have had
no installer run, and no launcher tile has ever been clicked.

Re-verified after the merge, all green: lock diff versions-only 32/32 against
origin/main, `cargo metadata --locked` resolves (35 members), `cargo fmt
--all --check` clean in both workspaces, doc lazy-continuation scanner 0 hits
over 521 files, notes body 0 internal-vocabulary hits above `## Under the hood`,
Play notes still 494/500 by android.yml's own gate logic. Wire 2, C ABI 16, and
the capability bytes are all unchanged from the bump commit — host_caps still has
exactly one free bit (0x80).

Play's "What's new" is left as it stands: at 494/500 there is no room, and the
only Android-facing additions here (the stats-overlay measurement change and a
certificate-strictness fix) are both worth less to a phone user than any line
already in it.
2026-08-06 00:45:25 +02:00
enricobuehler 745864423b Merge remote-tracking branch 'origin/main' into worktree-release-0250 2026-08-06 00:40:26 +02:00
enricobuehler f702f27bd3 chore(release): bump workspace version to 0.25.0
apple / swift (pull_request) Successful in 1m20s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m14s
ci / docs-site (pull_request) Successful in 1m19s
android / android (pull_request) Successful in 5m33s
ci / rust-arm64 (pull_request) Successful in 5m21s
ci / rust (pull_request) Failing after 5m56s
windows / build (aarch64-pc-windows-msvc) (pull_request) Failing after 41s
windows / build (x86_64-pc-windows-msvc) (pull_request) Failing after 43s
A minor bump: 98 commits since v0.24.0. The headline is DualSense pad audio
(PR #23) — a wired DualSense playing a game's voice-coil haptics and its own
speaker, streamed from the host, on Android and the desktop session client
against a Windows host with Steam's driver present. Behind it: the haptics
sweep's twelve milestones closing more than twenty controller faults across
every client and both hosts; the audio quality/latency work (256 kbps stereo,
the Steam Streaming Microphone endpoint root cause, and the de-jitter ratchet
that left audio permanently behind the picture); and MTU resilience plus
mid-session shard renegotiation, which turns the silent all-black stream on a
sub-1330-byte path into a diagnosed warning that heals itself. Plus the Decky
plugin reduced to a launcher, system-button routing with hold-Select, gamepad-UI
profiles on all three UIs, `discover`/`launch --request-access` in the CLI, and
the Sunshine false-conflict and crashed-host display-restore fixes.

The canary base is already 0.25 — scripts/ci/pf-version.sh derives it as one
minor ahead of the latest stable tag — so this is the version canary has been
publishing against all along.

Wire protocol stays at 2: every addition this cycle is optional or
capability-gated (an optional trailing max_shard_payload on Hello, the 0x08/0x09
renegotiation pair, the 0xD1 pad-audio plane, the 0xD2 redundant desktop-audio
plane, MAX_DATAGRAM_BYTES 2048 -> 9216). C ABI moves 14 -> 16 in two steps: 15
retroactively declares the floor that guarantees the rumble policy engine's C
surface (which shipped while the constant still read 7), and 16 adds the
pad-audio surface and mirrors its two capability bits. Four new capability bits
land in the client/host bytes (audio redundancy 0x04/0x20, pad audio 0x08/0x40);
the video-caps byte was NOT touched and stays full from 0.23.0, so the standing
"next video cap needs a second byte and an ABI bump" note still holds. host_caps
is now down to its last free bit (0x80). Virtual-display driver protocol 6 and
the Windows gamepad channel 3 are untouched — pf-driver-proto is byte-for-byte
identical to v0.24.0. The generated header is in sync (ABI 16, both cap mirrors).

Breaking for C embedders: 149 unprefixed macros are now PUNKTFUNK_-prefixed
(139 #defines renamed in the checked-in header). Mechanical to fix, and it
cannot break silently — the old spellings cease to exist, so it is always an
undeclared-identifier error rather than the wrong value a colliding #define
used to produce.

Lock touched for the 32 workspace members only, via `cargo update --workspace`:
diff against origin/main is versions-only, 32 insertions and 32 deletions. Unlike
the last cut there is no third-party crate sitting on the outgoing version to
trip the count — `wasapi` is at 0.23.0 and was never a candidate. `cargo metadata
--locked` resolves (35 members; fec-rs, pf-driver-proto and usbip-sim keep their
own versions by design). `cargo fmt --all --check` clean in both the main and
packaging/windows/drivers workspaces. Doc lazy-continuation scanner: 0 hits over
521 files — that regex is the exact defect that made the first v0.23.0 tag go red
on Windows clippy, and no Windows leg runs on a main push, so main being green
proves nothing about the tag fan-out.

api/openapi.json is deliberately left at 0.23.0: it tracks API edits and lags,
as in every prior cut. It is now two releases behind and worth a look.

Notes at docs/releases/v0.25.0.md, per docs/releases/README.md — authored with
the bump so CI's ensure_release seeds the release body at tag creation. Body
voice checked programmatically: 0 internal-vocabulary hits above `## Under the
hood`. Play's "What's new" at docs/releases/whatsnew/v0.25.0.txt (494/500 chars),
verified by running android.yml's gate logic verbatim against it, including the
byte-identical-to-another-release check.
2026-08-05 00:05:40 +02:00
301 changed files with 26402 additions and 2443 deletions
+16
View File
@@ -160,6 +160,22 @@ jobs:
key: gradle-${{ hashFiles('clients/android/**/*.gradle.kts', 'clients/android/gradle/wrapper/gradle-wrapper.properties') }}
restore-keys: gradle-
# Clippy for the ANDROID target. Like the kit tests below, this was running NOWHERE: ci.yml
# lints `--workspace` on the host, where `clients/android/native` and every
# `#[cfg(target_os = "android")]` module elsewhere compile out, and this workflow only ever
# built. Discovered in 2026-08 with five lints already resident — code no gate had ever read.
#
# Placed BEFORE assembleDebug deliberately: a lint failure should cost the ~10 s the lint
# takes, not the full three-ABI build first. It shares sccache and the target dir with the
# build that follows, so the compile is not paid twice.
#
# The task lints arm64-v8a AND armeabi-v7a, and reuses the build task's exact cargo-ndk
# environment — see the long note on `registerCargoNdkClippy` in kit/build.gradle.kts for why
# both pointer widths are load-bearing and why the environment must not be duplicated here.
- name: Clippy (Android target, deny warnings)
working-directory: clients/android
run: ./gradlew :kit:cargoNdkClippy --stacktrace
# The kit's JVM unit tests — the pure parsers, migrations and feedback policies. They were
# running nowhere: this workflow only assembled, and android-screenshots.yml runs the :app
# module's tests, so nothing enforced :kit's. Cheap (a couple of seconds against an already
+25
View File
@@ -173,6 +173,31 @@ jobs:
makepkg -f -d --holdver
ls -lh "$GITHUB_WORKSPACE/dist"
# The host must ship a VERSIONED libav soname dep, and nothing else in this pipeline proves
# it. packaging/arch/PKGBUILD lists bare `libavcodec.so` etc. and relies on makepkg rewriting
# each into `libavcodec.so=<soname>-<arch>` from the built binary's DT_NEEDED; if that
# rewrite ever stops happening — Arch dropping the soname `provides`, someone "tidying" the
# entries out of `depends`, a makepkg change — the dep silently degrades to an unversioned
# name that ANY ffmpeg satisfies. That is precisely the 2026-08-08 state in which `pacman
# -Syu` walked every Arch/CachyOS install across the FFmpeg 8 -> 9 soname bump and left the
# host unable to start (exit 127 before main(), restart loop). The failure is invisible in a
# green build and only shows up as a bricked box weeks later, so assert it here.
- name: Assert the host pins the FFmpeg soname
run: |
PKG="$(ls "$GITHUB_WORKSPACE"/dist/punktfunk-host-*.pkg.tar.zst | head -1)"
DEPS="$(bsdtar -xOf "$PKG" .PKGINFO | sed -n 's/^depend = //p')"
echo "$DEPS" | sed 's/^/ depend = /'
for lib in libavcodec libavutil; do
echo "$DEPS" | grep -qE "^$lib\.so=[0-9]+-[0-9]+$" || {
echo "::error::punktfunk-host declares no VERSIONED $lib.so dependency."
echo "::error::makepkg did not expand the bare soname from DT_NEEDED, so pacman can"
echo "::error::upgrade FFmpeg across a soname break and brick the install."
echo "::error::See the depends comment in packaging/arch/PKGBUILD."
exit 1
}
done
echo "OK: $(echo "$DEPS" | grep -E '^libav|^libsw' | tr '\n' ' ')"
# The optional HDR gamescope companion (packaging/gamescope) — a separate pkgbase with a
# completely different dependency set, published into the same repo so `pacman -S
# punktfunk-gamescope` is all an Arch/SteamOS box needs for 10-bit BT.2020 PQ.
+7
View File
@@ -66,6 +66,13 @@ jobs:
test -f node_modules/@punktfunk/host/package.json
test -f node_modules/@punktfunk/host/dist/index.d.ts
# The kit had no biome config and no lint step, while every plugin repo that consumes it does
# — so its source drifted (unused imports, formatting) with nothing to catch it. Now gated
# here, on the same config and pinned biome version the plugins use.
- name: Lint & format
working-directory: plugin-kit
run: bun run check
- name: Typecheck
working-directory: plugin-kit
run: bun run typecheck
+1 -1
View File
@@ -9,7 +9,7 @@
#
# What goes in: scripts/ci/gen-sbom.sh = syft over the checkout (every lockfile-pinned dep in
# both Rust workspaces + the JS trees + Swift Package.resolved) merged with
# compliance/sbom/manual-components.cdx.json (vendored C/C++, bundled DLLs, VB-CABLE, gamescope).
# compliance/sbom/manual-components.cdx.json (vendored C/C++, bundled DLLs, gamescope).
name: sbom
# One pending run per workflow+ref: a newer push supersedes the queued/running one and cancels
# it (a canary only needs the latest commit; each release tag is its own ref so tag runs never
-8
View File
@@ -150,13 +150,6 @@ jobs:
if (-not $env:FFMPEG_DIR) {
"FFMPEG_DIR=C:\Users\Public\ffmpeg" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
}
# VBCABLE_DIR: the pinned official VB-CABLE package (provisioned by
# provision-windows-punktfunk-extras.ps1) -> pack-host-installer.ps1 bundles the
# streaming virtual microphone. Same daemon-env-or-fallback pattern as FFMPEG_DIR
# (the daemon env only refreshes on a runner-task restart).
if (-not $env:VBCABLE_DIR) {
"VBCABLE_DIR=C:\Users\Public\vbcable" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
}
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$v = if ($env:GITHUB_REF -like 'refs/tags/v*') {
$env:GITHUB_REF_NAME -replace '^v', ''
@@ -406,7 +399,6 @@ jobs:
@{ n = 'bun runtime (BUN_EXE)'; p = $env:BUN_EXE; f = '' }
@{ n = 'plugin runner (SCRIPTING_BUNDLE)';p = $env:SCRIPTING_BUNDLE; f = '' }
@{ n = 'FFmpeg DLLs (FFMPEG_DIR\bin)'; p = $env:FFMPEG_DIR; f = 'bin' }
@{ n = 'VB-CABLE (VBCABLE_DIR)'; p = $env:VBCABLE_DIR; f = 'VBCABLE_Setup_x64.exe' }
)
$missing = @()
foreach ($x in $need) {
+699
View File
@@ -0,0 +1,699 @@
# Changelog
Protocol, ABI, driver and embedder detail, one section per stable release, newest first.
This is the **technical** half of a release. The other half — what changed for people who *use*
Punktfunk — is `docs/releases/vX.Y.Z.md`, and it deliberately contains no internal names. The two
were one document through v0.24.0; they split at v0.25.0 because the engineering section had grown
long enough to bury the user-facing half it was appended to. See `docs/releases/README.md`.
If you embed `punktfunk-core`, package Punktfunk, or write a plugin, this file is for you. Start
with the version table of the release you are moving to, then read **Breaking changes**.
---
## v0.25.0
407 commits since v0.24.0.
### Versions
| | v0.24.0 | v0.25.0 | Notes |
|---|---|---|---|
| Wire protocol | 2 | **2** | unchanged — every addition below is optional or capability-gated |
| C ABI | 14 | **17** | three steps; see below |
| Workspace crate dirs | 22 | **26** | `pf-bitstream` (+ vendored `cros-codecs`), `pf-vkdecode`, `pf-dxvadec`, `pf-vaadec` added; `pf-ffvk` removed |
| Virtual-display driver protocol | 6 | **6** | unchanged (minimum accepted still 3) |
| Windows virtual-gamepad channel | 3 | **3** | unchanged |
| Plugin index schema | 1 | **1** | unchanged |
| `api/openapi.json` | 0.23.0 | **0.24.0** | tracks API edits, lags one release by convention |
`crates/pf-driver-proto` is byte-for-byte identical to v0.24.0 — if you ship the virtual-display
driver or the gamepad channel, nothing in this release touches you.
**Why the wire did not move.** It grew a lot and still did not break: an optional trailing
`max_shard_payload: u16` on `Hello` (absent/0 = legacy, doubling as the renegotiation capability
flag and the jumbo receive ceiling); two control messages `ShardPayloadChanged` (`0x08`) and
`ShardPayloadAck` (`0x09`); a redundant desktop-audio datagram tag `0xD2` beside the plain `0xC9`; a
controller-audio plane at `0xD1`; a new `0xCD` kind `0x06`; arrival flag bits 8/9; and
`MAX_DATAGRAM_BYTES` 2048 → 9216. Old peers never send or read any of it. Bump `WIRE_VERSION` only
when the handshake or planes change *incompatibly* — riding a C-ABI bump onto the wire once locked
every new client out of every deployed host (`ABI mismatch: client 3 host 2`, observed live).
### C ABI 14 → 17
- **v15 — the rumble policy engine's C surface.** `punktfunk_connection_next_rumble_cmd`,
`punktfunk_connection_set_rumble_quirks`, `PUNKTFUNK_RUMBLE_QUIRK_*`. These symbols are **not
new**: they landed while the constant still read 7 and no bump was made, so every core since has
exported them while advertising a version that never promised them. A shipped binary says what it
says, so this cannot be corrected retroactively — **v15 is the floor that guarantees them.** At or
above 15 the surface is present; below it, probe for the symbol. No code changed with this bump.
- **v16 — the controller-audio client surface.** `punktfunk_connection_next_pad_audio` (the `0xD1`
per-gamepad DualSense haptics/speaker plane), `punktfunk_connection_set_pad_audio_caps`, and the
`PUNKTFUNK_CLIENT_CAP_PAD_AUDIO` / `PUNKTFUNK_HOST_CAP_PAD_AUDIO` mirrors.
- **v17 — session end reason.** `punktfunk_connection_end_reason` + the `PUNKTFUNK_END_REASON_*`
vocabulary: after a session ends, ask *why* — this client closed it, the host's launched game
exited (its close carried `APP_EXITED_CLOSE_CODE`, which the host had been sending for a long time
with nothing consuming it), the host ended it cleanly, the host reported a failure, or the
connection was lost. Purely a read of state the core already had: **no new call is required of an
embedder**, a client that never calls it is unchanged, and the host sends identical bytes either
way.
### ⚠ Breaking changes
**1. 149 unprefixed macros are now `PUNKTFUNK_`-prefixed** (139 `#define`s renamed in the checked-in
header). Names as generic as `MAX_PADS`, `TAG_LEN`, `ABI_VERSION`, `WIRE_VERSION`, `INPUT_MAGIC` and
the whole `BTN_*` / `AXIS_*` family were landing in the namespace of every program that included the
header.
*What to do:* add the prefix. Values are identical; the change is mechanical.
*It cannot break silently.* The old spellings cease to exist, so this is always an
undeclared-identifier error, never a wrong value — which is precisely the failure being removed. A
colliding `#define` does **not** fail to compile: the preprocessor silently takes the last
definition, so an embedder whose own header defined `MAX_PADS` previously got a wrong value at
runtime. Associated constants are untouched; the generator already qualifies those by type name.
**2. Linux hosts: the virtual Steam Deck controller moved to its own `punktfunk` group.** The
capability rode on `input`, which every gamepad guide tells users to join — but it can emulate
arbitrary USB hardware. Operators must `usermod -aG punktfunk "$USER"` and re-login or the pad stops
attaching. Ordinary virtual gamepads are unaffected.
**3. Plugins may no longer set `launch.command` or the pre-launch command.** Both run through a
shell and are now operator-token only; a plugin that sets them is refused. Third-party plugins that
populated them need updating — use the `launcher_ui` / `xbox` launch kinds instead.
**4. Plugin UIs moved to their own origin** on a second listener (default `PORT + 1`,
`PUNKTFUNK_UI_PLUGIN_PORT`). Reverse proxies and firewalls must forward that port; a self-signed
console needs it trusted separately.
### Capability bits
Four added, all in the handshake's client/host capability bytes:
| Bit | Constant | Meaning |
|---|---|---|
| client `0x04` | `CLIENT_CAP_AUDIO_RED` | can decode the redundant desktop-audio plane |
| host `0x20` | `HOST_CAP_AUDIO_RED` | is sending it |
| client `0x08` | `CLIENT_CAP_PAD_AUDIO` | can render controller audio |
| host `0x40` | `HOST_CAP_PAD_AUDIO` | is sending it |
**Pressure worth watching:** `client_caps` has four bits free; **`host_caps` is down to its last
one (`0x80`)**; `video_caps` has been full since 0.23.0 (`VIDEO_CAP_MULTI_SLICE = 0x80`). The next
video capability needs a second byte *and* an ABI bump — plan for it rather than discovering it.
### Wire planes
- **Controller audio, `0xD1`** — `[0xD1][u8 pad][u8 kind][u32 seq LE][u64 pts_ns LE][opus payload]`,
one Opus frame per datagram behind a 15-byte header. `PAD_AUDIO_KIND_HAPTICS = 0` is the pad's
BACK channel pair (the voice coils) at 5 ms frames; `PAD_AUDIO_KIND_SPEAKER = 1` is the FRONT pair
at 10 ms. Best-effort like every audio plane: loss is a sequence gap concealed by the gap tracker,
silence is a frozen sequence under the mic-mute discipline, host gating at 60 dBFS with a 250 ms
hangover. `0xD2` (redundant desktop audio) deliberately skipped `0xD1` to reserve it for this.
- **`HidOutput::AudioCtl`** — `0xCD` kind `0x06`, carrying the DualSense output report's
volume/routing bytes, change-only and value-deduped. Older clients drop it as an unknown kind.
- **Arrival flags** — bits 8 (haptics) and 9 (speaker), sent only toward a `HOST_CAP_PAD_AUDIO` host.
- **Adaptive-trigger effects are length-bounded** on encode and decode against one shared constant;
the header emits `uint8_t effect[PUNKTFUNK_HID_EFFECT_MAX]` in place of a literal `11` (same value,
so the struct layout is byte-identical). A zero-length effect body is now rejected rather than
decoding as an empty — that is, a *release* — effect.
- Out-of-range pad indices are dropped before **either** rumble consumer sees them. The reorder gate
bounds-checked and the legacy queue did not, so an embedder draining it could be handed an index it
would use to subscript its own array. The client also clamps the host's rumble lease receive-side
at 5 s, where the ceiling had been sender-side only.
### Host environment variables
| Variable | Default | Notes |
|---|---|---|
| `PUNKTFUNK_AUDIO_QUALITY` | `high` | `low`/`standard`/`high`; `high` = stereo 256 kbps. `standard` reproduces the pre-0.25 encoder exactly for an A/B. A typo warns once rather than silently downgrading. |
| `PUNKTFUNK_AUDIO_REDUNDANCY` | unset = automatic | on when the client supports it and the budget allows |
| `PUNKTFUNK_AUDIO_OUTPUT_MODE` | `client_only` | `client_only`/`host_and_client`/`follow_default`. **Windows host only.** |
| `PUNKTFUNK_PAD_AUDIO` | on | `0` disables controller audio host-wide |
| `PUNKTFUNK_PAD_AUDIO_SLOTS` | `1` | max 4; multi-pad needs an operator to raise it |
| `PUNKTFUNK_PAD_AUDIO_STAMPS` | unset | debug bisect hook |
| `PUNKTFUNK_WIRE_MTU` | unset | pins on-wire IP MTU for all sessions; above 1500 also enables jumbo |
| `PUNKTFUNK_JUMBO` | unset (off) | fixed 9000-MTU profile |
| `PUNKTFUNK_UI_PLUGIN_PORT` | `PORT + 1` | the plugin-UI origin |
| `PUNKTFUNK_LIBRARY_ART_ROOTS` | platform default | art-serving roots; POSIX now defaults to `$HOME` |
| `PUNKTFUNK_DECODER` | client | **values changed**: `native-vulkan` · `native-vaapi` (Linux) · `native-d3d11va` (Windows) · `software`. Legacy `vulkan`/`vaapi`/`d3d11va` still accepted and migrated. Now **trimmed** — a trailing space used to fall through to `auto` silently. |
| `PUNKTFUNK_VAAPI_DEVICE` | client | **new** — pin the VAAPI render node |
| `PUNKTFUNK_DUMP_VIDEO` / `PUNKTFUNK_AU_DUMP` | client | **new** — capture exact decoder input / the AU as it arrived from the host |
| `PUNKTFUNK_AU_FAULT=drop\|truncate\|flip[:period]` | client | **new** — deliberate decoder-input corruption for recovery testing; native rungs only |
| `PUNKTFUNK_NVENC_SPLIT_ARBITRATE=1` | host | **new** — opt-in live split-encode arbitration (Linux-wired) |
| `PUNKTFUNK_NO_AUDIO_MINT` | host (Win) | **new** — opt out of minted endpoints; restores the name ladder |
| `PUNKTFUNK_GPU_PRIORITY` | host (Win) | **removed** — superseded by `PUNKTFUNK_GPU_PRIORITY_CLASS`, a strict superset |
| `PUNKTFUNK_FFMPEG_LOG` | client | **removed** with the av_log machinery |
Legacy `PUNKTFUNK_HOST_AUDIO=1` and `PUNKTFUNK_KEEP_DEFAULT=1` still work, mapping to
`host_and_client` and `follow_default`; `follow_default` wins if both are set. New devtest command:
`punktfunk-host pad-endpoint ensure|remove|status`.
### Security
- **Origin isolation.** A second listener serves `/plugin-ui/**` and nothing else; the console origin
refuses those paths and the plugin origin refuses everything else, `/api/**` above all. Different
origin (scheme+host+port) so same-origin policy *is* the boundary; same site so the `SameSite=Lax`
session cookie still flows. Bind failure disables plugin UIs rather than falling back.
`x-pf-listener` is stripped inbound and set by the entry; active ports republish as
`*_PORT_ACTIVE`; the plugin origin's CSP names the console as its only `frame-ancestors`; the proxy
allowlist drops the plugin's `Clear-Site-Data`, `Access-Control-Allow-Origin` and `Set-Cookie`.
⚠ The kit's `postMessage(..., "*")` is **load-bearing** — narrowing it to `location.origin` would
target the plugin's own origin and drop every message.
- **Authorization is an allowlist with a build-time gate.** `plugin_may_access` is a list of
permitted `(method, path)` pairs with `{}` segment matching, enforced by a test that walks the live
route table and **fails the build on any unclassified route** — the block-list it replaces let new
endpoints through silently. Field authority is tracked separately from route reachability:
requests carry the lane that authorized them, and `prep` / `launch.kind = "command"` are
operator-token only.
- **Art serving** gained an extension whitelist plus magic-byte sniffing, canonicalize-or-refuse, UNC
refusal, config-dir exclusion and root checking, with `file://` percent-decoded *before*
canonicalization so `%2e%2e` cannot hide. Validation also runs at write time, so an unservable path
can no longer be persisted.
### Native decode — FFmpeg is gone from the client
268 files, +129k / 25k. `cargo tree -p punktfunk-client-session` finds zero `ffmpeg`. **The host
keeps `libavcodec` unconditionally** (pf-encode); no host workflow, packaging script or licence file
was touched.
| Platform | v0.24.0 | v0.25.0 |
|---|---|---|
| Linux desktop | ffmpeg-next: Vulkan hwcontext (`pf-ffvk`) → VAAPI → libavcodec sw | `pf-vkdecode` (ash, presenter's own `VkDevice`, zero-copy) → `pf-vaadec` (dlopen'd libva, DRM-PRIME dmabuf) → `openh264` + `rav1d` |
| Windows desktop | ffmpeg-next Vulkan → libavcodec D3D11VA half | `pf-vkdecode``pf-dxvadec` (plans into `ID3D11VideoDecoder`) → `openh264` + `rav1d` |
| Android | MediaCodec (never had FFmpeg) | unchanged |
| Apple | VideoToolbox (never had FFmpeg) | unchanged |
**Workspace members:** added `pf-bitstream` (+ vendored `cros-codecs`, compiler-enforced
`unsafe`-free), `pf-vkdecode`, `pf-dxvadec`, `pf-vaadec`; removed `pf-ffvk`. **Deleted:**
`video_vulkan.rs`, `video_vaapi.rs`, `video_libav.rs`, the libavcodec half of `video_d3d11.rs`, the
`av_log` machinery, `ffmpeg::codec::Id` as decoder vocabulary, `DecodedImage::VkFrame`/`::Dmabuf`,
the `ffmpeg-fallback` feature, and swscale — and with it the BT.601 default its correction code
existed to undo.
**Software rung:** `openh264 = "0.9"` (BSD-2) and `rav1d = { version = "1", default-features =
false, features = ["bitdepth_8"] }` (BSD-2). `dav1d-sys` was rejected because it is `system-deps`-
only and would add a system library plus a `.pc` to every client package. `default-features = false`
drops `asm` — rav1d's `build.rs` *panics* without nasm, unlike openh264-sys2, which degrades quietly.
**`bitdepth_8` only** ⇒ software AV1 refuses 10-bit by contract, read from the sequence header before
any byte reaches the decoder.
**⚠ HEVC has no CPU floor.** An HEVC session that exhausts its hardware rungs tears down and re-dials
advertising HEVC-less caps, and the host picks H.264 (`last_rung_verdict` / `NoSoftwareRung`). This is
a first-class path, not a failure.
**Rung × codec × hardware evidence** (`native_evidence`) — the admission filter is driven by this, so
an unproven rung yields only to one that is both verified for the codec and usable on the device:
| Rung | Codecs | Evidence |
|---|---|---|
| `native-vulkan` | H.264, H.265 Main/Main10/4:4:4 | **yes** — bit-exact vs libavcodec, 250/250 AUs on 3 drivers + 92-min soak |
| | AV1 | **yes** — 250/250 bit-identical on one vendor, no soak |
| `native-d3d11va` | H.264, H.265 | **yes** — frame-hash parity on RTX 4090 + AMD iGPU, 30-min soak |
| | AV1 | **not proven** — decoded 4K60 once, no parity, no soak ⇒ excluded from the filter |
| `native-vaapi` | H.264, H.265, AV1 | **NO — has never decoded a frame anywhere**; no VAAPI hardware was reachable |
| `software` | H.264 (openh264), AV1 (rav1d) | **not proven**; openh264 has never run on glass. No HEVC at all. |
Vendor order (unchanged): Linux NVIDIA/AMD `vk → vaapi → sw`; Linux Intel/unknown
`vaapi → vk → sw`; Windows NVIDIA/AMD `vk → d3d11va → sw`; Windows Intel/unknown
`d3d11va → vk → sw`.
**AV1 advertisement** now answers from device facts (`av1_hardware_decodable`: Vulkan `DECODE_AV1`
queue op, or the Windows D3D11 import path) rather than `ffmpeg::decoder::find(AV1)`, which was true
on any build linking libdav1d. **Settings migration:** stored `vulkan`/`vaapi`/`d3d11va` migrate to
`native-*` at decoder construction *and* at each dialog's lookup — the second is load-bearing, since
an unmatched value renders as "Automatic" and a save would silently rewrite the preference.
### The three decode data-loss bugs
**AV1 sub-frame truncation — shipped in v0.24.0, host-side.** NVENC sub-frame readback has two halves
armed by *different* conditions: `build_init_params` arms the writer from `subframe_on` alone, while
the chunked reader additionally requires `slices >= 2` — and `resolve_slices` returns `1` for AV1
unconditionally, because AV1 partitions via tiles, not slices. So an AV1 session told the driver to
publish tile-by-tile and then took only the first tile. Measured at 4K60: every AU carried a header
declaring two tile rows plus a single Tile Group OBU with `tg_start = tg_end = 0`; libdav1d rejected
**835/836** AUs. NVIDIA's *hardware* decoder accepts it (so Vulkan Video looked healthy at 60 fps);
its DXVA path did not. 1080p is one tile and unaffected; 4K splits into two tile rows and loses half
the picture. Fixed by disarming sub-frame for AV1 while leaving `split_mode` untouched — AV1 keeps
every engine. Arming the reader instead is *not* a drop-in: the reader cuts at
`bitstreamSizeInBytes` on the reasoning that slices are contiguous Annex-B, which AV1 OBUs are not.
Post-fix 654/654 clean. The test that had pinned the old behaviour as *correct* is replaced by one
pinning the disarm, plus one comparing the reader's gate against the writer's — the comparison
nothing made.
**HEVC DPB from the level ceiling — new in this release, client-side.** `dpb_limit` computed
`max(A-2_level_ceiling, sps_max_dec_pic_buffering_minus1 + 1)`. HEVC equation A-2 is a **ceiling on
what an SPS may legally signal**, not a statement of need, and it branches on picture size against
the *level's* `MaxLumaPs`. The host is blameless: NVENC autoselects L5.1 because the bitrate exceeds
L5.0's ceiling, and signals six pictures at every resolution. At 720p and 1080p the A-2 branch yields
16 frames / **17 slots** — one more than NVIDIA's `maxDpbSlots` of 16 — so every AU fell outside
device caps, flushed, waited for an IRAP, and the fresh IDR needed 17 again; rungs exhausted, and
there is no software HEVC. It hid because the path was only ever exercised at 4K, the one size that
falls through to the honest answer. Fixed to `buffering.min(16)`: the `max()` bought no tolerance,
since `Dpb::needs_bumping` already evicts at the signalled depth — it only over-allocated ten
surfaces per 1080p session. **H.264 escaped by luck** (its ceiling lands at 13 for 1080p) and is left
alone, because H.264's DPB size genuinely *is* level-derived absent a VUI `bitstream_restriction`.
**rav1d aborts the process — new in this release, client-side.** rav1d 1.1.0 `abort()`s on *any*
decode error while holding one frame context: the `c.fc.len() == 1` branch decodes inline, always
finishes in `rav1d_decode_frame_exit` which unconditionally takes `frame_hdr`, then on `Err` re-enters
an `on_error` whose first act is `frame_hdr.as_ref().unwrap()` on the `None` it just left. The panic
unwinds into `dav1d_send_data`, which is `extern "C"``panic_cannot_unwind``abort()`. **No
`catch_unwind`, no rung demotion and no refusal can catch it**, and every `rav1d_*` entry is
`pub(crate)`, so no in-process guard is possible. 4K was only *where* the first error happened — the
CPU rung does 3539 fps against a 60 fps stream, the backlog stopped draining, the pump flushed to
live, and the next AU referenced undecoded frames. Fixed by opening with `n_fc >= 2` and asking
`dav1d_get_frame_delay` what the settings actually bought. Decode now drains **past** the first
`EAGAIN`, which is why two frame contexts cost no latency (2042 ms/unit at `n_fc=2` vs 2153 at
`n_fc=1`). On glass: 4K60 AV1 was SIGABRT on the second frame every run; after, exit 0 with 1204
frames and 13 decode errors recovered across 17 backlog flushes. Reported upstream as **rav1d#1497**
with a reproducer. Does **not** make the CPU rung panic-proof.
**Settings loader BOM — shipped in v0.24.0, client-side.** `.and_then(|s| from_str(&s).ok())` turned
every parse failure into `Default`. `Set-Content -Encoding UTF8` writes `EF BB BF`, serde_json
correctly rejects at byte 0, and every setting vanished silently. A shared `load_json_or_default` now
strips the BOM and warns with path plus serde line/column, covering settings, known-hosts (where a
BOM silently unpaired every host) and profiles on both desktop clients. The result is deliberately
still `Default`, never an error.
### Other decode/encode
- **Intel Arc pNext ordering.** `vkGetPhysicalDeviceVideoCapabilitiesKHR` was called with the codec
caps struct chained *before* `VkVideoDecodeCapabilitiesKHR` (`push_next` prepends). Arc/Windows
fills those two **by position, not by sType**, and returned them swapped — we read a level as a
capability bitmask. Measured A/B: `decode_flags_raw=12 max_level_idc=1` before,
`decode_flags_raw=1 max_level_idc=12` after. NVIDIA and RADV dispatch by sType, which is why the
fleet stayed green. ⚠ **This does not yet give Arc Vulkan Video** — the refusal only moves down: the
device advertises only COINCIDE, and its NV12 coincide entry does not advertise `SAMPLED` usage,
which the zero-copy presenter needs. Unresolved whether that is ours or an Intel constraint.
- **NVENC split encode.** The 10-bit rule sat *above* the pixel-rate arm and took no codec, so it
vetoed 10-bit 4K120 — the exact case the pixel-rate arm exists for — and applied an
HEVC-Main10-on-Ada result to AV1 10-bit, which has no such measurement. Re-measured on Ada and
Blackwell: 4K60 2.06×, 5120×1440@240 1.31×, 4K120 1.89× — **split wins at every mode on both
architectures, including the configuration the veto came from.** New order: env override →
pixel-rate arm (now taking `max_forced_split_mode(engines)`, not a hard-coded 2) →
HEVC-Main10-below-the-bar → AUTO. Operator over-asks are clamped with a warning because **the driver
honours an over-ask and silently encodes narrower**. Also newly logged: HEVC + plain AUTO +
sub-frame is **silently single-engine** — the fleet's default shape, and nothing said so.
**Unvalidated consequence:** 5120×1440@240 Main10 now clears the pixel-rate bar and *will* be
forced to split — the exact configuration the old veto came from. `PUNKTFUNK_SPLIT_ENCODE=0` is the
escape.
- **PyroWave on Windows stamped over the host's GPU scheduling policy.** It raised the process WDDM
class to HIGH at every session open, while `auto_priority_gate` already owns that process-wide —
starting at HIGH, *upgrading* to REALTIME once safe, and leaving a monitor that drops back when VRAM
tightens (REALTIME + NVIDIA + HAGS + near-full VRAM is a documented NVENC hang). Opening PyroWave
stamped HIGH back and **orphaned the monitor's decision**. Removed rather than reconciled.
- **A `pf-vkdecode` AV1 use-after-free fix had stabilised the wrong pointer** —
`OwnedStdAv1SequenceHeader` kept the Std struct *inline*, so `pStdSequenceHeader` was a dead stack
address; it worked only because NVIDIA happened to retain `pColorConfig` instead. Std structs are
now boxed inside each owning wrapper, and create-time arrays are fields of the stored parameters
assembled at their final address. The same shape was fixed pre-emptively in H.264/H.265.
### A/V sync — it did not previously exist
The host has always stamped `pts_ns` on every audio datagram. **Every client decoded it into
`AudioPacket` / `AudioPCM` and never read it.** Video's `pts_ns` was used end to end; audio free-ran
at whatever depth its jitter ring reached; nothing compared them. The A/V offset was an emergent
property of buffer depths — it moved whenever the ring ratcheted under underrun pressure, and it got
**worse every time video got faster**, because a quicker decoder lowers the video leg and leaves
audio's where it was. That is why shaving milliseconds off the audio budget had never helped.
Two host defects were prerequisites:
- **`pts_ns` was stamped at encode time**, inside the loop draining an already-accumulated chunk, so
every frame of a chunk carried near-identical timestamps describing *when we got round to
encoding*. Now derived from the chunk's arrival instant minus queued-frame duration, re-anchored
per chunk.
- **The host did not pace.** One capture callback hands over a whole quantum (5 ms honoured, **21.3 ms
on a VM**, where stock PipeWire raises `min-quantum` to 1024), drained into back-to-back
`send_datagram` calls — a 45 frame burst then ~21 ms of nothing, which a ring could only absorb by
standing a burst period deep. Frames now leave on the audio clock (`FRAME_INTERVAL` 5 ms,
`PACE_MAX_SLEEP` 10 ms, `PACE_REANCHOR` 100 ms). Costs no average latency.
```
audio_e2e = (now + buffered_ahead + clock_offset) pts_ns
av_offset = audio_e2e video_e2e (> 0 ⇒ audio behind the picture)
```
`AvSync` EWMAs it (`AV_EWMA_TAU_MS = 2000`), ignores anything inside `AV_DEADBAND_MS = 10`, waits
`AV_MIN_OBSERVATIONS = 100` before a first correction, and **refuses rather than clamps** beyond
`AV_SANE_LIMIT_MS = 1000` — a wall-clock step must not steer the ring.
**Video is the master, and continuity outranks sync.** `JitterPolicy::set_sync_target` takes only a
*request*, clamped between the existing underrun-driven adaptive floor and the hard cap: a link whose
jitter genuinely needs more buffer than the picture is away keeps its buffer, and the residual is
reported rather than forced. `None`/`nil` reproduces prior behaviour bit-identically, which is how
the four rings adopted it one at a time.
Per client: the Rust desktop reference is a new `video_e2e_ns` atomic beside `clock_offset`, written
by the presenter and read by the audio thread. **Android** publishes `OnFrameRendered` — the one
place that knows a frame *latched***raw, not floor-shaved** (the HUD shaves the OS present floor;
sound must reach the ear when light reaches the eye), and stays inert below API 33 rather than
substituting the release instant, which targets a future vsync 821 ms ahead of glass. **Apple**
publishes its `LatencyMeter` sample as an *expiring level*, because that client has a backgrounded
keep-alive that keeps audio playing while dropping video decode; its clamp raises the ceiling to the
floor rather than `min(max(…))`, which on a device whose callback quantum alone exceeds the hard cap
would otherwise hand back the cap, silently below the continuity floor.
Escape hatches: `PUNKTFUNK_NO_AV_SYNC=1` everywhere, plus
`adb shell setprop debug.punktfunk.no_av_sync 1` on Android (a launcher-started app inherits no
environment). Observability: `buffer_ms`/`target_ms` had only ever been a `tracing::debug!` line —
and on a Deck the client runs under Steam's `reaper` with stdout on a pipe nobody can read, so the
one number identifying a deep ring was unobtainable *on the device reporting the latency*. Now on the
HUD and in the 1 Hz stats log on every client.
### Decode-target aliasing — caught before it shipped
**None of this ever shipped.** `git ls-tree v0.24.0 crates/` has no `pf-vkdecode`, `pf-dxvadec`,
`pf-vaadec` or `pf-bitstream`; v0.24.0's decode rungs were libavcodec. This was a ship-blocker for
the new stack, cleared — not a field bug.
Three of the four native rungs released a picture's surface **inside the plan→submission
conversion**, then assigned the decode target a slot. `SlotMap::assign` returns the *lowest free
slot* — the one just vacated. The submission then named one surface as both decode target and its own
reference: `CurrPicTextureIndex == RefFrameMapTextureIndex[k]` on DXVA, or `pSetupReferenceSlot`
sharing an array layer with `pReferenceSlots` on Vulkan. **Decode into the surface you are predicting
from.**
- **AV1 / D3D11VA** — AV1 applies `refresh_frame_flags` *after* decode (7.20), so "read a slot then
overwrite it" is the ordinary case: **268 of the vendored vector's 274 frames**, first at frame 6.
- **H.264 / both Vulkan and D3D11VA** — `H264Planner` snapshots `dpb_refs` in `begin_picture`, before
8.2.5 marking and the C.4.5.3 bump, so a picture the sliding window unmarks and the bump evicts
lands in *both* `dpb_refs` and `dpb.removed`. Both conditions coincide only in low-delay H.264 —
and NVENC guarantees it (`max_num_ref_frames = 3` alongside `max_dec_frame_buffering = 3`, plus
`max_num_reorder_frames = 0`). Result: **297 of every 300 access units of every stream a punktfunk
host emits**, at every resolution, on both rungs.
- **H.265 is exempt, now measured rather than argued** — 0 of 120 aliases, with a counterfactual that
moves the snapshot one call earlier and reproduces 115 of 120.
- **VAAPI's exemption was incidental**: the precondition is fully present (117 of 120 AUs) but
`plan_to_va` never invents a surface. That held only because three call sites happened to write
`free_surface()` and `surface_table()` adjacently; `acquire_target` now returns index, surface and
table together so a later edit cannot split them.
Fix is uniform: the plans grow `release_after_decode`, conversions hand removals back, callers
release once the decode op is issued. Costs no slot (`SlotMap::new` allocates `max_dpb_frames + 1`).
Both rungs hold the `Result` rather than `?`-ing it so the deferred release runs on failure paths —
seven exits sat between conversion and release, each of which would have leaked a slot.
**Why four gates missed it**, all recorded: the conformance vector is *structurally blind* (level 1.3,
no VUI `bitstream_restriction` ⇒ a 7-frame DPB against 2 reference frames, and it reorders) and
passed 250/250 for two milestones; **a test had encoded the bug as correct**; another assertion was
*vacuous* (it asserted the decode target was never also a reference while handing every picture its
own never-reused surface id — distinct integers cannot collide); and **it streamed clean** — *"the
2026-08-07 field sessions that looked clean were looking at wrong pixels."*
`gpu_parity` is now **11 legs** (not 9 — that note was written mid-PR): each decodes a vendored stream,
reads back every output frame's NV12, crops to the display region and SHA-256s in *display order*
against libavcodec goldens, frame count and flush tail included. The three new legs are our own
encoder's output rather than conformance vectors — H.264 because the vector is blind to the shape,
H.265 because an exemption with no stream behind it is how the H.264 defect survived two milestones,
AV1 because the vector is one tile on all 274 frames while our encoder splits 4K into two tile rows,
so every tile array the conversions fill had only ever been written at index 0. `video_vaapi_native`
parity is new entirely: 7 legs, bit-identical on RDNA3.
⚠ Promoting D3D11VA AV1 to `verified` **changes rung selection** on Windows Intel/unknown vendors, not
just a label. VAAPI stays `verified = false` deliberately — one vendor, never soaked; flipping it
would move `auto` off Vulkan Video on every Linux AMD/Intel client including the Deck.
### FFmpeg 9, and the Arch soname trap
`pf-encode` now builds against **FFmpeg 9**. The host still links libavcodec unconditionally; the
client has none (see above).
**`pacman` is the only one of our packaging formats that does not derive dependencies from ELF
`DT_NEEDED`.** rpm auto-generates `libavcodec.so.62()(64bit)`, `dpkg-shlibdeps` emits `libavcodec62`,
nix pins the closure — but a bare `depends=('ffmpeg')` let `pacman -Syu` walk the host across a
soname bump with no warning and no conflict. FFmpeg 8 → 9 (`2:9.0-5`: libavutil .60→.61, libavcodec
.62→.63, libavfilter .11→.12, libavdevice .62→.63, libswscale .9→.10) therefore **bricked every
Arch/CachyOS install**: the dynamic loader cannot start the binary, so it is **exit 127 before
`main()`** in a systemd restart loop, with nothing in the host's own log to explain it.
`ldd /usr/bin/punktfunk-host | grep "not found"` is the one-line diagnosis.
⭐ The fix is **SONAME deps, not a hand-written version bound**: `depends=(… 'libavcodec.so'
'libavutil.so' …)`. Arch's ffmpeg declares matching `provides=(libavcodec.so=63-64 …)`, and makepkg
rewrites each bare `libfoo.so` into `libfoo.so=<soname>-<arch>` by reading the built binary's
`DT_NEEDED` — so the bound tracks whatever FFmpeg the builder linked against with nothing to
maintain across the next bump. A literal `ffmpeg<2:9` would go stale on every bump. pacman now
refuses the upgrade instead of bricking the install. All seven libs are listed even though
`--as-needed` currently drops two: an unlinked soname is left bare by makepkg and satisfied by any
ffmpeg, so listing it costs nothing and a future link picks up the bound automatically.
### Linux playback filled the buffer ceiling
The PipeWire playback callback sized its writes from the mapped buffer's **capacity** — PipeWire's
quantum limit, 8192 frames ≈ 170 ms — instead of the graph's per-cycle ask (`pw_buffer.requested`).
Every cycle queued up to 170 ms of PCM downstream of the ring **and** taught `JitterPolicy` that the
device drains 170 ms per callback, so the underrun floor (want + one frame) rose above any depth the
A/V sync loop could request: sync measured audio ~280 ms late and was then forbidden — **by its own
continuity rule** — from draining it. The first on-glass run of the latency overhaul showed exactly
that: `audio buffer 272 ms, a/v +284 ms`, stable. Now honours `requested` (capacity remains both the
ceiling and the fallback when `requested == 0`) and logs requested-vs-capacity once per stream.
Needs libpipewire ≥ 0.3.49; every ship target clears it.
### Windows audio substrate
The host now mints its **own** devnodes from Valve's INFs (`SteamStreamingSpeakers.inf` /
`SteamStreamingMicrophone.inf` under `{CommonProgramFiles(x86)}\Steam\drivers\Windows10\…`) instead
of bundling VB-CABLE.
- **Two persistent endpoints**, `Punktfunk Speakers` (client-only loopback sink — the wiring plan
parks the default playback on it during a stream, its WASAPI loopback feeds the encoder, the host
stays silent) and `Punktfunk Microphone` (host writes decoded client voice into the render side;
the capture side surfaces as the mic). Both survive host restarts and re-resolve by marker.
- **Identity is the recorded endpoint id, never the name** — a minted instance is name-identical to
Steam's primaries. Durable marker `PunktfunkAudioRole` (1 = Speakers, 2 = Mic) under Device
Parameters. Name stamping is device-desc + device-name **only**: a wider stamp set makes
`AudioEndpointBuilder` re-mint under a new GUID. Best-effort via the SYSTEM ACL route; on failure
the endpoint still wires and simply keeps the driver's default name.
- **Format stamps are per-direction.** Render gets the PCM16-device / float-mix stereo split; capture
gets the **device-format key only** — mix and host-format keys are render-engine properties, and
stamping them onto a capture endpoint breaks its shared-mode graph (`IsFormatSupported` reports
2ch/48k fine, `Initialize` then fails `0x88890008`).
- **`MintedIds` is tier-0 in the wiring plan.** The mic takes its minted device outright (paired by
provider id — a name search cannot distinguish it from the primary); the loopback prefers the
minted sink at the head of the silent tier. Below that the old ladder is unchanged: Steam primaries
→ cable → real hardware. `PUNKTFUNK_MIC_DEVICE` still beats everything.
- **Mic-vs-loopback arbitration**: the mic may hold the Streaming Microphone only while the loopback
still gets a non-last-resort pick; otherwise the loopback takes it and `mic_withheld` is set. This
fixes a field case where a headless Steam-only host streamed **silence**.
- **New `AudioReadiness`** — `Full` / `AudioOnly` / `MicOnly` / `Nothing`, logged on every plan
change and surfaced at `GET /api/v1/status``RuntimeStatus.audio` (`AudioWiring`, Windows-only,
absent before the first wiring pass; a status poll triggers no COM work or `IPolicyConfig` writes).
The console Dashboard renders it as an "Audio wiring" card.
- **Requires Steam installed** (never running) — without the INFs the host streams video only, and
picks the drivers up automatically if Steam is installed later. Opt out entirely with
`PUNKTFUNK_NO_AUDIO_MINT`, which restores the previous name-based ladder exactly.
-**VB-CABLE is no longer bundled but is deliberately NOT uninstalled** — it is a third-party
shared component other apps may use, and it stays in the ladder as a live fallback. Demoting it was
considered and rejected: on a box where minting transiently fails, that would let the Steam
Streaming Microphone outrank an installed cable, steal the silent sink and make stream audio
audible on the host.
-**The minted endpoints survive Punktfunk's uninstall by design** (they are plain instances of
Steam's drivers and are inert without the host). There is no user-facing removal path; cleanup is
the devtest `punktfunk-host audio-probe cleanup`.
- New devtest: `punktfunk-host audio-probe ssm|sink|sss-primary|mint|plan|micpitch|micpins|cleanup`.
`plan` is the field-triage command; `micpins` maps exclusive+shared `IsFormatSupported` across
{1,2}ch × {16,32}bit × {44.1,48,96}kHz on both mic pins.
### Apple audio
- **The microphone was never in the render graph.** On the combined (voice-processing) engine — made
default a week earlier and never run on a device — the input node carried a tap and **no
connection**, so nothing pulled it: the IO unit came up, the recording indicator lit for a beat,
and not one buffer ever reached the tap, with no error and no failed start. The 10 s silence
tripwire counts *captured* frames, so it never fired. Input now runs through a silent sink into the
main mixer at `outputVolume = 0` (Apple's own voice-processing sample topology). Two more: the tap
read the input format **before** `prepare()`, and enabling voice processing swaps in the VPIO unit
and renegotiates, so the pre-swap read could be 0 Hz / 0 ch; and a mic-chain failure on the
voice-processed engine took the whole uplink down for the session — it now falls back to the split
path, because **the mic outranks the AEC**.
- **No packet-loss concealment on the one client that decodes Opus in core.** Linux, Windows and
Android all feed an `AudioGapTracker` and synthesize libopus PLC; the in-core path had the tracker
sitting unused in the same crate and decoded only packets that arrived. At ~200 packets/s of 5 ms
frames every lost datagram was a hard time-domain gap — one click per loss. The redundant plane
(`0xD2`) hides single losses, so the survivors were exactly the burstier gaps that most needed
concealing. Concealed frames now land in front of the arriving frame in one contiguous buffer, a
DTX marker advances accounting without being decoded, and the output buffer is pre-sized for a full
concealment run so the borrow-until-next-call pointer cannot dangle (50 ms cap).
- **The Apple jitter ring never grew.** The shared Rust `JitterPolicy` has an adaptive target floor;
the hand-written Apple mirror mirrored the *shed* half but not the *growth* half, pinning its
target at the 20 ms base forever. On Wi-Fi that bunches arrivals, 20 ms is regularly shorter than
one delivery stall, so the ring re-primed through every stall for the whole session. Now the full
`note_read` mirror: 3 underruns in a 5 s window grow the target 10 ms (capped at CoreAudio's 70),
30 s of quiet steps back, and the write-side hard trim follows the grown target.
### Clients
- **Nothing in the desktop console had ever been clickable.** `SkiaOverlay::handle_event` matched
only `KeyDown` and `TextInput`, so every mouse button, wheel and touch contact fell past the console
into the run loop, which routes pointer input exclusively at `stream.capture``None` while
browsing. New `Overlay::handle_pointer` carries mouse/touch in swapchain pixels; the run loop
converts (it owns the window and hence display scale); the console hit-tests the rects it drew last
frame. Only **direct** touch devices are offered — an indirect trackpad already drives the mouse.
Widgets act on **press**, not release, because both carousels scroll the focused item toward centre
and what you pressed would slide out from under your finger. Host menu on Up from a saved tile;
`UpdateHost` edits **in place** (remove-and-re-add would silently drop the fingerprint, learned MAC,
pinned cards and profile binding), and `ForgetHost` arms on first press and fires on second.
- **Discovery went permanently deaf three ways**, each needing an app relaunch: a failed resolve was
never retried (`browseResultsChangedHandler` fires only when the result *set* changes, and a host
whose resolve failed is still in the set); a stuck resolve never ended (`NWConnection` has no
timeout, so the throwaway UDP flow could sit in `.preparing` forever, and a service with a
connection in flight was skipped); and an `NWBrowser` parking in `.waiting` was ignored — **which is
exactly where iOS's local-network privacy prompt lands on first launch, and granting it does not
revive the browser that was already waiting.** A 1 Hz sweep now times out stuck resolves, retries
failed ones on a 1→30 s backoff, and re-arms a dead browser; the advert's TXT is re-read on every
browse report. `discovery::Rescan` forces a fresh mdns-sd query — the browse otherwise re-queries on
a doubling backoff **capped at one hour**, so a long-lived browse is effectively passive. ⚠
`clients/windows/src/discovery.rs` is a **second copy** of the browse that the earlier IPv4 pinning
missed; it took an arbitrary first address, so a host whose OS responder answered AAAA rendered a
card that failed on every click.
- **Phone gyro mirror**, off by default, player 1 / wire pad 0 only, and only while that pad has no
motion source of its own. iOS/iPadOS only on Apple (`DeviceGyro` wraps `CMDeviceMotion` at ~100 Hz
on a dedicated serial queue — the controller path's main-queue delivery is a known jitter source);
Android phones with a gyroscope at ~200 Hz with `maxReportLatencyUs = 0`, since batching is poison
for gyro aim. Both rotate from the device's natural frame into the controller frame by interface
orientation, and both send **one zero-gyro sample on stand-down** — the host holds motion as state
and re-emits it, so a leftover nonzero angular velocity reads as endless rotation.
- **Safe-area resolution** is purely a *sizing* change — no layout change, no input change; pointer
mapping follows for free since both clients derive the picture rect from the live host mode. Full
native height, width less left+right safe insets. Portrait settings screens report the housing on
`top` with zero horizontal insets, so the portrait top inset stands in (gated so an iPad's status
bar never fabricates one). Android adds the rounded-corner radius, which it does not count as
cutout. Both even-floor and clamp, because `validate_dimensions` rejects odd dimensions and an inset
subtraction lands odd about half the time.
- **Gamepad UI**: six sections (Stream · Video · Audio · Controller · Interface · Profiles, plus Input
on the desktop console) walked with L1/R1 with per-section cursor memory; 12 palettes under one
shared `ui_palette` key, Violet keeping its explicit sixteen colours so existing installs are an
identity transform. Presentation only → **device preference, never part of a profile**. Palette
maths ported three times (Rust/Swift/Kotlin) with the same assertions pinned in each language;
`every_palette_is_multi_tone` fails under 45° hue spread and caught Ember at 35° and Graphite at 3°.
Three render-only findings: additive blending blows out over a pale ground, a white scrim at the
dark field's strength bleaches the gradient, and white glass over a bright field needs more body.
### Session and game lifetime
- **`PunktfunkEndReason` replaces a single "closed" bit** (ABI 17, additive, wire untouched). Five
values — local, game exited, host ended, host error, lost — classified by the connection watcher
from close codes already on the wire (`APP_EXITED_CLOSE_CODE` had been sent for a long time with
nothing consuming it). **Latched before the shutdown flag**, because the two are read by different
threads and the reason must never arrive second. Exposed as `punktfunk_connection_end_reason` +
`is_normal()`. Shells fall back to the old wording when there is no verdict (older core, or a close
that raced the read).
- **The Steam `Running` registry hint was an unbounded veto.** Honouring it reset the absence window
every pass, so a flag Steam left set — Steam crashed, was closed first, the game re-parented —
pinned a lease in `running` for the life of the host process. The absence timer now runs
regardless; past `VETO_LIMIT` (30 s) with nothing of the game on the box, the session ends anyway
and logs at WARN. Extracted as a pure `exit_confirmed(gone_for, hint_running)` with tests — the
watch loop polls a live process table and cannot be unit-tested, which is exactly how the
unbounded veto shipped.
- **New `launchreg.rs`: one record per `(client fingerprint, library id)`**, written at launch and
independent of the termination policy. The old fingerprint-keyed reclaim only ran under
`GameOnSessionEnd::Always`, so under the default `Keep` nothing was recorded — and a client retry
re-sent `Hello::launch` verbatim, which the host obeyed unconditionally. Steam/Epic URIs hid it
(the launcher just focuses the running copy) but a `gog:`/`custom:` target genuinely started a
second instance over the same save files. The same retry also minted a fresh `launch_stamp`, so
procscan refused to adopt a game older than 2 s and **a reconnected session lost game-exit
detection for the rest of its life.** Identity now flows backwards from the watcher, which
publishes the concrete `ProcRef`s it adopted; liveness is `Scanner::alive` over that recorded set,
re-verified by `(pid, start)`. Tradeoffs: a `custom:` command with no detection hints stays
`Unknown` forever (trading exit detection for not double-spawning), and `IN_FLIGHT_WINDOW` is a
fixed 90 s, deliberately not `disconnect_grace_seconds`.
- **A launcher entry is `LeaseKind::Untracked` unconditionally**, checked ahead of
`nested`/`child`/`spec`. Its lifetime previously depended on invisible state: launcher not running
→ live child → `Child` lease → quitting the launcher ended the session; launcher already running →
command forwards and exits inside `SHIM_WINDOW``Untracked` → session persists. Steam Big Picture
is a *mode*, not a process (and on a Deck it is always running); Heroic is single-instance
Electron. The real trap was the GameStream path, whose `GsApp` intermediate silently dropped the
field.
### Library and plugins
- **Store claims keep identity across the scanner-to-plugin handover.** `library.json` gains a v2
`{entries, claims}` shape that reads the old bare array unchanged and rewrites on first mutation.
`PUT /library/provider/{p}?store=<s>` claims a store; entries then surface as
`<store>:<external_id>` rather than `custom:<id>`, so entry ids, GameStream app ids, client art
caches and Moonlight pins all survive. One provider per store (409 otherwise); while a claim is
held the matching built-in scanner is skipped, so the two never double-list.
- `GET/PUT /library/scanners` is now a **sources** endpoint over the same disabled-set file.
- New entry fields: `role: game|launcher`; launch kinds `steam_ui` (`bigpicture|desktop`),
`launcher_ui` (platform-gated, 400 on invalid) and `xbox`.
- **Plugin kit 0.3.0** adds a `./library` subpath: `defineLibraryPlugin` plus ported total parsers —
text VDF/ACF, the binary `shortcuts.vdf` walker with CRC-32 appid derivation, read-only immutable
SQLite, a registry wrapper that refuses HKCU, path-confinement joins. `GET/PUT /__config` returns
`{schema, value}` and persists raw, so a plugin with settings need not ship an SPA.
### Platform and packaging
- **The client's config writer** falls back to an in-place write when the atomic replace is
unavailable, verifies it by reading the bytes back, and records the last persistence failure
centrally so the UI can surface it. Scratch files are now per-process, closing a real collision
between the five processes that write these stores (shell, session, console UI, CLI, Decky) — one
could previously rename its half-written temp over another's target.
- **Host send pacing** gained a pure, unit-tested budget function: oversized frames are budgeted at
the pacing rate with a 100 ms absolute ceiling rather than compressed into one frame interval.
Steady-state schedules are byte-identical, the legacy behaviour stays reachable via an environment
escape hatch, and the GameStream-compatible path is untouched.
- **Mid-session shard renegotiation is gated off for PyroWave sessions**, which parse the video
stream in windows fixed at session start — re-sizing mid-stream would corrupt the parse. Those
sessions get the next-session clamp only and are excluded from jumbo. The ABR decode-cap latch
likewise does not apply to PyroWave, where adaptive bitrate is open-loop by design.
- **The Deck's Vulkan compatibility layer is built from source**, pinned to the same upstream
revision as the host's own packaged build — bump both together. ~4 MB of app content replaces a
94 MB external extension, and Flathub is no longer needed at install time. ⚠ `subprojects/vkroots`
is a gamescope **submodule** and flatpak-builder clones submodules by default; declaring it again
as an explicit source breaks the build during extraction. `glm` and `stb` are `.wrap` files, not
submodules, and *do* need explicit sources.
- **Build-container images push to an authenticated registry endpoint**, and `:latest` is reconciled
against the content key on every push to main — an out-of-band tag move is detected and repaired
rather than silently inherited.
- **Windows pad drivers** publish their sequence counters with release ordering (the host was already
loading with acquire and pairing with nothing) and serialize the output-ring publish. The
`/dev/uhid` event ABI, previously transcribed into all five Linux gamepad backends, is consolidated
into one module.
### Verification status
Honest about what has and has not been on hardware, because several things in this release have not:
- **Controller audio has never run on a real DualSense.** Its entire verification is unit tests and
compile checks, and its rumble arbitration rests on an explicitly retracted assumption about
whether the voice coils and the rumble motors are the same actuators. The evidence-based 500 ms
idle window is correct either way, but the underlying exclusivity is unsettled. Android's arbiter
is the evidence-based one; the desktop twin and the coil restore on Android's stop path are owed.
Some Android OEM kernels refuse the isochronous claim outright, which degrades to ordinary rumble.
- The **plugin-UI origin split** is validated against a fake console and a fake plugin, not yet in a
real browser.
- The **packaging default-on changes** have had no installer run or package build.
- **No launcher tile has been clicked on a real host** — the first source that would publish one does
not exist yet.
- Desktop-audio, packet-sizing and iPad-pointer work is build-verified only.
-**The FFmpeg-deletion milestone itself has never executed on a GPU.** It was gated on
cross-clippy, 160 tests, a workspace check and an `ffmpeg` count of 0 in the client / 2 in the host.
The software on-glass check, the D3D11 and VAAPI AV1 hardware legs and the field bake were all owed
at merge; later commits closed some of that but not all. The "no FFmpeg" claim is verified by
`cargo tree` and a notices-generator mention count, not by inspecting a shipped binary.
-**`pf-vaadec` has never decoded a frame anywhere** — no VAAPI hardware was reachable. It is the
*first* rung on Linux/Intel and unknown vendors; the evidence filter bars it there in favour of
`pf-vkdecode`, but an explicit pin reaches it.
- **openh264 has never run on glass**; the H.264 software rung is unit-tested only.
- **`native-d3d11va` AV1 is deliberately `verified = false`** — one 25 s 4K60 session, no parity.
- **Split arbitration is opt-in and Linux-wired only**; the Windows arm is built and unit-tested but
not on hardware. The 5120×1440@240 Main10 behaviour flip is explicitly unvalidated and is named as
the first thing to re-measure.
- **Software throughput is unmeasured in general** — the CPU rung does 3539 fps at 4K AV1 against a
60 fps stream, which is why the backlog flush that triggered the rav1d abort happens at all.
- **The Apple mic fix is a proven root cause, not a verified session.** Its own commits call it "a
strong inference plus one proven logic defect rather than a confirmed fix" and close "awaiting the
reporter's on-device confirmation" — which nothing later in the range records. It also leaves a
known gap: nothing reports whether the uplink actually opened, so the HUD still offers a Mute
Microphone button over a session that may be sending nothing.
- **The Windows audio substrate is, by contrast, well-evidenced on hardware** — repeated "measured on
the target box", a live bisect on a fresh endpoint, and a `micpitch` proof reading 440 Hz in →
440 Hz out at exact peak. The one thing not evidenced is a real client speaking through the minted
microphone end to end; the pitch proof is probe-driven.
- **The phone-gyro mirror is not recorded as hardware-verified** — remap matrices are pinned by unit
tests in both languages, but there is no "played a game with a clip-on pad" evidence in the tree.
- **The iOS gamepad-UI pale-palette sweep on glass is still owed**, per its own commit.
-**The CI runner scripts are hand-installed** (`/usr/local/bin/ci-docker-prune.sh`,
`/usr/local/sbin/ci-docker-reclaim.sh`). Merging does not deploy them — both runner hosts need the
files copied out of `scripts/ci/`, and the missing `192.168.1.58:5011` insecure-registry entry on
one host is routed around, not fixed.
Generated
+44 -43
View File
@@ -647,9 +647,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.65"
version = "1.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e228eec9be7c17ccb640b59b36a5cd805ea2a564a4c5e162c2f659fea30d3b96"
checksum = "9066c49992464636f92905fa096ec58baaa4d57ec19a5c096c68d3e25ef3d136"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -994,7 +994,7 @@ dependencies = [
[[package]]
name = "cursor-probe"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"pf-capture",
@@ -1114,7 +1114,7 @@ dependencies = [
[[package]]
name = "display-disturb"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
@@ -1290,9 +1290,9 @@ dependencies = [
[[package]]
name = "ffmpeg-next"
version = "8.1.0"
version = "9.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7c4bd5ab1ac61f29c634df1175d350ded29cf74c3c6d4f7030431a5ae3c7d5d"
checksum = "6380599799e175191eb7ffe82c97f36a2a90a36cbc54c738a903e5287d7f516a"
dependencies = [
"bitflags 2.13.0",
"ffmpeg-sys-next",
@@ -1301,9 +1301,9 @@ dependencies = [
[[package]]
name = "ffmpeg-sys-next"
version = "8.1.0"
version = "9.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a314bc0e022a33a99567ed4bd2576bd58ffd8fcff7891c29194cfecc26a62547"
checksum = "9b939bf79dd5949412a4b81cfe21a07f48ea21b47fcbb5f57816c8c2de5ae30b"
dependencies = [
"bindgen",
"cc",
@@ -1341,9 +1341,9 @@ dependencies = [
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
checksum = "26b73573e6edcd2af0cdf47bd6cb58f0b3839491263c314eaad1ccf24430e1de"
[[package]]
name = "fixedbitset"
@@ -2358,7 +2358,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.24.0"
version = "0.25.0"
[[package]]
name = "lazy_static"
@@ -2463,7 +2463,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"bindgen",
"cmake",
@@ -2498,7 +2498,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"punktfunk-core",
]
@@ -2988,7 +2988,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-bitstream"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"cros-codecs",
"tracing",
@@ -2996,7 +2996,7 @@ dependencies = [
[[package]]
name = "pf-capture"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3017,7 +3017,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ash",
@@ -3051,7 +3051,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3069,7 +3069,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ash",
@@ -3090,7 +3090,7 @@ dependencies = [
[[package]]
name = "pf-dxvadec"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"cros-codecs",
"pf-bitstream",
@@ -3100,7 +3100,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ash",
@@ -3124,7 +3124,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"libc",
@@ -3136,7 +3136,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"pf-host-config",
@@ -3150,11 +3150,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.24.0"
version = "0.25.0"
[[package]]
name = "pf-inject"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3183,19 +3183,20 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ash",
"async-channel",
"pf-client-core",
"pf-vkdecode",
"punktfunk-core",
"sdl3",
"tracing",
@@ -3204,7 +3205,7 @@ dependencies = [
[[package]]
name = "pf-update"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"serde",
"serde_json",
@@ -3212,7 +3213,7 @@ dependencies = [
[[package]]
name = "pf-update-check"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"base64",
@@ -3224,7 +3225,7 @@ dependencies = [
[[package]]
name = "pf-vaadec"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"cros-codecs",
"pf-bitstream",
@@ -3233,7 +3234,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3266,7 +3267,7 @@ dependencies = [
[[package]]
name = "pf-vkdecode"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"ash",
"cros-codecs",
@@ -3277,7 +3278,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"pf-paths",
@@ -3289,7 +3290,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ash",
@@ -3512,7 +3513,7 @@ dependencies = [
[[package]]
name = "punktfunk-cli"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"pf-client-core",
"punktfunk-core",
@@ -3523,7 +3524,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"android_logger",
"jni",
@@ -3541,7 +3542,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3558,7 +3559,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3573,7 +3574,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"async-channel",
"mdns-sd",
@@ -3592,7 +3593,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -3624,7 +3625,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"aes",
"aes-gcm",
@@ -3709,7 +3710,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3723,7 +3724,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"anyhow",
"ksni",
@@ -3746,7 +3747,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.24.0"
version = "0.25.0"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -57,7 +57,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.24.0"
version = "0.25.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+8 -8
View File
@@ -83,7 +83,7 @@ MANIFEST (crate version — SPDX license — source)
cast 0.3.0 — MIT OR Apache-2.0 — https://github.com/japaric/cast.rs
cbc 0.1.2 — MIT OR Apache-2.0 — https://github.com/RustCrypto/block-modes
cbindgen 0.29.4 — MPL-2.0 — https://github.com/mozilla/cbindgen
cc 1.2.65 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cc 1.4.1 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cesu8 1.1.0 — Apache-2.0/MIT — https://github.com/emk/cesu8-rs
cexpr 0.6.0 — Apache-2.0/MIT — https://github.com/jethrogb/rust-cexpr
cfg-expr 0.20.8 — MIT OR Apache-2.0 — https://github.com/EmbarkStudios/cfg-expr
@@ -148,12 +148,12 @@ MANIFEST (crate version — SPDX license — source)
fastbloom 0.14.1 — MIT OR Apache-2.0 — https://github.com/tomtomwombat/fastbloom/
fastrand 2.4.1 — Apache-2.0 OR MIT — https://github.com/smol-rs/fastrand
fdeflate 0.3.7 — MIT OR Apache-2.0 — https://github.com/image-rs/fdeflate
ffmpeg-next 8.1.0 — WTFPL — https://github.com/zmwangx/rust-ffmpeg
ffmpeg-sys-next 8.1.0 — WTFPL — https://github.com/zmwangx/rust-ffmpeg-sys
ffmpeg-next 9.0.0 — WTFPL — https://github.com/zmwangx/rust-ffmpeg
ffmpeg-sys-next 9.0.0 — WTFPL — https://github.com/zmwangx/rust-ffmpeg-sys
fiat-crypto 0.2.9 — MIT OR Apache-2.0 OR BSD-1-Clause — https://github.com/mit-plv/fiat-crypto
field-offset 0.3.6 — MIT OR Apache-2.0 — https://github.com/Diggsey/rust-field-offset
filetime 0.2.29 — MIT/Apache-2.0 — https://github.com/alexcrichton/filetime
find-msvc-tools 0.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
find-msvc-tools 0.1.10 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
fixedbitset 0.5.7 — MIT OR Apache-2.0 — https://github.com/petgraph/fixedbitset
flate2 1.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/flate2-rs
flume 0.12.0 — Apache-2.0/MIT — https://github.com/zesterer/flume
@@ -628,7 +628,7 @@ Crates whose package did not embed a license file (SPDX + source only)
atomig-macro 0.4.0 — MIT/Apache-2.0 — https://github.com/LukasKalbertodt/atomig/
cookie-factory 0.3.3 — MIT — https://github.com/rust-bakery/cookie-factory
defmt-parser 1.0.0 — MIT OR Apache-2.0 — https://github.com/knurling-rs/defmt
ffmpeg-sys-next 8.1.0 — WTFPL — https://github.com/zmwangx/rust-ffmpeg-sys
ffmpeg-sys-next 9.0.0 — WTFPL — https://github.com/zmwangx/rust-ffmpeg-sys
jni-sys-macros 0.4.1 — MIT OR Apache-2.0 — https://github.com/jni-rs/jni-sys
ndk 0.9.0 — MIT OR Apache-2.0 — https://github.com/rust-mobile/ndk
ndk-sys 0.6.0+11769913 — MIT OR Apache-2.0 — https://github.com/rust-mobile/ndk
@@ -2266,7 +2266,7 @@ SOFTWARE.
----------------------------------------------------------------------------
The following license (LICENSE-APACHE) applies to: asn1-rs 0.6.2, asn1-rs-derive 0.5.1, assert_matches 1.5.0, async-channel 2.5.0, async-executor 1.14.0, async-io 2.6.0, async-lock 3.4.2, async-process 2.5.0, async-recursion 1.1.1, async-signal 0.2.14, async-task 4.7.1, atomic-waker 1.1.2, autocfg 1.5.1, base64 0.22.1, bitflags 1.3.2, bitflags 2.13.0, blocking 1.6.2, bumpalo 3.20.3, cast 0.3.0, cc 1.2.65, cexpr 0.6.0, cfg-if 1.0.4, cmake 0.1.58, concurrent-queue 2.5.0, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, der-parser 9.0.0, displaydoc 0.2.6, either 1.16.0, equivalent 1.0.2, errno 0.3.14, event-listener 5.4.1, event-listener-strategy 0.5.4, fastrand 2.4.1, filetime 0.2.29, find-msvc-tools 0.1.9, fixedbitset 0.5.7, flate2 1.1.9, fnv 1.0.7, form_urlencoded 1.2.2, fs-err 3.3.0, futures-lite 2.6.1, gethostname 1.1.0, gif 0.14.2, glob 0.3.3, hashbrown 0.16.1, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, httparse 1.10.1, idna 1.1.0, idna_adapter 1.2.2, indexmap 2.14.0, itertools 0.10.5, itertools 0.13.0, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, khronos-egl 6.0.0, lazy_static 1.5.0, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, mime 0.3.17, nasm-rs 0.3.2, num-bigint 0.4.6, num-bigint-dig 0.8.6, num-derive 0.4.2, num-integer 0.1.46, num-iter 0.1.45, num-traits 0.2.19, num_cpus 1.17.0, oid-registry 0.7.1, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, ordered-stream 0.2.0, parking 2.2.1, parking_lot 0.12.5, parking_lot_core 0.9.12, percent-encoding 2.3.2, piper 0.2.5, pkg-config 0.3.33, png 0.18.1, polling 3.11.0, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, roxmltree 0.21.1, rsa 0.9.10, rustc_version 0.4.1, rusticata-macros 4.1.0, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, stable_deref_trait 1.2.1, system-deps 7.0.8, tar 0.4.46, tempfile 3.27.0, thread_local 1.1.9, tinytemplate 1.2.1, unicode-segmentation 1.13.3, unicode-width 0.2.2, url 2.5.8, uuid 1.23.4, vcpkg 0.2.15, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, weezl 0.1.12, wit-bindgen 0.57.1, x509-parser 0.16.0, xattr 1.6.1
The following license (LICENSE-APACHE) applies to: asn1-rs 0.6.2, asn1-rs-derive 0.5.1, assert_matches 1.5.0, async-channel 2.5.0, async-executor 1.14.0, async-io 2.6.0, async-lock 3.4.2, async-process 2.5.0, async-recursion 1.1.1, async-signal 0.2.14, async-task 4.7.1, atomic-waker 1.1.2, autocfg 1.5.1, base64 0.22.1, bitflags 1.3.2, bitflags 2.13.0, blocking 1.6.2, bumpalo 3.20.3, cast 0.3.0, cc 1.4.1, cexpr 0.6.0, cfg-if 1.0.4, cmake 0.1.58, concurrent-queue 2.5.0, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, der-parser 9.0.0, displaydoc 0.2.6, either 1.16.0, equivalent 1.0.2, errno 0.3.14, event-listener 5.4.1, event-listener-strategy 0.5.4, fastrand 2.4.1, filetime 0.2.29, find-msvc-tools 0.1.10, fixedbitset 0.5.7, flate2 1.1.9, fnv 1.0.7, form_urlencoded 1.2.2, fs-err 3.3.0, futures-lite 2.6.1, gethostname 1.1.0, gif 0.14.2, glob 0.3.3, hashbrown 0.16.1, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, httparse 1.10.1, idna 1.1.0, idna_adapter 1.2.2, indexmap 2.14.0, itertools 0.10.5, itertools 0.13.0, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, khronos-egl 6.0.0, lazy_static 1.5.0, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, mime 0.3.17, nasm-rs 0.3.2, num-bigint 0.4.6, num-bigint-dig 0.8.6, num-derive 0.4.2, num-integer 0.1.46, num-iter 0.1.45, num-traits 0.2.19, num_cpus 1.17.0, oid-registry 0.7.1, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, ordered-stream 0.2.0, parking 2.2.1, parking_lot 0.12.5, parking_lot_core 0.9.12, percent-encoding 2.3.2, piper 0.2.5, pkg-config 0.3.33, png 0.18.1, polling 3.11.0, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, roxmltree 0.21.1, rsa 0.9.10, rustc_version 0.4.1, rusticata-macros 4.1.0, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, stable_deref_trait 1.2.1, system-deps 7.0.8, tar 0.4.46, tempfile 3.27.0, thread_local 1.1.9, tinytemplate 1.2.1, unicode-segmentation 1.13.3, unicode-width 0.2.2, url 2.5.8, uuid 1.23.4, vcpkg 0.2.15, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, weezl 0.1.12, wit-bindgen 0.57.1, x509-parser 0.16.0, xattr 1.6.1
----------------------------------------------------------------------------
Apache License
Version 2.0, January 2004
@@ -4011,7 +4011,7 @@ Exhibit B - "Incompatible With Secondary Licenses" Notice
----------------------------------------------------------------------------
The following license (LICENSE-MIT) applies to: cc 1.2.65, cfg-if 1.0.4, cmake 0.1.58, filetime 0.2.29, find-msvc-tools 0.1.9, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
The following license (LICENSE-MIT) applies to: cc 1.4.1, cfg-if 1.0.4, cmake 0.1.58, filetime 0.2.29, find-msvc-tools 0.1.10, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
----------------------------------------------------------------------------
Copyright (c) 2014 Alex Crichton
@@ -6183,7 +6183,7 @@ DEALINGS IN THE SOFTWARE.
----------------------------------------------------------------------------
The following license (LICENSE) applies to: ffmpeg-next 8.1.0
The following license (LICENSE) applies to: ffmpeg-next 9.0.0
----------------------------------------------------------------------------
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
Version 2, December 2004
+56
View File
@@ -4045,6 +4045,51 @@
}
}
},
"AudioWiring": {
"type": "object",
"description": "The Windows host's audio wiring verdict — which endpoint carries each role. The names are\nthe endpoints' friendly names as the Sound settings show them (on current hosts the minted\n\"Punktfunk\" instances of Steam's streaming drivers).",
"required": [
"readiness",
"mic_withheld",
"last_resort"
],
"properties": {
"last_resort": {
"type": "boolean",
"description": "The loopback is the known-degraded last resort — desktop audio may be silent until the\nendpoint set changes."
},
"loopback": {
"type": [
"string",
"null"
],
"description": "Friendly name of the desktop-audio loopback source; absent = desktop audio unavailable."
},
"mic": {
"type": [
"string",
"null"
],
"description": "Friendly name of the virtual-mic write target; absent = mic passthrough unavailable."
},
"mic_withheld": {
"type": "boolean",
"description": "The mic was WITHHELD so game audio could keep the only working sink — mic passthrough\nneeds Steam installed (the host mints its own microphone) or a virtual cable."
},
"narrowing": {
"type": [
"string",
"null"
],
"description": "Why the chosen loopback endpoint NARROWS the desktop mix (rate/channels), when it does."
},
"readiness": {
"type": "string",
"description": "`full` | `audio_only` | `mic_only` | `none` — whether desktop audio and mic passthrough\neach have an endpoint at all.",
"example": "full"
}
}
},
"AvailableCompositor": {
"type": "object",
"description": "A compositor backend the host can drive a virtual output on, and whether it's usable now.",
@@ -6805,6 +6850,17 @@
"description": "Number of live streaming sessions across BOTH planes (GameStream + native punktfunk/1). The\nnative server admits concurrent sessions, so this can exceed 1; `session`/`stream` below\ndescribe a single representative session for the detail card.",
"minimum": 0
},
"audio": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/AudioWiring",
"description": "The audio wiring verdict (Windows hosts; absent on other platforms and before the first\nwiring pass). Present even while idle — the wiring exists for the host's lifetime."
}
]
},
"audio_streaming": {
"type": "boolean",
"description": "True while the audio stream thread is running."
+10 -1
View File
@@ -9,7 +9,16 @@
# from the last image rebuild instead of a fresh -Syu per run. That is the same staleness
# the gamescope cache already embraces ("a stale binary against newer system libs is the
# same risk the distro's own package carries between rebuilds"), and any ci/ edit — or
# bumping the date in this line (refreshed: 2026-07-29) — re-keys and re-snapshots it.
# bumping the date in this line (refreshed: 2026-08-08) — re-keys and re-snapshots it.
#
# ⚠ That staleness has a sharp edge, and 2026-08-08 is why the date above moved: this snapshot is
# what decides which FFmpeg the HOST links, and arch.yml deliberately runs no -Syu, so the builder
# stayed frozen on ffmpeg 8 (libavcodec 62) even after Arch shipped 2:9.0-5 (libavcodec 63) to
# every user. A canary built from the old snapshot therefore CANNOT satisfy the soname dep that
# packaging/arch/PKGBUILD now derives from the link (libavcodec.so=62-64 against a box that has
# 63-64), so it would simply refuse to install rather than start. Re-keying this image is the step
# that makes the ffmpeg-9 bump actually reach the package — a Cargo.toml bump alone does nothing
# here. Whenever Arch moves to an FFmpeg major, bump the date in the same commit.
FROM docker.io/library/archlinux:base-devel
# One transaction: the main build/runtime deps (first list) + the gamescope companion's
+8
View File
@@ -45,6 +45,14 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
# Sourced from the official FFmpeg GitHub mirror by release tag, NOT ffmpeg.org: the CI build network
# can't reach ffmpeg.org (curl times out) but reaches github.com fine. The `nX.Y` tag pins the version
# (n8.0 -> libavcodec 62); bump it to move FFmpeg. Immutable-tag clone, so no separate checksum needed.
#
# STAYING ON 8.0 THROUGH THE 2026-08-08 FFmpeg-9 BUMP IS DELIBERATE. `ffmpeg-next` moved to 9, but a
# crate major is a CEILING (ffmpeg-sys-next 9 spans libavcodec 56..63), so an 8.0 tree still compiles
# — and this .deb is the one package with NO exposure to the soname break that motivated the bump: it
# BUNDLES these libs into /usr/lib/punktfunk-host behind an rpath and strips the libav* sonames from
# its Depends, so nothing the user's apt does can move them underneath it. Bumping this tag would
# re-qualify the encode stack for every Ubuntu user and buy none of them anything, so it is its own
# change — and it drags NVHDR_TAG and the soname assertion below along with it.
ARG FFMPEG_TAG=n8.0
# nv-codec-headers must MATCH the FFmpeg version: its `master` is NVENC SDK 13, which renamed
# NV_ENC_CLOCK_TIMESTAMP_SET.countingType -> countingTypeLSB and won't compile against FFmpeg 8.0's
+3 -1
View File
@@ -13,7 +13,9 @@ ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
# toolchain + bindgen; nodejs runs the JS actions (checkout/cache); unzip is for the bun installer
build-essential clang libclang-dev pkg-config cmake git curl ca-certificates nodejs unzip \
# ffmpeg-next 8 (system FFmpeg 8 / libavcodec 62 on 26.04)
# ffmpeg-next 9, built against whatever libav* 26.04 ships (FFmpeg 8 / libavcodec 62 today).
# The crate major is a CEILING — ffmpeg-sys-next 9 spans libavcodec 56..63 — so this image does
# not need to move in lockstep with Arch's FFmpeg 9; it just links what the distro has.
libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libavfilter-dev \
libavdevice-dev \
# capture / audio / display stacks (+xkbcommon for the wlr input backend)
@@ -49,7 +49,7 @@ MANIFEST (crate version — SPDX license — source)
bytes 1.12.0 — MIT — https://github.com/tokio-rs/bytes
cast 0.3.0 — MIT OR Apache-2.0 — https://github.com/japaric/cast.rs
cbindgen 0.29.4 — MPL-2.0 — https://github.com/mozilla/cbindgen
cc 1.2.65 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cc 1.4.1 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cesu8 1.1.0 — Apache-2.0/MIT — https://github.com/emk/cesu8-rs
cfg-if 1.0.4 — MIT OR Apache-2.0 — https://github.com/rust-lang/cfg-if
cfg_aliases 0.2.1 — MIT — https://github.com/katharostech/cfg_aliases
@@ -88,7 +88,7 @@ MANIFEST (crate version — SPDX license — source)
fastbloom 0.14.1 — MIT OR Apache-2.0 — https://github.com/tomtomwombat/fastbloom/
fastrand 2.4.1 — Apache-2.0 OR MIT — https://github.com/smol-rs/fastrand
fiat-crypto 0.2.9 — MIT OR Apache-2.0 OR BSD-1-Clause — https://github.com/mit-plv/fiat-crypto
find-msvc-tools 0.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
find-msvc-tools 0.1.10 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
fixedbitset 0.5.7 — MIT OR Apache-2.0 — https://github.com/petgraph/fixedbitset
flume 0.12.0 — Apache-2.0/MIT — https://github.com/zesterer/flume
fnv 1.0.7 — Apache-2.0 / MIT — https://github.com/servo/rust-fnv
@@ -1390,7 +1390,7 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
----------------------------------------------------------------------------
The following license (LICENSE-APACHE) applies to: autocfg 1.5.1, base64 0.22.1, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.2.65, cfg-if 1.0.4, cmake 0.1.58, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, either 1.16.0, equivalent 1.0.2, errno 0.3.14, fastrand 2.4.1, find-msvc-tools 0.1.9, fixedbitset 0.5.7, fnv 1.0.7, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, indexmap 2.14.0, itertools 0.10.5, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking_lot 0.12.5, parking_lot_core 0.9.12, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, tempfile 3.27.0, tinytemplate 1.2.1, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1
The following license (LICENSE-APACHE) applies to: autocfg 1.5.1, base64 0.22.1, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.4.1, cfg-if 1.0.4, cmake 0.1.58, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, either 1.16.0, equivalent 1.0.2, errno 0.3.14, fastrand 2.4.1, find-msvc-tools 0.1.10, fixedbitset 0.5.7, fnv 1.0.7, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, indexmap 2.14.0, itertools 0.10.5, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking_lot 0.12.5, parking_lot_core 0.9.12, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, tempfile 3.27.0, tinytemplate 1.2.1, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1
----------------------------------------------------------------------------
Apache License
Version 2.0, January 2004
@@ -2435,7 +2435,7 @@ Exhibit B - "Incompatible With Secondary Licenses" Notice
----------------------------------------------------------------------------
The following license (LICENSE-MIT) applies to: cc 1.2.65, cfg-if 1.0.4, cmake 0.1.58, find-msvc-tools 0.1.9, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
The following license (LICENSE-MIT) applies to: cc 1.4.1, cfg-if 1.0.4, cmake 0.1.58, find-msvc-tools 0.1.10, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
----------------------------------------------------------------------------
Copyright (c) 2014 Alex Crichton
@@ -57,6 +57,7 @@ import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import dev.chrisbanes.haze.HazeState
import dev.chrisbanes.haze.hazeSource
import io.unom.punktfunk.kit.DeviceGyro
import io.unom.punktfunk.kit.deviceBodyVibrator
import io.unom.punktfunk.kit.security.KnownHost
import io.unom.punktfunk.kit.security.KnownHostStore
@@ -126,6 +127,8 @@ fun GamepadSettingsScreen(
val context = LocalContext.current
// Gates the "Rumble on this phone" row — a TV box has no body vibrator to mirror onto.
val hasBodyVibrator = remember { deviceBodyVibrator(context) != null }
// Gates "Gyro from this phone" the same way — a TV box has no gyroscope to mirror from.
val hasGyroscope = remember { DeviceGyro.available(context) }
// Gates the AV1 codec row the same way the touch settings do (see `codecOptionsFor`).
val av1Capable = remember { io.unom.punktfunk.kit.VideoDecoders.pickDecoder("video/av01") != null }
@@ -159,7 +162,7 @@ fun GamepadSettingsScreen(
// path there is this screen's own Controller-optimized UI toggle, which swaps in the standard
// interface remote-navigably. The strings branch on it.
val tv = remember { isTvDevice(context) }
val allRows = buildSettingsRows(s, hasBodyVibrator, av1Capable, ::update) +
val allRows = buildSettingsRows(s, hasBodyVibrator, hasGyroscope, av1Capable, ::update) +
buildProfileRows(profiles, savedHosts, tv) { pinProfile = it }
// Which section is showing, and where each one's focus was when it was last left — a detour
// into another tab shouldn't lose your place.
@@ -445,12 +448,13 @@ private fun SettingRowView(row: GpRow, focused: Boolean, adjustDir: Int, onClick
}
/** Build the console settings rows from the current [Settings], writing through [update].
* [hasBodyVibrator] gates the "Rumble on this phone" row (absent on TVs); [av1Capable] gates the
* AV1 codec entry (see `codecOptionsFor`). Every row declares its [GpTab]; the screen shows one
* tab at a time. */
* [hasBodyVibrator] gates the "Rumble on this phone" row and [hasGyroscope] the "Gyro from this
* phone" row (both absent on TVs); [av1Capable] gates the AV1 codec entry (see
* `codecOptionsFor`). Every row declares its [GpTab]; the screen shows one tab at a time. */
internal fun buildSettingsRows(
s: Settings,
hasBodyVibrator: Boolean,
hasGyroscope: Boolean,
av1Capable: Boolean,
update: (Settings) -> Unit,
): List<GpRow> {
@@ -598,6 +602,18 @@ internal fun buildSettingsRows(
} else {
null
},
// The rumble mirror's sibling, data flowing the other way — needs a gyroscope to
// mirror FROM, which a TV box lacks.
if (hasGyroscope) {
toggle(
"phoneGyro", GpTab.CONTROLLER, null, "Gyro from this phone",
"When the controller has no gyro of its own, send this phone's motion " +
"sensors as controller 1's — for clip-on pads without one.",
s.gyroOnPhone,
) { update(s.copy(gyroOnPhone = it)) }
} else {
null
},
) + listOf(
// NOT gated on the vibrator (the bug A2 fixed in the touch settings): an SC2 capture has
// nothing to do with this device's motor, and a TV box is where it matters most.
@@ -158,6 +158,16 @@ data class Settings(
* toggle is hidden on devices without a vibrator (TVs), where this would be a silent no-op.
*/
val rumbleOnPhone: Boolean = false,
/**
* Opt-in: use this phone's own gyroscope as controller 1's motion when the forwarded pad has
* none of its own — for clip-on gamepads without an IMU, where the phone body moves with the
* player's hands. The rumble mirror's sibling, data flowing the other way. Off by default;
* read once per session by StreamScreen (it starts a [io.unom.punktfunk.kit.DeviceGyro] only
* when set), and the mirror stands down by itself whenever wire pad 0 is fed by a capture
* link (USB DualSense / SC2 — pads with a real gyro). The toggle is hidden on devices
* without a gyroscope (TVs), where this would be a silent no-op.
*/
val gyroOnPhone: Boolean = false,
/**
* Capture a Steam Controller 2 (wired / Puck dongle over USB, or an already-paired BLE pad)
@@ -300,6 +310,7 @@ class SettingsStore(context: Context) {
smoothBuffer = prefs.getInt(K_SMOOTH_BUFFER, 0),
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
gyroOnPhone = prefs.getBoolean(K_GYRO_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
dsCapture = prefs.getBoolean(K_DS_CAPTURE, true),
padHaptics = prefs.getBoolean(K_PAD_HAPTICS, true),
@@ -340,6 +351,7 @@ class SettingsStore(context: Context) {
.putInt(K_SMOOTH_BUFFER, s.smoothBuffer)
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_GYRO_ON_PHONE, s.gyroOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_DS_CAPTURE, s.dsCapture)
.putBoolean(K_PAD_HAPTICS, s.padHaptics)
@@ -390,6 +402,7 @@ class SettingsStore(context: Context) {
const val K_SMOOTH_BUFFER = "smooth_buffer"
const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_GYRO_ON_PHONE = "gyro_on_phone"
const val K_SC2_CAPTURE = "sc2_capture"
const val K_DS_CAPTURE = "ds_capture"
const val K_PAD_HAPTICS = "pad_haptics"
@@ -77,6 +77,7 @@ import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.core.content.ContextCompat
import io.unom.punktfunk.kit.DeviceGyro
import io.unom.punktfunk.kit.VideoDecoders
import io.unom.punktfunk.kit.deviceBodyVibrator
import io.unom.punktfunk.kit.security.KnownHostStore
@@ -849,7 +850,8 @@ private fun ControllerSettings(s: Settings, update: (Settings) -> Unit, onOpenCo
field = "gamepad",
enabled = s.gamepadForwarding,
caption = "The virtual pad the host creates. Automatic matches your controller; " +
"every connected one is forwarded as its own player.",
"every connected one is forwarded as its own player. An X-Box type has no " +
"gyroscope, so pick a DualSense-class one if you want motion.",
) { g -> update(s.copy(gamepad = g)) }
SettingDropdown(
label = "Guide button",
@@ -888,6 +890,18 @@ private fun ControllerSettings(s: Settings, update: (Settings) -> Unit, onOpenCo
onCheckedChange = { on -> update(s.copy(rumbleOnPhone = on)) },
)
}
// The rumble mirror's sibling, data flowing the other way: needs a gyroscope to
// mirror FROM — a TV box has none, so the row would be a silent no-op there.
val hasGyroscope = remember { DeviceGyro.available(context) }
if (hasGyroscope) {
ToggleRow(
title = "Gyro from this phone",
subtitle = "When the controller has no gyro, send this phone's motion " +
"sensors as controller 1's",
checked = s.gyroOnPhone,
onCheckedChange = { on -> update(s.copy(gyroOnPhone = on)) },
)
}
// NOT gated on the vibrator: SC2 passthrough is a USB/BLE capture that has nothing to do
// with rumbling this device's body, and the gate hid the toggle on exactly the machines
// that most want it — TV boxes, where a Steam Controller 2 is the whole input story.
@@ -18,12 +18,13 @@ import kotlin.math.roundToInt
* The live stats overlay — the unified HUD (`design/stats-unification.md`): headline is
* `capture→displayed` tiled by `host+network` + `decode` + `display` when the platform delivered
* OnFrameRendered render callbacks this window (`dispValid`), falling back to the v1
* `capture→decoded` headline without the `display` term when it didn't. Reads the 33-double
* `capture→decoded` headline without the `display` term when it didn't. Reads the 35-double
* layout from [NativeBridge.nativeVideoStats] (that KDoc is the authoritative index list):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lostTotal, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms, lost, skipped,
* fec, frames, dispValid, displayP50Ms, e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms,
* presentsWindow, presenterActive, feedP50Ms, codecP50Ms, skippedOverflowWindow]`. Every read
* presentsWindow, presenterActive, feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs,
* audioAvOffsetMs]`. Every read
* is length-guarded, so an older native lib simply omits the lines it can't feed.
*
* The shown `display` and `end-to-end` numbers EXCLUDE the OS present floor (see [osFloorMs]) at
@@ -44,7 +45,7 @@ import kotlin.math.roundToInt
* reliability counters (1821) when nonzero.
* - [StatsVerbosity.DETAILED] — also the decoder label, the video-feed descriptor (1013), the
* stage equation (14/15, split into `host + network` when the Phase-2 terms at 16/17 are nonzero),
* and the excluded-floor line when one was measured.
* the excluded-floor line when one was measured, and the audio plane's own latency (33/34).
* [StatsVerbosity.OFF] renders nothing. Older native layouts simply omit the lines they lack (the
* counter line falls back to the cumulative `lostTotal` at index 9 on a pre-window lib).
*/
@@ -178,10 +179,42 @@ internal fun StatsOverlay(
}
}
}
if (detailed) {
audioLine(s)?.let { statLine(it, Color.White) }
}
counterLine(s, lost)?.let { statLine(it, Color(0xFFFFB0B0)) }
}
}
/**
* The audio plane's own latency from the live gauges at 33/34 — `audio buffer 42 ms · a/v +18 ms`,
* the same wording the desktop HUD uses. `buffer` is how much decoded audio is queued ahead of the
* speaker; `a/v` is where that PUTS it relative to the picture (positive = audio behind). `null`
* before any audio has been queued (buffer 0 — audio off, or the ring not yet primed) and on an
* older native layout.
*
* Both terms, not just the depth: a deep ring on a jittery link is correct behaviour — the
* underrun-driven floor earned that buffer — and only the offset distinguishes it from a ring that
* is simply holding audio late. The offset term is dropped at zero, which is both "aligned" and
* "no measurement yet"; the depth alone is still the triage number, and it is the one that did not
* exist at all before (the plane published nothing any surface could render, so a "the audio delay
* is way too high" report had no instrument behind it).
*
* NOT shaved by [osFloorMs], unlike every video figure above. That shave is a reporting policy —
* metrics report what Punktfunk controls — but sound has to reach the ear when the light reaches
* the eye, so the sync loop aligns against the RAW capture→displayed time (see the native
* `DisplayTracker`) and this offset is stated in those same terms. Subtracting the floor here would
* report an alignment the listener is not getting.
*/
private fun audioLine(s: DoubleArray): String? {
if (s.size < 35) return null
val bufferMs = s[33].roundToInt()
if (bufferMs <= 0) return null
val avOffset = s[34].roundToInt()
val avTerm = if (avOffset != 0) " · a/v ${if (avOffset > 0) "+" else ""}$avOffset ms" else ""
return "audio buffer $bufferMs ms$avTerm"
}
/** One monospace HUD line — the shared type ramp so every tier's rows line up. */
@Composable
private fun statLine(text: String, color: Color) {
@@ -67,11 +67,13 @@ import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.kit.DeviceGyro
import io.unom.punktfunk.kit.DsCapture
import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.GamepadRouter
import io.unom.punktfunk.kit.deviceBodyVibrator
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.PadSensors
import io.unom.punktfunk.kit.Sc2Capture
import io.unom.punktfunk.kit.SessionEndReason
import io.unom.punktfunk.kit.VideoDecoders
@@ -137,6 +139,19 @@ fun StreamScreen(session: ActiveSession, onSessionEnded: (SessionEndReason) -> U
micHint = null
}
}
// A captured pad has a gyro this session's virtual controller cannot carry (see
// GamepadRouter.onMotionUnreachable). Shown briefly, then gone: the failure is otherwise
// completely silent — the gyro simply does nothing, which from the couch is indistinguishable
// from a broken sensor — and the fix is a setting, so the notice has to name it.
var motionHint by remember { mutableStateOf(false) }
LaunchedEffect(motionHint) {
if (motionHint) {
// Longer than the mic chord's 1.6 s: that one confirms something the user just did,
// this one explains something they did not, in a sentence they have to read.
delay(6000)
motionHint = false
}
}
// The one place mute is toggled — Compose state + the native flag, always together.
val setMicMuted = { muted: Boolean ->
micMuted = muted
@@ -359,6 +374,9 @@ fun StreamScreen(session: ActiveSession, onSessionEnded: (SessionEndReason) -> U
// Select + Y toggles the mic — the couch reach for the on-screen mute button, which a
// gamepad/TV user has no pointer for. Ignored when no capture is running (there is nothing
// to mute, and claiming otherwise would be the lie the control exists to avoid).
// A captured Sony pad whose motion this session cannot carry. Fires once per pad, at the
// moment it is claimed, on the main thread.
router.onMotionUnreachable = { motionHint = true }
router.onMicChord = {
if (micRunning) {
val next = !micMuted
@@ -455,9 +473,39 @@ fun StreamScreen(session: ActiveSession, onSessionEnded: (SessionEndReason) -> U
router,
deviceVibrator = if (initialSettings.rumbleOnPhone) deviceBodyVibrator(context) else null,
).also { it.start() }
// "Gyro from this phone" (opt-in): this device's IMU speaks for controller 1's motion
// while wire pad 0 is a controller without a gyro of its own — the rumble mirror's
// sibling, data flowing the other way. The mirror gates itself per sample (it stands
// down whenever pad 0's controller has motion of its own — a capture link below, or a
// pad whose own sensors PadSensors is reading), so it composes without coordination here.
val phoneGyro = if (initialSettings.gyroOnPhone && initialSettings.gamepadForwarding) {
DeviceGyro(context, handle, router).also { it.start() }
} else {
null
}
// A Bluetooth controller's OWN gyro, through the platform sensor framework (API 31+):
// a BT DualSense / DS4 / Switch Pro / 8BitDo is an ordinary InputDevice, so none of the
// capture links below ever sees it and its motion used to go nowhere at all. No separate
// setting — this is the pad's own IMU doing what the pad is for, and unlike the USB
// captures it claims nothing; forwarding being off is the only thing that silences it.
val padSensors = if (initialSettings.gamepadForwarding) {
PadSensors(router).also { it.start() }
} else {
null
}
// Free a disconnected controller's rumble/lights bindings promptly (else the open lights
// session leaks until the session ends). The router owns hot-plug; the feedback owns the binds.
router.onSlotClosed = feedback::onDeviceRemoved
// session leaks until the session ends), and take its sensor listeners off with it — the
// same callback also fires when a USB capture below CLAIMS the pad, which is what keeps
// the claimed pad from being fed motion twice. The router owns hot-plug; the feedback owns
// the binds. Assigned before the captures are constructed, so their claims land on it.
router.onSlotClosed = { deviceId ->
feedback.onDeviceRemoved(deviceId)
padSensors?.onSlotClosed(deviceId)
}
// The other edge: a controller that arrives (or first speaks) mid-session gets its sensors
// read too. The pads already connected were swept by PadSensors.start() above — both run
// on the main thread with nothing between them, so no controller falls through the gap.
router.onSlotOpened = { deviceId -> padSensors?.onSlotOpened(deviceId) }
// Steam Controller 2 as-is passthrough (opt-out): capture a wired/Puck USB pad — or an
// already-paired BLE one — and forward its raw reports; the host mirrors a real
// 28DE:1302 that its Steam drives directly, and Steam's rumble/settings writes come back
@@ -587,12 +635,17 @@ fun StreamScreen(session: ActiveSession, onSessionEnded: (SessionEndReason) -> U
feedback.onHidRaw = null
feedback.sink = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
phoneGyro?.stop() // join the sensor thread + park pad 0's rotation at zero, same ordering rule
// After the mirror, so it cannot resume writing pad 0 in the gap when a pad's own
// sensors let go of it; before the router is released, so the parks still find slots.
padSensors?.stop()
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
sc2?.stop() // release the USB/BLE link + free the wire slot (host tears the pad down)
dsUsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
ds?.stop() // rumble-stop on the physical pad + release the USB link + free the wire slot
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.onMicChord = null // same: no mute toggle on buttons released during teardown
router.onMotionUnreachable = null // same: no notice raised by a slot closing at teardown
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
@@ -853,6 +906,11 @@ fun StreamScreen(session: ActiveSession, onSessionEnded: (SessionEndReason) -> U
}
// Chord confirmation (gamepad/TV) — the counterpart to the button changing under a finger.
micHint?.let { MicChordHint(it, Modifier.align(Alignment.TopCenter).padding(top = 16.dp)) }
// Bottom, not top: this can coincide with a mic-chord confirmation or the exit cue, and a
// notice landing on top of one of those would cost the user both.
if (motionHint) {
MotionUnreachableHint(Modifier.align(Alignment.BottomCenter).padding(bottom = 24.dp))
}
}
}
@@ -939,6 +997,28 @@ private fun MicChordHint(text: String, modifier: Modifier = Modifier) {
)
}
/**
* "This pad's gyro can't reach the game" — shown briefly when a captured controller with motion
* meets a session whose virtual pad has no motion plane (the X-Box classes have no gyro in their
* HID contract, so every sample would be decoded and dropped host-side).
*
* It names the setting because that is the whole point: without it the player has a gyro that
* silently does nothing and no way to tell that from a broken sensor. Not a control — the setting
* applies from the next session, so offering to change it here would promise something this stream
* cannot deliver. [GamepadRouter.onMotionUnreachable] raises it.
*/
@Composable
private fun MotionUnreachableHint(modifier: Modifier = Modifier) {
Text(
"Motion won't reach this session — set Controller type to DualSense",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/**
* The "hold to quit" cue shown while the gamepad exit chord (Select + Start + L1 + R1) is held. The
* chord no longer quits on a quick press — the router debounces it on a ~1 s hold — so this confirms
@@ -123,7 +123,9 @@ class GamepadPaletteTest {
*/
@Test
fun everySettingsRowHasATab() {
val rows = buildSettingsRows(Settings(), hasBodyVibrator = true, av1Capable = true) {}
val rows = buildSettingsRows(
Settings(), hasBodyVibrator = true, hasGyroscope = true, av1Capable = true,
) {}
assertTrue(rows.isNotEmpty())
assertEquals(rows.size, rows.map { it.id }.toSet().size)
// Profiles is built separately (from the catalog), so no settings row claims it.
@@ -137,7 +139,9 @@ class GamepadPaletteTest {
@Test
fun backgroundRowStepsTheSharedKey() {
var s = Settings()
fun rows() = buildSettingsRows(s, hasBodyVibrator = false, av1Capable = false) { s = it }
fun rows() = buildSettingsRows(
s, hasBodyVibrator = false, hasGyroscope = false, av1Capable = false,
) { s = it }
fun palette() = rows().first { it.id == "palette" }
assertEquals("violet", s.uiPalette)
@@ -24,6 +24,7 @@ class GamepadSettingsRowsTest {
): List<GpRow> = buildSettingsRows(
Settings(gamepadForwarding = forwarding),
hasBodyVibrator = true,
hasGyroscope = true,
av1Capable = true,
) { sink += it }
@@ -0,0 +1,94 @@
package io.unom.punktfunk
import androidx.activity.ComponentActivity
import androidx.compose.ui.test.junit4.createAndroidComposeRule
import androidx.compose.ui.test.onNodeWithText
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.annotation.Config
/**
* The stats HUD's audio line — `audio buffer N ms · a/v ±N ms`, from the live gauges at indexes
* 33/34 (`design/audio-latency-overhaul.md`).
*
* Worth pinning because the whole point of the overhaul's stats half is that the audio plane became
* OBSERVABLE. Before it, ring depth and A/V offset existed only as a log line, and on a device
* launched by a game launcher that goes to a pipe nobody can read — so the single number that
* identifies a deep ring was unobtainable on the exact device reporting the latency, and a field
* investigation ran to its conclusion without it. A measurement that never reaches a surface is
* indistinguishable from no measurement, which is what this asserts.
*
* `sdk = [36]` for the same reason as the screenshot tests: Robolectric ships android-all jars only
* up to API 36 while the app's compileSdk is 37.
*/
@RunWith(RobolectricTestRunner::class)
@Config(sdk = [36])
class StatsOverlayAudioTest {
@get:Rule
val compose = createAndroidComposeRule<ComponentActivity>()
/**
* A plausible 35-double window with the audio gauges dialled in. Everything before 33 is the
* DETAILED-renderable shape the ShotScenes fixture uses; only the last two matter here.
*/
private fun stats(bufferMs: Double, avOffsetMs: Double, size: Int = 35): DoubleArray {
val full = doubleArrayOf(
238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 2.0,
10.0, 9.0, 16.0, 1.0, 0.9, 0.4, 0.6, 0.3,
2.0, 1.0, 5.0, 238.0,
1.0, 0.5, 1.8, 2.6,
0.2, 0.3, 236.0, 1.0,
0.1, 0.3, 0.0,
bufferMs, avOffsetMs,
)
return full.copyOf(size)
}
private fun show(s: DoubleArray, verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
compose.setContent { StatsOverlay(s, verbosity = verbosity) }
}
@Test
fun detailedShowsDepthAndOffset() {
show(stats(bufferMs = 42.0, avOffsetMs = 18.0))
// Positive = audio playing BEHIND the picture, and the sign is explicit so a glance tells
// which way the loop still has to move.
compose.onNodeWithText("audio buffer 42 ms · a/v +18 ms").assertExists()
}
@Test
fun audioAheadOfThePictureReadsNegative() {
show(stats(bufferMs = 42.0, avOffsetMs = -12.0))
compose.onNodeWithText("audio buffer 42 ms · a/v -12 ms").assertExists()
}
/** Aligned (or not yet measured) drops the offset term; the depth alone is still the triage number. */
@Test
fun alignedShowsDepthAlone() {
show(stats(bufferMs = 42.0, avOffsetMs = 0.0))
compose.onNodeWithText("audio buffer 42 ms").assertExists()
}
/** Nothing queued (audio off, or the ring not yet primed) — the line has nothing to say. */
@Test
fun silentPlaneRendersNoLine() {
show(stats(bufferMs = 0.0, avOffsetMs = 0.0))
compose.onNodeWithText("audio buffer", substring = true).assertDoesNotExist()
}
/** The line is DETAILED-only, like every other per-stage figure. */
@Test
fun normalTierOmitsTheLine() {
show(stats(bufferMs = 42.0, avOffsetMs = 18.0), verbosity = StatsVerbosity.NORMAL)
compose.onNodeWithText("audio buffer", substring = true).assertDoesNotExist()
}
/** An older native lib emits 33 doubles; the overlay must omit the line, not index past the end. */
@Test
fun olderNativeLayoutOmitsTheLine() {
show(stats(bufferMs = 42.0, avOffsetMs = 18.0, size = 33))
compose.onNodeWithText("audio buffer", substring = true).assertDoesNotExist()
}
}
@@ -355,10 +355,12 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// The full 26-double unified layout (design/stats-unification.md): [fps, mbps, e2eP50,
// e2eP95, latValid, skew, w, h, hz, lostTotal, bitDepth, colorPrimaries, colorTransfer,
// chromaFormatIdc, hostNetP50, decodeP50, hostP50, netP50, lost, skipped, fec, frames,
// dispValid, displayP50, e2eDispP50, e2eDispP95].
// The full 35-double unified layout — NativeBridge.nativeVideoStats' KDoc is the
// authoritative index list: [fps, mbps, e2eP50, e2eP95, latValid, skew, w, h, hz,
// lostTotal, bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50,
// decodeP50, hostP50, netP50, lost, skipped, fec, frames, dispValid, displayP50,
// e2eDispP50, e2eDispP95, paceP50, latchP50, presents, presenterActive, feedP50, codecP50,
// skippedOverflow, audioBufferMs, audioAvOffsetMs].
// 10/9/16/1 = a 10-bit BT.2020 PQ (HDR) 4:2:0 feed so the DETAILED HUD renders its
// video-feed line; the display stage is valid (dispValid 1) so the headline is the
// directly-measured capture→displayed pair, less the excluded OS present floor (the 0.3
@@ -376,6 +378,12 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
1.0, 0.5, 1.8, 2.6,
// Timeline-presenter split: pace + latch tile the display term; presents ≈ fps.
0.2, 0.3, 236.0, 1.0,
// The decode term's own split (feed + codec = 0.4), and no overflow — the one
// `skipped` above is benign newest-wins pacing, not a decoder falling behind.
0.1, 0.3, 0.0,
// The audio plane: a 28 ms ring placed 4 ms behind the picture — a converged sync
// loop, i.e. inside the deadband it deliberately leaves alone.
28.0, 4.0,
),
verbosity = verbosity,
decoderLabel = "c2.qti.hevc.decoder · low-latency",
+61 -19
View File
@@ -67,30 +67,37 @@ fun androidSdkDir(): String {
return "${System.getProperty("user.home")}/Library/Android/sdk"
}
// Every cargo-ndk invocation needs the same discovery environment, and they must not drift apart:
// a lint that ran against a different toolchain/sysroot than the build is a lint about a different
// program. Applied by both `registerCargoNdk` (build) and `registerCargoNdkClippy` (lint).
fun Exec.cargoNdkEnvironment() {
val sdk = androidSdkDir()
// A GUI Android Studio launch does not source the login shell, so make cargo, the NDK, and
// cmake (libopus builds via the cmake crate) discoverable explicitly — same as a bare CLI.
val cmakeBin = "$sdk/cmake/3.22.1/bin"
environment(
"PATH",
cargoBin + File.pathSeparator + cmakeBin + File.pathSeparator + System.getenv("PATH"),
)
environment("ANDROID_HOME", sdk)
environment("ANDROID_NDK_HOME", "$sdk/ndk/$ndkVer")
// CMake's built-in Android support (used by the cmake crate for libopus) finds the NDK via
// these, and uses Ninja (bundled next to the SDK cmake) since there's no `make`.
environment("ANDROID_NDK_ROOT", "$sdk/ndk/$ndkVer")
environment("ANDROID_NDK", "$sdk/ndk/$ndkVer")
environment("CMAKE_GENERATOR", "Ninja")
// audiopus_sys picks static-vs-dynamic by HOST not target — force the bundled static libopus
// (pure C) so the android .so links it instead of looking for the host's libopus.so.
environment("LIBOPUS_STATIC", "1")
environment("LIBOPUS_NO_PKG", "1")
}
fun registerCargoNdk(taskName: String, release: Boolean) =
tasks.register<Exec>(taskName) {
group = "rust"
description = "cargo-ndk build of punktfunk-client-android (${if (release) "release" else "debug"})"
workingDir = repoRoot
val sdk = androidSdkDir()
// A GUI Android Studio launch does not source the login shell, so make cargo, the NDK, and
// cmake (libopus builds via the cmake crate) discoverable explicitly — same as a bare CLI.
val cmakeBin = "$sdk/cmake/3.22.1/bin"
environment(
"PATH",
cargoBin + File.pathSeparator + cmakeBin + File.pathSeparator + System.getenv("PATH"),
)
environment("ANDROID_HOME", sdk)
environment("ANDROID_NDK_HOME", "$sdk/ndk/$ndkVer")
// CMake's built-in Android support (used by the cmake crate for libopus) finds the NDK via
// these, and uses Ninja (bundled next to the SDK cmake) since there's no `make`.
environment("ANDROID_NDK_ROOT", "$sdk/ndk/$ndkVer")
environment("ANDROID_NDK", "$sdk/ndk/$ndkVer")
environment("CMAKE_GENERATOR", "Ninja")
// audiopus_sys picks static-vs-dynamic by HOST not target — force the bundled static libopus
// (pure C) so the android .so links it instead of looking for the host's libopus.so.
environment("LIBOPUS_STATIC", "1")
environment("LIBOPUS_NO_PKG", "1")
cargoNdkEnvironment()
// Resolve cargo by ABSOLUTE path: Gradle's Exec resolves command[0] via the JVM's
// inherited PATH, NOT the environment("PATH", …) set above (that only reaches the spawned
// child). A GUI Android Studio launch (and any daemon it started) has no ~/.cargo/bin on
@@ -113,6 +120,41 @@ fun registerCargoNdk(taskName: String, release: Boolean) =
commandLine(cmd)
}
// ------------------------------------------------------------------------------------------------
// Lint the ANDROID target. `punktfunk-client-android` and every `#[cfg(target_os = "android")]`
// module elsewhere in the workspace were, until this task existed, **completely unlinted**: ci.yml
// runs `cargo clippy --workspace` on the HOST, where all of that code is compiled out, and this
// workflow only ever ran `build`. The gap was found in 2026-08 with five lints sitting in
// clients/android/native (two of them `unnecessary_cast`, which is exactly the class that decides
// whether a cast is redundant BY POINTER WIDTH).
//
// Both widths are linted, and that is the load-bearing part: arm64-v8a is 64-bit and armeabi-v7a is
// 32-bit, so a cast that is redundant on one can be required on the other. Linting only the primary
// ABI would license "fixes" that break the 32-bit build — the shipping ABI for the many 32-bit
// Google TV / Android TV boxes this client targets. x86_64 is deliberately omitted: it is
// emulator-only and shares its pointer width with arm64, so it costs a third of the job's lint time
// for no signal these two do not already carry.
//
// `--all-targets` for the same reason ci.yml spells it out: without it the `#[cfg(test)]` modules
// are never compiled, and un-compiled test code drifts silently.
fun registerCargoNdkClippy(taskName: String) =
tasks.register<Exec>(taskName) {
group = "verification"
description = "clippy (deny warnings) for punktfunk-client-android on both Android widths"
workingDir = repoRoot
cargoNdkEnvironment()
commandLine(
// Absolute cargo path for the same reason as the build task above.
"$cargoBin/cargo", "ndk",
"-t", "arm64-v8a", "-t", "armeabi-v7a",
"--platform", "28",
"clippy", "-p", "punktfunk-client-android", "--all-targets",
"--", "-D", "warnings",
)
}
val cargoNdkClippy = registerCargoNdkClippy("cargoNdkClippy")
// Post-link floor check: every undefined symbol in the built .so must exist in the API-28 stubs,
// else System.loadLibrary fails on devices at the minSdk floor (see the script header for the
// 0.9.0 incident this guards against). Runs right after its cargo-ndk task; the APK build depends
@@ -0,0 +1,170 @@
package io.unom.punktfunk.kit
import android.content.Context
import android.hardware.Sensor
import android.hardware.SensorEvent
import android.hardware.SensorEventListener
import android.hardware.SensorManager
import android.os.Build
import android.os.Handler
import android.os.HandlerThread
import android.view.Display
import android.view.Surface
import android.view.WindowManager
/**
* The opt-in phone-gyro mirror ("Gyro from this phone", off by default): while wire pad 0 is a
* controller with no motion source of its own, THIS device's IMU speaks for it on the rich-input
* motion plane — for clip-on and third-party pads that ship without a gyro, where the phone body
* is rigidly attached to (or simply is) the thing in the player's hands. [GamepadFeedback]'s
* rumble-on-phone mirror with the data flowing the other way.
*
* On Android the only motion sources are the capture links (USB DualSense / SC2 — pads with a
* real IMU, claimed as [GamepadRouter.ExternalPad]s), so the stand-down rule is exactly
* [GamepadRouter.padHasOwnMotion]: when a capture link holds pad 0, the mirror sends nothing —
* two motion writers on one wire pad would fight. It also sends nothing while pad 0 has no slot
* at all (motion never creates a host pad; a controller must have arrived first).
*
* Two properties this class enforces itself:
* - samples ride a dedicated [HandlerThread] with batching disabled (`maxReportLatencyUs = 0`) —
* sensor batching would trade the exact latency gyro aim exists to avoid;
* - a stand-down edge (capture link claims pad 0, or [stop]) sends ONE zero-gyro sample, so the
* host's virtual pad never keeps integrating an angular velocity this device stopped
* producing (the gyro-sweep "stale angular velocity re-sent forever" failure mode).
*
* Units are the wire contract, converted by [Gamepad.motionGyroWire] / [Gamepad.motionAccelWire] —
* the same two functions [PadSensors] uses, so a scale this client ever has to correct is corrected
* once for every sender rather than once per sender that someone remembers. The one thing the
* phone adds is a frame remap: sensors report in the device's natural-portrait frame, while
* the wire wants the controller frame the player sees (x right, y up, z out of the screen), so
* each sample is rotated by the current display rotation — a phone clipped landscape must yaw
* when the player yaws, not roll. The matrix is derived and pinned by `DeviceGyroTest`;
* correctable in one place if on-glass says otherwise.
*/
class DeviceGyro(
context: Context,
private val handle: Long,
private val router: GamepadRouter,
) : SensorEventListener {
private val sensorManager: SensorManager? =
context.getSystemService(SensorManager::class.java)
/** For the live rotation; null on contexts without a display association (then portrait). */
private val display: Display? = runCatching {
if (Build.VERSION.SDK_INT >= 30) {
context.display
} else {
@Suppress("DEPRECATION")
context.getSystemService(WindowManager::class.java)?.defaultDisplay
}
}.getOrNull()
private val thread = HandlerThread("pf-phone-gyro")
/** Latest converted accel, paired with each gyro send (the wire fuses both per sample). */
private val lastAccel = intArrayOf(0, Gamepad.MOTION_ACCEL_LSB_PER_G, 0)
/** Whether the last gyro event actually went to pad 0 — the stand-down zero-send edge. */
private var wasWriting = false
/** Register the listeners; a device without a gyroscope makes this a no-op. */
fun start() {
val sm = sensorManager ?: return
val gyro = sm.getDefaultSensor(Sensor.TYPE_GYROSCOPE) ?: return
thread.start()
val h = Handler(thread.looper)
// ~200 Hz requested (the framework clamps to what the hardware offers), zero report
// latency: batching is poison for gyro aim.
sm.registerListener(this, gyro, SAMPLING_PERIOD_US, 0, h)
sm.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)?.let {
sm.registerListener(this, it, SAMPLING_PERIOD_US, 0, h)
}
}
/**
* Unregister and join the sensor thread, then park the host pad's rotation at zero if this
* mirror was the live writer. Call BEFORE the router is released / the handle freed —
* teardown-ordered like the feedback threads.
*/
fun stop() {
sensorManager?.unregisterListener(this)
thread.quitSafely()
runCatching { thread.join() }
if (wasWriting) {
wasWriting = false
sendZero()
}
}
override fun onSensorChanged(event: SensorEvent) {
val rotation = display?.rotation ?: Surface.ROTATION_0
when (event.sensor.type) {
Sensor.TYPE_ACCELEROMETER -> {
val v = remap(rotation, event.values[0], event.values[1], event.values[2])
for (i in 0..2) lastAccel[i] = Gamepad.motionAccelWire(v[i])
}
Sensor.TYPE_GYROSCOPE -> {
// The write gate, per sample: pad 0 must exist (motion never creates a pad)
// and must not be a capture link's (its own IMU is streaming).
val write = router.padPresent(0) && !router.padHasOwnMotion(0)
if (!write) {
// Stand-down edge: never leave the last angular velocity latched host-side.
if (wasWriting) {
wasWriting = false
sendZero()
}
return
}
wasWriting = true
val v = remap(rotation, event.values[0], event.values[1], event.values[2])
NativeBridge.nativeSendPadMotion(
handle, 0,
Gamepad.motionGyroWire(v[0]),
Gamepad.motionGyroWire(v[1]),
Gamepad.motionGyroWire(v[2]),
lastAccel[0], lastAccel[1], lastAccel[2],
)
}
}
}
override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) {}
/** Zero rotation, last-known accel — "at rest", not free-fall. */
private fun sendZero() {
NativeBridge.nativeSendPadMotion(
handle, 0, 0, 0, 0, lastAccel[0], lastAccel[1], lastAccel[2],
)
}
companion object {
/** Whether this device can source motion at all — gates the settings rows (a TV box
* without an IMU would make the toggle a silent no-op, the rumble mirror's rule). */
fun available(context: Context): Boolean =
context.getSystemService(SensorManager::class.java)
?.getDefaultSensor(Sensor.TYPE_GYROSCOPE) != null
/**
* ~200 Hz — between the sensor's usual FASTEST (~250-500 Hz) and GAME (~50 Hz), and also
* the ceiling the framework grants an app without `HIGH_SAMPLING_RATE_SENSORS` (API 31+),
* so asking for more would only be silently capped. Shared with [PadSensors].
*/
internal const val SAMPLING_PERIOD_US = 5000
/**
* Rotate one device-frame vector (rotation rate or acceleration — both transform the
* same way under an in-plane rotation) into the controller frame for [rotation]
* ([Surface].ROTATION_*). Sensors report in the natural-portrait frame (+x right edge,
* +y top, +z out of the screen); the controller frame keeps +z (the screen always faces
* the player) and rotates x/y to mean "player's right" and "player's up". ROTATION_90 =
* the device physically turned counter-clockwise, top to the player's LEFT.
*/
fun remap(rotation: Int, x: Float, y: Float, z: Float): FloatArray = when (rotation) {
Surface.ROTATION_90 -> floatArrayOf(-y, x, z) // top left: right = bottom, up = +x
Surface.ROTATION_270 -> floatArrayOf(y, -x, z) // top right: right = top, up = x
Surface.ROTATION_180 -> floatArrayOf(-x, -y, z)
else -> floatArrayOf(x, y, z)
}
}
}
@@ -22,9 +22,11 @@ import android.view.InputDevice
*
* Input: parse ([DsDevice.parseState]) → typed mirror on an [GamepadRouter.ExternalPad] (buttons
* diffed, axes on-change — the exit chord participates like any pad) + the rich plane (touch
* normalized to the wire's 0..65535 screen space on-change; motion forwarded per report in raw
* device units, the wire's contract). The wire slot is claimed when the capture engages, with the
* first parsed report as the fallback for a claim that found no free index, and freed on
* normalized to the wire's 0..65535 screen space on-change; motion forwarded per report, rescaled
* into the wire's units by this pad's own calibration — read once per claim, off the claiming
* thread, with the nominal scaling standing in for the millisecond that read is in flight rather
* than the UI waiting on a control transfer). The wire slot is claimed when the capture engages,
* with the first parsed report as the fallback for a claim that found no free index, and freed on
* unplug/[stop], so indices never leak.
*
* Feedback: implements [GamepadFeedback.PadFeedbackSink] — rumble / trigger / lightbar / player
@@ -55,6 +57,13 @@ class DsCapture(
@Volatile private var model: DsDevice.Model? = null
@Volatile private var pad: GamepadRouter.ExternalPad? = null
/** This pad's factory motion scale, read once per capture on [calReader] and handed to the
* link thread, which scales nominally until it lands — see [MotionCalHandoff]. */
private val motionCal = MotionCalHandoff()
/** The thread doing the claim-time calibration read, kept for the teardown wait. */
@Volatile private var calReader: Thread? = null
// Typed-mirror diff state (wire units) + rich-plane on-change mirrors. Link thread only.
private val state = DsDevice.State()
private var wireButtons = 0
@@ -124,6 +133,11 @@ class DsCapture(
if (model != null) return false
val m = DsDevice.modelFor(dev.productId) ?: return false
if (!usb.start(dev)) return false
// Before `model`, which is what lets the link thread into the parse at all: opening the
// claim forgets the last pad's calibration, so reports arriving while this pad's own read
// (below, off this thread) is in flight fall back to the nominal scaling rather than to
// another unit's factory numbers.
val claim = motionCal.begin()
model = m
for (id in InputDevice.getDeviceIds()) {
val d = InputDevice.getDevice(id) ?: continue
@@ -135,9 +149,88 @@ class DsCapture(
Log.i(TAG, "Sony pad captured over USB: PID=0x%04x model=%s".format(dev.productId, m))
ensureSlot(m)
onActiveChanged?.invoke(true)
readMotionCalAsync(m, claim)
return true
}
/**
* Start this claim's calibration read, on its own thread.
*
* Off the caller's thread because [startUsb] runs on the main one — stream setup, and the
* USB-permission broadcast — and the read is a blocking EP0 control transfer: a pad that is
* there answers in about a millisecond, but one that is stalling takes the link's whole write
* timeout, and the interface must wait for neither. The pad is live throughout, its motion
* nominally scaled until this lands ([onReport]), so even a pad that never answers costs
* precision rather than the UI or the controller.
*
* One thread per claim, daemon and named, matching how [HidUsbLink] runs its reader; it is
* awaited by [awaitCalRead] before the connection it reads from can be closed.
*/
private fun readMotionCalAsync(m: DsDevice.Model, claim: Int) {
val t = Thread({
// A read that throws would otherwise leave the capture on the nominal scaling with
// nothing in the log to say why — the one outcome that looks identical to a pad whose
// calibration is genuinely nominal. Publish the fallback explicitly, and say so.
val cal = runCatching { readMotionCal(m) }.getOrElse {
Log.w(TAG, "motion calibration read failed — nominal scaling", it)
DsDevice.MotionCal.NOMINAL
}
// Discarded when the claim is already over (unplug, stop, or a re-claim beat us here):
// scaling the NEXT pad by this one's factory numbers would be worse than not reading.
if (!motionCal.publish(claim, cal)) {
Log.i(TAG, "motion calibration arrived after the claim ended — discarded")
}
}, "pf-ds-cal")
calReader = t
t.isDaemon = true
t.start()
}
/**
* Wait for an in-flight calibration read to let go of the USB connection, before a teardown
* closes it.
*
* Not politeness: the read is a control transfer on the very connection [HidUsbLink.stop] is
* about to close, and closing a descriptor with a transfer in flight pulls it out from under
* the kernel — the same rule the pad-audio borrow follows. Bounded, and in every case but a
* pad that has stopped answering the thread is long gone, so this returns immediately. It can
* never deadlock: the reading thread waits on nothing this one holds ([MotionCalHandoff] has
* its own monitor, and the read itself takes no lock).
*/
private fun awaitCalRead() {
val t = calReader ?: return
calReader = null
if (!t.isAlive) return
runCatching { t.join(CAL_JOIN_MS) }
if (t.isAlive) Log.w(TAG, "calibration read still in flight at teardown")
}
/**
* Read this pad's IMU calibration — the feature report that says how many raw counts this
* individual unit puts on a °/s and on a g ([DsDevice.MotionCal]).
*
* Once, at claim time, and nowhere else: the calibration is fixed for the life of the
* connection, so doing it per input report would buy nothing and cost the capture its latency.
* A pad that refuses keeps the nominal scaling rather than losing motion altogether.
*/
private fun readMotionCal(m: DsDevice.Model): DsDevice.MotionCal {
val blob = usb.getReport(HidUsbLink.REPORT_TYPE_FEATURE, m.calReportId, m.calReportLen)
val cal = DsDevice.MotionCal.parse(blob, m.calReportId)
// Worth a line either way: this is the number the owed on-glass check reads back — a pad
// whose blob was read declares its own resolution, the fallback declares the wire's.
if (cal === DsDevice.MotionCal.NOMINAL) {
Log.w(
TAG,
"motion calibration 0x%02x unreadable (%d/%d B) — nominal scaling (%s)".format(
m.calReportId, blob?.size ?: 0, m.calReportLen, cal,
),
)
} else {
Log.i(TAG, "motion calibration 0x%02x: %s".format(m.calReportId, cal))
}
return cal
}
/** Stop the link and free the wire slot (host tears the virtual pad down). Idempotent. */
fun stop() {
// Before anything touches the link: the pad-audio renderer borrows this connection's
@@ -157,6 +250,10 @@ class DsCapture(
resetRichFeedback(m)
}
disarmBackstop()
// End the claim before waiting on it: a calibration that lands after this publishes
// nothing, and then the wait makes sure nothing is still reading the connection below.
motionCal.end()
awaitCalRead()
usb.stop()
val wasActive = model != null
model = null
@@ -168,7 +265,10 @@ class DsCapture(
private fun onReport(report: ByteArray, len: Int) {
val m = model ?: return
if (!DsDevice.parseState(m, report, len, state)) return
// Nominal scaling until this claim's calibration read lands (see MotionCalHandoff): for
// that millisecond the pad behaves as it did before the read existed, which nobody can
// feel — unlike a pad whose buttons wait on a control transfer.
if (!DsDevice.parseState(m, report, len, state, motionCal.effective)) return
// Normally claimed already, at capture time; this is the retry for a capture that engaged
// while every wire index was taken.
val p = pad ?: ensureSlot(m) ?: return // all 16 taken — drop until one frees
@@ -189,7 +289,9 @@ class DsCapture(
@Synchronized
private fun ensureSlot(m: DsDevice.Model): GamepadRouter.ExternalPad? {
pad?.let { return it }
val p = router.openExternal(m.pref) ?: return null
// hasGyro: every pad this link captures is a Sony one with an IMU, and its motion goes out
// on the rich plane — so a session that cannot carry it is worth saying out loud.
val p = router.openExternal(m.pref, hasGyro = true) ?: return null
pad = p
Log.i(TAG, "captured $m → wire pad ${p.index}")
// The wire index exists from here on, and the host addresses pad audio by it.
@@ -279,6 +381,10 @@ class DsCapture(
val wasActive = model != null
model = null
releaseSlot()
// As in stop(): end the claim so a late calibration publishes nothing, then wait for the
// read to let go of the connection the line below closes.
motionCal.end()
awaitCalRead()
// Release the transport too: the link only *signals* the drop, so without this an unplug
// left its connection open, its interfaces claimed and its detach receiver registered.
usb.stop()
@@ -310,8 +416,8 @@ class DsCapture(
/**
* The rich plane: touch contacts normalized to the wire's 0..65535 screen space, forwarded
* on change per slot; motion forwarded every report (raw device units — the wire is a unit
* passthrough into the host's virtual pad, and sensor noise makes per-report dedup pointless).
* on change per slot; motion forwarded every report (already in wire units — the parse applies
* this pad's calibration, and sensor noise makes per-report dedup pointless).
*/
private fun mirrorRich(p: GamepadRouter.ExternalPad, m: DsDevice.Model) {
for (f in 0 until 2) {
@@ -483,5 +589,9 @@ class DsCapture(
/** How soon to retry a rumble stop whose write was rejected. Short: the motors are running
* and the host has already moved on, so nothing else is coming to silence them. */
const val STOP_RETRY_MS = 100L
/** Teardown's budget for an in-flight calibration read. Comfortably past the link's own
* EP0 timeout, so it only ever elapses for a pad that has stopped answering entirely. */
const val CAL_JOIN_MS = 500L
}
}
@@ -1,5 +1,7 @@
package io.unom.punktfunk.kit
import kotlin.math.abs
/**
* Sony DualSense / DualSense Edge / DualShock 4 **USB** protocol constants: the input-report
* parser and the output-report builders the capture link ([DsCapture]) needs. Unlike the SC2's
@@ -28,14 +30,168 @@ object DsDevice {
/**
* One captured model: its `GamepadPref` wire byte (the virtual pad the host builds — matching
* the physical one), its output-report size (the descriptor-declared size the firmware
* expects: DS5 48 = id + 47, Edge 64 = id + 63, DS4 32 = id + 31), and its touchpad extent
* expects: DS5 48 = id + 47, Edge 64 = id + 63, DS4 32 = id + 31), its touchpad extent
* (`dualsense_proto::DS_TOUCH_W/H`, `dualshock4_proto::DS4_TOUCH_*`) for normalizing touches
* onto the wire's 0..65535 space.
* onto the wire's 0..65535 space, and the IMU-calibration feature report it answers
* ([MotionCal]): DS5/Edge `0x05` (id + 40 B), DS4 over USB `0x02` (id + 36 B).
*/
enum class Model(val pref: Int, val outputSize: Int, val touchW: Int, val touchH: Int) {
DUALSENSE(Gamepad.PREF_DUALSENSE, 48, 1920, 1080),
DUALSENSE_EDGE(Gamepad.PREF_DUALSENSEEDGE, 64, 1920, 1080),
DUALSHOCK4(Gamepad.PREF_DUALSHOCK4, 32, 1920, 942),
enum class Model(
val pref: Int,
val outputSize: Int,
val touchW: Int,
val touchH: Int,
val calReportId: Int,
val calReportLen: Int,
) {
DUALSENSE(Gamepad.PREF_DUALSENSE, 48, 1920, 1080, 0x05, 41),
DUALSENSE_EDGE(Gamepad.PREF_DUALSENSEEDGE, 64, 1920, 1080, 0x05, 41),
DUALSHOCK4(Gamepad.PREF_DUALSHOCK4, 32, 1920, 942, 0x02, 37),
}
/**
* One pad's own IMU calibration: the factory scale factors that turn its raw motion counts
* into the wire's fixed units (`punktfunk_core::input::gamepad` — 20 LSB per °/s, 10000 LSB
* per g), read out of the calibration feature report the pad serves on EP0.
*
* **Why the pad's blob and not a constant.** Measured on glass 2026-08-07: a DualSense flat
* and face up arrived as 0.811 g where 1.000 was owed, because this path forwarded the raw
* i16s verbatim. The nominal ×10000/8192 rescale that first closed that gap ([NOMINAL]) still
* leaves that unit's factory bias — about 1 % — on acceleration, and provably cannot fix gyro
* at all: the same still-average showed this pad's gyro calibration is nowhere near identity,
* and a near-identity one would mean 1024 LSB per °/s, i.e. ±32 °/s full scale, which no
* controller has. The scale is per unit; only the pad knows it.
*
* The arithmetic is `hid-playstation`'s, and the host's contract test
* (`crates/pf-inject/tests/motion_contract.rs`, `SonyImuCalibration`) is the same math read
* from the other end — it applies it to the blobs our *virtual* pads declare and asserts they
* land on the wire constants. Per axis: gyro `raw × speed_2x × 20 / (|plus bias| +
* |minus bias|)`, accel `(raw (plus range/2)) × 20000 / range`, where `range = plus
* minus` spans 2 g.
*/
class MotionCal private constructor(
/** Per axis: `speed_2x × 20`, over `|plus bias| + |minus bias|`. */
private val gyroNumer: LongArray,
private val gyroDenom: LongArray,
/** Per axis: the raw count the pad reads at 0 g, and the raw span of 2 g. */
private val accelBias: LongArray,
private val accelRange: LongArray,
) {
/** Raw gyro count on [axis] (0 = pitch, 1 = yaw, 2 = roll) → the wire's 20 LSB per °/s. */
fun gyroToWire(axis: Int, raw: Int): Int =
clampWire(raw.toLong() * gyroNumer[axis] / gyroDenom[axis])
/** Raw acceleration count on [axis] → the wire's 10000 LSB per g, zero point removed. */
fun accelToWire(axis: Int, raw: Int): Int =
clampWire((raw - accelBias[axis]) * ACCEL_NUMER / accelRange[axis])
/**
* The derived resolutions, for the capture's one-line claim log — the number that says
* whether a pad's blob was actually read (a real DualSense declares ≈16 LSB/°·s and ≈8192
* LSB/g; the [NOMINAL] fallback reads back as exactly 20 and 8192).
*/
override fun toString(): String = buildString {
append("gyro ")
for (i in 0 until 3) {
if (i > 0) append('/')
append(gyroDenom[i] * WIRE_GYRO_LSB_PER_DEG_S / gyroNumer[i])
}
append(" LSB/°·s, accel ")
for (i in 0 until 3) {
if (i > 0) append('/')
append(accelRange[i] / 2)
}
append(" LSB/g at ")
append(accelBias.joinToString("/"))
}
/**
* Both conversions are a >1 multiplier on every pad measured so far, so a real ±4 g slam
* or a fast flick near full scale would otherwise wrap the i16 and read as an impossible
* motion in the opposite direction.
*/
private fun clampWire(v: Long): Int = v.coerceIn(-32768L, 32767L).toInt()
companion object {
/** The pads' nominal acceleration resolution — `hid-playstation`'s `DS_ACC_RES_PER_G`. */
private const val RAW_ACCEL_LSB_PER_G = 8192L
/**
* The wire's gyro scale, taken from [Gamepad] rather than restated. These were literal
* `20L` / `10000L` until the sensor path hoisted the same numbers into one place; a
* second copy of a unit constant is precisely the defect this whole program opened
* with, and two of them in one module would be worse than the original.
*
* `val`, not `const val`, only because the widening to Long is not a compile-time
* constant expression. Long here on purpose: the arithmetic below multiplies raw counts
* by the calibration's speed term before dividing, which overflows an Int.
*/
private val WIRE_GYRO_LSB_PER_DEG_S = Gamepad.MOTION_GYRO_LSB_PER_DEG_S.toLong()
/** `MOTION_ACCEL_LSB_PER_G`, doubled — the declared accel range spans 2 g, not 1. */
private val ACCEL_NUMER = 2L * Gamepad.MOTION_ACCEL_LSB_PER_G
/** Bytes the layout below reads; the reports themselves are longer (41 / 37). */
private const val MIN_LEN = 35
/**
* What an unreadable pad gets: gyro straight through and accel on the nominal 8192
* LSB/g. Wrong by that unit's factory bias, and for gyro wrong by however far its
* scale sits from the wire's 20 — but a pad whose calibration cannot be read is far
* better off slightly mis-scaled than silent, so this never zeroes motion.
*/
val NOMINAL = MotionCal(
LongArray(3) { 1 },
LongArray(3) { 1 },
LongArray(3),
LongArray(3) { 2 * RAW_ACCEL_LSB_PER_G },
)
/**
* Parse a calibration feature report ([Model.calReportId]) — all little-endian i16:
* `[0]` report id, `[1..7)` gyro bias (pitch, yaw, roll), `[7..19)` gyro plus/minus
* INTERLEAVED (pitch+, pitch, yaw+, yaw, roll+, roll), `[19..23)` the two speed
* words, `[23..35)` accel plus/minus (x+, x, y+, y, z+, z).
*
* ⚠ Interleaved is the **USB** order. A Bluetooth DualShock 4 groups the three plusses
* before the three minuses and consumers switch layout on the transport — this path is
* USB-only by construction (see the file header), so do not "generalise" it.
*
* Falls back to [NOMINAL] for a failed read (null), a truncated or foreign reply, and
* per axis for a degenerate declaration — a clone or broken pad that declares zeroes
* would otherwise divide by zero (`hid-playstation` guards the same case, for the same
* reason).
*/
fun parse(blob: ByteArray?, reportId: Int): MotionCal {
if (blob == null || blob.size < MIN_LEN) return NOMINAL
if ((blob[0].toInt() and 0xFF) != reportId) return NOMINAL
val w = { o: Int ->
((blob[o + 1].toInt() shl 8) or (blob[o].toInt() and 0xFF)).toShort().toLong()
}
val speed2x = w(19) + w(21)
val gyroNumer = LongArray(3)
val gyroDenom = LongArray(3)
val accelBias = LongArray(3)
val accelRange = LongArray(3)
for (i in 0 until 3) {
val bias = w(1 + 2 * i)
val denom = abs(w(7 + 4 * i) - bias) + abs(w(9 + 4 * i) - bias)
if (speed2x > 0 && denom > 0) {
gyroNumer[i] = speed2x * WIRE_GYRO_LSB_PER_DEG_S
gyroDenom[i] = denom
} else {
gyroNumer[i] = 1 // passthrough, as before any calibration existed
gyroDenom[i] = 1
}
val plus = w(23 + 4 * i)
val range = plus - w(25 + 4 * i)
if (range > 0) {
accelBias[i] = plus - range / 2
accelRange[i] = range
} else {
accelBias[i] = 0 // nominal, as NOMINAL above
accelRange[i] = 2 * RAW_ACCEL_LSB_PER_G
}
}
return MotionCal(gyroNumer, gyroDenom, accelBias, accelRange)
}
}
}
/** The captured [Model] for a USB PID, or null for anything we don't capture. */
@@ -50,8 +206,9 @@ object DsDevice {
* The client-consumed fields of one input report. `buttons` is already the WIRE bitmask
* (`Gamepad.BTN_*`) — the parse maps device bits straight to the wire, the exact inverse of
* the host's `DsState::from_gamepad` (BTN_A ↔ cross, BTN_B ↔ circle, BTN_X ↔ square,
* BTN_Y ↔ triangle; positional, not glyph-order). Gyro/accel stay in raw device units — the
* wire's `Motion` is a unit passthrough into the virtual pad's report. Touch coordinates stay
* BTN_Y ↔ triangle; positional, not glyph-order). Gyro/accel arrive in WIRE units — the wire's
* `Motion` is a unit passthrough into the virtual pad's report, so the pad's raw counts are
* rescaled during the parse by the [MotionCal] handed to [parseState]. Touch coordinates stay
* device-raw here; [DsCapture] normalizes against the model's extent when forwarding.
*/
class State {
@@ -59,8 +216,8 @@ object DsDevice {
var lsX = 0; var lsY = 0 // wire i16, +y = up (device is +y down — inverted in the parse)
var rsX = 0; var rsY = 0
var lt = 0; var rt = 0 // 0..255
val gyro = IntArray(3) // raw i16 units (pitch/yaw/roll)
val accel = IntArray(3)
val gyro = IntArray(3) // wire i16: 20 LSB per °/s (pitch/yaw/roll)
val accel = IntArray(3) // wire i16: 10000 LSB per g
val touchActive = BooleanArray(2)
val touchX = IntArray(2) // raw device coords (0..touchW-1 / 0..touchH-1)
val touchY = IntArray(2)
@@ -108,15 +265,25 @@ object DsDevice {
* short read (the pad also emits `0x09`-family getMAC responses etc. on EP0 — those never hit
* the interrupt endpoint, but be defensive). Motion/touch fields update only when the report
* is long enough to carry them (it always is on glass — 64-byte interrupt transfers).
*
* [cal] is this pad's own motion calibration, read once when the capture claims it; the
* default is the nominal fallback, which is all a caller without a live pad (the tests) can
* have.
*/
fun parseState(model: Model, report: ByteArray, len: Int, out: State): Boolean =
fun parseState(
model: Model,
report: ByteArray,
len: Int,
out: State,
cal: MotionCal = MotionCal.NOMINAL,
): Boolean =
if (model == Model.DUALSHOCK4) {
parseDs4(report, len, out)
parseDs4(report, len, out, cal)
} else {
parseDs5(model, report, len, out)
parseDs5(model, report, len, out, cal)
}
private fun parseDs5(model: Model, r: ByteArray, len: Int, out: State): Boolean {
private fun parseDs5(model: Model, r: ByteArray, len: Int, out: State, cal: MotionCal): Boolean {
if (len < 11 || (r[0].toInt() and 0xFF) != DS5_INPUT_ID) return false
out.lsX = stickX(u8(r, 1))
out.lsY = stickY(u8(r, 2))
@@ -152,8 +319,8 @@ object DsDevice {
}
out.buttons = w
if (len >= 28) {
for (i in 0 until 3) out.gyro[i] = i16(r, 16 + 2 * i)
for (i in 0 until 3) out.accel[i] = i16(r, 22 + 2 * i)
for (i in 0 until 3) out.gyro[i] = cal.gyroToWire(i, i16(r, 16 + 2 * i))
for (i in 0 until 3) out.accel[i] = cal.accelToWire(i, i16(r, 22 + 2 * i))
}
if (len >= 41) {
unpackTouch(r, 33, out, 0)
@@ -162,7 +329,7 @@ object DsDevice {
return true
}
private fun parseDs4(r: ByteArray, len: Int, out: State): Boolean {
private fun parseDs4(r: ByteArray, len: Int, out: State, cal: MotionCal): Boolean {
if (len < 10 || (r[0].toInt() and 0xFF) != DS5_INPUT_ID) return false // DS4 shares id 0x01
out.lsX = stickX(u8(r, 1))
out.lsY = stickY(u8(r, 2))
@@ -188,8 +355,8 @@ object DsDevice {
if (b7 and DS4_TOUCHPAD != 0) w = w or Gamepad.BTN_TOUCHPAD
out.buttons = w
if (len >= 25) {
for (i in 0 until 3) out.gyro[i] = i16(r, 13 + 2 * i)
for (i in 0 until 3) out.accel[i] = i16(r, 19 + 2 * i)
for (i in 0 until 3) out.gyro[i] = cal.gyroToWire(i, i16(r, 13 + 2 * i))
for (i in 0 until 3) out.accel[i] = cal.accelToWire(i, i16(r, 19 + 2 * i))
}
if (len >= 43) {
unpackTouch(r, 35, out, 0)
@@ -3,6 +3,7 @@ package io.unom.punktfunk.kit
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
import kotlin.math.roundToInt
/**
* Android gamepad capture → punktfunk/1 gamepad wire (the `input.rs::gamepad` contract; the host
@@ -54,6 +55,31 @@ object Gamepad {
const val AXIS_LT = 4
const val AXIS_RT = 5
// Motion wire units — must equal punktfunk-core `input.rs::gamepad::MOTION_*`. Every motion
// sender on this client goes through the two converters below, so a scale that ever has to
// change changes in ONE place: the gyro program's first finding was a client sending 40× hot
// because a second copy of the number had drifted.
const val MOTION_GYRO_LSB_PER_DEG_S = 20
const val MOTION_ACCEL_LSB_PER_G = 10_000
/** Standard gravity, `punktfunk-core`'s `G` — the divisor that turns m/s² into g. */
const val GRAVITY = 9.80665f
/** [MOTION_GYRO_LSB_PER_DEG_S] restated for Android's rad/s sensors: 1 rad/s ⇒ ~1145.9 raw. */
const val MOTION_GYRO_LSB_PER_RAD_S = MOTION_GYRO_LSB_PER_DEG_S * 180f / Math.PI.toFloat()
/** One angular-rate component, Android's rad/s → the wire's signed-16 raw units. */
fun motionGyroWire(radPerSec: Float): Int =
(radPerSec * MOTION_GYRO_LSB_PER_RAD_S).roundToInt().coerceIn(-32768, 32767)
/**
* One acceleration component, Android's m/s² → the wire's signed-16 raw units. Android reports
* specific force (the axis pointing up reads +1 g at rest), which is the DualSense report's own
* convention — no sign flip, and a pad lying flat lands on the host's neutral +1 g exactly.
*/
fun motionAccelWire(mPerSecSq: Float): Int =
(mPerSecSq / GRAVITY * MOTION_ACCEL_LSB_PER_G).roundToInt().coerceIn(-32768, 32767)
// GamepadPref wire bytes — must equal punktfunk-core `config.rs::GamepadPref::to_u8`.
const val PREF_AUTO = 0
const val PREF_XBOX360 = 1
@@ -7,6 +7,7 @@ import android.os.Looper
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
import java.util.Collections
import java.util.concurrent.ConcurrentHashMap
/**
@@ -31,7 +32,8 @@ import java.util.concurrent.ConcurrentHashMap
*
* Threading: slot mutation + dispatch run on the main thread (Android input dispatch and the
* InputManager hot-plug callbacks both land there). [deviceForPad] is read from the feedback poll
* threads, so the slot table is a [ConcurrentHashMap].
* threads, [padPresent]/[padHasOwnMotion] from the phone-gyro thread and [deviceMotion] from the
* pad-sensor thread, so the slot table is a [ConcurrentHashMap].
*/
class GamepadRouter(
context: Context,
@@ -69,7 +71,18 @@ class GamepadRouter(
) {
/** One forwarded controller: its stable wire pad index, per-device axis state, and held buttons. */
private class Slot(val index: Int, val mapper: Gamepad.AxisMapper) {
private class Slot(
val index: Int,
val mapper: Gamepad.AxisMapper,
/**
* Whether motion sent for this pad can reach the game at all, asked once at open off the
* kind it declared ([NativeBridge.nativePadMotionReaches]). False means the host built it a
* backend with no motion plane, so [deviceMotion] drops the sample here rather than paying
* to send one the host will decode and discard — at a controller's full sensor rate, for
* the whole session. The capture-link pads carry the same flag on [ExternalPad].
*/
val motionReaches: Boolean = true,
) {
/** Forwarded button bits currently held (Gamepad.BTN_*) — for release-on-close + chord detection. */
var held = 0
@@ -85,12 +98,33 @@ class GamepadRouter(
private val slots = ConcurrentHashMap<Int, Slot>()
/**
* Invoked (main thread) with the deviceId whenever a slot closes — hot-unplug or session teardown.
* `StreamScreen` wires this to `GamepadFeedback.onDeviceRemoved` so a disconnected pad's rumble /
* lights bindings are released promptly instead of leaking until the feedback threads stop.
* deviceIds whose own gyro [PadSensors] is currently reading — see [setDeviceHasSensorMotion].
* Written on the main thread, read from the phone-gyro thread, hence a concurrent set.
*/
private val sensorDevices: MutableSet<Int> =
Collections.newSetFromMap(ConcurrentHashMap<Int, Boolean>())
/**
* Invoked (main thread) with the deviceId whenever a slot closes — hot-unplug, a capture link's
* [releaseDevice] claim, or session teardown. `StreamScreen` wires this to
* `GamepadFeedback.onDeviceRemoved` so a disconnected pad's rumble / lights bindings are
* released promptly instead of leaking until the feedback threads stop, and to
* [PadSensors.onSlotClosed] so the controller's own sensor listeners come off with it.
*/
var onSlotClosed: ((deviceId: Int) -> Unit)? = null
/**
* Invoked (main thread) with the deviceId whenever a slot opens for a REAL controller — the
* hot-plug callback or the first input from a pad the session started without. Not fired for
* [openExternal]: a capture link's pad has no [InputDevice] behind it and streams motion from
* its own IMU already. `StreamScreen` wires this to [PadSensors.onSlotOpened].
*
* Slots opened in `init` (every controller already connected) predate any assignment here, so
* a listener must sweep [forwardedDevices] once when it starts. Both happen on the main thread
* inside one composition block, so nothing can slip between the sweep and the assignment.
*/
var onSlotOpened: ((deviceId: Int) -> Unit)? = null
/**
* Invoked (main thread) when the emergency-exit chord has been HELD for [EXIT_HOLD_MS] — the caller
* leaves the stream. `StreamScreen` wires this to the deliberate-quit exit.
@@ -115,6 +149,17 @@ class GamepadRouter(
*/
var onMicChord: (() -> Unit)? = null
/**
* Invoked (main thread) once per pad when a captured controller WITH a gyro turns out to be in
* a session whose virtual pad has no motion plane — its motion is not being sent, because every
* sample would be decoded and dropped host-side.
*
* It exists because the failure is otherwise completely silent: the gyro just does nothing, and
* from the couch that is indistinguishable from a broken sensor. The fix is the Controller type
* setting, so whatever shows this has to name it. `StreamScreen` wires it to a brief notice.
*/
var onMotionUnreachable: (() -> Unit)? = null
private val mainHandler = Handler(Looper.getMainLooper())
/** The pending exit-chord hold timer, or null when the chord isn't currently armed. */
private var pendingExit: Runnable? = null
@@ -320,13 +365,82 @@ class GamepadRouter(
return null
}
/** Whether ANY live slot currently holds wire pad [pad]. Read from the phone-gyro thread. */
fun padPresent(pad: Int): Boolean = slots.values.any { it.index == pad }
/**
* Whether wire pad [pad]'s motion already comes from the controller's OWN IMU — either a
* capture-link slot ([ExternalPad] — USB DualSense / SC2; synthetic ids are negative
* ([EXTERNAL_ID_BASE]), real [InputDevice] ids positive), or a real controller whose gyro
* [PadSensors] is reading through the platform sensor framework (a Bluetooth DualSense /
* Switch Pro / 8BitDo). The phone-gyro mirror stands down for both: two motion writers on one
* wire pad would fight, and the pad's own IMU is the one attached to the player's hands.
* Read from the phone-gyro thread (both tables are concurrent).
*/
fun padHasOwnMotion(pad: Int): Boolean =
slots.any { (id, slot) -> slot.index == pad && (id < 0 || id in sensorDevices) }
/**
* Declare (or withdraw) that real controller [deviceId] is sourcing its own rotation — see
* [padHasOwnMotion]. Called by [PadSensors] as it registers and unregisters listeners, on the
* main thread; read from the phone-gyro thread, hence the concurrent set. Keyed by device
* rather than by pad index so a controller that changes wire index (a lower one freed up while
* it was captured) carries the fact with it.
*/
fun setDeviceHasSensorMotion(deviceId: Int, has: Boolean) {
if (has) sensorDevices.add(deviceId) else sensorDevices.remove(deviceId)
// This is the first moment we know a Bluetooth pad actually HAS a gyro — `openSlot` only
// knows what kind it declared. So it is the honest place to raise the notice when that
// gyro has nowhere to go, and the only one that cannot nag about a pad that never had one.
if (has && forwarding && slots[deviceId]?.motionReaches == false) {
onMotionUnreachable?.invoke()
}
}
/**
* One motion sample from real controller [deviceId]'s own sensors, on whatever wire index its
* slot currently holds — [ExternalPad.motion] for pads the input stack still owns. Silently
* drops when the slot is gone (unplugged, or claimed by a capture link between the sensor
* callback and here) rather than writing to an index that may already belong to someone else.
* Called from [PadSensors]' sensor thread.
*/
fun deviceMotion(deviceId: Int, gyro: IntArray, accel: IntArray) {
val slot = slots[deviceId] ?: return
if (!forwarding) return
// The same gate the USB capture path takes: a backend with no motion plane decodes every
// sample and discards it, so sending is pure cost. Notified once per pad by
// [setDeviceHasSensorMotion], which is where we first know the controller HAS a gyro to
// lose — a pad without one must not produce a warning about motion.
if (!slot.motionReaches) return
NativeBridge.nativeSendPadMotion(
handle, slot.index,
gyro[0], gyro[1], gyro[2],
accel[0], accel[1], accel[2],
)
}
/** Snapshot of the REAL controllers currently forwarded, as deviceIds — the set [PadSensors]
* sweeps at start for the pads that were already connected when the session opened. */
fun forwardedDevices(): List<Int> = slots.keys.filter { it >= 0 }
/**
* A capture-link pad occupying a wire slot without an Android [InputDevice] — the as-is Steam
* Controller 2 passthrough (USB/BLE claimed directly, invisible to the input stack). Shares
* the real slots' lifecycle: a stable lowest-free index, Arrival-before-input, held-state
* flush + Remove on [close], and full participation in the emergency exit chord.
*/
inner class ExternalPad internal constructor(private val syntheticId: Int, val index: Int) {
inner class ExternalPad internal constructor(
private val syntheticId: Int,
val index: Int,
/**
* Whether this pad's motion can reach the game at all, asked once at open (see
* [NativeBridge.nativePadMotionReaches]). False means the host built this pad a backend
* without a motion plane, so [motion] drops the sample here instead of paying to send one
* the host will decode and discard — at a controller's full report rate, for the whole
* session.
*/
private val motionReaches: Boolean,
) {
// Live lookup instead of a captured reference: after [close] (or a router release) the
// slot is gone from the table and every entry point below degrades to a safe no-op.
private val slot get() = slots[syntheticId]
@@ -357,7 +471,7 @@ class GamepadRouter(
/** One motion sample on the rich plane (gyro pitch/yaw/roll + accel, raw device i16
* units — the host passes them straight into the virtual pad's report). Per report. */
fun motion(gyro: IntArray, accel: IntArray) {
if (slot != null && forwarding) {
if (slot != null && forwarding && motionReaches) {
NativeBridge.nativeSendPadMotion(
handle, index,
gyro[0], gyro[1], gyro[2],
@@ -373,15 +487,26 @@ class GamepadRouter(
/**
* Open a slot for a capture-link pad, declaring [pref] as its kind; null when all 16 wire
* indices are taken. Main thread (like the hot-plug callbacks).
*
* [hasGyro] says whether this link forwards motion on the RICH plane ([ExternalPad.motion]) —
* true for the Sony pads, whose IMU is a headline feature, and false for the Steam Controller 2,
* whose motion rides inside the opaque passthrough report that [ExternalPad.hidReport] carries
* and which nothing here may second-guess. It gates only the notice: a pad that never sends
* motion must not produce a warning about motion.
*/
fun openExternal(pref: Int): ExternalPad? {
fun openExternal(pref: Int, hasGyro: Boolean = false): ExternalPad? {
val index = lowestFreeIndex() ?: return null
// Synthetic ids live below any real InputDevice id (those are positive), so they can't
// collide and InputDevice.getDevice(id) resolves them to null for the feedback path.
val syntheticId = EXTERNAL_ID_BASE - index
if (forwarding) NativeBridge.nativeSendGamepadArrival(handle, pref, index)
// Asked once, here, off the kind this pad just DECLARED — not off the session's resolved
// backend, which under Automatic answers for whichever pad happened to be active at dial
// time. Cheap enough to ask unconditionally; the answer holds for the pad's lifetime.
val motionReaches = NativeBridge.nativePadMotionReaches(handle, pref)
if (forwarding && hasGyro && !motionReaches) onMotionUnreachable?.invoke()
slots[syntheticId] = Slot(index, Gamepad.AxisMapper(handle, index))
return ExternalPad(syntheticId, index)
return ExternalPad(syntheticId, index, motionReaches)
}
/**
@@ -437,8 +562,18 @@ class GamepadRouter(
// to that type (a single global choice — matches the handshake's session-default pref).
val pref = if (setting == Gamepad.PREF_AUTO) Gamepad.prefFor(dev) else setting
if (forwarding) NativeBridge.nativeSendGamepadArrival(handle, pref, index)
val slot = Slot(index, Gamepad.AxisMapper(handle, index))
// Asked here, off the kind this pad just DECLARED — not off the session's resolved backend,
// which under Automatic answers for whichever pad happened to be active at dial time. Held
// for the slot's life; the sensor path reads it on every sample.
val slot = Slot(
index,
Gamepad.AxisMapper(handle, index),
NativeBridge.nativePadMotionReaches(handle, pref),
)
slots[dev.id] = slot
// After the table holds the slot, so a listener that sends on this device the moment it is
// told ([PadSensors]) finds an index to send on rather than dropping its first samples.
onSlotOpened?.invoke(dev.id)
return slot
}
@@ -442,6 +442,42 @@ class HidUsbLink(
return n >= 0
}
/**
* Read one report back OUT of the device HID `GET_REPORT`, the EP0 mirror of [sendReport].
* [type] is [REPORT_TYPE_FEATURE] (or output), [id] the report number, [len] the report's full
* declared size INCLUDING its leading id byte, which a numbered report echoes back in byte 0
* (hidapi framing). Returns what arrived truncated if the device answered short or null
* when the device refuses the request or the link is down.
*
* **Once, at claim time; never per input report.** EP0 is independent of the interrupt
* endpoints (see [sendReport]), so this is safe alongside the reader thread but it BLOCKS the
* calling thread for up to [WRITE_TIMEOUT_MS], and a blocking control transfer in the report
* path would wreck capture latency. The one caller reads a Sony pad's fixed motion calibration
* when the capture engages ([DsCapture]).
*/
fun getReport(type: Int, id: Int, len: Int): ByteArray? {
if (len <= 0) return null
val conn = connection ?: return null
val ifId = (activeClaim ?: claims.firstOrNull())?.iface?.id ?: return null
val buf = ByteArray(len)
val n = runCatching {
conn.controlTransfer(
0xA1, // device→host, class, interface
0x01, // GET_REPORT
(type shl 8) or id,
ifId,
buf,
buf.size,
WRITE_TIMEOUT_MS,
)
}.getOrDefault(-1)
return when {
n >= len -> buf
n > 0 -> buf.copyOf(n)
else -> null
}
}
/**
* Stop the read loop and release the interfaces. Idempotent; does not fire [onClosed].
*
@@ -469,12 +505,13 @@ class HidUsbLink(
device = null
}
private companion object {
const val READ_TIMEOUT_MS = 100L
const val WRITE_TIMEOUT_MS = 250
companion object {
private const val READ_TIMEOUT_MS = 100L
private const val WRITE_TIMEOUT_MS = 250
/** Hard `requestWait` ERRORS (not timeouts) persisting this long = the fd is dead. */
const val ERROR_UNPLUG_MS = 2000L
const val REPORT_TYPE_OUTPUT = 0x02
private const val ERROR_UNPLUG_MS = 2000L
private const val REPORT_TYPE_OUTPUT = 0x02
/** HID feature-report type — public for [getReport] callers ([writeRaw] takes a kind). */
const val REPORT_TYPE_FEATURE = 0x03
}
}
@@ -0,0 +1,63 @@
package io.unom.punktfunk.kit
/**
* The hand-off of one claim's motion calibration, from the thread that reads it off the pad to the
* link thread that scales every input report with it.
*
* [DsCapture] reads a captured Sony pad's calibration feature report **off** the claiming thread
* it is a blocking EP0 control transfer and the claim runs on the UI's thread so the value lands
* a moment after the capture goes live. Reports in that gap are scaled by
* [DsDevice.MotionCal.NOMINAL] and forwarded like any other ([effective]): for about a millisecond
* the pad behaves exactly as it did before the calibration read existed acceleration a little
* short, gyro unscaled which nobody can feel, whereas a pad that ignores its buttons until an
* EP0 read comes back is very obvious.
*
* What the hand-off is actually for is the two things that gap must NOT do, neither of which a
* plain field gives:
*
* - **Fall back to the previous pad's numbers instead of the nominal ones.** Calibration is per
* unit, so the last controller's scale factors are simply wrong for this one more wrong, in
* general, than the nominal constants. [begin] forgets them, which is what makes the gap
* nominal rather than inherited.
* - **Let a read that outlived its claim publish.** An unplug, a [DsCapture.stop] and a fast
* re-claim can all land while a read is in flight; [publish] only accepts a value whose token is
* still the live claim's, so a straggler can never scale a pad it never read.
*
* Thread-safe: claimed and ended by the claiming thread, published by the reading thread, read by
* the link thread.
*/
internal class MotionCalHandoff {
/** Handed out by [begin] and burned by [end] — never reused, so a straggler can't match. */
private var token = 0
@Volatile private var cal: DsDevice.MotionCal? = null
/**
* The calibration to scale the next report with: the live claim's own, or the nominal fallback
* while its read is still in flight. Never null a report is always forwarded, never held
* back waiting for a control transfer.
*/
val effective: DsDevice.MotionCal get() = cal ?: DsDevice.MotionCal.NOMINAL
/** Open a claim: forget the previous pad's calibration, and take this claim's token. */
@Synchronized
fun begin(): Int {
cal = null
return ++token
}
/** End the live claim. Nothing read under an older token can land after this. */
@Synchronized
fun end() {
cal = null
token++
}
/** Publish [value] if [claim] is still the live claim; returns whether it landed. */
@Synchronized
fun publish(claim: Int, value: DsDevice.MotionCal): Boolean {
if (claim != token) return false
cal = value
return true
}
}
@@ -264,12 +264,12 @@ object NativeBridge {
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 33 doubles (unified stats spec, `design/stats-unification.md`):
* Returns 35 doubles (unified stats spec, `design/stats-unification.md`):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
* netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
* e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
* feedP50Ms, codecP50Ms, skippedOverflowWindow]`
* feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs, audioAvOffsetMs]`
* (the flags are 1.0/0.0; indexes 2/3 are the end-to-end capturedecoded headline; 1013
* describe the negotiated video feed bit depth 8/10, CICP primaries/transfer, and the HEVC
* chroma_format_idc 1=4:2:0 / 3=4:4:4; 14/15 are the stage p50s tiling the headline
@@ -285,7 +285,10 @@ object NativeBridge {
* the window's on-glass confirm count, and whether the presenter is active at all; 30/31
* split `decode` (15) the same way `feed` = receivedqueued (hand-off + input-slot wait),
* `codec` = queueddecoded, the decoder's own time; 32 is the parked-AU overflow subset of
* `skipped` (19), i.e. the decoder falling behind rather than benign newest-wins pacing).
* `skipped` (19), i.e. the decoder falling behind rather than benign newest-wins pacing;
* 33/34 are the AUDIO plane the playback ring's live depth in ms and the A/V sync loop's
* smoothed offset in ms, positive meaning audio plays BEHIND the picture. Those two are live
* gauges, not windowed samples, and the offset reads 0 until the loop has a video reference).
* Poll ~1 Hz; each call resets the measurement window.
*/
external fun nativeVideoStats(handle: Long): DoubleArray?
@@ -516,6 +519,23 @@ object NativeBridge {
/** Signal wire pad [pad] (0..15) was unplugged so the host tears its virtual device down. The core stamps the seq + re-sends. */
external fun nativeSendGamepadRemove(handle: Long, pad: Int)
/**
* Whether motion sent for a pad that declared [declaredPref] (the [Gamepad].PREF_* byte passed
* to [nativeSendGamepadArrival]) can actually reach the game, or would be decoded and dropped
* by a host backend without a motion plane the X-Box classes have no gyro in their HID
* contract.
*
* Answered natively, off `punktfunk_core::config::pad_motion_reaches`, rather than
* reconstructed here from the session's requested/resolved prefs. The rule is subtler than it
* looks (the host builds each pad from its OWN declaration and folds what it cannot build, so
* neither the declaration nor the session echo answers it alone) and every way of getting it
* wrong is silent, so it lives in one place with one set of tests.
*
* Ask ONCE when a pad opens, not per sample. `true` when the session handle is dead "don't
* suppress" is the safe answer whenever we cannot tell.
*/
external fun nativePadMotionReaches(handle: Long, declaredPref: Int): Boolean
/**
* One raw HID input report from a client-captured controller (the as-is Steam Controller 2
* passthrough), forwarded verbatim on the rich-input plane. [buf] is a DIRECT ByteBuffer whose
@@ -0,0 +1,247 @@
package io.unom.punktfunk.kit
import android.hardware.Sensor
import android.hardware.SensorEvent
import android.hardware.SensorEventListener
import android.os.Build
import android.os.Handler
import android.os.HandlerThread
import android.util.Log
import android.view.InputDevice
import java.util.concurrent.ConcurrentHashMap
/**
* Motion from a controller the Android input stack owns the Bluetooth pads.
*
* Before this, the only motion sources on Android were the capture links: [DsCapture] (a Sony pad
* claimed over USB, raw HID) and [Sc2Capture] (Steam Controller 2 passthrough). A DualSense, a
* DualShock 4, a Switch Pro or an 8BitDo paired over BLUETOOTH is neither it arrives as an
* ordinary [InputDevice], its buttons and sticks work, and its gyro was silently dead. That is a
* whole class of controller with no motion at all.
*
* Android 12 (API 31) exposes those sensors: [InputDevice.getSensorManager] hands back a
* [android.hardware.SensorManager] scoped to that one controller, carrying the usual
* TYPE_GYROSCOPE / TYPE_ACCELEROMETER. This class registers a listener per forwarded controller
* that has a gyroscope, converts each sample to wire units, and sends it on that pad's wire index
* through [GamepadRouter.deviceMotion]. Below API 31 nothing is registered and the class is inert
* those pads keep working, minus motion, exactly as they did.
*
* It follows [DeviceGyro] (the phone-gyro mirror) wherever the two solve the same problem:
* - samples ride ONE dedicated [HandlerThread] with batching disabled (`maxReportLatencyUs = 0`)
* sensor batching would trade away the exact latency gyro aim exists to avoid, and the main
* thread is where Compose recomposition lives;
* - a feed torn down while its wire pad is still alive parks the rotation at zero first, because
* the host holds motion as STATE and re-emits it in every virtual-pad report: an angular
* velocity left behind reads as a pad rotating forever (the gyro sweep's "stale rate re-sent
* forever" finding).
*
* One writer per pad, three ways:
* 1. A USB capture claims the physical device away from the input stack; [DsCapture.startUsb]
* calls [GamepadRouter.releaseDevice] at claim time, which closes the slot, which fires
* `onSlotClosed`, which lands on [onSlotClosed] here and unregisters. The claim also makes the
* controller's [InputDevice] vanish outright, so even a reopened slot would find nothing to
* register but the explicit teardown is what makes the ordering deterministic instead of a
* race against the platform's own removal callback.
* 2. The phone-gyro mirror stands down: registering flips
* [GamepadRouter.setDeviceHasSensorMotion], [GamepadRouter.padHasOwnMotion] reports it, and
* [DeviceGyro] re-reads that gate on every sample (sending its own zero park on the edge).
* 3. Exactly one feed exists per deviceId [onSlotOpened] is idempotent, and it is the only
* thing that ever constructs one.
*
* Frame: see [gyroToWire] the mapping is straight through, and NOT yet verified on hardware.
*/
class PadSensors(private val router: GamepadRouter) {
/** One controller's live sensor feed: its listener state and the accel it pairs with each
* rotation. Its arrays belong to the sensor thread; [stop] reads them only after the join. */
private inner class Feed(private val deviceId: Int) : SensorEventListener {
/** Latest converted accel, paired with each gyro send (the wire fuses both per sample).
* Starts at the host's neutral 1 g on the up axis, NOT [0,0,0], which is free fall. */
private val accel = intArrayOf(0, Gamepad.MOTION_ACCEL_LSB_PER_G, 0)
private val gyro = IntArray(3)
/** Whether any rotation has gone out on this pad gates the park on teardown, so a pad
* that never sent motion is not handed a sample it did not earn. */
@Volatile
var wroteMotion = false
private set
override fun onSensorChanged(event: SensorEvent) {
when (event.sensor.type) {
Sensor.TYPE_ACCELEROMETER -> accelToWire(event.values, accel)
Sensor.TYPE_GYROSCOPE -> {
gyroToWire(event.values, gyro)
// One line per controller per session, on the first sample that carries both
// planes: it is the cheapest possible version of the frame measurement
// [gyroToWire] asks for. Hold the pad flat and still while a stream starts and
// the accel triple says which slot gravity lands on — the one thing that
// settles whether the straight-through mapping is right.
if (!wroteMotion) {
Log.i(
TAG,
"controller $deviceId first motion sample: " +
"gyro ${gyro.joinToString()} accel ${accel.joinToString()}",
)
}
wroteMotion = true
router.deviceMotion(deviceId, gyro, accel)
}
}
}
override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) {}
/** Zero rotation, last-known accel — "at rest", not free fall. */
fun park() {
gyro.fill(0)
router.deviceMotion(deviceId, gyro, accel)
}
}
/** deviceId live feed. Concurrent: the main thread mutates it while the sensor thread is
* running (hot-plug, a capture link's claim). */
private val feeds = ConcurrentHashMap<Int, Feed>()
private val thread = HandlerThread("pf-pad-sensors")
private var handler: Handler? = null
/**
* Start the sensor thread and attach to every controller the router already forwards the
* pads connected before the session opened, which will never fire a hot-plug callback.
* Everything after that arrives through [onSlotOpened]. Main thread.
*/
fun start() {
if (!supported()) return
thread.start()
handler = Handler(thread.looper)
for (deviceId in router.forwardedDevices()) onSlotOpened(deviceId)
}
/**
* A slot opened for real controller [deviceId] attach if it has a gyroscope of its own.
* Idempotent, and a no-op before [start] or on a platform without the API. Main thread, from
* [GamepadRouter.onSlotOpened].
*/
fun onSlotOpened(deviceId: Int) {
val h = handler ?: return
if (feeds.containsKey(deviceId)) return
// API 31+ only — getSensorManager does not exist below it. Re-checked here rather than
// relying on start()'s gate, so the entry point is safe on its own terms.
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.S) return
val dev = InputDevice.getDevice(deviceId) ?: return
// Declared non-null: a controller with no sensors gets an empty manager, not a null one.
val sm = dev.sensorManager
// A gyroscope is the entry price; the accelerometer alone does not buy a feed. The rotation
// is what gyro aim is for, and an accel-only feed would send gravity while pinning rotation
// at zero on a pad the phone-gyro mirror is otherwise entitled to speak for — precisely the
// two-writers-on-one-pad fight this program has spent its day unpicking. Such a pad stays
// on the mirror's terms instead, where at least the accel agrees with the gyro beside it.
// Nothing found here is not proof the pad has no IMU. A DualSense's motion arrives on its
// own evdev node, and whether InputReader merges that node onto the gamepad InputDevice
// (shared descriptor) or leaves it standing alone is the platform's business, not ours —
// and a standalone one is exactly what GamepadRouter.isForwardable filters out, so this
// would never see it. Android 12's own controller-sensor documentation cites the DualShock
// 4 and DualSense, which says the merge happens; it is not something this code can assert.
// If a Bluetooth Sony pad ever turns up here with no gyroscope, THAT is the thing to check.
val gyroSensor = sm.getDefaultSensor(Sensor.TYPE_GYROSCOPE) ?: return
val feed = Feed(deviceId)
feeds[deviceId] = feed
// ~200 Hz requested, zero report latency: batching is poison for gyro aim, and 200 Hz is
// what the framework grants an app without HIGH_SAMPLING_RATE_SENSORS anyway.
sm.registerListener(feed, gyroSensor, DeviceGyro.SAMPLING_PERIOD_US, 0, h)
sm.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)?.let {
sm.registerListener(feed, it, DeviceGyro.SAMPLING_PERIOD_US, 0, h)
}
// The pad sources its own rotation from here on → the phone-gyro mirror stands down for it.
router.setDeviceHasSensorMotion(deviceId, true)
Log.i(TAG, "controller $deviceId (${dev.name}) has a gyro — forwarding its motion")
}
/**
* The slot for [deviceId] closed unplug, session teardown, or a capture link claiming the
* device. Unregister and hand the pad back to the phone-gyro mirror. Main thread, from
* [GamepadRouter.onSlotClosed].
*
* No park-at-zero here, on purpose: the router removed the slot BEFORE invoking the callback
* and has already sent that pad's Remove, so the host tore the virtual pad down and there is no
* latched rotation left to clear while writing to a wire index that is free again would be
* addressing whoever claims it next. [stop] is the case where the pad outlives the feed.
*/
fun onSlotClosed(deviceId: Int) {
unregister(deviceId)
router.setDeviceHasSensorMotion(deviceId, false)
}
/**
* Unregister every listener, join the sensor thread, then park at zero each pad that was
* rotating. Call BEFORE the router is released and the session handle freed the same
* teardown ordering rule as the feedback poll threads and [DeviceGyro.stop]. The parks come
* AFTER the join for two reasons: a sample still in flight would re-latch the rotation just
* cleared, and the join is what publishes the sensor thread's writes to this one.
*/
fun stop() {
val parked = feeds.keys.toList().mapNotNull { id -> unregister(id)?.let { id to it } }
for ((deviceId, _) in parked) router.setDeviceHasSensorMotion(deviceId, false)
thread.quitSafely()
runCatching { thread.join() }
handler = null
for ((_, feed) in parked) if (feed.wroteMotion) feed.park()
}
/**
* Drop [deviceId]'s listeners, returning the feed that held them (null if there was none).
* Safe for a controller that is already gone: the sensor manager is reached through the
* [InputDevice], and a vanished device simply leaves nothing to unregister the platform has
* stopped calling the listener either way.
*/
private fun unregister(deviceId: Int): Feed? {
val feed = feeds.remove(deviceId) ?: return null
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
InputDevice.getDevice(deviceId)?.sensorManager?.unregisterListener(feed)
}
return feed
}
companion object {
private const val TAG = "PadSensors"
/** Whether this platform can read a controller's own sensors at all (API 31+). */
fun supported(): Boolean = Build.VERSION.SDK_INT >= Build.VERSION_CODES.S
/**
* One gyroscope sample (Android: rad/s) the wire's three signed-16 components, in place.
*
* The axis frame is straight through, and that is now MEASURED rather than assumed.
*
* The wire is a unit passthrough into a virtual DualSense report, whose frame was measured
* over raw HID on 2026-08-07: slot 0 = Right (pitch), slot 1 = Up (yaw), slot 2 = Backward
* toward the player (roll), right-handed. Android hands a controller's own sensors over in
* that same frame which was the documented expectation, but the numbers pass through a
* HID driver and InputFlinger's sensor mapper, either of which could have permuted or
* negated without saying so.
*
* Verified 2026-08-07 end to end: a DualSense on Bluetooth to an Android phone, streaming
* to a Linux host. This path's own first-sample log read `accel 0, 10000, 0` exactly 1 g
* on slot 1 and at the far end `hid-playstation` published gravity as +0.991 g on ABS_Y
* with every rotation driving its correctly-named axis (yawRY, pitchRX, rollRZ) and the
* signs agreeing with gravity's independent witness on 95 of 100 rotating samples.
*
* So: no remap. If a future device disagrees, the remap belongs HERE with its own
* expectations in `PadSensorsTest` not spread across callers.
*/
fun gyroToWire(values: FloatArray, out: IntArray) {
for (i in 0..2) out[i] = Gamepad.motionGyroWire(values.getOrElse(i) { 0f })
}
/**
* One accelerometer sample (Android: m/, specific force) the wire's three signed-16
* components, in place. Same measured frame as [gyroToWire] and the same straight-through
* mapping; the sign needs no flip, because Android and the DualSense report agree that the
* axis pointing up reads +1 g at rest (see [Gamepad.motionAccelWire]) which is precisely
* what the on-glass run read back, `accel 0, 10000, 0` with the pad lying flat.
*/
fun accelToWire(values: FloatArray, out: IntArray) {
for (i in 0..2) out[i] = Gamepad.motionAccelWire(values.getOrElse(i) { 0f })
}
}
}
@@ -0,0 +1,54 @@
package io.unom.punktfunk.kit
import android.view.Surface
import org.junit.Assert.assertEquals
import org.junit.Test
/**
* Pins the phone-gyro mirror's devicecontroller frame remap and its wire-unit constants
* ([DeviceGyro]). Pure JVM: [Surface]'s ROTATION_* are compile-time constants and remap is
* plain math. The matrix is derived (like the wire scale constants) if on-glass says an axis
* is wrong, fix [DeviceGyro.remap] AND these expectations together.
* Run: `./gradlew :kit:testDebugUnitTest`.
*/
class DeviceGyroTest {
/** A distinct value per axis so a swapped or flipped component can't cancel out. */
private fun remap(rotation: Int) = DeviceGyro.remap(rotation, 1f, 2f, 3f).toList()
@Test
fun naturalPortraitIsIdentity() = assertEquals(listOf(1f, 2f, 3f), remap(Surface.ROTATION_0))
@Test
fun upsideDownFlipsInPlane() = assertEquals(listOf(-1f, -2f, 3f), remap(Surface.ROTATION_180))
/** ROTATION_90 = device turned counter-clockwise, top to the player's LEFT:
* player-right = device-bottom (y), player-up = device-right (+x); z never changes. */
@Test
fun rotation90TopLeft() = assertEquals(listOf(-2f, 1f, 3f), remap(Surface.ROTATION_90))
/** ROTATION_270 = top to the player's RIGHT: player-right = +y, player-up = x. */
@Test
fun rotation270TopRight() = assertEquals(listOf(2f, -1f, 3f), remap(Surface.ROTATION_270))
/** Every remap stays a proper (right-handed) rotation: x̂ × ŷ = ẑ after mapping. */
@Test
fun handednessPreserved() {
for (r in listOf(
Surface.ROTATION_0, Surface.ROTATION_90, Surface.ROTATION_180, Surface.ROTATION_270,
)) {
val x = DeviceGyro.remap(r, 1f, 0f, 0f)
val y = DeviceGyro.remap(r, 0f, 1f, 0f)
assertEquals("left-handed remap at rotation $r", 1f, x[0] * y[1] - x[1] * y[0], 0f)
}
}
/** The wire contract, shared with pf-client-core / the Swift client and now with every other
* Android motion sender ([Gamepad.motionGyroWire]): 20 LSB/°·s means 1 rad/s ~1145.9 raw;
* 1 g 10000 raw. */
@Test
fun wireUnitConstants() {
assertEquals(20f * 180f / Math.PI.toFloat(), Gamepad.MOTION_GYRO_LSB_PER_RAD_S, 0f)
assertEquals(1145.9156f, Gamepad.MOTION_GYRO_LSB_PER_RAD_S, 0.001f)
assertEquals(10_000, Gamepad.MOTION_ACCEL_LSB_PER_G)
}
}
@@ -151,6 +151,185 @@ class DsDeviceTest {
assertFalse(DsDevice.parseState(DsDevice.Model.DUALSHOCK4, ds4Report(), 8, s))
}
// ---- IMU calibration (the pad's own scale factors) ----
/**
* A calibration feature report in the pads' USB layout: report id, three gyro bias words, six
* INTERLEAVED gyro plus/minus words, the two speed words, six accel plus/minus words all
* little-endian i16, exactly what [DsDevice.MotionCal.parse] reads and what
* `crates/pf-inject/tests/motion_contract.rs` writes from the other end.
*/
private fun calBlob(
id: Int,
gyroBias: IntArray,
gyroPlus: IntArray,
gyroMinus: IntArray,
speed: Int,
accelPlus: IntArray,
accelMinus: IntArray,
len: Int = 41,
): ByteArray = ByteArray(len).also { b ->
fun put(o: Int, v: Int) {
b[o] = (v and 0xFF).toByte()
b[o + 1] = ((v shr 8) and 0xFF).toByte()
}
b[0] = id.toByte()
for (i in 0 until 3) {
put(1 + 2 * i, gyroBias[i])
put(7 + 4 * i, gyroPlus[i])
put(9 + 4 * i, gyroMinus[i])
put(23 + 4 * i, accelPlus[i])
put(25 + 4 * i, accelMinus[i])
}
put(19, speed)
put(21, speed)
}
/**
* A realistic DualSense blob: gyro measured at 512 °/s each way over ±8192 counts about a
* small factory bias 16384/1024 = 16 raw LSB per °/s, the ±2000 °/s full scale a real pad
* has and accel spanning about ±8192 counts (`DS_ACC_RES_PER_G`) about a per-axis zero point
* that is NOT zero. Both are the shape a nominal constant cannot express.
*/
private fun realisticCal(): DsDevice.MotionCal = DsDevice.MotionCal.parse(
calBlob(
id = 0x05,
gyroBias = intArrayOf(10, -6, 3),
gyroPlus = intArrayOf(10 + 8192, -6 + 8192, 3 + 8192),
gyroMinus = intArrayOf(10 - 8192, -6 - 8192, 3 - 8192),
speed = 512, // speed_plus + speed_minus = 1024
accelPlus = intArrayOf(8300, 8200, 8000),
accelMinus = intArrayOf(-8100, -8192, -8384),
),
0x05,
)
@Test
fun calibrationRescalesRawCountsOntoTheWireUnits() {
val cal = realisticCal()
// 100 °/s at this pad's 16 LSB per °/s = 1600 raw → the wire's 20 LSB per °/s = 2000.
for (axis in 0 until 3) {
assertEquals(2000, cal.gyroToWire(axis, 1600))
assertEquals(-2000, cal.gyroToWire(axis, -1600))
assertEquals(0, cal.gyroToWire(axis, 0))
}
// 1 g = the axis's zero point plus half its declared 2 g range → 10000 wire units.
val zero = intArrayOf(100, 4, -192) // plus range/2, per axis
val oneG = intArrayOf(8300, 8200, 8000) // = accelPlus
for (axis in 0 until 3) {
assertEquals(10000, cal.accelToWire(axis, oneG[axis]))
assertEquals(0, cal.accelToWire(axis, zero[axis]))
assertEquals(-10000, cal.accelToWire(axis, zero[axis] - (oneG[axis] - zero[axis])))
}
// Both rescales are >1 here, so full-scale raw must clamp rather than wrap the i16.
assertEquals(32767, cal.gyroToWire(0, 30000))
assertEquals(-32768, cal.gyroToWire(0, -30000))
assertEquals(32767, cal.accelToWire(0, 30000))
// The capture logs this, and it is the discriminator the owed on-glass check reads: a pad
// whose blob was read declares its own resolution, the fallback declares the wire's.
assertTrue(cal.toString().startsWith("gyro 16/16/16 LSB/°·s"))
assertTrue(DsDevice.MotionCal.NOMINAL.toString().startsWith("gyro 20/20/20 LSB/°·s"))
}
/**
* The host's own virtual pads declare `DS_FEATURE_CALIBRATION` (`dualsense_proto.rs`) a blob
* that states the wire's units exactly. Reading it back must therefore be a passthrough: if
* this ever stops holding, the client and the host disagree about what a motion sample means.
*/
@Test
fun theHostsOwnBlobIsAPassthrough() {
val cal = DsDevice.MotionCal.parse(
calBlob(
id = 0x05,
gyroBias = intArrayOf(0, 0, 0),
gyroPlus = intArrayOf(10000, 10000, 10000),
gyroMinus = intArrayOf(-10000, -10000, -10000),
speed = 500,
accelPlus = intArrayOf(10000, 10000, 10000),
accelMinus = intArrayOf(-10000, -10000, -10000),
),
0x05,
)
for (axis in 0 until 3) {
assertEquals(2000, cal.gyroToWire(axis, 2000)) // 100 °/s
assertEquals(10000, cal.accelToWire(axis, 10000)) // 1 g
assertEquals(-1234, cal.gyroToWire(axis, -1234))
}
}
/**
* Anything unusable keeps the pre-calibration behaviour accel on the nominal 8192 LSB/g,
* gyro straight through. A pad with no readable calibration is better off slightly mis-scaled
* than silent, so nothing here may zero motion.
*/
@Test
fun unusableCalibrationFallsBackInsteadOfZeroing() {
val degenerate = calBlob(
id = 0x02,
gyroBias = intArrayOf(0, 0, 0),
gyroPlus = intArrayOf(0, 0, 0),
gyroMinus = intArrayOf(0, 0, 0),
speed = 0,
accelPlus = intArrayOf(0, 0, 0),
accelMinus = intArrayOf(0, 0, 0),
len = 37,
)
val cals = listOf(
DsDevice.MotionCal.NOMINAL,
DsDevice.MotionCal.parse(null, 0x05), // the GET_REPORT failed
DsDevice.MotionCal.parse(ByteArray(8) { if (it == 0) 0x05 else 0 }, 0x05), // short reply
DsDevice.MotionCal.parse(degenerate, 0x02), // a clone pad's zeroes
DsDevice.MotionCal.parse(degenerate, 0x05), // someone else's report id
)
for (cal in cals) {
for (axis in 0 until 3) {
assertEquals(1234, cal.gyroToWire(axis, 1234)) // passthrough
assertEquals(10000, cal.accelToWire(axis, 8192)) // 8192 raw LSB = 1 g
assertEquals(-10000, cal.accelToWire(axis, -8192))
}
}
}
/** The parse applies the calibration at the motion offsets, per model, and defaults to nominal. */
@Test
fun parseStateAppliesTheCalibration() {
val cal = realisticCal()
// DS5: gyro at [16..22), accel at [22..28). Pitch = 1600 raw (100 °/s), accel z = 8000 (1 g).
val ds5 = ds5Report {
it[16] = 0x40; it[17] = 0x06 // 1600
it[26] = 0x40; it[27] = 0x1F // 8000
}
val five = DsDevice.State()
assertTrue(DsDevice.parseState(DsDevice.Model.DUALSENSE, ds5, 64, five, cal))
assertEquals(2000, five.gyro[0])
assertEquals(10000, five.accel[2])
// DS4: gyro at [13..19), accel at [19..25). Same numbers, same answers.
val ds4 = ds4Report {
it[13] = 0x40; it[14] = 0x06
it[23] = 0x40; it[24] = 0x1F
}
val four = DsDevice.State()
assertTrue(DsDevice.parseState(DsDevice.Model.DUALSHOCK4, ds4, 64, four, cal))
assertEquals(2000, four.gyro[0])
assertEquals(10000, four.accel[2])
// No calibration argument = the nominal fallback: gyro through, accel ×10000/8192.
val nominal = DsDevice.State()
assertTrue(DsDevice.parseState(DsDevice.Model.DUALSENSE, ds5, 64, nominal))
assertEquals(1600, nominal.gyro[0])
assertEquals(8000L * 10000 / 8192, nominal.accel[2].toLong())
}
/** Each model asks for the feature report its firmware actually serves over USB. */
@Test
fun calibrationReportIdentityPerModel() {
assertEquals(0x05, DsDevice.Model.DUALSENSE.calReportId)
assertEquals(41, DsDevice.Model.DUALSENSE.calReportLen)
assertEquals(0x05, DsDevice.Model.DUALSENSE_EDGE.calReportId)
assertEquals(41, DsDevice.Model.DUALSENSE_EDGE.calReportLen)
assertEquals(0x02, DsDevice.Model.DUALSHOCK4.calReportId)
assertEquals(37, DsDevice.Model.DUALSHOCK4.calReportLen)
}
// ---- output builders (offsets = the host parser's: `parse_ds_output` / `parse_ds4_output`) ----
@Test
@@ -0,0 +1,152 @@
package io.unom.punktfunk.kit
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNotEquals
import org.junit.Assert.assertSame
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* The claim/read hand-off that lets [DsCapture] read a pad's motion calibration off the claiming
* thread. Two things are pinned here, and both are about the gap before the read comes back.
*
* What the gap DOES: the pad streams, scaled by the nominal calibration the behaviour that
* shipped before the read existed. What it must NOT do: inherit the previous pad's factory numbers
* (calibration is per unit), or accept a read that outlived its claim, which an unplug, a stop, or
* a re-claim can all cause.
*/
class MotionCalHandoffTest {
/**
* A calibration whose gyro reads [rawLsbPerDegS] raw LSB per °/s and whose accel sits at
* [accelZero] raw counts at 0 g, so two of them are told apart by what they DO identity
* alone would let a regression that returns the wrong instance still look right.
*/
private fun cal(rawLsbPerDegS: Int, accelZero: Int = 0): DsDevice.MotionCal {
val speed = 500 // speed_plus = speed_minus, so speed_2x = 1000
val span = rawLsbPerDegS * 1000 // |plus bias| + |minus bias| = span
val blob = ByteArray(41)
fun put(o: Int, v: Int) {
blob[o] = (v and 0xFF).toByte()
blob[o + 1] = ((v shr 8) and 0xFF).toByte()
}
blob[0] = 0x05
for (i in 0 until 3) {
put(7 + 4 * i, span / 2) // gyro plus
put(9 + 4 * i, -span / 2) // gyro minus
put(23 + 4 * i, accelZero + 8192) // accel plus / minus: 8192 raw LSB per g
put(25 + 4 * i, accelZero - 8192)
}
put(19, speed)
put(21, speed)
return DsDevice.MotionCal.parse(blob, 0x05)
}
/** One DS5 input report: cross held, sticks centred, gyro pitch 1600 raw, accel z 8000 raw. */
private fun report(): ByteArray = ByteArray(64).also {
it[0] = 0x01
it[1] = 0x80.toByte(); it[2] = 0x80.toByte(); it[3] = 0x80.toByte(); it[4] = 0x80.toByte()
it[8] = (0x08 or 0x20).toByte() // hat neutral | cross
it[16] = 0x40; it[17] = 0x06 // gyro pitch = 1600
it[26] = 0x40; it[27] = 0x1F // accel z = 8000
it[33] = 0x80.toByte(); it[37] = 0x80.toByte() // no touch contacts
}
@Test
fun `a claim scales nominally until its read lands`() {
val h = MotionCalHandoff()
assertSame(DsDevice.MotionCal.NOMINAL, h.effective)
val claim = h.begin()
assertSame("the read is in flight — scale nominally, do not wait", DsDevice.MotionCal.NOMINAL, h.effective)
val read = cal(16)
assertTrue(h.publish(claim, read))
assertSame(read, h.effective)
}
/**
* The whole point of scaling nominally instead of holding reports back: a pad answers its
* buttons from the first report, and only its motion changes when the calibration arrives.
*/
@Test
fun `a report in the gap is forwarded, nominally scaled, and rescales once the read lands`() {
val h = MotionCalHandoff()
val claim = h.begin()
val r = report()
val gap = DsDevice.State()
assertTrue(
"a report must still be parsed while the read is in flight",
DsDevice.parseState(DsDevice.Model.DUALSENSE, r, 64, gap, h.effective),
)
assertEquals("buttons reach the wire immediately", Gamepad.BTN_A, gap.buttons)
assertEquals("and so do sticks", 128, gap.lsX)
assertEquals("nominal gyro is the raw count", 1600, gap.gyro[0])
assertEquals("nominal accel is ×10000/8192", 8000L * 10000 / 8192, gap.accel[2].toLong())
assertTrue(h.publish(claim, cal(16, accelZero = 100)))
val live = DsDevice.State()
assertTrue(DsDevice.parseState(DsDevice.Model.DUALSENSE, r, 64, live, h.effective))
assertEquals("buttons do not depend on the calibration", gap.buttons, live.buttons)
assertEquals("1600 raw at 16 LSB/°·s = 100 °/s = 2000 wire", 2000, live.gyro[0])
assertNotEquals("the same raw report must convert differently now", gap.gyro[0], live.gyro[0])
assertNotEquals(gap.accel[2], live.accel[2])
}
@Test
fun `a read that outlived its claim publishes nothing`() {
val h = MotionCalHandoff()
val claim = h.begin()
h.end() // unplug, or DsCapture.stop, while the read was in flight
assertFalse("a straggler may not publish into a dead claim", h.publish(claim, cal(16)))
assertSame(DsDevice.MotionCal.NOMINAL, h.effective)
}
@Test
fun `a new claim scales nominally rather than inheriting the previous pad's calibration`() {
val h = MotionCalHandoff()
val first = h.begin()
val hot = cal(4, accelZero = 400) // a pad reading 4 raw LSB per °/s, well off nominal
assertTrue(h.publish(first, hot))
assertSame(hot, h.effective)
// Re-claimed without an end() in between — the pad was swapped while a read was in flight.
val second = h.begin()
assertNotEquals(first, second)
assertSame(
"the next pad starts on the nominal scaling, NOT the last pad's factory numbers",
DsDevice.MotionCal.NOMINAL,
h.effective,
)
assertFalse("the first pad's read may not scale the second pad", h.publish(first, hot))
assertSame(DsDevice.MotionCal.NOMINAL, h.effective)
// And that fallback is a real difference, not two names for the same numbers: the inherited
// calibration would have turned this pad's motion into something else entirely.
val r = report()
val nominal = DsDevice.State()
val inherited = DsDevice.State()
DsDevice.parseState(DsDevice.Model.DUALSENSE, r, 64, nominal, h.effective)
DsDevice.parseState(DsDevice.Model.DUALSENSE, r, 64, inherited, hot)
assertNotEquals(inherited.gyro[0], nominal.gyro[0])
assertNotEquals(inherited.accel[2], nominal.accel[2])
val slow = cal(32)
assertTrue(h.publish(second, slow))
assertSame(slow, h.effective)
}
@Test
fun `ending a claim twice still refuses every outstanding token`() {
val h = MotionCalHandoff()
val claim = h.begin()
h.end() // DsCapture.stop
h.end() // …and the unplug that followed it
assertFalse(h.publish(claim, cal(16)))
assertSame(DsDevice.MotionCal.NOMINAL, h.effective)
val next = h.begin()
assertNotEquals(claim, next)
val read = cal(16)
assertTrue(h.publish(next, read))
assertSame(read, h.effective)
}
}
@@ -0,0 +1,92 @@
package io.unom.punktfunk.kit
import org.junit.Assert.assertArrayEquals
import org.junit.Assert.assertEquals
import org.junit.Test
/**
* Pins the unit scaling and the axis mapping of the controller-sensor path ([PadSensors]) and the
* shared converters it goes through ([Gamepad.motionGyroWire] / [Gamepad.motionAccelWire]). Pure
* JVM the two `*ToWire` functions take plain float arrays and touch no Android class.
*
* The scale is MEASURED FACT (`punktfunk_core::input::gamepad`: 20 LSB/°·s, 10000 LSB/g) and must
* not drift. The axis mapping is straight through and NOT yet verified against hardware see
* [PadSensors.gyroToWire] for the measurement that would settle it. [straightThroughFrame] exists
* to make a future remap a deliberate, visible edit rather than a quiet one.
* Run: `./gradlew :kit:testDebugUnitTest`.
*/
class PadSensorsTest {
private fun gyro(x: Float, y: Float, z: Float) =
IntArray(3).also { PadSensors.gyroToWire(floatArrayOf(x, y, z), it) }
private fun accel(x: Float, y: Float, z: Float) =
IntArray(3).also { PadSensors.accelToWire(floatArrayOf(x, y, z), it) }
/** 20 LSB/°·s from Android's rad/s: π rad/s is exactly 180 °/s, so exactly 3600 raw. */
@Test
fun gyroScaleFromRadiansPerSecond() {
assertEquals(3600, gyro(Math.PI.toFloat(), 0f, 0f)[0])
assertEquals(-3600, gyro(-Math.PI.toFloat(), 0f, 0f)[0])
assertEquals(1146, gyro(1f, 0f, 0f)[0]) // 1 rad/s ⇒ 1145.9156, rounded
assertEquals(0, gyro(0f, 0f, 0f)[0])
}
/** 10000 LSB/g from Android's m/: standard gravity is exactly 1 g. Android reports specific
* force, so a pad at rest reads +1 g on the axis pointing up no sign flip anywhere. */
@Test
fun accelScaleFromMetresPerSecondSquared() {
assertEquals(10_000, accel(0f, Gamepad.GRAVITY, 0f)[1])
assertEquals(-10_000, accel(0f, -Gamepad.GRAVITY, 0f)[1])
assertEquals(0, accel(0f, 0f, 0f)[1])
}
/** A controller lying flat and still lands exactly on the host's neutral for a virtual
* DualSense 1 g on wire slot 1 (`punktfunk-core` `MOTION_NEUTRAL_ACCEL = [0, 10000, 0]`),
* not the [0,0,0] that means free fall. */
@Test
fun restingPadIsTheHostNeutral() {
assertArrayEquals(intArrayOf(0, 10_000, 0), accel(0f, Gamepad.GRAVITY, 0f))
}
/**
* The frame: component i of the sensor sample becomes component i of the wire triple, for both
* planes, with no permutation and no negation. UNVERIFIED against hardware if a Bluetooth
* DualSense says otherwise, the remap goes into [PadSensors.gyroToWire] and this test changes
* with it. Distinct magnitudes per axis so a swap or a flip cannot cancel out.
*/
@Test
fun straightThroughFrame() {
assertArrayEquals(intArrayOf(1146, 2292, 3438), gyro(1f, 2f, 3f))
assertArrayEquals(
intArrayOf(10_000, 20_000, -30_000),
accel(Gamepad.GRAVITY, 2f * Gamepad.GRAVITY, -3f * Gamepad.GRAVITY),
)
}
/** Both planes clamp to signed 16 bits rather than wrapping a flick past 1638 °/s or a knock
* past 3.27 g saturates, where a wrap would send a full-speed rotation the other way. */
@Test
fun clampsToSigned16() {
assertArrayEquals(intArrayOf(32767, -32768, 32767), gyro(100f, -100f, 1e9f))
assertArrayEquals(intArrayOf(32767, -32768, 32767), accel(1000f, -1000f, 1e9f))
}
/** Rounds to nearest rather than truncating: a truncating converter loses up to a whole LSB
* off every sample, always toward zero, and a gyro whose every sample is biased the same way
* is a gyro that drifts. */
@Test
fun roundsToNearestNotTowardZero() {
assertEquals(1, gyro(0.0006f, 0f, 0f)[0]) // 0.688 raw — truncation would say 0
assertEquals(-1, gyro(-0.0006f, 0f, 0f)[0])
assertEquals(1, accel(0.0007f, 0f, 0f)[0]) // 0.714 raw
}
/** A sensor that hands back fewer than three components (or none the framework reuses one
* array across types) contributes zero rather than throwing on the sensor thread. */
@Test
fun shortSampleIsZeroFilled() {
val out = IntArray(3) { 7 }
PadSensors.gyroToWire(floatArrayOf(Math.PI.toFloat()), out)
assertArrayEquals(intArrayOf(3600, 0, 0), out)
}
}
+106 -12
View File
@@ -20,6 +20,16 @@
//! (2) is now the SHARED `punktfunk_core::audio::JitterPolicy` at `JitterTuning::AAUDIO`, which also
//! fixed what this ring was missing: it had a hard cap but nothing that walked the depth back down,
//! so drift and arrival bursts raised latency permanently and Android settled on its ceiling.
//!
//! It is also **A/V synchronised** (`design/audio-latency-overhaul.md`): the decode thread reads the
//! host capture `pts_ns` every `AudioPacket` has always carried, compares where this frame will
//! actually play against where the picture it belongs with reached glass
//! (`decode::DisplayTracker` publishes that), and asks the ring for a depth that closes the gap.
//! Only ASKS — `JitterPolicy` clamps the request between its own underrun-driven floor and the hard
//! cap, so continuity outranks sync and a link whose jitter genuinely needs more buffer than the
//! picture is away keeps its buffer, with the residual reported on the HUD instead of taken out of
//! the listener's stream. With no video reference (below API 33 there are no render callbacks, so
//! nothing confirms a present) the target stays `None` and the ring behaves exactly as it did.
use ndk::audio::{
AudioCallbackResult, AudioContentType, AudioDirection, AudioFormat, AudioPerformanceMode,
@@ -34,6 +44,14 @@ use std::sync::mpsc::{sync_channel, Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
/// What one playback open attempt yields: the stream, plus both halves of the PCM hand-off — the
/// sender the decode thread fills and the receiver that returns drained buffers for refill.
///
/// Named rather than written inline because the closure's return type trips
/// `clippy::type_complexity`, which the Android target is now linted for (`:kit:cargoNdkClippy`)
/// after years of nothing checking it.
type OpenedPlayback = ndk::audio::Result<(AudioStream, SyncSender<Vec<f32>>, Receiver<Vec<f32>>)>;
const SAMPLE_RATE: i32 = 48_000;
/// Decoded-chunk hand-off depth: 64 × 5 ms = 320 ms slack (matches the core's AUDIO_QUEUE).
const RING_CHUNKS: usize = 64;
@@ -94,15 +112,45 @@ impl AudioDec {
/// Diagnostics — written by the decode thread + the realtime callback, logged periodically. The
/// audio analogue of the video `fed`/`rendered` counters (we can't "screenshot" sound).
///
/// The ring's DEPTH is not here: the A/V sync loop needs the same number in the same units, so it
/// is published once through [`punktfunk_core::audio::AudioSyncCell`] and read from there by the
/// log line below. One publisher, one reading — a second copy is a second thing to go stale.
#[derive(Default)]
struct Counters {
opus_decoded: AtomicU64, // Opus packets decoded OK (~200/s at 5 ms frames)
pcm_written: AtomicU64, // PCM frames copied out to AAudio (device clock is pulling)
underruns: AtomicU64, // callbacks that emitted silence (ring not primed / drained)
ring_depth: AtomicU64, // ring sample count at the last callback
target_ms: AtomicU64, // the policy's LIVE target depth (it grows on this device's underruns)
}
/// Whether the A/V sync loop runs this session. `false` leaves `JitterPolicy`'s sync target at
/// `None`, which reproduces the pre-overhaul ring behaviour exactly — the point of the hatch.
///
/// Two levers because Android has neither of the other clients' launch surfaces. `PUNKTFUNK_NO_AV_SYNC`
/// keeps the contract the desktop clients document (and works when the client is driven from a
/// shell), but an app started from the launcher inherits no such environment, so the one a field
/// tester can actually reach is the sysprop — `adb shell setprop debug.punktfunk.no_av_sync 1`,
/// no rebuild, exactly like `debug.punktfunk.presenter`. A loop that steers PLAYBACK has to be
/// bisectable on the device that reports the regression, not only on the bench.
fn av_sync_enabled() -> bool {
if matches!(
std::env::var("PUNKTFUNK_NO_AV_SYNC").as_deref(),
Ok("1") | Ok("true")
) {
return false;
}
let mut buf = [0u8; 92]; // PROP_VALUE_MAX
// SAFETY: __system_property_get with a valid name + PROP_VALUE_MAX buffer is always safe.
let n = unsafe {
libc::__system_property_get(
c"debug.punktfunk.no_av_sync".as_ptr(),
buf.as_mut_ptr().cast(),
)
};
!(n > 0 && matches!(&buf[..n as usize], b"1" | b"true"))
}
/// Owned by [`crate::session::SessionHandle`]: the live AAudio stream + the decode thread.
pub struct AudioPlayback {
_stream: AudioStream, // dropping it stops + closes the AAudio stream
@@ -127,15 +175,15 @@ impl AudioPlayback {
// Worst transient the ring can hold before the policy trims it.
let hard_cap_max = tuning.hard_cap_ms as usize * ms;
let counters = Arc::new(Counters::default());
// The A/V sync hand-off: the realtime callback owns the ring (so it publishes the depth and
// consumes the target), the decode thread owns the timestamps (so it computes the target).
// Two atomics, because the callback must not block on the thread that decodes Opus.
let sync: Arc<punktfunk_core::audio::AudioSyncCell> = Arc::default();
// One open attempt at a given sharing mode. Everything the realtime callback captures
// (channels, ring, prime state) is rebuilt per attempt — `open_stream` consumes the builder
// AND the callback, so nothing survives a failed try to reuse.
let try_open = |sharing: AudioSharingMode| -> ndk::audio::Result<(
AudioStream,
SyncSender<Vec<f32>>,
Receiver<Vec<f32>>,
)> {
let try_open = |sharing: AudioSharingMode| -> OpenedPlayback {
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list: drained PCM buffers go BACK to the decode thread to be refilled, so
// the realtime callback never frees heap (Android's Scudo allocator has unbounded free()
@@ -146,6 +194,7 @@ impl AudioPlayback {
// Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from
// a single high-priority thread, and the decode thread only touches `tx`/`free_rx`.
let cb_counters = counters.clone();
let cb_sync = sync.clone();
// Pre-reserve the ring so `extend` never reallocates on the realtime thread. Worst
// transient before the trim below = the hard cap plus one full channel of 5 ms (480-f32)
// frames — the punktfunk protocol always sends 5 ms Opus frames (host `audio_thread`); a
@@ -171,6 +220,13 @@ impl AudioPlayback {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
// A/V sync: take whatever depth the decode thread's sync loop last asked for, and
// publish where the ring actually is so it can measure the result. The policy
// clamps the request between its own underrun floor and the hard cap — continuity
// outranks sync, always (see `JitterPolicy::set_sync_target`). Read AFTER the
// drain, so the depth is everything a frame queued right now must wait behind.
policy.set_sync_target(cb_sync.target());
cb_sync.publish_depth(ring.len());
// Jitter buffer: the shared policy decides prime/silence, trims a burst, and —
// new here — sheds ONE crossfaded 5 ms frame when the depth average has sat above
// target long enough to be drift rather than jitter. Without that shed this ring
@@ -201,9 +257,6 @@ impl AudioPlayback {
// No-op while un-primed, so a deliberate priming silence is never counted as an
// underrun (which would otherwise drive the adaptive floor up for no reason).
policy.note_read(ran_short);
cb_counters
.ring_depth
.store(ring.len() as u64, Ordering::Relaxed);
cb_counters
.target_ms
.store(policy.target_ms() as u64, Ordering::Relaxed);
@@ -303,7 +356,7 @@ impl AudioPlayback {
let sd = shutdown.clone();
let join = std::thread::Builder::new()
.name("pf-audio".into())
.spawn(move || decode_loop(client, tx, free_rx, sd, counters, channels))
.spawn(move || decode_loop(client, tx, free_rx, sd, counters, channels, sync))
.ok();
Some(AudioPlayback {
@@ -334,6 +387,7 @@ fn decode_loop(
shutdown: Arc<AtomicBool>,
counters: Arc<Counters>,
channels: usize,
sync: Arc<punktfunk_core::audio::AudioSyncCell>,
) {
// Fold this Opus→AAudio thread into the client's hot-thread set so the ADPF session the decode
// thread opens also keeps audio decode on a fast core (registered before the video pump's first
@@ -354,9 +408,44 @@ fn decode_loop(
let mut window_peak = 0f32; // loudest |sample| since the last log — tells a tone from silence
let mut gaps = punktfunk_core::audio::AudioGapTracker::new();
let mut frame_samples = 0usize; // per-channel samples of the last decoded frame — the PLC unit
// A/V sync (audio latency overhaul). This thread is the only place holding all three
// ingredients at once: the packet's host capture `pts_ns`, the ring depth (via the sync cell)
// and the video plane's end-to-end figure. `pts_ns` arrived in every `AudioPacket` and was
// dropped on the floor here for the plane's whole existence, which is why audio ran at whatever
// depth its jitter ring settled at with nothing ever placing it against the picture.
let av_sync_enabled = av_sync_enabled();
let mut av = punktfunk_core::audio::AvSync::new(channels as u8);
let video_e2e = client.video_e2e_shared();
let av_offset_out = client.audio_av_offset_shared();
let buffer_ms_out = client.audio_buffer_ms_shared();
if !av_sync_enabled {
log::info!("audio: A/V sync disabled (PUNKTFUNK_NO_AV_SYNC / debug.punktfunk.no_av_sync)");
}
'pump: while !shutdown.load(Ordering::Relaxed) {
match client.next_audio(Duration::from_millis(5)) {
Ok(pkt) => {
// Place this frame against the picture it belongs with, BEFORE it is queued:
// `buffered_ahead` is everything that must still play first, so the depth read here
// is exactly what delays it.
let depth = sync.depth();
// Published unconditionally — the ring's depth is worth seeing even with sync off,
// and it is what makes a "the audio delay is way too high" report triageable at all.
buffer_ms_out.store((depth / ms.max(1)) as u32, Ordering::Relaxed);
if av_sync_enabled {
let ve2e = video_e2e.load(Ordering::Relaxed);
av.observe(punktfunk_core::audio::AvSyncObservation {
pts_ns: pkt.pts_ns,
now_local_ns: punktfunk_core::client::now_realtime_ns(),
clock_offset_ns: client.clock_offset_now_ns(),
buffered_ahead: depth,
// 0 = nothing confirmed on the glass yet (no render callback below API 33,
// or the stream has not presented a frame); no reference, no correction.
video_e2e_ns: (ve2e > 0).then_some(ve2e),
});
sync.set_target(av.desired_depth(depth));
av_offset_out.store(av.offset_ms() as i64, Ordering::Relaxed);
}
// Conceal lost packets (a seq gap) with libopus PLC before decoding the one that
// arrived: empty input synthesizes `frame_samples` of interpolation per missing
// packet — an inaudible fade instead of the click a hard gap makes in the ring.
@@ -404,12 +493,17 @@ fn decode_loop(
Err(TrySendError::Disconnected(_)) => break,
}
if count % 600 == 0 {
// `av_ms` is the sync loop's smoothed placement error (+ = audio behind
// the picture); 0 with sync off, or before it has a video reference.
// Logged next to the depth because a deep ring on a jittery link is
// correct and only the offset separates that from audio held late.
log::info!(
"audio: opus={count} pcm_frames={} underruns={} buffer_ms={} target_ms={} peak={window_peak:.3}",
"audio: opus={count} pcm_frames={} underruns={} buffer_ms={} target_ms={} av_ms={} peak={window_peak:.3}",
counters.pcm_written.load(Ordering::Relaxed),
counters.underruns.load(Ordering::Relaxed),
counters.ring_depth.load(Ordering::Relaxed) / ms.max(1) as u64,
(depth / ms.max(1)) as u64,
counters.target_ms.load(Ordering::Relaxed),
av.offset_ms(),
);
window_peak = 0.0;
}
@@ -204,7 +204,15 @@ pub(super) fn run_async(
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
let meter = Arc::new(PresentMeter::new());
let tracker = DisplayTracker::new(stats.clone(), clock_offset.clone(), meter.clone());
// The tracker also publishes each confirmed present's end-to-end into the shared cell the audio
// plane steers its jitter ring by (`design/audio-latency-overhaul.md`) — video is the master,
// and this is the only point that knows when a frame actually reached glass.
let tracker = DisplayTracker::new(
stats.clone(),
clock_offset.clone(),
client.video_e2e_shared(),
meter.clone(),
);
let render_cb = install_render_callback(&codec, &tracker);
// The timeline presenter (see `presenter.rs`): newest-wins / smoothing store, one-in-flight
+45 -9
View File
@@ -5,7 +5,7 @@ use ndk::media::media_codec::MediaCodec;
use ndk::native_window::NativeWindow;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicI64, Ordering};
use std::sync::atomic::{AtomicI64, AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use super::latency::now_realtime_ns;
@@ -35,6 +35,16 @@ pub(super) struct DisplayTracker {
/// loaded per callback so mid-stream re-syncs apply. Holding the handle (not the client)
/// keeps the leaked render-callback refcount from pinning the whole session alive.
clock_offset: Arc<AtomicI64>,
/// Where the AUDIO plane reads the video leg it has to land with (ns) — `displayed +
/// clock_offset pts`, published on every confirmed present. Written here, read by
/// [`crate::audio`]'s sync loop; the two planes never touch each other directly (the presenter
/// must not know about audio, and the audio thread cannot see the glass).
///
/// Published RAW. The HUD shaves the OS present floor off its shown display / end-to-end
/// numbers (`StatsOverlay.osFloorMs` — metrics report what Punktfunk controls), but sound has
/// to reach the ear when the light reaches the eye, and a floor-shaved reference would place
/// audio a whole latch period early on every device. Presentation policy, not physics.
video_e2e: Arc<AtomicU64>,
/// Always-on latch/display accumulator for the presenter's 1 Hz `pf-present` line —
/// independent of the HUD gate, so a HUD-off A/B stays measurable from logcat.
meter: Arc<super::presenter::PresentMeter>,
@@ -48,11 +58,13 @@ impl DisplayTracker {
pub(super) fn new(
stats: Arc<crate::stats::VideoStats>,
clock_offset: Arc<AtomicI64>,
video_e2e: Arc<AtomicU64>,
meter: Arc<super::presenter::PresentMeter>,
) -> Arc<DisplayTracker> {
Arc::new(DisplayTracker {
stats,
clock_offset,
video_e2e,
meter,
rendered: Mutex::new(VecDeque::new()),
})
@@ -105,7 +117,14 @@ pub(super) fn install_render_callback(
}
let sym = libc::dlsym(lib, c"AMediaCodec_setOnFrameRenderedCallback".as_ptr());
if sym.is_null() {
log::info!("decode: no render callback on this API level (<33) — no display stage");
// No confirmed present ⇒ no `display` stage AND no reference for the audio plane's A/V
// sync, which then stays inert and leaves the ring exactly as it was. The release
// instant is NOT substituted: releases target a future vsync, so it runs a whole latch
// period (8-21 ms measured) ahead of glass — well outside the loop's deadband, i.e. it
// would place audio early on every frame while looking like it was working.
log::info!(
"decode: no render callback on this API level (<33) — no display stage, no A/V sync"
);
return None;
}
std::mem::transmute::<*mut c_void, SetOnFrameRenderedFn>(sym)
@@ -145,8 +164,10 @@ pub(super) unsafe fn release_render_callback(ud: *const DisplayTracker) {
/// between the frame rendering and the (batchable) callback delivery — to subtract against the
/// receipt/decode stamps and the host capture pts. Records the HUD's `displayed` point:
/// `end-to-end` = capture→displayed (skew-corrected) and `display` = decoded→displayed
/// (single-clock local). Panic-free by construction (poison-proof lock, saturating math) — an
/// unwind out of an `extern "C"` fn would abort the process.
/// (single-clock local) — and publishes that end-to-end figure for the audio plane to align
/// against, which is the only place in the client that knows when a frame truly reached glass.
/// Panic-free by construction (poison-proof lock, saturating math) — an unwind out of an
/// `extern "C"` fn would abort the process.
unsafe extern "C" fn on_frame_rendered(
_codec: *mut ndk_sys::AMediaCodec,
userdata: *mut c_void,
@@ -186,13 +207,28 @@ unsafe extern "C" fn on_frame_rendered(
let latch_us = paired.and_then(|(_, r)| clamp(displayed_ns - r));
// Always-on half: the presenter's pf-present line reads these with the HUD off.
t.meter.note_latch(latch_us);
if !t.stats.enabled() {
return; // HUD hidden — skip the skew math + the stats lock
}
// The glass-to-glass figure, computed ABOVE the HUD gate: the audio plane steers its ring by it
// (see `video_e2e`), and a sync loop that only worked while the overlay was up would be off on
// the exact devices that report latency — on a Deck-class report the overlay is precisely what
// the field cannot reach. The cost is one relaxed load and some integer arithmetic per confirmed
// present (≤ the panel rate); the stats LOCK stays behind the gate, which is what that
// early-return was really protecting.
let e2e_ns =
displayed_ns + t.clock_offset.load(Ordering::Relaxed) as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
t.stats.note_displayed(e2e_us, display_us, latch_us);
// Same (0, 10 s) clamp as every other e2e sample — a vendor's first render callbacks can carry
// a garbage `system_nano`, and here that would step the audio ring rather than just a p95.
let e2e_valid = e2e_ns > 0 && e2e_ns < 10_000_000_000;
if e2e_valid {
t.video_e2e.store(e2e_ns as u64, Ordering::Relaxed);
}
if !t.stats.enabled() {
return; // HUD hidden — skip the stats lock
}
t.stats.note_displayed(
e2e_valid.then_some((e2e_ns / 1000) as u64),
display_us,
latch_us,
);
}
/// React to an output-format change by signalling the stream's HDR dataspace on the Surface (SDR
@@ -185,9 +185,12 @@ pub(super) fn run_sync(
// render = true are parked in the tracker; the OnFrameRendered callback pairs them with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
// The `video_e2e` cell is the audio plane's alignment reference (see `DisplayTracker`): this
// legacy loop feeds it too, so A/V sync works with "Low-latency mode" off as well.
let tracker = DisplayTracker::new(
stats.clone(),
clock_offset.clone(),
client.video_e2e_shared(),
std::sync::Arc::new(super::presenter::PresentMeter::new()),
);
let render_cb = install_render_callback(&codec, &tracker);
+15 -2
View File
@@ -33,8 +33,21 @@ pub(super) fn now_monotonic_ns() -> i64 {
};
// SAFETY: `clock_gettime` with a valid out-pointer is an always-safe syscall.
unsafe { libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut ts) };
// Explicit widening: timespec's fields are 32-bit on armv7 (time_t/c_long).
ts.tv_sec as i64 * 1_000_000_000 + ts.tv_nsec as i64
// Explicit widening: `timespec`'s fields are 32-bit on armv7 (`time_t`/`c_long`) and 64-bit on
// arm64, so these casts are REQUIRED on one shipping ABI and redundant on the other.
//
// `:kit:cargoNdkClippy` lints both widths, so it sees the redundant half and flags it; taking
// its advice would break the 32-bit build, which is the ABI for the many 32-bit Google TV /
// Android TV boxes this client targets. `i64::from`/`.into()` do not escape it either — they
// just trade `unnecessary_cast` for `useless_conversion` on the 64-bit side. So the cast stays
// and the lint is answered here rather than in whichever build breaks first.
#[allow(
clippy::unnecessary_cast,
reason = "required on 32-bit ABIs; redundant only on 64-bit"
)]
{
ts.tv_sec as i64 * 1_000_000_000 + ts.tv_nsec as i64
}
}
/// One upcoming frame timeline (API 33+ payload): when SurfaceFlinger expects to present the
+10 -7
View File
@@ -26,6 +26,15 @@ use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, SyncSender, TryS
use std::sync::Arc;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
/// What one capture open attempt yields: the stream, plus both halves of the PCM hand-off — the
/// receiver the encode worker drains and the sender that returns emptied buffers for reuse. Note
/// the pair is the mirror image of [`crate::audio::OpenedPlayback`]'s: here the callback produces
/// and the worker consumes.
///
/// Named rather than written inline for the same reason as that one — `clippy::type_complexity`,
/// now that the Android target is actually linted (`:kit:cargoNdkClippy`).
type OpenedCapture = ndk::audio::Result<(AudioStream, Receiver<Vec<f32>>, SyncSender<Vec<f32>>)>;
const CHANNELS: usize = 1;
const SAMPLE_RATE: i32 = 48_000;
/// 10 ms per channel @ 48 kHz — half the desktop clients' 20 ms frame, trading a little Opus
@@ -84,13 +93,7 @@ impl MicCapture {
// One open attempt at a given sharing mode (same pattern as [`crate::audio`]: `open_stream`
// consumes the builder AND the callback, so each try rebuilds the channels it captures).
let try_open = |sharing: AudioSharingMode,
voice: bool|
-> ndk::audio::Result<(
AudioStream,
Receiver<Vec<f32>>,
SyncSender<Vec<f32>>,
)> {
let try_open = |sharing: AudioSharingMode, voice: bool| -> OpenedCapture {
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list, mirroring the playback path: the realtime capture callback must
// not touch the allocator (Android's Scudo has unbounded malloc/free tail latency — an
+2 -2
View File
@@ -408,8 +408,8 @@ pub(crate) unsafe fn self_test(fd: i32, seconds: i32, hz: i32) -> i32 {
frame.fill(0);
// Channels 2 and 3 are the voice coils; the speaker pair stays silent so a pass is
// unambiguously FELT rather than merely audible.
for c in 2..channels {
frame[c] = sample;
for slot in frame.iter_mut().take(channels).skip(2) {
*slot = sample;
}
}
if let Err(e) = playback.write_interleaved(&chunk) {
@@ -361,6 +361,40 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepad
);
}
/// `NativeBridge.nativePadMotionReaches(handle, declaredPref)` — whether motion sent for a pad that
/// declared `declaredPref` (the `GamepadPref` wire byte it passed to `nativeSendGamepadArrival`) can
/// actually reach the game, or would be decoded and dropped by a host backend with no motion plane.
///
/// The whole question is answered here rather than in Kotlin so the reasoning lives in exactly one
/// place — [`punktfunk_core::config::pad_motion_reaches`], which carries the argument and the tests.
/// A third transcription of it would be a third thing to get subtly wrong, and every way of getting
/// it wrong is silent: too strict kills a working gyro, too lax keeps ~250 Hz of samples flowing
/// into a host that drops every one.
///
/// A `0` handle answers `true` — "don't suppress" is the safe answer when we cannot tell, matching
/// the `Auto` rule inside the predicate itself.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativePadMotionReaches(
_env: JNIEnv,
_this: JObject,
handle: jlong,
declared_pref: jint,
) -> jboolean {
if handle == 0 {
return 1;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; both fields are plain Copy
// values read behind `&self`.
let h = unsafe { &*(handle as *const SessionHandle) };
let declared =
punktfunk_core::config::GamepadPref::from_u8(declared_pref.clamp(0, u8::MAX as jint) as u8);
u8::from(punktfunk_core::config::pad_motion_reaches(
declared,
h.client.requested_gamepad,
h.client.resolved_gamepad,
))
}
/// `NativeBridge.nativeSendGamepadRemove(handle, pad)` — signal that wire pad index `pad` was
/// unplugged so the host tears its virtual device down. `pad` (rides `flags`) is the only field; the
/// core stamps the per-pad seq (in the snapshot seq space, so a reordered snapshot can't resurrect the
+16 -4
View File
@@ -177,12 +177,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD
/// (unified stats spec, `design/stats-unification.md`). Returns 33 doubles
/// (unified stats spec, `design/stats-unification.md`). Returns 35 doubles
/// `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
/// netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
/// e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
/// feedP50Ms, codecP50Ms, skippedOverflowWindow]`
/// feedP50Ms, codecP50Ms, skippedOverflowWindow, audioBufferMs, audioAvOffsetMs]`
/// (the flags are 1.0/0.0; indexes 021 match the previous 22-double layout — 013 the original
/// 14-double one with the latency pair re-based to the end-to-end capture→decoded headline, 14/15
/// the stage p50s tiling it: `host+network` = capture→received, `decode` = received→decoded; 16/17
@@ -203,7 +203,10 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
/// received→queued (hand-off + input-slot wait) at 30 and `codec` = queued→decoded (codec-pure,
/// from the AU's last piece) at 31, both 0.0 when no sample landed (sync loop); 32 is the
/// parked-AU overflow subset of the window's `skipped` at 19 (decoder fell behind, vs benign
/// newest-wins pacing)), or `null` when no decode thread is running.
/// newest-wins pacing); 33/34 are the AUDIO plane's latency — the playback ring's live depth in ms
/// and the A/V sync loop's smoothed offset in ms (positive = audio behind the picture) — both live
/// gauges rather than windowed samples, like the cumulative drop total at 9), or `null` when no
/// decode thread is running.
/// Poll ~1 Hz from the UI; each call
/// resets the measurement window. Not android-gated — pure `jni` + connector reads, so it links on
/// the host build too (Kotlin only ever calls it on device).
@@ -227,7 +230,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
.drain(h.client.frames_dropped(), h.client.fec_recovered_shards());
let mode = h.client.mode();
let color = h.client.color;
let buf: [f64; 33] = [
let buf: [f64; 35] = [
snap.fps,
snap.mbps,
snap.e2e_p50_ms,
@@ -281,6 +284,15 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
snap.feed_p50_ms,
snap.codec_p50_ms,
snap.skipped_overflow as f64,
// The audio plane's own latency (`design/audio-latency-overhaul.md`): how much decoded
// audio is queued ahead of the speaker, and where the A/V sync loop measures that
// PUTS it relative to the picture (+ = audio behind). Both, because a deep ring on a
// jittery link is correct behaviour and only the offset tells that apart from audio
// simply held late. Live gauges written by the audio thread — before this the whole
// plane published nothing any surface could render, so a "the audio delay is way too
// high" report had no instrument behind it at all.
h.client.audio_buffer_ms() as f64,
h.client.audio_av_offset_ms() as f64,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
+8
View File
@@ -19,6 +19,14 @@
<array>
<string>_punktfunk._udp</string>
</array>
<!-- NOTE: there is deliberately NO NSAppTransportSecurity dict here. ATS stays fully ON.
The host is self-signed at a user-supplied address, which default ATS can never accept
(it exempts only .local, unqualified names, and RFC1918/link-local literals — notably NOT
Tailscale's 100.64/10 CGNAT range), so the management API talks over MgmtTransport
(Network.framework), which is outside the URL loading system and pins the host by
SHA-256 fingerprint instead. That leaves cover-art CDN fetches as the app's only
URLSession traffic, and they keep the full ATS policy — which is the whole reason not to
reach for NSAllowsArbitraryLoads here. See MgmtTransport.swift. -->
<!-- Background keep-alive (opt-in, iOS/iPadOS): the ONLY sanctioned way to keep the long-lived
QUIC socket + pump-thread set alive while backgrounded is the audio background mode, backed
by the session's real, audible remote audio (AVAudioEngine keeps rendering). Video decode is
@@ -384,7 +384,13 @@ struct ContentView: View {
.frame(minWidth: 940, minHeight: 620)
}
#else
.fullScreenCover(item: $libraryTarget) { host in
// iOS: the cover is the TOUCH UI's presentation only. In gamepad mode the library is one
// of GamepadHomeView's in-place layers (the console shell no bottom-up cover), so the
// proxy hides the target from the cover while that mode owns it; every writer (Y on a
// tile, `returnToLibrary`) keeps writing the same `libraryTarget` either way, and a
// controller arriving or leaving mid-browse hands the open library to whichever
// presentation the new mode owns.
.fullScreenCover(item: touchLibraryTarget) { host in
NavigationStack {
LibraryView(store: store, host: host, onLaunch: { launchTitle(host, $0) })
}
@@ -401,6 +407,14 @@ struct ContentView: View {
Binding(get: { deepLinkNotice != nil }, set: { if !$0 { deepLinkNotice = nil } })
}
/// The iOS library cover's item: `libraryTarget`, hidden while the gamepad shell presents
/// the library in place (see the cover's comment).
private var touchLibraryTarget: Binding<StoredHost?> {
Binding(
get: { gamepadUIActive ? nil : libraryTarget },
set: { libraryTarget = $0 })
}
private var approvalChoicePresented: Binding<Bool> {
Binding(get: { approvalChoice != nil }, set: { if !$0 { approvalChoice = nil } })
}
@@ -558,7 +572,8 @@ struct ContentView: View {
GamepadHomeView(
store: store, model: model, discovery: discovery,
libraryTarget: $libraryTarget, waker: waker,
connect: { connect($0, profile: $1) }, connectDiscovered: connectDiscovered)
connect: { connect($0, profile: $1) }, connectDiscovered: connectDiscovered,
launchTitle: launchTitle)
} else {
HomeView(
store: store, model: model, discovery: discovery,
@@ -574,7 +589,8 @@ struct ContentView: View {
GamepadHomeView(
store: store, model: model, discovery: discovery,
libraryTarget: $libraryTarget, waker: waker,
connect: { connect($0, profile: $1) }, connectDiscovered: connectDiscovered)
connect: { connect($0, profile: $1) }, connectDiscovered: connectDiscovered,
launchTitle: launchTitle)
// On tvOS pairing/library normally present from HomeView's navigationDestinations
// which aren't mounted while the gamepad launcher is up. Give the launcher its
// own presenters (exactly one of the two homes is mounted at a time, so these can
@@ -764,6 +780,15 @@ struct ContentView: View {
// other in the seconds where they overlap.
.overlay(alignment: .bottom) {
VStack(spacing: 8) {
// A forwarded pad has a gyro this session's virtual controller cannot
// carry. Shown briefly at every stats tier and with the overlay off: the
// failure is otherwise completely silent the gyro just does nothing
// and the fix is a setting, so the hint has to name it. Every platform,
// including tvOS, where a DualSense is an ordinary way to play.
if captureEnabled, model.motionUnreachableKind != nil {
MotionUnreachableBadge()
.transition(.opacity.combined(with: .scale(scale: 0.9)))
}
#if !os(tvOS)
// Shown for as long as the mic is muted, at every stats tier and with the
// overlay off see MicMutedBadge. tvOS has no microphone to mute.
@@ -83,6 +83,11 @@ struct ConnectOverlay: View {
}
}
/// The overlay's text/glyph colour: the palette's ink in the console takeover over a pale
/// aurora, literal white was the one console surface that stayed white-on-white and white
/// in the touch modal, whose branch is deliberately forced dark over a black scrim.
private var overlayFG: Color { gamepadUI ? ink.fg : .white }
@ViewBuilder private func content(_ phase: Phase) -> some View {
// The takeover carries larger type than the compact modal.
let titleSize: CGFloat = gamepadUI ? 24 : 19
@@ -90,21 +95,24 @@ struct ConnectOverlay: View {
VStack(spacing: gamepadUI ? 16 : 14) {
switch phase {
case .connecting(let name):
ProgressView().controlSize(.large).tint(.white)
ProgressView().controlSize(.large).tint(overlayFG)
Text("Connecting to \(name)")
.font(.geist(titleSize, .bold, relativeTo: .title3)).foregroundStyle(.white)
.font(.geist(titleSize, .bold, relativeTo: .title3)).foregroundStyle(overlayFG)
.multilineTextAlignment(.center)
Text("Establishing a secure connection…")
.font(.geist(bodySize, relativeTo: .caption)).foregroundStyle(.white.opacity(0.6))
.font(.geist(bodySize, relativeTo: .caption))
.foregroundStyle(overlayFG.opacity(0.6))
Button("Cancel") { onCancelConnect() }.buttonStyle(.bordered).padding(.top, 6)
case .waking(let w) where w.timedOut:
Image(systemName: "moon.zzz.fill")
.font(.system(size: gamepadUI ? 40 : 34)).foregroundStyle(.white.opacity(0.9))
.font(.system(size: gamepadUI ? 40 : 34))
.foregroundStyle(overlayFG.opacity(0.9))
Text("\(w.hostName) didn't wake")
.font(.geist(titleSize, .bold, relativeTo: .title3)).foregroundStyle(.white)
.font(.geist(titleSize, .bold, relativeTo: .title3)).foregroundStyle(overlayFG)
.multilineTextAlignment(.center)
Text("It may still be booting, or it's powered off / off this network.")
.font(.geist(bodySize, relativeTo: .caption)).foregroundStyle(.white.opacity(0.6))
.font(.geist(bodySize, relativeTo: .caption))
.foregroundStyle(overlayFG.opacity(0.6))
.multilineTextAlignment(.center)
HStack(spacing: 12) {
Button("Cancel") { waker.cancel() }.buttonStyle(.bordered)
@@ -112,12 +120,13 @@ struct ConnectOverlay: View {
}
.padding(.top, 6)
case .waking(let w):
ProgressView().controlSize(.large).tint(.white)
ProgressView().controlSize(.large).tint(overlayFG)
Text("Waking \(w.hostName)")
.font(.geist(titleSize, .bold, relativeTo: .title3)).foregroundStyle(.white)
.font(.geist(titleSize, .bold, relativeTo: .title3)).foregroundStyle(overlayFG)
.multilineTextAlignment(.center)
Text("Waiting for it to come online · \(w.seconds)s")
.font(.geistFixed(bodySize)).foregroundStyle(.white.opacity(0.6)).monospacedDigit()
.font(.geistFixed(bodySize)).foregroundStyle(overlayFG.opacity(0.6))
.monospacedDigit()
// A wake-only wait (no dial after) offers "Stop Waiting"; a wake-&-connect is "Cancel".
Button(w.connectsAfter ? "Cancel" : "Stop Waiting") { waker.cancel() }
.buttonStyle(.bordered).padding(.top, 6)
@@ -14,7 +14,15 @@ import SwiftUI
struct GamepadAddHostView: View {
@Environment(\.gamepadInk) private var ink
@Environment(\.dismiss) private var dismiss
@Environment(\.gamepadHostedInShell) private var hostedInShell
let onAdd: (StoredHost) -> Void
/// How the in-place shell (iOS) closes this screen; nil (the macOS sheet, the tvOS cover)
/// falls back to the environment dismiss. Declared AFTER `onAdd` so the existing trailing-
/// closure call sites keep binding to it, not to this.
var close: (() -> Void)?
/// Whether this screen owns the controller false while the shell is mid-transition or the
/// connect takeover is up (see GamepadSettingsView's twin).
var controllerActive = true
#if os(iOS)
/// `.compact` in a landscape phone window tighter chrome so the keyboard tray still fits.
@@ -36,8 +44,8 @@ struct GamepadAddHostView: View {
items: rows,
focusID: $focusID,
onActivate: { activate(id: $0.id) },
onBack: { dismiss() },
isActive: editing == nil
onBack: { performClose() },
isActive: controllerActive && editing == nil
) { row, focused in
rowView(row, focused: focused)
.frame(maxWidth: GamepadFormMetrics.rowMaxWidth)
@@ -45,7 +53,8 @@ struct GamepadAddHostView: View {
}
.frame(maxWidth: .infinity)
.safeAreaInset(edge: .top, spacing: 0) {
VStack(spacing: 4) {
VStack(alignment: .leading, spacing: gamepadHeaderSpacing(compact: compact)) {
// Leading, like every gamepad heading and no close chrome (B is the exit).
Text("Add Host")
.font(.geist(gamepadTitleSize(compact: compact), .bold, relativeTo: .title))
.foregroundStyle(ink.fg)
@@ -54,14 +63,14 @@ struct GamepadAddHostView: View {
+ "for everything else.")
.font(.geist(GamepadFormMetrics.detailFont, relativeTo: .caption))
.foregroundStyle(ink.fg(0.55))
.multilineTextAlignment(.center)
.frame(maxWidth: GamepadFormMetrics.rowMaxWidth * 0.72)
.multilineTextAlignment(.leading)
.frame(maxWidth: GamepadFormMetrics.rowMaxWidth * 0.72, alignment: .leading)
}
}
.padding(.horizontal, 24)
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.frame(maxWidth: .infinity)
.overlay(alignment: .topTrailing) { closeButton.padding(.top, 20).padding(.trailing, 20) }
.padding(.bottom, gamepadTitleBottomPadding(compact: compact))
.frame(maxWidth: .infinity, alignment: .leading)
.background { GamepadTrayScrim(edge: .top) }
}
.safeAreaInset(edge: .bottom, spacing: 0) {
@@ -73,7 +82,10 @@ struct GamepadAddHostView: View {
.background { GamepadTrayScrim(edge: .bottom) }
}
// No aurora the same clean Liquid-Glass-over-dark base as the gamepad settings screen.
.background { GamepadFormBackground() }
// Hosted in the shell, the field is the shell's (see GamepadSettingsView's twin).
.background {
if !hostedInShell { GamepadFormBackground() }
}
// Publish the palette's ink to this screen (text, glass, accent, scrims) a
// pale palette flips all of them, and no leaf should have to read the setting.
.gamepadPaletteInk()
@@ -81,6 +93,18 @@ struct GamepadAddHostView: View {
.onChange(of: port) { _, value in
if value.count > 5 { port = String(value.prefix(5)) }
}
#if !os(tvOS)
// The visible close is gone (a gamepad UI exits with B) this keeps a hardware
// keyboard's Esc and the macOS sheet's cancel working without chrome.
.background {
Button("Cancel") { performClose() }
.keyboardShortcut(.cancelAction)
.buttonStyle(.plain)
.frame(width: 0, height: 0)
.opacity(0)
.accessibilityHidden(true)
}
#endif
#if os(tvOS)
// tvOS types with the SYSTEM fullscreen keyboard (TVTextEntry) instead of the custom
// tray the remote and the pad both drive it natively. Same `editing` state as the
@@ -141,22 +165,10 @@ struct GamepadAddHostView: View {
#endif
}
/// Touch/click fallback for closing the controller path is B, a hardware keyboard's Esc
/// rides the cancel action.
private var closeButton: some View {
Button { dismiss() } label: {
Image(systemName: "xmark")
.font(.system(size: GamepadFormMetrics.closeFont, weight: .semibold))
.foregroundStyle(ink.fg)
.frame(width: GamepadFormMetrics.closeSide, height: GamepadFormMetrics.closeSide)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
.buttonStyle(.plain)
#if !os(tvOS)
.keyboardShortcut(.cancelAction) // unavailable on tvOS (Menu is the cancel there)
#endif
.accessibilityLabel("Cancel")
/// Close this screen through whichever mechanism presents it: the shell's layer pop on iOS,
/// the environment dismiss under a macOS sheet / tvOS cover.
private func performClose() {
if let close { close() } else { dismiss() }
}
// MARK: - Rows
@@ -237,7 +249,7 @@ struct GamepadAddHostView: View {
name: name.trimmingCharacters(in: .whitespaces),
address: address.trimmingCharacters(in: .whitespaces),
port: UInt16(port) ?? 9777))
dismiss()
performClose()
default:
openKeyboard(id)
}
@@ -55,7 +55,14 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
/// otherwise poll the SAME controller at once driving both. The parent sets this false while
/// something is presented on top so only the front-most carousel consumes the gamepad.
var isActive: Bool = true
@ViewBuilder let card: (Item) -> Card
/// Whether the cards are worth showing off yet the entrance holds until this is true. The
/// library passes "the first covers have their artwork" (see LibraryCoverflowView); anything
/// whose cards are ready the moment they mount leaves it alone.
var contentReady: Bool = true
/// Builds one card. The `CardEntrance` handed along is the card's share of the strip's
/// arrival, and the caller MUST apply it (`.modifier(entrance)`) *underneath* its own
/// `.scrollTransition` see `CardEntrance` for why that placement is load-bearing.
@ViewBuilder let card: (Item, CardEntrance) -> Card
@State private var input = GamepadMenuInput(manager: .shared)
@State private var haptics = MenuHaptics(manager: .shared)
@@ -83,6 +90,26 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
/// confirm and end-stop events (moves trigger on `cursor`).
@State private var activateTick = 0
@State private var boundaryTick = 0
/// The strip's entrance, as ONE timeline: 0 = every card still away, 1 = every card landed
/// (see `CardEntrance`, which slices its own window out of this). Animated exactly once per
/// mount a strip that re-played its entrance every time a screen popped off the top of it
/// would be noise, and the shell's push/pop carries that motion already. So it plays when a
/// screen is entered: the launcher when the gamepad UI comes up, the coverflow each time the
/// library opens (its layer mounts fresh).
///
/// One animated Double rather than a Bool behind per-card `.animation(_:value:)` modifiers,
/// because those modifiers wrap the caller's card INCLUDING its `.scrollTransition` and a
/// delayed spring flipping while the scroll view was still settling captured the transition's
/// own per-frame phase updates, stranding the centred card in a half-receded state until the
/// next scroll re-drove it. Nothing here wraps the card in an animation at all.
@State private var entranceProgress: Double = 0
/// Which card the entrance fans out from the cursor as it stood when the strip was armed,
/// so a restored selection assembles around where the eye already is instead of sweeping in
/// from the left.
@State private var entranceAnchor = 0
/// The entrance has been scheduled; it plays exactly once per mount.
@State private var entranceArmed = false
@Environment(\.accessibilityReduceMotion) private var reduceMotion
/// Read-back from a touch drag is honoured only once the gamepad has been quiet this long
/// (longer than a move animation, so overlapping held-stick moves never let it through).
@@ -94,24 +121,27 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
ScrollViewReader { proxy in
ScrollView(.horizontal) {
HStack(spacing: spacing) {
ForEach(items) { item in
// Enumerated for the entrance stagger only identity stays `item.id`,
// which is what `.scrollTargetLayout()` and `scrollPosition` key on.
ForEach(Array(items.enumerated()), id: \.element.id) { idx, item in
#if os(tvOS)
// A focusable Button per card: the focus engine does the navigating
// (remote swipes and pad dpad alike), select activates. The bare style
// below keeps the tile's own look the `.scrollTransition` center pop
// is the focus treatment, since focus and center track each other.
Button { activate(item) } label: {
card(item)
card(item, entrance(idx))
.frame(width: itemWidth)
}
.buttonStyle(ConsoleBareButtonStyle())
.focused($focusedID, equals: item.id)
.id(item.id)
#else
card(item)
card(item, entrance(idx))
.frame(width: itemWidth)
.contentShape(Rectangle())
.onTapGesture { tap(item) }
.id(item.id) // explicit scroll-target identity for scrollPosition
#endif
}
}
@@ -165,7 +195,10 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
reconcile()
wire()
if isActive { input.start() }
armEntrance()
}
// The cards became worth showing (the library's covers got their art) play now.
.onChange(of: contentReady) { _, _ in armEntrance() }
.onDisappear {
input.stop()
haptics.stop()
@@ -200,9 +233,55 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
.onChange(of: items.map(\.id)) { _, _ in
reconcile()
wire()
// A strip that mounted empty (its content arrived after) still gets its entrance.
armEntrance()
}
}
// MARK: - Entrance
/// Run the entrance, once, as soon as the strip is mounted AND its cards are worth showing.
///
/// Deferred one runloop turn ON PURPOSE: a state change made inside `onAppear` lands in the
/// same transaction as the view's insertion, where SwiftUI runs with animations disabled so
/// the cards would simply BE there. Note the failure mode is benign either way: progress
/// reaching 1 without animating leaves every card at exact identity, never stranded.
private func armEntrance() {
guard !entranceArmed, contentReady, !items.isEmpty else { return }
entranceArmed = true
// After `reconcile`, so the fan-out anchors on the seeded/restored cursor.
entranceAnchor = cursor
// Not just the next runloop turn (a change made inside `onAppear` lands in the
// insertion's transaction, where animations are disabled) but a couple of frames: the
// GeometryReader's first pass can report no width at all, so the strip has to lay out
// for real and the scroll view has to centre itself on the cursor before this starts.
// Cards are invisible until then (progress 0 opacity 0), so the wait never shows.
DispatchQueue.main.asyncAfter(deadline: .now() + 0.05) {
// Linear on purpose: the master timeline is a clock, and each card eases its OWN
// slice of it (see `CardEntrance`) a spring here would warp every card's curve.
withAnimation(
reduceMotion ? .easeOut(duration: 0.28) : .linear(duration: CardEntrance.total)
) {
entranceProgress = 1
}
}
}
/// The card's share of the strip's entrance: it swings in on the drum, the anchored card
/// landing first and its neighbours fanning outward to either side.
private func entrance(_ idx: Int) -> CardEntrance {
// Capped so a several-hundred-title library never queues a card behind a visibly long
// wait everything past the cap lands together, well off-screen anyway.
let delay = min(CardEntrance.maxDelay, Double(abs(idx - entranceAnchor)) * 0.07)
return CardEntrance(
progress: entranceProgress,
start: delay / CardEntrance.total,
// Never zero: the anchor is the card the eye is ON, so it must swing like the rest
// giving it "no rotation" left the one card you actually watch merely sliding up.
side: idx < entranceAnchor ? -1 : 1,
reduceMotion: reduceMotion)
}
// MARK: - Input wiring
private func wire() {
@@ -346,4 +425,87 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
withAnimation(.spring(response: 0.34, dampingFraction: 0.7).delay(0.1)) { bumpOffset = 0 }
}
}
/// How a card arrives when its strip does: turned away on the drum, small, low and invisible
/// then it swings flat, grows and rises into place on a spring soft enough to overshoot. Cards to
/// the left of the anchor hinge on their trailing edge and cards to its right on their leading
/// one, so the strip FANS OPEN from the cursor rather than sweeping past it; the anchor card
/// itself only grows, since it is already facing you. Each card carries its own delay (see
/// `entrance(_:)`) that stagger is what makes the strip read as one gesture instead of a
/// simultaneous flash, and it is the same hinge/perspective language the coverflow's own recede
/// speaks, so the arrival and the scrolling feel like one object.
///
/// APPLY THIS UNDERNEATH THE CARD'S OWN `.scrollTransition`, never around it. A scroll
/// transition derives its phase from the geometry of the view it wraps, so an entrance layered
/// on the OUTSIDE moves the very thing the transition is measuring: every card read as far from
/// centre for the whole travel, its phase pinned at fully-receded, and the centred card only
/// collapsed into its focused look as the entrance ended arriving as a jump. Underneath, the
/// transition measures a card that never moves and simply composes its own scale/rotation on top.
///
/// Transforms only nothing here touches layout, so the scroll view's snapping and the tvOS
/// focus engine are untouched either. Reduce Motion drops every bit of travel for a plain,
/// unstaggered cross-fade.
struct CardEntrance: ViewModifier, Animatable {
/// How long ONE card takes to travel, and the most any card waits before it starts.
static let perCard: Double = 0.6
static let maxDelay: Double = 0.42
/// The master timeline the carousel animates 0 1.
static var total: Double { perCard + maxDelay }
/// The interpolated master progress. `Animatable` is the whole point: SwiftUI hands this
/// modifier a fresh value every frame and re-runs `body`, so the card's transforms are a pure
/// FUNCTION of the clock. No `.animation` modifier wraps the card, so nothing here can catch
/// the caller's `.scrollTransition` mid-scroll and strand it.
var progress: Double
/// Where this card's window opens on that timeline, 01.
let start: Double
/// Which way the card swings in: -1 hinged on its trailing edge (it sits left of the anchor),
/// +1 hinged on its leading edge (right of it). Never 0 every card turns, including the
/// centred one.
let side: Double
let reduceMotion: Bool
var animatableData: Double {
get { progress }
set { progress = newValue }
}
func body(content: Content) -> some View {
// This card's own 01, sliced out of the master clock.
let span = Self.perCard / Self.total
let raw = min(max((progress - start) / span, 0), 1)
// The travel eases out with a whisker of overshoot, so a card settles rather than stops.
let travel = Self.easeOutBack(raw)
// The fade is FAR quicker than the travel it finishes in the first third of the window.
// Sharing one curve meant the card spent its whole swing at near-zero opacity and only
// the last few degrees ever showed, which is why this read as a small slide.
let fade = Self.easeOut(min(raw / 0.34, 1))
// Deep turn, well down, well shrunk the card is genuinely edge-on and travelling. The
// sign matches the coverflow's own recede (right of centre turns negative about its
// leading edge), so the arrival deepens the turn the card wears at rest and unwinds into
// it instead of swinging the opposite way.
let away = reduceMotion ? 0 : 1 - travel
return content
.opacity(reduceMotion ? raw : fade)
.scaleEffect(1 - 0.26 * away)
.rotation3DEffect(
.degrees(side * -64 * away),
axis: (x: 0, y: 1, z: 0),
anchor: .center,
perspective: 0.65)
.offset(y: 34 * away)
}
/// `1 - (1-t)³`, with a small overshoot past 1 before it settles.
private static func easeOutBack(_ t: Double) -> Double {
let c1 = 1.2, c3 = c1 + 1
let u = t - 1
return 1 + c3 * u * u * u + c1 * u * u
}
private static func easeOut(_ t: Double) -> Double {
let u = 1 - t
return 1 - u * u * u
}
}
#endif
@@ -23,22 +23,47 @@ func buttonGlyph(
/// Top padding for a gamepad screen's pinned title. macOS gets extra clearance the launcher
/// title sits right under the window titlebar and the settings/add-host sheets have no titlebar
/// at all, so the iOS value hugs the top edge there.
/// at all. The other values follow the console shell's rhythm (title top = 18 design units,
/// k-floored to 10 for a landscape phone): the title needs air to the screen edge or the whole
/// header reads pressed against the bezel, which the tab strip's extra band made obvious.
func gamepadTitleTopPadding(compact: Bool) -> CGFloat {
#if os(macOS)
26
#elseif os(tvOS)
24
#else
compact ? 4 : 10
compact ? 18 : 28
#endif
}
/// Padding under a gamepad screen's pinned header block (title, and the tab strip where there is
/// one) before the content: the console leaves ~14 units of air under its tab pills, and without
/// it the first row sits shoulder-to-shoulder with the header.
func gamepadTitleBottomPadding(compact: Bool) -> CGFloat {
#if os(tvOS)
16
#else
compact ? 8 : 12
#endif
}
/// Spacing between a header's stacked elements (title over tab strip / subtitle).
func gamepadHeaderSpacing(compact: Bool) -> CGFloat {
#if os(tvOS)
13
#else
compact ? 6 : 10
#endif
}
/// Point size for a gamepad screen's pinned title: TV-large on tvOS (read from the couch), the
/// in-hand compact-aware sizes elsewhere.
/// in-hand compact-aware sizes elsewhere. Sized as a proper screen heading the field verdict
/// on the smaller first cut was "way too small" once the title moved off-centre.
func gamepadTitleSize(compact: Bool) -> CGFloat {
#if os(tvOS)
44
#else
compact ? 20 : 30
compact ? 24 : 34
#endif
}
@@ -58,8 +83,7 @@ enum GamepadFormMetrics {
static let rowCorner: CGFloat = 18
static let rowMaxWidth: CGFloat = 920
static let detailFont: CGFloat = 19
static let closeFont: CGFloat = 20
static let closeSide: CGFloat = 48
static let bandWidth: CGFloat = 380
#else
static let headerFont: CGFloat = 12
static let labelFont: CGFloat = 16
@@ -72,8 +96,8 @@ enum GamepadFormMetrics {
static let rowCorner: CGFloat = 14
static let rowMaxWidth: CGFloat = 620
static let detailFont: CGFloat = 13
static let closeFont: CGFloat = 14
static let closeSide: CGFloat = 34
/// The option band's (GamepadOptionBand) fixed stage inside a choice row.
static let bandWidth: CGFloat = 240
#endif
}
@@ -147,8 +171,21 @@ struct GamepadHintBar: View {
/// header). Honors Reduce Motion by freezing the field at a fixed phase.
struct GamepadScreenBackground: View {
@Environment(\.gamepadInk) private var ink
/// Quiet the field for a form screen (see the type comment).
var calm = false
/// How far toward the form screens' quiet the field sits: 0 = the launcher's full aurora,
/// 1 = calm, fractional mid-chase. Continuous (not a Bool) so the in-place shell can CHASE
/// it during a push/pop the console does the same with its `bg_mix` and every
/// calm-dependent factor below rides an `.opacity` modifier, which animates reliably where
/// re-built gradient stops do not.
var calmMix: Double
/// The Bool spelling every non-shell call site uses (see the type comment for `calm`).
init(calm: Bool = false) {
calmMix = calm ? 1 : 0
}
init(calmMix: Double) {
self.calmMix = calmMix
}
@Environment(\.accessibilityReduceMotion) private var reduceMotion
@AppStorage(DefaultsKey.uiPalette) private var paletteID = "violet"
@@ -184,21 +221,22 @@ struct GamepadScreenBackground: View {
// ±8° over ~5 min the whole field very slowly warms and cools.
.hueRotation(.degrees(sin(t * 0.021) * 8))
// Calm = col·0.6 + ground·0.4: over the ground, `.opacity` IS the multiply
.opacity(calm ? 0.6 : 1)
if calm {
// and a plusLighter wash of the palette's own ground IS the add. Chosen so the
// ground lands exactly where it was and the bright pools come down to meet it.
Self.color(palette.ground)
.opacity(0.4)
.blendMode(.plusLighter)
}
.opacity(1 - 0.4 * calmMix)
// and a plusLighter wash of the palette's own ground IS the add. Chosen so the
// ground lands exactly where it was and the bright pools come down to meet it.
// Mounted unconditionally at opacity 0 a plusLighter layer contributes nothing,
// and an always-present layer is what lets the mix animate instead of popping.
Self.color(palette.ground)
.opacity(0.4 * calmMix)
.blendMode(.plusLighter)
// Cinematic vignette: the edges settle toward the scrim so the cards sit in the
// pooled light. Soft (extends past the frame) so the corners deepen rather than
// crush. Halved under calm: a launcher's cards sit in the pooled centre, but a form
// screen's rows run out toward the edges, where crushing them just eats the list.
EllipticalGradient(
colors: [.clear, scrim.opacity((calm ? 0.21 : 0.42) * strength)],
colors: [.clear, scrim.opacity(0.42 * strength)],
center: .center, startRadiusFraction: 0.25, endRadiusFraction: 1.15)
.opacity(1 - 0.5 * calmMix)
// Legibility grounding for the pinned title (top) and hint pill (bottom). This one
// works on the field itself (it's the backdrop's bottom layer nothing behind it to
// blur), so it stays a gradient, just a light one.
@@ -342,20 +380,39 @@ struct GamepadTrayScrim: View {
// to keep the pinned title legible, so it has to frost dark under white ink and
// light under dark ink.
.environment(\.colorScheme, ink.isLight ? .light : .dark)
// Fade the whole blur out toward the content so it dissolves rather than ending on a line.
// Sink the material's grey luminance lift toward the palette's shade (black on a
// dark field field ask: the frost read GREY over the aurora). Inside the mask, so
// the tint dissolves with the blur.
.overlay(ink.shade(0.35))
// Fade the whole blur out toward the content so it dissolves rather than ending on a
// line. The strong region sits deep (0.65) because the first stretch of the gradient
// now runs over the fixed 80 pt outer overhang below.
.mask {
LinearGradient(
stops: [
.init(color: .black, location: 0),
.init(color: .black.opacity(0.9), location: 0.5),
.init(color: .black.opacity(0.92), location: 0.65),
.init(color: .clear, location: 1),
],
startPoint: fromEdge, endPoint: toContent)
}
// Grow past the tray so the fade-to-clear happens OUTSIDE its bounds the tray's own
// text always sits on the strong part, rows blur out before they reach it.
.padding(edge == .top ? .bottom : .top, -32)
.ignoresSafeArea()
// text always sits on the strong part, rows blur out before they reach it. The bottom
// gets the longer runway: its tray sits over SCROLLING rows plus the detail line, and
// the field verdict on the short reach was rows colliding visibly with the legend.
.padding(edge == .top ? .bottom : .top, edge == .top ? -44 : -72)
// Full-bleed by LAYOUT, not by `.ignoresSafeArea()`: safe-area expansion resolves a
// beat after insertion (outside any geometry group and outside this view's own
// transaction), which is exactly the pop the field kept seeing vertically first,
// then, once the vertical runway became padding, on the X axis alone (the landscape
// side insets). 80 pt clears every inset on every device; backgrounds never clip,
// so the overhang simply draws.
.padding(edge == .top ? .top : .bottom, -80)
.padding(.horizontal, -80)
// And the shape must NEVER animate: mounted inside a pushed shell layer, any late
// geometry would ride the push's transaction and visibly grow into place. The
// layer's own fade/slide still carries the scrim; only its SHAPE is pinned.
.transaction { $0.animation = nil }
}
}
@@ -74,6 +74,9 @@ struct GamepadHomeView: View {
@ObservedObject var waker: HostWaker
let connect: (StoredHost, ProfileSelection) -> Void
let connectDiscovered: (DiscoveredHost) -> Void
/// Launch a library title on a host the in-place library layer's activate path (iOS; the
/// cover/sheet presentations wire ContentView's `launchTitle` into LibraryView themselves).
let launchTitle: (StoredHost, String) -> Void
/// The profile catalog pinned host+profile combos render as their own tiles here, which is
/// how a controller picks a profile: one focus-and-press instead of a menu (design §5.4).
@@ -93,29 +96,56 @@ struct GamepadHomeView: View {
private let compact = false // no size classes on macOS; the window minimum keeps room
#endif
@ObservedObject private var gamepads = GamepadManager.shared
@Environment(\.accessibilityReduceMotion) private var reduceMotion
@State private var selection: GamepadHomeTarget?
@State private var showSettings = false
@State private var showAddHost = false
/// The console's input drop: true for the transition's 0.26 s, during which NO layer polls
/// the controller a double-tapped A can't push two screens, and the held button that
/// caused the change is long released before the next poller starts (whose own
/// `needsSnapshot` seed swallows it if not).
@State private var transitioning = false
/// Guards the gate's release against an interrupted transition: only the newest hold clears.
@State private var transitionEpoch = 0
var body: some View {
GeometryReader { geo in
hero(for: geo.size)
// The in-place shell (see GamepadShell.swift): the launcher is the base layer, the
// current sub-screen a transparent layer over it, both over ONE persistent backdrop
// that never unmounts a push slides the screen up out of a fade while the launcher
// recedes underneath, the console's own choreography. On macOS/tvOS `topScreen` is
// constantly nil and this ZStack degenerates to the plain launcher, presented over by
// the sheets/covers below exactly as before.
ZStack {
homeLayer
.opacity(covered ? 0 : 1)
.scaleEffect(covered ? GamepadShellMotion.underScale : 1)
// The covers used to swallow touch; the recessed layer must too.
.allowsHitTesting(!covered)
#if os(iOS)
if let screen = topScreen {
screenLayer(screen)
// Settle the screen's internal layout before the insertion animates, so
// descendants never lerp from a half-resolved first frame. (Not sufficient
// for the tray blurs on its own safe-area expansion resolves outside a
// geometry group; GamepadTrayScrim pins its own geometry too.)
.geometryGroup()
.zIndex(1)
.id(screen.id)
.transition(.gamepadScreen(slide: GamepadShellMotion.slide(compact: compact)))
}
#endif
}
// Pinned inside the safe area, out of the carousel's vertical budget never clipped.
.safeAreaInset(edge: .top, spacing: 0) {
titleBar
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
// Value-keyed rather than `withAnimation` at the triggers: pushes originate outside
// this view too (`model.returnToLibrary` writes `libraryTarget`), and keying on the
// derived id catches every writer. Reduce Motion snaps.
.animation(reduceMotion ? nil : GamepadShellMotion.screen, value: topScreenID)
// ONE living field for every layer, still a `.background` (the layout rule in this
// file's header). Its calm is CHASED between the launcher's aurora and the form
// screens' quiet, never crossfaded per screen the console's `bg_mix`.
.background {
GamepadScreenBackground(calmMix: calmTarget)
.animation(reduceMotion ? nil : GamepadShellMotion.calm, value: calmTarget)
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
GamepadHintBar(hints: hints)
// Equal distance from the left and bottom edges the pill's corner inset was the
// real asymmetry (leading 22 vs bottom 10), not its internal padding.
.padding(.leading, compact ? 12 : 18)
.padding(.bottom, compact ? 12 : 18)
.padding(.top, compact ? 4 : 8)
}
.background { GamepadScreenBackground() }
// Publish the palette's ink to this screen (text, glass, accent, scrims) a
// pale palette flips all of them, and no leaf should have to read the setting.
.gamepadPaletteInk()
@@ -129,6 +159,17 @@ struct GamepadHomeView: View {
try? await Task.sleep(for: .seconds(10))
}
}
#if os(iOS)
.onChange(of: topScreenID) { _, _ in
transitionEpoch += 1
let epoch = transitionEpoch
transitioning = true
let hold = reduceMotion ? 0.05 : GamepadShellMotion.duration + 0.02
DispatchQueue.main.asyncAfter(deadline: .now() + hold) {
if epoch == transitionEpoch { transitioning = false }
}
}
#endif
// The remote's Play/Pause mirrors the pad's X (Settings): the focus engine never surfaces
// X, and historically tvOS maps a pad's X to this same press the poll and this command
// double-firing just sets the same Bool twice.
@@ -136,8 +177,9 @@ struct GamepadHomeView: View {
.onPlayPauseCommand { showSettings = true }
#endif
// The settings / add-host screens take over the controller (the carousel's `isActive`
// gate above). iOS presents them full screen the immersive console feel; macOS has no
// fullScreenCover, so they become generously sized sheets over the dimmed launcher.
// gate above). macOS has no fullScreenCover they are generously sized sheets over the
// dimmed launcher; tvOS keeps its focus-engine covers. iOS needs nothing here: the
// shell's layers above ARE the presentation.
#if os(macOS)
.sheet(isPresented: $showSettings) {
GamepadSettingsView(store: store)
@@ -148,7 +190,7 @@ struct GamepadHomeView: View {
.frame(width: 660, height: 620)
}
.frame(minWidth: 640, minHeight: 420)
#else
#elseif os(tvOS)
.fullScreenCover(isPresented: $showSettings) { GamepadSettingsView(store: store) }
.fullScreenCover(isPresented: $showAddHost) {
GamepadAddHostView { store.add($0) }
@@ -156,6 +198,110 @@ struct GamepadHomeView: View {
#endif
}
// MARK: - The shell's layers (see GamepadShell.swift)
/// The launcher itself everything the pre-shell body was, minus the backdrop (hoisted to
/// the shell) and the presentation modifiers (below).
private var homeLayer: some View {
GeometryReader { geo in
hero(for: geo.size)
}
// Pinned inside the safe area, out of the carousel's vertical budget never clipped.
.safeAreaInset(edge: .top, spacing: 0) {
titleBar
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, gamepadTitleBottomPadding(compact: compact))
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
GamepadHintBar(hints: hints)
// Equal distance from the left and bottom edges the pill's corner inset was the
// real asymmetry (leading 22 vs bottom 10), not its internal padding.
.padding(.leading, compact ? 12 : 18)
.padding(.bottom, compact ? 12 : 18)
.padding(.top, compact ? 4 : 8)
}
}
#if os(iOS)
/// The screen the shell shows over the launcher derived from the same triggers every
/// platform sets, so `returnToLibrary`, the tiles, X and Y all keep writing what they wrote.
private var topScreen: GamepadScreen? {
if showSettings { return .settings }
if showAddHost { return .addHost }
if let host = libraryTarget { return .library(host) }
return nil
}
@ViewBuilder private func screenLayer(_ screen: GamepadScreen) -> some View {
// The layer owns the controller only once the push settles and nothing rides over the
// shell (the connect/wake takeover is an overlay in ContentView, above these layers).
let active = !transitioning && waker.waking == nil && model.phase != .connecting
Group {
switch screen {
case .settings:
GamepadSettingsView(
store: store,
close: { if !transitioning { showSettings = false } },
controllerActive: active)
case .addHost:
GamepadAddHostView(
onAdd: { store.add($0) },
close: { if !transitioning { showAddHost = false } },
controllerActive: active)
case .library(let host):
GamepadLibraryScreen(
store: store, host: host,
onLaunch: { launchTitle(host, $0) },
close: { if !transitioning { libraryTarget = nil } },
controllerActive: active)
}
}
.environment(\.gamepadHostedInShell, true)
}
#endif
private var covered: Bool {
#if os(iOS)
topScreen != nil
#else
false
#endif
}
private var topScreenID: String? {
#if os(iOS)
topScreen?.id
#else
nil
#endif
}
/// The backdrop's calm target: 1 under a form screen, 0 under the launcher/library. The
/// macOS sheets / tvOS covers mount their own calmed field, so the launcher behind them
/// keeps its aurora exactly what shipped.
private var calmTarget: Double {
#if os(iOS)
topScreen?.isForm == true ? 1 : 0
#else
0
#endif
}
/// Stop consuming the controller while another screen (or the connect/wake takeover) is on
/// top otherwise the launcher navigates behind it (invisibly on iPhone, visibly on iPad),
/// and a second A during a dial would launch a concurrent connect. `.connecting` covers the
/// takeover's Connecting phase; `waker.waking` its Waking phase. On iOS the shell adds the
/// transition's input drop, during which NOBODY polls.
private var homeOwnsController: Bool {
#if os(iOS)
topScreen == nil && !transitioning
&& waker.waking == nil && model.phase != .connecting
#else
libraryTarget == nil && !showSettings && !showAddHost
&& waker.waking == nil && model.phase != .connecting
#endif
}
// MARK: - Hero (carousel + detail), sized to fit the space between the pinned title and hints
@ViewBuilder private func hero(for size: CGSize) -> some View {
@@ -181,32 +327,27 @@ struct GamepadHomeView: View {
// MARK: - Chrome
private var titleBar: some View {
// The chip used to be a trailing `.overlay`, which reserves no width: on a portrait phone
// it sat directly on top of the centred title ("Select a Host" ran straight into the pad
// name). Laying it out as a row with a hidden mirror on the leading side keeps the title
// optically centred AND clear of the chip at every width; the title shrinks a little
// before it would ever truncate.
// Leading title (a console heading, not a floating label field ask), chip trailing.
// The old hidden-mirror trick existed only to keep a CENTRED title clear of the chip;
// a leading title needs none of it the flexible frame keeps the two apart, and the
// title shrinks a little before it would ever truncate.
HStack(spacing: 12) {
statusChip(hidden: true)
Text("Select a Host")
.font(.geist(gamepadTitleSize(compact: compact), .bold, relativeTo: .title))
.foregroundStyle(ink.fg)
.lineLimit(1)
.minimumScaleFactor(0.75)
.frame(maxWidth: .infinity)
statusChip(hidden: false)
.frame(maxWidth: .infinity, alignment: .leading)
statusChip
}
.padding(.horizontal, 20)
.padding(.horizontal, 24)
}
/// Which pad is driving this UI (name + battery) quiet, and only where there's room; a
/// compact-height phone gives the pixels to the carousel instead. `hidden` renders the same
/// chip purely as a width reserve.
@ViewBuilder private func statusChip(hidden: Bool) -> some View {
/// compact-height phone gives the pixels to the carousel instead.
@ViewBuilder private var statusChip: some View {
if !compact, let active = gamepads.active {
ControllerStatusChip(controller: active)
.opacity(hidden ? 0 : 1)
.accessibilityHidden(hidden)
}
}
@@ -229,14 +370,9 @@ struct GamepadHomeView: View {
onActivate: { $0.activate() },
onSecondary: { openLibraryForSelected() },
onTertiary: { showSettings = true },
// Stop consuming the controller while another screen (or the connect/wake takeover) is on
// top otherwise the launcher navigates behind it (invisibly on iPhone, visibly on iPad),
// and a second A during a dial would launch a concurrent connect. `.connecting` covers the
// takeover's Connecting phase; `waker.waking` covers its Waking phase.
isActive: libraryTarget == nil && !showSettings && !showAddHost
&& waker.waking == nil && model.phase != .connecting
) { tile in
hostCard(tile, size: CGSize(width: cardWidth, height: cardHeight))
isActive: homeOwnsController
) { tile, entrance in
hostCard(tile, size: CGSize(width: cardWidth, height: cardHeight), entrance: entrance)
}
.frame(height: cardHeight + 40)
}
@@ -245,8 +381,12 @@ struct GamepadHomeView: View {
/// per-frame `phase` (real distance-from-centered), so the look always matches what's on screen
/// mid-scroll. `.shadow`/`.overlay` aren't part of `VisualEffect`, so the focus pop is scale +
/// brightness/saturation + a depth blur on the recessed neighbors.
private func hostCard(_ tile: HomeTile, size: CGSize) -> some View {
private func hostCard(
_ tile: HomeTile, size: CGSize, entrance: CardEntrance
) -> some View {
GamepadHostTile(tile: tile, size: size)
// Beneath the scroll transition, never around it see CardEntrance.
.modifier(entrance)
.scrollTransition { content, phase in
let d = CGFloat(min(abs(phase.value), 1))
let scale = 1 - d * 0.12
@@ -402,10 +542,15 @@ private struct GamepadHostTile: View {
.foregroundStyle(ink.fg(0.5))
}
if tile.isOnline {
// Status colours stay palette-independent (a pip must not change meaning
// with the wallpaper) only the glow softens on a pale field, where it
// reads as a smudge at full strength.
Circle()
.fill(Color.green)
.fill(GamepadInk.onlineGreen)
.frame(width: Self.pipSide, height: Self.pipSide)
.shadow(color: .green.opacity(0.7), radius: 5)
.shadow(
color: GamepadInk.onlineGreen.opacity(ink.isLight ? 0.45 : 0.7),
radius: 5)
}
}
}
@@ -441,7 +586,7 @@ private struct GamepadHostTile: View {
startPoint: .top, endPoint: .bottom),
style: StrokeStyle(lineWidth: 1, dash: tile.filled ? [] : [6, 5]))
}
.shadow(color: .black.opacity(0.45), radius: 20, y: 14)
.shadow(color: ink.shadow(0.45), radius: 20, y: 14)
}
private var monogramBadge: some View {
@@ -37,6 +37,12 @@ struct GamepadInk: Equatable, Sendable {
func accent(_ alpha: Double) -> Color { accent.opacity(alpha) }
/// A wash under text: `alpha` is the dark-field strength, scaled for a pale one.
func shade(_ alpha: Double) -> Color { shade.opacity(alpha * shadeScale) }
/// The glass base at `alpha` what a surface's material is washed with so it carries the
/// palette's hue (the console fills its panels with exactly this colour).
func glass(_ alpha: Double) -> Color { glass.opacity(alpha) }
/// A drop shadow: always black a white shadow is not a shadow but softened on a pale
/// field, where full-strength black under every card reads as a smear rather than depth.
func shadow(_ alpha: Double) -> Color { .black.opacity(alpha * (isLight ? 0.4 : 1)) }
static func of(_ p: GamepadPalette) -> GamepadInk {
let accent = Color(red: p.accent.x, green: p.accent.y, blue: p.accent.z)
@@ -60,6 +66,10 @@ struct GamepadInk: Equatable, Sendable {
/// The shipped dark look what a preview or a test composition gets.
static let dark = GamepadInk.of(GamepadPalette.named("violet"))
/// The online pip deliberately NOT palette-derived: a status colour must not change
/// meaning with the wallpaper (the console's rule; this is its `ONLINE_GREEN` verbatim).
static let onlineGreen = Color(red: 0.20, green: 0.84, blue: 0.29)
}
private struct GamepadInkKey: EnvironmentKey {
@@ -111,7 +111,9 @@ struct GamepadKeyboard: View {
.font(.geist(15, .semibold, relativeTo: .callout))
}
}
.foregroundStyle(focused ? Color.black : ink.fg)
// The focused keycap sits on `ink.accent`, so `onAccent` is what reads on it a dark
// accent palette got black-on-dark with the old literal black.
.foregroundStyle(focused ? ink.onAccent : ink.fg)
.frame(maxWidth: .infinity, minHeight: compact ? 34 : 42)
.background {
RoundedRectangle(cornerRadius: 9, style: .continuous)
@@ -0,0 +1,55 @@
// The library as one of the gamepad shell's in-place layers (iOS): console chrome a pinned
// title and a close styled like the settings screen's around the shared LibraryView, whose
// gamepad branch renders the coverflow. The cover presentation used to get its title and Close
// from the wrapping NavigationStack's bar; a shell layer has no bar, so this restores both in
// the console's own grammar. Everything data-shaped (the fetch, the loading/error/empty states,
// the image session lifecycle) stays LibraryView's.
import PunktfunkKit
import SwiftUI
#if os(iOS)
struct GamepadLibraryScreen: View {
@Environment(\.gamepadInk) private var ink
@ObservedObject var store: HostStore
let host: StoredHost
let onLaunch: (String) -> Void
let close: () -> Void
var controllerActive = true
/// `.compact` in a landscape phone window tighter chrome, like every gamepad screen.
@Environment(\.verticalSizeClass) private var vSizeClass
private var compact: Bool { vSizeClass == .compact }
var body: some View {
LibraryView(
store: store, host: host, onLaunch: onLaunch,
onClose: close, controllerActive: controllerActive)
.safeAreaInset(edge: .top, spacing: 0) {
// Leading, like every gamepad heading no close chrome, B is the exit (the
// coverflow's, or LibraryView's own back-catcher before the coverflow exists).
Text("\(host.displayName) — Library")
.font(.geist(gamepadTitleSize(compact: compact), .bold, relativeTo: .title))
.foregroundStyle(ink.fg)
.lineLimit(1)
.minimumScaleFactor(0.75)
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.horizontal, 24)
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, gamepadTitleBottomPadding(compact: compact))
.background { GamepadTrayScrim(edge: .top) }
}
// A hardware keyboard's Esc still closes, without chrome.
.background {
Button("Close") { close() }
.keyboardShortcut(.cancelAction)
.buttonStyle(.plain)
.frame(width: 0, height: 0)
.opacity(0)
.accessibilityHidden(true)
}
.gamepadPaletteInk()
}
}
#endif
@@ -0,0 +1,95 @@
// The gamepad UI's screen-shell vocabulary (iOS): which screen sits over the launcher, and the
// console push/pop choreography that presents it. On iOS the launcher's sub-screens (settings,
// add-host, library) are NOT system covers they are transparent layers composited in
// GamepadHomeView's ZStack over ONE persistent living backdrop, exactly the model
// `pf-console-ui`'s shell renders on the desktop clients: a push slides the incoming screen up
// out of a fade while the outgoing one recedes; a pop mirrors it; the field underneath never
// moves and never leaves. A system `fullScreenCover` an opaque sheet sliding up from the
// bottom edge, mounting its own backdrop was exactly the wrong grammar for a console.
// (macOS keeps its windowed sheets and tvOS its focus-engine covers; this file's motion
// constants are iOS-only in practice, but compile everywhere for the shared call sites.)
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS) || os(tvOS)
/// The screen the shell currently shows over the launcher. Derived, not stored: the presentation
/// triggers (`showSettings`, `showAddHost`, `libraryTarget`) stay authoritative on every
/// platform this enum is just their iOS rendering. Depth is 1 by construction (the settings
/// pin picker is an in-screen layer, and every trigger is only reachable from the launcher), so
/// there is no stack to model.
enum GamepadScreen: Identifiable {
case settings
case addHost
case library(StoredHost)
var id: String {
switch self {
case .settings: return "settings"
case .addHost: return "addHost"
case .library(let host): return "library-\(host.id.uuidString)"
}
}
/// The backdrop's calm target while this screen is up: the form screens quiet the field
/// (`Bg::Form` in the console); the library keeps the launcher's full aurora.
var isForm: Bool {
switch self {
case .settings, .addHost: return true
case .library: return false
}
}
}
/// The console shell's motion constants, mapped to SwiftUI. Source of truth:
/// `crates/pf-console-ui/src/shell/render.rs` (push/pop) and `shell.rs` (`TRANSITION_S`).
enum GamepadShellMotion {
/// One transition, both layers the console's `TRANSITION_S`.
static let duration: TimeInterval = 0.26
/// `1-(1-t)³` as a bezier: the standard ease-out-cubic control points.
static let screen = Animation.timingCurve(0.33, 1, 0.68, 1, duration: duration)
/// The backdrop's calm chase. The console runs an exponential approach (τ 0.12 s); the same
/// ease-out at 0.30 s lands within a few percent of it and settles together with the screen.
static let calm = Animation.timingCurve(0.33, 1, 0.68, 1, duration: 0.30)
/// The push/pop travel the console's `36 * k`, k-floored for a landscape phone.
static func slide(compact: Bool) -> CGFloat { compact ? 27 : 36 }
/// The incoming screen grows from this; the revealed launcher grows back from `underScale`.
static let inScale: CGFloat = 0.985
static let underScale: CGFloat = 0.96
}
extension AnyTransition {
/// The console push/pop for the top layer. Insertion: up out of a fade, growing from 0.985.
/// Removal: down into a fade at full size (the console's pop leaves scale alone). The
/// launcher's recede underneath is NOT a transition it never unmounts it is the
/// `covered` opacity/scale in GamepadHomeView, animated in the same transaction.
///
/// Known deviation from the console: a pop there re-reveals the launcher from α 0.4; a
/// SwiftUI opacity animates from 0. Same duration, same landing the revealed screen just
/// reads a beat later in the fade, not worth an explicitly-driven progress machine.
static func gamepadScreen(slide: CGFloat) -> AnyTransition {
.asymmetric(
insertion: .opacity
.combined(with: .offset(y: slide))
.combined(with: .scale(scale: GamepadShellMotion.inScale)),
removal: .opacity.combined(with: .offset(y: slide)))
}
}
private struct GamepadHostedInShellKey: EnvironmentKey {
static let defaultValue = false
}
extension EnvironmentValues {
/// True for a screen mounted as one of the shell's layers: it must NOT mount its own
/// backdrop (the shell's single persistent field is behind everything already a second
/// one would double the mesh cost and break the "field never moves" illusion). The same
/// screens presented as macOS sheets / tvOS covers read the default `false` and keep
/// mounting their own, exactly as before.
var gamepadHostedInShell: Bool {
get { self[GamepadHostedInShellKey.self] }
set { self[GamepadHostedInShellKey.self] = newValue }
}
}
#endif
@@ -21,11 +21,16 @@ import GameController
struct LibraryCoverflowView: View {
@Environment(\.gamepadInk) private var ink
let games: [GameEntry]
let imageSession: URLSession?
let artLoader: LibraryArtLoader?
var onLaunch: ((String) -> Void)?
/// Button B (back) dismisses the library screen. No touch equivalent needed here (the toolbar
/// Close button already covers that); this is what makes gamepad-only exit possible.
var onDismiss: (() -> Void)?
/// Whether the carousel owns the controller the in-place shell gates it (mid-transition,
/// and under the connect takeover after A launches a title, where this coverflow used to
/// keep polling underneath). Cover/sheet presentations keep the default.
var controllerActive = true
@Environment(\.gamepadHostedInShell) private var hostedInShell
#if os(iOS)
/// `.compact` in a landscape phone window drives a tighter poster so everything still fits.
@@ -36,6 +41,18 @@ struct LibraryCoverflowView: View {
private let compact = false // no size classes on macOS
#endif
@State private var selection: String?
/// How many covers have settled (art loaded, or every candidate exhausted).
@State private var artSettled = 0
/// The backstop below has fired: play the entrance regardless of what the art is doing.
@State private var artWaitOver = false
/// Whether the strip may play its entrance yet. Cards swinging in as grey placeholders and
/// then filling with artwork afterwards is the whole effect wasted, so the entrance waits for
/// the first few covers every poster is fetched in parallel, so those land together and
/// cover the visible strip. The wait is capped: a slow or artless library still animates.
private var contentReady: Bool {
artWaitOver || artSettled >= min(4, games.count)
}
var body: some View {
GeometryReader { geo in
@@ -46,10 +63,19 @@ struct LibraryCoverflowView: View {
.padding(.leading, 22)
.padding(.vertical, compact ? 6 : 10)
}
.background { GamepadScreenBackground() }
// Hosted in the shell, the field is the shell's own persistent aurora (the library is
// an aurora screen the calm mix simply stays 0, so nothing even chases).
.background {
if !hostedInShell { GamepadScreenBackground() }
}
// Publish the palette's ink to this screen (text, glass, accent, scrims) a
// pale palette flips all of them, and no leaf should have to read the setting.
.gamepadPaletteInk()
// The entrance's backstop (see `contentReady`).
.task {
try? await Task.sleep(for: .milliseconds(700))
artWaitOver = true
}
}
@ViewBuilder private func content(for size: CGSize) -> some View {
@@ -81,9 +107,11 @@ struct LibraryCoverflowView: View {
spacing: 34,
onActivate: { onLaunch?($0.id) },
onBack: { onDismiss?() },
shoulderJump: 5
) { game in
cover(game, width: coverWidth, height: coverHeight)
shoulderJump: 5,
isActive: controllerActive,
contentReady: contentReady
) { game, entrance in
cover(game, width: coverWidth, height: coverHeight, entrance: entrance)
}
.frame(height: coverHeight + 44)
}
@@ -92,18 +120,26 @@ struct LibraryCoverflowView: View {
/// per-frame `phase` (real distance-from-centered), so the tilt tracks what's actually on screen
/// mid-scroll. `.shadow` isn't a `VisualEffect`, so it's baked constant into the card; the
/// scale/rotation/opacity ramp already makes the centered cover prominent.
private func cover(_ game: GameEntry, width: CGFloat, height: CGFloat) -> some View {
PosterImage(candidates: game.art.posterCandidates, title: game.title, session: imageSession)
private func cover(
_ game: GameEntry, width: CGFloat, height: CGFloat, entrance: CardEntrance
) -> some View {
PosterImage(
candidates: game.art.posterCandidates, title: game.title, loader: artLoader,
onLoaded: { artSettled += 1 })
.frame(width: width, height: height)
.clipShape(RoundedRectangle(cornerRadius: 16, style: .continuous))
.overlay(alignment: .topLeading) {
StoreBadge(label: game.storeLabel, isLauncher: game.isLauncher)
// `solid`: a frosted chip can't sample a backdrop through this card's own
// composited transform, so it would only show up on the centred card.
StoreBadge(label: game.storeLabel, isLauncher: game.isLauncher, solid: true)
}
.overlay {
RoundedRectangle(cornerRadius: 16, style: .continuous)
.strokeBorder(ink.fg(0.12), lineWidth: 1)
}
.shadow(color: .black.opacity(0.5), radius: 16, y: 12)
.shadow(color: ink.shadow(0.5), radius: 16, y: 12)
// Beneath the scroll transition, never around it see CardEntrance.
.modifier(entrance)
.scrollTransition { content, phase in
let v = phase.value
let d = CGFloat(min(abs(v), 1))
@@ -12,15 +12,21 @@ struct LibraryView: View {
/// Tapping a title starts a session that asks the host to launch it (the library id is passed
/// through). `nil` browse-only (cards aren't tappable).
var onLaunch: ((String) -> Void)? = nil
/// How the gamepad shell (GamepadLibraryScreen) closes this screen; nil every sheet/cover
/// presentation falls back to the environment dismiss.
var onClose: (() -> Void)? = nil
/// Whether the gamepad coverflow owns the controller the shell gates it during a push/pop
/// and while the connect takeover is up. Presentations that cover the launcher keep the
/// default (their being up IS the launcher's gate).
var controllerActive = true
@Environment(\.dismiss) private var dismiss
@State private var games: [GameEntry] = []
@State private var loading = false
@State private var errorText: String?
/// Authenticated session for cover-art fetches (the same paired identity + host pinning as the
/// list fetch, reused across every poster in the grid). Built alongside `games` in `load()`;
/// torn down on disappear since it isn't one-shot like `LibraryClient.fetch`'s own session.
@State private var imageSession: URLSession?
/// Cover-art loader (the same paired identity + host pinning as the list fetch, reused across
/// every poster in the grid). Built alongside `games` in `load()`; dropped on disappear.
@State private var artLoader: LibraryArtLoader?
#if os(iOS) || os(macOS) || os(tvOS)
// Gamepad-driven browsing see ContentView's identical gate. With no controller (or the
// setting off) every platform keeps the plain-grid presentation of this same view.
@@ -55,9 +61,23 @@ struct LibraryView: View {
}
.task { await load() }
.onDisappear {
imageSession?.finishTasksAndInvalidate()
imageSession = nil
// Hand the loader off before clearing it, so its pooled connections are closed
// rather than left open on a screen the user has left.
let leaving = artLoader
artLoader = nil
Task { await leaving?.close() }
}
#if os(iOS) || os(macOS)
// B closes the library even before the coverflow exists (loading / error / empty):
// the coverflow's carousel owns B once games render; until then this zero-size
// listener does without it a controller-only user is trapped on an error screen
// (the gamepad screens carry no close chrome).
.background {
if gamepadUIActive && games.isEmpty {
LibraryBackCatcher(active: controllerActive) { (onClose ?? { dismiss() })() }
}
}
#endif
}
@ViewBuilder private var content: some View {
@@ -71,8 +91,9 @@ struct LibraryView: View {
} else {
if gamepadUIActive {
LibraryCoverflowView(
games: games, imageSession: imageSession, onLaunch: onLaunch,
onDismiss: { dismiss() })
games: games, artLoader: artLoader, onLaunch: onLaunch,
onDismiss: { (onClose ?? { dismiss() })() },
controllerActive: controllerActive)
} else {
grid
}
@@ -105,10 +126,10 @@ struct LibraryView: View {
LazyVGrid(columns: columns, spacing: 18) {
ForEach(entries) { game in
if let onLaunch {
Button { onLaunch(game.id) } label: { GameCard(game: game, imageSession: imageSession) }
Button { onLaunch(game.id) } label: { GameCard(game: game, artLoader: artLoader) }
.buttonStyle(.plain)
} else {
GameCard(game: game, imageSession: imageSession)
GameCard(game: game, artLoader: artLoader)
}
}
}
@@ -187,8 +208,7 @@ struct LibraryView: View {
keyPEM: identity.keyPEM,
hostFingerprint: current.pinnedSHA256
).launchersFirst
imageSession?.finishTasksAndInvalidate()
imageSession = try LibraryImageLoader.session(
artLoader = try LibraryArtLoader(
address: current.address,
port: current.effectiveMgmtPort,
certPEM: identity.certPEM,
@@ -202,15 +222,39 @@ struct LibraryView: View {
}
}
#if os(iOS) || os(macOS)
/// Zero-size controller listener for the library's pre-coverflow states B backs out. The same
/// shape as ConnectOverlay's `ConnectControllerInput`; `GamepadMenuInput.needsSnapshot` swallows
/// the held press that opened the screen. Unmounts the moment the coverflow (and its own B) is up.
private struct LibraryBackCatcher: View {
let active: Bool
let onBack: () -> Void
@State private var input = GamepadMenuInput(manager: .shared)
var body: some View {
Color.clear
.frame(width: 0, height: 0)
.onAppear {
input.onBack = onBack
if active { input.start() }
}
.onChange(of: active) { _, nowActive in
if nowActive { input.start() } else { input.stop() }
}
.onDisappear { input.stop() }
}
}
#endif
/// One poster tile. Steam vs custom is marked with a badge; the art walks the candidate URLs
/// (portrait header hero) and finally a text placeholder.
private struct GameCard: View {
let game: GameEntry
let imageSession: URLSession?
let artLoader: LibraryArtLoader?
var body: some View {
VStack(alignment: .leading, spacing: 6) {
PosterImage(candidates: game.art.posterCandidates, title: game.title, session: imageSession)
PosterImage(candidates: game.art.posterCandidates, title: game.title, loader: artLoader)
.aspectRatio(2.0 / 3.0, contentMode: .fit)
.frame(maxWidth: .infinity)
.clipShape(RoundedRectangle(cornerRadius: 10, style: .continuous))
@@ -17,16 +17,29 @@ struct StoreBadge: View {
/// A launcher entry (design D4) gets the brand fill, so "opens Steam" is legible at poster size
/// without reading the title.
var isLauncher: Bool = false
/// Fill the chip with a flat wash instead of a frosted material.
///
/// The coverflow MUST pass true. Its cards ride a `.scrollTransition` that composites them
/// with `opacity < 1` and a 3D rotation, and a material cannot sample a backdrop through an
/// offscreen composite so the frost stayed blank on every card and only appeared on the one
/// card sitting at exactly full opacity in the centre, reading as a flash on focus. A flat
/// wash has no backdrop to sample: it is simply always there. (Deliberately black, not
/// palette ink: the chip sits on cover art, whose colours the palette has no business
/// fighting.)
var solid: Bool = false
private var fill: AnyShapeStyle {
if isLauncher { return AnyShapeStyle(Color.brand) }
return solid ? AnyShapeStyle(Color.black.opacity(0.58)) : AnyShapeStyle(.ultraThinMaterial)
}
var body: some View {
Text(label)
.font(.geist(11, .semibold, relativeTo: .caption2))
.foregroundStyle(isLauncher ? AnyShapeStyle(.white) : AnyShapeStyle(.primary))
.foregroundStyle(isLauncher || solid ? AnyShapeStyle(.white) : AnyShapeStyle(.primary))
.padding(.horizontal, 6)
.padding(.vertical, 3)
.background(
isLauncher ? AnyShapeStyle(Color.brand) : AnyShapeStyle(.ultraThinMaterial),
in: Capsule())
.background(fill, in: Capsule())
.padding(6)
}
}
@@ -47,8 +60,8 @@ private extension Image {
}
}
/// Sequentially tries cover-art URLs over `session` (so a paired client can reach the host's own
/// art proxy, not just public CDNs see `LibraryImageLoader`), advancing past any that fail to
/// Sequentially tries cover-art URLs over `loader` (so a paired client can reach the host's own
/// art proxy, not just public CDNs see `LibraryArtLoader`), advancing past any that fail to
/// load, then a placeholder. The loaded image is hard-clipped to fill the card's actual frame
/// regardless of its own aspect ratio: a portrait capsule fills it as intended, but a fallback
/// banner (wide hero/header art, used when a title has no portrait capsule) would otherwise report
@@ -57,7 +70,11 @@ private extension Image {
struct PosterImage: View {
let candidates: [URL]
let title: String
let session: URLSession?
let loader: LibraryArtLoader?
/// Fires once this poster has settled art loaded, or every candidate exhausted and the
/// placeholder is what it will be. The gamepad coverflow waits on a few of these before
/// playing its entrance, so the cards swing in carrying artwork rather than grey rectangles.
var onLoaded: (() -> Void)?
@State private var index = 0
@State private var image: PlatformImage?
@@ -67,26 +84,38 @@ struct PosterImage: View {
Image(platformImage: image)
.resizable()
.scaledToFill()
.transition(.opacity)
} else if index < candidates.count {
ZStack { placeholder; ProgressView() }
.transition(.opacity)
} else {
placeholder
.transition(.opacity)
}
}
// Art crosses over its placeholder instead of replacing it between two frames. Cover
// fetches land one by one, so without this a freshly opened library is a run of cards
// visibly snapping from grey to artwork after the strip has already settled.
.animation(.easeOut(duration: 0.3), value: image != nil)
.frame(maxWidth: .infinity, maxHeight: .infinity)
.clipped()
.task(id: index) { await loadCurrent() }
}
private func loadCurrent() async {
guard index < candidates.count else { return }
guard let session, let data = try? await session.data(from: candidates[index]).0,
// Past the end: the placeholder IS the final look, so this poster has settled.
guard index < candidates.count else {
onLoaded?()
return
}
guard let loader, let data = try? await loader.data(for: candidates[index]),
let loaded = PlatformImage(data: data)
else {
index += 1 // advance to the next candidate (or past the end placeholder)
return
}
image = loaded
onLoaded?()
}
private var placeholder: some View {
@@ -243,7 +243,7 @@ private struct ShotGamepadHome: View {
GamepadHomeView(
store: store, model: model, discovery: discovery,
libraryTarget: .constant(nil), waker: waker,
connect: { _, _ in }, connectDiscovered: { _ in })
connect: { _, _ in }, connectDiscovered: { _ in }, launchTitle: { _, _ in })
}
}
@@ -301,7 +301,7 @@ private struct ShotConnect: View {
GamepadHomeView(
store: store, model: model, discovery: discovery,
libraryTarget: .constant(nil), waker: waker,
connect: { _, _ in }, connectDiscovered: { _ in })
connect: { _, _ in }, connectDiscovered: { _ in }, launchTitle: { _, _ in })
} else {
ShotHome()
}
@@ -132,6 +132,17 @@ final class SessionModel: ObservableObject {
/// and under stage-1.
@Published var osFloorP50Ms = 0.0
@Published var osFloorValid = false
/// The AUDIO plane's latency, from the playback ring (`SessionAudio.Stats`): how much decoded
/// audio is queued ahead of the speaker, and where that PUTS it relative to the picture
/// (positive = audio behind). `audioValid` is false until playback runs.
///
/// Both numbers, never just the depth a deep ring on a jittery link is the adaptive floor
/// doing its job, and only the offset separates that from audio simply being held late. They
/// existed nowhere a surface could render them until now, which is why a field report of "the
/// audio delay seems way too high" was triaged all the way to a conclusion without them.
@Published var audioBufferMs = 0
@Published var audioAvOffsetMs = 0
@Published var audioValid = false
/// The floor-shaved values every HUD tier displays (raw floor, never below 0). Identical
/// to the raw values whenever no floor is measured.
@@ -153,6 +164,20 @@ final class SessionModel: ObservableObject {
/// background's privacy mute never clears the user's choice. Local and instant: it gates
/// capture on this device, nothing is sent to the host.
@Published private(set) var micMuted = false
/// The kind a controller declared when it turned out this session cannot carry its motion
/// set once per such pad, cleared after `motionHintSeconds`. Nil the rest of the time.
///
/// It exists because the failure is otherwise entirely silent: the gyro simply does nothing,
/// with no way for the player to tell a dead sensor from a session that resolved a backend
/// without a motion plane. The fix is a settings change, so the hint has to name it.
@Published private(set) var motionUnreachableKind: PunktfunkConnection.GamepadType?
/// Drops `motionUnreachableKind` again held so a second pad's hint replaces the first
/// cleanly, and so ending the session cancels a pending clear rather than letting it fire
/// into a torn-down model.
private var motionHintTimer: Task<Void, Never>?
/// How long the motion hint stays up the start-of-stream shortcut banner's 6 s, since the
/// two share the bottom-centre stack and a player reads them the same way.
private static let motionHintSeconds: UInt64 = 6
/// Resize overlay (design/midstream-resolution-resize.md client resize UX): true from the
/// instant a Match-window resize starts steering toward a new size until a frame at that size
/// decodes (or a safety timeout). Drives the blur+spinner so the unavoidable host-rebuild delay
@@ -524,6 +549,21 @@ final class SessionModel: ObservableObject {
applyMicMute()
}
/// A forwarded controller has a gyro this session cannot carry (see
/// `GamepadCapture.onMotionUnreachable`). Show it briefly, then let it go.
///
/// Last pad wins, and its timer restarts: two such pads are the same one fact to a player, and
/// a second hint appearing under a still-visible first would only read as a stutter.
private func noteMotionUnreachable(_ kind: PunktfunkConnection.GamepadType) {
motionUnreachableKind = kind
motionHintTimer?.cancel()
motionHintTimer = Task { [weak self] in
try? await Task.sleep(for: .seconds(Self.motionHintSeconds))
guard !Task.isCancelled else { return }
self?.motionUnreachableKind = nil
}
}
/// Push the EFFECTIVE mute the user's choice OR the background keep-alive's privacy mute
/// onto the audio engine. The two reasons are composed here and nowhere else: whichever one
/// changed, the other still holds, so returning from the background can't un-mute a user who
@@ -573,6 +613,11 @@ final class SessionModel: ObservableObject {
// The mic mute is per-session and never persisted: the next stream starts live (if the
// mic is enabled), rather than silently carrying a mute nobody remembers making.
micMuted = false
// Cancel before clearing: a pending clear firing into a torn-down session would be
// harmless but pointless, and leaving the hint set would carry it into the next stream.
motionHintTimer?.cancel()
motionHintTimer = nil
motionUnreachableKind = nil
let audio = self.audio
self.audio = nil
// Gamepad capture is main-actor (releases held buttons on the wire while the
@@ -628,6 +673,7 @@ final class SessionModel: ObservableObject {
displayValid = false
clientQueueValid = false
osFloorValid = false
audioValid = false
lostFrames = 0
lostPct = 0
mouseCaptured = false
@@ -702,7 +748,14 @@ final class SessionModel: ObservableObject {
micUID: settings.micUID,
micChannel: settings.micChannel,
micEnabled: settings.micEnabled,
echoCancel: settings.echoCancel)
echoCancel: settings.echoCancel,
// The A/V sync reference: `endToEnd` is captureon-glass, the one figure that says
// where the picture actually IS, and the audio ring steers its depth to land with it.
// The same meter object the presenter writes per presented frame, so audio reads the
// video plane's own measurement rather than a second estimate of it and under the
// stage-1 fallback presenter, which stamps nothing, it stays empty and the loop
// correctly declines to correct.
videoLatency: endToEnd)
self.audio = audio
// Gamepads: forward every controller GamepadManager selected each on its own wire pad
// index (a pin forwards only one, Automatic forwards all) and render the host's feedback
@@ -722,6 +775,9 @@ final class SessionModel: ObservableObject {
// The cross-client escape chord (hold L1+R1+Start+Select 1.5 s) on tvOS the only
// controller way out of a stream (B/Menu is swallowed during sessions; see ContentView).
capture.onDisconnectRequest = { [weak self] in self?.disconnect() }
// A pad with a gyro that this session cannot carry say so once, briefly, and name the
// setting that fixes it. Already main-actor (GamepadCapture fires it there).
capture.onMotionUnreachable = { [weak self] kind in self?.noteMotionUnreachable(kind) }
capture.start()
gamepadCapture = capture
let feedback = GamepadFeedback(connection: conn, manager: .shared)
@@ -860,6 +916,15 @@ final class SessionModel: ObservableObject {
} else {
self.clientQueueValid = false
}
// The audio plane is a LEVEL, not a window: the ring's depth and the sync loop's
// smoothed offset are both current values, so they are read rather than drained.
if let a = self.audio?.stats {
self.audioBufferMs = a.bufferMS
self.audioAvOffsetMs = a.avOffsetMS
self.audioValid = true
} else {
self.audioValid = false
}
// Mirror the window to the unified log (see statsLog) one line per second,
// stages in ms, only while frames actually flowed. `fps` counts RECEIVED AUs;
// `presents` counts frames that reached glass (the display meter's sample count)
@@ -875,7 +940,12 @@ final class SessionModel: ObservableObject {
// the whole line (a cascade error that also mis-blames the float args).
format: "fps=%lld presents=%lld e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%lld "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f",
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f "
// Appended LAST, so every existing parser of this line is unaffected.
// In the log as well as on the HUD because the overlay is only up when
// someone thought to turn it on, and the reports that need these
// numbers arrive after the fact.
+ "audio_buffer=%lld audio_av_offset=%lld",
frames,
displayWindow?.count ?? 0,
self.endToEndValid ? self.endToEndP50Ms : -1,
@@ -887,7 +957,9 @@ final class SessionModel: ObservableObject {
self.osFloorValid ? self.osFloorP50Ms : -1,
self.displayValid ? self.displayAdjP50Ms : -1,
self.endToEndValid ? self.endToEndAdjP50Ms : -1,
self.clientQueueValid ? self.clientQueueP50Ms : -1)
self.clientQueueValid ? self.clientQueueP50Ms : -1,
self.audioValid ? self.audioBufferMs : -1,
self.audioValid ? self.audioAvOffsetMs : 0)
statsLog.info("\(line, privacy: .public)")
}
}
@@ -154,6 +154,28 @@ struct StreamHUDView: View {
.foregroundStyle(.secondary)
}
}
// The AUDIO plane's own latency (detailed tier). Deliberately OUTSIDE the video branch
// above: it is not a term of that equation audio is steered to MEET the video total,
// never summed into it and the depth is exactly as worth seeing under the stage-1
// fallback presenter, which measures no end-to-end at all.
//
// `buffer` is how much decoded audio is queued ahead of the speaker; `a/v` is where
// that puts it relative to the picture (+ = audio behind). Both, not just the depth: a
// deep ring on a jittery link is the adaptive floor doing its job, and only the offset
// distinguishes that from a ring holding audio late. Neither number was renderable
// anywhere before they lived in a periodic log line which is how a report of "the
// audio delay seems way too high" got triaged to a conclusion with no instrument.
if verbosity == .detailed && model.audioValid && model.audioBufferMs > 0 {
// String(format:) for the signed offset: `%+d` has no specifier-interpolation
// equivalent, and Swift's Int is 64-bit (%lld, never the 32-bit %d).
Text(model.audioAvOffsetMs == 0
? "audio buffer \(model.audioBufferMs) ms"
: String(
format: "audio buffer %lld ms · a/v %+lld ms",
model.audioBufferMs, model.audioAvOffsetMs))
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.tertiary)
}
if model.lostFrames > 0 {
// Unrecoverable network drops this window; hidden while the link is clean.
// String(format:) rather than specifier interpolation: the literal % would
@@ -224,9 +246,18 @@ struct StreamHUDView: View {
/// The card's inner content padding. Roomier on tvOS the stat text auto-scales for the
/// couch (relative system styles), so the card's chrome must keep pace or it reads cramped.
///
/// On iOS it also has to CLEAR THE CORNER. A rounded corner of radius `r` pulls the card's
/// edge inward by `r (r² (ry)²)` at a distance `y` below the top, so the first and last
/// lines of a padded stack sit inside the arc unless the padding keeps pace with the radius.
/// At `0.45 · r` that intrusion stays well inside the padding across the whole range this
/// card can wear (4.6 pt of arc against 12.6 pt of padding at the 28 pt cap), so no line
/// ever runs into the curve.
private var cardPadding: CGFloat {
#if os(tvOS)
return 16
#elseif os(iOS)
return max(10, cardCornerRadius * 0.45)
#else
return 10
#endif
@@ -246,13 +277,20 @@ struct StreamHUDView: View {
#endif
}
/// The card's corner radius. On iOS it's concentric with the physical display corner
/// `displayCornerRadius edgeInset`, so the gap to the screen edge stays uniform right around the
/// corner instead of a small-radius card cutting into the very rounded glass. Clamped so a
/// flat-cornered device (or a hidden radius) still gets a sensibly rounded card.
/// The card's corner radius. On iOS it aims to be concentric with the physical display
/// corner `displayCornerRadius edgeInset`, so the gap to the screen edge stays uniform
/// right around the corner instead of a small-radius card cutting into the very rounded
/// glass but that aim is BOUNDED by what a card this small can actually carry.
///
/// Unbounded, a modern phone (~62 pt of display radius) asked for a 48 pt corner on a card
/// whose lines sit 10 pt from the edge: the arc reaches ~19 pt inward at the first line, so
/// the top and bottom lines rendered INSIDE the curve. Concentricity is only a virtue while
/// the radius is small next to the card; past that it is just a blob eating its own text.
/// 28 pt is the most this card's stack can wear (with `cardPadding` scaling alongside), and
/// devices whose display radius asks for less than that still get a truly concentric corner.
private var cardCornerRadius: CGFloat {
#if os(iOS)
return max(12, DeviceMetrics.displayCornerRadius - edgeInset)
return min(28, max(12, DeviceMetrics.displayCornerRadius - edgeInset))
#elseif os(tvOS)
return 16 // scales with the roomier padding
#else
@@ -267,6 +305,39 @@ struct StreamHUDView: View {
}
}
/// "This pad's gyro can't reach the game" shown briefly when a forwarded controller with motion
/// meets a session whose virtual controller has no motion plane (an X-Box class pad has no gyro in
/// its HID contract, so every sample would be decoded and dropped).
///
/// Not a control, unlike `MicMutedBadge`: the fix is the Controller type setting, which is not
/// reachable mid-stream on every platform, and changing it applies from the next session anyway.
/// So this states the fact and names the setting, in the HUD's glass language, and gets out of the
/// way the alternative is what shipped before, which was a gyro that silently did nothing with
/// no way to tell that from a broken sensor.
///
/// Every platform: a DualSense on an Apple TV is an ordinary way to play, and it is exactly the
/// pad this can happen to.
struct MotionUnreachableBadge: View {
var body: some View {
HStack(spacing: 7) {
Image(systemName: "gyroscope")
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(.yellow)
Text("Motion won't reach this session — set Controller type to DualSense")
.font(.geist(12, .medium, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.9))
}
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.environment(\.colorScheme, .dark) // reads over any frame, like the resize overlay
.accessibilityElement(children: .combine)
.accessibilityLabel(
"This controller's motion will not reach the game. "
+ "Set Controller type to DualSense to enable it.")
}
}
#if !os(tvOS)
/// The muted-microphone badge the mute STATE, as opposed to the buttons that flip it. It rides
/// over the stream whenever the mic is muted, INDEPENDENT of the stats overlay (which the user
@@ -0,0 +1,180 @@
// The gamepad settings' "select" value as a REAL band: the options sit side by side on a drum
// segment curving about a vertical axis the current one faces you flat, and a step rotates the
// next one in with perspective. The old presentation animated a single Text keyed by its value
// (an old-out/new-in crossfade that merely implied motion), which fell apart under fast repeated
// steps: each press restarted the fade. Here the drum's position is one continuous value driven
// by a spring, and SwiftUI's spring retargeting preserves velocity rapid presses accumulate
// into one accelerating travel instead of five restarted crossfades.
//
// The band is LINEAR, not a ring (field verdict on the first cut): a ring showed the first
// option waiting to the right of the last one, which left/right can't reach (adjust clamps)
// a promise the navigation doesn't keep. And on a 2-option ring the unselected option flipped
// sides with every step. So positions are fixed: option i sits i steps from the start, the ends
// are the ends, and A's wrap from the last option travels BACK across the list to the first.
// Options other than the facing one exist only while the drum is actually moving at rest a row
// shows exactly its value (a resting neighbour under a long label rendered as overlapping,
// unreadable text).
//
// The band is purely presentational: stepping semantics (left/right clamps with a boundary thud,
// A cycles forward wrapping, disabled rows refuse input) stay in GamepadSettingsView's row
// closures. Font and ink come from the environment the row applies the same value font/colour
// it always did, and the drum's own opacity ramp multiplies on top.
import Foundation
import SwiftUI
#if os(iOS) || os(macOS) || os(tvOS)
struct GamepadOptionBand: View {
let options: [String]
/// The committed selection the caller's clamp/wrap already applied.
let selection: Int
let focused: Bool
/// The band's footprint, FIXED by the row: a step must never reflow the row (the old
/// free-width value shifted the chevrons with every label), and the drum needs its stage
/// even when the facing label is short.
let width: CGFloat
@Environment(\.accessibilityReduceMotion) private var reduceMotion
/// Where the drum rests, in option steps always chasing `Double(selection)`; only the
/// spring's interpolation ever puts it between integers.
@State private var drumPosition: Double
init(options: [String], selection: Int, focused: Bool, width: CGFloat) {
self.options = options
self.selection = selection
self.focused = focused
self.width = width
_drumPosition = State(initialValue: Double(selection))
}
var body: some View {
Group {
if reduceMotion {
// No drum, no travel: today's quiet crossfade, minus even the 14 pt slip.
ZStack {
Text(current)
.lineLimit(1)
.id(selection)
.transition(.opacity)
}
.animation(.smooth(duration: 0.2), value: selection)
} else {
Drum(
options: options,
rotation: drumPosition,
target: drumPosition,
// Puts the ±1 neighbour ~40 % of the band off-centre, curling to the edge.
radius: width * 0.72)
}
}
.frame(width: width)
.clipped()
// Soft edges: the drum dissolves before it reaches the chevrons instead of ending on a cut.
.mask {
LinearGradient(
stops: [
.init(color: .clear, location: 0),
.init(color: .black, location: 0.12),
.init(color: .black, location: 0.88),
.init(color: .clear, location: 1),
],
startPoint: .leading, endPoint: .trailing)
}
.onChange(of: selection) { old, new in step(from: old, to: new) }
// The options list itself can mutate under the drum (a custom resolution appears, a
// controller connects, the buffer options re-derive from a new refresh rate) re-seat
// without a travel.
.onChange(of: options.count) { _, _ in snap() }
// One element to VoiceOver the neighbour texts are rendering, not content.
.accessibilityElement(children: .ignore)
.accessibilityLabel(current)
}
private var current: String {
options.indices.contains(selection) ? options[selection] : ""
}
/// A step (or A's wrap which on a linear band is a fast travel back to the start) springs
/// the drum; anything else (an external write from the touch settings, a re-derived options
/// list) re-seats it a travel to a value the user didn't step to would read as the UI
/// acting on its own.
private func step(from old: Int, to new: Int) {
let wrapped = options.count > 1 && old == options.count - 1 && new == 0
guard (abs(new - old) == 1 || wrapped), !reduceMotion else { return snap() }
withAnimation(.spring(response: 0.32, dampingFraction: 0.78)) {
drumPosition = Double(new)
}
}
private func snap() {
var tx = Transaction()
tx.disablesAnimations = true
withTransaction(tx) { drumPosition = Double(selection) }
}
}
/// The rotating drum itself. `Animatable` so SwiftUI re-evaluates the body with the INTERPOLATED
/// rotation every frame of the spring each option's offset/scale/opacity follows the real arc,
/// and options along the travel genuinely enter and leave mid-flight. (A plain `.animation` on
/// independent modifiers can't do that: each modifier would lerp its own endpoints and the
/// in-between options would never appear.)
private struct Drum: View, Animatable {
let options: [String]
/// The interpolated drum position, in option steps.
var rotation: Double
/// Where the spring is headed (jumps instantly on a step; only `rotation` chases it). The
/// distance between them is "how mid-flight are we" the neighbours exist exactly as long
/// as the drum is moving, fading continuously as it lands, so a resting row is one flat
/// Text and a long label never sits under a resting neighbour.
let target: Double
/// Drum radius in points (from the band width see the caller).
let radius: Double
var animatableData: Double {
get { rotation }
set { rotation = newValue }
}
/// Angular pitch between adjacent options on the drum.
private static let stepAngle = 34.0 * .pi / 180.0
var body: some View {
let flight = min(1, abs(rotation - target) * 3)
let content = ZStack {
ForEach(0..<options.count, id: \.self) { i in
// Plain signed distance the band is linear, so option i has ONE home and the
// ends are the ends (nothing waits beyond the last option).
let d = Double(i) - rotation
if abs(d) < 0.5 || (flight > 0.001 && abs(d) <= 2.5) {
option(i, distance: d, gate: flight)
}
}
}
#if os(tvOS)
// Flatten the transform stack while travelling the 10-foot GPU already made these
// rows drop Liquid Glass, and five projected texts per step is the same class of cost.
content.drawingGroup()
#else
content
#endif
}
@ViewBuilder private func option(_ i: Int, distance d: Double, gate: Double) -> some View {
let angle = d * Self.stepAngle
let depth = cos(angle)
// The facing option never gates: a resting row still shows its value.
let alpha = pow(max(depth, 0), 3) * (abs(d) < 0.5 ? 1 : gate)
Text(options[i])
.lineLimit(1)
.scaleEffect(0.70 + 0.30 * depth)
// Foreshorten the label as it turns away this is what sells the cylinder.
.rotation3DEffect(.radians(angle), axis: (x: 0, y: 1, z: 0), perspective: 0.4)
.offset(x: radius * sin(angle))
.opacity(alpha)
.zIndex(depth)
}
}
#endif
@@ -47,10 +47,18 @@ enum GpSettingsTab: String, CaseIterable, Hashable {
struct GamepadSettingsView: View {
@Environment(\.gamepadInk) private var ink
@Environment(\.dismiss) private var dismiss
@Environment(\.gamepadHostedInShell) private var hostedInShell
/// The saved-host store the pin picker writes `setPinned` through it and the profile rows
/// count pins from its live hosts. Threaded in from GamepadHomeView like the home screen
/// itself (ContentView owns the instance).
@ObservedObject var store: HostStore
/// How the in-place shell (iOS) closes this screen; nil (the macOS sheet, the tvOS cover)
/// falls back to the environment dismiss. See `performClose`.
var close: (() -> Void)?
/// Whether this screen owns the controller. The shell holds it false during a push/pop (the
/// console's input drop) and while the connect takeover is up; a system presentation never
/// needs the gate and keeps the default.
var controllerActive = true
@AppStorage(DefaultsKey.streamWidth) private var width = 1920
@AppStorage(DefaultsKey.streamHeight) private var height = 1080
@AppStorage(DefaultsKey.streamHz) private var hz = 60
@@ -85,6 +93,7 @@ struct GamepadSettingsView: View {
#endif
#if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
@AppStorage(DefaultsKey.gyroFromDevice) private var gyroFromDevice = false
#endif
@ObservedObject private var gamepads = GamepadManager.shared
/// The profile catalog (ProfileStore.shared, like every other surface that reads it) the
@@ -126,7 +135,8 @@ struct GamepadSettingsView: View {
onAdjust: { row, delta in adjust(id: row.id, by: delta) },
onActivate: { activate(id: $0.id) },
onBack: { back() },
onShoulder: { step(tabBy: $0) }
onShoulder: { step(tabBy: $0) },
isActive: controllerActive
) { row, focused in
rowView(row, focused: focused)
.frame(maxWidth: GamepadFormMetrics.rowMaxWidth)
@@ -134,18 +144,20 @@ struct GamepadSettingsView: View {
}
.frame(maxWidth: .infinity)
.safeAreaInset(edge: .top, spacing: 0) {
VStack(spacing: compact ? 4 : 8) {
VStack(alignment: .leading, spacing: gamepadHeaderSpacing(compact: compact)) {
// Leading, like a console section heading centred read as a floating label,
// and a gamepad UI needs no close chrome next to it (B is the exit).
Text(title)
.font(.geist(gamepadTitleSize(compact: compact), .bold, relativeTo: .title))
.foregroundStyle(ink.fg)
.frame(maxWidth: .infinity)
.overlay(alignment: .trailing) { closeButton.padding(.trailing, 20) }
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.horizontal, 24)
// The picker is one layer deeper its rows aren't sections of anything, so the
// strip would be a control that does nothing while it's up.
if pinTarget == nil { tabStrip }
}
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.padding(.bottom, gamepadTitleBottomPadding(compact: compact))
.background { GamepadTrayScrim(edge: .top) }
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
@@ -167,8 +179,12 @@ struct GamepadSettingsView: View {
}
// The launcher's living field, calmed (GamepadFormBackground) the glass rows keep real
// colour and luminance to lens without the launcher's contrast, and the palette setting
// applies here too, so this screen previews the row you're stepping.
.background { GamepadFormBackground() }
// applies here too, so this screen previews the row you're stepping. Hosted in the
// shell, the field is the SHELL's (one persistent backdrop, calm-chased) mounting a
// second would double the mesh and snap where the shell crossfades.
.background {
if !hostedInShell { GamepadFormBackground() }
}
// Publish the palette's ink to this screen (text, glass, accent, scrims) a
// pale palette flips all of them, and no leaf should have to read the setting.
.gamepadPaletteInk()
@@ -177,6 +193,18 @@ struct GamepadSettingsView: View {
gamepads.startDiscovery()
}
.onDisappear { gamepads.stopDiscovery() }
#if !os(tvOS)
// The visible close is gone (a gamepad UI exits with B) this keeps a hardware
// keyboard's Esc and the macOS sheet's cancel working without chrome.
.background {
Button("Close") { performClose() }
.keyboardShortcut(.cancelAction)
.buttonStyle(.plain)
.frame(width: 0, height: 0)
.opacity(0)
.accessibilityHidden(true)
}
#endif
}
/// The section switcher. Horizontally scrollable so a narrow phone in landscape never has to
@@ -229,10 +257,12 @@ struct GamepadSettingsView: View {
.padding(.vertical, 7)
.background {
// One shared capsule that MOVES between pills, rather than one per pill fading
// in and out the highlight travels the way the press did.
// in and out the highlight travels the way the press did. A Liquid Glass
// surface (accent-tinted through consoleGlass), so the strip wears the same
// material language as the rows it sits above.
if selected {
Capsule()
.fill(ink.accent(0.85))
Color.clear
.consoleGlass(Capsule(), tint: ink.accent(0.85))
.matchedGeometryEffect(id: "tab", in: tabHighlight)
}
}
@@ -274,22 +304,10 @@ struct GamepadSettingsView: View {
focusID = landing
}
/// Touch/click fallback for closing the controller path is B, a hardware keyboard's Esc
/// rides the cancel action.
private var closeButton: some View {
Button { dismiss() } label: {
Image(systemName: "xmark")
.font(.system(size: GamepadFormMetrics.closeFont, weight: .semibold))
.foregroundStyle(ink.fg)
.frame(width: GamepadFormMetrics.closeSide, height: GamepadFormMetrics.closeSide)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
.buttonStyle(.plain)
#if !os(tvOS)
.keyboardShortcut(.cancelAction) // unavailable on tvOS (Menu is the cancel there)
#endif
.accessibilityLabel("Close settings")
/// Close this screen through whichever mechanism presents it: the shell's layer pop on iOS,
/// the environment dismiss under a macOS sheet / tvOS cover.
private func performClose() {
if let close { close() } else { dismiss() }
}
/// "Settings", or "Pin Work" while the pin picker is up the title is what says which
@@ -337,7 +355,7 @@ struct GamepadSettingsView: View {
pinTarget = nil
focusID = "profile-\(profile.id)"
} else {
dismiss()
performClose()
}
}
@@ -363,24 +381,31 @@ struct GamepadSettingsView: View {
.font(.system(size: m.chevronFont, weight: .semibold))
.foregroundStyle(
ink.fg(focused && row.adjustable && row.enabled ? 0.6 : 0))
// Keyed by the value so a change slides the new option in instead of
// hard-swapping the string a QUIET horizontal slip following the user's
// motion (a right-step enters from the right), crossfading over ~14 pt.
// Deliberately not `.push`: that travels the whole container width, loud
// and visibly outside the row. The ZStack is the stable home the
// removed/inserted texts transition within.
let slide: CGFloat = lastAdjustDelta >= 0 ? 14 : -14
ZStack {
Text(row.value)
if let labels = row.optionLabels, let idx = row.selectedIndex {
// A choice row's value is a REAL band the options ride a rotating
// drum, so fast repeated steps spin it instead of restarting a fade.
GamepadOptionBand(
options: labels, selection: idx, focused: focused, width: bandWidth)
.font(.geist(m.valueFont, .medium, relativeTo: .callout))
.foregroundStyle(focused ? ink.fg : ink.fg(0.6))
.lineLimit(1)
.id(row.value)
.transition(.asymmetric(
insertion: .offset(x: slide).combined(with: .opacity),
removal: .offset(x: -slide).combined(with: .opacity)))
} else {
// The flat rows (profile pin counts, placeholders) keep the quiet slip:
// keyed by the value so a change slides the new string in following the
// user's motion, crossfading over ~14 pt. The ZStack is the stable home
// the removed/inserted texts transition within.
let slide: CGFloat = lastAdjustDelta >= 0 ? 14 : -14
ZStack {
Text(row.value)
.font(.geist(m.valueFont, .medium, relativeTo: .callout))
.foregroundStyle(focused ? ink.fg : ink.fg(0.6))
.lineLimit(1)
.id(row.value)
.transition(.asymmetric(
insertion: .offset(x: slide).combined(with: .opacity),
removal: .offset(x: -slide).combined(with: .opacity)))
}
.animation(.smooth(duration: 0.22), value: row.value)
}
.animation(.smooth(duration: 0.22), value: row.value)
Image(systemName: "chevron.right")
.font(.system(size: m.chevronFont, weight: .semibold))
.foregroundStyle(
@@ -410,6 +435,17 @@ struct GamepadSettingsView: View {
rows.first { $0.id == focusID }?.detail ?? " "
}
/// The option band's fixed stage. A portrait phone is the one place the full 240 pt starves
/// the row's label (everywhere else the 620 pt row cap leaves room to spare), so it alone
/// narrows the stage.
private var bandWidth: CGFloat {
#if os(iOS)
hSizeClass == .compact && vSizeClass == .regular ? 170 : GamepadFormMetrics.bandWidth
#else
GamepadFormMetrics.bandWidth
#endif
}
// MARK: - Row model
private struct Row: Identifiable {
@@ -422,6 +458,11 @@ struct GamepadSettingsView: View {
let value: String
/// One-line explanation shown near the hint bar while this row is focused.
let detail: String
/// A choice row's full option list (labels only the tags stay inside the closures)
/// and where its drum currently rests. nil the value renders as plain text (toggles,
/// actions, profiles a two-position switch is not a drum; see GamepadOptionBand).
var optionLabels: [String]?
var selectedIndex: Int?
/// Whether left/right means anything here false hides the value's chevrons (the
/// Profiles rows navigate, and the placeholder rows do nothing at all).
var adjustable = true
@@ -649,6 +690,22 @@ struct GamepadSettingsView: View {
value: $rumbleOnDevice),
at: at + 1)
}
// The phone-gyro mirror sits beside the rumble mirror: same clip-on-pad audience,
// opposite data direction. Hidden where the device has no motion hardware; engages
// in-session only while player 1's controller reports no rotation rate of its own.
if DeviceGyro.isAvailable,
let anchor = list.firstIndex(where: { $0.id == "deviceRumble" })
?? list.firstIndex(where: { $0.id == "padType" }) {
list.insert(
toggleRow(
id: "deviceGyro", tab: .controller,
icon: "gyroscope",
label: "Gyro from this device",
detail: "When the controller has no gyro, send this device's motion "
+ "sensors as player 1's — for clip-on pads without one of their own.",
value: $gyroFromDevice),
at: anchor + 1)
}
#endif
return list + profileRows
}
@@ -710,6 +767,8 @@ struct GamepadSettingsView: View {
value: pinned ? "Pinned" : "Off",
detail: "A pinned profile appears as its own card on the host — one press "
+ "connects with it.",
optionLabels: ["Off", "Pinned"],
selectedIndex: pinned ? 1 : 0,
adjust: { delta in
let target = delta > 0
guard pinned != target else { return false }
@@ -776,6 +835,10 @@ struct GamepadSettingsView: View {
id: id, tab: tab, icon: icon, label: label,
value: index.map { options[$0].label } ?? "",
detail: detail,
// The band mounts only once the value is a known option the "" of an unknown
// current renders flat, and the first step's snap-to-first seats the drum.
optionLabels: index != nil ? options.map(\.label) : nil,
selectedIndex: index,
enabled: enabled,
adjust: { delta in
// Unknown current value: snap to the first option on any step.
@@ -803,6 +866,10 @@ struct GamepadSettingsView: View {
id: id, tab: tab, icon: icon, label: label,
value: value.wrappedValue ? "On" : "Off",
detail: detail,
// Toggles ride the band too (field ask): Off sits left of On, matching the
// directional semantics below, so a right-step slides On in from the right.
optionLabels: ["Off", "On"],
selectedIndex: value.wrappedValue ? 1 : 0,
enabled: enabled,
adjust: { delta in
// Directional semantics: left = off, right = on; a no-op reads as a boundary.
@@ -712,6 +712,15 @@ extension SettingsView {
Toggle("Rumble on this iPhone", isOn: $rumbleOnDevice)
}
}
// The rumble mirror's sibling, data flowing the other way: hidden where the
// device has no motion hardware, engages only while the player-1 controller
// reports no rotation rate of its own.
if !inProfileScope, DeviceGyro.isAvailable {
described("When the controller has no gyro of its own, sends this device's "
+ "motion sensors as player 1's — for clip-on pads without one.") {
Toggle("Gyro from this device", isOn: $gyroFromDevice)
}
}
#endif
#if !os(tvOS)
if !inProfileScope {
@@ -91,6 +91,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.pointerCapture) var pointerCapture = true
@AppStorage(DefaultsKey.touchMode) var touchMode = TouchInputMode.trackpad.rawValue
@AppStorage(DefaultsKey.rumbleOnDevice) var rumbleOnDevice = false
@AppStorage(DefaultsKey.gyroFromDevice) var gyroFromDevice = false
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
@@ -70,12 +70,17 @@ extension View {
// MARK: - Console glass (gamepad host tiles + settings rows)
/// Liquid Glass tuned for the gamepad UI's dark "console" surfaces the host-carousel tiles and
/// Liquid Glass tuned for the gamepad UI's "console" surfaces the host-carousel tiles and
/// the settings rows. Unlike `glassBackground` (floating-overlay only, per HIG), this deliberately
/// clads content tiles / dense rows: a chosen part of the 10-foot console look. `tint` washes the
/// glass toward a color (the brand violet on the focused / primary surface); `interactive` makes
/// it flex on press. The pre-26 fallback is `.ultraThinMaterial` forced dark these surfaces
/// always sit on the near-black backdrop, so the material must stay dark even in a light appearance.
/// glass toward a color (the palette accent on the focused / primary surface); `interactive` makes
/// it flex on press.
///
/// Every tier is WASHED with the palette's `ink.glass` the same surface colour the console
/// fills its panels with so switching the background palette recolours the surfaces, not just
/// the text on them. The wash alphas are tune-on-device values with one fixed direction: the
/// pale palettes' white frost needs MORE body than the dark glass (the console's 0.66-vs-0.62
/// pair), because a thin white wash over a colourful field reads as haze, not as a surface.
private struct ConsoleGlass<S: Shape>: ViewModifier {
let shape: S
var tint: Color?
@@ -86,16 +91,19 @@ private struct ConsoleGlass<S: Shape>: ViewModifier {
@Environment(\.gamepadInk) private var ink
private var scheme: ColorScheme { ink.isLight ? .light : .dark }
/// The palette wash over the material tiers (the material itself supplies the blur body).
private var materialWash: Color { ink.glass(ink.isLight ? 0.55 : 0.40) }
func body(content: Content) -> some View {
#if os(tvOS)
// ALWAYS the material fallback on tvOS: the gamepad settings list is 15+ of these
// surfaces, and live Liquid Glass per row made the whole screen visibly laggy on the
// Apple TV's GPU (same class of call GlassProminentButton already makes glass fights
// the 10-foot platform). The tint rides an overlay so the focused row keeps its wash.
// the 10-foot platform). The wash and tint ride overlays two flat fills, no GPU cost.
content.background {
shape.fill(.ultraThinMaterial)
.environment(\.colorScheme, scheme)
.overlay { shape.fill(materialWash) }
.overlay {
if let tint { shape.fill(tint) }
}
@@ -104,7 +112,14 @@ private struct ConsoleGlass<S: Shape>: ViewModifier {
if #available(iOS 26, macOS 26, *) {
content.glassEffect(glass, in: shape).environment(\.colorScheme, scheme)
} else {
content.background { shape.fill(.ultraThinMaterial).environment(\.colorScheme, scheme) }
content.background {
shape.fill(.ultraThinMaterial)
.environment(\.colorScheme, scheme)
.overlay { shape.fill(materialWash) }
.overlay {
if let tint { shape.fill(tint) }
}
}
}
#endif
}
@@ -112,8 +127,13 @@ private struct ConsoleGlass<S: Shape>: ViewModifier {
#if !os(tvOS)
@available(iOS 26, macOS 26, *)
private var glass: Glass {
var g: Glass = .regular
if let tint { g = g.tint(tint) }
// Liquid Glass has ONE tint channel, so the palette wash and the caller's tint share
// it: mixed 60 % toward the caller's (the focused row must still read accented on
// every palette) over the palette base. If device QA finds the mixed focus wash too
// weak, the escape hatch is `tint ?? wash` today's focused look, bit for bit.
let wash = ink.glass(ink.isLight ? 0.60 : 0.45)
var g: Glass = .regular.tint(
tint.map { wash.mix(with: $0, by: 0.6) } ?? wash)
if interactive { g = g.interactive() }
return g
}
@@ -121,9 +141,51 @@ private struct ConsoleGlass<S: Shape>: ViewModifier {
}
extension View {
/// Liquid Glass for a dark console surface (a host tile / settings row), or `.ultraThinMaterial`
/// (forced dark) pre-26. Pass the surface's shape explicitly glass defaults to a Capsule.
/// Liquid Glass for a console surface (a host tile / settings row), or `.ultraThinMaterial`
/// pre-26 both washed with the palette's own glass colour, both frosting to the palette's
/// scheme. Pass the surface's shape explicitly glass defaults to a Capsule.
func consoleGlass<S: Shape>(_ shape: S, tint: Color? = nil, interactive: Bool = false) -> some View {
modifier(ConsoleGlass(shape: shape, tint: tint, interactive: interactive))
}
}
// MARK: - Console floating glass (the gamepad screens' close buttons)
/// `glassBackground` for a floating control INSIDE the gamepad UI (the close ): same shape
/// contract, but washed with the palette's ink and frosted to the palette's scheme plain
/// `glassBackground` follows the SYSTEM appearance, which leaves the frost dark under dark ink
/// when a pale palette is up. The non-gamepad floating surfaces (the HUD, the trust card, the
/// touch connect modal) keep plain `glassBackground`: they sit over video or the touch UI,
/// where the palette means nothing.
private struct ConsoleGlassBackground<S: Shape>: ViewModifier {
let shape: S
var interactive = false
@Environment(\.gamepadInk) private var ink
private var scheme: ColorScheme { ink.isLight ? .light : .dark }
func body(content: Content) -> some View {
if #available(iOS 26, macOS 26, tvOS 26, *) {
content
.glassEffect(
(interactive ? Glass.regular.interactive() : .regular)
.tint(ink.glass(ink.isLight ? 0.60 : 0.45)),
in: shape)
.environment(\.colorScheme, scheme)
} else {
content.background {
shape.fill(.regularMaterial)
.environment(\.colorScheme, scheme)
.overlay { shape.fill(ink.glass(ink.isLight ? 0.55 : 0.40)) }
}
}
}
}
extension View {
/// Palette-washed floating glass for the gamepad screens' own controls. Same fallback story
/// as `glassBackground` (`.regularMaterial` pre-26), plus the ink wash and scheme flip.
func consoleGlassBackground<S: Shape>(_ shape: S, interactive: Bool = false) -> some View {
modifier(ConsoleGlassBackground(shape: shape, interactive: interactive))
}
}
@@ -21,6 +21,12 @@ import os
/// long quiet spell relaxes it back toward the base so a session on Wi-Fi that bunches arrivals
/// deepens until it stops crackling, while a clean LAN keeps the tight base latency. Keep the
/// constants here in step with `JitterTuning.COREAUDIO`.
///
/// **A/V sync.** On top of all that the depth can be STEERED, by `setSyncTarget` from the drain
/// thread's `AvSync` because a ring that is the right depth for the link is not thereby the
/// right depth for the picture. Continuity still outranks sync: the request is clamped between
/// the underrun-driven floor above and the hard cap, so the loop can never buy alignment with a
/// dropout. `nil` (the default) is exactly the pre-sync behaviour.
final class AudioRing: @unchecked Sendable {
/// Mirrors `JitterTuning::COREAUDIO` see that type for the rationale.
private static let targetMS = 20
@@ -48,6 +54,13 @@ final class AudioRing: @unchecked Sendable {
private static let growWindowMS = 5_000
private static let growStepMS = 10
private static let shrinkQuietMS = 30_000
/// The same quiet span, while the A/V sync loop is actively asking to run shallower. A grown
/// target normally relaxes only after a long spell because, absent other evidence, the only
/// thing that can justify giving up hard-won slack is time; a sync request IS that evidence
/// a measurement saying the extra depth is costing alignment right now so a smaller target
/// gets tested sooner. Wrong guesses are cheap and self-correcting (one underrun and the
/// growth path takes it straight back). Mirrors `SHRINK_QUIET_SYNC_MS`.
private static let shrinkQuietSyncMS = 5_000
private var buf: [Float]
private var readIdx = 0
@@ -70,6 +83,14 @@ final class AudioRing: @unchecked Sendable {
/// which is a different problem from the depth being wrong.
private var underrunCount = 0
private var shedCount = 0
/// The depth the A/V sync loop would like, in interleaved samples (`AvSync.desiredDepth`).
/// `nil` the default, and what an un-wired session keeps reproduces the pre-sync
/// behaviour exactly, so this ring could adopt sync without the other three diverging.
private var syncTarget: Int?
/// The sync loop's smoothed offset in ms, STORED not computed: the ring owns the depth but has
/// no timestamps, so the drain thread (which has both a packet's `pts_ns` and the video leg)
/// hands the number back for reporting. Mirrors `NativeClient::audio_av_offset_ms`.
private var avOffsetMS = 0
private let channels: Int
private let perMS: Int
private let lock = OSAllocatedUnfairLock()
@@ -85,9 +106,64 @@ final class AudioRing: @unchecked Sendable {
/// Effective target depth in interleaved samples: the (adaptively grown) live target, lifted
/// so it can always serve one device quantum plus a packet (a large-buffer device cannot
/// sustain a target below its own quantum).
/// sustain a target below its own quantum) then, if the A/V sync loop has asked for a depth,
/// its request CLAMPED into that band. Mirrors `JitterPolicy::effective_target`.
///
/// The clamp order is the whole safety argument for steering playback depth off a network
/// measurement at all: sync may pull the ring shallower to catch the picture up, or push it
/// deeper when audio runs early, but never below what underrun pressure has proven this link
/// needs, and never past the hard cap that bounds added latency. A link whose jitter genuinely
/// demands more buffer than the picture is away keeps its buffer and the residual is REPORTED
/// (`Stats.avOffsetMS`) rather than taken out of the listener's stream.
///
/// The ceiling is raised to the floor rather than used as-is: a device whose callback quantum
/// alone exceeds `hardCapMS` makes `floor > cap`, and a plain `min(max(s, floor), cap)` would
/// then return the CAP i.e. quietly below the continuity floor, inverting the very ordering
/// this exists to guarantee, on exactly the awkward hardware it exists to survive. (Rust's
/// `Ord::clamp` announces the same condition by panicking; Swift would just get it wrong.)
private var target: Int {
max(targetLive, renderQuantum + Self.frameMS * perMS)
let floor = max(targetLive, renderQuantum + Self.frameMS * perMS)
guard let want = syncTarget else { return floor }
let cap = max(Self.hardCapMS * perMS, floor)
return min(max(want, floor), cap)
}
/// The sync loop is asking to run shallower than the adaptive target has grown to the
/// evidence `noteRead` relaxes a grown target on. Compared against the LIVE target, not the
/// effective one: it is the underrun-driven growth that a sync request is evidence against,
/// not the device-quantum lift, which no amount of measurement can argue with.
private var syncWantsLess: Bool {
guard let want = syncTarget else { return false }
return want < targetLive
}
/// Hand the ring the depth the A/V sync loop wants (`AvSync.desiredDepth`), in interleaved
/// samples, or `nil` to run unsynchronised. Called from the drain thread.
///
/// This is a REQUEST, not a command see `target` for what happens to it. `nil` is the
/// default and reproduces the pre-sync behaviour exactly.
func setSyncTarget(_ samples: Int?) {
lock.lock()
defer { lock.unlock() }
syncTarget = samples
}
/// Store the sync loop's smoothed A/V offset for reporting (positive = audio behind the
/// picture). The ring cannot compute this it has no timestamps but it is where the two
/// numbers a listener's complaint needs, depth and offset, can be read under one lock.
func noteAvOffset(_ ms: Int) {
lock.lock()
defer { lock.unlock() }
avOffsetMS = ms
}
/// Buffered depth in interleaved samples what the sync loop measures against (`bufferedMS`
/// is the same quantity rounded for humans). Everything queued here must play before the frame
/// the drain thread is about to write, which is exactly what delays it.
var bufferedSamples: Int {
lock.lock()
defer { lock.unlock() }
return writeIdx - readIdx
}
func write(_ samples: UnsafePointer<Float>, count: Int) {
@@ -196,7 +272,12 @@ final class AudioRing: @unchecked Sendable {
} else {
emptyReads = 0
quietRun += count
if quietRun >= Self.shrinkQuietMS * perMS {
// Without a sync request, time is the only evidence that hard-won slack is no longer
// needed, so a grown target waits out the long window. A request for less IS evidence,
// and without this branch a ring that ratcheted to the ceiling during a transient would
// hold audio a ceiling's worth late for minutes after the cause had gone.
let quietNeeded = syncWantsLess ? Self.shrinkQuietSyncMS : Self.shrinkQuietMS
if quietRun >= quietNeeded * perMS {
quietRun = 0
targetLive = max(targetLive - Self.growStepMS * perMS, Self.targetMS * perMS)
}
@@ -239,6 +320,12 @@ final class AudioRing: @unchecked Sendable {
let targetMS: Int
let underruns: Int
let sheds: Int
/// The A/V sync loop's smoothed offset (ms): **positive = audio playing BEHIND the
/// picture**, negative = ahead of it. `0` before the loop has evidence, or with sync off.
///
/// Reported next to the depth, never instead of it: a deep ring on a jittery link is
/// CORRECT behaviour, and only the offset separates that from a ring holding audio late.
let avOffsetMS: Int
}
var stats: Stats {
@@ -248,7 +335,154 @@ final class AudioRing: @unchecked Sendable {
bufferedMS: (writeIdx - readIdx) / max(perMS, 1),
targetMS: target / max(perMS, 1),
underruns: underrunCount,
sheds: shedCount)
sheds: shedCount,
avOffsetMS: avOffsetMS)
}
}
// MARK: - A/V sync
/// The A/V synchronisation controller: turns "when will this audio actually play" and "when did
/// the picture it belongs with reach the glass" into a ring depth `AudioRing` should aim for.
/// The Swift mirror of `punktfunk_core::audio::AvSync` keep the two in step.
///
/// **The defect it exists to fix.** The host stamps `pts_ns` on every audio datagram and the
/// client decoded it into `AudioPCM` and then never read it. Video's `pts_ns`, by contrast, is
/// used end to end (`LatencyMeter` computes a true glass-to-glass `displayed + clockOffset pts`
/// per presented frame). So audio free-ran at whatever depth its jitter ring happened to settle
/// at, video was presented on a wholly independent path, and nothing ever compared them: the A/V
/// offset was an accident of buffer depths. It moved whenever the ring ratcheted under underrun
/// pressure, and the way this surfaced in the field it got WORSE every time video got faster,
/// because a quicker decoder lowers the video leg while leaving the audio leg exactly where it was.
///
/// **Video is the master.** In a game streamer the video leg is the input-feel budget and must
/// never be inflated to satisfy the audio clock; audio tolerates small, crossfaded, rate-limited
/// corrections that are inaudible, and `AudioRing.shedOneFrame` already applies them. So audio
/// moves.
///
/// **Continuity outranks sync.** This type only ever PROPOSES a depth. `AudioRing` clamps the
/// proposal to its own underrun-driven floor (see `AudioRing.target`), so a link whose jitter
/// genuinely needs more buffer than the picture is away keeps its buffer and the residual is
/// reported instead of being taken out of the listener's stream.
///
/// Not a class and not locked: it is owned outright by the drain thread that observes packets.
struct AvSync {
/// Smoothing time constant for the measured offset, in ms of consumed audio. Long enough that
/// network jitter and a single late datagram do not move it; short enough to track real drift.
private static let ewmaTauMS = 2_000
/// Offsets inside this band are left alone. Correcting a few ms costs a (crossfaded, but real)
/// discontinuity and buys nothing a listener can perceive detectability for A/V misalignment
/// sits an order of magnitude above it. The deadband is what keeps the loop from hunting
/// forever around zero, which would be audible in a way the misalignment it chased was not.
private static let deadbandMS = 10
/// Observations folded before the first correction is offered. The offset is derived from a
/// clock skew estimate and a video figure that both need a moment to settle after connect;
/// acting on the first sample would chase the handshake, not the stream.
private static let minObservations = 100
/// An offset larger than this is not believed. A wall-clock step, a paused host, or a stale
/// video figure can all produce an enormous apparent misalignment, and steering the ring by it
/// would empty or overfill it outright. Beyond this the loop reports and waits rather than acts.
private static let saneLimitMS = 1_000
/// The protocol's frame, in ms the EWMA is weighted by it so the time constant means the
/// same thing however often the caller observes.
private static let frameMS = 5
/// Interleaved samples per millisecond at the negotiated layout (48 × channels).
private let perMS: Int
/// EWMA of the measured offset in ns. Positive = audio is scheduled to play LATE relative to
/// the picture it belongs with.
private var offsetAvgNs: Double = 0
private var observations = 0
/// Set once an observation lands outside `saneLimitMS`, for reporting.
private(set) var implausible = false
/// `channels` is the negotiated interleaved channel count (2/6/8).
init(channels: Int) {
perMS = 48 * max(channels, 1)
}
/// One measurement handed to `observe`. Every field is in the units its source already
/// produces, so no caller has to do clock arithmetic to use it correctly.
struct Observation {
/// The host capture timestamp carried by the audio frame being queued (host clock).
let ptsNs: UInt64
/// Local `CLOCK_REALTIME` now the same basis `LatencyMeter` stamps video in.
let nowLocalNs: Int64
/// Host clock minus client clock, from the skew handshake (`clockOffsetNs`).
///
/// It very nearly CANCELS: the video figure this is differenced against was computed with
/// the same offset and the same sign, so as long as both terms use one value the skew
/// drops out of the result entirely. That is what makes the connect-time offset good
/// enough here even though the absolute legs would prefer a re-synced one.
let clockOffsetNs: Int64
/// How much audio is already queued AHEAD of this frame, in interleaved samples
/// everything that must play before it does.
let bufferedAhead: Int
/// The video plane's current end-to-end figure in ns: `displayed + clockOffset pts`, as
/// `LatencyMeter` already computes it per presented frame. `nil` while nothing has reached
/// the glass recently no reference, no correction.
let videoE2eNs: Int64?
}
/// Fold one measurement. Returns the smoothed offset in ns once there is enough evidence to
/// believe it (positive = audio late), or `nil` while still settling.
///
/// Rejecting the implausible rather than clamping it is deliberate: a wall-clock step or a
/// stale video figure produces a huge apparent offset, and a clamped-but-wrong value would be
/// acted on as though it were a small real one.
@discardableResult
mutating func observe(_ o: Observation) -> Int64? {
// No frame on the glass yet no reference to align against, so nothing to say.
guard let videoE2eNs = o.videoE2eNs else { return nil }
// When this frame's samples will actually reach the speaker, expressed in the host's
// capture clock the same clock, and the same shape, as the video figure it is compared
// against.
let bufferedNs = Int64(o.bufferedAhead / max(perMS, 1)) * 1_000_000
// Overflow-reporting arithmetic, NOT the wrapping `&+`/`&-` the meters use. Every term is
// a nanosecond count on the same epoch (~1.8e18), so the DIFFERENCE is tiny while the
// operands sit within a factor of five of `Int64.max` and a garbage `pts_ns` would wrap
// a nonsense value round into a small, plausible-looking offset. This loop's entire
// defence is that it can tell nonsense from a real misalignment, so an overflow takes the
// same exit the sanity limit does rather than being silently believed.
let (playAtLocal, o1) = o.nowLocalNs.addingReportingOverflow(bufferedNs)
let (playAtHost, o2) = playAtLocal.addingReportingOverflow(o.clockOffsetNs)
let (audioE2eNs, o3) = playAtHost.subtractingReportingOverflow(Int64(bitPattern: o.ptsNs))
let (offsetNs, o4) = audioE2eNs.subtractingReportingOverflow(videoE2eNs)
guard !o1, !o2, !o3, !o4, abs(offsetNs) <= Int64(Self.saneLimitMS) * 1_000_000 else {
implausible = true
return nil
}
implausible = false
let alpha = min(1.0, Double(Self.frameMS) / Double(Self.ewmaTauMS))
if observations == 0 {
offsetAvgNs = Double(offsetNs)
} else {
offsetAvgNs += (Double(offsetNs) - offsetAvgNs) * alpha
}
observations += 1
return settled ? Int64(offsetAvgNs) : nil
}
/// Enough evidence folded to act on.
var settled: Bool { observations >= Self.minObservations }
/// The smoothed offset in ms (positive = audio late), for the HUD. Reported as soon as it is
/// measured, including while still settling a number the operator can watch converge is more
/// useful than a blank that hides whether the loop is working at all.
var offsetMS: Int { Int(offsetAvgNs / 1_000_000) }
/// The ring depth that would place audio with the picture, given where the ring is now.
/// `nil` while unsettled or inside the deadband the caller then leaves the ring alone.
///
/// Audio late (offset > 0) means there is too much queued: aim shallower. Audio early means
/// aim deeper.
func desiredDepth(currentDepth: Int) -> Int? {
guard settled else { return nil }
let offsetMs = offsetAvgNs / 1_000_000
guard abs(offsetMs) >= Double(Self.deadbandMS) else { return nil }
let delta = Int(offsetMs * Double(perMS))
return max(0, currentDepth - delta)
}
}
@@ -62,6 +62,13 @@ public final class SessionAudio {
/// not the ring, so the drain thread never has to be re-pointed). Main-thread confined,
/// like every start path.
private var ring: AudioRing?
/// The video plane's end-to-end meter (captureon-glass), if the owner wired one the
/// reference the A/V sync loop steers the ring against. `nil` leaves the loop inert and the
/// ring exactly as it was before sync existed, which is also what the stage-1 fallback
/// presenter gets: it decodes and presents inside the layer with no per-frame stamp, so it can
/// offer no reference, and a loop with no reference must not invent one. Main-thread confined,
/// like `ring`; the meter itself is internally locked and read from the drain thread.
private var videoLatency: LatencyMeter?
#if !os(macOS)
/// AVAudioSession `setCategory`/`setActive` are synchronous and block on the audio server, so
/// they must not run on the main thread (UI stall AVFoundation warns about it). PROCESS-WIDE
@@ -91,9 +98,16 @@ public final class SessionAudio {
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending.
/// `echoCancel` picks the engine topology see the header note and `wantsCombined`.
///
/// `videoLatency` is the session's END-TO-END latency meter (captureon-glass). Pass it to arm
/// A/V sync: it is the only thing that tells the audio plane where the picture actually is, and
/// without it the ring keeps today's free-running behaviour. Omit it for a playback-only or
/// stage-1 session, where no such figure is measured.
public func start(
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool, echoCancel: Bool
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool, echoCancel: Bool,
videoLatency: LatencyMeter? = nil
) {
self.videoLatency = videoLatency
#if os(macOS)
// No AVAudioSession on macOS start the engines directly (caller's thread, as before).
startEngines(
@@ -305,6 +319,31 @@ public final class SessionAudio {
}
}
// MARK: - Stats
/// The playback plane's two latency numbers, for the stats overlay.
///
/// Both, never just the depth: a deep ring on a jittery link is CORRECT behaviour the
/// adaptive floor put it there because the link kept starving and only the offset separates
/// that from a ring that is simply holding audio late. Before this pair existed the plane
/// published nothing any surface could render (depth and target lived in a periodic log line),
/// and a field investigation into "the audio delay seems way too high" ran all the way to its
/// conclusion without either number.
public struct Stats: Sendable {
/// Decoded audio queued ahead of the speaker (ms).
public let bufferMS: Int
/// The A/V sync loop's smoothed offset (ms): positive = audio playing BEHIND the picture.
/// `0` before the loop has evidence, with sync unwired, or genuinely aligned.
public let avOffsetMS: Int
}
/// A snapshot of `Stats`, or nil before playback starts. Main thread (`ring` is main-confined;
/// the ring's own numbers are taken under its lock, so they describe one instant).
public var stats: Stats? {
guard let s = ring?.stats else { return nil }
return Stats(bufferMS: s.bufferedMS, avOffsetMS: s.avOffsetMS)
}
// MARK: - Playback (host speaker)
/// The playback jitter ring + the source node draining it shared by the plain playback
@@ -401,9 +440,25 @@ public final class SessionAudio {
}
drainStarted = true
stateLock.unlock()
// A/V sync. This thread is the only place that holds all three ingredients at once: the
// packet's host capture `ptsNs`, the ring depth, and the video plane's end-to-end figure.
// `ptsNs` was decoded into `AudioPCM` and then dropped on the floor right here for the
// plane's entire existence, which is why audio ran at whatever depth its jitter ring
// happened to settle at and nothing ever placed it against the picture.
//
// The escape hatch mirrors the Rust clients': a field regression in a loop that steers
// PLAYBACK should be bisectable without a rebuild. macOS honours it from the environment;
// elsewhere it simply never trips, which is the same as today's behaviour.
let syncEnabled = !["1", "true"].contains(
ProcessInfo.processInfo.environment["PUNKTFUNK_NO_AV_SYNC"] ?? "")
// nil disarms the loop entirely no reference, no correction (see `videoLatency`).
let videoLatency = syncEnabled ? self.videoLatency : nil
if !syncEnabled { log.info("A/V sync disabled by PUNKTFUNK_NO_AV_SYNC") }
let channels = Int(connection.resolvedAudioChannels)
let thread = Thread { [connection, flag, drainDone] in
defer { drainDone.signal() }
var drained = 0
var av = AvSync(channels: channels)
// Decode happens IN-CORE (libopus multistream) AudioToolbox's Opus path is
// stereo-only and is handed back as interleaved f32 PCM in wire channel order.
// Per-iteration autorelease pool: no runloop on this thread (see Stage2Pipeline).
@@ -417,6 +472,25 @@ public final class SessionAudio {
return false // session closed
}
guard let pcm, pcm.frameCount > 0 else { return true }
// Place this frame against the picture it belongs with BEFORE queueing it: the
// depth read here is everything that must still play first, which is exactly what
// delays it. Skipped wholesale when no meter was wired, so an un-armed session
// does not even read the ring.
if let videoLatency {
let depth = ring.bufferedSamples
var ts = timespec()
clock_gettime(CLOCK_REALTIME, &ts)
let nowNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec)
// Half a second of tolerance on the reference: long enough to ride out a
// stalled or hitching present path, short enough that a backgrounded session
// (video decode dropped, audio still playing) stops steering almost at once.
av.observe(AvSync.Observation(
ptsNs: pcm.ptsNs, nowLocalNs: nowNs,
clockOffsetNs: connection.clockOffsetNs, bufferedAhead: depth,
videoE2eNs: videoLatency.latestSample(asOfNs: nowNs, maxAgeMs: 500)))
ring.setSyncTarget(av.desiredDepth(currentDepth: depth))
ring.noteAvOffset(av.offsetMS)
}
pcm.samples.withUnsafeBufferPointer { p in
if let base = p.baseAddress {
ring.write(base, count: pcm.frameCount * pcm.channels)
@@ -430,7 +504,7 @@ public final class SessionAudio {
if drained % 2_000 == 0 {
let s = ring.stats
log.info(
"audio: buffer_ms=\(s.bufferedMS) target_ms=\(s.targetMS) underruns=\(s.underruns) drift_sheds=\(s.sheds)"
"audio: buffer_ms=\(s.bufferedMS) target_ms=\(s.targetMS) underruns=\(s.underruns) drift_sheds=\(s.sheds) av_offset_ms=\(s.avOffsetMS)"
)
}
return true
@@ -483,19 +557,52 @@ public final class SessionAudio {
}
engine.attach(source)
engine.connect(source, to: engine.mainMixerNode, format: format)
guard installMicTap(on: input, micUID: micUID, micChannel: micChannel) else {
// Mic chain unavailable (logged) keep the session audible on the plain playback
// engine rather than playing through an idle voice processor.
// The capture side must be PULLED, and only the render graph pulls anything. An input
// node carrying nothing but a tap is not part of that graph, so on the combined engine
// nobody drove it: the IO unit came up (the recording indicator lit for a beat, then went
// out as the input went idle) and NOT ONE BUFFER ever reached the tap no error, no
// failed start, just a session that quietly sent no microphone at all. Routing the input
// through a silent sink puts it in the graph, which is what Apple's own voice-processing
// sample does. The split path never needed it: a capture-only engine has the input node
// AS its graph, so it is pulled by definition which is why this only broke when the
// combined topology became the default.
//
// `outputVolume = 0` on the sink: the mic has to reach the graph, never the speaker. At
// any audible volume this is a microphone wired straight to the earpiece.
let micSink = AVAudioMixerNode()
engine.attach(micSink)
micSink.outputVolume = 0
engine.connect(engine.inputNode, to: micSink, format: nil)
engine.connect(micSink, to: engine.mainMixerNode, format: nil)
// BEFORE the tap reads a format. Enabling voice processing swaps the engine's IO unit
// for the VPIO one and renegotiates its formats, and until the engine is prepared the
// input node can still report the pre-swap state 0 Hz / 0 channels included, which
// `installMicTap` (correctly) refuses as "no usable input device". Preparing first means
// the chain is built against what the voice processor will actually emit.
engine.prepare()
guard installMicTap(on: engine.inputNode, micUID: micUID, micChannel: micChannel) else {
// Mic chain unavailable on the VOICE-PROCESSED engine (logged). The mic outranks the
// echo cancellation, so fall back to the split path its own engine, no voice
// processor, the topology that shipped before AEC existed rather than dropping the
// uplink for the rest of the session. (The sibling failure above, where the voice
// processor won't engage at all, already does exactly this; this arm used to give up
// on the mic instead, which is how a whole session could go silent uplink-only.)
engine.stop()
startPlayback(speakerUID: speakerUID)
startCapture(micUID: micUID, micChannel: micChannel)
return
}
engine.prepare()
do {
try engine.start()
} catch {
log.error("combined engine failed to start: \(error.localizedDescription)")
input.removeTap(onBus: 0)
startPlayback(speakerUID: speakerUID) // no echo cancellation beats no audio
engine.inputNode.removeTap(onBus: 0)
engine.stop()
// Same rule: a working mic without echo cancellation beats no mic at all.
startPlayback(speakerUID: speakerUID)
startCapture(micUID: micUID, micChannel: micChannel)
return
}
stateLock.lock()
@@ -533,8 +640,16 @@ public final class SessionAudio {
}
}
#endif
guard installMicTap(on: input, micUID: micUID, micChannel: micChannel) else { return }
// Prepared before the tap reads a format, for the same reason the combined path does it:
// a node that hasn't been through `prepare()` can still report the pre-negotiation
// format (0 Hz / 0 channels on a device that is perfectly fine), which reads downstream
// as "no microphone".
engine.prepare()
guard installMicTap(on: engine.inputNode, micUID: micUID, micChannel: micChannel) else {
log.error("mic uplink unavailable — this session sends no microphone audio")
engine.stop()
return
}
do {
try engine.start()
} catch {
@@ -0,0 +1,116 @@
// On-disk cache for library cover art.
//
// Posters are the bulk of what the library screen transfers and they essentially never change, so
// re-fetching them on every visit is pure waste the Windows client has cached them on disk for
// this reason and Apple did not. It matters more now that host art rides `MgmtTransport`: a cache
// hit costs no connection at all.
//
// Lives in the CACHES directory on purpose: every byte here is re-derivable from the host, so the
// system is welcome to evict it under storage pressure. Entries are keyed by the SHA-256 of the
// absolute URL, which covers both host-proxy paths and store CDN URLs without either colliding.
//
// Deliberately free of any Network.framework / PunktfunkCore dependency, so it can be unit-tested
// against a temporary directory.
import CryptoKit
import Foundation
/// A size- and age-bounded blob cache. An actor so disk work stays off whichever thread the
/// SwiftUI poster view happens to be on, and so pruning can never race a write.
actor ArtCache {
private let directory: URL
private let maxBytes: Int
private let maxAge: TimeInterval
private let fileManager = FileManager.default
/// `directory` is created on demand. Defaults: 128 MB a 200-title library of 600×900
/// capsules lands far under that and 30 days, which only matters for art a host later
/// replaces.
init(directory: URL, maxBytes: Int = 128 * 1024 * 1024, maxAge: TimeInterval = 30 * 24 * 3600) {
self.directory = directory
self.maxBytes = maxBytes
self.maxAge = maxAge
}
/// The app's standard location, or nil if the caches directory is unavailable (in which case
/// callers simply run without a cache rather than failing).
static func standard() -> ArtCache? {
guard let caches = FileManager.default.urls(
for: .cachesDirectory, in: .userDomainMask).first
else { return nil }
return ArtCache(directory: caches.appendingPathComponent("PunktfunkArt", isDirectory: true))
}
func data(for url: URL) -> Data? {
let file = path(for: url)
guard let data = try? Data(contentsOf: file) else { return nil }
// Age out stale art rather than serving it forever.
if let modified = modificationDate(of: file), Date().timeIntervalSince(modified) > maxAge {
try? fileManager.removeItem(at: file)
return nil
}
// Touch, so eviction can order by last USE rather than last write.
try? fileManager.setAttributes([.modificationDate: Date()], ofItemAtPath: file.path)
return data
}
func store(_ data: Data, for url: URL) {
// An empty body is not art, and a `data:` URL is already inline caching either is a
// pure loss.
guard !data.isEmpty, url.scheme?.lowercased() != "data" else { return }
do {
try fileManager.createDirectory(at: directory, withIntermediateDirectories: true)
try data.write(to: path(for: url), options: .atomic)
} catch {
return // a cache that can't write is a slow cache, not a broken app
}
prune()
}
/// Drop the oldest entries until the directory fits the budget. Also removes anything past
/// `maxAge` so a cache that is under budget still doesn't hoard stale art forever.
func prune() {
let keys: [URLResourceKey] = [.contentModificationDateKey, .fileSizeKey]
guard let entries = try? fileManager.contentsOfDirectory(
at: directory, includingPropertiesForKeys: keys, options: .skipsHiddenFiles)
else { return }
var files: [(url: URL, date: Date, size: Int)] = []
var total = 0
let now = Date()
for entry in entries {
let values = try? entry.resourceValues(forKeys: Set(keys))
let date = values?.contentModificationDate ?? .distantPast
let size = values?.fileSize ?? 0
if now.timeIntervalSince(date) > maxAge {
try? fileManager.removeItem(at: entry)
continue
}
files.append((entry, date, size))
total += size
}
guard total > maxBytes else { return }
// Oldest first `data(for:)` touches on read, so this is least-recently-USED.
for file in files.sorted(by: { $0.date < $1.date }) {
guard total > maxBytes else { break }
try? fileManager.removeItem(at: file.url)
total -= file.size
}
}
/// Wipe the cache for a "clear cached data" affordance, and for tests.
func clear() {
try? fileManager.removeItem(at: directory)
}
private func path(for url: URL) -> URL {
let digest = SHA256.hash(data: Data(url.absoluteString.utf8))
let name = digest.map { String(format: "%02x", $0) }.joined()
return directory.appendingPathComponent(name, isDirectory: false)
}
private func modificationDate(of file: URL) -> Date? {
(try? file.resourceValues(forKeys: [.contentModificationDateKey]))?.contentModificationDate
}
}
@@ -18,9 +18,6 @@
import CryptoKit
import Foundation
import Security
import os
private let tlsLog = Logger(subsystem: "io.unom.punktfunk", category: "library-tls")
enum ClientTLS {
enum TLSError: LocalizedError {
@@ -134,62 +131,8 @@ enum ClientTLS {
}
}
/// URLSession delegate that pins the host's self-signed cert (by the fingerprint the client
/// already trusts) and presents the client identity for the mTLS client-cert challenge but ONLY
/// for challenges from `host`:`port` (the punktfunk host itself). A session built with this
/// delegate is safe to reuse for OTHER origins too (e.g. a GOG/Heroic/Xbox cover-art CDN): a
/// non-matching origin falls through to `.performDefaultHandling`, i.e. normal system trust
/// evaluation and no client cert exactly what `URLSession.shared` would have done. Without the
/// host scoping, pinning would reject every external origin's cert (its fingerprint never matches
/// the host's) and the client identity would leak to servers that didn't ask for it.
final class LibraryTLSDelegate: NSObject, URLSessionDelegate {
private let identity: SecIdentity
private let pinnedHostFingerprint: Data? // SHA-256 of the host cert DER; nil = accept any (TOFU)
private let host: String
private let port: Int
init(identity: SecIdentity, pinnedHostFingerprint: Data?, host: String, port: UInt16) {
self.identity = identity
self.pinnedHostFingerprint = pinnedHostFingerprint
self.host = host
self.port = Int(port)
}
func urlSession(
_ session: URLSession,
didReceive challenge: URLAuthenticationChallenge,
completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void
) {
let space = challenge.protectionSpace
guard space.host == host, space.port == port else {
completionHandler(.performDefaultHandling, nil)
return
}
switch space.authenticationMethod {
case NSURLAuthenticationMethodServerTrust:
// Pin the host cert by fingerprint the host is self-signed (the client trusts it the
// same way the QUIC session does). No pin yet (TOFU) accept the presented leaf.
guard let trust = space.serverTrust,
let leaf = (SecTrustCopyCertificateChain(trust) as? [SecCertificate])?.first
else {
completionHandler(.cancelAuthenticationChallenge, nil)
return
}
let der = SecCertificateCopyData(leaf) as Data
let fp = Data(SHA256.hash(data: der))
if let pinned = pinnedHostFingerprint, pinned != fp {
tlsLog.warning("library: host cert fingerprint mismatch — refusing")
completionHandler(.cancelAuthenticationChallenge, nil)
return
}
completionHandler(.useCredential, URLCredential(trust: trust))
case NSURLAuthenticationMethodClientCertificate:
completionHandler(.useCredential,
URLCredential(identity: identity, certificates: nil, persistence: .forSession))
default:
completionHandler(.performDefaultHandling, nil)
}
}
}
// The URLSession pinning delegate that used to live here is gone: the management API now speaks
// over `MgmtTransport` (Network.framework), which states the same trust rule in a
// `sec_protocol_options_set_verify_block` and unlike the URL loading system is not subject to
// App Transport Security. That is what lets ATS stay ON for the cover-art CDN fetches, which are
// the only URLSession traffic left in the app. See MgmtTransport.swift for the full rationale.
@@ -0,0 +1,195 @@
// Minimal HTTP/1.1 response parsing for `MgmtTransport`.
//
// We speak HTTP ourselves because the management API has to be reached OUTSIDE the URL loading
// system: App Transport Security applies to URLSession and cannot be relaxed for the arbitrary,
// user-supplied addresses a punktfunk host lives at (see MgmtTransport for the full story). What
// we need is GETs against one host, so this covers exactly that and nothing more no redirects,
// no request bodies, no content negotiation.
//
// It does need to find where a response ENDS without waiting for the peer to hang up, because the
// connection is reused across a grid's worth of poster fetches (`messageLength`). Both framings
// hyper emits are handled: `Content-Length`, and `Transfer-Encoding: chunked` for the art proxy.
//
// Deliberately free of any Network.framework / PunktfunkCore dependency: pure bytes-in,
// value-out, so it can be unit-tested (and typechecked) on its own.
import Foundation
/// A parsed HTTP/1.1 response. `headers` keys are lowercased, so lookups are case-insensitive the
/// way the grammar requires.
struct HTTPResponse: Sendable {
let status: Int
let headers: [String: String]
let body: Data
func header(_ name: String) -> String? { headers[name.lowercased()] }
/// Did the peer ask to close? HTTP/1.1 keeps the connection open unless it says otherwise.
var wantsClose: Bool {
header("connection")?.lowercased().contains("close") ?? false
}
}
enum HTTPParseError: Error, Sendable {
/// The header block never terminated, or the body is shorter than `Content-Length` promised
/// i.e. the peer hung up mid-response. Never surface a truncated body as success: a clipped
/// JSON array would read as "this host has no games" rather than as the failure it is.
case truncated
case malformedStatusLine
case malformedHeader
case malformedChunk
}
enum HTTPResponseParser {
/// Byte length of the first complete response in `raw`, or nil when more bytes are needed.
///
/// Also nil when the response carries no framing header at all, since then the body runs
/// until the peer closes and has no knowable length a connection that answers that way
/// cannot be reused.
static func messageLength(in raw: Data) throws -> Int? {
let b = [UInt8](raw)
guard let head = try parseHead(b) else { return nil }
if head.headers["transfer-encoding"]?.lowercased().contains("chunked") == true {
return try chunkedEnd(b, from: head.bodyStart)
}
if let field = head.headers["content-length"] {
guard let length = Int(field.trimmingCharacters(in: .whitespaces)), length >= 0 else {
throw HTTPParseError.malformedHeader
}
let end = head.bodyStart + length
return b.count >= end ? end : nil
}
return nil // framed by connection close
}
/// Parse one complete response. `raw` must hold exactly one message (use `messageLength` to
/// slice it) or, for a close-framed response, everything read up to EOF.
static func parse(_ raw: Data) throws -> HTTPResponse {
let b = [UInt8](raw)
guard let head = try parseHead(b) else { throw HTTPParseError.truncated }
let rest = Data(b[head.bodyStart...])
let body: Data
if head.headers["transfer-encoding"]?.lowercased().contains("chunked") == true {
body = try decodeChunked(rest)
} else if let field = head.headers["content-length"] {
guard let length = Int(field.trimmingCharacters(in: .whitespaces)), length >= 0 else {
throw HTTPParseError.malformedHeader
}
guard rest.count >= length else { throw HTTPParseError.truncated }
body = rest.prefix(length)
} else {
body = rest // framed by connection close: what we read is what there is
}
return HTTPResponse(status: head.status, headers: head.headers, body: body)
}
private struct Head {
let status: Int
let headers: [String: String]
/// Offset of the first body byte (just past the CRLFCRLF).
let bodyStart: Int
}
/// Status line + header block, or nil if the block hasn't fully arrived.
private static func parseHead(_ b: [UInt8]) throws -> Head? {
guard let headEnd = findHeaderEnd(b) else { return nil }
let text = String(decoding: b[0..<headEnd], as: UTF8.self)
var lines = text.components(separatedBy: "\r\n")
guard !lines.isEmpty else { throw HTTPParseError.malformedStatusLine }
// "HTTP/1.1 200 OK" the reason phrase is optional and ignored.
let statusLine = lines.removeFirst().split(separator: " ", maxSplits: 2,
omittingEmptySubsequences: false)
guard statusLine.count >= 2, statusLine[0].hasPrefix("HTTP/"),
let status = Int(statusLine[1])
else { throw HTTPParseError.malformedStatusLine }
var headers: [String: String] = [:]
for line in lines where !line.isEmpty {
// A leading space/tab marks an obsolete folded continuation line. Nothing we talk to
// emits them, and silently mis-parsing one as a field is worse than refusing it.
guard !line.hasPrefix(" "), !line.hasPrefix("\t"),
let colon = line.firstIndex(of: ":")
else { throw HTTPParseError.malformedHeader }
let name = String(line[line.startIndex..<colon]).lowercased()
let value = String(line[line.index(after: colon)...])
.trimmingCharacters(in: .whitespaces)
// Repeated fields join with ", " per RFC 9110; none of ours repeat, but dropping one
// silently would be a lie.
headers[name] = headers[name].map { "\($0), \(value)" } ?? value
}
return Head(status: status, headers: headers, bodyStart: headEnd + 4)
}
/// Index just past the CRLFCRLF that ends the header block.
private static func findHeaderEnd(_ b: [UInt8]) -> Int? {
guard b.count >= 4 else { return nil }
for i in 0...(b.count - 4) where b[i] == 0x0D && b[i + 1] == 0x0A
&& b[i + 2] == 0x0D && b[i + 3] == 0x0A {
return i
}
return nil
}
/// Offset just past a complete chunked body (terminal chunk plus any trailers), or nil if it
/// hasn't all arrived.
private static func chunkedEnd(_ b: [UInt8], from start: Int) throws -> Int? {
var i = start
while true {
guard let lineEnd = findCRLF(b, from: i) else { return nil }
guard let size = chunkSize(b, i, lineEnd) else { throw HTTPParseError.malformedChunk }
i = lineEnd + 2
if size == 0 {
// Terminal chunk. Trailers (if any) run to the next empty line.
var j = i
while true {
guard let end = findCRLF(b, from: j) else { return nil }
if end == j { return j + 2 }
j = end + 2
}
}
guard i + size + 2 <= b.count else { return nil }
i += size + 2 // payload plus its trailing CRLF
}
}
/// `Transfer-Encoding: chunked` decoding. hyper streams the art proxy this way, so this is a
/// live path, not defensive dead code.
static func decodeChunked(_ data: Data) throws -> Data {
let b = [UInt8](data)
var i = 0
var out = Data()
while true {
guard let lineEnd = findCRLF(b, from: i) else { throw HTTPParseError.malformedChunk }
guard let size = chunkSize(b, i, lineEnd) else { throw HTTPParseError.malformedChunk }
i = lineEnd + 2
if size == 0 { return out } // terminal chunk; trailers are ignored
guard i + size <= b.count else { throw HTTPParseError.malformedChunk }
out.append(contentsOf: b[i..<(i + size)])
i += size
guard i + 1 < b.count, b[i] == 0x0D, b[i + 1] == 0x0A else {
throw HTTPParseError.malformedChunk
}
i += 2
}
}
/// "1a" or "1a;ext=value" 26. Nil if it isn't a hex size.
private static func chunkSize(_ b: [UInt8], _ from: Int, _ to: Int) -> Int? {
let field = String(decoding: b[from..<to], as: UTF8.self)
.split(separator: ";", maxSplits: 1, omittingEmptySubsequences: false)[0]
.trimmingCharacters(in: .whitespaces)
guard let size = Int(field, radix: 16), size >= 0 else { return nil }
return size
}
private static func findCRLF(_ b: [UInt8], from: Int) -> Int? {
guard from >= 0, b.count >= 2 else { return nil }
var i = from
while i + 1 < b.count {
if b[i] == 0x0D && b[i + 1] == 0x0A { return i }
i += 1
}
return nil
}
}
@@ -83,6 +83,10 @@ public extension Array where Element == GameEntry {
/// Errors surfaced to the UI so it can guide setup (the common case is "not paired yet").
public enum LibraryError: LocalizedError {
case unauthorized
/// The host's certificate didn't hash to the fingerprint pinned at pairing an impostor, or
/// a host reinstalled/re-keyed since. Distinct from `unreachable` because the remedy is
/// completely different: re-pair, don't go hunting the network.
case pinMismatch
case http(Int)
case unreachable(String)
@@ -91,12 +95,22 @@ public enum LibraryError: LocalizedError {
case .unauthorized:
return "The host didn't recognize this device. Pair with the host first — it "
+ "authorizes paired clients by their certificate (no token needed)."
case .pinMismatch:
return "The host's certificate doesn't match the one this device paired with. "
+ "If the host was reinstalled, forget it here and pair again."
case .http(let code):
return "The management API returned HTTP \(code)."
case .unreachable(let why):
return "Couldn't reach the host's management API: \(why). It binds the LAN by default, "
+ "so check the host is updated and reachable (a host pinned to "
+ "`--mgmt-bind 127.0.0.1` is loopback-only and can't be browsed remotely)."
// The library rides a DIFFERENT port than the stream (the management API, 47990 by
// default; the stream is QUIC on 9777), so it can fail while streaming to the same
// host works perfectly say that first, because the opposite assumption has sent
// more than one person hunting the wrong layer. Opening that URL in a browser is the
// fastest way to tell "port unreachable" apart from anything client-side.
return "Couldn't reach the host's management API: \(why). The library uses a "
+ "different port than the stream (47990 by default), so streaming can work "
+ "while this doesn't. Check that port is reachable from this device, and that "
+ "the host isn't pinned to `--mgmt-bind 127.0.0.1`, which serves it to the "
+ "host itself only."
}
}
}
@@ -115,47 +129,66 @@ public enum LibraryClient {
keyPEM: String,
hostFingerprint: Data?
) async throws -> [GameEntry] {
guard let url = URL(string: "https://\(address):\(port)/api/v1/library") else {
throw LibraryError.unreachable("invalid host address")
}
let identity: SecIdentity
do {
identity = try ClientTLS.makeIdentity(certPEM: certPEM, keyPEM: keyPEM)
} catch {
throw LibraryError.unreachable(
(error as? LocalizedError)?.errorDescription ?? error.localizedDescription)
}
let delegate = LibraryTLSDelegate(
identity: identity, pinnedHostFingerprint: hostFingerprint, host: address, port: port)
let session = URLSession(configuration: .ephemeral, delegate: delegate, delegateQueue: nil)
defer { session.finishTasksAndInvalidate() }
let req = URLRequest(url: url, timeoutInterval: 10)
let (data, response): (Data, URLResponse)
do {
(data, response) = try await session.data(for: req)
} catch {
throw LibraryError.unreachable(error.localizedDescription)
}
guard let http = response as? HTTPURLResponse else {
throw LibraryError.unreachable("not an HTTP response")
}
switch http.statusCode {
guard let base = URL(string: "\(baseURL(address: address, port: port))/api/v1/library")
else { throw LibraryError.unreachable("invalid host address") }
let identity = try clientIdentity(certPEM: certPEM, keyPEM: keyPEM)
let response = try await send(
path: "/api/v1/library", address: address, port: port,
identity: identity, hostFingerprint: hostFingerprint)
switch response.status {
case 200:
var games = try JSONDecoder().decode([GameEntry].self, from: data)
// Steam art now comes back as host-relative proxy paths (`/api/v1/library/art/...`,
// see the host's `library::steam_art`) so they work the same regardless of which
var games = try JSONDecoder().decode([GameEntry].self, from: response.body)
// Steam art comes back as host-relative proxy paths (`/api/v1/library/art/...`, see
// the host's `library::steam_art`) so they work the same regardless of which
// interface/port the client reached the host on. Resolve them against THIS host now,
// so every other consumer just sees ordinary absolute URLs.
let base = url
for i in games.indices {
games[i].art = games[i].art.resolved(against: base)
}
return games
case 401:
// 403 joins 401 here: both are the host declining this certificate, and the remedy the
// user needs is the same one.
case 401, 403:
throw LibraryError.unauthorized
default:
throw LibraryError.http(http.statusCode)
throw LibraryError.http(response.status)
}
}
/// `https://addr:port`, IPv6 literals bracketed the mirror of the Rust client's `base_url`.
static func baseURL(address: String, port: UInt16) -> String {
let bare = address.hasPrefix("[") && address.hasSuffix("]")
? String(address.dropFirst().dropLast()) : address
return bare.contains(":") ? "https://[\(bare)]:\(port)" : "https://\(bare):\(port)"
}
/// Build the paired identity, restating any keychain failure in the UI's vocabulary.
static func clientIdentity(certPEM: String, keyPEM: String) throws -> SecIdentity {
do {
return try ClientTLS.makeIdentity(certPEM: certPEM, keyPEM: keyPEM)
} catch {
throw LibraryError.unreachable(
(error as? LocalizedError)?.errorDescription ?? error.localizedDescription)
}
}
/// One GET against the host, with transport failures mapped onto `LibraryError`.
static func send(
path: String, address: String, port: UInt16,
identity: SecIdentity, hostFingerprint: Data?
) async throws -> HTTPResponse {
do {
return try await MgmtTransport.get(
host: address, port: port, path: path,
identity: identity, pinnedHostFingerprint: hostFingerprint)
} catch MgmtTransportError.pinMismatch {
throw LibraryError.pinMismatch
} catch MgmtTransportError.timedOut {
throw LibraryError.unreachable("timed out")
} catch let error as MgmtTransportError {
throw LibraryError.unreachable(String(describing: error))
} catch {
throw LibraryError.unreachable(error.localizedDescription)
}
}
}
@@ -179,23 +212,77 @@ extension Artwork {
}
}
/// Builds the authenticated `URLSession` the library UI uses to fetch cover-art images the same
/// paired identity + host pinning as [`LibraryClient.fetch`], reused across a whole grid's worth of
/// poster loads (this session is NOT one-shot: callers own its lifetime and should invalidate it
/// when the view goes away). Safe to use for every candidate URL a `GameEntry`'s `Artwork` carries:
/// `LibraryTLSDelegate` only pins/presents-cert for the host itself, deferring to normal system
/// trust + no client cert for any other origin (an external CDN URL).
public enum LibraryImageLoader {
public static func session(
/// Loads cover art for the library UI, routing each URL to the transport that suits its origin.
///
/// A `GameEntry`'s art candidates mix two very different things: the host's own art proxy
/// (`/api/v1/library/art/...`, resolved to absolute URLs against this host) and public store CDN
/// URLs carried verbatim on custom/GOG/Heroic entries. Host URLs go over [`MgmtTransport`] with
/// the paired identity and the pinned fingerprint outside the URL loading system, so App
/// Transport Security can stay ON app-wide. Every other origin keeps ordinary `URLSession` with
/// full system trust evaluation and no client certificate, which is exactly what it should get.
///
/// Posters are cached on disk (`ArtCache`), so a second visit to a library costs no network at
/// all and the connections behind a first visit are pooled and kept alive rather than paying a
/// TLS handshake per tile.
///
/// Built once per library screen and reused across a whole grid's worth of posters.
public final class LibraryArtLoader: @unchecked Sendable {
private let address: String
private let port: UInt16
private let identity: SecIdentity
private let hostFingerprint: Data?
/// Third-party origins only. No delegate: these are ordinary public HTTPS URLs and get the
/// system's normal certificate validation.
private let cdn = URLSession(configuration: .default)
/// nil when the caches directory is unavailable then we simply always fetch.
private let cache = ArtCache.standard()
public init(
address: String,
port: UInt16 = punktfunkDefaultMgmtPort,
certPEM: String,
keyPEM: String,
hostFingerprint: Data?
) throws -> URLSession {
let identity = try ClientTLS.makeIdentity(certPEM: certPEM, keyPEM: keyPEM)
let delegate = LibraryTLSDelegate(
identity: identity, pinnedHostFingerprint: hostFingerprint, host: address, port: port)
return URLSession(configuration: .default, delegate: delegate, delegateQueue: nil)
) throws {
self.address = address
self.port = port
self.identity = try LibraryClient.clientIdentity(certPEM: certPEM, keyPEM: keyPEM)
self.hostFingerprint = hostFingerprint
}
public func data(for url: URL) async throws -> Data {
if let cache, let cached = await cache.data(for: url) { return cached }
let fetched = try await fetch(url)
if let cache { await cache.store(fetched, for: url) }
return fetched
}
/// Release this host's pooled connections call when the library screen goes away, so we
/// don't sit on open TLS sockets the user is finished with.
public func close() async {
await MgmtConnectionPool.shared.closeAll(
matching: "\(MgmtTransport.unbracketed(address)):\(port):")
}
private func fetch(_ url: URL) async throws -> Data {
guard isHostOrigin(url) else { return try await cdn.data(from: url).0 }
var path = url.path.isEmpty ? "/" : url.path
if let query = url.query { path += "?\(query)" }
let response = try await LibraryClient.send(
path: path, address: address, port: port,
identity: identity, hostFingerprint: hostFingerprint)
guard response.status == 200 else { throw LibraryError.http(response.status) }
return response.body
}
/// Does this URL point at the host's own art proxy? Compared on host + port rather than a
/// string prefix, so a differently-spelled but equivalent URL still takes the pinned path.
private func isHostOrigin(_ url: URL) -> Bool {
guard let host = url.host else { return false }
let bare = address.hasPrefix("[") && address.hasSuffix("]")
? String(address.dropFirst().dropLast()) : address
let scheme = url.scheme?.lowercased()
return host.caseInsensitiveCompare(bare) == .orderedSame
&& (url.port ?? (scheme == "http" ? 80 : 443)) == Int(port)
}
}
@@ -0,0 +1,377 @@
// HTTPS transport for the host's management REST API, built on Network.framework rather than
// URLSession.
//
// WHY NOT URLSession. App Transport Security governs the URL loading system, and its default
// policy exempts only "local" destinations `.local` names, unqualified names, and RFC1918 /
// link-local IP literals. Everything else must present a certificate that passes system trust
// evaluation. A punktfunk host is self-signed by construction (there is no CA that could vouch
// for a box on someone's LAN), so the library worked at 192.168.x and died at the TLS layer on
// every other address: a Tailscale peer (100.64/10 is CGNAT, NOT RFC1918), a WireGuard peer, or a
// public IP. No ATS key can express "any address the user typed" the exception keys are
// domain-scoped so the only ways out were disabling ATS app-wide (which also drops the TLS
// floor and the cleartext block on third-party cover-art fetches, the one surface we did NOT want
// to open) or leaving the URL loading system for this one origin. This is that second option.
//
// Network.framework is not subject to ATS, and `sec_protocol_options_set_verify_block` lets us
// state the trust rule we actually mean: the leaf certificate must hash to the fingerprint the
// user pinned during PIN pairing. That is a stronger check than CA trust here, not a weaker one,
// and it is the same rule the QUIC stream plane has always applied via punktfunk-core which is
// precisely why streaming kept working over Tailscale while the library did not.
//
// Connections are POOLED and kept alive: a library screen fetches one JSON payload and then a
// poster per title, and giving each its own TLS handshake was pure latency. `MgmtConnectionPool`
// keeps a small number of connections per host, hands them out one request at a time, and makes
// callers wait rather than opening an unbounded number.
import CryptoKit
import Foundation
import Network
import Security
enum MgmtTransportError: Error, Sendable {
/// The host's certificate did not hash to the pinned fingerprint an impostor, or a host
/// that was reinstalled/re-keyed since pairing.
case pinMismatch
case connection(String)
case timedOut
case tooLarge
case invalidPort(UInt16)
}
enum MgmtTransport {
/// Largest response we will buffer. The host's art proxy serves Steam hero images that run to
/// a few MB; anything past this is not a poster and not a library payload.
static let maxResponseBytes = 16 * 1024 * 1024
/// `GET https://host:port/path`, authenticated by mTLS (`identity`) and pinned by
/// `pinnedHostFingerprint` (nil = trust-on-first-use, matching the QUIC connect's semantics).
///
/// Runs over a pooled keep-alive connection. A connection the host has since dropped is
/// indistinguishable from a live one until we write to it, so a REUSED connection that fails
/// is retried once on a fresh one; a fresh connection that fails is a real error.
static func get(
host: String,
port: UInt16,
path: String,
identity: SecIdentity,
pinnedHostFingerprint: Data?,
timeout: TimeInterval = 15
) async throws -> HTTPResponse {
guard let nwPort = NWEndpoint.Port(rawValue: port) else {
throw MgmtTransportError.invalidPort(port)
}
let pin = pinnedHostFingerprint
let key = "\(unbracketed(host)):\(port):\(pin.map(hex) ?? "tofu")"
var lastError: Error = MgmtTransportError.connection("no attempt made")
for attempt in 0..<2 {
let connection = await MgmtConnectionPool.shared.acquire(key: key) {
MgmtConnection(host: unbracketed(host), port: nwPort, identity: identity, pin: pin)
}
let wasReused = connection.hasServedRequest
do {
let response = try await connection.perform(path: path, timeout: timeout)
await MgmtConnectionPool.shared.release(connection, key: key)
return response
} catch {
await MgmtConnectionPool.shared.release(connection, key: key)
lastError = error
// Only a reused connection earns a second try, and only once: retrying a fresh
// connection would just double every genuine failure's latency.
if !wasReused || attempt == 1 { throw error }
}
}
throw lastError
}
static func hex(_ data: Data) -> String {
data.map { String(format: "%02x", $0) }.joined()
}
/// Saved hosts store bare addresses, but a user who pasted a bracketed IPv6 literal shouldn't
/// get an unresolvable endpoint out of it.
static func unbracketed(_ host: String) -> String {
guard host.hasPrefix("["), host.hasSuffix("]"), host.count > 2 else { return host }
return String(host.dropFirst().dropLast())
}
}
/// A pool of keep-alive connections, at most `maxPerHost` per host. Callers past that wait for one
/// to come back rather than opening more a library grid can ask for dozens of posters at once,
/// and answering that with dozens of TLS handshakes is what this exists to prevent.
actor MgmtConnectionPool {
static let shared = MgmtConnectionPool()
private var available: [String: [MgmtConnection]] = [:]
/// Connections created and not yet closed, per host the cap this pool enforces.
private var live: [String: Int] = [:]
private var waiters: [String: [CheckedContinuation<Void, Never>]] = [:]
private let maxPerHost = 4
func acquire(key: String, make: () -> MgmtConnection) async -> MgmtConnection {
while true {
if var idle = available[key], let connection = idle.popLast() {
available[key] = idle
if connection.isHealthy { return connection }
connection.close()
live[key] = max(0, (live[key] ?? 1) - 1)
continue
}
if (live[key] ?? 0) < maxPerHost {
live[key] = (live[key] ?? 0) + 1
return make()
}
await withCheckedContinuation { (continuation: CheckedContinuation<Void, Never>) in
waiters[key, default: []].append(continuation)
}
}
}
/// Always call this, on success AND on failure: a connection that is never returned leaks a
/// slot, and enough leaked slots would hang every later request on the waiter queue.
func release(_ connection: MgmtConnection, key: String) {
if connection.isHealthy, (available[key]?.count ?? 0) < maxPerHost {
available[key, default: []].append(connection)
} else {
connection.close()
live[key] = max(0, (live[key] ?? 1) - 1)
}
if var queue = waiters[key], !queue.isEmpty {
let next = queue.removeFirst()
waiters[key] = queue
next.resume()
}
}
/// Drop every idle connection for a host used when a library screen goes away, so we don't
/// sit on sockets the user is done with.
func closeAll(matching prefix: String) {
for (key, connections) in available where key.hasPrefix(prefix) {
connections.forEach { $0.close() }
live[key] = max(0, (live[key] ?? 0) - connections.count)
available[key] = []
}
}
}
/// One TLS connection to a host, serving requests one at a time. The pool guarantees a single
/// caller at a time, so there is no request queueing here.
///
/// Everything mutable is touched only on `queue`, which also runs the connection's callbacks, the
/// verify block and the timeout so the state below needs no locking and two callbacks can never
/// race to resume the same continuation.
final class MgmtConnection: @unchecked Sendable {
private let queue = DispatchQueue(label: "io.unom.punktfunk.mgmt-connection")
private let connection: NWConnection
private let host: String
private let port: UInt16
private enum Phase { case idle, connecting, ready, dead }
private var phase: Phase = .idle
private var pending: CheckedContinuation<HTTPResponse, Error>?
private var pendingRequest: Data?
/// Bytes read past the end of the last response. Non-empty only if a host pipelines ahead of
/// us, which none do but dropping them would silently corrupt the next read.
private var buffer = Data()
private var operation = 0
private var pinRejected = false
private var servedRequest = false
/// False once the connection has failed; the pool discards these instead of handing them out.
private(set) var isHealthy = true
/// Has this connection completed at least one request? Drives the retry-once rule in
/// `MgmtTransport.get` only a connection the host may have dropped since is worth retrying.
var hasServedRequest: Bool { servedRequest }
init(host: String, port: NWEndpoint.Port, identity: SecIdentity, pin: Data?) {
self.host = host
self.port = port.rawValue
let options = NWProtocolTLS.Options()
let sec = options.securityProtocolOptions
sec_protocol_options_set_min_tls_protocol_version(sec, .TLSv12)
// Our half of the mTLS handshake: the same paired identity the host authorizes the
// read-only library routes by (mgmt/auth.rs `cert_may_access`).
if let secIdentity = sec_identity_create(identity) {
sec_protocol_options_set_local_identity(sec, secIdentity)
}
let rejected = RejectionFlag()
// Replaces system trust evaluation wholesale, which is the point: the host is self-signed
// and carries no SAN, so there is nothing for the system policy to succeed at. Pinning the
// leaf's SHA-256 is the real check.
sec_protocol_options_set_verify_block(sec, { _, trust, complete in
let secTrust = sec_trust_copy_ref(trust).takeRetainedValue()
guard let chain = SecTrustCopyCertificateChain(secTrust) as? [SecCertificate],
let leaf = chain.first
else {
rejected.value = true
complete(false)
return
}
guard let pin else {
complete(true) // trust-on-first-use: no pin recorded for this host yet
return
}
let fingerprint = Data(SHA256.hash(data: SecCertificateCopyData(leaf) as Data))
let matches = fingerprint == pin
if !matches { rejected.value = true }
complete(matches)
}, queue)
self.connection = NWConnection(
to: .hostPort(host: NWEndpoint.Host(host), port: port),
using: NWParameters(tls: options, tcp: NWProtocolTCP.Options()))
self.rejection = rejected
self.connection.stateUpdateHandler = { [weak self] state in
self?.handle(state)
}
}
/// Set from the verify block, read when mapping the resulting handshake failure. Its own
/// object because the block is built before `self` exists.
private let rejection: RejectionFlag
private final class RejectionFlag: @unchecked Sendable { var value = false }
func perform(path: String, timeout: TimeInterval) async throws -> HTTPResponse {
try await withCheckedThrowingContinuation { continuation in
queue.async {
guard self.phase != .dead else {
continuation.resume(throwing: MgmtTransportError.connection("connection closed"))
return
}
self.operation += 1
let op = self.operation
self.pending = continuation
self.pendingRequest = self.requestBytes(path: path)
self.buffer.removeAll(keepingCapacity: true)
self.queue.asyncAfter(deadline: .now() + timeout) { [weak self] in
guard let self, self.operation == op else { return }
self.finish(.failure(MgmtTransportError.timedOut))
}
switch self.phase {
case .idle:
self.phase = .connecting
self.connection.start(queue: self.queue)
case .ready:
self.send()
case .connecting, .dead:
break // `.ready` (or a failure) will pick the pending request up
}
}
}
}
func close() {
queue.async {
self.phase = .dead
self.isHealthy = false
self.connection.cancel()
}
}
// MARK: - Queue-confined internals
private func handle(_ state: NWConnection.State) {
switch state {
case .ready:
phase = .ready
if pendingRequest != nil { send() }
case .failed(let error):
phase = .dead
isHealthy = false
finish(.failure(mapped(error)))
case .cancelled:
phase = .dead
isHealthy = false
finish(.failure(MgmtTransportError.connection("cancelled")))
default:
break
}
}
private func send() {
guard let request = pendingRequest else { return }
pendingRequest = nil
connection.send(content: request, completion: .contentProcessed { [weak self] error in
guard let self else { return }
if let error {
self.isHealthy = false
self.finish(.failure(self.mapped(error)))
return
}
self.receive()
})
}
private func receive() {
connection.receive(minimumIncompleteLength: 1, maximumLength: 64 * 1024) {
[weak self] chunk, _, isComplete, error in
guard let self else { return }
if let chunk, !chunk.isEmpty { self.buffer.append(chunk) }
if self.buffer.count > MgmtTransport.maxResponseBytes {
self.isHealthy = false
self.finish(.failure(MgmtTransportError.tooLarge))
return
}
if let error {
self.isHealthy = false
self.finish(.failure(self.mapped(error)))
return
}
do {
if let length = try HTTPResponseParser.messageLength(in: self.buffer) {
let message = self.buffer.prefix(length)
self.buffer = Data(self.buffer.dropFirst(length))
let response = try HTTPResponseParser.parse(message)
// A response the peer means to be last leaves nothing reusable behind. Nor
// does a stream with bytes left over: we never pipeline, so anything trailing
// means we are out of sync, and reusing the connection would misread the next
// response rather than fail cleanly.
if response.wantsClose || !self.buffer.isEmpty { self.isHealthy = false }
self.servedRequest = true
self.finish(.success(response))
return
}
if isComplete {
// No framing header: the body ran to EOF, so what we have is the whole thing
// and the connection is spent.
self.isHealthy = false
let response = try HTTPResponseParser.parse(self.buffer)
self.servedRequest = true
self.finish(.success(response))
return
}
} catch {
self.isHealthy = false
self.finish(.failure(error))
return
}
self.receive()
}
}
private func finish(_ result: Result<HTTPResponse, Error>) {
guard let continuation = pending else { return }
pending = nil
operation += 1 // invalidate this operation's timeout
continuation.resume(with: result)
}
/// A rejected pin surfaces as a generic handshake failure; the flag is how we recover what
/// actually happened, so the UI can say "re-pair" instead of "offline".
private func mapped(_ error: NWError) -> MgmtTransportError {
rejection.value ? .pinMismatch : .connection(String(describing: error))
}
private func requestBytes(path: String) -> Data {
// An IPv6 literal is bracketed in the Host header (RFC 9110 §7.2); a name or IPv4 is not.
let authority = host.contains(":") ? "[\(host)]:\(port)" : "\(host):\(port)"
let request = """
GET \(path) HTTP/1.1\r
Host: \(authority)\r
User-Agent: punktfunk-apple\r
Accept: */*\r
\r
"""
return Data(request.utf8)
}
}
@@ -311,6 +311,51 @@ public final class PunktfunkConnection {
default: return nil
}
}
/// Whether this backend has a motion plane at all whether a `sendMotion` sample to a
/// host running it can reach the game, or is decoded and dropped. Mirrors the host's
/// `GamepadPref::has_motion`; the X-Box classes have no gyro in their HID contract.
///
/// This answers for ONE backend. To ask it of a particular pad, go through
/// `PunktfunkConnection.motionReaches(declared:)` `resolvedGamepad` is not that pad's
/// answer, because the host builds each virtual device from the pad's own
/// `gamepadArrival` and falls back to the session default only for a pad that never
/// declared one.
///
/// `.auto` answers `true` on purpose: it means "unknown" an older host that omitted the
/// echo, which may well have resolved a DualSense. Suppressing on unknown would silently
/// break a working gyro, which is the worse of the two failures.
public var hasMotion: Bool {
switch self {
case .auto: return true // unknown; assume it can, see above
case .xbox360, .xboxOne: return false
case .dualSense, .dualShock4, .dualSenseEdge, .switchPro,
.steamController, .steamDeck, .steamController2:
return true
}
}
/// Whether motion sent for ONE pad can reach the game: `declared` is the kind that pad
/// announced in its `gamepadArrival`, `asked` is the session default the handshake carried,
/// and `resolved` is the host's echo. Mirrors punktfunk-core's `pad_motion_reaches`, which
/// carries the full argument; in short:
///
/// - the host builds each virtual device from that pad's declaration, so the echo is simply
/// not this pad's answer when the two differ (under "Automatic" the handshake carries the
/// ACTIVE pad's kind, so a couch with an X-Box pad and a DualSense echoes X-Box 360 while
/// the host builds the DualSense a working motion plane);
/// - the host FOLDS what it cannot build a Switch Pro on Windows, a UHID backend on a
/// host whose `/dev/uhid` is unusable and nothing here can predict that;
/// - but the echo IS one observed sample of that fold, for the kind we asked about, so it
/// is authoritative for a pad that declared exactly that.
///
/// Static and pure so it can be tested without a live session; the connection's
/// `motionReaches(declared:)` is the call site that fills in the other two.
public static func motionReaches(
declared: GamepadType, asked: GamepadType, resolved: GamepadType
) -> Bool {
declared == asked ? resolved.hasMotion : declared.hasMotion
}
}
/// The virtual gamepad backend the host actually resolved (the Welcome's echo of the
@@ -318,6 +363,18 @@ public final class PunktfunkConnection {
/// DualSense feedback.
public private(set) var resolvedGamepad: GamepadType = .auto
/// The session default this connection's handshake ASKED for, kept beside the host's answer
/// above. The pair is what makes the echo usable per pad see `motionReaches(declared:)`.
public private(set) var requestedGamepad: GamepadType = .auto
/// Whether motion sent for ONE pad can reach the game, given the kind that pad DECLARED in its
/// `gamepadArrival` (`GamepadManager.declaredKind(for:)`) this session's two halves of
/// `GamepadType.motionReaches(declared:asked:resolved:)`, which carries the reasoning.
public func motionReaches(declared: GamepadType) -> Bool {
GamepadType.motionReaches(
declared: declared, asked: requestedGamepad, resolved: resolvedGamepad)
}
/// The compositor the host actually resolved for this session's virtual output (the
/// Welcome's echo of the requested `compositor`, with `.auto` resolved to a concrete
/// backend). `.auto` = an older host that didn't say. Clients use it to decide
@@ -572,6 +629,9 @@ public final class PunktfunkConnection {
var gp: UInt32 = 0
_ = punktfunk_connection_gamepad(handle, &gp)
resolvedGamepad = GamepadType(rawValue: gp) ?? .auto
// What we asked for, straight off the parameter the echo above only speaks for a pad
// that declared this same kind (see `motionReaches(declared:)`).
requestedGamepad = gamepad
var comp: UInt32 = 0
_ = punktfunk_connection_compositor(handle, &comp)
resolvedCompositor = Compositor(rawValue: comp) ?? .auto
@@ -0,0 +1,221 @@
// The opt-in phone-gyro mirror (`DefaultsKey.gyroFromDevice`): when player 1's forwarded
// controller has no rotation sensor of its own, THIS device's IMU speaks for it on the wire's
// motion plane for clip-on and third-party pads that ship without a gyro, where the phone
// body is rigidly attached to (or simply is) the thing in the player's hands. The sibling of
// `GamepadFeedback`'s rumble-on-device mirror, with the data flowing the other way.
//
// GamepadCapture owns the engage/stand-down decision (it knows the pad-0 slot and whether its
// controller reports a rotation rate); this class only turns CoreMotion on and off and converts
// samples. Two invariants it enforces itself:
// - one motion writer per pad: samples go out only between `start` and `stop`, and capture
// suppresses pad 0's controller-motion forwarding while this runs;
// - no stale rotation: `stop` sends a single zero-gyro sample after the last real one, so the
// host's virtual pad never keeps integrating an angular velocity this device stopped
// producing (the gyro-sweep "stale angular velocity re-sent forever" failure mode).
//
// Samples are CMDeviceMotion (sensor-fused: bias-corrected rotation rate, gravity split from
// user acceleration) at the ~100 Hz CoreMotion ceiling below a DualSense's 250 Hz, but the
// host's motion plane is event-driven, not cadence-locked, so a slower producer just means
// fewer samples. Units and axis semantics match `GamepadCapture.forwardMotion` exactly (the
// `GamepadWire` constants; accel = gravity + user acceleration the same convention, so a
// future sign/scale correction lands in one place for both sources). The one thing the phone
// adds is a frame remap: CoreMotion reports in the device's portrait frame, while the wire
// wants the controller frame the player sees (x right, y up, z out of the screen), so each
// sample is rotated by the current interface orientation a phone clipped landscape must yaw
// when the player yaws, not roll.
#if os(iOS)
import CoreMotion
import Foundation
import UIKit
/// Device-frame controller-frame axis remap for one interface orientation. CoreMotion's
/// frame is fixed to the portrait device (+x right edge, +y top, +z out of the screen); the
/// controller frame keeps +z (the screen always faces the player) and rotates x/y so they
/// mean "player's right" and "player's up". Derived, like the wire scale constants pinned
/// by `DeviceGyroRemapTests`, correctable in one place if on-glass says otherwise.
/// File-scope rather than nested so the sample thread can use it without actor isolation.
enum DeviceGyroRemap {
case identity
/// Upside-down portrait: both in-plane axes flip.
case flipped
/// Landscape, device top to the player's LEFT (interface `.landscapeRight`):
/// player-right = device-bottom, player-up = device-right.
case topLeft
/// Landscape, device top to the player's RIGHT (interface `.landscapeLeft`).
case topRight
init(_ orientation: UIInterfaceOrientation) {
switch orientation {
case .portraitUpsideDown: self = .flipped
case .landscapeRight: self = .topLeft
case .landscapeLeft: self = .topRight
default: self = .identity
}
}
/// Rotate one device-frame vector (rotation rate or acceleration both transform the
/// same way under an in-plane rotation) into the controller frame.
func apply(x: Float, y: Float, z: Float) -> (x: Float, y: Float, z: Float) {
switch self {
case .identity: return (x, y, z)
case .flipped: return (-x, -y, z)
case .topLeft: return (-y, x, z)
case .topRight: return (y, -x, z)
}
}
}
@MainActor
public final class DeviceGyro {
/// Whether this device can source motion at all gates the settings rows (a device
/// without an IMU would make the toggle a silent no-op, the rumble mirror's rule).
/// One shared probe: Apple recommends a single `CMMotionManager` per app, and the
/// settings UI asking per-render must not allocate one each time.
public static let isAvailable: Bool = CMMotionManager().isDeviceMotionAvailable
/// Everything the sample thread touches, behind one lock: the orientation remap (written
/// on main when the device rotates), the last converted accel, and whether a real sample
/// went out (so `stop` knows it owes the wire a zero). Kept off the actor deliberately
/// `forward` runs on the delivery queue.
private final class SampleState: @unchecked Sendable {
let lock = NSLock()
var remap: DeviceGyroRemap = .identity
var sentSample = false
/// Re-sent with the closing zero-gyro sample so "rotation stopped" doesn't also
/// overwrite a plausible gravity vector with free-fall.
var lastAccel: (Int16, Int16, Int16) = (0, 0, 0)
}
/// Ship one converted sample (wire pad 0). Must be thread-safe invoked from the
/// delivery queue (`PunktfunkConnection.sendMotion` locks internally).
private let send: @Sendable (_ gyro: (Int16, Int16, Int16), _ accel: (Int16, Int16, Int16)) -> Void
private let motion = CMMotionManager()
/// Dedicated serial delivery queue deliberately NOT main (the controller path's
/// main-queue delivery is a known jitter source; the mirror starts clean).
private let queue: OperationQueue = {
let q = OperationQueue()
q.name = "punktfunk.device-gyro"
q.maxConcurrentOperationCount = 1
return q
}()
private let state = SampleState()
private var orientationObserver: NSObjectProtocol?
/// Whether the mirror is between `start` and `stop` read by GamepadCapture to keep the
/// controller path off pad 0's motion while this runs.
public private(set) var isRunning = false
public init(
send: @escaping @Sendable (_ gyro: (Int16, Int16, Int16), _ accel: (Int16, Int16, Int16)) -> Void
) {
self.send = send
}
/// Begin sourcing pad-0 motion from this device. Idempotent.
public func start() {
guard !isRunning, motion.isDeviceMotionAvailable else { return }
isRunning = true
updateRemap()
// Interface orientation only changes alongside a device-orientation notification, so
// this is the one signal needed; re-reading the scene keeps a rotation lock stable.
orientationObserver = NotificationCenter.default.addObserver(
forName: UIDevice.orientationDidChangeNotification, object: nil, queue: .main
) { [weak self] _ in
MainActor.assumeIsolated { self?.updateRemap() }
}
// CoreMotion's practical ceiling; requesting faster just clamps.
motion.deviceMotionUpdateInterval = 1.0 / 100.0
motion.startDeviceMotionUpdates(to: queue) { [state, send] m, _ in
guard let m else { return }
Self.forward(m, state: state, send: send)
}
}
/// Stop sourcing and, if anything was sent, park the host pad's rotation at zero. The
/// zero rides the same serial queue as the samples, so it is guaranteed last without
/// blocking the caller.
public func stop() {
guard isRunning else { return }
isRunning = false
motion.stopDeviceMotionUpdates()
if let o = orientationObserver {
NotificationCenter.default.removeObserver(o)
orientationObserver = nil
}
queue.addOperation { [state, send] in
state.lock.lock()
let owed = state.sentSample
state.sentSample = false
let accel = state.lastAccel
state.lock.unlock()
if owed { send((0, 0, 0), accel) }
}
}
private func updateRemap() {
let o = UIApplication.shared.connectedScenes
.compactMap { $0 as? UIWindowScene }
.first?.interfaceOrientation ?? .portrait
state.lock.lock()
state.remap = DeviceGyroRemap(o)
state.lock.unlock()
}
/// Runs on the delivery queue: remap, scale, ship.
nonisolated private static func forward(
_ m: CMDeviceMotion, state: SampleState,
send: (_ gyro: (Int16, Int16, Int16), _ accel: (Int16, Int16, Int16)) -> Void
) {
state.lock.lock()
let r = state.remap
state.lock.unlock()
let rot = r.apply(
x: Float(m.rotationRate.x), y: Float(m.rotationRate.y), z: Float(m.rotationRate.z))
// Total acceleration, NEGATED the same convention as GamepadCapture.forwardMotion, which
// this file's header promises to track. Apple reports the gravity VECTOR (pointing down);
// an accelerometer measures proper acceleration (pointing up at rest), and the wire carries
// the latter. Without the minus a still phone told the host it was accelerating downward at
// 1 g.
let acc = r.apply(
x: -Float(m.gravity.x + m.userAcceleration.x),
y: -Float(m.gravity.y + m.userAcceleration.y),
z: -Float(m.gravity.z + m.userAcceleration.z))
// NO frame conversion here, and that is not an oversight `GamepadCapture.forwardMotion`
// applies `GamepadWire.appleMotionToWire` and this deliberately does not.
//
// The trap is that two different frames are both called "the controller frame". GCMotion
// reports a CONTROLLER in (Right, Forward, Up) measured on a real DualSense which is
// not the wire's frame, hence the conversion over there. `r` above resolves THIS DEVICE
// into the frame the header describes: x right, y up, z out of the screen. For the pose
// this mirror exists to serve a phone clipped upright, screen facing the player "out of
// the screen" points AT the player, so that frame is (Right, Up, Backward), which IS the
// wire's frame. Straight through is already correct.
//
// Applying the controller path's conversion here was tried and was WRONG: a phone at rest
// would have reported gravity as 1 g on the roll axis instead of +1 g up, i.e. lying on
// its edge. Caught by measuring the Android twin, which does the same thing straight
// through and reads +1 g on the up axis end to end. If a future capture path needs a
// conversion, decide it from that source's OWN measured frame rather than by analogy.
let gs = GamepadWire.gyroLSBPerRadS
let as_ = GamepadWire.accelLSBPerG
let gyro = (
GamepadWire.motionRaw(rot.x, scale: gs),
GamepadWire.motionRaw(rot.y, scale: gs),
GamepadWire.motionRaw(rot.z, scale: gs)
)
let accel = (
GamepadWire.motionRaw(acc.x, scale: as_),
GamepadWire.motionRaw(acc.y, scale: as_),
GamepadWire.motionRaw(acc.z, scale: as_)
)
state.lock.lock()
state.lastAccel = accel
state.sentSample = true
state.lock.unlock()
send(gyro, accel)
}
}
#endif
@@ -66,7 +66,13 @@ public final class GamepadCapture {
var buttons: UInt32 = 0
var axes: [Int32] = [0, 0, 0, 0, 0, 0]
var fingerActive: [Bool] = [false, false]
var lastMotionNs: UInt64 = 0
/// A motion sample went out on this pad `flush` then owes the wire a zero-gyro
/// sample: the host holds motion as STATE and re-emits it, so a nonzero angular
/// velocity left behind reads as endless rotation (the gyro-sweep latch).
var motionSent = false
/// The last accel sent, re-used by the flush zero so "rotation stopped" doesn't
/// also replace a plausible gravity vector with free-fall.
var lastAccel: (Int16, Int16, Int16) = (0, 0, 0)
// Hold-Selectguide gesture state (pf-client-core's `SelectGesture`, adapted to
// this class's mask-diff model): a Select pressed ALONE is held out of the mask
// until it resolves into a tap (delivered on release) or past `guideHold` a
@@ -89,9 +95,6 @@ public final class GamepadCapture {
/// against `manager.forwarded` (empty until a session's `start`, cleared by `stop`).
private var slots: [Slot] = []
/// Motion forwarding floor: 4 ms between samples ( 250 Hz, the DualSense's own rate).
private static let motionIntervalNs: UInt64 = 4_000_000
/// The cross-client controller escape chord (pf-client-core's `ESCAPE_CHORD`):
/// L1+R1+Start+Select held together four simultaneous buttons no game uses, so normal
/// play can't trip it. Held for `disconnectHold` it ends the session via
@@ -128,6 +131,15 @@ public final class GamepadCapture {
/// gameplay can't end it (see ContentView's tvOS session branch).
public var onDisconnectRequest: (() -> Void)?
/// Fired ON MAIN, once per slot at open, when a controller that HAS a gyro was given a host
/// backend without a motion plane its motion is not being sent, because every sample would
/// be decoded and dropped. The argument is the kind this pad declared, so the UI can name it.
///
/// It fires at open rather than on the first sample precisely because nothing is sampled: the
/// IMU is never powered in this case (see `openSlot`), which is also what stops the pad
/// burning battery streaming gyro nobody reads.
public var onMotionUnreachable: ((PunktfunkConnection.GamepadType) -> Void)?
/// Forward this device's controllers to the host at all (`Settings.gamepadForwarding`,
/// default true). Off is for a couch whose controller reaches the host another way USB
/// passthrough such as VirtualHere, or a pad plugged into the host itself where
@@ -153,6 +165,15 @@ public final class GamepadCapture {
/// everywhere but macOS). See `guideHold`.
public let guideGesture: Bool
#if os(iOS)
/// Opt-in phone-gyro mirror (`DefaultsKey.gyroFromDevice`): while player 1's forwarded
/// controller has no rotation sensor, this device's IMU sources pad 0's motion instead
/// for clip-on pads without a gyro. Session-scoped (the setting is read once here); nil
/// when off, unavailable, or forwarding is off (the mirror is wire-only, so with nothing
/// to send there is nothing to mirror). Engage/stand-down lives in `updateDeviceGyro`.
private let deviceGyro: DeviceGyro?
#endif
public init(
connection: PunktfunkConnection, manager: GamepadManager, forwarding: Bool = true,
systemForward: Bool = true, guideGesture: Bool = false
@@ -162,6 +183,17 @@ public final class GamepadCapture {
self.forwarding = forwarding
self.systemForward = systemForward
self.guideGesture = guideGesture
#if os(iOS)
if forwarding, DeviceGyro.isAvailable,
UserDefaults.standard.bool(forKey: DefaultsKey.gyroFromDevice) {
deviceGyro = DeviceGyro { [weak connection] gyro, accel in
// Thread-safe (sendMotion locks); pad 0 by the same rule as the rumble mirror.
connection?.sendMotion(pad: 0, gyro: gyro, accel: accel)
}
} else {
deviceGyro = nil
}
#endif
}
public func start() {
@@ -187,6 +219,9 @@ public final class GamepadCapture {
MainActor.assumeIsolated {
self?.suspended = true
self?.releaseAll()
// The mirror pauses with capture (its stop parks the host pad's rotation
// at zero an overlay pull-down must not leave the game spinning).
self?.updateDeviceGyro()
}
})
observers.append(NotificationCenter.default.addObserver(
@@ -199,11 +234,15 @@ public final class GamepadCapture {
for slot in self.slots {
if let ext = slot.controller.extendedGamepad { self.sync(slot, ext) }
}
self.updateDeviceGyro()
}
})
}
public func stop() {
#if os(iOS)
deviceGyro?.stop()
#endif
closeAllSlots()
forwardedSub = nil
observers.forEach { NotificationCenter.default.removeObserver($0) }
@@ -224,6 +263,8 @@ public final class GamepadCapture {
}
// A chord-holding pad may have just unplugged re-evaluate so a stale hold disarms.
updateEscapeChord()
// Pad 0 may have changed hands re-evaluate whether this device's IMU speaks for it.
updateDeviceGyro()
}
/// Open one forwarded controller on its assigned wire index: attach GC handlers, claim its
@@ -299,10 +340,43 @@ public final class GamepadCapture {
// local feature reads it. Powering the IMU anyway costs the pad real battery (it streams
// gyro + accel continuously over Bluetooth, which is why `closeSlot` is careful to power
// it back down), so with nothing to forward we simply never turn it on.
if forwarding, let motion = c.motion {
if motion.sensorsRequireManualActivation { motion.sensorsActive = true }
motion.valueChangedHandler = { [weak self, weak slot] m in
MainActor.assumeIsolated { if let self, let slot { self.forwardMotion(slot, m) } }
//
// A host that built this pad a backend WITHOUT a motion plane is the same situation: every
// sample would be decoded and dropped, so there is equally nothing to forward. Asked per
// pad off what this slot declared, not off the session echo under "Automatic" a couch
// with an X-Box pad on 0 and a DualSense on 1 echoes X-Box 360 while the host builds pad 1
// a DualSense whose gyro works.
//
// Gated on `hasRotationRate`, not on `motion != nil`. An X-Box controller exposes a
// `GCMotion` that reports gravity and NOTHING else attaching to it streamed a
// permanently-zero `rotationRate` to the host as authoritative gyro, under a declaration
// that says this pad has one. A game reading it sees a controller being held perfectly
// still forever, which is worse than seeing no motion plane at all: there is nothing to
// fall back to and nothing to notice.
let motionCanReach = connection.motionReaches(declared: slot.pref)
if forwarding, let motion = c.motion, motion.hasRotationRate {
if motionCanReach {
if motion.sensorsRequireManualActivation { motion.sensorsActive = true }
// Delivered on the MAIN queue, like every other handler here, and deliberately so
// even though ~250 Hz of samples on main is not free.
//
// GameController's `handlerQueue` is a property of the CONTROLLER, not of an
// element, so there is no way to move motion off main without moving buttons,
// sticks, the touchpad and the escape chord with it. This whole class is
// `@MainActor` eight `assumeIsolated` sites, the slot table, the gesture timers
// so that is a rewrite of the isolation model, not a queue assignment. It would
// also put the tvOS escape chord (the ONLY controller way out of a stream there)
// on a background queue, which is a real risk taken for a speculative gain.
//
// If main-queue contention ever shows up as motion jitter, the measurement to make
// first is `motion_cadence`'s per-pad inter-arrival histogram on the host it
// already reports exactly this, and would say whether the delay is here or on the
// wire before anyone restructures the class for it.
motion.valueChangedHandler = { [weak self, weak slot] m in
MainActor.assumeIsolated { if let self, let slot { self.forwardMotion(slot, m) } }
}
} else {
onMotionUnreachable?(slot.pref)
}
}
}
@@ -561,36 +635,96 @@ public final class GamepadCapture {
private func forwardMotion(_ slot: Slot, _ m: GCMotion) {
guard !suspended else { return }
let now = DispatchTime.now().uptimeNanoseconds
guard now &- slot.lastMotionNs >= Self.motionIntervalNs else { return }
slot.lastMotionNs = now
// Total acceleration in g: gravity + user when split, else the raw vector.
#if os(iOS)
// While the phone-gyro mirror speaks for pad 0, the controller's own motion
// necessarily rotation-less, that's the engage condition stays off the wire:
// two writers on one pad's motion state would fight, and this accel-only stream
// would keep stomping the mirror's gyro with zeros.
if slot.pad == 0, deviceGyro?.isRunning == true { return }
#endif
// Every sample goes out. There used to be a 4 ms floor here, and it was a DROP: a sample
// arriving 3.9 ms after the last one was discarded outright.
//
// That is the wrong shape for this signal. Buttons and sticks are absolute state, so a
// dropped frame costs nothing the next one says everything it would have. Angular
// velocity is a RATE, and a consumer integrates it into an angle; a dropped sample is
// rotation that happened and can never be recovered. GameController's delivery is jittery
// around the pad's own ~250 Hz, so a floor set AT that rate does not shed a rare extra
// sample, it sheds a steady fraction of every turn and the error is one-signed, so it
// accumulates into aim drifting short rather than into noise.
//
// Nothing needed the ceiling: GC delivers at the sensor's rate rather than faster, the SDL
// client has always forwarded every sample, and the host's own idle watchdog runs on a
// 100 ms timeout this cannot outpace. The throttle's `lastMotionNs`/`motionIntervalNs` went
// with it rather than being left set-but-unread nothing else consumed either.
// Total acceleration in g: gravity + user when split, else the raw vector then NEGATED
// into the wire's convention.
//
// Apple reports acceleration as the gravity VECTOR: a device lying flat face-up reads
// z = 1, because gravity points down. An accelerometer physically measures proper
// acceleration, which at rest is the +1 g normal force pushing UP, and that is what a
// DualSense's report the wire's convention carries. The two are exact negatives, so
// every sample we sent was upside down, on both branches (`m.acceleration` follows the
// same Apple convention as the gravity/user split).
//
// Measured on glass 2026-08-07 (G16): a DualSense flat and face-up, streamed from an
// iPhone to a Linux host, arrived at hid-playstation as z = 0.99 g where +1.00 was owed.
// Magnitude was 1.006 g, so the SCALE was already right this is purely direction.
// `rotationRate` is a true angular rate and needs no flip; the same session confirmed yaw
// came through with the correct sign.
let ax: Float
let ay: Float
let az: Float
if m.hasGravityAndUserAcceleration {
ax = Float(m.gravity.x + m.userAcceleration.x)
ay = Float(m.gravity.y + m.userAcceleration.y)
az = Float(m.gravity.z + m.userAcceleration.z)
ax = -Float(m.gravity.x + m.userAcceleration.x)
ay = -Float(m.gravity.y + m.userAcceleration.y)
az = -Float(m.gravity.z + m.userAcceleration.z)
} else {
ax = Float(m.acceleration.x)
ay = Float(m.acceleration.y)
az = Float(m.acceleration.z)
ax = -Float(m.acceleration.x)
ay = -Float(m.acceleration.y)
az = -Float(m.acceleration.z)
}
let gs = GamepadWire.gyroLSBPerRadS
let as_ = GamepadWire.accelLSBPerG
wire?.sendMotion(
pad: UInt8(slot.pad),
gyro: (
GamepadWire.motionRaw(Float(m.rotationRate.x), scale: gs),
GamepadWire.motionRaw(Float(m.rotationRate.y), scale: gs),
GamepadWire.motionRaw(Float(m.rotationRate.z), scale: gs)
),
accel: (
GamepadWire.motionRaw(ax, scale: as_),
GamepadWire.motionRaw(ay, scale: as_),
GamepadWire.motionRaw(az, scale: as_)
))
// Into the DualSense report frame. GameController and the pad's own report do not agree
// about which slot is which axis measured, both from the same controller, on 2026-08-07
// so forwarding GC's x/y/z straight through sent yaw where the game reads roll. See
// `GamepadWire.appleMotionToWire`. One change of basis, applied to both planes.
let g = GamepadWire.appleMotionToWire(
(Float(m.rotationRate.x), Float(m.rotationRate.y), Float(m.rotationRate.z)))
let a = GamepadWire.appleMotionToWire((ax, ay, az))
let gyro = (
GamepadWire.motionRaw(g.0, scale: gs),
GamepadWire.motionRaw(g.1, scale: gs),
GamepadWire.motionRaw(g.2, scale: gs)
)
let accel = (
GamepadWire.motionRaw(a.0, scale: as_),
GamepadWire.motionRaw(a.1, scale: as_),
GamepadWire.motionRaw(a.2, scale: as_)
)
// Recorded AFTER the frame conversion, deliberately: `flush` replays `lastAccel` beside a
// zero gyro, so it has to be the vector that actually went on the wire. Stashing the
// pre-conversion one would park a still pad's gravity in the wrong axis.
if wire != nil {
slot.motionSent = true
slot.lastAccel = accel
}
wire?.sendMotion(pad: UInt8(slot.pad), gyro: gyro, accel: accel)
}
/// Engage or stand down the phone-gyro mirror: it speaks for pad 0 exactly while a
/// forwarded controller holds that index but can't rotate for itself no `GCMotion`,
/// or a motion object without a rotation rate (gravity-only pads, e.g. an Xbox pad on
/// iOS). Re-evaluated on every reconcile and on suspend/resume; `DeviceGyro.stop`
/// parks the host pad's rotation at zero, so standing down never strands a spin.
private func updateDeviceGyro() {
#if os(iOS)
guard let gyro = deviceGyro else { return }
let pad0 = slots.first { $0.pad == 0 }
let wants = !suspended && pad0 != nil && pad0!.controller.motion?.hasRotationRate != true
if wants { gyro.start() } else { gyro.stop() }
#endif
}
/// Arm the disconnect timer when ANY forwarded pad holds the full escape chord, disarm the
@@ -634,6 +768,14 @@ public final class GamepadCapture {
wire?.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(f), active: false, x: 0, y: 0)
slot.fingerActive[f] = false
}
// Motion is host-side STATE, re-emitted until replaced a nonzero angular velocity
// left behind reads as endless rotation (the gyro-sweep latch: Control Center
// pull-down froze the last sample for as long as the overlay stayed up). Rest means
// zero rotation; the last accel is kept so gravity doesn't become free-fall.
if slot.motionSent {
slot.motionSent = false
wire?.sendMotion(pad: UInt8(slot.pad), gyro: (0, 0, 0), accel: slot.lastAccel)
}
}
/// Flush every open slot's held state (app deactivation) keeps the slots open (GC just stops
@@ -41,6 +41,10 @@ public final class GamepadManager: ObservableObject {
public let kind: PunktfunkConnection.GamepadType
public let hasLight: Bool
public let hasHaptics: Bool
/// This controller has a GYROSCOPE not merely a `GCMotion`. The distinction is the whole
/// point: an X-Box pad exposes a `GCMotion` that reports gravity and nothing else, so
/// `motion != nil` is true for a controller with no angular rate to give. Read
/// `hasRotationRate`, which is GameController's own answer to the question we mean.
public let hasMotion: Bool
public let hasAdaptiveTriggers: Bool
/// Specifically a DualSense (incl. the Edge same feedback surface) gates the
@@ -265,7 +269,10 @@ public final class GamepadManager: ObservableObject {
kind: kind,
hasLight: c.light != nil,
hasHaptics: c.haptics != nil,
hasMotion: c.motion != nil,
// `hasRotationRate`, not `motion != nil` see the property. The settings row shows a
// gyroscope badge off this, and promising a gyro an X-Box pad does not have is the
// same lie as streaming its non-existent rotation to the host.
hasMotion: c.motion?.hasRotationRate ?? false,
// GCDualSenseGamepad's triggers are GCDualSenseAdaptiveTrigger by declaration (the
// Edge included); the DualShock 4 has none.
hasAdaptiveTriggers: kind == .dualSense || kind == .dualSenseEdge,
@@ -73,6 +73,30 @@ public enum GamepadWire {
public static func motionRaw(_ value: Float, scale: Float) -> Int16 {
Int16((value * scale).rounded().clamped(to: Float(Int16.min)...Float(Int16.max)))
}
/// GameController's motion frame the DualSense report frame the wire is defined in.
///
/// The wire is a unit passthrough: the host writes these three components, in order, into the
/// virtual DualSense's report bytes 16../22.. the same slots a real pad fills. So the frame
/// the wire is defined in is the pad's OWN report frame, and a client that forwards its
/// platform's axes unconverted is simply speaking a different language.
///
/// Both frames were measured on 2026-08-07 from ONE physical DualSense on one desk the pad
/// read twice, over raw HID and through GameController:
///
/// DualSense report frame: (Right, Up, Backward) axis 0 carries pitch, 1 yaw, 2 roll
/// GameController frame: (Right, Forward, Up)
///
/// Matching them up: Right is already slot 0; Up is GC's z, so it moves to slot 1; and slot 2
/// wants Backward, which is GC's y negated. Hence `(x, z, -y)`.
///
/// Applied to gyro AND acceleration, because it is a change of basis and both are expressed in
/// that basis. The negation `forwardMotion` already does for acceleration is a separate matter
/// that one converts Apple's gravity-VECTOR convention into the proper acceleration a real
/// pad reports, and it composes with this rather than replacing it.
public static func appleMotionToWire(_ v: (Float, Float, Float)) -> (Float, Float, Float) {
(v.0, v.2, -v.1)
}
}
extension Float {
@@ -47,7 +47,7 @@ MANIFEST (crate version — SPDX license — source)
bytes 1.12.0 — MIT — https://github.com/tokio-rs/bytes
cast 0.3.0 — MIT OR Apache-2.0 — https://github.com/japaric/cast.rs
cbindgen 0.29.4 — MPL-2.0 — https://github.com/mozilla/cbindgen
cc 1.2.65 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cc 1.4.1 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cesu8 1.1.0 — Apache-2.0/MIT — https://github.com/emk/cesu8-rs
cfg-if 1.0.4 — MIT OR Apache-2.0 — https://github.com/rust-lang/cfg-if
cfg_aliases 0.2.1 — MIT — https://github.com/katharostech/cfg_aliases
@@ -85,7 +85,7 @@ MANIFEST (crate version — SPDX license — source)
fastbloom 0.14.1 — MIT OR Apache-2.0 — https://github.com/tomtomwombat/fastbloom/
fastrand 2.4.1 — Apache-2.0 OR MIT — https://github.com/smol-rs/fastrand
fiat-crypto 0.2.9 — MIT OR Apache-2.0 OR BSD-1-Clause — https://github.com/mit-plv/fiat-crypto
find-msvc-tools 0.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
find-msvc-tools 0.1.10 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
fixedbitset 0.5.7 — MIT OR Apache-2.0 — https://github.com/petgraph/fixedbitset
fnv 1.0.7 — Apache-2.0 / MIT — https://github.com/servo/rust-fnv
foldhash 0.2.0 — Zlib — https://github.com/orlp/foldhash
@@ -908,7 +908,7 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
----------------------------------------------------------------------------
The following license (LICENSE-APACHE) applies to: autocfg 1.5.1, base64 0.22.1, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.2.65, cfg-if 1.0.4, cmake 0.1.58, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, either 1.16.0, equivalent 1.0.2, errno 0.3.14, fastrand 2.4.1, find-msvc-tools 0.1.9, fixedbitset 0.5.7, fnv 1.0.7, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, indexmap 2.14.0, itertools 0.10.5, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking_lot 0.12.5, parking_lot_core 0.9.12, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, tempfile 3.27.0, tinytemplate 1.2.1, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1
The following license (LICENSE-APACHE) applies to: autocfg 1.5.1, base64 0.22.1, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.4.1, cfg-if 1.0.4, cmake 0.1.58, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, either 1.16.0, equivalent 1.0.2, errno 0.3.14, fastrand 2.4.1, find-msvc-tools 0.1.10, fixedbitset 0.5.7, fnv 1.0.7, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, indexmap 2.14.0, itertools 0.10.5, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking_lot 0.12.5, parking_lot_core 0.9.12, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, tempfile 3.27.0, tinytemplate 1.2.1, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1
----------------------------------------------------------------------------
Apache License
Version 2.0, January 2004
@@ -1953,7 +1953,7 @@ Exhibit B - "Incompatible With Secondary Licenses" Notice
----------------------------------------------------------------------------
The following license (LICENSE-MIT) applies to: cc 1.2.65, cfg-if 1.0.4, cmake 0.1.58, find-msvc-tools 0.1.9, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
The following license (LICENSE-MIT) applies to: cc 1.4.1, cfg-if 1.0.4, cmake 0.1.58, find-msvc-tools 0.1.10, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
----------------------------------------------------------------------------
Copyright (c) 2014 Alex Crichton
@@ -24,6 +24,10 @@ public final class LatencyMeter: @unchecked Sendable {
private let lock = NSLock()
private var samplesUs: [Int64] = []
private var skewCorrected = false
/// The most recent sample and the instant it ended, for `latestSample(asOfNs:maxAgeMs:)`
/// a LEVEL, not a window, so `drain` deliberately leaves both alone.
private var latestNs: Int64 = 0
private var latestAtNs: Int64 = 0
public init() {}
@@ -49,10 +53,42 @@ public final class LatencyMeter: @unchecked Sendable {
guard latNs > 0, latNs < 10_000_000_000 else { return }
lock.lock()
samplesUs.append(latNs / 1000)
latestNs = latNs
latestAtNs = atNs
if offsetNs != 0 { skewCorrected = true }
lock.unlock()
}
/// The most recent single sample in ns, or `nil` if none has landed or the last one ended more
/// than `maxAgeMs` before `nowNs` (both `CLOCK_REALTIME`). Unlike `drain`, this reports a level
/// rather than a window, and reading it consumes nothing.
///
/// **What it is for.** Read off the END-TO-END meter, this is the video plane's live
/// glass-to-glass figure `displayed + clockOffset pts`, exactly the shape `AvSync` compares
/// audio against and it is the reference the A/V sync loop needs. It is published from
/// `record`, so BOTH present paths (arrival and deadline) feed it without either knowing that
/// audio exists.
///
/// **Why staleness is not optional.** The number is a level, so absent an age check it would
/// simply keep its last value forever. This client has a state where that matters: the
/// backgrounded keep-alive keeps audio playing and DROPS video decode entirely, so the loop
/// would go on steering the ring against a reference minutes old and frozen. Expiring it
/// returns `nil`, which is the same "no reference yet" case as session start the loop holds
/// its last correction and stops chasing. `nowNs` is caller-supplied rather than read fresh so
/// the audio side compares against exactly the instant it timestamped its own frame at.
///
/// Only the PAST is bounded. A present stamp can legitimately sit a hair ahead of the reader's
/// clock (the deadline presenter stamps at the link's target present time), and discarding the
/// only reference we have over a fraction of a refresh would make it flap in and out; a stamp
/// wildly in the future instead yields a huge offset, which `AvSync` refuses on its own terms.
public func latestSample(asOfNs nowNs: Int64, maxAgeMs: Int) -> Int64? {
lock.lock()
defer { lock.unlock() }
guard latestNs > 0 else { return nil }
guard (nowNs &- latestAtNs) <= Int64(maxAgeMs) * 1_000_000 else { return nil }
return latestNs
}
public struct Stats: Sendable {
public let p50Ms: Double
public let p95Ms: Double
@@ -193,6 +193,14 @@ public enum DefaultsKey {
/// once per session by `GamepadFeedback`. The toggle is shown only where the device actually
/// has a haptic actuator (no iPad/Mac/TV).
public static let rumbleOnDevice = "punktfunk.rumbleOnDevice"
/// Use this device's own gyroscope as player 1's motion when the forwarded controller has
/// none of its own for clip-on and third-party pads without an IMU, where the device body
/// moves with the player's hands. The rumble mirror's sibling, data flowing the other way.
/// Off by default (opt-in); read once per session by `GamepadCapture`, whose `DeviceGyro`
/// mirror engages only while pad 0's controller reports no rotation rate (a real gyro pad
/// always wins). The toggle is shown only where the device has motion hardware
/// (`DeviceGyro.isAvailable`).
public static let gyroFromDevice = "punktfunk.gyroFromDevice"
/// Auto-wake on connect: when connecting to a saved host that isn't advertising on mDNS, fire
/// Wake-on-LAN and, if the dial fails, wait for it to come back before retrying (the "Waking"
/// overlay). On by default. Turn off if a host that's already on just isn't seen on mDNS (a
@@ -198,5 +198,315 @@ final class AudioRingDriftTests: XCTestCase {
silentTail, 0,
"after adapting, the last 3 s must play through the bunching without a dropout")
}
// MARK: - A/V sync (audio latency overhaul, W6)
//
// The second half of the same story. Depth alone is not correctness: a ring can be exactly as
// deep as its link needs and still put audio in the wrong place, because nothing ever compared
// it to the picture. `AvSync` measures that comparison and asks the ring to move; the ring is
// free to refuse. These pin both halves that the loop DOES act (the previous pass in this
// area shipped a correction that was structurally unreachable and had a green test), and that
// it can never act far enough to starve the callback.
/// Build an observation whose measured offset is exactly `offsetMS` (positive = audio late).
/// Mirrors the Rust `obs` helper: pin now/skew/pts so the only free term is the buffered depth,
/// then choose the video figure so the difference lands where we want it.
private func obs(offsetMS: Int, depth: Int) -> AvSync.Observation {
let bufferedMS = depth / perMS
let audioE2eMS = bufferedMS + 40 // 40 ms of transport, arbitrary but fixed
let videoE2eMS = audioE2eMS - offsetMS
return AvSync.Observation(
ptsNs: 1_000_000_000,
nowLocalNs: 1_000_000_000 + 40 * 1_000_000,
clockOffsetNs: 0,
bufferedAhead: depth,
videoE2eNs: Int64(max(0, videoE2eMS)) * 1_000_000)
}
/// Fold `n` identical observations in.
private func settle(_ sync: inout AvSync, offsetMS: Int, depth: Int, count: Int = 100) {
for _ in 0..<count { sync.observe(obs(offsetMS: offsetMS, depth: depth)) }
}
func testAvSyncNeedsEvidenceBeforeActing() {
var s = AvSync(channels: channels)
// One sample is never enough the skew estimate and the video figure both settle after
// connect, and acting on the first would chase the handshake, not the stream.
XCTAssertNil(s.observe(obs(offsetMS: 50, depth: 30 * perMS)))
XCTAssertFalse(s.settled)
XCTAssertNil(s.desiredDepth(currentDepth: 30 * perMS))
settle(&s, offsetMS: 50, depth: 30 * perMS, count: 99) // 1 + 99 = 100
XCTAssertTrue(s.settled, "should act once the evidence is in")
}
/// No frame on the glass no reference the loop says nothing, however many observations
/// arrive. This is the state every session starts in, and the one the stage-1 fallback
/// presenter stays in for its whole life.
func testAvSyncWithoutAVideoReferenceNeverActs() {
var s = AvSync(channels: channels)
for _ in 0..<500 {
s.observe(AvSync.Observation(
ptsNs: 1_000_000_000, nowLocalNs: 1_040_000_000, clockOffsetNs: 0,
bufferedAhead: 30 * perMS, videoE2eNs: nil))
}
XCTAssertFalse(s.settled)
XCTAssertNil(s.desiredDepth(currentDepth: 30 * perMS))
}
func testAvSyncAimsShallowerWhenAudioIsLate() {
let depth = 60 * perMS
var s = AvSync(channels: channels)
settle(&s, offsetMS: 40, depth: depth, count: 400)
guard let want = s.desiredDepth(currentDepth: depth) else {
return XCTFail("a 40 ms offset is actionable")
}
XCTAssertLessThan(want, depth, "audio late must aim shallower")
// The correction is the offset, not a guess at it.
let shedMS = (depth - want) / perMS
XCTAssertTrue((35...45).contains(shedMS), "should aim to shed ~40 ms, got \(shedMS)")
XCTAssertEqual(s.offsetMS, 40, "and report it, sign and all")
}
func testAvSyncAimsDeeperWhenAudioIsEarly() {
let depth = 20 * perMS
var s = AvSync(channels: channels)
settle(&s, offsetMS: -30, depth: depth, count: 400)
guard let want = s.desiredDepth(currentDepth: depth) else {
return XCTFail("a 30 ms offset is actionable")
}
XCTAssertGreaterThan(want, depth, "audio early must aim deeper")
XCTAssertEqual(s.offsetMS, -30)
}
func testAvSyncDeadbandsWhatNoOneCanHear() {
let depth = 30 * perMS
var s = AvSync(channels: channels)
settle(&s, offsetMS: 8, depth: depth, count: 400) // inside the 10 ms deadband
XCTAssertNil(
s.desiredDepth(currentDepth: depth),
"an offset inside the deadband must not provoke a (real, if crossfaded) discontinuity")
XCTAssertEqual(s.offsetMS, 8, "…but it is still REPORTED — the HUD shows the residual")
}
/// A wall-clock step or a stale video figure produces an enormous apparent misalignment.
/// Clamping it would act on a wrong number as though it were a small real one, so it is
/// refused outright and the running average is left untouched.
func testAvSyncRejectsTheImplausibleInsteadOfClampingIt() {
let depth = 30 * perMS
var s = AvSync(channels: channels)
settle(&s, offsetMS: 30, depth: depth, count: 400)
let before = s.offsetMS
// Built directly rather than through `obs`: that helper floors the video figure at zero,
// which would cap the offset at a merely LARGE value and let this pass without ever
// exercising the rejection.
let wild = AvSync.Observation(
ptsNs: 0, nowLocalNs: 5_000_000_000, clockOffsetNs: 0,
bufferedAhead: depth, videoE2eNs: 40_000_000)
XCTAssertNil(s.observe(wild))
XCTAssertTrue(s.implausible, "a ~5 s offset must be refused, not folded")
XCTAssertEqual(before, s.offsetMS, "an implausible sample must be discarded, not folded in")
}
/// The same refusal for arithmetic that cannot even be CARRIED OUT, which is why the terms are
/// combined with overflow-reporting operators rather than the wrapping `&-` the latency meters
/// use.
///
/// This input is not arbitrary. `ptsNs = 1 << 63` reads as `Int64.min` in two's complement, so
/// the audio leg overflows and the difference lands on EXACTLY `Int64.min` and `abs()` of
/// `Int64.min` has no representable result, so in Swift it traps. Check the overflow flags
/// after the sanity limit instead of before and this observation does not mis-measure the
/// stream, it aborts the process, from the audio drain thread. The guard's short-circuit
/// ordering is what makes the sanity check itself safe to run.
func testAvSyncRefusesAnOffsetItCannotEvenCompute() {
var s = AvSync(channels: channels)
let wild = AvSync.Observation(
ptsNs: 1 << 63, nowLocalNs: 40_000_000, clockOffsetNs: 0,
bufferedAhead: 0, videoE2eNs: 40_000_000)
XCTAssertNil(s.observe(wild))
XCTAssertTrue(s.implausible)
XCTAssertFalse(s.settled, "a refused sample is not evidence")
XCTAssertEqual(s.offsetMS, 0, "and nothing of it was folded in")
}
// MARK: - and what the ring does with the proposal
/// Drive one read so the ring knows the device quantum (`renderQuantum` seeds the floor).
private func primeQuantum(_ ring: AudioRing, quantumMS: Int) {
var scratch = [Float](repeating: 0, count: quantumMS * perMS)
scratch.withUnsafeMutableBufferPointer { ring.read(into: $0.baseAddress!, count: $0.count) }
}
/// The loop is NOT inert: a settled proposal inside the ring's legal band actually moves the
/// effective target. Without this the whole feature could ship as unreachable code with every
/// other test still green which is exactly how the previous drift correction shipped dead.
func testSyncActuallyMovesTheTarget() {
let ring = AudioRing(capacity: 48_000 * channels, channels: channels)
primeQuantum(ring, quantumMS: 5)
XCTAssertEqual(ring.stats.targetMS, 20, "base target (JitterTuning.COREAUDIO)")
// Audio 30 ms EARLY at a 20 ms depth aim 50 ms deep: above the floor, under the 90 ms
// cap, so the ring has no reason to refuse.
var s = AvSync(channels: channels)
settle(&s, offsetMS: -30, depth: 20 * perMS, count: 400)
ring.setSyncTarget(s.desiredDepth(currentDepth: 20 * perMS))
XCTAssertEqual(ring.stats.targetMS, 50, "the ring must adopt a legal request")
// And releasing it returns the ring to exactly where it was.
ring.setSyncTarget(nil)
XCTAssertEqual(ring.stats.targetMS, 20)
}
/// THE safety invariant: sync only ever proposes. Continuity the underrun-driven floor
/// outranks it, or a lossy link would be "synced" into dropouts. Pinned against a GROWN floor,
/// not just the base, because the floor sync is most likely to argue with is the one a bad link
/// earned.
func testSyncCanNeverStarveTheRing() {
let ring = AudioRing(capacity: 48_000 * channels, channels: channels)
let want = 5 * perMS
var scratch = [Float](repeating: 0, count: want)
let feed = [Float](repeating: 0.5, count: 25 * perMS)
func write(ms: Int) {
feed.withUnsafeBufferPointer { ring.write($0.baseAddress!, count: ms * perMS) }
}
func read() {
scratch.withUnsafeMutableBufferPointer { ring.read(into: $0.baseAddress!, count: want) }
}
// Grow the floor above the base with three clustered genuine underruns (same shape as
// testTargetGrowsOnUnderrunsAndRelaxesWhenQuiet).
write(ms: 25)
for _ in 0..<5 { read() }
read()
write(ms: 5); read()
read()
write(ms: 5); read()
read()
let floor = ring.stats.targetMS
XCTAssertGreaterThan(floor, 20, "the test needs a GROWN floor to be meaningful")
// Ask for an absurdly shallow ring zero.
ring.setSyncTarget(0)
XCTAssertEqual(
ring.stats.targetMS, floor,
"sync pulled the target below the continuity floor — a link that needs the buffer must "
+ "keep it, and the residual gets reported instead")
// One frame under the floor is still under the floor.
ring.setSyncTarget(floor * perMS - perMS)
XCTAssertEqual(ring.stats.targetMS, floor)
// And it may not blow past the hard cap either added latency stays bounded.
ring.setSyncTarget(Int.max / 2)
XCTAssertLessThanOrEqual(ring.stats.targetMS, 90, "sync pushed the target past the hard cap")
}
/// A device whose callback quantum alone exceeds the hard cap puts the continuity floor ABOVE
/// the ceiling. The floor must win: clamping naively (`min(max(s, floor), cap)`) would hand
/// back the cap quietly below the floor, inverting the whole ordering on exactly the
/// awkward hardware this code exists to survive.
func testAHugeDeviceQuantumDoesNotInvertTheClamp() {
let ring = AudioRing(capacity: 48_000 * channels * 2, channels: channels)
let quantumMS = 500 // absurd, but not a reason to starve the callback
primeQuantum(ring, quantumMS: quantumMS)
ring.setSyncTarget(0)
XCTAssertGreaterThanOrEqual(
ring.stats.targetMS, quantumMS,
"the target must still be able to serve one callback")
}
/// A ring that ratcheted during a transient must not hold audio late for minutes after the
/// cause is gone: with sync asking for less, the relax window is the short one.
func testSyncPressureRelaxesAGrownTargetSoonerThanTimeAlone() {
let want = 5 * perMS
func grow(_ ring: AudioRing) {
var scratch = [Float](repeating: 0, count: want)
let feed = [Float](repeating: 0.5, count: 25 * perMS)
func write(ms: Int) {
feed.withUnsafeBufferPointer { ring.write($0.baseAddress!, count: ms * perMS) }
}
func read() {
scratch.withUnsafeMutableBufferPointer {
ring.read(into: $0.baseAddress!, count: want)
}
}
write(ms: 25)
for _ in 0..<5 { read() }
read()
write(ms: 5); read()
read()
write(ms: 5); read()
read()
}
/// Quiet (full) reads needed before the grown target relaxes one step.
func quietToRelax(_ ring: AudioRing) -> Int {
var scratch = [Float](repeating: 0, count: want)
let feed = [Float](repeating: 0.5, count: 5 * perMS)
let start = ring.stats.targetMS
var reads = 0
while ring.stats.targetMS == start, reads < 200_000 {
feed.withUnsafeBufferPointer { ring.write($0.baseAddress!, count: 5 * perMS) }
scratch.withUnsafeMutableBufferPointer {
ring.read(into: $0.baseAddress!, count: want)
}
reads += 1
}
return reads
}
let slow = AudioRing(capacity: 48_000 * channels, channels: channels)
grow(slow)
slow.setSyncTarget(nil)
let slowReads = quietToRelax(slow)
let fast = AudioRing(capacity: 48_000 * channels, channels: channels)
grow(fast)
fast.setSyncTarget(perMS) // strictly shallower than the grown target
let fastReads = quietToRelax(fast)
XCTAssertLessThan(
fastReads, slowReads,
"sync pressure should relax sooner: \(fastReads) vs \(slowReads) quiet reads")
}
/// The four client rings adopt sync one at a time; an un-wired one must behave exactly as it
/// did. `nil` is the default, so this pins the initializer too and every other test in this
/// file runs without a sync target, which is the real guard that nothing moved underneath them.
func testNoSyncTargetLeavesTheRingExactlyAsItWas() {
let a = AudioRing(capacity: 48_000 * channels, channels: channels)
let b = AudioRing(capacity: 48_000 * channels, channels: channels)
b.setSyncTarget(nil)
let want = 5 * perMS
var sa = [Float](repeating: 0, count: want)
var sb = [Float](repeating: 0, count: want)
let feed = [Float](repeating: 0.5, count: 30 * perMS)
for step in 0..<4_000 {
// Uneven delivery so the depth actually moves around and the two rings have something
// to disagree about.
if step % 7 == 0 {
for r in [a, b] {
feed.withUnsafeBufferPointer { r.write($0.baseAddress!, count: 30 * perMS) }
}
}
sa.withUnsafeMutableBufferPointer { a.read(into: $0.baseAddress!, count: want) }
sb.withUnsafeMutableBufferPointer { b.read(into: $0.baseAddress!, count: want) }
XCTAssertEqual(sa, sb, "step \(step): an explicit nil diverged from the default")
XCTAssertEqual(a.stats.targetMS, b.stats.targetMS, "step \(step)")
}
}
/// The reporting half of §1.3: the offset must reach the same snapshot the depth does, because
/// a depth on its own cannot distinguish "deep because the link needs it" from "deep and
/// therefore late". This is the number the HUD and the 1 Hz log line read.
func testAvOffsetIsReportedAlongsideTheDepth() {
let ring = AudioRing(capacity: 48_000 * channels, channels: channels)
XCTAssertEqual(ring.stats.avOffsetMS, 0, "no evidence yet reads as zero, not as noise")
let feed = [Float](repeating: 0.5, count: 30 * perMS)
feed.withUnsafeBufferPointer { ring.write($0.baseAddress!, count: 30 * perMS) }
var s = AvSync(channels: channels)
settle(&s, offsetMS: 37, depth: 30 * perMS, count: 400)
ring.noteAvOffset(s.offsetMS)
let stats = ring.stats
XCTAssertEqual(stats.bufferedMS, 30)
XCTAssertEqual(stats.avOffsetMS, 37, "positive = audio behind the picture")
}
}
#endif
@@ -0,0 +1,62 @@
// Pins the phone-gyro mirror's devicecontroller frame remap (DeviceGyro.swift). The matrix is
// derived (like the wire scale constants), so these tests are the contract: if on-glass says an
// axis is wrong, fix the enum AND these expectations together.
#if os(iOS)
import UIKit
import XCTest
@testable import PunktfunkKit
final class DeviceGyroRemapTests: XCTestCase {
/// A distinct vector per axis so a swapped or flipped component can't cancel out.
private let v: (x: Float, y: Float, z: Float) = (1, 2, 3)
func testPortraitIsIdentity() {
let r = DeviceGyroRemap.identity.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [1, 2, 3])
}
func testUpsideDownFlipsInPlane() {
let r = DeviceGyroRemap.flipped.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [-1, -2, 3])
}
/// Device top to the player's LEFT: player-right = device-bottom (y), player-up =
/// device-right (+x). z (out of the screen) never changes the screen faces the player.
func testTopLeftLandscape() {
let r = DeviceGyroRemap.topLeft.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [-2, 1, 3])
}
/// Device top to the player's RIGHT: player-right = device-top (+y), player-up =
/// device-left (x).
func testTopRightLandscape() {
let r = DeviceGyroRemap.topRight.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [2, -1, 3])
}
/// Interface orientation remap: `.landscapeRight` means the Home edge is on the
/// player's right, i.e. the device top points LEFT (and vice versa).
func testOrientationMapping() {
XCTAssertEqual(DeviceGyroRemap(.portrait), .identity)
XCTAssertEqual(DeviceGyroRemap(.portraitUpsideDown), .flipped)
XCTAssertEqual(DeviceGyroRemap(.landscapeRight), .topLeft)
XCTAssertEqual(DeviceGyroRemap(.landscapeLeft), .topRight)
XCTAssertEqual(DeviceGyroRemap(.unknown), .identity)
}
/// Every remap must stay a proper rotation (right-handed): x̂ × ŷ = after mapping.
func testHandednessPreserved() {
for remap in [DeviceGyroRemap.identity, .flipped, .topLeft, .topRight] {
let x = remap.apply(x: 1, y: 0, z: 0)
let y = remap.apply(x: 0, y: 1, z: 0)
// Cross product of the two mapped in-plane basis vectors.
let cross = (
x: x.y * 0 - 0 * y.y, y: 0 * y.x - x.x * 0, z: x.x * y.y - x.y * y.x
)
XCTAssertEqual(cross.z, 1, "left-handed remap: \(remap)")
}
}
}
#endif
@@ -0,0 +1,91 @@
// The motion frame conversion, pinned against the readings it was derived from.
//
// On 2026-08-07 one physical DualSense was read twice on one desk over raw HID (the pad's own
// report) and through GameController so both frames come from the same controller in the same
// orientations rather than from two documents:
//
// DualSense report frame: (Right, Up, Backward) axis 0 pitch, 1 yaw, 2 roll
// GameController frame: (Right, Forward, Up)
//
// The numbers below are those measurements. They are the reason the conversion is `(x, z, -y)` and
// not one of the five other permutations that also move gravity to slot 1, so they belong in a test
// rather than only in a commit message.
import XCTest
@testable import PunktfunkKit
final class GamepadMotionFrameTests: XCTestCase {
private func wire(_ v: (Float, Float, Float)) -> (Float, Float, Float) {
GamepadWire.appleMotionToWire(v)
}
/// Gravity at rest, face up. MEASURED: GameController read (+0.005, -0.192, +0.992) g while raw
/// HID on the same pad read (+0.021, +0.997, +0.160). The conversion has to carry one into the
/// other including the small tilt term, which is what distinguishes this mapping from the one
/// that merely gets gravity onto the right slot.
func testRestingGravityLandsInTheDualSenseFrame() {
let apple: (Float, Float, Float) = (0.005, -0.192, 0.992)
let w = wire(apple)
XCTAssertEqual(w.0, 0.005, accuracy: 0.001, "right stays on slot 0")
XCTAssertEqual(w.1, 0.992, accuracy: 0.001, "up moves to slot 1 — the pad reads +1 g here")
XCTAssertEqual(w.2, 0.192, accuracy: 0.001, "slot 2 is Backward, so GC's Forward negates")
// The hardware's own reading of the same pose, to the precision two sessions of holding a
// controller by hand can agree to.
XCTAssertEqual(w.1, 0.997, accuracy: 0.02)
XCTAssertEqual(w.2, 0.160, accuracy: 0.05)
}
/// The tilt term's SIGN is the whole point: before this conversion the client sent Apple's y
/// straight through, so a pad tilted nose-up reported itself tilted nose-down.
func testTheForeAftAxisIsNegatedNotJustMoved() {
XCTAssertEqual(wire((0, 1, 0)).2, -1, "GC +y (Forward) is the wire's -Backward")
XCTAssertEqual(wire((0, -1, 0)).2, 1)
XCTAssertEqual(wire((0, 1, 0)).0, 0, "and it must not leak into the other slots")
XCTAssertEqual(wire((0, 1, 0)).1, 0)
}
/// Each rotation, as measured, must reach the slot the wire reads it from: the wire's gyro is
/// documented pitch/yaw/roll in slots 0/1/2, and the raw-HID run confirmed the pad agrees.
func testEachRotationReachesItsWireSlot() {
// Yaw is the reliable direct measurement a continuous one-way spin, clockwise from above,
// read as NEGATIVE on GC's z. It must arrive negative on slot 1, where the pad puts yaw.
let yaw = wire((-0.2, 21.7, -122.2))
XCTAssertEqual(yaw.1, -122.2, accuracy: 0.01)
XCTAssertLessThan(yaw.1, 0, "clockwise-from-above is negative about +Up, both frames agree")
// Pitch: nose-down about Right stays on slot 0 and keeps its sign.
let pitch = wire((-79.4, 0, 0))
XCTAssertEqual(pitch.0, -79.4, accuracy: 0.01)
// Roll: about the fore-aft axis, which moves to slot 2 AND flips.
let roll = wire((0, 61.8, 0))
XCTAssertEqual(roll.2, -61.8, accuracy: 0.01)
}
/// A change of basis is linear and orthonormal: it may not stretch a vector, and applying it to
/// gyro and to acceleration must be the same operation. Both are asserted because the capture
/// path calls it twice, on two different quantities.
func testConversionIsAnIsometry() {
for v in [(1, 2, 3), (-4, 5, -6), (0, 0, 1), (7, 0, 0)] as [(Float, Float, Float)] {
let w = wire(v)
let before = (v.0 * v.0 + v.1 * v.1 + v.2 * v.2).squareRoot()
let after = (w.0 * w.0 + w.1 * w.1 + w.2 * w.2).squareRoot()
XCTAssertEqual(before, after, accuracy: 1e-4, "must not change magnitude")
}
}
/// Right-handed in, right-handed out. A permutation with the wrong number of sign flips is a
/// REFLECTION, which reads as plausible on every single axis and inverts every rotation the
/// exact failure this measurement exists to prevent.
func testHandednessIsPreserved() {
let x = wire((1, 0, 0))
let y = wire((0, 1, 0))
// x cross y must equal the image of z, not its negative.
let cx = (x.1 * y.2 - x.2 * y.1, x.2 * y.0 - x.0 * y.2, x.0 * y.1 - x.1 * y.0)
let z = wire((0, 0, 1))
XCTAssertEqual(cx.0, z.0, accuracy: 1e-5)
XCTAssertEqual(cx.1, z.1, accuracy: 1e-5)
XCTAssertEqual(cx.2, z.2, accuracy: 1e-5)
}
}
@@ -0,0 +1,62 @@
// Whether a given pad's motion can reach the game. The Swift half of punktfunk-core's
// `pad_motion_reaches` same rows as `config::tests::motion_reach_is_answered_per_pad_not_per_session`,
// because a client that disagrees with the host about this either kills a working gyro or keeps
// streaming ~250 Hz of samples nobody reads, and both failures are silent.
import PunktfunkCore
import XCTest
@testable import PunktfunkKit
final class GamepadMotionReachTests: XCTestCase {
private typealias Pad = PunktfunkConnection.GamepadType
func testOnlyTheXboxClassesLackAMotionPlane() {
for kind: Pad in [.xbox360, .xboxOne] {
XCTAssertFalse(kind.hasMotion, "\(kind) should have no motion plane")
}
for kind: Pad in [
.dualSense, .dualShock4, .dualSenseEdge, .switchPro,
.steamController, .steamDeck, .steamController2,
] {
XCTAssertTrue(kind.hasMotion, "\(kind) should carry motion")
}
// Unknown must not suppress: an older host that omitted the echo may well have resolved a
// DualSense, and silently killing its gyro is worse than sending into a void.
XCTAssertTrue(Pad.auto.hasMotion)
}
/// The per-pad question, case by case. Each row is a session a player can actually sit down to;
/// the comment says which of the three inputs decides it.
func testMotionReachIsAnsweredPerPadNotPerSession() {
// The case this predicate exists for, and the one a session-level check gets WRONG:
// "Automatic" with mixed pads. The handshake carries the active pad's kind (an X-Box pad),
// so the echo says X-Box 360 but pad 1 declared a DualSense and the host built it one,
// with a motion plane. Reading the echo here kills a gyro that works.
XCTAssertTrue(Pad.motionReaches(declared: .dualSense, asked: .xbox360, resolved: .xbox360))
// Its mirror: the pad that DID declare the X-Box kind still has nowhere to put motion.
XCTAssertFalse(Pad.motionReaches(declared: .xbox360, asked: .xbox360, resolved: .xbox360))
// An explicit Switch Pro against a WINDOWS host, which folds it to X-Box 360. Declared ==
// asked, so the echo is this pad's answer and catches a fold nothing local could predict.
XCTAssertFalse(
Pad.motionReaches(declared: .switchPro, asked: .switchPro, resolved: .xbox360))
// The same declaration against a Linux host that builds it: unchanged, motion reaches.
XCTAssertTrue(
Pad.motionReaches(declared: .switchPro, asked: .switchPro, resolved: .switchPro))
// A DualSense wish on a host with no usable /dev/uhid degrades the same way.
XCTAssertFalse(
Pad.motionReaches(declared: .dualSense, asked: .dualSense, resolved: .xbox360))
// Nobody connected at dial time, so the handshake asked `.auto` and the host resolved it
// from its own env. A pad that shows up later declares its own kind and is judged on that.
XCTAssertTrue(Pad.motionReaches(declared: .dualSense, asked: .auto, resolved: .xbox360))
XCTAssertFalse(Pad.motionReaches(declared: .xbox360, asked: .auto, resolved: .dualSense))
// An old host that echoes nothing leaves `.auto`, which must not suppress.
XCTAssertTrue(Pad.motionReaches(declared: .dualSense, asked: .dualSense, resolved: .auto))
// Even then the declaration still speaks when it is the thing without a plane.
XCTAssertFalse(Pad.motionReaches(declared: .xbox360, asked: .dualSense, resolved: .auto))
}
}
@@ -71,4 +71,65 @@ final class LatencyMeterTests: XCTestCase {
m.record(ptsNs: now - 20_000_000_000, offsetNs: 0)
XCTAssertNil(m.drain())
}
// MARK: - latestSample: the A/V sync loop's video reference
/// The end-to-end meter doubles as the reference the audio ring steers against, so its most
/// recent sample must be readable as a LEVEL without consuming it, and independently of the
/// 1 Hz percentile window the HUD drains.
func testLatestSampleSurvivesDrainAndIsNotAWindow() {
let m = LatencyMeter()
let atNs: Int64 = 1_000_000_000_000
m.record(ptsNs: UInt64(atNs - 12_000_000), atNs: atNs, offsetNs: 0) // 12 ms
XCTAssertEqual(m.latestSample(asOfNs: atNs, maxAgeMs: 500), 12_000_000)
_ = m.drain()
XCTAssertEqual(
m.latestSample(asOfNs: atNs, maxAgeMs: 500), 12_000_000,
"the reference is a level — draining the percentile window must not clear it")
// and it tracks the newest frame.
m.record(ptsNs: UInt64(atNs - 20_000_000), atNs: atNs, offsetNs: 0)
XCTAssertEqual(m.latestSample(asOfNs: atNs, maxAgeMs: 500), 20_000_000)
}
/// No frame yet no reference. This is what keeps the sync loop inert at session start and
/// under the stage-1 presenter, which stamps no present at all.
func testLatestSampleIsNilBeforeAnyFrame() {
XCTAssertNil(LatencyMeter().latestSample(asOfNs: 1_000_000_000_000, maxAgeMs: 500))
}
/// THE staleness gate: video can stop while audio keeps playing (the backgrounded keep-alive
/// drops decode entirely). A level with no expiry would go on offering a minutes-old figure as
/// though it were live, and the ring would be steered against a frozen reference.
func testLatestSampleExpires() {
let m = LatencyMeter()
let atNs: Int64 = 1_000_000_000_000
m.record(ptsNs: UInt64(atNs - 12_000_000), atNs: atNs, offsetNs: 0)
XCTAssertNotNil(m.latestSample(asOfNs: atNs + 499_000_000, maxAgeMs: 500))
XCTAssertNil(
m.latestSample(asOfNs: atNs + 501_000_000, maxAgeMs: 500),
"a stale reference must read as NO reference, not as a live one")
// A stamp marginally ahead of the reader's clock is normal (the deadline presenter stamps
// at the link's TARGET present time) and must not drop the only reference we have.
XCTAssertNotNil(m.latestSample(asOfNs: atNs - 8_000_000, maxAgeMs: 500))
}
/// A sample the meter refused must not become a reference either the sync loop would then be
/// steered by a value the percentile window itself judged absurd.
///
/// The ABSURDLY LARGE case is the load-bearing one: a negative interval would also be stopped
/// by `latestSample`'s own `> 0` check, so on its own it proves nothing about where the publish
/// sits relative to the guard.
func testRefusedSampleIsNotPublishedAsAReference() {
let m = LatencyMeter()
let atNs: Int64 = 1_000_000_000_000
m.record(ptsNs: UInt64(atNs - 20_000_000_000), atNs: atNs, offsetNs: 0) // 20 s refused
XCTAssertNil(
m.latestSample(asOfNs: atNs, maxAgeMs: 500),
"a sample too absurd for the window is too absurd to steer the ring")
m.record(ptsNs: UInt64(atNs + 1), atNs: atNs, offsetNs: 0) // negative interval
XCTAssertNil(m.latestSample(asOfNs: atNs, maxAgeMs: 500))
// and a good sample after them still lands, so the refusals cost nothing.
m.record(ptsNs: UInt64(atNs - 9_000_000), atNs: atNs, offsetNs: 0)
XCTAssertEqual(m.latestSample(asOfNs: atNs, maxAgeMs: 500), 9_000_000)
}
}
@@ -78,4 +78,234 @@ final class LibraryClientTests: XCTestCase {
XCTAssertEqual(resolved.hero, "https://cdn.example.com/hero.jpg") // unchanged
XCTAssertNil(resolved.logo)
}
// MARK: - HTTP response parsing (MgmtTransport)
// The management API is reached over Network.framework rather than URLSession (ATS cannot be
// relaxed for the arbitrary addresses a host lives at see MgmtTransport), so we parse HTTP
// ourselves. These cover the framings hyper actually emits, plus the failure modes where
// getting it wrong would be silent.
private func raw(_ text: String) -> Data { Data(text.utf8) }
func testParsesContentLengthFramedJSON() throws {
let body = #"[{"id":"steam:570"}]"#
let response = try HTTPResponseParser.parse(raw(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\n"
+ "Content-Length: \(body.utf8.count)\r\n\r\n\(body)"))
XCTAssertEqual(response.status, 200)
XCTAssertEqual(String(decoding: response.body, as: UTF8.self), body)
// Field names are case-insensitive per RFC 9110.
XCTAssertEqual(response.header("CONTENT-TYPE"), "application/json")
}
func testParsesChunkedBody() throws {
// How hyper streams the art proxy.
let response = try HTTPResponseParser.parse(raw(
"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n"
+ "5\r\nhello\r\n6\r\n world\r\n0\r\n\r\n"))
XCTAssertEqual(String(decoding: response.body, as: UTF8.self), "hello world")
}
func testChunkExtensionsAndTrailersAreIgnored() throws {
let response = try HTTPResponseParser.parse(raw(
"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n"
+ "3;foo=bar\r\nabc\r\n0\r\nX-Trailer: 1\r\n\r\n"))
XCTAssertEqual(String(decoding: response.body, as: UTF8.self), "abc")
}
func testUnauthorizedStatusSurvives() throws {
// What an unpaired certificate gets from the host the status is the whole signal.
let response = try HTTPResponseParser.parse(
raw("HTTP/1.1 401 Unauthorized\r\nContent-Length: 0\r\n\r\n"))
XCTAssertEqual(response.status, 401)
XCTAssertTrue(response.body.isEmpty)
}
func testBodyRunsToEOFWithoutFramingHeaders() throws {
let response = try HTTPResponseParser.parse(raw("HTTP/1.1 200 OK\r\n\r\nraw-to-eof"))
XCTAssertEqual(String(decoding: response.body, as: UTF8.self), "raw-to-eof")
}
func testTruncatedBodyThrowsRatherThanReturningPartialJSON() {
// The one that matters: a body cut short must NOT come back as success. A clipped JSON
// array would decode to fewer games "this host has no games" instead of an error.
XCTAssertThrowsError(
try HTTPResponseParser.parse(raw("HTTP/1.1 200 OK\r\nContent-Length: 99\r\n\r\nshort")))
}
func testOverLongBodyIsClippedToContentLength() throws {
let response = try HTTPResponseParser.parse(
raw("HTTP/1.1 200 OK\r\nContent-Length: 3\r\n\r\nabcdef"))
XCTAssertEqual(String(decoding: response.body, as: UTF8.self), "abc")
}
func testMalformedResponsesThrow() {
// Header block never terminated (peer hung up), a non-HTTP greeting, and a chunked stream
// cut mid-chunk.
XCTAssertThrowsError(
try HTTPResponseParser.parse(raw("HTTP/1.1 200 OK\r\nContent-Length: 3\r\n")))
XCTAssertThrowsError(try HTTPResponseParser.parse(raw("NOT-HTTP\r\n\r\n")))
XCTAssertThrowsError(try HTTPResponseParser.parse(raw(
"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n9\r\nabc")))
}
func testMultiWordReasonPhraseParses() throws {
let response = try HTTPResponseParser.parse(
raw("HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\n\r\n"))
XCTAssertEqual(response.status, 404)
}
func testBinaryBodySurvivesByteForByte() throws {
// Posters are PNG/JPEG: the body must never be round-tripped through a String.
let png: [UInt8] = [0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, 0x00, 0xFF, 0x0D, 0x0A]
var message = raw("HTTP/1.1 200 OK\r\nContent-Length: \(png.count)\r\n\r\n")
message.append(contentsOf: png)
let response = try HTTPResponseParser.parse(message)
XCTAssertEqual([UInt8](response.body), png)
}
// MARK: - Message framing (keep-alive)
// Connections are pooled and reused, so a response has to be delimited WITHOUT waiting for
// the peer to hang up. Getting this wrong either truncates a response or bleeds one response
// into the next, and both would be silent.
func testMessageLengthDelimitsContentLengthFraming() throws {
let complete = raw("HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nhello")
XCTAssertEqual(try HTTPResponseParser.messageLength(in: complete), complete.count)
XCTAssertNil(try HTTPResponseParser.messageLength(
in: raw("HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nhel")))
}
func testMessageLengthDelimitsChunkedFraming() throws {
let complete = raw("HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n\r\n")
XCTAssertEqual(try HTTPResponseParser.messageLength(in: complete), complete.count)
// Mid-chunk, and terminal chunk without its closing blank line.
XCTAssertNil(try HTTPResponseParser.messageLength(
in: raw("HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhel")))
XCTAssertNil(try HTTPResponseParser.messageLength(
in: raw("HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n5\r\nhello\r\n0\r\n")))
// Trailers belong to the message and must be consumed with it.
let trailered = raw("HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n"
+ "3\r\nabc\r\n0\r\nX-Trailer: 1\r\n\r\n")
XCTAssertEqual(try HTTPResponseParser.messageLength(in: trailered), trailered.count)
}
func testMessageLengthIsNilWithoutFraming() throws {
// No Content-Length and not chunked the body runs to EOF and the connection can't be
// reused. Partial header blocks are likewise "not yet".
XCTAssertNil(try HTTPResponseParser.messageLength(in: raw("HTTP/1.1 200 OK\r\n\r\nto-eof")))
XCTAssertNil(try HTTPResponseParser.messageLength(in: raw("HTTP/1.1 200 OK\r\nContent-Len")))
}
func testBackToBackResponsesSplitExactly() throws {
// The reuse case that matters: two responses arriving in one read.
let first = raw("HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nhello")
let second = raw("HTTP/1.1 200 OK\r\nContent-Length: 3\r\n\r\nbye")
let both = first + second
XCTAssertEqual(try HTTPResponseParser.messageLength(in: both), first.count)
XCTAssertEqual(
String(decoding: try HTTPResponseParser.parse(both.prefix(first.count)).body,
as: UTF8.self), "hello")
XCTAssertEqual(
String(decoding: try HTTPResponseParser.parse(both.dropFirst(first.count)).body,
as: UTF8.self), "bye")
}
func testConnectionCloseIsDetected() throws {
let response = try HTTPResponseParser.parse(
raw("HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 0\r\n\r\n"))
XCTAssertTrue(response.wantsClose)
// HTTP/1.1 keeps the connection open unless told otherwise.
XCTAssertFalse(try HTTPResponseParser.parse(
raw("HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n")).wantsClose)
}
// MARK: - Art cache
private func temporaryCacheDirectory() -> URL {
URL(fileURLWithPath: NSTemporaryDirectory())
.appendingPathComponent("pf-art-test-\(UUID().uuidString)", isDirectory: true)
}
func testArtCacheRoundTripsBinaryAndSeparatesKeys() async throws {
let directory = temporaryCacheDirectory()
defer { try? FileManager.default.removeItem(at: directory) }
let cache = ArtCache(directory: directory)
let png = Data([0x89, 0x50, 0x4E, 0x47, 0x00, 0xFF, 0x0D, 0x0A])
let portrait = URL(string: "https://100.64.1.2:47990/api/v1/library/art/steam:570/portrait")!
let header = URL(string: "https://100.64.1.2:47990/api/v1/library/art/steam:570/header")!
var hit = await cache.data(for: portrait)
XCTAssertNil(hit, "cold cache must miss")
await cache.store(png, for: portrait)
hit = await cache.data(for: portrait)
XCTAssertEqual(hit, png, "posters are binary; the body must survive byte-for-byte")
// Sibling art of the same title must not collide.
let sibling = await cache.data(for: header)
XCTAssertNil(sibling)
}
func testArtCacheRefusesEmptyAndInlineData() async throws {
let directory = temporaryCacheDirectory()
defer { try? FileManager.default.removeItem(at: directory) }
let cache = ArtCache(directory: directory)
let empty = URL(string: "https://cdn.example.com/empty.jpg")!
await cache.store(Data(), for: empty)
let emptyHit = await cache.data(for: empty)
XCTAssertNil(emptyHit, "an empty body is not art")
let inline = URL(string: "data:image/png;base64,iVBORw0KGgo=")!
await cache.store(Data("x".utf8), for: inline)
let inlineHit = await cache.data(for: inline)
XCTAssertNil(inlineHit, "a data: URL is already inline — caching it is a pure loss")
}
func testArtCacheAgesEntriesOut() async throws {
let directory = temporaryCacheDirectory()
defer { try? FileManager.default.removeItem(at: directory) }
let cache = ArtCache(directory: directory, maxAge: 0.4)
let url = URL(string: "https://cdn.example.com/stale.jpg")!
await cache.store(Data("stale".utf8), for: url)
let fresh = await cache.data(for: url)
XCTAssertNotNil(fresh)
try await Task.sleep(nanoseconds: 700_000_000)
let expired = await cache.data(for: url)
XCTAssertNil(expired)
}
func testArtCacheEvictsLeastRecentlyUsedOverBudget() async throws {
let directory = temporaryCacheDirectory()
defer { try? FileManager.default.removeItem(at: directory) }
// Budget holds three of these; the fourth store must evict.
let cache = ArtCache(directory: directory, maxBytes: 300)
let blob = Data(repeating: 0x41, count: 100)
var urls: [URL] = []
for i in 0..<4 {
let url = URL(string: "https://cdn.example.com/blob\(i).jpg")!
urls.append(url)
await cache.store(blob, for: url)
try await Task.sleep(nanoseconds: 60_000_000) // distinct mtimes for LRU ordering
}
let evicted = await cache.data(for: urls[0])
XCTAssertNil(evicted, "the oldest entry should have been evicted")
let newest = await cache.data(for: urls[3])
XCTAssertEqual(newest, blob)
}
func testBaseURLBracketsIPv6Only() {
XCTAssertEqual(LibraryClient.baseURL(address: "192.168.1.70", port: 47990),
"https://192.168.1.70:47990")
XCTAssertEqual(LibraryClient.baseURL(address: "100.101.102.103", port: 47990),
"https://100.101.102.103:47990")
XCTAssertEqual(LibraryClient.baseURL(address: "fd7a:115c::1", port: 47990),
"https://[fd7a:115c::1]:47990")
// An address the user pasted already bracketed must not end up double-bracketed.
XCTAssertEqual(LibraryClient.baseURL(address: "[fd7a:115c::1]", port: 47990),
"https://[fd7a:115c::1]:47990")
}
}
+5 -5
View File
@@ -1,7 +1,7 @@
THIRD-PARTY SOFTWARE NOTICES
============================================================================
punktfunk (https://git.unom.io/unom/punktfunk) is licensed under MIT OR Apache-2.0.
Punktfunk (https://git.unom.io/unom/punktfunk) is licensed under MIT OR Apache-2.0.
The binaries it ships statically/dynamically link the third-party Rust crates listed
below. Each is distributed under its own permissive license; the full license texts
follow the manifest. This file is generated by scripts/gen-third-party-notices.py
@@ -62,7 +62,7 @@ MANIFEST (crate version — SPDX license — source)
cairo-sys-rs 0.22.0 — MIT — https://github.com/gtk-rs/gtk-rs-core
cast 0.3.0 — MIT OR Apache-2.0 — https://github.com/japaric/cast.rs
cbindgen 0.29.4 — MPL-2.0 — https://github.com/mozilla/cbindgen
cc 1.2.65 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cc 1.4.1 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cesu8 1.1.0 — Apache-2.0/MIT — https://github.com/emk/cesu8-rs
cexpr 0.6.0 — Apache-2.0/MIT — https://github.com/jethrogb/rust-cexpr
cfg-expr 0.20.8 — MIT OR Apache-2.0 — https://github.com/EmbarkStudios/cfg-expr
@@ -117,7 +117,7 @@ MANIFEST (crate version — SPDX license — source)
fiat-crypto 0.2.9 — MIT OR Apache-2.0 OR BSD-1-Clause — https://github.com/mit-plv/fiat-crypto
field-offset 0.3.6 — MIT OR Apache-2.0 — https://github.com/Diggsey/rust-field-offset
filetime 0.2.29 — MIT/Apache-2.0 — https://github.com/alexcrichton/filetime
find-msvc-tools 0.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
find-msvc-tools 0.1.10 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
fixedbitset 0.5.7 — MIT OR Apache-2.0 — https://github.com/petgraph/fixedbitset
flate2 1.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/flate2-rs
flume 0.12.0 — Apache-2.0/MIT — https://github.com/zesterer/flume
@@ -1625,7 +1625,7 @@ DEALINGS IN THE SOFTWARE.
----------------------------------------------------------------------------
The following license (LICENSE-APACHE) applies to: assert_matches 1.5.0, async-channel 2.5.0, autocfg 1.5.1, base64 0.22.1, bitflags 1.3.2, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.2.65, cexpr 0.6.0, cfg-if 1.0.4, cmake 0.1.58, concurrent-queue 2.5.0, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, displaydoc 0.2.6, either 1.16.0, equivalent 1.0.2, errno 0.3.14, event-listener 5.4.1, event-listener-strategy 0.5.4, fastrand 2.4.1, filetime 0.2.29, find-msvc-tools 0.1.9, fixedbitset 0.5.7, flate2 1.1.9, fnv 1.0.7, form_urlencoded 1.2.2, glob 0.3.3, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, idna 1.1.0, idna_adapter 1.2.2, indexmap 2.14.0, itertools 0.10.5, itertools 0.13.0, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, lazy_static 1.5.0, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, nasm-rs 0.3.2, num-integer 0.1.46, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking 2.2.1, parking_lot 0.12.5, parking_lot_core 0.9.12, percent-encoding 2.3.2, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, stable_deref_trait 1.2.1, system-deps 7.0.8, tar 0.4.46, tempfile 3.27.0, thread_local 1.1.9, tinytemplate 1.2.1, unicode-segmentation 1.13.3, unicode-width 0.2.2, url 2.5.8, vcpkg 0.2.15, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1, xattr 1.6.1
The following license (LICENSE-APACHE) applies to: assert_matches 1.5.0, async-channel 2.5.0, autocfg 1.5.1, base64 0.22.1, bitflags 1.3.2, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.4.1, cexpr 0.6.0, cfg-if 1.0.4, cmake 0.1.58, concurrent-queue 2.5.0, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, displaydoc 0.2.6, either 1.16.0, equivalent 1.0.2, errno 0.3.14, event-listener 5.4.1, event-listener-strategy 0.5.4, fastrand 2.4.1, filetime 0.2.29, find-msvc-tools 0.1.10, fixedbitset 0.5.7, flate2 1.1.9, fnv 1.0.7, form_urlencoded 1.2.2, glob 0.3.3, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, idna 1.1.0, idna_adapter 1.2.2, indexmap 2.14.0, itertools 0.10.5, itertools 0.13.0, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, lazy_static 1.5.0, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, nasm-rs 0.3.2, num-integer 0.1.46, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking 2.2.1, parking_lot 0.12.5, parking_lot_core 0.9.12, percent-encoding 2.3.2, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, stable_deref_trait 1.2.1, system-deps 7.0.8, tar 0.4.46, tempfile 3.27.0, thread_local 1.1.9, tinytemplate 1.2.1, unicode-segmentation 1.13.3, unicode-width 0.2.2, url 2.5.8, vcpkg 0.2.15, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1, xattr 1.6.1
----------------------------------------------------------------------------
Apache License
Version 2.0, January 2004
@@ -2921,7 +2921,7 @@ Exhibit B - "Incompatible With Secondary Licenses" Notice
----------------------------------------------------------------------------
The following license (LICENSE-MIT) applies to: cc 1.2.65, cfg-if 1.0.4, cmake 0.1.58, filetime 0.2.29, find-msvc-tools 0.1.9, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
The following license (LICENSE-MIT) applies to: cc 1.4.1, cfg-if 1.0.4, cmake 0.1.58, filetime 0.2.29, find-msvc-tools 0.1.10, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
----------------------------------------------------------------------------
Copyright (c) 2014 Alex Crichton
+2 -1
View File
@@ -1575,7 +1575,8 @@ pub fn show_scoped(
&dialog,
inline,
"Gamepad type",
"The virtual pad on the host — Automatic matches your controller",
"The virtual pad on the host — Automatic matches your controller. An X-Box type has no \
gyroscope, so pick a DualSense-class one if you want motion.",
&[
"Automatic",
"Xbox 360",
+5 -3
View File
@@ -1156,7 +1156,7 @@ async fn session(args: Args) -> Result<()> {
if args.rich_input_test {
let conn2 = conn.clone();
tokio::spawn(async move {
use punktfunk_core::input::gamepad::AXIS_LS_X;
use punktfunk_core::input::gamepad::{AXIS_LS_X, MOTION_ACCEL_LSB_PER_G};
use punktfunk_core::quic::RichInput;
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
// A neutral gamepad axis event makes the host create the virtual DualSense pad 0.
@@ -1184,12 +1184,14 @@ async fn session(args: Args) -> Result<()> {
for i in 0..60u32 {
let x = ((i * 65535) / 60) as u16;
let _ = conn2.send_datagram(touch(true, x, 32768).encode().into());
let g = (((i as i32 % 20) - 10) * 500) as i16; // gyro wobble
let g = (((i as i32 % 20) - 10) * 500) as i16; // gyro wobble, ±250 °/s
let _ = conn2.send_datagram(
RichInput::Motion {
pad: 0,
gyro: [g, 0, 0],
accel: [0, 0, 16384],
// At rest, gravity is 1 g on +Z — in WIRE units, which are 10000 LSB/g
// and not the 8192 or 16384 a particular driver happens to use.
accel: [0, 0, MOTION_ACCEL_LSB_PER_G as i16],
}
.encode()
.into(),
+2 -2
View File
@@ -69,8 +69,8 @@ Video, then the platform's own (pf-dxvadec on Windows, pf-vaadec on Linux), then
rung (openh264/rav1d). The libavcodec rungs that used to sit under each of them are
deleted, along with `pf-ffvk` and the `ffmpeg-next` dependency.
Two of the native rungs have never decoded a frame on real hardware (native VAAPI at all;
native D3D11VA's AV1 leg). They run anyway — with the libavcodec twins gone, the only
One of the native rungs has never decoded a frame on real hardware (native VAAPI's H.264 and
H.265 legs; its AV1 leg has decoded but has never been parity-checked). It runs anyway — with the libavcodec twins gone, the only
thing below them is the CPU, so barring them would cost the session hardware decode
outright rather than move it one rung down. What replaces the safety net is the log: every
session names the rung it landed on with its evidence state,
+63 -8
View File
@@ -266,6 +266,11 @@ pub fn run(target: Option<&str>) -> u8 {
ActionOutcome::Start(Box::new(params))
}
OverlayAction::CancelConnect => ActionOutcome::Handled, // run-loop-side
// Also run-loop-side: the clipboard belongs to SDL, which this callback
// has no handle on. Unreachable in practice — listed so adding an action
// to the enum keeps failing loudly here instead of falling into a
// wildcard that silently drops it.
OverlayAction::CopyText(_) => ActionOutcome::Handled,
OverlayAction::Quit => ActionOutcome::Quit,
}
});
@@ -286,6 +291,21 @@ pub fn run(target: Option<&str>) -> u8 {
}
}
/// A console row key → its index in the known-hosts store. The key is the pinned
/// fingerprint when there is one, else `addr:port` (see the row builder), and a pinned
/// CARD's key carries the profile id past a NUL — the console strips that before it
/// sends a command, so nothing here has to.
fn index_for_key(known: &trust::KnownHosts, key: &str) -> Option<usize> {
known
.hosts
.iter()
.position(|h| !h.fp_hex.is_empty() && h.fp_hex == key)
.or_else(|| {
let (addr, port) = key.rsplit_once(':')?;
known.index_by_addr(addr, port.parse().ok()?)
})
}
fn host_display_name(name: &str, addr: &str) -> String {
if name.trim().is_empty() {
addr.to_string()
@@ -483,6 +503,48 @@ impl ServiceState {
}
self.last_probe = Instant::now() - Duration::from_secs(60); // probe it now
}
ConsoleCmd::UpdateHost {
key,
name,
addr,
port,
} => {
let mut known = trust::KnownHosts::load();
let Some(h) = index_for_key(&known, &key).and_then(|i| known.hosts.get_mut(i))
else {
tracing::warn!(%key, "edit for an unknown host — ignoring");
return;
};
// Edited IN PLACE rather than removed and re-added: the fingerprint, the
// learned MAC, the pinned cards and the profile binding all hang off this
// entry, and re-adding would silently unpair a host the user only renamed.
h.name = if name.trim().is_empty() {
addr.clone()
} else {
name
};
h.addr = addr;
h.port = port;
if let Err(e) = known.save() {
tracing::warn!(error = %format!("{e:#}"), "saving known hosts");
}
self.last_probe = Instant::now() - Duration::from_secs(60); // the address moved
}
ConsoleCmd::ForgetHost { key } => {
let mut known = trust::KnownHosts::load();
let Some(i) = index_for_key(&known, &key) else {
tracing::warn!(%key, "forget for an unknown host — ignoring");
return;
};
let gone = known.hosts.remove(i);
if let Err(e) = known.save() {
tracing::warn!(error = %format!("{e:#}"), "saving known hosts");
}
tracing::info!(name = %gone.name, addr = %gone.addr, "host forgotten");
// It may still be advertising, in which case it comes straight back as a
// DISCOVERED row — unsaved and unpaired, which is the honest state.
self.last_probe = Instant::now() - Duration::from_secs(60);
}
ConsoleCmd::Wake { key, then_connect } => {
if let Some(c) = self.wake_cancel.take() {
c.store(true, Ordering::SeqCst);
@@ -534,14 +596,7 @@ impl ServiceState {
// end; never touches `profile_id` (the default binding). Idempotent, so
// a repeated press inside one refresh window can't double-pin.
let mut known = trust::KnownHosts::load();
let idx = known
.hosts
.iter()
.position(|h| !h.fp_hex.is_empty() && h.fp_hex == key)
.or_else(|| {
let (addr, port) = key.rsplit_once(':')?;
known.index_by_addr(addr, port.parse().ok()?)
});
let idx = index_for_key(&known, &key);
let Some(h) = idx.and_then(|i| known.hosts.get_mut(i)) else {
tracing::warn!(%key, "pin toggle for an unknown host — ignoring");
return;
+181
View File
@@ -510,6 +510,65 @@ mod session_main {
);
}
/// The driver's own answers about video images, printed with nothing in front of
/// them (`--probe-decode`).
///
/// Passing the five conjuncts above only says Vulkan Video EXISTS on a device; this
/// says whether the zero-copy pipeline can be BUILT on it — a different question
/// with, on at least one shipping driver, a different answer. Verbatim on purpose:
/// the Intel Arc refusal was twice diagnosed from punktfunk's own error text and
/// twice the diagnosis was wrong, and what broke it open both times was reading what
/// the driver actually said.
fn print_video_formats(a: &pf_presenter::vk::AdapterDecode) {
use pf_presenter::vk::probe::{describe_create_flags, describe_usage};
for p in &a.formats {
println!(" {} (wants {:?}):", p.profile, p.wanted);
for u in &p.usages {
let answer = match &u.formats {
Err(e) => format!("query failed: {e:?}"),
Ok(entries) if entries.is_empty() => "no formats offered".to_string(),
Ok(entries) => entries
.iter()
.map(|f| {
format!(
"{:?} usage={} create={} {:?} {:?}",
f.format,
describe_usage(f.image_usage),
describe_create_flags(f.image_create_flags),
f.image_type,
f.image_tiling,
)
})
.collect::<Vec<_>>()
.join("; "),
};
println!(" {:<24} {answer}", u.label);
// The second opinion, printed only where it differs from the video
// format query. Worded as "also asked" rather than "disagrees" on
// purpose: measured on both vendors this call answers "creatable" for
// combinations the video query rejects (NVIDIA included, for SAMPLED
// alone), so it does not honour the profile list and a difference here
// is NOT the driver contradicting itself. Printed anyway because the
// question gets re-asked by everyone who reads a refusal.
let listed = u
.wanted_entry(p.wanted)
.is_some_and(|f| f.image_usage.contains(u.usage));
if listed != u.image_format_support.is_ok() {
let second = match &u.image_format_support {
Ok(()) => "creatable".to_string(),
Err(e) => format!("{e:?}"),
};
println!(
" {:<24} (also asked: \
vkGetPhysicalDeviceImageFormatProperties2 says {second} that \
call does not honour the profile list; not authority)",
""
);
}
}
}
}
pub fn run() -> u8 {
// Logs to STDERR — stdout is the machine interface (ready/stats/error lines).
tracing_subscriber::fmt()
@@ -537,6 +596,128 @@ mod session_main {
};
}
// `--probe-decode`: per-adapter Vulkan Video decode capability, then exit. Human
// output on purpose — this is a triage tool, not a picker source, which is also
// why it is a separate flag: `--list-adapters` is parsed line-by-line by the
// desktop shells' GPU picker and must keep printing bare names.
if arg_flag("--probe-decode") {
return match pf_presenter::vk::probe_decode() {
Ok(adapters) => {
if adapters.is_empty() {
println!("no Vulkan physical devices");
}
for (i, a) in adapters.iter().enumerate() {
// The bracketed number is the PUNKTFUNK_VK_DEVICE value, and the
// FIRST listed entry is what
// a default run presents on — the decoder shares that device, so
// on a hybrid box this line is usually the answer.
let kind = if a.discrete { "discrete" } else { "integrated" };
// `a.index`, NOT the loop position. This list is sorted
// discrete-first for reading, but PUNKTFUNK_VK_DEVICE indexes the
// raw enumeration, which puts the iGPU first on some hybrids —
// printing the loop position would name the other GPU on exactly
// the machines this flag is for. The `i == 0` marker is still the
// loop position, because sorted-first IS what pick_device lands on
// when nothing overrides it.
println!(
"[{}] {} ({kind}){}",
a.index,
a.name,
if i == 0 { " <- default presenter" } else { "" }
);
println!(
" vulkan video decode: {}",
if a.usable { "YES" } else { "no" }
);
// Name every bit, and ACCOUNT for the ones we cannot name. The
// 5070 Ti reports 0xF — four bits — while punktfunk decodes three
// codecs, so the first version of this line printed three names
// beside a four-bit mask and looked complete. VP9 (bit 3) is a
// real decode operation this client has no rung for; a codec the
// tool cannot name must not silently vanish from a mask it prints,
// or the reader is left to trust that the words cover the number.
const OPS: [(u32, &str); 4] = [
(0x1, "H.264"),
(0x2, "H.265"),
(0x4, "AV1"),
(0x8, "VP9 (no punktfunk rung)"),
];
let mut codecs: Vec<String> = OPS
.iter()
.filter(|(bit, _)| a.codec_ops & bit != 0)
.map(|(_, n)| (*n).to_string())
.collect();
let named: u32 = OPS.iter().map(|(b, _)| b).sum();
let unknown = a.codec_ops & !named;
if unknown != 0 {
codecs.push(format!("unrecognised bits 0x{unknown:X}"));
}
println!(
" driver decode ops: {}",
if codecs.is_empty() {
format!("none (0x{:X})", a.codec_ops)
} else {
format!("{} (0x{:X})", codecs.join(", "), a.codec_ops)
}
);
if !a.usable {
// Say which conjunct failed. "no" with no reason is the thing
// this whole flag exists to stop.
let mut why: Vec<String> = Vec::new();
if !a.api_1_3 {
why.push("device is not Vulkan 1.3".into());
}
if !a.features_ok {
why.push(
"missing samplerYcbcrConversion / timelineSemaphore / \
synchronization2"
.into(),
);
}
if a.decode_family.is_none() {
why.push("no queue family advertises VIDEO_DECODE".into());
}
if !a.base_missing.is_empty() {
why.push(format!("missing {}", a.base_missing.join(", ")));
}
if a.codec_exts.is_empty() {
why.push("no VK_KHR_video_decode_{h264,h265,av1} extension".into());
}
println!(" why not: {}", why.join("; "));
} else {
println!(" extensions: {}", a.codec_exts.join(", "));
}
print_video_formats(a);
}
if adapters.len() > 1 {
// The single most common misreading of this output: seeing a
// capable GPU listed and concluding the decoder will use it.
// Vulkan Video decodes on the PRESENTER's device, and the decoder
// preference does not move the presenter.
println!();
println!(
"Vulkan Video decodes on the presenter's device. PUNKTFUNK_DECODER \
picks the rung,"
);
println!(
"not the GPU — move the presenter with PUNKTFUNK_VK_DEVICE=<index \
above> or"
);
println!(
"PUNKTFUNK_VK_ADAPTER=<name substring>, which is the safer knob \
where two"
);
println!("adapters share a name.");
}
0
}
Err(e) => {
eprintln!("probe-decode: {e:#}");
EXIT_PRESENTER_FAILED
}
};
}
// `--list-audio`: the PipeWire endpoints the settings pickers offer, as
// `sink|source<TAB>node.name<TAB>description` lines — a debug window into the
// same enumeration the GTK shell probes.
+5 -5
View File
@@ -1,7 +1,7 @@
THIRD-PARTY SOFTWARE NOTICES
============================================================================
punktfunk (https://git.unom.io/unom/punktfunk) is licensed under MIT OR Apache-2.0.
Punktfunk (https://git.unom.io/unom/punktfunk) is licensed under MIT OR Apache-2.0.
The binaries it ships statically/dynamically link the third-party Rust crates listed
below. Each is distributed under its own permissive license; the full license texts
follow the manifest. This file is generated by scripts/gen-third-party-notices.py
@@ -60,7 +60,7 @@ MANIFEST (crate version — SPDX license — source)
bytes 1.12.0 — MIT — https://github.com/tokio-rs/bytes
cast 0.3.0 — MIT OR Apache-2.0 — https://github.com/japaric/cast.rs
cbindgen 0.29.4 — MPL-2.0 — https://github.com/mozilla/cbindgen
cc 1.2.65 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cc 1.4.1 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
cesu8 1.1.0 — Apache-2.0/MIT — https://github.com/emk/cesu8-rs
cexpr 0.6.0 — Apache-2.0/MIT — https://github.com/jethrogb/rust-cexpr
cfg-expr 0.20.8 — MIT OR Apache-2.0 — https://github.com/EmbarkStudios/cfg-expr
@@ -114,7 +114,7 @@ MANIFEST (crate version — SPDX license — source)
fastrand 2.4.1 — Apache-2.0 OR MIT — https://github.com/smol-rs/fastrand
fiat-crypto 0.2.9 — MIT OR Apache-2.0 OR BSD-1-Clause — https://github.com/mit-plv/fiat-crypto
filetime 0.2.29 — MIT/Apache-2.0 — https://github.com/alexcrichton/filetime
find-msvc-tools 0.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
find-msvc-tools 0.1.10 — MIT OR Apache-2.0 — https://github.com/rust-lang/cc-rs
fixedbitset 0.5.7 — MIT OR Apache-2.0 — https://github.com/petgraph/fixedbitset
flate2 1.1.9 — MIT OR Apache-2.0 — https://github.com/rust-lang/flate2-rs
flume 0.12.0 — Apache-2.0/MIT — https://github.com/zesterer/flume
@@ -1598,7 +1598,7 @@ DEALINGS IN THE SOFTWARE.
----------------------------------------------------------------------------
The following license (LICENSE-APACHE) applies to: assert_matches 1.5.0, async-channel 2.5.0, autocfg 1.5.1, base64 0.22.1, bitflags 1.3.2, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.2.65, cexpr 0.6.0, cfg-if 1.0.4, cmake 0.1.58, concurrent-queue 2.5.0, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, displaydoc 0.2.6, either 1.16.0, equivalent 1.0.2, errno 0.3.14, event-listener 5.4.1, event-listener-strategy 0.5.4, fastrand 2.4.1, filetime 0.2.29, find-msvc-tools 0.1.9, fixedbitset 0.5.7, flate2 1.1.9, fnv 1.0.7, form_urlencoded 1.2.2, glob 0.3.3, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, idna 1.1.0, idna_adapter 1.2.2, indexmap 2.14.0, itertools 0.10.5, itertools 0.13.0, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, lazy_static 1.5.0, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, nasm-rs 0.3.2, num-integer 0.1.46, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking 2.2.1, parking_lot 0.12.5, parking_lot_core 0.9.12, percent-encoding 2.3.2, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, stable_deref_trait 1.2.1, system-deps 7.0.8, tar 0.4.46, tempfile 3.27.0, thread_local 1.1.9, tinytemplate 1.2.1, unicode-segmentation 1.13.3, unicode-width 0.2.2, url 2.5.8, vcpkg 0.2.15, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1, xattr 1.6.1
The following license (LICENSE-APACHE) applies to: assert_matches 1.5.0, async-channel 2.5.0, autocfg 1.5.1, base64 0.22.1, bitflags 1.3.2, bitflags 2.13.0, bumpalo 3.20.3, cast 0.3.0, cc 1.4.1, cexpr 0.6.0, cfg-if 1.0.4, cmake 0.1.58, concurrent-queue 2.5.0, core-foundation 0.10.1, core-foundation-sys 0.8.7, criterion 0.5.1, criterion-plot 0.5.0, crossbeam-deque 0.8.6, crossbeam-epoch 0.9.20, crossbeam-utils 0.8.21, curve25519-dalek-derive 0.1.1, displaydoc 0.2.6, either 1.16.0, equivalent 1.0.2, errno 0.3.14, event-listener 5.4.1, event-listener-strategy 0.5.4, fastrand 2.4.1, filetime 0.2.29, find-msvc-tools 0.1.10, fixedbitset 0.5.7, flate2 1.1.9, fnv 1.0.7, form_urlencoded 1.2.2, glob 0.3.3, hashbrown 0.17.1, heck 0.5.0, hermit-abi 0.5.2, idna 1.1.0, idna_adapter 1.2.2, indexmap 2.14.0, itertools 0.10.5, itertools 0.13.0, jni 0.21.1, jobserver 0.1.34, js-sys 0.3.103, lazy_static 1.5.0, linux-raw-sys 0.12.1, lock_api 0.4.14, log 0.4.33, nasm-rs 0.3.2, num-integer 0.1.46, num-traits 0.2.19, once_cell 1.21.4, openssl-probe 0.2.1, opus 0.3.1, parking 2.2.1, parking_lot 0.12.5, parking_lot_core 0.9.12, percent-encoding 2.3.2, pkg-config 0.3.33, proptest 1.11.0, rayon 1.12.0, rayon-core 1.13.0, regex 1.12.4, regex-automata 0.4.14, regex-syntax 0.8.11, rustc_version 0.4.1, rustix 1.1.4, rustls 0.23.41, rustls-native-certs 0.8.4, rusty-fork 0.3.1, scopeguard 1.2.0, security-framework 3.7.0, security-framework-sys 2.17.0, signal-hook-registry 1.4.8, smallvec 1.15.2, socket2 0.6.4, stable_deref_trait 1.2.1, system-deps 7.0.8, tar 0.4.46, tempfile 3.27.0, thread_local 1.1.9, tinytemplate 1.2.1, unicode-segmentation 1.13.3, unicode-width 0.2.2, url 2.5.8, vcpkg 0.2.15, version_check 0.9.5, wait-timeout 0.2.1, wasi 0.11.1+wasi-snapshot-preview1, wasip2 1.0.4+wasi-0.2.12, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126, wit-bindgen 0.57.1, xattr 1.6.1
----------------------------------------------------------------------------
Apache License
Version 2.0, January 2004
@@ -2853,7 +2853,7 @@ Exhibit B - "Incompatible With Secondary Licenses" Notice
----------------------------------------------------------------------------
The following license (LICENSE-MIT) applies to: cc 1.2.65, cfg-if 1.0.4, cmake 0.1.58, filetime 0.2.29, find-msvc-tools 0.1.9, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
The following license (LICENSE-MIT) applies to: cc 1.4.1, cfg-if 1.0.4, cmake 0.1.58, filetime 0.2.29, find-msvc-tools 0.1.10, jobserver 0.1.34, js-sys 0.3.103, openssl-probe 0.2.1, pkg-config 0.3.33, socket2 0.6.4, wait-timeout 0.2.1, wasm-bindgen 0.2.126, wasm-bindgen-macro 0.2.126, wasm-bindgen-macro-support 0.2.126, wasm-bindgen-shared 0.2.126
----------------------------------------------------------------------------
Copyright (c) 2014 Alex Crichton
@@ -59,14 +59,6 @@
"licenses": [{ "license": { "id": "Zlib" } }],
"externalReferences": [{ "type": "vcs", "url": "https://github.com/libsdl-org/SDL" }]
},
{
"type": "application",
"name": "VB-CABLE",
"version": "redistributed installer, see packaging/windows/install-vbcable.ps1",
"description": "Third-party kernel-mode virtual audio driver redistributed with the Windows host; notice at packaging/windows/licenses/VB-CABLE-NOTICE.txt. Planned to be replaced by an attestation-signed first-party driver.",
"licenses": [{ "license": { "name": "Proprietary freeware (VB-Audio Software, redistribution permitted per notice)" } }],
"externalReferences": [{ "type": "website", "url": "https://vb-audio.com/Cable/" }]
},
{
"type": "application",
"name": "punktfunk-gamescope",
+252 -13
View File
@@ -240,6 +240,16 @@ pub enum PlanWarning {
/// hosts never emit MMCO 5, so this warning is the field signal if that
/// assumption ever breaks.
Mmco5Rebase,
/// The SPS carried no VUI `bitstream_restriction`, so the DPB had to be sized from
/// A.3.1's LEVEL ceiling — and that ceiling demands more hardware slots than a
/// mainstream decoder provides. See [`dpb_limit`] for why this is the H.264 shape
/// of the defect #96 fixed for HEVC, and why it is a warning here rather than a
/// clamp. Every H.264 encoder a punktfunk host can reach writes the restriction
/// (measured 2026-08-07), so this warning is the field signal if that ever breaks.
LevelDerivedDpb {
max_dpb_frames: usize,
level_idc: u8,
},
}
/// The AU cannot be planned at all.
@@ -296,12 +306,72 @@ impl From<&Sps> for NegotiationInfo {
bit_depth_luma_minus8: sps.bit_depth_luma_minus8,
bit_depth_chroma_minus8: sps.bit_depth_chroma_minus8,
chroma_format_idc: sps.chroma_format_idc,
max_dpb_frames: sps.max_dpb_frames(),
max_dpb_frames: dpb_limit(sps),
interlaced: !sps.frame_mbs_only_flag,
}
}
}
/// The hardware DPB slot count a mainstream decoder provides. NVIDIA's Vulkan Video
/// reports `maxDpbSlots = 16` (RADV 17), and DXVA/D3D11VA's H.264 picture-parameter
/// format indexes the DPB with a 16-entry array. Backends allocate one slot per DPB
/// frame plus one for the picture in flight, so a stream whose DPB is sized at 16
/// demands 17 and is refused outright — losing the codec, not merely a slower path.
const MAINSTREAM_MAX_DPB_SLOTS: usize = 16;
/// DPB size in frames — the same question [`crate::h265::dpb_limit`] answers, and the
/// same trap, but H.264 gets a different answer and it is worth writing down why.
///
/// HEVC's SPS states its own requirement outright (`sps_max_dec_pic_buffering_minus1`),
/// so #96 could simply stop consulting equation A-2's level ceiling. H.264 has no such
/// unconditional field: the stream declares its need ONLY in the VUI's
/// `bitstream_restriction` (`max_dec_frame_buffering`, E.2.1). Absent that, A.3.1's
/// level ceiling — `min(MaxDpbMbs / (PicWidthInMbs * FrameHeightInMbs), 16)` — is all
/// the spec leaves, and it is genuinely the correct inference, not a bug. It is also
/// what the stream MAY use rather than what it needs, which is exactly the shape that
/// killed HEVC at 720p and 1080p.
///
/// The ceiling saturates at 16 — thus [`MAINSTREAM_MAX_DPB_SLOTS`] + 1 hardware slots,
/// the fatal value — whenever the picture is small relative to its level's `MaxDpbMbs`:
///
/// | picture | ceiling by level |
/// |---|---|
/// | 720p (3600 MBs) | L3.2 → 5, L4.2 → 9, **L5.0+ → 16** |
/// | 1080p (8160 MBs) | L4.2 → 4, L5.0 → 13, **L5.1+ → 16** |
/// | 1440p (14400 MBs) | L5.1 → 12, **L6.0+ → 16** |
/// | 2160p (32400 MBs) | L5.2 → 5, **L6.0+ → 16** |
///
/// So H.264 escaped #96 twice over rather than by one piece of luck, and BOTH escapes
/// are properties of the encoders, not of the format. Measured 2026-08-07 by reading
/// the SPS each encoder actually emitted, at 720p/1080p/1440p/2160p:
///
/// | encoder | level picked | `bitstream_restriction` | `max_dec_frame_buffering` |
/// |---|---|---|---|
/// | NVENC (RTX 5070 Ti, 610.57.04) | 3.2 / 4.2 / 5.1 / 5.2 | present | 3 |
/// | VAAPI via libavcodec (RDNA3, Mesa 26.0.3) | 4.1 / 4.2 / 5.1 / 5.2 | present | 1 |
/// | openh264 (software rung) | 3.2 / 4.2 / 5.1 / 5.2 | present | 1 |
///
/// Every one of them picks a level proportionate to the picture AND states its real
/// need in the VUI, so the ceiling is never reached and never consulted. That is why
/// this function does NOT clamp: with the restriction present the value below IS the
/// stream's own statement, and clamping a stream that genuinely asked for a deep DPB
/// would corrupt its output. Absent the restriction there is no honest smaller number
/// to substitute — [`PlanWarning::LevelDerivedDpb`] names the situation instead, so the
/// field tells us if a driver ever stops writing the VUI, rather than a user silently
/// losing H.264 the way #96's users silently lost HEVC.
fn dpb_limit(sps: &Sps) -> usize {
// A.3.1's cap, the VUI override and the `max_num_ref_frames` floor all live in the
// vendored parser; this is the one named place the RESULT is interpreted.
sps.max_dpb_frames()
}
/// Whether [`dpb_limit`] had to fall back to A.3.1's level ceiling because the SPS
/// carried no VUI `bitstream_restriction` — i.e. whether the number is the stream's own
/// statement of need or merely the largest DPB its level permits.
fn dpb_is_level_derived(sps: &Sps) -> bool {
!(sps.vui_parameters_present_flag && sps.vui_parameters.bitstream_restriction_flag)
}
#[derive(Copy, Clone, Debug)]
enum RefPicList {
RefPicList0,
@@ -606,7 +676,7 @@ impl H264Planner {
// uncapped. No hardware decoder implements a deeper DPB — a larger value is a
// corrupt (or hostile) VUI, not a feature request — and backends size real
// slot pools from this number, so it is gated here, at SPS activation.
if sps.max_dpb_frames() > 16 {
if dpb_limit(sps) > MAINSTREAM_MAX_DPB_SLOTS {
return Err(PlanError::OutsideEnvelope(
"DPB deeper than 16 frames (max_dec_frame_buffering)",
));
@@ -1279,10 +1349,19 @@ impl H264Planner {
}
// Apply the parameters of `sps` to the planning state.
fn apply_sps(&mut self, sps: &Sps) {
fn apply_sps(&mut self, sps: &Sps, warnings: &mut Vec<PlanWarning>) {
self.negotiation_info = NegotiationInfo::from(sps);
let max_dpb_frames = sps.max_dpb_frames();
let max_dpb_frames = dpb_limit(sps);
// Sized from the level ceiling rather than the stream's own declaration, and
// large enough that backends will ask for more slots than they can get. See
// [`dpb_limit`]: warned, not clamped, because there is no honest smaller number.
if dpb_is_level_derived(sps) && max_dpb_frames + 1 > MAINSTREAM_MAX_DPB_SLOTS {
warnings.push(PlanWarning::LevelDerivedDpb {
max_dpb_frames,
level_idc: sps.level_idc as u8,
});
}
let interlaced = !sps.frame_mbs_only_flag;
let max_num_order_frames = sps.max_num_order_frames() as usize;
let max_num_reorder_frames = if max_num_order_frames > max_dpb_frames {
@@ -1300,13 +1379,17 @@ impl H264Planner {
*old_negotiation_info != negotiation_info
}
fn renegotiate_if_needed(&mut self, sps: &Sps) -> Result<(), PlanError> {
fn renegotiate_if_needed(
&mut self,
sps: &Sps,
warnings: &mut Vec<PlanWarning>,
) -> Result<(), PlanError> {
if Self::negotiation_possible(sps, &self.negotiation_info) {
Self::check_envelope(sps)?;
// Make sure all the frames planned so far are display-ready before the
// stream parameters change under them.
self.drain_dpb();
self.apply_sps(sps);
self.apply_sps(sps, warnings);
}
Ok(())
@@ -1412,7 +1495,7 @@ impl H264Planner {
)?);
// A picture's SPS may require renegotiation.
self.renegotiate_if_needed(&pps.sps)?;
self.renegotiate_if_needed(&pps.sps, warnings)?;
let first_field = self.find_first_field(hdr).map_err(PlanError::Parse)?;
@@ -1706,7 +1789,7 @@ impl H264Planner {
bit_depth_luma_minus8: sps.bit_depth_luma_minus8,
bit_depth_chroma_minus8: sps.bit_depth_chroma_minus8,
chroma_format_idc: sps.chroma_format_idc,
max_dpb_frames: sps.max_dpb_frames(),
max_dpb_frames: dpb_limit(sps),
recovery_point,
}
}
@@ -1900,6 +1983,21 @@ mod tests {
(sps, pps)
}
/// The warnings a plan carries, minus the DPB-sizing signal.
///
/// [`base_sps`]'s fixtures are 64x64 with no VUI bitstream restriction, so A.3.1's
/// ceiling saturates (`MaxDpbMbs(L1) = 396` over 16 macroblocks) and every plan
/// built on them carries [`PlanWarning::LevelDerivedDpb`]. That is the arithmetic
/// `the_level_ceiling_alone_would_reproduce_96_and_is_warned_about` exists to pin —
/// SMALL pictures saturate the ceiling most easily, not large ones — and it says
/// nothing about the picture, which is what the tests below are checking.
fn picture_warnings(plan: &AuPlan) -> Vec<&PlanWarning> {
plan.warnings
.iter()
.filter(|w| !matches!(w, PlanWarning::LevelDerivedDpb { .. }))
.collect()
}
fn param_set_au(sps: &Sps, pps: &Pps) -> Vec<u8> {
let mut au = Vec::new();
Synthesizer::<'_, Sps, _>::synthesize(3, sps, &mut au, true).unwrap();
@@ -2022,7 +2120,7 @@ mod tests {
let p4 = planner.plan_au(&au4).unwrap();
for plan in [&p0, &p1, &p2, &p3, &p4] {
assert!(
plan.warnings.is_empty(),
picture_warnings(plan).is_empty(),
"authored stream must plan clean: {plan:?}"
);
}
@@ -2101,7 +2199,10 @@ mod tests {
let p1 = planner.plan_au(&au1).unwrap();
let p2 = planner.plan_au(&au2).unwrap();
for plan in [&p0, &p1, &p2] {
assert!(plan.warnings.is_empty(), "must plan clean: {plan:?}");
assert!(
picture_warnings(plan).is_empty(),
"must plan clean: {plan:?}"
);
}
let idr_id = p0.dpb.stored.unwrap();
let lt_id = p1.dpb.stored.unwrap();
@@ -2364,6 +2465,144 @@ mod tests {
);
}
/// An SPS at a given picture size and level, with the VUI bitstream restriction
/// either absent (so [`dpb_limit`] must fall back to A.3.1's level ceiling) or
/// present with an explicit `max_dec_frame_buffering`.
fn sps_at(width: u32, height: u32, level: Level, declared: Option<u32>) -> Sps {
Sps {
profile_idc: Profile::Main as u8,
level_idc: level,
frame_mbs_only_flag: true,
direct_8x8_inference_flag: true,
// Every H.264 encoder measured below sits at or under this; it is the
// A.3.1 floor `max_dpb_frames` applies, never the value under test.
max_num_ref_frames: 3,
pic_width_in_mbs_minus1: (width / 16 - 1) as u16,
pic_height_in_map_units_minus1: (height / 16 - 1) as u16,
vui_parameters_present_flag: declared.is_some(),
vui_parameters: VuiParams {
bitstream_restriction_flag: declared.is_some(),
max_dec_frame_buffering: declared.unwrap_or(0),
..Default::default()
},
..Default::default()
}
}
/// The H.264 half of `pf-encode`'s `rfi_dpb_fits_a_mainstream_vulkan_decoder`.
///
/// That test guards the PRODUCER end for HEVC — that `RFI_DPB` never grows past
/// what a client can allocate. This is the CONSUMER end for H.264, where the
/// number is not ours to choose: the client derives it from whatever SPS arrives,
/// and #96 proved that deriving a DPB from a level's ceiling instead of a stream's
/// need costs the codec outright.
///
/// Pinned here are the (picture, level) pairs a punktfunk host can actually reach,
/// with the level each shipping encoder was MEASURED to pick on 2026-08-07 (see
/// [`dpb_limit`] for the full table and the hardware). The property that has to
/// hold is the one the backends enforce: `dpb_limit + 1 <= 16` slots.
#[test]
fn every_reachable_h264_stream_fits_a_mainstream_slot_pool() {
// (picture, level measured on NVENC / VAAPI / openh264, their VUI declaration)
let measured = [
((1280, 720), Level::L3_2, 3), // NVENC, openh264
((1280, 720), Level::L4_1, 1), // VAAPI via libavcodec
((1920, 1080), Level::L4_2, 3), // all three
((2560, 1440), Level::L5_1, 3),
((3840, 2160), Level::L5_2, 3),
];
for ((w, h), level, declared) in measured {
let sps = sps_at(w, h, level, Some(declared));
assert!(
!dpb_is_level_derived(&sps),
"{w}x{h} L{:?}: the measured encoders all write the VUI restriction — \
an SPS that carries it must never be treated as level-derived",
level
);
let slots = dpb_limit(&sps) + 1;
assert!(
slots <= MAINSTREAM_MAX_DPB_SLOTS,
"{w}x{h} L{level:?} declaring {declared} needs {slots} DPB slots, \
mainstream hardware caps at {MAINSTREAM_MAX_DPB_SLOTS}"
);
}
}
/// The cliff the measured encoders walk past, pinned so it stays visible.
///
/// These are the SAME resolutions, at levels a host could legally pick, with no
/// VUI restriction to state the real need. Each computes a 16-frame DPB — 17
/// hardware slots — which is precisely the arithmetic that killed HEVC at 720p and
/// 1080p before #96. H.264 escapes it because every encoder both picks a
/// proportionate level AND writes the restriction, not because the format is safe.
#[test]
fn the_level_ceiling_alone_would_reproduce_96_and_is_warned_about() {
// (picture, a level that saturates A.3.1's ceiling for it)
let cliff = [
((1280, 720), Level::L5),
((1280, 720), Level::L5_2),
((1920, 1080), Level::L5_1),
((2560, 1440), Level::L6),
((3840, 2160), Level::L6_2),
];
for ((w, h), level) in cliff {
let sps = sps_at(w, h, level, None);
assert!(dpb_is_level_derived(&sps), "{w}x{h} L{level:?}");
assert_eq!(
dpb_limit(&sps),
16,
"{w}x{h} L{level:?} should saturate A.3.1's 16-frame ceiling"
);
assert!(
dpb_limit(&sps) + 1 > MAINSTREAM_MAX_DPB_SLOTS,
"{w}x{h} L{level:?} is the #96 arithmetic and must be recognised as such"
);
// ...and the planner must NAME it rather than let a user silently lose the
// codec. The SPS activates on the first slice, so the warning rides out on
// the IDR's plan.
let sps = Rc::new(sps);
let pps = PpsBuilder::new(Rc::clone(&sps))
.pic_parameter_set_id(0)
.pic_init_qp(26)
.build();
let mut au = param_set_au(&sps, &pps);
au.extend(write_idr_slice());
let plan = H264Planner::new()
.plan_au(&au)
.unwrap_or_else(|e| panic!("{w}x{h} L{level:?} should plan, got {e:?}"));
assert!(
plan.warnings.contains(&PlanWarning::LevelDerivedDpb {
max_dpb_frames: 16,
level_idc: level as u8,
}),
"{w}x{h} L{level:?}: expected LevelDerivedDpb, got {:?}",
plan.warnings
);
}
}
/// A proportionate level is the other half of the escape: at the levels the
/// encoders actually pick, the ceiling is small enough that even a stream with no
/// VUI at all fits — so neither escape is doing all the work alone.
#[test]
fn a_proportionate_level_fits_even_without_a_vui_restriction() {
let proportionate = [
((1280, 720), Level::L3_2, 5),
((1280, 720), Level::L4_1, 9),
((1920, 1080), Level::L4_2, 4),
((2560, 1440), Level::L5_1, 12),
((3840, 2160), Level::L5_2, 5),
];
for ((w, h), level, expected) in proportionate {
let sps = sps_at(w, h, level, None);
assert_eq!(dpb_limit(&sps), expected, "{w}x{h} L{level:?}");
let slots = dpb_limit(&sps) + 1; // + the picture in flight
assert!(slots <= MAINSTREAM_MAX_DPB_SLOTS, "{w}x{h} L{level:?}");
}
}
/// MMCO 5 writer: op 5 takes NO argument (Table 7-9), so the generic
/// [`write_p_slice`] — whose supported ops all take exactly one — cannot author
/// it.
@@ -2688,11 +2927,11 @@ mod tests {
let plan = planner.plan_au(&write_idr_slice()).unwrap();
assert!(plan.picture.is_idr);
assert_eq!(plan.picture.pic_order_cnt, 0);
assert!(plan.warnings.is_empty());
assert!(picture_warnings(&plan).is_empty());
// And the stream continues cleanly on the reset state.
let plan = planner.plan_au(&write_p_slice(1, 2, 1, 1, None)).unwrap();
assert!(plan.warnings.is_empty());
assert!(picture_warnings(&plan).is_empty());
assert_eq!(plan.slices[0].ref_list0.len(), 1);
}
@@ -2725,7 +2964,7 @@ mod tests {
au.extend(write_idr_slice_at(8, 1));
let plan = H264Planner::new().plan_au(&au).unwrap();
assert!(plan.warnings.is_empty());
assert!(picture_warnings(&plan).is_empty());
assert_eq!(plan.slices.len(), 2);
// The picture parameters come from the FIRST slice's PPS (the uncropped
// SPS 0); they must not drift to the last slice's.
+176 -13
View File
@@ -158,8 +158,9 @@ pub struct PicturePlan {
pub bit_depth_luma_minus8: u8,
pub bit_depth_chroma_minus8: u8,
pub chroma_format_idc: u8,
/// DPB size in frames per A.4 (equation A-2, capped at 16) — backends size their
/// slot pool from this.
/// DPB size in frames: the stream's own `sps_max_dec_pic_buffering_minus1 + 1`,
/// capped at 16 — backends size their slot pool from this. See [`dpb_limit`] for
/// why this is NOT equation A-2's level ceiling.
pub max_dpb_frames: usize,
/// Bits of the `st_ref_pic_set()` the FIRST slice carried inline (0 when the RPS
/// came from the SPS by index) — Vulkan's `NumBitsForSTRefPicSetInSlice`.
@@ -311,16 +312,52 @@ impl From<&Sps> for NegotiationInfo {
}
}
/// The DPB size the planner enforces: equation A-2 (via the vendored
/// `Sps::max_dpb_size`), never above 16 — and never below the stream's
/// `sps_max_dec_pic_buffering_minus1 + 1`, which C.5.2.2's bumping uses as its
/// fullness bound. A conforming stream keeps the latter within A-2 (its constraint
/// clause), so the `max` only widens the pool for streams that already violate A.4 —
/// storing their pictures beats erroring the AU.
/// The DPB size the planner enforces: the stream's own
/// `sps_max_dec_pic_buffering_minus1[HighestTid] + 1`, capped at 16.
///
/// This deliberately does NOT consult equation A-2 (the vendored
/// `Sps::max_dpb_size`). A-2 is a CEILING on what an SPS may signal — 7.4.3.2.1
/// constrains `sps_max_dec_pic_buffering_minus1[i]` to `0..=MaxDpbSize - 1` — not a
/// statement of what the stream needs. The size the stream actually needs is the
/// signalled buffering, which is precisely what C.5.2.2's fullness clause bumps
/// against, and A.4.1 bounds the total RPS entries by the same number: `buffering`
/// pictures hold `buffering - 1` references plus the current one, exactly.
///
/// Reading A-2 as a requirement cost us HEVC outright, measured on `.21`
/// (RTX 5070 Ti, 2026-08-07). The host's SPS is minimal and honest at every
/// resolution — `general_level_idc = 153` (L5.1 High, which NVENC autoselects
/// because 130 Mbps needs it), `sps_max_dec_pic_buffering_minus1 = 5`, i.e. six
/// pictures: `RFI_DPB` references plus the current one. But A-2 branches on picture
/// size against the LEVEL's `MaxLumaPs`, and at 1080p the coded 1920x1088 =
/// 2 088 960 samples fall under `MaxLumaPs(L5.1) >> 2` = 2 228 224, taking the first
/// branch: `min(4 * MaxDpbPicBuf, 16)` = 16. The old `max(A-2, buffering)` therefore
/// reported 16 where the stream asked for 6, backends added the current picture and
/// demanded 17 hardware DPB slots, and NVIDIA's Vulkan Video caps `maxDpbSlots` at
/// 16 — so every access unit was refused, the ladder ran out of rungs, and the
/// session reconnected without HEVC (fatal on a build with no software HEVC
/// decoder).
///
/// A resolution sweep on the same box put the blast radius at the two commonest
/// streaming resolutions and nowhere else, which is A-2's branch table exactly:
/// 720p (1280x720 = 921 600) and 1080p died on branch 1 at 16 frames; 1440p
/// (2560x1440 = 3 686 400, under `MaxLumaPs >> 1`) survived on branch 2 at 12; 4K
/// (3840x2176 = 8 355 840, past `3 * MaxLumaPs >> 2`) survived on the `else` branch
/// at 6 and decoded at 1.9 ms. One host, one level, one six-picture requirement —
/// only which branch the picture size landed in decided whether HEVC worked at all.
///
/// The `max()` that produced the 16 bought nothing even for the malformed streams
/// it was written for: `Dpb::needs_bumping` (C.5.2.2) already keys on the signalled
/// buffering, not on `max_num_pics`, so a stream referencing more pictures than it
/// declared was ALREADY being bumped below its own declared depth before every
/// store. All the widened limit ever did was over-allocate hardware surfaces.
///
/// The cap stays: `check_envelope` rejects `buffering > 16` at activation, but this
/// also runs from `NegotiationInfo::from`, which sees SPSes that have not reached
/// the gate yet, and backends size real slot pools from the result.
fn dpb_limit(sps: &Sps) -> usize {
let buffering =
usize::from(sps.max_dec_pic_buffering_minus1[usize::from(sps.max_sub_layers_minus1)]) + 1;
sps.max_dpb_size().max(buffering).min(16)
buffering.min(16)
}
/// The RefPicSet data (8.3.2), derived once per picture.
@@ -1755,6 +1792,10 @@ mod tests {
width: u32,
height: u32,
bit_depth_minus8: u32,
/// `general_level_idc` (30 x the level number: 120 = L4, 153 = L5.1). Only the
/// DPB-sizing tests care, and they care because equation A-2 keys on it — the
/// planner deliberately does not.
level_idc: u32,
/// (left, right, top, bottom) conf_win offsets, in chroma units.
conf_win: Option<(u32, u32, u32, u32)>,
max_dec_pic_buffering_minus1: u32,
@@ -1771,6 +1812,7 @@ mod tests {
width: 64,
height: 64,
bit_depth_minus8: 0,
level_idc: 120, // L4
conf_win: None,
max_dec_pic_buffering_minus1: 4,
max_num_reorder_pics: 0,
@@ -1801,7 +1843,7 @@ mod tests {
s.bits(31, 0);
s.bits(12, 0); // 43 zero bits total
s.bit(0); // general_inbld_flag / reserved
s.bits(8, 120); // general_level_idc: level 4
s.bits(8, o.level_idc); // general_level_idc
s.ue(0); // sps_seq_parameter_set_id
s.ue(o.chroma_format_idc);
@@ -2093,7 +2135,11 @@ mod tests {
assert_eq!(plan.picture.general_profile_idc, 1);
assert_eq!(plan.picture.level_idc, Level::L4);
assert_eq!(plan.picture.chroma_format_idc, 1);
assert_eq!(plan.picture.max_dpb_frames, 16, "A-2 for a 64x64 L4 stream");
assert_eq!(
plan.picture.max_dpb_frames, 5,
"the DPB is the stream's sps_max_dec_pic_buffering_minus1 + 1, NOT A-2's \
level ceiling (which would say 16 for a 64x64 L4 stream)"
);
// Zero-reorder low-delay: the picture is display-ready in its own plan.
assert_eq!(plan.dpb.outputs, vec![plan.dpb.stored.unwrap()]);
// An IDR carries no RPS.
@@ -3072,8 +3118,10 @@ mod tests {
other => panic!("the rebind must not activate the rejected SPS: {other:?}"),
}
// Leg B: a 17-frame DPB, same geometry (dpb_limit caps both sides at 16, so
// NegotiationInfo alone cannot catch the rebind).
// Leg B: a 17-frame DPB, same geometry. The envelope gate is what must catch
// this on BOTH the direct AU and the PPS-only rebind: renegotiation runs only
// after `check_envelope` has passed, so a NegotiationInfo difference (5 vs the
// capped 16) is never reached and cannot stand in for the gate.
let mut planner = H265Planner::new();
planner
.plan_au(&opening_idr_au(&SpsOpts::default()))
@@ -3094,6 +3142,121 @@ mod tests {
));
}
/// The DPB the planner reports is the stream's own declared depth, and it must fit
/// a mainstream Vulkan Video decoder — pinned at the exact stream that broke it.
///
/// Field defect, `.21` (RTX 5070 Ti), 2026-08-07: a punktfunk 1080p HEVC session
/// refused EVERY access unit with "stream needs 17 DPB slots, device caps at 16",
/// exhausted the decode ladder and reconnected without HEVC — fatal, because there
/// is no permissively licensed software HEVC decoder to fall back to. The host was
/// blameless: its SPS asked for six pictures at both resolutions. The planner was
/// reporting equation A-2's LEVEL ceiling instead, and A-2 branches on picture size
/// against `MaxLumaPs`, so the identical stream reported 16 at 1080p and 6 at 4K.
///
/// The three assertions below are the whole defect: the two resolutions must agree,
/// they must agree on the number the STREAM signalled, and `+ 1` for the current
/// picture must clear the 16-slot floor that NVIDIA's `maxDpbSlots` sets. The host
/// side of the same arithmetic — that `RFI_DPB` never grows past what this leaves
/// room for — is pinned in `pf-encode`'s `rfi_dpb_fits_a_mainstream_vulkan_decoder`.
#[test]
fn the_reported_dpb_is_the_streams_own_depth_and_fits_a_16_slot_decoder() {
/// `VkVideoCapabilitiesKHR::maxDpbSlots` on NVIDIA — the lowest cap among the
/// decoders punktfunk targets (RADV reports 17). A stream needing more than
/// this cannot be decoded natively at all.
const VULKAN_MAX_DPB_SLOTS: usize = 16;
// The field stream, byte for byte on the fields that matter: L5.1 High (which
// NVENC autoselects at 130 Mbps), coded 1920x1088, six pictures declared.
let field = SpsOpts {
width: 1920,
height: 1088,
level_idc: 153,
max_dec_pic_buffering_minus1: 5,
..Default::default()
};
let mut planner = H265Planner::new();
let at_1080p = planner.plan_au(&opening_idr_au(&field)).unwrap();
assert_eq!(
at_1080p.picture.max_dpb_frames, 6,
"sps_max_dec_pic_buffering_minus1 = 5 means six pictures — five references \
plus the current one. A-2 would have said 16 here, because 1920x1088 = \
2088960 luma samples fall under MaxLumaPs(L5.1) >> 2 = 2228224."
);
// What the backends ask the driver for: one slot per DPB picture, plus one for
// the picture in flight (`pf_vkdecode::slots` pins the same convention).
let slots_for = |plan: &AuPlan| plan.picture.max_dpb_frames + 1;
assert!(
slots_for(&at_1080p) <= VULKAN_MAX_DPB_SLOTS,
"{} DPB frames need {} slots, over the {VULKAN_MAX_DPB_SLOTS} a mainstream \
Vulkan Video decoder offers this is the refusal that cost HEVC",
at_1080p.picture.max_dpb_frames,
slots_for(&at_1080p)
);
// The same stream at the other three resolutions punktfunk streams at. 720p and
// 1080p died in the field and 1440p and 4K survived, purely because A-2 branches
// on picture size: 1280x720 and 1920x1088 fall under MaxLumaPs >> 2 (16 frames),
// 2560x1440 under MaxLumaPs >> 1 (12), and 3840x2176 past 3 * MaxLumaPs >> 2 (6).
// All four must now be indistinguishable, because the stream is.
for (w, h) in [(1280, 720), (2560, 1440), (3840, 2176)] {
let mut planner = H265Planner::new();
let plan = planner
.plan_au(&opening_idr_au(&SpsOpts {
width: w,
height: h,
..field.clone()
}))
.unwrap();
assert_eq!(
plan.picture.max_dpb_frames, at_1080p.picture.max_dpb_frames,
"{w}x{h}: the DPB requirement is a property of the stream, not of which \
A-2 branch the picture size lands in"
);
}
// The invariant behind all of them: the reported depth is the declared one, and
// every depth that leaves a slot for the picture in flight fits a 16-slot device.
for minus1 in 0..=14u32 {
let mut planner = H265Planner::new();
let plan = planner
.plan_au(&opening_idr_au(&SpsOpts {
max_dec_pic_buffering_minus1: minus1,
..field.clone()
}))
.unwrap();
assert_eq!(plan.picture.max_dpb_frames, minus1 as usize + 1);
assert!(slots_for(&plan) <= VULKAN_MAX_DPB_SLOTS);
}
// The one honest residue: A.4 lets a conforming stream declare a full 16-picture
// DPB, and 16 pictures plus the one in flight is 17 slots, which NVIDIA does not
// have. Refusing that is right — it genuinely does not fit, and decoding it with
// fewer slots would silently corrupt references. No punktfunk host comes near it
// (`RFI_DPB` puts us at 6), and this is pinned so the distinction stays visible:
// what was fixed is streams that never needed the slots, not this one.
let mut planner = H265Planner::new();
let deepest = planner
.plan_au(&opening_idr_au(&SpsOpts {
max_dec_pic_buffering_minus1: 15,
..field.clone()
}))
.unwrap();
assert_eq!(deepest.picture.max_dpb_frames, 16);
assert_eq!(slots_for(&deepest), VULKAN_MAX_DPB_SLOTS + 1);
let mut planner = H265Planner::new();
assert!(
matches!(
planner.plan_au(&opening_idr_au(&SpsOpts {
max_dec_pic_buffering_minus1: 16,
..field.clone()
})),
Err(PlanError::OutsideEnvelope(what)) if what.contains("DPB")
),
"a 17-picture DPB is past A.4's cap and must be refused at activation, not \
clamped silently into a slot pool that cannot hold it"
);
}
/// Finding 4: the RASL refusal must run BEFORE renegotiation — a RASL AU carrying
/// a renegotiating SPS must not drain the DPB on its way out.
#[test]
+15 -6
View File
@@ -118,7 +118,11 @@ rand = "0.9"
# need the hidapi driver). Linux links the system SDL3; Windows builds it from source
# (no system SDL3 there — same choice as clients/windows).
[target.'cfg(target_os = "linux")'.dependencies]
pipewire = "0.9"
# `v0_3_49` for `Buffer::requested` — the graph's per-cycle frame ask, without which the
# playback callback can only size writes from the buffer CEILING (quantum-limit, ~170 ms).
# Pure cfg gate; needs libpipewire ≥ 0.3.49 (2022-03) at runtime, which every ship target
# (SteamOS, flatpak runtimes, Arch, Ubuntu ≥ 22.10) clears.
pipewire = { version = "0.9", features = ["v0_3_49"] }
sdl3 = { version = "0.18", features = ["hidapi"] }
# Native VAAPI decode (M6 of the native-decode program): the hand-declared libva buffer
# layouts, the profile/format/surface decisions, the AuPlan → picparams/IQ/slice
@@ -165,11 +169,16 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "acb5a1a74410
"winuser",
] }
[target.'cfg(windows)'.dev-dependencies]
# The native D3D11VA rung's frame-hash parity test compares decoded surfaces against the
# libavcodec goldens M5 captured — the same SHA-256 list, and the same crate, pf-vkdecode's
# Vulkan parity legs use (already in the workspace lock). The goldens are checked-in
# hashes; nothing links FFmpeg to read them.
[target.'cfg(any(target_os = "linux", windows))'.dev-dependencies]
# The two platform native rungs' frame-hash parity tests compare decoded surfaces against
# the libavcodec goldens M5 captured — the same SHA-256 list, and the same crate,
# pf-vkdecode's Vulkan parity legs use (already in the workspace lock). The goldens are
# checked-in hashes; nothing links FFmpeg to read them.
#
# Windows was the only platform here until the VAAPI rung grew a readback: `cfg(windows)`
# for `video_d3d11_native::parity`, now `cfg(linux)` as well for
# `video_vaapi_native::parity`. A DEV dependency, so no shipped binary gains anything —
# which is also part of why the VAAPI readback cannot reach the production video path.
sha2 = "0.10"
[features]

Some files were not shown because too many files have changed in this diff Show More