95 Commits

Author SHA1 Message Date
enricobuehler 882a3d57f6 chore(release): bump workspace version to 0.7.0
android-screenshots / screenshots (push) Successful in 2m12s
windows-host / package (push) Successful in 6m43s
android / android (push) Successful in 9m24s
release / apple (push) Successful in 9m35s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m8s
apple / swift (push) Successful in 1m10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 53s
linux-client-screenshots / screenshots (push) Successful in 1m41s
apple / screenshots (push) Successful in 5m37s
web-screenshots / screenshots (push) Successful in 2m37s
audit / cargo-audit (push) Successful in 1m12s
ci / web (push) Successful in 1m8s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 4m34s
deb / build-publish (push) Successful in 3m0s
ci / bench (push) Successful in 4m57s
decky / build-publish (push) Successful in 12s
flatpak / build-publish (push) Successful in 4m7s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m2s
docker / deploy-docs (push) Successful in 22s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
The [workspace.package] version (inherited by every crate via
version.workspace) is the release being cut. Refresh the 9 workspace entries
in Cargo.lock to match (CI builds --locked). Canary derives from the tag now
(scripts/ci/pf-version.sh), so no canary-base edit is needed — cutting v0.7.0
auto-advances canary to 0.8.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:45:45 +00:00
enricobuehler fa28fa19a0 Merge remote-tracking branch 'origin/main'
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
2026-07-03 22:41:42 +00:00
enricobuehler 42595b5558 fix(flatpak): keep both channels in the OSTree summary (fixes stable "No such ref")
The surfaced install command
  flatpak install --user https://flatpak.unom.io/io.unom.Punktfunk.flatpakref
failed with "No such ref 'app/io.unom.Punktfunk/x86_64/stable'". The stable
commit's objects are on the server, but the repo *summary* (what flatpak reads
to resolve refs) listed only canary.

Root cause: each CI run builds a fresh SINGLE-branch local OSTree repo,
build-update-repo regenerates the summary from that one branch, and rsync
uploads it without --delete. Objects for both channels accumulate, but the
summary is overwritten every run and only names that run's branch. Canary runs
on every main push, stable only on tags — so a tag published stable, then the
next canary push clobbered the summary back to canary-only.

Fix: seed the local repo from the live server (rsync repo/ DOWN) before the
build, so it carries every published branch; the build only adds this run's
commit and the regenerated+signed summary keeps both channels. Single shared
repo kept (no URL/Caddyfile change; existing installs fixed transparently).
Adds a refs log after build-update-repo as a clobber tripwire. Also adopts
scripts/ci/pf-version.sh for the canary base (see previous commit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:40:35 +00:00
enricobuehler 4de543c146 ci(release): derive canary version from git tags (single source of truth)
Every release workflow hardcoded a canary base version (0.5.0 in
Apple/Android/rpm/flatpak/deb, 0.3 in windows-msix/windows-host/decky) that
had to be hand-bumped on each stable release and wasn't. With stable at
v0.6.0, every canary was a version *behind* stable — e.g. the Apple canary
showed up on TestFlight as 0.5.0 while 0.6.0 was already published.

Add scripts/ci/pf-version.{sh,ps1} (bash + pwsh twin) as the single source of
truth: stable = the vX.Y.Z tag; canary = latest stable tag with minor+1,
patch 0 (v0.6.0 -> 0.7.0), so canary is always exactly one minor ahead of the
newest release with zero maintenance. Falls back to the workspace Cargo.toml
version when no tag is fetchable. All workflows now eval/call it and format
their own channel suffix off $PF_BASE; only the canary branch changed, stable
branches and per-channel suffixes are untouched. channels.md drops the old
manual "bump the canary base" release step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:40:25 +00:00
enricobuehler 42d1c74663 fix(apple-client/audio): capture the right channel of a multi-channel mic + diagnostics
apple / swift (push) Successful in 1m5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
The mic uplink handed the host pure digital silence on a multi-channel
interface: AVAudioConverter's N→stereo downmix takes channels 0/1, but a
pro interface puts the mic on ONE higher discrete channel. Fold the input
to a mono bus ourselves instead — pick the mic's channel (or sum all) and
resample that to the encoder's 48 kHz stereo, so the silent 0/1 downmix
never happens.

- New "Microphone channel" setting (macOS): Auto (sum every channel — a
  lone hot mic passes at full level) or pin 1-based channel N. Picker
  appears only for multi-channel devices, driven by the device's input
  channel count.
- Diagnostics that make this class of failure self-naming next session:
  log the actual live capture device + format + fold mode, warn on a
  silent UID fallback, and a one-shot silence tripwire on the EXTRACTED
  signal (WARN on 10 s of zeros, else peak dBFS).
- foldToMono extracted as a pure, unit-tested helper (pin / sum-clamp x
  interleaved / deinterleaved / mono / out-of-range).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 00:36:35 +02:00
enricobuehler 136f6e8f0e feat(probe): --mic-burst — real-client mic pacing for jitter-buffer regression tests
apple / swift (push) Successful in 1m8s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
The steady 5 ms mic-test cadence never trips host-side buffering bugs:
the WASAPI crackle (fixed in the previous commit) only reproduced under
a real client's bursty input tap. --mic-burst paces the tone the same
way (two 20 ms Opus packets every 40 ms), so recording the host mic and
counting silence gaps regression-tests the jitter buffer headlessly.
Validated against the fixed Windows host on the lab box: 15 s of bursty
tone, zero mid-stream gaps >=3 ms (gaps confined to the first 40 ms
priming window).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 22:35:49 +00:00
enricobuehler 00acf5e44e fix(host/audio): WASAPI virtual mic — port the priming jitter buffer (crackling fix)
apple / swift (push) Successful in 1m8s
ci / rust (push) Successful in 1m56s
apple / screenshots (push) Successful in 5m17s
ci / bench (push) Successful in 4m41s
decky / build-publish (push) Successful in 24s
ci / web (push) Successful in 59s
android / android (push) Successful in 3m41s
ci / docs-site (push) Successful in 1m0s
windows-host / package (push) Successful in 7m6s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m39s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 1m7s
deb / build-publish (push) Successful in 9m15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Mac → Windows mic passthrough crackled heavily while the identical
stream was clean on the Linux host. Cause: clients push mic audio in
BURSTS on their own clock (the Mac input tap yields ~two 20 ms Opus
packets every ~42 ms) while the WASAPI render loop pulled a block every
~10 ms device period and greedily drained whatever was queued, padding
the rest with zeros — the queue sat near-empty and most periods
inserted mid-stream silence. The Linux backend has absorbed this since
day one with its priming jitter buffer; the WASAPI loop had none.

Port the same semantics: emit silence until ~48 ms is buffered (covers
the worst inter-burst gap), then play from the cushion (zero-filling
only a momentary shortfall), re-prime only after a genuine full drain
(client went quiet). Queue cap raised 80 → 120 ms for burst headroom;
steady-state added latency ≈ the 48 ms cushion.

Diagnosed live on .173: probe tone recording from CABLE Output proved
the endpoint wiring, then the burst-vs-period math explained the
crackle. Build-verified on Windows; on-glass listen pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 22:31:18 +00:00
enricobuehler 38078fe7ee feat(linux-client): gamepad library launcher — console-style coverflow (--browse)
ci / rust (push) Successful in 1m54s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
windows-host / package (push) Successful in 6m43s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m12s
ci / bench (push) Successful in 4m47s
apple / swift (push) Successful in 1m9s
android / android (push) Successful in 3m33s
deb / build-publish (push) Successful in 4m36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
decky / build-publish (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
release / apple (push) Successful in 8m30s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 48s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m15s
flatpak / build-publish (push) Successful in 4m4s
apple / screenshots (push) Successful in 5m31s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m48s
docker / deploy-docs (push) Successful in 24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m20s
A controller-driven, chrome-less library launcher for the Steam Deck flow
(the Decky plugin's "Open library on screen" + pinned games, 8470419):
`--browse host[:port]` opens a paired host's game library as a coverflow
over a drifting aurora — A streams the focused title (the id rides the
Hello), session end returns to the launcher, B quits back to Gaming Mode.
`--connect` gains `--launch <id>` for direct-to-game starts; `--mgmt`
overrides the library port. Scope is deliberately library-only: host
selection/settings stay in the touch UI, pairing stays in the plugin (no
dialog can map under gamescope — every state renders in-page).

- gamepad.rs menu mode: the worker holds the active pad open while idle
  (WITHOUT the Valve HIDAPI drivers — Deck lizard mode survives) and
  translates it through a pure MenuNav state machine: edge-triggered
  buttons, held-state snapshot on entry/detach (the escape chord that ends
  a stream can't ghost-fire in the menu), 380/160 ms stick/dpad repeat,
  menu rumble ticks. Keyboard fallback (arrows/Enter/Esc) drives the same
  handler — fully usable with no pad, no host (PUNKTFUNK_FAKE_LIBRARY).
- Coverflow: ±38° corridor-facing tilt under per-card perspective
  (gsk rotate_3d), dense overlapping side shelves with paint-order
  restacking (gtk::Fixed draws in child order), opaque card faces + a
  darkening veil for the recede (translucency would bleed the stack
  through). The strip lives in an External-policy ScrolledWindow because
  a bare gtk::Fixed measures its TRANSFORMED children and inflates the
  page min-width past the window.
- Spring-driven motion: semi-implicit Euler in ≤8 ms substeps (a raw
  50 ms frame leaves the stiff recoil spring ringing at ω·dt ≈ 1.2 —
  regression-tested), ζ≈0.85 cursor chase + ζ≈0.55 boundary wobble;
  velocity carries across retargets so held-repeat scrolling glides.
- Shot scene `gamepad-library` (GTK animations force-disabled in shot mode
  — nav transitions froze mid-slide in headless captures); shared poster
  fetch extracted to library::spawn_art_fetch.

Verified here: 21 unit tests (MenuNav, cursor stepping, spring
convergence/stability), clippy -D warnings clean, screenshot scene
pixel-checked, --browse smoke runs (fake-library + unpaired) on the
headless session. On-Deck validation pending (virtual-pad input, lizard
mode, rumble via Steam Input, full Decky→browse→stream→launcher loop).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:41:43 +00:00
enricobuehler 69609945a3 feat(clients): host/network split in every stats HUD (stats phase 2, client side)
Consumes the 0xCF host-timing plane (449a67c) on all four GUI clients: each
keeps a bounded pending ring of receipt samples keyed by pts, matches the
host's per-AU capture→sent reports against it, and the HUD equation becomes

  = host 3.1 + network 6.7 + decode 2.1 + display 2.3

falling back to the combined `= host+network …` term whenever no timing
matched the window (old host / datagram loss) — same total, one split
fewer, never a misleading zero. Apple additionally gains the split as the
only equation line under the stage-1 fallback presenter (receipt is
presenter-independent), a `nextHostTiming` wrapper with its own plane lock,
and a unit-tested `HostNetworkSplitter`; Android extends the JNI stats
array 16→18 doubles (0–15 unchanged); Windows/Linux thread the split
through `Stats` into the HUD and the headless/debug logs.

Docs updated: design/stats-unification.md Phase 2 → implemented (wire
format, fallback semantics), and the docs-site matrix's Sunshine "Host
processing latency" row is now a direct match (ours includes the paced
send; avg vs p50).

Verified here: linux client clippy -D warnings green on the live tree,
windows stub check + hand-verified diff, android cargo-ndk arm64 check
green, apple loopback test extended (needs the rebuilt xcframework + swift
test on the mac). On-glass: pending on all platforms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:31:49 +00:00
enricobuehler 8470419433 feat(decky): pinned-games library + self-update robustness; fix gamepad tab-nav
Decky client batch:
- Pinned games / library picker: per-host game grid (GamePickerModal),
  pin/unpin, one-tap streams surfaced on the Hosts tab and QAM
  (usePins/streamPin/resolvePinHost, new src/library.tsx).
- Self-update + client-update plumbing (main.py check_update, hooks.ts
  applyUpdate) with a CA-bundle-resolving SSL context and per-channel
  manifest polling; steam.ts / punktfunkrun.sh launch tweaks.
- scripts/test-backend.py harness for the backend RPCs; README refresh.

Fix: the fullscreen page wrapped <Tabs> in an overflow-visible box, so
Valve's L1/R1 tab slide + autoFocusContents scrollIntoView panned
#GamepadUI itself — the whole Steam UI slid left until a tab was clicked.
Clip the Tabs wrapper (overflow:hidden), matching Valve's own Tabs
containers. (On-glass verification pending — Deck offline this session.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 21:25:07 +00:00
enricobuehler 449a67ce8d feat(protocol): per-AU host-timing plane (0xCF) — split host+network latency (stats phase 2)
The unified-stats equation's host+network stage was one opaque number
because the wire carried nothing but pts_ns. Now the host reports its own
share per frame: when the client's Hello sets VIDEO_CAP_HOST_TIMING (0x08),
the send thread emits a 13-byte 0xCF datagram — [tag][pts_ns u64][host_us
u32] — right after the AU's last packet leaves the socket, so host_us =
capture→fully-sent (capture read/convert, encode, FEC+seal, paced send)
against the same anchor the wire pts carries. Clients correlate by pts_ns
and derive network = (received + clock_offset − pts) − host_us; the two
terms tile per frame by construction.

Back-compat is free in all four combinations: old clients ignore unknown
datagram tags, old hosts ignore unknown cap bits (client keeps the combined
stage). The hardened data-plane format is untouched — this rides the
established QUIC side-plane pattern (0xC8…0xCE). NativeClient ORs the bit
in unconditionally and exposes next_host_timing(); the C ABI gains
PunktfunkHostTiming + punktfunk_connection_next_host_timing (additive).
The synthetic host emits 0xCF too, so pure-loopback protocol tests cover
the plane.

The probe reports the split (host_p50/p95_us · net_p50/p95_us) and is our
direct analogue of Sunshine's "host processing latency" — ours additionally
includes the paced send.

Validated on loopback (synthetic host + probe, debug build): 240/240 AUs
matched, host_p50 6.5 ms + net_p50 6.4 ms ≈ capture→received p50 13.0 ms.
Core suite + new 0xCF roundtrip/truncation test green; host+core+probe
clippy clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:22:12 +00:00
enricobuehler 09a5957c6d feat(clients): unified stats vocabulary across every client + Moonlight comparison docs
One stat model everywhere (design/stats-unification.md): four measurement
points (capture/received/decoded/displayed), three stages that tile the
interval exactly, and a HUD that shows the addition explicitly —

  end-to-end 14.2 ms p50 · 19.8 p95 · capture→on-glass
  = host+network 9.8 + decode 2.1 + display 2.3

replacing each client's ad-hoc mix of overlapping absolutes (the Apple HUD's
three arrow lines that looked sequential but weren't), mean-vs-median decode
times (Windows/Linux), missing same-host-clock flags (Windows/Linux), and
three different names for the same capture→received measurement (probe's
"reassembled", Apple/Android's "client", Windows/Linux's post-decode "lat").

Per client: Apple threads receivedNs through the VT decode via the frame
refcon bit pattern so the decode stage exists at all (stage-1 fallback
honestly degrades to a capture→received headline); Windows carries
FrameTimes through the existing frame channel to the render thread and adds
e2e p50/p95 post-Present; Linux stamps received at AU pop and rides
decoded_ns on DecodedFrame to the paintable-set site; Android pairs receipt
stamps with MediaCodec output buffers via the codec's pts round-trip (JNI
stats array 14→16 doubles, indexes 0-13 unchanged). fps now uniformly counts
received AUs; lost/(received+lost) per window, hidden at zero.

docs-site gains "Understanding the Stats Overlay": what each line means, why
the equation only approximately sums (percentiles), and a line-by-line
Moonlight/Sunshine matrix — including that Moonlight has no end-to-end
number and its "network latency" is an ENet control RTT, so punktfunk's
headline must not be compared against any single Moonlight line.

Verified here: linux client + probe + core check/clippy/fmt green, android
native cargo-ndk arm64 check green. Pending: Windows CI + on-glass, swift
test on the mac, on-device Android.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:01:29 +00:00
enricobuehler c7630ff5dc fix(host/audio): mic pump — open handshake on Linux + rapid-death backoff
apple / swift (push) Successful in 1m8s
apple / screenshots (push) Successful in 5m18s
android / android (push) Successful in 3m21s
windows-host / package (push) Successful in 6m58s
ci / rust (push) Successful in 1m58s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m1s
ci / bench (push) Successful in 4m49s
deb / build-publish (push) Successful in 4m37s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m17s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m59s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m34s
Found by a live boot-order test (host started before the user session's
PipeWire): PwMicSource::open returned Ok before the daemon connection was
attempted, so a PipeWire that wasn't running surfaced as an instantly-dead
instance instead of an open failure — and the pump churned
open→die→reopen at heartbeat rate (1 Hz "virtual mic ready" log spam)
instead of backing off.

- PwMicSource::open now has a bring-up handshake (mirrors the Windows
  backend): ready only after connect + stream connect succeed, so a
  down daemon is an open ERROR and the pump's backoff engages.
- The pump triages deaths: an instance that lived >= 5 s (a one-off
  daemon restart) reopens immediately with the backoff reset; one that
  died right after opening counts as a failed open and backs off
  (2 s → 60 s cap). New pump test rapid_death_backs_off.

Re-validated live: host started with PipeWire stopped → throttled
"unavailable" warns, zero churn; daemon started → mic node up on the
next retry; exactly one pump + one loop thread (no leak).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:58:06 +00:00
enricobuehler 2c7ded0f3c fix(host/audio): rebuild mic passthrough — eager, self-healing virtual mic on both hosts
apple / swift (push) Successful in 1m7s
ci / rust (push) Successful in 1m57s
ci / web (push) Successful in 59s
android / android (push) Successful in 3m19s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m12s
windows-host / package (push) Successful in 7m2s
ci / bench (push) Successful in 4m52s
decky / build-publish (push) Successful in 14s
deb / build-publish (push) Successful in 4m37s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m14s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m40s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m28s
Mic passthrough silently died on real hosts. Root causes, all fixed:

- No liveness anywhere: a PipeWire restart (Linux) or any WASAPI device
  error (Windows) killed the backend worker; push() fed the dead queue
  for the rest of the host's life. VirtualMic now has a liveness
  contract (push -> bool, alive(), discard()) and the new shared
  audio::MicPump reopens with backoff, probing on an idle heartbeat so
  the mic heals BETWEEN sessions too. Validated live: systemctl restart
  pipewire -> node back in ~0.5 s, tone flows through the reopened
  backend.

- Lazy creation: the mic device didn't exist until the first 0xCB
  frame, but games bind their capture device at launch and never
  re-follow. The pump opens eagerly at host start (node exists with
  zero clients, elected default source).

- Windows headless dead-end: with VB-CABLE as the ONLY render endpoint
  (exactly what the installer ships), the anti-echo guard rejected the
  cable as the default render endpoint -> mic permanently dead. The new
  wiring_plan (pure, unit-tested on every platform) assigns the mic its
  endpoint FIRST (cable reserved for the mic), points the loopback at a
  DIFFERENT endpoint, and the capture side now yields (explicit
  endpoint or honest error) instead of the mic dying. Plan recomputed
  per (re)open — endpoints churn at boot/logon/driver installs.

- Stale bursts: buffered audio from a previous session played into a
  newly-attached recorder (observed live). Timestamped chunks + a
  consumer-gap check in the process callback age everything past 1 s.

The Linux node mechanism stays the stream-based Audio/Source with
RT_PROCESS + priority.session: the canonical null-audio-sink adapter
recipe was tested on this box (PipeWire 1.6.2) and never gets a clock
(QUANT 0 -> pure silence), and WirePlumber reroutes a feeder targeting
it to the default sink (echo). Decision documented in the module docs.

Live-validated on this box (synthetic host + probe --mic-test,
pw-record): eager node, both attach orderings, PipeWire-restart
self-heal, post-session silence. Windows side compile/CI + on-glass
validation pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:41:19 +00:00
enricobuehler b7048446c4 fix(windows-host): IDD-push compose kick — idle desktop no longer fails the attach gate
windows-drivers / probe-and-proto (push) Successful in 24s
apple / swift (push) Successful in 1m8s
ci / rust (push) Successful in 1m42s
windows-drivers / driver-build (push) Successful in 1m45s
ci / web (push) Successful in 54s
android / android (push) Successful in 3m39s
ci / docs-site (push) Successful in 1m8s
deb / build-publish (push) Successful in 4m40s
ci / bench (push) Successful in 4m58s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
decky / build-publish (push) Successful in 25s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
release / apple (push) Successful in 8m9s
windows-host / package (push) Successful in 7m35s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m27s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m11s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 54s
flatpak / build-publish (push) Successful in 4m26s
apple / screenshots (push) Successful in 5m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m46s
docker / deploy-docs (push) Successful in 24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m21s
DWM presents a display only when something dirties it. On an idle desktop a
perfectly healthy session sat at E_PENDING: the driver attached but no
first frame ever landed, so wait_for_attach's 4 s gate failed the open (and
a mid-session ring recreate hit the same stall against the 3 s
recover-or-drop). A real client escaped only because its own input soon
dirtied the desktop; a headless probe / input-less connect never did.

kick_dwm_compose() injects two net-zero 1 px relative mouse moves via
SendInput — pf-vdisplay has no hardware-cursor plane, so a cursor move is
composited into the frame, a guaranteed real present onto the IDD
swap-chain (the mechanism --input-test always relied on; the pointer ends
where it started). Wired into wait_for_attach (first kick at 600 ms, then
every 800 ms) and, rate-limited, into the GB1 recovery window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 19:24:57 +00:00
enricobuehler 3039626b87 design(host): add vrr design doc 2026-07-03 19:22:19 +00:00
enricobuehler 3f33ed30ae fix(windows-host): claim the vdisplay single-instance guard eagerly at serve startup
On-glass the lazy (first-session) claim let a second host started while the
freshly-restarted service sat idle win the mutex and ADD a monitor on the
real driver — priority backwards. The claim is now a process-global,
retryable slot (a failed claim is not memoized, so it heals once the other
instance exits), and `serve` claims it before any client can connect;
ensure_device keeps the lazy claim for standalone punktfunk1-host runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:57:47 +00:00
enricobuehler 7e31020c1c fix(windows-host): second-host guard — classify ACCESS_DENIED on the instance mutex as in-use
On-glass the SCM service creates Global\punktfunk-vdisplay-manager as
SYSTEM, so a second elevated-admin host's CreateMutexW fails ACCESS_DENIED
(the implicit open is checked against the SYSTEM DACL) before the
ALREADY_EXISTS branch can fire — right refusal, wrong message. Map it to
the same loud "another instance is live" error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:53:54 +00:00
enricobuehler fe54aff658 fix(windows-host): cross-plane IDD serialization, linger-expiry race, second-host guard
Batch C of the audit's medium tier (M7+M8+M9):

- M7: GameStream sessions now run the same begin_idd_setup dance as
  punktfunk/1 before creating the shared monitor. A GS connect could
  previously ADD/reconfigure the monitor while a native session was
  mid-build (and vice versa), and its sealed-channel delivery replaced the
  native ring (newest-wins) — each plane could freeze the other. GS has no
  cooperative stop plumbing, so it registers a flag nobody reads: a later
  session signals it, waits the 3 s grace, then force-preempts — the
  intended handover.
- M8: the linger-expiry teardown now runs UNDER the state lock. Running it
  outside let a concurrent acquire see Idle and ADD+isolate while the old
  monitor's pinger-join / CCD-restore / REMOVE were still in flight — a
  failed or de-isolated session exactly at the expiry boundary. A racing
  acquire now waits the few teardown seconds instead. Lock order stays
  state → device everywhere; the pinger takes only the device lock.
- M9: a named mutex (Global\punktfunk-vdisplay-manager) makes a SECOND host
  process fail its vdisplay open loudly instead of firing a startup
  CLEAR_ALL that razes the live host's monitors mid-stream (the admin
  footgun the shared watchdog then masked).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:28:22 +00:00
enricobuehler b46aa15afb fix(windows-drivers): pf-vdisplay robustness — AdapterInitStatus gate, pooled-device TDR check, MMCSS-optional worker
Batch B of the audit's medium tier (M4+M5+M6):

- M4: adapter_init_finished now reads AdapterInitStatus (was ignored) and
  only stashes the adapter on NT_SUCCESS, per the MS sample. A failed async
  init previously produced a HUSK adapter: monitors created on it arrive
  but the OS never assigns a swap-chain — every session black-screens with
  no visible cause (the exact signature live fault-injection produced after
  a WUDFHost kill). Unset adapter → ADD fails cleanly (host-retryable) and
  a re-entrant D0 retries the init; the status is now in the debug log.
- M5: pooled_device checks GetDeviceRemovedReason on a cache hit — a TDR'd
  device was returned for its LUID forever (SetDevice fail-loop, black
  virtual display until device teardown); now it falls through to a fresh
  create.
- M6: an AvSetMmThreadCharacteristicsW failure no longer aborts the worker
  before draining (which stalled the monitor and leaked the WDF swap-chain
  object) — continue unprioritized like the MS sample; revert only if MMCSS
  actually engaged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:20:48 +00:00
enricobuehler 058630f542 feat(decky): visible branded Steam shortcut, one-tap client updates, fullscreen-page polish
- The "Punktfunk" shortcut is no longer hidden: it now ships committed
  artwork (grid/wide/hero/logo/icon, generated by scripts/gen-steam-art.py
  — a pure-stdlib SDF renderer drawing the lens mark + a monoline
  "punktfunk" wordmark) applied via SetCustomArtworkForApp /
  SetShortcutIcon. Existing installs are unhidden and re-arted once per
  ART_VERSION; relaunching the library entry streams to the last host.
- Updates cover the flatpak CLIENT too: check_update compares the
  user-scope installed commit against its remote, applyUpdate runs
  `flatpak update --user` first (awaited) and the plugin reinstall —
  which reloads the panel — last; docs spell out the sudo-less --user
  update ("sudo flatpak update" silently skips per-user installs).
- Fullscreen page: DialogButton stretches to 100% width in the gamepad
  UI, so the Stream/Pair/Refresh/… actions filled whole rows — sized to
  content + right-aligned now; the header drops its Update button (About
  tab + QAM banner keep the flow) and the back button gets a real 40px
  hit target.
- Settings: the disable-Steam-Input note also shows for Automatic — on a
  Deck that now forwards the built-in controller as a Steam Deck pad
  (paddles/trackpads/gyro), which needs Steam Input off for the shortcut.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:16:40 +00:00
enricobuehler e9c1f4083a fix(client-linux): Deck Gaming Mode — auto pad type, real chrome-less fullscreen, leave-to-Gaming-Mode, colour bisect
- "Automatic" gamepad type resolves to the virtual Steam Deck pad on Deck
  hardware (env SteamDeck / DMI Jupiter|Galileo): the built-in 28DE:1205
  identity is invisible at Hello time — the Valve HIDAPI drivers run
  in-session only and Steam Input shadows the pad with its virtual X360 —
  so auto always fell through to Xbox 360. "steamdeck" is now also
  selectable in Settings.
- Chrome-less launches flatten the window CSS (border-radius/box-shadow)
  and fullscreen at startup: gamescope never ACKs the xdg fullscreen
  state, so adwaita kept the floating-CSD rounded corners + shadow
  visible over the stream.
- Gaming-Mode --connect launches quit on session end, so Steam ends the
  "game" and the Deck returns to Gaming Mode — previously the app popped
  to its own hosts page, stranding the user fullscreen and making the
  escape chord read as broken.
- The capture hint is controller-aware; the chromeless hint teaches the
  hold-chord ("hold L1+R1+Start+Select to leave") and a quick chord press
  re-flashes it.
- Colour bisect for the reported off-colours on the VAAPI dmabuf path:
  graphics offload defaults OFF under gamescope (a subsurface hands the
  NV12 CSC to the compositor), PUNKTFUNK_OFFLOAD=1|0 overrides, and each
  colour-signaling change logs whether GDK accepted the BT.709-narrow
  color state (fallback = GDK's BT.601 dmabuf default).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:16:26 +00:00
enricobuehler 20f0d2802f feat(client/android): Snapdragon latency tuning — ADPF pipeline hints, game mode, max-clock decode
Three levers to lower and steady decode latency on Snapdragon (Adreno) devices:

- ADPF (Adaptive Performance Framework): a new dlsym-resolved hint session
  (native/src/adpf.rs; API-33+, resolved at runtime so there's no build-time
  link dependency and libpunktfunk_android.so still loads on API 31/32) tells
  the CPU governor the video pipeline runs a per-frame real-time workload, so it
  keeps those threads on fast cores at high clocks. It now covers all three
  latency-critical threads — the pf-decode feed/drain/present loop, the core
  data-plane pump (UDP receive + FEC reassembly), and the audio thread — via a
  new generic hot-thread registry on NativeClient (register_hot_thread /
  hot_thread_ids; the pump self-registers). The session is built lazily on the
  first presented frame, since ADPF createSession rejects a set containing any
  not-yet-live tid.

- operating-rate -> Short.MAX ("as fast as possible"): pushes the Qualcomm
  decoder to run each frame at max clocks instead of merely sustaining the
  display rate at a power-saving clock that adds per-frame decode latency.

- appCategory="game": makes the app eligible for OEM Game Mode / Game Dashboard
  performance profiles.

The core registry is cross-platform (gettid on Linux/Android, a no-op
elsewhere) — no Android-specific pollution of the shared core. Host workspace +
64 core tests green; Android arm64-v8a + x86_64 (platform 31) build + clippy
clean. On-device Snapdragon validation pending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 17:16:11 +00:00
enricobuehler 6f8fb15c9b fix(windows-host): self-heal the hostless-zombie pf-vdisplay device (adapter cycle + re-probe)
Fault-injection on-glass showed a killed/crashed WUDFHost leaves the devnode
"started" but HOSTLESS: PnP Status OK, no WUDFHost process, zero device-
interface instances — is_available() then fails every future session at the
vdisplay::open gate (and a reopen inside VdisplayDriver::open finds nothing),
until something cycles the device. Port reset-pf-vdisplay.ps1's adapter
disable→enable step in-process (restart_vdisplay_device): the open gate now
uses ensure_available() (cycle once + bounded re-probe; a genuinely
uninstalled driver — no adapter devnode — still fails fast), and
VdisplayDriver::open retries open_device over a short arrival window after a
cycle, covering the manager's reopen path too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:12:43 +00:00
enricobuehler 89455032a0 fix(windows-host): IDD-push resilience — driver-death recovery, reopenable control device, full interface discovery
Batch A of the audit's medium tier (M1+M2+M3):

- M1 driver-death detection: a dead WUDFHost stops publishing, which at the
  ring is indistinguishable from an idle desktop — SDR sessions streamed a
  frozen frame forever (next_frame's 20 s bail is unreachable once anything
  presented). The ChannelBroker's process handle now doubles as a liveness
  probe (SYNCHRONIZE at OpenProcess); while no fresh frame arrives,
  try_consume polls it (rate-limited) and fails the capturer, landing in the
  session's bounded in-place rebuild.
- M2 reopenable control device: the manager's OnceLock-cached handle is now
  a retire/reopen DeviceSlot — a gone-classified IOCTL failure (driver
  upgrade / WUDFHost restart; pinger, create, or REMOVE) retires the handle
  and the next use reopens + re-handshakes. Retired handles are deliberately
  kept alive forever: bare-HANDLE holders (pinger, ChannelBroker) rely on
  never-closed, and a retired handle only fails IOCTLs. CLEAR_ALL runs on
  the FIRST open only (a reopen races live-ish sessions); acquire retries
  the monitor create once after a reopen. The JOIN path now probes the
  active monitor's WUDFHost pid and preempts a DEAD monitor instead of
  handing the rebuilding session its stale target — without this the whole
  recovery chain starved to the rebuild budget.
- M3 interface discovery: enumerate ALL interface instances with an
  SPINT_ACTIVE filter (a Code-10 devnode at index 0 no longer shadows the
  live interface), HDEVINFO behind RAII (error paths leaked one per probe),
  the raw device handle wrapped before GET_INFO (leaked on handshake
  failure), and the detail-sizing result guarded before the cbSize write.
- pf-driver-proto: SetFrameChannelRequest doc now states the real
  adopt-on-success contract (the old wording invited a driver-side
  close-on-error — a cross-process double-close against the host's reap).
- install: pf_vdisplay_present() passes /connected so a phantom devnode
  can't suppress creating a live ROOT node.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:04:19 +00:00
enricobuehler 0da9d8ec10 fix(windows): IDD-push audit highs — keyed-mutex timeout, two per-frame leaks, IDD_PUSH knob, pooled-device threading
Five verified findings from the IDD-push/pf-vdisplay deep audit:

- Keyed-mutex acquire (BOTH endpoints): AcquireSync returns WAIT_TIMEOUT
  (0x102) / WAIT_ABANDONED (0x80) as SUCCESS-severity HRESULTs, which the
  windows-rs Result wrapper erases — a busy slot read as "acquired", so
  driver and host could race the same ring texture (torn frames) and the
  designed busy-skip backpressure was dead code. Both sides now classify
  the raw vtable HRESULT; WAIT_ABANDONED counts as acquired (ownership
  transfers — refusing it would wedge the slot forever).
- Host SDR hot path leaked one ID3D11VideoProcessorInputView per converted
  frame: the D3D11_VIDEO_PROCESSOR_STREAM ManuallyDrop field suppressed the
  release after VideoProcessorBlt. Released by hand now, success or not.
- Driver leaked IddCx's per-acquire surface reference (from_raw_borrowed on
  a TRANSFERRED reference — the MS sample Attach/Reset's it): the swap-chain
  surface set survived swap-chain destruction, the likely true root cause of
  the ~50 MB-per-reconnect VRAM loss that device pooling only mitigated.
  Now adopted via from_raw (publisher or not) and dropped pre-Finished.
- PUNKTFUNK_IDD_PUSH removed: capture is unconditionally IDD-push, but the
  vdisplay manager still gated the lingering-monitor preempt (and render
  pin) on the knob, whose default was OFF — dev/CLI runs reused a lingering
  monitor whose IddCx swap-chain is dead (black reconnect). The preempt and
  the render-GPU pin are now unconditional; host.env comments no longer
  promise the removed DDA/WGC fallback.
- Driver D3D device: dropped D3D11_CREATE_DEVICE_SINGLETHREADED (unsound
  since DEVICE_POOL shares one device across processors) and the pooled
  immediate context is now SetMultithreadProtected — two concurrent
  monitors' workers otherwise race an unlocked context (UB in the UMD).

No wire-contract change (pf-driver-proto untouched); the driver fixes take
effect on the next pf-vdisplay redeploy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:27:13 +00:00
enricobuehler fbf3fea0c8 fix(packaging/windows): Inno [Code] comment brace-nesting trap broke ISCC
apple / swift (push) Successful in 1m17s
windows-host / package (push) Successful in 7m3s
apple / screenshots (push) Successful in 5m37s
android / android (push) Successful in 3m22s
ci / web (push) Successful in 48s
ci / rust (push) Successful in 1m18s
ci / bench (push) Successful in 4m59s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 4m35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m31s
docker / deploy-docs (push) Successful in 15s
The PublicFwParam doc comment contained a literal code-constant token; Inno's
{ } comments don't nest, so its closing brace ended the comment early and the
trailing text parsed as code ("'BEGIN' expected", compile aborted). Reworded to
avoid the literal braces + added a warning note. Verified: the [Code] section
has no other nested-brace-in-comment traps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:27:27 +00:00
enricobuehler c52ae119e1 fix: regenerate Cargo.lock for punktfunk-host's winresource build-dep
apple / swift (push) Successful in 1m10s
audit / cargo-audit (push) Successful in 1m12s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m8s
android / android (push) Successful in 3m49s
ci / web (push) Successful in 48s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 45s
ci / docs-site (push) Successful in 56s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 50s
release / apple (push) Successful in 8m51s
ci / rust (push) Successful in 5m51s
ci / bench (push) Successful in 4m51s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
deb / build-publish (push) Successful in 4m46s
apple / screenshots (push) Successful in 5m34s
flatpak / build-publish (push) Successful in 5m33s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m52s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m43s
docker / deploy-docs (push) Failing after 18s
windows-host / package (push) Has been cancelled
The "Punktfunk Host" identity work added winresource to the host crate but
didn't update the lock, so every --locked CI job failed to resolve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:17:31 +00:00
enricobuehler 5d7aabe8f0 fix(scripts/windows): keep deploy-host.ps1 pure ASCII (em-dash -> hyphen)
apple / swift (push) Successful in 1m8s
ci / rust (push) Failing after 50s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m8s
android / android (push) Successful in 3m24s
deb / build-publish (push) Failing after 45s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
windows-host / package (push) Failing after 6m18s
apple / screenshots (push) Successful in 5m35s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m26s
docker / deploy-docs (push) Successful in 17s
Two comment em-dashes I added tripped the installer-run ASCII guard (PS 5.1
mis-parses non-ASCII on non-UTF-8 locales).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:09:17 +00:00
enricobuehler f204a89cef perf(encode/windows): AMF quality=speed + bf=0; drop the useless poll spin
ci / rust (push) Failing after 48s
windows-host / package (push) Failing after 10s
apple / swift (push) Successful in 1m6s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m8s
android / android (push) Successful in 3m20s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
apple / screenshots (push) Successful in 5m10s
ci / bench (push) Successful in 4m43s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m25s
deb / build-publish (push) Failing after 44s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m21s
On-box A/B on the .173 Ryzen 7000 iGPU (720p60, real composition via input
injection — an idle virtual desktop composes ~1 fps and gives meaningless
encode timings): the encode-time-first `quality=speed` preset + explicit `bf=0`
cut host-side encode_us from ~36 ms to ~19.5 ms.

The blocking-poll idea from the prior commit was WRONG and is reverted to a
single non-blocking receive (default PUNKTFUNK_FFWIN_POLL_MS=0): libavcodec's
hevc_amf holds ~2 frames before releasing the oldest (needs frame N+2 to flush
N), so a spin between submits provably never yields the owed AU — verified with
a 150 ms cap pegging at exactly 150 ms across every usage preset and pipeline
depth. That ~2-frame buffer is inherent to the libavcodec wrapper, not host
scheduling; the real latency lever is a direct AMF SDK encoder (the AMF
analogue of the direct-NVENC path), tracked as the next AMD work item. The
env knob is retained for a future VCN/driver where a bounded spin can help.

Also measured and rejected: PUNKTFUNK_ZEROCOPY=1 on AMF is ~2x WORSE (68 ms vs
36 ms) — the D3D11 import path adds sync overhead beyond the readback it saves,
so the system-memory default stays. GPU-priority elevation is already
process-wide (dxgi.rs), so it covers the iGPU encode session with no change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:57:39 +00:00
enricobuehler 24fa018c70 chore(encode/windows): AMF forensics knobs — PUNKTFUNK_AMF_USAGE + PUNKTFUNK_FFWIN_POLL_MS
apple / swift (push) Successful in 1m6s
ci / web (push) Successful in 53s
deb / build-publish (push) Failing after 44s
windows-host / package (push) Failing after 10s
ci / rust (push) Failing after 49s
android / android (push) Successful in 3m33s
apple / screenshots (push) Successful in 5m18s
ci / docs-site (push) Successful in 57s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 5m0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m27s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m18s
The blocking poll landed but wait_us pegs at exactly the 2-frame-period cap:
AMF holds the AU ~2 frame periods regardless of retrieval. Field knobs to
bisect on-box (usage preset × poll cap) without rebuild cycles.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:39:36 +00:00
enricobuehler 51a6ca7e02 fix(encode/windows): AMF latency — honor the loop's blocking-poll contract + preset polish
apple / swift (push) Successful in 1m6s
windows-drivers / driver-build (push) Successful in 1m34s
windows-drivers / probe-and-proto (push) Successful in 20s
ci / rust (push) Failing after 47s
ci / web (push) Successful in 52s
windows-host / package (push) Failing after 11s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 3m20s
deb / build-publish (push) Failing after 46s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
apple / screenshots (push) Successful in 5m11s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m27s
ci / bench (push) Successful in 4m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m24s
The session loop's pipeline deferral was designed around direct NVENC, whose
poll() BLOCKS in lock_bitstream; libavcodec's AMF wrapper is truly async
(EAGAIN until the ASIC finishes), so a single non-blocking receive quantized AU
retrieval to the submit cadence: +1–2 frame periods flat (~43 ms p50 at 720p60
on the Ryzen iGPU vs ~3.5 ms of actual encode). FfmpegWinEncoder now tracks
in-flight frames and, while an AU is owed, spin-polls with short sleeps bounded
to ~2 frame periods (an overloaded encoder degrades to next-tick pickup instead
of stalling capture). Also: quality=speed (latency-first, iGPU-class VCN),
explicit bf=0 (h264_amf defaults >0 on RDNA3+), AMF low-latency submission
mode (FFmpeg ≥6.1, ignored on older).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:32:41 +00:00
enricobuehler b9fde03f1e feat(security): finish Windows firewall Public opt-in wiring + vuln-disclosure + doc cleanup
Firewall (the service.rs core landed in efb1ba2): scope the web-console rule
(TCP 47992) to Domain+Private by default with a `--allow-public-network` opt-in
that deletes-then-re-adds the rule, and add the installer "Allow connections on
Public networks" task (unchecked) forwarding the flag to `service install` and
`web setup`. Default is now trusted-networks-only; Public is explicit.

Vulnerability disclosure: SECURITY.md (report to security@punktfunk.com, scope,
SLAs, safe harbor), a Gitea issue-template contact link, a README security line,
and a Reporting section on the docs Security page.

Docs: the Security page now documents the Private/Domain firewall default (and
how to fix a misclassified-Public network / opt in); removed internal design-doc
and CLAUDE.md links from the user-facing docs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 14:08:17 +00:00
enricobuehler efb1ba26d7 fix(windows): opt-in pad-driver file logs + size-capped service log rotation
Two disk-write fixes:

- pf-xusb/pf-dualsense no longer write C:\Users\Public\pf*-driver.log
  unconditionally — the file log is now opt-in (debug builds, or the
  PFXUSB_DEBUG_LOG / PFDS_DEBUG_LOG system env var), mirroring the audit-§4.4
  fix pf-vdisplay already got: a release driver never writes the world-writable
  Public file (info-leak/DoS surface), and the per-report OUTPUT/SET_STATE hex
  dumps stop being a sustained per-rumble disk-write path during gameplay.
  OutputDebugStringA stays unconditional; the host's driver-silence WARN and
  the gamepad-driver-health failure-mode table now say the log is opt-in.

- service.log/host.log get one-generation rotation: at each (re)open a file
  over 10 MB is renamed to .old, so a crash-restart loop or a RUST_LOG=debug
  left in host.env can't grow the append-forever logs without bound. Rotation
  runs only before an open (never under a live appender — host.log's handle
  lacks FILE_SHARE_DELETE, so a racing rename harmlessly fails).

Windows CI compile/clippy pending (drivers workspace + host are not
Linux-cross-checkable); rides along with the next pad-driver redeploy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:03:32 +00:00
enricobuehler 1320e3dc66 fix(scripts/windows): deploy-host.ps1 builds all-vendor when an FFmpeg tree exists
windows-host / package (push) Failing after 20s
apple / swift (push) Successful in 1m9s
apple / screenshots (push) Successful in 5m26s
ci / rust (push) Failing after 48s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 3m22s
deb / build-publish (push) Failing after 43s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m40s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m27s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m18s
The dev deploy built --features nvenc only, so a web-console GPU preference
pointing at an AMD/Intel adapter made every session die at encoder open
(NV_ENC_ERR_NO_ENCODE_DEVICE) — the exact "can't connect" just hit on the RTX
box's Ryzen iGPU. The script now enables amf-qsv when FFMPEG_DIR (machine env,
process env, or C:\Users\Public\ffmpeg) has a dev tree, and copies the FFmpeg
runtime DLLs next to the exe after a successful build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:52:55 +00:00
enricobuehler 1be83575b6 feat(host/windows): "Punktfunk Host" identity in Task Manager (icon + version info)
punktfunk-host.exe embedded no icon or version resources, so Task Manager and
Explorer showed a bare lowercase exe name with a generic icon. build.rs now
embeds the branded .ico + FileDescription "Punktfunk Host" / ProductName
"Punktfunk" via winresource (same pattern as the Windows client and the tray;
Linux packaging builds skip the block). The tray gets a matching "Punktfunk
Tray" description, and the SCM display name moves off lowercase
"punktfunk streaming host" to "Punktfunk Host" (applied idempotently by
`service install` on upgrade).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:52:55 +00:00
enricobuehler 4d1d20f832 chore: untrack CLAUDE.md
apple / swift (push) Successful in 1m14s
apple / screenshots (push) Successful in 5m45s
ci / rust (push) Successful in 1m15s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m5s
android / android (push) Successful in 3m25s
deb / build-publish (push) Successful in 4m35s
ci / bench (push) Successful in 4m55s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m27s
docker / deploy-docs (push) Successful in 18s
Local per-box assistant instructions (incl. internal environment detail) don't
belong in the published tree; the file stays on disk, now gitignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:22:23 +00:00
enricobuehler 6e875fea44 fix(apple/ios): the ACTUAL type-checker bomb was pointerSection's footer ternary chain
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 8m36s
apple / screenshots (push) Successful in 5m44s
ci / rust (push) Successful in 1m31s
ci / web (push) Successful in 56s
android / android (push) Successful in 10m1s
deb / build-publish (push) Successful in 4m33s
ci / bench (push) Successful in 4m52s
ci / docs-site (push) Successful in 1m24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m20s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m19s
docker / deploy-docs (push) Failing after 14s
4f3cd24 split the wrong expression — act's log masking hid the real line number.
The unmasked retry pinpointed it: the pointerSection footer, a ten-segment
string + chain with an isPad ternary nesting four more, evaluated inside the
ViewBuilder. Moved the copy into a plain computed String built with +=
statements (linear to type-check); no text change. The two remaining 5-6
segment chains in Settings are compiled by the passing macOS slice, so they
are proven cheap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:17:01 +00:00
enricobuehler 4f3cd24036 fix(apple/ios): split streamModeSection — the inline iOS branch blew the type-checker budget
release / apple (push) Successful in 6m29s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m13s
apple / swift (push) Successful in 1m5s
apple / screenshots (push) Successful in 4m5s
ci / bench (push) Successful in 4m36s
ci / rust (push) Successful in 11m23s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 4m26s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m45s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m37s
android / android (push) Successful in 9m37s
docker / deploy-docs (push) Successful in 19s
The Section's iOS content (resolution wheel + 3-way refresh rows + bitrate
rows) as ONE ViewBuilder expression hit "the compiler is unable to type-check
this expression in reasonable time" — failing exactly one build slice, the iOS
archive, so swift test (macOS) and the tvOS/macOS archives never saw it and the
0.6.0 iOS TestFlight upload soft-failed. Extracted iosResolutionWheel /
iosRefreshRows / bitrateRows; no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:06:48 +00:00
enricobuehler af13f0b749 chore(release): 0.6.0
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
android-screenshots / screenshots (push) Successful in 2m18s
android / android (push) Successful in 4m13s
decky / build-publish (push) Successful in 26s
windows-host / package (push) Successful in 6m36s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m10s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m50s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 35s
release / apple (push) Successful in 7m53s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m32s
deb / build-publish (push) Successful in 9m52s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
web-screenshots / screenshots (push) Successful in 2m38s
ci / web (push) Successful in 48s
ci / rust (push) Successful in 11m43s
linux-client-screenshots / screenshots (push) Successful in 1m33s
flatpak / build-publish (push) Successful in 4m8s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m10s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m49s
docker / deploy-docs (push) Successful in 25s
ci / docs-site (push) Successful in 57s
ci / bench (push) Successful in 5m9s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:19:18 +00:00
enricobuehler d285d4a0b2 fix(tray): live-probe the web console instead of sniffing the install layout
windows-drivers / probe-and-proto (push) Successful in 29s
audit / cargo-audit (push) Successful in 1m31s
apple / swift (push) Successful in 1m8s
windows-drivers / driver-build (push) Successful in 1m35s
android / android (push) Successful in 4m45s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m0s
release / apple (push) Successful in 7m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
The "Open web console" entry was gated on {exe dir}\web\web-run.cmd (Windows)
/ the punktfunk-web unit file (Linux) — which misses consoles run from a repo
checkout (the RTX box, caught on-glass) and shows a dead entry while an
installed console is stopped. The poller now probes https://127.0.0.1:<web
port>/ each cycle (any HTTP response = up, transport failure = down) and the
menu follows live on both platforms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:17:01 +00:00
enricobuehler 04f370999c fix(web): pin the sidebar at viewport height
Sticky h-dvh sidebar: long pages scroll the content, not the nav — the flex
stretch was pushing the language switcher below the fold; overflow-y-auto keeps
the nav usable on short viewports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:52 +00:00
enricobuehler 2c937855b3 fix(packaging/windows): Windows 11 22H2 floor + tray install task + stale console-port fixes
The OS floor is now enforced at install time (MinVersion=10.0.22621 with an
explanatory [Messages] override): pf-vdisplay is built against IddCx 1.10, and
on Windows 10 (incl. LTSC) / Win11 21H2 the device fails start with Code 10
STATUS_DEVICE_POWER_FAILURE (field-reported). Docs (site requirements/install/
windows-host pages + README) state the floor; new docs-site Security page.

Installer also gains the trayicon task (punktfunk-tray.exe file + HKLM Run key,
post-install launch as the signed-in user, upgrade taskkill + uninstall
--quit/taskkill choreography before file deletion), and the wizard/cleanup
text/port sweeps move off the stale :3000 web-console references to :47992
(cleanups sweep both for upgrades from old installs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:52 +00:00
enricobuehler 8005b11faf feat(tray): system-tray status icon for the host (Windows + Linux)
New crates/punktfunk-tray — a small per-user companion showing the host service
state at a glance (running / stopped / starting / degraded / failed + the live
session in the tooltip) with one-click actions: open web console, approve a
pending pairing request, start/stop/restart, open logs. No more digging through
logs to learn whether the service came back after a reboot or an update.

Status is service-manager-FIRST (SCM / systemd user unit — a port squatter can
never fake Running), then the new loopback-only unauthenticated
GET /api/v1/local/summary (counts/booleans only; the mgmt token and cert.pem
are SYSTEM/Admins-DACL'd on Windows, so a non-elevated tray cannot bearer-auth).

Windows: windows_subsystem binary (a console exe in the Run key would flash a
terminal at sign-in), Shell_NotifyIcon + hidden window, per-session single
instance, TaskbarCreated re-add, --quit for the uninstaller; service actions
elevate per click via ShellExecuteW "runas" onto the new
`punktfunk-host service restart` (stop → wait Stopped → start).
Linux: ksni/StatusNotifierItem over zbus, systemctl --user actions (no polkit),
/etc/xdg/autostart entry whose --autostart self-gates to actual host users.
Icons: scripts/gen-tray-icons.py (pure stdlib) renders the brand lens + status
dot into committed .ico/hicolor assets; deb/rpm/arch ship binary+autostart+icons.

Live-validated: Linux on the headless KDE session (SNI registration, state
transitions, menu-driven start, dbusmenu layout); Windows on the RTX box
(session-1 launch with no NIM_ADD failure, single instance, --quit, restart
round-trip, summary loopback-200/LAN-401).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:35 +00:00
enricobuehler 01fcb01019 fix(encode/windows): resolve NVENC at runtime — AMD/Intel hosts no longer crash at start
The nvenc build linked nvEncodeAPI64.dll's entry points at load time, so a
--features nvenc binary hard-crashed on any box without the NVIDIA driver
(AMD/Intel). Entry points now come from a runtime LoadLibrary table
(encode/windows/nvenc.rs load_api); a missing DLL just falls through the
encoder auto-detect to AMF/QSV/software. The generated import lib and all its
plumbing (gen-nvenc-importlib.ps1, nvenc.def, PUNKTFUNK_NVENC_LIB_DIR,
setup-build-env wiring) are gone.

Live-validated on the RTX 4090 box (NVENC session, 7000+ frames).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:18 +00:00
enricobuehler 95a08e99c3 feat(host/windows): seal the host↔driver channels (frame + gamepad, proto v2)
Frame ring (pf-vdisplay) and both gamepad SHM channels move off named Global\
objects (openable by any sibling LocalService) to UNNAMED sections/events whose
handles the host DuplicateHandles into the driver's verified WUDFHost with least
access — frame delivery over the SYSTEM+admins-only IOCTL_SET_FRAME_CHANNEL,
pads over a 32-byte named bootstrap mailbox (pid + handle value only, DoS-bounded;
HID minidrivers have no control device). Driver-validated pad_index kills
cross-pad redirects; v1↔v2 mixes fail closed with diagnosis logs on both sides.
Sibling-LocalService denial proven empirically (design/idd-push-security.md,
design/gamepad-channel-sealing.md).

Driver-side raw ops now live behind pf-umdf-util (checked shm accessors, the
forbid(unsafe_code) ChannelClient state machine, WDF request tokens) — the pad
drivers' logic is 100% safe Rust; whole drivers workspace clippy-gated in CI.

driver install --gamepad now sweeps SWD\punktfunk phantom devnodes: a re-created
SwDevice REVIVES the old devnode with its previously-bound driver (never
re-ranks), so an upgrade otherwise leaves the old driver serving — or, across
the v1→v2 fence, a dead pad (found live on the RTX box).

On-glass validated on the RTX 4090 box: frame path 7007 frames p50 2.06 ms
cross-machine; DualSense + XUSB "sealed pad channel mapped"/proto=2 attach via
both the test harness and a real streaming session; phantom-sweep repro.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:08:56 +00:00
enricobuehler a3e1ea2b44 fix(android/ci): retry transient Play API failures in play-upload.py
apple / swift (push) Successful in 1m9s
apple / screenshots (push) Successful in 4m2s
android / android (push) Successful in 11m51s
ci / web (push) Successful in 1m0s
ci / docs-site (push) Successful in 1m13s
ci / rust (push) Successful in 4m30s
deb / build-publish (push) Successful in 3m35s
ci / bench (push) Successful in 4m47s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
decky / build-publish (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m3s
docker / deploy-docs (push) Successful in 20s
The uploader only caught HTTPError — a URLError (TLS "EOF occurred in
violation of protocol", the failure that dropped two release uploads on
2026-07-02) or a Google 5xx killed the job outright. Retry those with
3/9/27 s backoff; 4xx still fails fast. The edits API is transactional
until commit, so re-sending is safe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:05:27 +00:00
enricobuehler 6686fcdded fix(gamestream/tests): sender_delivers_batches flaked under CI load — burst overflowed the default socket buffer
apple / swift (push) Successful in 1m12s
apple / screenshots (push) Successful in 4m26s
windows-host / package (push) Successful in 6m25s
ci / rust (push) Successful in 5m5s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m4s
android / android (push) Failing after 10m7s
deb / build-publish (push) Successful in 3m35s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
ci / bench (push) Successful in 4m38s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m53s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m53s
docker / deploy-docs (push) Successful in 18s
The test burst 3×100 1200 B datagrams into an undrained loopback socket: at
~2.5 KB kernel truesize each, the default ~212 KB rmem holds only ~80, so on
a starved CI runner (parallel release builds) the kernel silently dropped the
overflow and the recv loop could never reach 300 — surfacing as WouldBlock
after the 3 s timeout. Size the burst (3×20) to fit the default buffer even
with zero concurrent draining, and give recv a starvation-tolerant 10 s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:35:23 +00:00
enricobuehler 31c382fde0 chore(release): 0.5.1
audit / cargo-audit (push) Successful in 54s
apple / swift (push) Successful in 1m15s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m1s
ci / bench (push) Successful in 4m40s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 40s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 46s
release / apple (push) Successful in 7m51s
windows-host / package (push) Successful in 6m46s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m7s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
apple / screenshots (push) Successful in 4m4s
android-screenshots / screenshots (push) Successful in 1m14s
decky / build-publish (push) Successful in 15s
deb / build-publish (push) Successful in 3m25s
flatpak / build-publish (push) Successful in 4m20s
linux-client-screenshots / screenshots (push) Successful in 6m12s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m31s
web-screenshots / screenshots (push) Successful in 2m37s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m16s
docker / deploy-docs (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / rust (push) Successful in 4m32s
android / android (push) Failing after 11m14s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:05:43 +00:00
enricobuehler d707ee4d4e feat(apple,android): three-way touch input — trackpad cursor (default), direct pointer, real multi-touch passthrough
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
The two touch clients had exactly complementary gaps: iOS forwarded fingers
ONLY as raw wire touches (no way to drive the host cursor from the touch
screen), Android had the two mouse modes but no passthrough. Both now share
one three-way "Touch input" setting: Trackpad (default) / Direct pointer /
Touch passthrough.

iOS/iPadOS: Input/TouchMouse.swift ports the Android gesture engine 1:1
(same px-based acceleration curve; tap=click, two-finger tap=right-click,
two-finger drag=scroll, tap-then-drag=held drag, three-finger tap=stats
HUD via the shared hudEnabled default); direct-pointer mode maps through
the aspect-fit letterbox; the previous always-on behavior lives on as the
passthrough option. The mode latches per gesture (a Settings change never
splits one gesture across models), touchesCancelled releases held state
without synthesizing a click, and session stop flushes a mid-drag button.
Settings picker on iPhone + iPad next to the iPad-only pointer-capture
toggle. Deliberate default change: trackpad, not passthrough.

Android: new nativeSendTouch JNI shim → wire TouchDown/Move/Up (the host
already injects real touch on every backend — libei touchscreen, wlroots,
KWin fake-input, SendInput); streamTouchPassthrough forwards every finger
with stable ids and lifts still-held contacts on teardown; the trackpadMode
Boolean becomes the TouchMode enum (old pref migrated on load, never
rewritten) with a Settings dropdown.

Verified: macOS swift build + full suite (incl. new TouchMouseTests), iOS
Simulator Swift compile, cargo check/fmt/clippy on the native crate, Kotlin
app+kit compile + unit tests. On-glass feel of the iOS ballistics and
Android passthrough against a touch-aware app still pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:02:12 +02:00
enricobuehler e8196b33b8 feat(client/linux): Steam Deck batch — idle gamepad grab, fullscreen streams, in-band HDR colors, gamescope-safe settings, pad-pin persistence
windows-host / package (push) Successful in 6m41s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m5s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m6s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 47s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 54s
apple / swift (push) Successful in 1m17s
audit / cargo-audit (push) Successful in 17s
android / android (push) Successful in 3m46s
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 57s
release / apple (push) Successful in 8m41s
deb / build-publish (push) Has been cancelled
ci / bench (push) Successful in 4m39s
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / rust (push) Successful in 8m21s
Root-caused fixes from on-Deck testing (owner + first external tester):

- System input broke while the app was merely OPEN: SDL's Steam Deck HIDAPI
  driver clears the built-in controller's "lizard mode" (trackpad-mouse,
  clicky pads) at device ENUMERATION and keeps feeding the firmware watchdog
  (SDL_hidapi_steamdeck.c InitDevice/UpdateDevice) — and we enabled that
  driver at startup and held every pad open app-lifetime. The Valve HIDAPI
  hints are now enabled only while a session is attached, and only the active
  pad is opened (Settings enumerates via SDL's ID-based metadata getters, no
  open). Close/detach hands the hardware back; the watchdog restores lizard
  mode within seconds. This also unblocks click-to-capture on the Deck (the
  dead trackpad made "input not passed through" a symptom, not a cause).
- Washed-out colors from a Windows host with an HDR desktop: the host ships
  Main10 BT.2020 PQ IN-BAND (correct VUI) while the Welcome still says SDR;
  this client rendered everything as BT.709 narrow. Colour signaling is now
  read per-frame (video::ColorDesc from the AVFrame CICP fields) and drives
  the GdkDmabufTexture color state, the software path's swscale matrix/range
  plus a tagged MemoryTexture for PQ, and an "· HDR" HUD chip — GTK tone-maps
  correctly on SDR displays, mid-session SDR↔HDR flips included. Regression-
  tested against a checked-in Main10 PQ fixture (tests/pq-frame.h265).
- Streams start fullscreen by default (Settings toggle; F11 / the controller
  chord lead out, and the pointer at the top edge reveals the header while
  input isn't captured — a Deck desktop has no F11). Gaming-Mode launches
  (--fullscreen / Deck env) build the stream page with NO header bar at all:
  gamescope doesn't reliably ACK xdg_toplevel fullscreen, so anything keyed
  on is_fullscreen() could leave the title bar drawn over the stream.
- Game Mode settings were uneditable: GTK popovers are xdg_popups, which
  gamescope never maps for nested apps — every ComboRow dropdown flashed and
  died. Under gamescope the preferences dialog now uses in-window selection
  subpages (PreferencesDialog::push_subpage) via a ChoiceRow that stays a
  stock ComboRow on desktops. Covered by an in-process GTK test
  (choice_row_modes, #[ignore]d — needs a display).
- Forwarded-controller pin persists across restarts (Settings::forward_pad,
  stable vid:pid:name key — SDL instance ids are per-run) and survives
  disconnects; automatic selection skips Steam Input's sensor-less virtual
  pad (28de:11ff) so gyro doesn't silently die on Bazzite/Deck.
- "Punktfunk" branding in the About dialog.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:37:43 +00:00
enricobuehler fd699b3e2c feat(decky): plugin overhaul — on-Deck update check, exec-bit-free runner, About/host-detail UI, Punktfunk branding
Fixes from live debugging on the Deck:

- check_update() was dead on-device: Decky Loader's embedded (PyInstaller)
  Python has no usable default CA paths, so every HTTPS fetch failed with
  CERTIFICATE_VERIFY_FAILED. Build the SSL context explicitly: default paths
  first, then the known system bundles (SteamOS/Arch, Debian, Fedora/Bazzite,
  openSUSE), then certifi if importable. Verification stays on; the check
  stays offline-tolerant with its 30-min cache.
- "could not chmod runner" on every use: Decky extracts plugin zips without
  exec bits into a root-owned dir the unprivileged backend can't chmod. The
  Steam shortcut now launches the runner through /bin/sh with the script as a
  %command% argument — no exec bit needed, existing shortcuts migrate on
  reuse, the chmod attempt is gone.

UI/structure:

- index.tsx (660 lines) split into page/pair/settings/hooks/boundary modules;
  PluginErrorBoundary kept guarding every surface.
- New About section/tab: visible version + channel, explicit check-for-updates
  (forces past the cache, always toasts an outcome), setup-guide link, leave-
  chord help, and a Force-stop backstop for a wedged stream.
- Host rows open a details modal (address, protocol, pairing policy, paired
  state, fingerprint). Settings gain 1280×800 (Deck native), Xbox One and
  DualShock 4 pad types, and a host-compositor picker.
- Update flows note the Decky store contact can stall a couple of minutes on
  networks that blackhole plugins.deckbrew.xyz (observed live).
- "Punktfunk" in all user-facing strings; plugin id/paths/env unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:37:43 +00:00
enricobuehler 79dd8f58e3 docs(readme): status refresh — Windows client streaming live, console features
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:37:43 +00:00
enricobuehler be879c946a fix(host/logs): mdns-sd noise gate + tracing-log target normalization in the log ring
log-crate events arrive through the tracing-log bridge under the shim target
"log" — normalize them back to the real module path (NormalizeEvent) so the
console's target column and the noise gate see mdns_sd::… , and suppress the
bridge's log.* bookkeeping fields like the stderr fmt layer does.

Gate known-chatty third-party DEBUG targets (mdns-sd DEBUG-logs every
unparseable multicast packet — one AirPlay device floods thousands of entries
per hour) to INFO-and-up in the ring, so ambient LAN noise can't evict the
tail the ring exists to preserve. stderr under RUST_LOG is unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:37:43 +00:00
enricobuehler f3646d4e7c feat(apple/gamepad): claim controller system gestures during capture — PS button opens the Steam overlay, share/create stops screenshotting locally
apple / swift (push) Successful in 1m6s
ci / rust (push) Successful in 2m1s
ci / web (push) Successful in 56s
android / android (push) Successful in 3m19s
ci / docs-site (push) Successful in 58s
deb / build-publish (push) Successful in 3m13s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m43s
release / apple (push) Successful in 8m1s
apple / screenshots (push) Successful in 5m33s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m23s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
While a pad drives a stream, GamepadCapture now sets EVERY element's
preferredSystemGestureState to .disabled (restored to .enabled on unbind).
iOS/macOS attach system gestures to several controller buttons — share/create
took a LOCAL screenshot instead of reaching the game, and only the Home
element was opted out before. With the gestures claimed, the already-wired
chains do their job: PS/Home → wire guide → BTN_MODE on the virtual xpad
(the Steam-overlay button) / the PS bit on the virtual DualSense.

Also fold the share/create/capture element (GCInputButtonShare) into the
back/select wire bit — clone pads like the GameSir G8 expose their screenshot
button only as the share element, not buttonOptions (OR onto the same bit, so
double-exposed pads are harmless). The G8's other extra button (M) is a
firmware-local modifier (turbo/hair-trigger/swap) invisible to the OS.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:36:16 +02:00
enricobuehler 396c3453f5 feat(apple/gamepad): rewrite rumble renderer — bounded divergence + iOS 27 plain-player fix
apple / swift (push) Successful in 1m8s
ci / rust (push) Successful in 1m59s
ci / web (push) Successful in 51s
android / android (push) Successful in 3m44s
ci / docs-site (push) Successful in 1m3s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m47s
release / apple (push) Successful in 8m38s
apple / screenshots (push) Successful in 5m27s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m26s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m16s
Ground-up RumbleRenderer rewrite around one principle: rumble is idempotent
state on a lossy channel, and the actuator's divergence from it must be
bounded, not best-effort. The old renderer rebuilt an infinite-duration
CHHapticAdvancedPatternPlayer per 0xCA datagram via an async stop; one stop
lost inside CoreHaptics left an unstoppable player buzzing forever (the
"entered the menu and rumble never stopped" bug).

- Finite 4 s segments, never infinite events — a leaked player self-silences;
  steady levels re-arm seamlessly ON the engine timeline (no stop/start race)
- GamepadFeedback drains the rumble plane DRY per cycle, newest-wins (was one
  datagram per 8 ms through a 16-deep drop-newest queue = lag + shed stops)
- Host 500 ms state refreshes dedupe to a liveness stamp; zero applies
  immediately; nonzero ramps throttle to one rebake/25 ms per motor
- Throwing player stop escalates to engine.stop() (kills leaked players);
  1.6 s staleness watchdog (Policy.session) force-silences on a dead channel;
  the test panel holds levels via Policy.manual
- Plain makePlayer, NEVER makeAdvancedPlayer: gamecontrollerd's controller
  haptics server advertises `adv players: 0`, and iOS 27 beta 2 hard-drops
  advanced loads with an XPC decode fault (-4811/4097, rumble silently dead).
  Live-verified on an iOS 27 beta 2 iPhone: DualSense rumble works
- Split-handle engines fall back to one combined .default engine on repeated
  failure; renderer publishes health transitions and the test panel shows
  them (a refused system service no longer reads as silent app breakage)
- Per-motor sharpness on split handles (0.3 heavy / 0.7 light); macOS
  DualSense raw-HID path gains a ~1 s keepalive re-write while nonzero
- RumbleTuningTests pin the scheduling math, tuning relations, and a
  queue/ticker teardown smoke test

Stuck-rumble streaming repro revalidation on glass still pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:06:45 +02:00
enricobuehler 6921e147dd ci(release): idempotent registry publish — survive re-tagged releases
apple / swift (push) Successful in 1m3s
ci / rust (push) Successful in 2m2s
ci / web (push) Successful in 56s
android / android (push) Successful in 3m22s
ci / docs-site (push) Successful in 58s
apple / screenshots (push) Successful in 5m38s
deb / build-publish (push) Successful in 3m12s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m41s
flatpak / build-publish (push) Successful in 4m8s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m12s
docker / deploy-docs (push) Successful in 19s
A moved release tag re-fires the publish workflows, and the Gitea
registries reject duplicate uploads with 409 (deb pool, rpm group, and
the generic packages' versioned URLs; the channel aliases already
pre-deleted). Delete any prior copy of the exact version before
uploading (404 on first publish tolerated), so a republished tag
overwrites instead of wedging — v0.5.0's retag left stale no-port-change
artifacts published and every re-run red.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:23:04 +00:00
enricobuehler 861da54066 feat(web,host/windows): move the web console off :3000 to :47992
apple / swift (push) Successful in 1m6s
apple / screenshots (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
android-screenshots / screenshots (push) Successful in 50s
android / android (push) Successful in 3m25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
windows-host / package (push) Successful in 6m28s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 52s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m3s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m5s
linux-client-screenshots / screenshots (push) Successful in 2m9s
release / apple (push) Successful in 9m25s
docker / deploy-docs (push) Successful in 20s
web-screenshots / screenshots (push) Successful in 2m33s
deb / build-publish (push) Successful in 3m19s
decky / build-publish (push) Successful in 19s
flatpak / build-publish (push) Successful in 5m9s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m38s
Port 3000 collides with half the dev-server ecosystem; 47992 sits next
to the mgmt API (47990) in the punktfunk port family. Updates the run
scripts, systemd/scheduled-task units, Dockerfile, Windows firewall
rule + installer, packaging, and every doc that referenced :3000.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 18:17:42 +00:00
enricobuehler 0c17343a50 fix(mgmt): version-agnostic OpenAPI drift test + regenerate the 0.5.0 snapshot
apple / swift (push) Successful in 1m11s
apple / screenshots (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
android-screenshots / screenshots (push) Successful in 50s
windows-host / package (push) Successful in 6m40s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m5s
android / android (push) Successful in 3m23s
decky / build-publish (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m7s
release / apple (push) Successful in 10m8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
deb / build-publish (push) Successful in 3m34s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
linux-client-screenshots / screenshots (push) Successful in 2m1s
flatpak / build-publish (push) Successful in 4m28s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m14s
web-screenshots / screenshots (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
The snapshot comparison now normalizes info.version on both sides and
compares structurally — a version bump alone can never invalidate the
checked-in spec again (the 0.5.0 release tripped on exactly this; the
API surface is what drift-control protects). Snapshot regenerated so
the docs-site copy shows the current version.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:53:33 +00:00
enricobuehler 38f8f18fe8 chore(release): 0.5.0
audit / cargo-audit (push) Successful in 17s
apple / swift (push) Successful in 1m8s
ci / rust (push) Failing after 1m49s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 59s
ci / bench (push) Successful in 5m0s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 56s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 50s
windows-host / package (push) Successful in 6m43s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m6s
release / apple (push) Successful in 9m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
apple / screenshots (push) Successful in 5m48s
android-screenshots / screenshots (push) Successful in 2m34s
decky / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
linux-client-screenshots / screenshots (push) Has been cancelled
android / android (push) Successful in 3m22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
web-screenshots / screenshots (push) Has been cancelled
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:33:01 +00:00
enricobuehler 9a58746aa5 fix(host/windows): clippy while_let_loop in the async poll drain
The rebase onto main picked up the pre-fix loop{match} variant of the
async retrieve drain — the exact shape the Windows clippy gate rejects
(run 6722 failed on it; the while-let form passed run 6724 on the CI
branch). Restore the gated form.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:31:45 +00:00
enricobuehler c21549c136 feat(host/windows,drivers): gamepad driver attach/heartbeat health surfaced in logs
apple / swift (push) Successful in 1m12s
windows-drivers / probe-and-proto (push) Successful in 14s
windows-drivers / driver-build (push) Successful in 1m15s
apple / screenshots (push) Successful in 5m30s
android / android (push) Successful in 3m35s
ci / web (push) Successful in 51s
ci / rust (push) Successful in 1m44s
ci / docs-site (push) Successful in 58s
deb / build-publish (push) Successful in 4m6s
ci / bench (push) Successful in 4m50s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7s
decky / build-publish (push) Successful in 13s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 35s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
windows-host / package (push) Failing after 2m28s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m40s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m40s
docker / deploy-docs (push) Successful in 5s
The gamepad drivers have no IOCTL plane (hidclass gates the stack), so
until now the host had ZERO visibility into whether a driver ever
bound: a pad could be "created" with no driver installed and nothing
was logged. Two health fields are carved from reserved shm space
(layout-compatible; pf-driver-proto pins the offsets): driver_proto —
stamped by pf-xusb at device add + per serviced XInput IOCTL (movement
= the game-visible path) and by pf-dualsense/DS4 from its ~125Hz timer
— and driver_heartbeat. Host-side, every pad owns a DriverAttach
watcher fed from the existing service() poll: INFO on attach (WARN on
proto mismatch), and after 3s of silence ONE diagnosis WARN combining
a cached pnputil /enum-drivers store check, the devnode's CM problem
code (CM_Locate_DevNodeW/CM_Get_DevNode_Status on the instance id now
captured from the create callback, with plain-language hints: 28 = not
installed, 52 = signature/Memory Integrity, …) and the driver's debug
log path. Also fixes a real bug both SwDeviceCreate wrappers shared:
the 10s WaitForSingleObject result was ignored and the callback
HRESULT zero-initialised, so a PnP timeout read as SUCCESS (now E_FAIL
init + explicit timeout error). Failure-mode table:
design/gamepad-driver-health.md.

Linux workspace green; Windows host + drivers CI-compile only, on-box
recipe at the bottom of the design doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 8af1a15aa6 feat(host,web): host log ring + GET /api/v1/logs + console Logs page
Remote debugging without shell access: a tracing layer tees every
event at DEBUG-and-up — independent of the RUST_LOG filter gating
stderr/host.log, so console-side debugging never needs a restart —
into a bounded in-memory ring (log_capture.rs, 4096 newest entries,
OnceLock singleton like config()), installed at both init sites
(stderr path in main, the Windows service file path). The mgmt API
serves it cursor-paged at GET /api/v1/logs?after=&limit= — bearer-only
and deliberately NOT on the mTLS cert allowlist (log lines can name
client identities and host paths). The web console grows a Logs page
(follow/pause · min-level filter · text search · eviction-gap badge);
polling self-paces: a non-empty page advances the after-cursor (new
query key → immediate refetch, drains backlogs), an empty page idles
at the 2s interval. OpenAPI regenerated; ring pagination/eviction,
layer wiring, and the authed route are unit-tested; Storybook story
included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 7ced80c4e3 feat(android): connected-controllers debug view (Settings → Host)
The client end of the "host doesn't see my gamepad" triage chain: a
new ControllersScreen lists every InputDevice Android classifies as a
gamepad/joystick (name, VID:PID, source classes, the punktfunk pad
type it resolves to, rumble test) plus an "Other input devices"
section — a pad behind a BT→USB adapter (the Pico 2W tester case)
often enumerates with the adapter's identity or not as a gamepad at
all, and this makes that visible on the device instead of over a bug
report. A live input test (button chips + axis bars + raw last-keycode
line) consumes pad events via new MainActivity probe hooks ahead of
the focus-nav remap; hold B 1.2s to exit since the pad can't reach the
toggle while captured. Gamepad grows pads()/isPad() (firstPad
generalized).

Kotlin compiles green (kit + app); on-device validation pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 1a483aae06 feat(host/windows): two-thread async NVENC retrieve (PUNKTFUNK_NVENC_ASYNC, opt-in)
The gpu-contention plan's §5.B lever: today submit and the blocking
lock_bitstream share one thread, so under a GPU-saturating game the
pipeline serializes on the WDDM scheduling wait (1000/17ms ≈ 59fps —
the depth-1 collapse; the old 'deeper pipeline just stacks latency'
result was a same-thread implementation, not a disproof). Async mode
opens the session enableEncodeAsync=1, registers an auto-reset
completion event per pool bitstream, and moves the wait+lock+copy+
unlock onto an internal retrieve thread feeding poll() through a
channel — the exact split the NVENC guide mandates. Register/map/unmap
stay on the encode thread; teardown drops the job channel, joins the
thread, THEN destroys the session. In-flight depth is bounded by
PUNKTFUNK_NVENC_ASYNC_DEPTH (default 4, hard cap POOL-1) — both for
output-buffer reuse and because NVENC encodes the capture ring's
textures in place. Idle latency cost ≈ 0 (same-tick pickup); under
contention completed frames queue instead of stalling capture.

CI-compile validated only — on-glass A/B under game load on the RTX
box still pending (box offline).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 49e6021ece feat(host,probe): controlled loss injection for the native path + probe keyframe-on-drop
PUNKTFUNK_VIDEO_DROP now also covers the native data plane (N% of
sealed wire packets discarded before send in paced_submit — the same
FEC-test knob the GameStream path has; no netem/root needed), and the
probe grows the real clients' recovery trigger: the data loop publishes
the session's unrecoverable-frame count and the control task sends
RequestKeyframe when it rises (100ms poll = natural coalescing).
Together these make the IDR-vs-intra-refresh recovery A/B runnable
against any host.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler fa4c798a25 feat(host/linux): amdgpu session clock pin — gpuclocks grows the AMD arm
nvclocks.rs -> gpuclocks.rs. PUNKTFUNK_PIN_CLOCKS=1 now also pins every
amdgpu card's power_dpm_force_performance_level to high for the host
lifetime (prior level restored on exit) — the measured AMD encode-
latency lever: VCN per-frame time doubles when a 60fps paced trickle
lets clocks sag (8 -> 4.4ms/frame at 1440p on the 780M with clocks
hot). Root-gated by sysfs ownership; non-root degrades to a logged
recipe (validated live on the AMD box). Opt-in stays deliberate:
box-wide power-management override, wrong on battery/Deck.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler fd1086074b feat(host/vaapi): submit-split instrumentation + async_depth knob (depth 1 stays default)
Chasing the 8ms submit at 1440p on the 780M: the sampled PUNKTFUNK_PERF
split (push/pull/send) shows desc+buffersrc at ~5us, hwmap-import+VPP
CSC at ~0.2-0.5ms, and avcodec_send_frame owning the rest — so neither
a VA-surface import cache nor CSC overlap would help. Two facts landed:
(1) async_depth>=2 in libavcodec's vaapi_encode is a structural
+1-frame latency (frame N's packet only materializes when N+1 queues;
measured 18ms vs 8.3ms p50 at depth 1) — depth 1 stays the default,
PUNKTFUNK_VAAPI_ASYNC_DEPTH exists for pixel rates beyond the ASIC's
serial budget, and poll() now does a bounded in-flight wait so a deeper
depth still ships the AU as soon as the ASIC finishes. (2) The residual
send_frame block tracks GPU CLOCKS, not the ASIC: ~8ms/frame at a 60fps
duty cycle vs ~4.4ms at 120fps pacing vs 3.5ms back-to-back (270fps CLI
benchmark, even at -async_depth 1) — the clock-sag fix lands in
gpuclocks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 12a3944156 feat(host/linux): default the tiled zero-copy path to GPU NV12 (NVENC fed native YUV)
A/B'd on the Bazzite box (RTX 5070 Ti, KWin 6.6, driver 595, 1080p60
over the LAN): pixel-correct decode (full desktop, no tint/banding),
latency-neutral idle (p50 1.47ms RGB vs 1.52ms NV12, both 2400/2400
frames), CPU-neutral — and it deletes NVENC's internal RGB->YUV CSC
from the SM/3D engine a game saturates (video 40%+SM 15% -> video
26%+SM 2% measured on Windows). Matches the Windows host default.
PUNKTFUNK_NV12=0 restores the RGB feed; LINEAR/gamescope captures are
unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 73f14bc725 feat(host/linux): NVIDIA clock hygiene — P2-cap driver profile + opt-in NVML clock floor
Two halves of the easy-scene p99 lever (host-latency plan Tier 1B):
CudaNoStablePerfLimit application profile (no root; NVIDIA's supported
opt-out of the CUDA/NVENC P2 memory-clock clamp, raw key 0x166c5e=0 per
open-gpu-kernel-modules#333, shipped for obs/Discord in R595) installed
into ~/.nv/nvidia-application-profiles-rc.d/ keyed on procname, opt-out
PUNKTFUNK_NV_PROFILE=0; and PUNKTFUNK_PIN_CLOCKS=1 arming an NVML
SetGpuLockedClocks(TDP, UNLIMITED) core-clock floor (base floor, boost
headroom — never a max pin) held for the host lifetime, reset-on-start
self-healing a crashed run's stale pin, NO_PERMISSION degrading to a
logged sudoers/oneshot recipe. libnvidia-ml is dlopen'd like libcuda —
no link-time dependency, clean no-op off NVIDIA.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 21eded8d88 feat(host): intra-refresh loss recovery + delivery-anchored latency instrumentation
Intra-refresh (opt-in PUNKTFUNK_INTRA_REFRESH=1 until on-glass
validated): NVENC runs a moving intra band + recovery-point SEI
(gop_size becomes the wave period, ffmpeg forces the real GOP infinite;
default fps/2, PUNKTFUNK_IR_PERIOD_FRAMES overrides; ENOSYS latches a
fallback to IDR-only). Clients request a keyframe on every
FEC-unrecoverable frame, so under intra-refresh the session glue serves
the first request instantly and suppresses the rest for a 2s window —
the wave heals loss without the 20-40x IDR spike cascade. VAAPI/software
keep IDR recovery.

Instrumentation: the wire pts now anchors at the PipeWire delivery stamp
(client-measured latency covers delivery + queue age, not just
submit->glass; repeats/synthetic stamps fall back to now), encode_us
keeps its submit->AU meaning via a separate inflight stamp, and a new
'queue' stage (delivery->submit age of fresh frames) rides
PUNKTFUNK_PERF and the web-console stats samples.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 315eb6ef7c feat(host/gamestream): priority-boost the video + send threads like the native path
The GameStream video thread ran unboosted on Linux and the send thread
only got the Windows MMCSS call; both now use boost_thread_priority
(Linux nice -10/-5, Windows HIGHEST/ABOVE_NORMAL + session tuning).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler a333d5a15b feat(host/capture): zero-copy by default on VAAPI hosts (dmabuf passthrough)
PUNKTFUNK_ZEROCOPY unset now defaults ON when the encode backend is
VAAPI — a stock AMD/Intel install gets the LINEAR-dmabuf -> GPU-CSC path
instead of three full-frame CPU touches (measured on the 780M at 1440p:
0.8s vs 7.9s CPU per 600 frames, pixel-identical). NVENC stays opt-in.
A dmabuf offer the compositor never accepts latches a one-shot downgrade
so the pipeline rebuild renegotiates on the CPU offer; explicit =1 keeps
erroring loudly. The EGL->CUDA importer is no longer built on VAAPI
backends (an NVIDIA box forced to PUNKTFUNK_ENCODER=vaapi now correctly
takes the passthrough instead of producing CUDA frames the encoder
rejects), and a VAAPI session landing on the CPU path warns with the
reason.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler 34bdda7d96 feat(host/vaapi): fall back to the low-power (VDEnc) entrypoint — unblocks modern Intel
Gen12+/Arc iHD exposes ONLY EncSliceLP, so the default open fails with
'no usable encoding entrypoint'. Try full-feature first (AMD unchanged,
validated on the 780M), retry low_power=1, cache the mode per codec;
PUNKTFUNK_VAAPI_LOW_POWER pins it. Probes inherit the ladder. Docs note
the Intel HuC firmware requirement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:33:56 +00:00
enricobuehler fbeac16c96 feat(clients/windows): WinUI UX batch - tile hover, Settings NavigationView, modal slide-up
audit / cargo-audit (push) Successful in 1m13s
apple / swift (push) Successful in 1m14s
release / apple (push) Successful in 8m2s
android / android (push) Successful in 10m42s
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 58s
ci / rust (push) Successful in 12m23s
apple / screenshots (push) Successful in 5m27s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m43s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m21s
ci / bench (push) Successful in 4m49s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m10s
deb / build-publish (push) Successful in 4m0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
decky / build-publish (push) Successful in 26s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m20s
windows-host / package (push) Failing after 23s
flatpak / build-publish (push) Successful in 4m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m42s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m16s
docker / deploy-docs (push) Successful in 34s
Bump windows-reactor + windows to a4f7b2cb (from b4129fcc) for the new
PointerEntered/PointerExited events; migration is mechanical renames only
(SymbolGlyph->Symbol, placeholder->placeholder_text, on_changed->
on_text_changed/on_toggled, on_menu_item_clicked->on_item_clicked,
on_ready->on_mounted). New runtime model: reactor lost its build.rs, so the
client build.rs stages the WinAppSDK bootstrap via
windows-reactor-setup::as_framework_dependent() and main calls
windows_reactor::bootstrap() (missing either = 0x80040154 at launch);
staged filenames unchanged, so pack-msix and the MSIX manifest are untouched.

- Host tiles: WinUI pointer-over fill (ControlFillSecondary) via the new
  pointer enter/exit events, hover id in root state (backend-wired handlers
  bypass the reconciler flush, like the flyout clicks).
- Settings: stock NavigationView sidebar (Windows-Settings pattern) with
  Display/Video/Input/Audio/About panes, built-in back arrow, wide content
  column, and a per-section content slide-up tween. The section card is
  KEYED by section: an in-place diff across sections re-sets a reused
  ComboBox's items (clearing WinUI's selection) but skips selected_index
  when the values compare equal, rendering a blank selection - the key
  forces a remount. Card titles/descriptions dropped; per-control guidance
  moved to hover tooltips (ToolTipService).
- New "Show the stats overlay (HUD)" setting (show_hud, default on),
  honored mid-stream via the 400 ms HUD re-render.
- Add-host modal: entrance fade + slide-up tween (scrim fades with it).
- Self-initiated disconnect (Ctrl+Alt+Shift+D -> Ended(None)) returns to
  the host list silently instead of raising the error banner.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 18:23:25 +02:00
enricobuehler bf799b41e3 feat(clients/windows): GPU picker, disconnect shortcut, richer stream HUD
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m16s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 59s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m6s
apple / swift (push) Successful in 1m11s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m16s
apple / screenshots (push) Successful in 5m30s
android / android (push) Successful in 3m21s
ci / web (push) Successful in 52s
ci / rust (push) Successful in 1m26s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 3m19s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m36s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m50s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m45s
docker / deploy-docs (push) Successful in 17s
- Settings gains a GPU selector (shown only on multi-GPU boxes): the picked
  DXGI adapter drives decode + present, persisted as Settings.adapter and
  applied at the next stream - gpu.rs now caches the shared device keyed by
  the resolved preference (env PUNKTFUNK_ADAPTER > Settings > the window's
  monitor's adapter) so a change needs no app restart.
- Ctrl+Alt+Shift+D disconnects the session (consumed locally, captured or
  released): the hook releases capture and trips the session stop flag,
  plumbed through the stream-page handoff; the pump winds down and the UI
  navigates back to the host list.
- Stream HUD extended: codec chip (HEVC/H.264/AV1), display-side line from
  the render thread (presents/s + capture-to-decoded vs capture-to-on-glass
  p50), session line (host name, duration, network-lost frames, skipped
  backlog frames), and both shortcut hints incl. the new disconnect.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:41:16 +02:00
enricobuehler 5ef63756ea fix(host/linux,clients/android): honor the host/device keyboard layout in keymaps
apple / swift (push) Successful in 1m5s
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 56s
ci / rust (push) Successful in 7m4s
audit / cargo-audit (push) Successful in 1m19s
android / android (push) Successful in 4m16s
windows-host / package (push) Successful in 7m53s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m22s
release / apple (push) Successful in 8m22s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m20s
ci / bench (push) Successful in 4m43s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 3m21s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m9s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m11s
apple / screenshots (push) Successful in 5m34s
flatpak / build-publish (push) Successful in 4m33s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m33s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m12s
wlroots injector: the virtual keyboard keymap now defers to the standard
XKB_DEFAULT_RULES/MODEL/LAYOUT/VARIANT/OPTIONS env vars (libxkbcommon
built-ins as fallback) instead of hardcoding evdev/pc105/us, matching the
libei path where the session compositor's own keymap applies. Android:
Keymap gains the same positional-key coverage for non-US layouts (+ tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:25:07 +02:00
enricobuehler a4c84ac620 feat(clients/windows): all-vendor video pipeline rewrite + app icon + hosts-page tiles
Decode+present rewrite (first real pixels on glass for this client):

- Decode: FFmpeg D3D11VA on NVIDIA/AMD/Intel. get_format now only returns
  AV_PIX_FMT_D3D11 and lets libavcodec build the decode pool from
  hw_device_ctx (hand-built frames contexts failed three different ways:
  NVIDIA rejects DECODER|SHADER_RESOURCE arrays, BindFlags=0 fails texture
  creation, Intel rejects non-128-aligned HEVC surfaces at the first
  SubmitDecoderBuffers). A DXVA profile probe before the hwdevice commits
  hardware-vs-software up front instead of burning the opening IDR;
  extra_hw_frames covers the frames the client holds.
- Present: the decoded slice is copied with ONE display-size-boxed
  CopySubresourceRegion (a planar slice is a single subresource in D3D11;
  the old two-copy D3D12-style code silently no-opped - the black screen)
  into a sampleable NV12/P010 texture, per-plane SRVs + YUV->RGB shaders.
- New dedicated render thread (render.rs): presenting is decoupled from the
  XAML thread; frame-latency-waitable swapchain + SetMaximumFrameLatency(1),
  newest-wins drain after the wait, crossbeam frame channel with pts for a
  capture->presented p50 log.
- HiDPI: pixel-sized buffers + SetMatrixTransform(96/dpi) - was blurry at
  125/150 % scaling.
- Software fallback now feeds the same shaders (swscale -> NV12/P010 planes
  -> two dynamic plane textures); ps_rgba/X2BGR10 path deleted, hw/sw colour
  math identical.
- Adapter selection for hybrid boxes: PUNKTFUNK_ADAPTER > the window's
  monitor's adapter > default; PUNKTFUNK_D3D_DEBUG=1 debug layer.
- Session pump: request_keyframe at start and on hw->sw demotion (infinite
  GOP would otherwise sit on a black screen).

Validated live on the Arc Pro + RTX 3500 Ada laptop against the local
Windows host: 60 fps D3D11VA on both vendors, software path, GUI on glass.

Also: embedded app icon (build.rs winresource + WM_SETICON, MSIX
Square44x44 targetsize assets, pack-msix stages them) and the hosts-page
tile rework (tap-to-connect tiles with sibling overflow menu - fixes
forget-also-connects - in-tile rename editor, add-host modal via root state).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:24:23 +02:00
enricobuehler 2c416a4bff fix(host/windows): layout-correct keyboard injection - semantic vs positional VKs
First-party punktfunk clients send US-positional VKs (the physical key's
US-layout VK), GameStream/Moonlight clients send layout-semantic VKs
(Sunshine's model). The SendInput injector previously resolved everything
through the SYSTEM service's layout - on a German host that is the y/z swap
and u-umlaut-on-o-umlaut scramble. GameStream ingest now tags its key events
KEY_FLAG_SEMANTIC_VK (stripped from punktfunk/1 wire events so a network
client can't flip the convention); the injector maps semantic VKs under the
foreground app's layout and positional VKs through a fixed scancode table.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:24:04 +02:00
enricobuehler 019f2677a7 feat(host,web): multi-GPU selection — GPU inventory + preference API, web-console GPU card
apple / swift (push) Successful in 1m9s
ci / rust (push) Successful in 1m50s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 57s
decky / build-publish (push) Successful in 11s
android / android (push) Successful in 3m13s
apple / screenshots (push) Successful in 5m32s
deb / build-publish (push) Successful in 3m15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
windows-host / package (push) Successful in 7m35s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 31s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m53s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m58s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m54s
docker / deploy-docs (push) Successful in 18s
- new crate::gpu (compiled on all platforms so the OpenAPI doc stays
  platform-independent): DXGI / sysfs GPU inventory with reboot-stable ids
  (PCI vendor:device + occurrence — LUIDs are per-boot), persisted auto/manual
  preference (<config>/gpu-settings.json, atomic temp+rename with in-memory
  rollback), one selection with precedence console preference >
  PUNKTFUNK_RENDER_ADAPTER > max VRAM and graceful fallback when the preferred
  GPU is absent, plus a live "in use" record (RAII session guard wrapped around
  every encoder open_video returns)
- fix: windows_gpu_vendor derived the encoder backend from DXGI adapter 0
  instead of the selected render adapter — on a hybrid box (e.g. Intel iGPU at
  index 0 + NVIDIA dGPU) the backend could disagree with the GPU the capture
  ring / IddCx render pin sit on. The NVENC 4:4:4 probe now also runs on the
  selected adapter (was: OS default), the codec/4:4:4 probe caches are keyed
  per selected GPU (were process-lifetime OnceLocks), and an explicit
  PUNKTFUNK_ENCODER conflicting with the selected GPU's vendor warns up front
- mgmt API: GET /api/v1/gpus (inventory + mode + preferred + next-session
  selection with reason + in-use GPU/backend/session-count) and
  PUT /api/v1/gpus/preference (validates mode/gpu_id before writing);
  openapi.json regenerated; the vdisplay render pin now also engages for a
  console preference (not just the env pin)
- web console: GPU card on the Host page — list with vendor + VRAM,
  Automatic / Prefer controls, Preferred / Next session / "In use · backend"
  badges, missing-preferred-GPU warning and env-pin note; en + de messages
- Linux: a matched manual preference picks the VAAPI render node and the
  NVENC-vs-VAAPI auto choice; auto mode is exactly the previous behavior

Validated live on the hybrid laptop (RTX 3500 Ada + Intel Arc Pro, which
enumerates twice — the occurrence ids disambiguate): enumerate, prefer,
bad-id 400, restart persistence, auto-restore keeping the stored pick.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 13:57:26 +02:00
enricobuehler 40fefd73ca feat(nav): add Ko-fi Support link to docs nav
apple / swift (push) Successful in 1m8s
ci / rust (push) Successful in 1m21s
ci / web (push) Successful in 51s
android / android (push) Successful in 3m41s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m31s
deb / build-publish (push) Successful in 3m14s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
ci / bench (push) Successful in 4m44s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m54s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m57s
docker / deploy-docs (push) Successful in 18s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 11:00:46 +00:00
enricobuehler b5fc017b19 ci(audit): ignore quick-xml build-time DoS advisories (RUSTSEC-2026-0194/0195)
quick-xml 0.39.4 enters only via wayland-scanner, a build-time proc-macro
that parses trusted crate-shipped protocol XML at compile time — never a
shipped binary, never runtime/attacker-controlled input, so neither DoS is
reachable. wayland-scanner 0.31.10 (latest) pins quick-xml ^0.39; the fixes
land only in >=0.41, so there is nothing to bump to.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 10:18:37 +00:00
enricobuehler f48dc5dfce feat(host/windows,packaging): installer overhaul - branding, VB-CABLE, GameStream choice, driver uninstall
ci / docs-site (push) Successful in 1m3s
android / android (push) Successful in 3m34s
decky / build-publish (push) Successful in 11s
apple / swift (push) Successful in 1m7s
ci / rust (push) Successful in 1m36s
ci / web (push) Successful in 49s
apple / screenshots (push) Successful in 5m20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
windows-host / package (push) Successful in 6m41s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m17s
ci / bench (push) Successful in 4m41s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m22s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m37s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m8s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m13s
docker / deploy-docs (push) Successful in 16s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m0s
deb / build-publish (push) Successful in 3m6s
- Modern branded wizard: WizardStyle=modern dynamic windows11 (Inno >= 6.6,
  plain-modern fallback for older compilers; CI provisioning upgrades a
  pre-6.6 Inno). Brand-mark wizard side panels + header tiles (100-200% DPI)
  and a multi-size punktfunk.ico (SetupIconFile + Apps & Features), generated
  AND committed by branding/gen-branding.ps1 from the canonical brand geometry.
  Gotcha encoded in the script: ISCC rejects all-PNG icons, so entries <= 64px
  are classic DIBs (PNG only at 128/256), and the ICO is load-verified.

- VB-CABLE actually ships now: windows-host.yml never set VBCABLE_DIR, so every
  published installer silently omitted the virtual mic (broken mic passthrough
  in the field). CI provisions the pinned, SHA-256-verified official Pack45
  (provision-windows-punktfunk-extras.ps1) and the pack now FAILS on a
  supplied-but-invalid dir instead of shipping mic-less again. Attribution per
  VB-Audio's bundling grant surfaced in the visible wizard task text (vendor,
  vb-cable.com, donationware) on top of the licenses notice.

- GameStream (Moonlight) compat is a wizard task (checked by default) ->
  service install --gamestream=on|off writes PUNKTFUNK_HOST_CMD=
  serve[ --gamestream] into host.env. Only the two canonical values are ever
  rewritten - a hand-customized command line survives upgrades. Silent
  installs: /MERGETASKS="!gamestream".

- Driver uninstall (field report: our virtual-device drivers survived
  uninstall): new `driver uninstall [--gamepad]` removes the pf-vdisplay
  device node(s) + the pf-vdisplay/pf-dualsense/pf-xusb driver-store packages,
  wired into [UninstallRun] after service uninstall. Locale-safe by
  construction: devices matched on unlocalized VALUES (never pnputil's
  localized labels), packages found by INF content scan - validated against a
  German-locale box ("Instanz-ID:" parse; 7/7 punktfunk INFs matched, no
  foreign hits). VB-CABLE is deliberately left installed (shared third-party
  component with its own uninstaller).

Installer compile, cargo check/clippy/fmt, and the ASCII locale gate are green;
the wizard look + uninstall flow still need one on-glass pass on a disposable
box (this box runs the live host).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 12:16:19 +02:00
enricobuehler 9074781acd feat(clients/windows): screen-module restructure + parity features (speed test, native mode, capture UX)
Structure: split the 1400-line app.rs into per-screen app/ modules (mod=root/
router, hosts, connect, pair, speed, settings, licenses, stream, style) with
shared card/header/busy-page builders and setting_combo/toggle helpers; the
re-render rule (thread-driven state lives in root use_async_state, flows down
as props) is now documented at the module root.

Parity features the other clients already had:
- "Native display" resolves the real monitor mode at connect
  (MonitorFromWindow -> EnumDisplaySettingsW; was a hardcoded 1080p60)
- per-host network speed test: saved-host card button + a results screen
  (probe burst -> goodput/loss -> ~70% recommended bitrate applied in one
  tap; stale runs invalidated by generation) and `--headless --speed-test`;
  the bitrate setting becomes a free-form NumberBox so the recommendation
  round-trips
- forget host (ContentDialog confirm -> KnownHosts::remove_by_fp)
- settings: forwarded-controller picker (pads/pinned/set_pinned now wired),
  gamepad type, host compositor, capture-system-shortcuts; the previously
  dead Settings.compositor / inhibit_shortcuts are honored (shortcuts off =
  Alt+Tab/Alt+Esc/Ctrl+Esc/Win act locally)
- click-to-recapture after a Ctrl+Alt+Shift+Q release; the HUD hint tracks
  the live capture state

Perf: the input hook caches lock geometry (clip rect + contain-fit scale) at
engage instead of GetClientRect per WM_MOUSEMOVE; the audio jitter ring trims
via drain() and reuses the render scratch buffer.

Validated on the bare-metal box: --discover, synthetic-host loopback E2E
(TOFU -> clock skew -> HEVC negotiate -> D3D11VA init -> session end),
speed-test E2E, and the WinUI shell rendering in the console session via
PsExec (SSH/session-0 cannot create windows, pre-existing 0x80070005).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 12:16:19 +02:00
enricobuehler cac5b31535 docs: status updates — apple gamepad UI v2 (+ macOS), android AAudio wording
apple / swift (push) Successful in 1m4s
audit / cargo-audit (push) Failing after 1m4s
android / android (push) Successful in 5m18s
windows-host / package (push) Successful in 6m24s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m7s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m8s
release / apple (push) Successful in 8m0s
ci / rust (push) Successful in 9m37s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 57s
decky / build-publish (push) Successful in 12s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 57s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 50s
ci / bench (push) Successful in 4m32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
deb / build-publish (push) Successful in 3m14s
apple / screenshots (push) Successful in 5m37s
flatpak / build-publish (push) Successful in 4m4s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m31s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m58s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:24:44 +02:00
enricobuehler 133e25849d feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS
Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit:
Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split
along the same seams.

The gamepad mode is couch-complete, and now on macOS too (the living-room
Mac case), not just iOS/iPadOS:

- GamepadSettingsView: a console-style, fully controller-navigable settings
  screen (X from the launcher) — up/down moves focus, left/right steps values
  (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a
  one-line description. Backed by GamepadMenuList, the vertical sibling of
  GamepadCarousel, and SettingsOptions — the option lists hoisted out of
  SettingsView statics and shared by the touch, tvOS and gamepad settings.
- GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad
  — field rows open an on-screen controller keyboard (dpad grid, A types,
  X backspaces, B done); the launcher carousel ends in an Add Host tile, so
  the dead-end "add one with touch first" empty state is gone.
- Launcher polish: contextual hint bar with the pad's real button glyphs,
  controller name + battery chip, one shared console chrome.
- GamepadScreenBackground: an animated aurora (TimelineView-driven drifting
  blobs in the brand's violet family, breathing radii, slow hue shift,
  legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a
  .metal library only bundles reliably in one of the two build systems (SPM vs
  the xcodeproj's synced folders) these sources compile under.
- macOS port: settings/add-host/library present as sized sheets (a macOS sheet
  takes its content's IDEAL size, and the GeometryReader-driven screens
  collapsed to nothing), NSScreen-based mode lists, scroll indicators .never
  (the "always show scroll bars" setting overrides .hidden), tray scrims so
  scrolled rows dim under the pinned title/hints, extra title clearance, and a
  PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/
  library render-verified live on a real Mac + LAN hosts.
- GamepadMenuInput: X button support, and (re)start now snapshots held buttons
  so a controller handoff press never fires twice (the B that closed the
  keyboard no longer also cancels the screen underneath).
- Cleanups: one "Connection failed" alert in ContentView instead of one per
  home screen; HostDiscovery.advertises/unsaved shared by both home screens.
- host: can_encode_444 stub for the non-Linux/Windows host build (the macOS
  synthetic-source loopback used by the Swift tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:24:44 +02:00
enricobuehler e925d00194 feat(linux): game library browser; split app.rs into cli/launch/ui_trust
- library.rs + ui_library.rs: the host's unified game library over the
  management API (the Apple LibraryClient/LibraryView ported) — mTLS with the
  paired identity, host verified by its pinned cert fingerprint (ureq + rustls,
  unified with the workspace rustls 0.23); posters load async with monogram
  placeholders, and picking a title starts a session that asks the host to
  launch it (the library id rides the Hello).
- app.rs (~800 lines lighter) splits into cli.rs (argv/headless
  pairing/--connect/screenshot scenes), launch.rs (mode resolve + session
  worker + event stream into the UI) and ui_trust.rs (TOFU / SPAKE2 PIN /
  delegated-approval dialogs); ui_hosts/ui_stream reworked around the split.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:24:44 +02:00
enricobuehler bd4e15b68d refactor(android): split session JNI into modules, HUD-gated stats, AAudio open retry
- native: the 756-line session.rs becomes session/{mod,connect,input,planes}.rs
  around a SessionHandle (connect lifecycle + trust, input plane shims, plane
  start/stop + stats drain).
- Decode-stats sampling is HUD-gated (nativeSetVideoStatsEnabled): with the
  overlay hidden the decode thread skips the per-AU clock read + lock; enabling
  resets the measurement window.
- audio: the AAudio open path is a per-sharing-mode try_open closure — the
  realtime callback state (ring, prime, free-list) is rebuilt per attempt, so a
  failed exclusive-mode try can't leak state into the shared-mode retry.
- Kotlin: ConnectScreen/StreamScreen slimmed by extracting ConnectDialogs,
  StatsOverlay and TouchInput.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:08:56 +02:00
enricobuehler 3678c182d5 feat(clients): codec preference on Windows/Apple/Android clients (Phase 2b)
apple / swift (push) Failing after 52s
windows-drivers / probe-and-proto (push) Successful in 50s
windows-drivers / driver-build (push) Successful in 1m20s
android / android (push) Failing after 2m55s
ci / web (push) Successful in 1m5s
release / apple (push) Successful in 3m38s
apple / screenshots (push) Has been skipped
ci / rust (push) Successful in 4m47s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 2m49s
decky / build-publish (push) Successful in 21s
windows-host / package (push) Successful in 7m35s
ci / bench (push) Successful in 5m10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m41s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m17s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m22s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m59s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 52s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m7s
flatpak / build-publish (push) Successful in 4m20s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m58s
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m45s
Rounds out codec negotiation across the last three clients — each advertises what it can decode,
builds its decoder from the resolved Welcome.codec, and exposes a "Video codec" preference picker.

**Windows** (Rust, mirrors Linux): `decodable_codecs()` + `ffmpeg_codec_id()`; the D3D11VA and
software FFmpeg decoders (and the mid-session D3D11VA→software demotion) open the negotiated codec
instead of hardcoding HEVC; settings gain a `codec` field + reactor ComboBox; `--codec` CLI flag.

**Apple** (Swift/C-ABI): AnnexB is now codec-aware — a `VideoCodec` enum drives H.264 vs HEVC NAL
parsing / parameter-set extraction (`CMVideoFormatDescriptionCreateFromH264ParameterSets` for H.264,
no VPS) and AVCC repacking; `PunktfunkConnection` advertises H264|HEVC via `punktfunk_connect_ex7`,
reads `resolvedCodec` (`punktfunk_connection_codec`), and threads `videoCodec` into the stage-1/2
pipelines + `VideoDecoder`; SettingsView "Video codec" Picker (auto/HEVC/H.264). AV1 is left out
(hosts don't emit it on the native path, and it's not an AnnexB codec). Test call sites updated.

**Android** (Kotlin + Rust JNI): the JNI `nativeConnect` gains `preferredCodec`; the native decode
loop picks the AMediaCodec MIME (`video/hevc`|`video/avc`) from `connector.codec` and advertises
H264|HEVC; Settings `codec` field + Compose dropdown.

Core/host/probe/Linux clippy + tests green (unchanged from 2a). Windows/Apple/Android compile on
their platform CI (this Linux box can't build them — Windows toolchain / Xcode / the Android NDK's
opus-cmake toolchain). All follow the Linux client's validated pattern.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 00:29:38 +00:00
enricobuehler 12843fe253 feat(protocol,clients): codec preference negotiation + Linux client decodes per Welcome (Phase 2a)
Adds a client-selectable **preferred codec** and wires the core + ABI + probe + Linux client to
negotiate and decode it. (Windows/Apple/Android follow in 2b.)

**Core:**
- `Hello.preferred_codec` (a single CODEC_* bit, 0 = auto) — a soft hint appended after
  `video_codecs`. `resolve_codec(client, host, preferred)` now honors the preference when the host
  can also emit it, else falls back to precedence (HEVC > AV1 > H.264). Roundtrip + preference tests.
- `NativeClient::connect` takes `video_codecs` + `preferred_codec`; `NativeClient.codec` exposes the
  resolved `Welcome.codec`.
- ABI: `punktfunk_connect_ex7` (adds the two codec params; `ex6` delegates to it advertising
  HEVC-only) + `punktfunk_connection_codec` getter + `PUNKTFUNK_CODEC_{H264,HEVC,AV1}` constants
  (drift-guarded against the wire values). Header regenerated.

**Host:** passes `hello.preferred_codec` into `resolve_codec`.

**probe:** `--codec h264|hevc|av1|auto` sets the preference (still advertises it can decode all
three); the dump extension already follows the resolved codec.

**Linux client:** advertises the codecs FFmpeg can actually decode (`decodable_codecs()`), threads
the user's `codec` setting as the preference, and builds the decoder — both the software and VAAPI
paths, plus the mid-session VAAPI→software demotion — from the negotiated `Welcome.codec` instead of
hardcoding HEVC. New "Video codec" dropdown in Preferences (Automatic/HEVC/H.264/AV1).

Live-validated on the dev box: probe `--codec hevc` against a software (H.264-only) host resolves to
H.264 (graceful soft-preference fallback), no failure. clippy + core (57) + host (133) tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 00:13:26 +00:00
enricobuehler ffc0b07b46 feat(protocol,host): negotiate video codec + add a GPU-less software (openh264) encode path
Phase 1 of codec negotiation, and the Linux software H.264 encode path it unblocks.

**Codec negotiation (core `quic`):**
- `Hello.video_codecs` (bitfield: CODEC_H264/HEVC/AV1) — the client advertises what it can
  decode; appended as a trailing byte (older client → 0 = HEVC-only, back-compat).
- `Welcome.codec` — the single codec the host resolved and will emit; trailing byte (older
  host → HEVC).
- `resolve_codec(client, host_capable)` picks the shared codec (precedence HEVC > AV1 > H.264)
  or `None` → the host refuses honestly rather than sending an undecodable stream.
- Roundtrip + back-compat tests; cbindgen exports the CODEC_* constants.

**Software encoder (host):**
- The openh264 `OpenH264Encoder` (was Windows-only) is now built on Linux too — it's
  platform-agnostic (consumes CPU RGB `CapturedFrame`s, statically-bundled openh264). `openh264`
  moved to the shared linux+windows Cargo target.
- `PUNKTFUNK_ENCODER=software` selects it: `open_video` gains a `software` branch (H.264 only),
  and `session_plan::resolve_encoder` / `capture::gpu_encode` resolve `EncoderBackend::Software`
  → `output_format().gpu = false`, so the portal capturer delivers CPU RGB. Explicit-only (auto
  never picks it — a box with a dead driver still has /dev/nvidiactl and would mis-resolve NVENC).

**Host codec resolution (`punktfunk1`):**
- The native path no longer hardcodes HEVC: it resolves the codec from the client's advertised
  set ∩ the host's capability (`Codec::host_wire_caps`: software→H.264, else HEVC), threads it
  through `SessionPlan.codec`, and opens the encoder + validates reconfigures at that codec. A
  software host + HEVC-only client is refused with a clear error.
- 4:4:4 is gated on HEVC (it's HEVC-only).

**Probe:** advertises H264|HEVC|AV1 and logs the resolved codec.

Validated on the GPU-less dev box: negotiation is live end-to-end (probe advertises 0x07 → host
resolves H.264 → Welcome reports it → plan = Software/H264), and the openh264 unit test (CPU RGB →
AnnexB IDR) now runs on Linux. Full capture→encode still needs a GPU on this box — every
compositor screencast path (KWin GL, gamescope VK_EXT_physical_device_drm, wlroots EGL) requires
one; software render (llvmpipe/pixman) can't be captured — so this box exercises negotiation +
encoder, not live capture. The software path unblocks GPU-less-*encode* boxes that still have a
display GPU. Phase 2 (clients advertising real codecs + decoding per Welcome.codec) is a follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 23:13:39 +00:00
enricobuehler e7b07d2363 fix(host): make client game launches work on every Linux compositor
Client-initiated launches (Hello.launch / GameStream applist) were only
wired to gamescope's bare-spawn path via the process-global
PUNKTFUNK_GAMESCOPE_APP env — which leaked across sessions, was never
read by kwin/mutter/wlroots (launch was a silent no-op there), and was
unreachable on gamescope anyway because apply_input_env unconditionally
defaulted to the managed session (which bails on non-Bazzite/SteamOS
boxes and ignores the launch command in all its modes).

- Thread the launch per-session: resolve the library id at handshake,
  carry it on SessionContext (Windows: id; else: resolved command), and
  hand it to the backend instance via set_launch_command — the global
  env write is gone (the env stays as an operator fallback in spawn).
- Gamescope sub-mode ladder (pick_gamescope_mode, pure + unit-tested):
  managed only when session-plus/SteamOS infra exists, attach for an
  explicit request or a foreign (non-host-descendant) gamescope, else
  bare spawn — which nests the launch and is now reachable on plain
  distros instead of the guaranteed managed-mode bail.
- launch_session_command: one launch entry point for both planes once
  capture is live — desktop compositors plain-spawn into the retargeted
  session (the virtual output is primary); managed/attached gamescope
  spawns with the live session's DISPLAY/GAMESCOPE_WAYLAND_DISPLAY
  discovered from /proc (steam:// URIs also forward over Steam's own
  pipe). launch_is_nested gates bare spawn against double-launching.
- GameStream unified onto the same dispatch; also nests library-id
  picks into gamescope (previously only apps.json cmd was nested).

Validated live on the dev box up to the missing-GPU wall: handshake
resolution, Spawn sub-mode on plain Ubuntu, gamescope spawned with the
command nested. On-glass validation (kwin spawn on the streamed output,
Bazzite/Deck managed forward) pending GPU reattach.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-01 22:02:52 +00:00
enricobuehler 7c976bc8c3 fix(host/audio): make the Linux virtual mic the default source (was silent)
apple / swift (push) Successful in 1m9s
android / android (push) Successful in 4m7s
ci / rust (push) Successful in 4m42s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 58s
apple / screenshots (push) Successful in 5m18s
windows-host / package (push) Successful in 6m42s
deb / build-publish (push) Successful in 2m47s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
ci / bench (push) Successful in 4m46s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m40s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m6s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m52s
The punktfunk/1 virtual microphone was created as a plain Audio/Source with
no session priority, which caused two failures — both diagnosed live against a
Bazzite host on PipeWire 1.4.10:

1. It was never WirePlumber's default source, so any app recording the *default*
   input (games, Discord, arecord) heard silence. This is the Linux analogue of
   the Windows host forcing the default recording endpoint (audio_control.rs).

2. The real killer on PipeWire 1.4.x: a *non-default* Audio/Source recorded via
   `--target` never gets a driver assigned — the {source, recorder} group stays
   orphaned (pw-top QUANT/RATE 0, driver-node None), so the RT process() callback
   never fires and even an explicitly-selected mic is pure silence. PipeWire 1.6
   drives any recorded source regardless, which is why the host worked on a 1.6
   box but not the 1.4.10 Bazzite host.

Fix: advertise a high priority.session on the source so WirePlumber elects it as
the default source and keeps it driven. Reproduced with a faithful standalone
copy of the node on the same 1.4.10 daemon: no priority.session -> silent,
priority.session set -> audio. Only overrides WirePlumber's auto default; a
user's explicit default.configured.audio.source still wins.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 21:31:51 +00:00
enricobuehler dd4da9e04d docs: full env/config reference + fix outdated Bazzite gamescope-only framing
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m34s
ci / rust (push) Successful in 4m47s
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m15s
apple / screenshots (push) Successful in 5m14s
deb / build-publish (push) Successful in 2m46s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
ci / bench (push) Successful in 4m41s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m47s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m47s
docker / deploy-docs (push) Successful in 20s
Rebuild configuration.md into a complete PUNKTFUNK_* reference (verified
against config.rs, the host.env templates, and the env read sites): core,
gamescope/session-following, compositor, video quality, gamepads, audio,
Windows host, auth/paths, perf tuning, diagnostics, and client-side knobs.

Rework bazzite.md: it now documents both Steam Gaming Mode (gamescope) and
the KDE Plasma desktop with auto-detect/session-following, attach vs managed,
and the Desktop screencast + kde-desktop-setup.sh input grant — previously it
only described the managed gamescope model.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 20:06:28 +00:00
enricobuehler d6596ff81b docs: rework client/crate READMEs, add missing ones
windows-drivers / probe-and-proto (push) Successful in 24s
windows-drivers / driver-build (push) Successful in 1m18s
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m21s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 2m48s
windows-host / package (push) Successful in 7m10s
decky / build-publish (push) Successful in 24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m38s
release / apple (push) Successful in 9m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m32s
flatpak / build-publish (push) Successful in 4m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m49s
Rework the client READMEs to be accurate and inviting to first-time
visitors, and fill in the gaps where crates and tools had none.

- Rewrite clients/{apple,android,decky} READMEs (features-first, trim
  dense internal narrative; drop the stale "one session at a time" /
  "renegotiation not implemented" section from the Apple README).
- Add READMEs for clients/{linux,windows,probe}, which had none.
- Add crate READMEs for punktfunk-host, punktfunk-core, pf-driver-proto.
- Add brief READMEs for tools/{loss-harness,latency-probe}.
- Fix packaging/README duplicate "Option B" heading (bootc -> Option C).
- Fix docs-site/README stale docs/ -> design/ reference.
- De-stale packaging/windows/drivers/pf-dualsense README (drop "M0 spike"
  / external-checkout framing; reflect in-tree workspace + shipped +
  installer-bundled + multi-pad), keeping the driver-authoring lore.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 19:31:06 +00:00
426 changed files with 39536 additions and 12893 deletions
+13
View File
@@ -34,4 +34,17 @@ ignore = [
# a constant-time rsa ships (then drop this), the host ever signs an attacker-chosen message with
# this key, or any RSA decryption / key-transport using the private key is added.
"RUSTSEC-2023-0071",
# quick-xml DoS advisories (RUSTSEC-2026-0194 quadratic-time duplicate-attribute check;
# RUSTSEC-2026-0195 unbounded namespace-declaration allocation in NsReader). Both are
# exploited by feeding attacker-controlled XML to a running parser. In this tree quick-xml is
# a BUILD-TIME-ONLY, transitive dependency of `wayland-scanner` (a proc-macro that parses the
# TRUSTED wayland protocol XML files shipped with the wayland-rs crates at compile time). It is
# never linked into any shipped binary and never parses runtime/attacker-controlled input, so
# neither DoS is reachable. There is no fix to bump to: wayland-scanner 0.31.10 (latest) pins
# `quick-xml ^0.39`, and the fixes only exist in quick-xml >=0.41. Revisit (drop these) when
# wayland-scanner releases against quick-xml >=0.41, or if quick-xml is ever pulled onto a
# runtime path that parses untrusted XML.
"RUSTSEC-2026-0194",
"RUSTSEC-2026-0195",
]
+9
View File
@@ -0,0 +1,9 @@
# Shown on the "new issue" chooser so security reports go to the private channel, not a public issue.
blank_issues_enabled: true
contact_links:
- name: 🔒 Report a security vulnerability
url: https://git.unom.io/unom/punktfunk/src/branch/main/SECURITY.md
about: >-
Found a security issue? Please report it privately by email to security@punktfunk.com — do not
open a public issue, so other users aren't exposed before a fix ships. See SECURITY.md for the
full policy.
+2 -1
View File
@@ -78,9 +78,10 @@ jobs:
- name: Version + channel
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
case "$GITHUB_REF" in
refs/tags/v*) VN="${GITHUB_REF_NAME#v}"; TRACK="alpha" ;; # alpha = built-in closed testing
*) VN="0.5.0-ci${GITHUB_RUN_NUMBER}"; TRACK="internal" ;;
*) VN="${PF_BASE}-ci${GITHUB_RUN_NUMBER}"; TRACK="internal" ;;
esac
echo "VERSION_NAME=$VN" >> "$GITHUB_ENV"
echo "PLAY_TRACK=$TRACK" >> "$GITHUB_ENV"
+14 -5
View File
@@ -36,16 +36,17 @@ jobs:
- name: Version + channel
# vX.Y.Z tag -> X.Y.Z, published to the `stable` apt distribution (a real release).
# A main push -> 0.5.0~ciN.g<sha>, published to the `canary` distribution: the '~' sorts
# below the eventual 0.5.0 tag, it climbs monotonically by run number, and the canary base
# stays one minor AHEAD of the latest stable so a stable->canary box re-point still moves
# forward (see channels.md). Computed BEFORE the build so it's stamped into the binary
# A main push -> <next-minor>~ciN.g<sha>, published to the `canary` distribution: the '~' sorts
# below the eventual tag, it climbs monotonically by run number, and the canary base is
# derived one minor AHEAD of the latest stable tag (scripts/ci/pf-version.sh) so a
# stable->canary box re-point still moves forward (see channels.md). Computed BEFORE the build so it's stamped into the binary
# (PUNKTFUNK_BUILD_VERSION -> build.rs -> --version).
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; DIST=stable ;;
*) V="0.5.0~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;;
*) V="${PF_BASE}~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;;
esac
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "DISTRIBUTION=$DIST" >> "$GITHUB_ENV"
@@ -126,6 +127,14 @@ jobs:
run: |
for DEB in dist/*.deb; do
echo "uploading $DEB"
# A re-tagged release re-fires this workflow and the apt registry 409s on duplicate
# package versions — delete any prior copy of this exact name/version/arch first
# (404 on the first publish is fine).
NAME=$(dpkg-deb -f "$DEB" Package)
VER=$(dpkg-deb -f "$DEB" Version)
ARCH=$(dpkg-deb -f "$DEB" Architecture)
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/$NAME/$VER/$ARCH" || true
# PAT owner (enricobuehler), not the push actor — matches docker.yml's registry login.
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$DEB" \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/upload"
+13 -4
View File
@@ -63,7 +63,8 @@ jobs:
pnpm run build # rollup -> clients/decky/dist/index.js
- name: Version + channel + stamp
# Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> 0.3.<run>
# Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> <next-minor>.<run>
# (base one minor ahead of the latest stable tag via scripts/ci/pf-version.sh)
# (`canary/` alias). Decky reads a plugin's INSTALLED version from package.json (NOT
# plugin.json), and the plugin's own update check (clients/decky/main.py check_update)
# compares against it — so the build version is STAMPED into package.json here (mirrored
@@ -72,9 +73,12 @@ jobs:
# (ci10 < ci9), which would break update detection; the run number is monotonic.
working-directory: ${{ gitea.workspace }}
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_MAJOR/PF_MINOR (base one minor ahead of latest stable)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; ALIAS=latest ;;
*) V="0.3.${GITHUB_RUN_NUMBER}"; ALIAS=canary ;;
# Canary MUST be a plain monotonic numeric semver (see the note above): <major>.<minor>.<run>,
# where major.minor track one minor ahead of the latest stable and the run number climbs.
*) V="${PF_MAJOR}.${PF_MINOR}.${GITHUB_RUN_NUMBER}"; ALIAS=canary ;;
esac
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
echo "VERSION=$V" >> "$GITHUB_ENV"
@@ -122,8 +126,13 @@ jobs:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
# 1) Immutable, versioned URL + its update manifest (the manifest's `artifact` points
# here, so the published sha256 keeps matching what Decky later downloads).
# 1) Versioned URL + its update manifest (the manifest's `artifact` points here, so the
# published sha256 keeps matching what Decky later downloads). A re-tagged release
# re-fires this workflow and the registry 409s on duplicate uploads — delete any
# prior copy of this version first (404 on the first publish is fine).
for f in punktfunk.zip manifest.json; do
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE "$BASE/$VERSION/$f" || true
done
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/punktfunk.zip" \
"$BASE/$VERSION/punktfunk.zip"
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/manifest.json" \
+52 -6
View File
@@ -73,15 +73,17 @@ jobs:
- name: Version + channel
# Tag vX.Y.Z -> X.Y.Z on the OSTree `stable` branch (a real release); a main push ->
# 0.5.0-ciN.g<sha> on the `canary` branch. The two branches live side-by-side in one repo
# <next-minor>-ciN.g<sha> on the `canary` branch (base one minor ahead of the latest stable
# tag via scripts/ci/pf-version.sh). The two branches live side-by-side in one repo
# (rsync runs without --delete), each tracked by its own .flatpakref, so `flatpak update`
# on a stable box never jumps to a canary build. The generic-registry version string allows
# letters/dots/hyphens.
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; BRANCH=stable; ALIAS=latest ;;
*) V="0.5.0-ci${GITHUB_RUN_NUMBER}.g${SHORT}"; BRANCH=canary; ALIAS=canary ;;
*) V="${PF_BASE}-ci${GITHUB_RUN_NUMBER}.g${SHORT}"; BRANCH=canary; ALIAS=canary ;;
esac
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "BUNDLE=punktfunk-client-${V}.flatpak" >> "$GITHUB_ENV"
@@ -106,6 +108,40 @@ jobs:
python3 /tmp/flatpak-cargo-generator.py /tmp/Cargo.flatpak.lock \
-o packaging/flatpak/cargo-sources.json
- name: Seed the local OSTree repo from the live server (keep BOTH channels in the summary)
# Each CI run builds only ONE branch (canary on main, stable on a tag). The deploy step's
# `flatpak build-update-repo` regenerates the repo SUMMARY from whatever refs are in the
# LOCAL repo, and the rsync publishes it (without --delete). A fresh single-branch local
# repo therefore produces a single-branch summary that CLOBBERS the other channel on the
# server — the exact bug that made `app/io.unom.Punktfunk/x86_64/stable` unresolvable
# ("No such ref") after a canary main-push overwrote the post-release summary, even though
# the stable commit's objects were still on disk. Fix: mirror the published repo DOWN first,
# so the local repo carries every existing branch; the build below then only ADDS this run's
# commit and the regenerated+signed summary keeps both channels. No-op on a fresh repo (first
# publish) or when the deploy secrets aren't set (the build still produces a valid bundle).
env:
DEPLOY_HOST: ${{ secrets.DEPLOY_HOST }}
DEPLOY_USER: ${{ secrets.DEPLOY_USER }}
DEPLOY_PORT: ${{ secrets.DEPLOY_PORT }}
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
run: |
set -euo pipefail
if [ -z "${DEPLOY_HOST:-}" ] || [ -z "${DEPLOY_SSH_KEY:-}" ]; then
echo "::warning::DEPLOY_* not set — no seed; building a fresh single-branch repo."
exit 0
fi
install -d -m700 ~/.ssh
printf '%s\n' "$DEPLOY_SSH_KEY" > ~/.ssh/deploy; chmod 600 ~/.ssh/deploy
SSH="ssh -i $HOME/.ssh/deploy -p ${DEPLOY_PORT:-22} -o StrictHostKeyChecking=accept-new"
DEST="${DEPLOY_USER}@${DEPLOY_HOST}"
mkdir -p "$PWD/repo"
# Pull the currently-published repo (all channels' objects + refs) into the repo the build
# will extend. No --delete: the local repo starts empty, so this only ADDS. A missing
# server repo (very first publish) is fine — we continue with a fresh repo.
rsync -az --info=stats1 -e "$SSH" "$DEST:$DEPLOY_DIR/site/repo/" "$PWD/repo/" \
|| echo "::warning::no published repo to seed (first publish?) — continuing fresh"
echo "seeded refs:"; ls "$PWD/repo/refs/heads/app/$APP_ID/x86_64/" 2>/dev/null || echo " (none)"
- name: Build the flatpak (install deps from Flathub, offline build)
run: |
# --install-deps-from=flathub pulls everything the manifest declares: the GNOME 50
@@ -133,7 +169,10 @@ jobs:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
# 1) Immutable, versioned URL.
# 1) Versioned URL. A re-tagged release re-fires this workflow and the registry 409s on
# duplicate uploads — delete any prior copy first (404 on the first publish is fine).
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"$BASE/$VERSION/$BUNDLE" || true
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$BUNDLE" \
"$BASE/$VERSION/$BUNDLE"
echo "published $BASE/$VERSION/$BUNDLE"
@@ -174,6 +213,10 @@ jobs:
--gpg-sign="$KEYID" --gpg-homedir="$GNUPGHOME"
flatpak build-update-repo --generate-static-deltas \
--gpg-sign="$KEYID" --gpg-homedir="$GNUPGHOME" "$PWD/repo"
# The regenerated summary advertises exactly these refs — must include EVERY channel that
# has ever published (the seed step ensures the other channel's commit is present). If this
# ever shows only one branch on a repo that had two, the seed didn't run — investigate.
echo "published summary advertises:"; ls "$PWD/repo/refs/heads/app/$APP_ID/x86_64/" 2>/dev/null || echo " (none)"
# 2) Build the install descriptors (GPGKey = the committed public key, base64).
GPGKEY="$(base64 -w0 packaging/flatpak/unom-flatpak.gpg)"
rm -rf site && mkdir -p site
@@ -185,9 +228,12 @@ jobs:
Comment=unom Flatpak applications
GPGKey=$GPGKEY
EOF
# Two refs, one per channel — both regenerated every run and rsync'd without --delete, so
# the server always offers both (the stable ref only resolves once a release has built the
# `stable` branch). A box installs ONE; `flatpak update` then tracks that channel's branch.
# Two refs, one per channel. Both descriptor files are regenerated every run and rsync'd
# without --delete; the repo SUMMARY carries both branches because the build was seeded
# from the live repo above (so build-update-repo below re-signs a summary listing every
# published channel, not just this run's). The stable ref resolves for good once any
# release has built the `stable` branch. A box installs ONE; `flatpak update` then tracks
# that channel's branch.
write_ref() { # <filename> <branch> <title>
cat > "site/$1" <<EOF
[Flatpak Ref]
+3 -2
View File
@@ -99,13 +99,14 @@ jobs:
- name: Version from tag
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE, PF_CHANNEL, PF_STABLE_TAG (single source of truth)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; V="${V%%-*}" ;; # App Store marketing version is numeric X.Y.Z (drop -rc)
*) V="0.5.0" ;; # canary marketing version; the build number disambiguates
*) V="$PF_BASE" ;; # canary marketing version = one minor ahead of the latest stable tag; the build number disambiguates
esac
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "BUILD_NUM=$GITHUB_RUN_NUMBER" >> "$GITHUB_ENV"
echo "version $V build $GITHUB_RUN_NUMBER"
echo "version $V build $GITHUB_RUN_NUMBER (channel $PF_CHANNEL, latest stable ${PF_STABLE_TAG})"
- name: Rust toolchain (mac + iOS + tvOS slices)
run: |
+14 -5
View File
@@ -68,16 +68,17 @@ jobs:
restore-keys: cargo-home-
- name: Version + channel
# vX.Y.Z tag -> X.Y.Z-1 in the base group (a real release); main push -> 0.5.0-0.ciN.g<sha>
# in the `<base>-canary` group, whose "0." release sorts below the eventual 0.5.0-1 yet
# climbs by run number. The canary base stays one minor ahead of the latest stable so a
# stable->canary box re-point still moves forward. The spec %build stamps
# vX.Y.Z tag -> X.Y.Z-1 in the base group (a real release); main push -> <next-minor>-0.ciN.g<sha>
# in the `<base>-canary` group, whose "0." release sorts below the eventual <next-minor>-1 yet
# climbs by run number. The canary base is derived one minor ahead of the latest stable tag
# (scripts/ci/pf-version.sh) so a stable->canary box re-point still moves forward. The spec %build stamps
# PUNKTFUNK_BUILD_VERSION from these macros into the binary (--version provenance).
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; R="1"; GROUP="${{ matrix.group }}" ;;
*) V="0.5.0"; R="0.ci${GITHUB_RUN_NUMBER}.g${SHORT}"; GROUP="${{ matrix.group }}-canary" ;;
*) V="$PF_BASE"; R="0.ci${GITHUB_RUN_NUMBER}.g${SHORT}"; GROUP="${{ matrix.group }}-canary" ;;
esac
echo "PF_VERSION=$V" >> "$GITHUB_ENV"
echo "PF_RELEASE=$R" >> "$GITHUB_ENV"
@@ -103,6 +104,14 @@ jobs:
for rpm in dist/*.rpm; do
case "$rpm" in *debuginfo*|*debugsource*) echo "skip $rpm"; continue;; esac
echo "uploading $rpm"
# A re-tagged release re-fires this workflow and the rpm registry 409s on duplicate
# package versions — delete any prior copy of this exact name/version-release/arch
# first (404 on the first publish is fine).
NAME=$(rpm -qp --qf '%{NAME}' "$rpm" 2>/dev/null)
VR=$(rpm -qp --qf '%{VERSION}-%{RELEASE}' "$rpm" 2>/dev/null)
ARCH=$(rpm -qp --qf '%{ARCH}' "$rpm" 2>/dev/null)
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"https://$REGISTRY/api/packages/$OWNER/rpm/$GROUP/package/$NAME/$VR/$ARCH" || true
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$rpm" \
"https://$REGISTRY/api/packages/$OWNER/rpm/$GROUP/upload"
done
+13 -3
View File
@@ -131,11 +131,21 @@ jobs:
# dispatched provisioning workflow landing on a different one. Path is relative to the job
# working-directory (packaging/windows/drivers). Near-noop once the toolchain is present.
run: ../../../scripts/ci/ensure-windows-toolchain.ps1
- name: cargo build the driver workspace (wdk-probe + wdk-iddcx + pf-vdisplay)
- name: cargo build the driver workspace (wdk-probe + wdk-iddcx + pf-vdisplay + gamepad drivers)
# Whole workspace: wdk-probe (toolchain/surface-assert probe) + wdk-iddcx (DDI wrappers) +
# pf-vdisplay (the real IddCx driver). pf-vdisplay linking proves the IddCx call sites resolve
# against IddCxStub end-to-end (M1 step 2 gate).
# pf-vdisplay (the real IddCx driver) + pf-umdf-util (the safe UMDF primitive layer) + the two
# gamepad drivers. pf-vdisplay linking proves the IddCx call sites resolve against IddCxStub
# end-to-end (M1 step 2 gate); the gamepad drivers prove pf-umdf-util's WDF dispatch links.
run: cargo build -v
- name: cargo clippy the shipped drivers (-D warnings — enforces the unsafe-audit gates)
# The gamepad drivers' business logic is 100% safe (it moved onto pf-umdf-util, the audited
# unsafe layer); pf-vdisplay + wdk-iddcx are inherently FFI-bound but every `unsafe {}` carries a
# `// SAFETY:` proof. Both invariants are lint-gated (`unsafe_op_in_unsafe_fn` +
# `undocumented_unsafe_blocks`); this step keeps them from regressing. (wdk-probe is a
# toolchain-only probe crate and is excluded.)
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense --check
- name: Inspect /INTEGRITYCHECK (before) — expect FORCE_INTEGRITY set by wdk-build
run: |
# explicit --target (.cargo/config.toml) -> output under the triple subdir.
+25 -12
View File
@@ -16,15 +16,17 @@
# Versioning (free-form; not MSIX's 4-part rule) — single project version:
# vX.Y.Z tag -> X.Y.Z (THE release; published + stable `latest/` alias + attached to the
# unified Gitea Release).
# main push / dispatch -> 0.3.<run_number> (canary; `canary/` alias; climbs by run number).
# main push / dispatch -> <next-minor>.<run_number> (canary; `canary/` alias; base one minor
# ahead of the latest stable tag via scripts/ci/pf-version.ps1, run climbs).
#
# Signing reuses the client's MSIX_CERT_PFX_B64 / MSIX_CERT_PASSWORD secrets (CN=unom). Without them
# an ephemeral self-signed cert is generated and its public .cer published next to the installer
# (import once to LocalMachine\TrustedPublisher). See packaging/windows/pack-host-installer.ps1.
#
# GPU backends: the host builds with --features nvenc,amf-qsv = all three vendors in one installer.
# - NVENC (NVIDIA, direct SDK): the only link need is nvencodeapi.lib, synthesised from a 2-export
# .def with llvm-dlltool (no GPU/SDK at build time).
# - NVENC (NVIDIA, direct SDK): nothing needed at build time — the entry points are resolved at
# RUNTIME from the driver's nvEncodeAPI64.dll (a link-time import would kill the binary on
# AMD/Intel-only boxes before main).
# - AMF/QSV (AMD/Intel, libavcodec): link-imports the FFmpeg libs from FFMPEG_DIR (the BtbN lgpl-shared
# tree the client uses; includes the *_amf/*_qsv encoders) and bundles its DLLs into the installer.
# lgpl-shared (not gpl-shared) keeps those bundled DLLs LGPL (we never use the GPL-only x264/x265).
@@ -37,6 +39,7 @@ on:
paths:
- 'crates/punktfunk-host/**'
- 'crates/punktfunk-core/**'
- 'crates/punktfunk-tray/**'
- 'packaging/windows/**'
- 'scripts/windows/**'
- 'web/**'
@@ -93,30 +96,40 @@ jobs:
if (-not $env:FFMPEG_DIR) {
"FFMPEG_DIR=C:\Users\Public\ffmpeg" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
}
# VBCABLE_DIR: the pinned official VB-CABLE package (provisioned by
# provision-windows-punktfunk-extras.ps1) -> pack-host-installer.ps1 bundles the
# streaming virtual microphone. Same daemon-env-or-fallback pattern as FFMPEG_DIR
# (the daemon env only refreshes on a runner-task restart).
if (-not $env:VBCABLE_DIR) {
"VBCABLE_DIR=C:\Users\Public\vbcable" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
}
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$v = if ($env:GITHUB_REF -like 'refs/tags/v*') {
$env:GITHUB_REF_NAME -replace '^v', ''
} else {
"0.3.$($env:GITHUB_RUN_NUMBER)"
# Canary: <major>.<minor>.<run> — major.minor track one minor ahead of stable, run climbs monotonically.
"$($pf.PF_MAJOR).$($pf.PF_MINOR).$($env:GITHUB_RUN_NUMBER)"
}
"HOST_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"PUNKTFUNK_BUILD_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
Write-Output "host version $v"
- name: Generate NVENC import lib
shell: pwsh
run: |
& packaging/windows/nvenc/gen-nvenc-importlib.ps1 -OutDir C:\t\nvenc
"PUNKTFUNK_NVENC_LIB_DIR=C:\t\nvenc" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Build (release, nvenc + amf-qsv)
shell: pwsh
# All-vendor host: NVENC (NVIDIA, direct SDK) + AMF/QSV (AMD/Intel, libavcodec via FFMPEG_DIR).
run: cargo build --release -p punktfunk-host --features nvenc,amf-qsv
- name: Clippy (host, Windows)
- name: Build (release, status tray)
shell: pwsh
# The per-user notification-area companion the installer bundles (punktfunk-tray.exe).
run: cargo build --release -p punktfunk-tray
- name: Clippy (host + tray, Windows)
shell: pwsh
# First-ever Windows lint coverage for the host (Linux CI never lints the windows-cfg code).
run: cargo clippy -p punktfunk-host --features nvenc,amf-qsv -- -D warnings
run: |
cargo clippy -p punktfunk-host --features nvenc,amf-qsv -- -D warnings; if ($LASTEXITCODE) { throw "host clippy" }
cargo clippy -p punktfunk-tray -- -D warnings; if ($LASTEXITCODE) { throw "tray clippy" }
- name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer)
shell: pwsh
+5 -2
View File
@@ -16,7 +16,8 @@
# vX.Y.Z tag -> X.Y.Z.0 (THE release; any -rc/+meta pre-release suffix is dropped for MSIX).
# Published to the generic registry + the stable `latest/` alias + attached to the
# unified Gitea Release alongside every other platform's artifact.
# main push / dispatch -> 0.3.<run_number>.0 (canary; climbs monotonically by run number).
# main push / dispatch -> <next-minor>.<run_number>.0 (canary; base is one minor ahead of the
# latest stable tag via scripts/ci/pf-version.ps1, run number climbs monotonically).
# Published to the generic registry + the `canary/` alias.
# Both arches share the version; artifacts are arch-suffixed (..._x64.msix / ..._arm64.msix).
#
@@ -78,11 +79,13 @@ jobs:
"CARGO_TARGET_DIR=${{ matrix.td }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"FFMPEG_DIR=${{ matrix.ffmpeg }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
rustup target add ${{ matrix.target }}
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$parts = if ($env:GITHUB_REF -like 'refs/tags/v*') {
# MSIX needs a purely-numeric 4-part version: drop any -rc/+meta pre-release suffix.
(($env:GITHUB_REF_NAME -replace '^v', '') -replace '[-+].*$', '').Split('.')
} else {
@('0', '3', $env:GITHUB_RUN_NUMBER)
# Canary: <major>.<minor>.<run>.0 — major.minor track one minor ahead of stable, run climbs monotonically.
@($pf.PF_MAJOR, $pf.PF_MINOR, $env:GITHUB_RUN_NUMBER)
}
while ($parts.Count -lt 4) { $parts += '0' }
$v = ($parts[0..3] -join '.')
+3
View File
@@ -31,3 +31,6 @@ xcuserdata/
# Python bytecode (e.g. clients/android/ci tooling)
__pycache__/
*.pyc
# Claude Code project instructions — local to each dev box, not part of the repo.
CLAUDE.md
-405
View File
@@ -1,405 +0,0 @@
# CLAUDE.md — punktfunk
Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protocol core
(`punktfunk-core`) exposed over a C ABI and native clients per platform. Full design:
[`design/implementation-plan.md`](design/implementation-plan.md). Status table: `README.md`.
## Where the work stands
- **Core (`punktfunk-core` + C ABI): complete and hardened.** FEC recovery, loopback-under-loss,
proptests, C ABI harness all green; 13 adversarial-review findings fixed +
regression-tested (`a913042`).
- **GameStream host: working end-to-end with a stock Moonlight client.** Validated live
on this box: pairing (persists across restarts), serverinfo/applist (app catalog from
`~/.config/punktfunk/apps.json` → each entry picks a compositor + nested command), RTSP, ENet
control, audio, and video at the **client's native resolution and refresh** — the host
creates a per-session virtual output via per-compositor `VirtualDisplay` backends:
**KWin** (`zkde_screencast stream_virtual_output`, needs KWin ≥ 6.5.6 headless; >60 Hz via
custom modes), **gamescope** (spawned headless at WxH@Hz, its PipeWire node captured, needs
gamescope ≥ 3.16.22 — older deadlocks on PipeWire ≥ 1.6), **Mutter** (D-Bus
`RecordVirtual` virtual monitor; validated live on headless GNOME Shell 50, zero-copy),
**Sway/wlroots** (`swaymsg create_output` + custom mode, xdpw portal capture with a
managed chooser config; validated live on sway 1.11, zero-copy).
Performance work landed and measured: GPU **zero-copy** on all paths (tiled dmabuf →
EGL/GL → CUDA; LINEAR dmabuf → **Vulkan bridge** → CUDA → NVENC), auto 2-way NVENC
split-encode above ~1 Gpix/s (5K@240), infinite GOP + RFI keyframes (killed the periodic
freeze), encode|send thread split with `sendmmsg` batching. Stable 240 fps at 5120×1440.
Input: mouse/keyboard (libei via RemoteDesktop portal on KWin/GNOME, gamescope's own EIS
socket, wlr protocols on Sway) and **gamepads** (uinput X-Box-360 pads + rumble
back-channel; validated live — pad created/destroyed with the session). Management REST API +
checked-in OpenAPI doc (`mgmt.rs`). **Web-console performance capture** (`stats_recorder.rs`,
design: [`design/stats-capture-plan.md`](design/stats-capture-plan.md)): the operator arms stats
recording from the web console, plays, stops, and reviews the run as graphs (per-stage latency
breakdown · fps new/repeat · goodput · loss/FEC). A shared `Arc<StatsRecorder>` ring (the hot-path
gate is a runtime `AtomicBool`, replacing the startup-only `PUNKTFUNK_PERF`) is fed by **both** the
native `virtual_stream` and the GameStream encode loop at their existing ~2 s/~1 s aggregation
boundary, and finished captures are saved as on-disk recordings
(`~/.config/punktfunk/captures/*.json`) browsable/exportable from the console's **Performance** page
(recharts). Endpoints `/api/v1/stats/*` (bearer-only). *Implemented; not yet on-glass validated.*
- **Native protocol (`punktfunk/1`): full session planes, validated live.** QUIC
control plane (`punktfunk-core` `quic` feature: Hello{mode}/Welcome{full Config}/Start), data
plane = the hardened core `Session` over raw UDP with **GF(2¹⁶) Leopard FEC + AES-GCM**
(inexpressible in GameStream), host creates the native virtual output at the client's
requested mode. `punktfunk1-host` is a **persistent listener** (sessions back to back;
`--max-sessions`). QUIC datagrams carry the side planes, demuxed by first byte: input
0xC8 (incl. **gamepads** — incremental events accumulated into the uinput xpad), **Opus
audio** 0xC9 (48 kHz stereo, 5 ms, host→client), **rumble** 0xCA (host→client). **Trust:**
host serves its persistent identity (`~/.config/punktfunk/cert.pem`, shared with GameStream
pairing) and logs the SHA-256 fingerprint; clients pin it, established by a **SPAKE2 PIN pairing
ceremony** (host arms pairing and displays a 4-digit PIN; a PAKE binds both cert fingerprints so an
attacker gets one online guess, no offline dictionary attack) — PIN pairing is the default for new
hosts. **TOFU on first connect** (`endpoint::client_pinned`) stays as an explicit host opt-in
(`punktfunk1-host --allow-tofu` / `serve --open`, advertised as `pair=optional`) for fully trusted LANs;
clients only offer the TOFU "Trust" path for a host that advertised `pair=optional`, route every
other new host straight to the PIN ceremony, and on a pinned-fingerprint change force re-pairing
(no re-TOFU shortcut). Clients present persistent identities via QUIC client auth, the host stores
paired fingerprints (`punktfunk1-paired.json`) and gates sessions with `--require-pairing` (the
default; `--allow-tofu`/`--open` accept unpaired clients).
**LAN auto-discovery**: both `serve` and `punktfunk1-host` advertise the native service over
mDNS (`_punktfunk._udp`, `crate::discovery`) with TXT `proto`/`fp`(cert fingerprint to
pin)/`pair`(required|optional)/`id`; `punktfunk-probe --discover` lists hosts, Apple clients
browse the same service via NWBrowser (validated cross-LAN 2026-06-12).
**Mid-stream mode renegotiation**: `Reconfigure` on the still-open control stream — the
host rebuilds output+encoder at the new mode in ~90 ms while the data plane runs on
(validated live: one .h265 with 720p and 1080p segments). Measured on-box at 720p120: 1680/1680 frames, **p50 0.83 ms**
capture→…→reassembled; audio measured live (~200 pkts/s). A **wall-clock skew handshake**
(`ClockProbe`/`ClockEcho`, 8 NTP rounds after `Start`, `clock_offset_ns`) aligns the client to the
host clock, so that latency is now valid **cross-machine** (`skew_corrected=true`) — measured GNOME
box → dev box over the LAN: **p50 1.30 ms** (the 1.57 ms inter-box clock offset removed).
`punktfunk-probe` is the
working reference client (`--pin`, datagram counters, `--input-test` incl. gamepad).
The embeddable connector (`NativeClient`) exposes it all over the C ABI: `punktfunk_connect`
(pin/TOFU) + `next_au`/`next_audio`/`next_rumble`/`next_hidout`/`send_input`/
`send_rich_input`. **Client-negotiated virtual pad type**: the Hello carries a gamepad
preference byte (same trailing-byte back-compat pattern as the compositor), the Welcome
echoes the resolved backend — precedence: explicit client choice > `PUNKTFUNK_GAMEPAD`
env > uinput Xbox 360. Backends: **Xbox 360** (uinput / ViGEm), **Xbox One/Series** (the same
XInput backend with the One/Series USB identity for matching glyphs — no extra game-visible
capability; impulse-trigger rumble is unreachable through a virtual pad), and the UHID
`hid-playstation` pads — **DualSense** (adaptive triggers, lightbar, touchpad, motion) and
**DualShock 4** (lightbar, touchpad, motion, rumble; DualSense minus adaptive triggers / player
LEDs / mute). DualSense and DualShock 4 each have a Linux (UHID `hid-playstation`) **and a Windows
(UMDF minidriver)** backend — `inject/dualsense_windows.rs` + `inject/dualshock4_windows.rs`, one
driver serving either identity per a `device_type` byte the host stamps into shared memory (the DS4
reuses the same SwDeviceCreate game-detection identity fix as the DualSense). One/Series stays
Linux-only and folds into Xbox 360 off it. Clients auto-resolve the type from the physical controller
(DS5→DualSense, DS4→DualShock 4, Xbox One→Xbox One). **Windows uses ZERO external gamepad
dependencies — ViGEmBus is gone.** Xbox 360 (XInput) runs on a UMDF2 **XUSB companion** driver
(`packaging/windows/xusb-driver/`, `inject/gamepad_windows.rs`) that registers `GUID_DEVINTERFACE_XUSB`
and answers the buffered XInput IOCTLs from a shared section, so classic `XInputGetState`/`SetState`
work with no kernel bus driver (validated live: slot connected, state + rumble round-trip; Xbox One
folds to this 360 path). All three UMDF drivers (DualSense/DS4 + XUSB) are bundled + pnputil-installed
by the Inno Setup installer (`packaging/windows/gamepad-drivers/` + `install-gamepad-drivers.ps1`).
**Multi-pad ready**: the host stamps each pad's index into the device Location (`pszDeviceLocation`),
the driver reads it (`WdfDeviceAllocAndQueryProperty`) to map its own `*-shm-<index>`, and
`UmdfHostProcessSharing=ProcessSharingDisabled` gives each pad its own host (per-pad statics) —
validated live with 2 distinct XInput slots + 2 DualSense pads. (Client-side multi-pad forwarding is
the remaining piece.)
- **Windows host: implemented and shipping (all-vendor, x64-only).** `#[cfg(windows)]` backends
behind the same traits as Linux — DXGI Desktop Duplication capture (`capture/dxgi.rs`), **SudoVDA**
virtual display per session (`vdisplay/sudovda.rs`), GPU encode (NVENC `--features nvenc`; AMD/Intel
`--features amf-qsv`), SendInput + **ViGEm** gamepads (`inject/gamepad_windows.rs`), WASAPI loopback
+ virtual mic (`audio/wasapi_*`). Ships as a **signed Inno Setup installer** that registers a
`LocalSystem` SCM service launching into the interactive session for secure-desktop (UAC/lock-screen)
capture (`service.rs`), bundles the SudoVDA driver + the FFmpeg DLLs, and is published by
`windows-host.yml`. **Encoder is GPU-aware** (`encode.rs` `open_video` + `windows_resolved_backend`):
`PUNKTFUNK_ENCODER=auto` (the host.env default) detects the DXGI adapter vendor → **NVENC** (NVIDIA,
direct SDK, `encode/nvenc.rs`), **AMF** (AMD) / **QSV** (Intel) via libavcodec
(`encode/ffmpeg_win.rs`, the Windows analogue of the Linux VAAPI backend — `WinVendor{Amf,Qsv}`,
system-memory NV12/P010 readback default + opt-in zero-copy D3D11 behind `PUNKTFUNK_ZEROCOPY` with a
system fallback), or software H.264 (`encode/sw.rs`, GPU-less). GameStream codec advertisement is
probed per-GPU on AMF/QSV (`windows_codec_support``serverinfo`, AV1 gated). **HDR (10-bit)**: WGC
captures the HDR desktop as FP16/Rgb10a2 (DDA FP16 for the secure desktop), the encoder forces HEVC
Main10 + BT.2020 PQ (NVENC ABGR10/P010; AMF/QSV P010 + a swscale Rgb10a2→P010 fallback), the client
auto-detects PQ from the HEVC VUI — gated by `PUNKTFUNK_10BIT` + client `VIDEO_CAP_10BIT`; **Windows
host only** (the Linux host stays 8-bit, blocked upstream). **Vulkan-game HDR over the virtual
display**: NVIDIA/AMD Vulkan ICDs refuse to *advertise* an HDR color space for a surface on an IddCx
indirect display (so Vulkan games — Doom: The Dark Ages, id Tech, etc. — say "device does not support
HDR"), even though the ICD happily *accepts + presents* a forced HDR swapchain there. A tiny always-on
Vulkan **implicit layer** (`packaging/windows/pf-vkhdr-layer/`, `VK_LAYER_PUNKTFUNK_hdr_inject`)
injects the `HDR10_ST2084`/scRGB surface formats into `vkGetPhysicalDeviceSurfaceFormats[2]KHR`,
self-gated on the display's actual advanced-color state (no-op on SDR / real monitors); bundled +
HKLM-registered by the installer. **Live-validated: Doom: The Dark Ages enables HDR over the virtual
display.** **AMF/QSV is CI-green but not yet on-glass validated** (no AMD/Intel Windows box in the
lab); NVENC is live-validated. Newer/less battle-tested than the Linux host. Packaging: `packaging/windows/`.
## What's left
1. **Native clients — decode + present: macOS stage 1 done, first light achieved
(2026-06-10).** PunktfunkKit compiles and is tested on macOS (AnnexB → VideoToolbox →
`AVSampleBufferDisplayLayer`, GCMouse/GCKeyboard capture, `PunktfunkClient` app shell);
validated live Mac ↔ this box at 720p60 — vkcube on glass, input injected via gamescope
EIS. The app speaks the full ABI v2 trust surface: Keychain-persisted client identity
presented on every connect, SPAKE2 PIN pairing UI (host-card context menu + the trust
prompt's "Pair with PIN instead…"), TOFU fingerprint prompt. **Gamepads (2026-06-11):**
controller discovery + selection in Settings (`GamepadManager` — exactly one pad
forwarded as pad 0, auto or pinned; pad TYPE auto-resolves from the physical
controller, user-overridable), capture incl. DualSense touchpad/motion
(`GamepadCapture`/`GamepadWire`), feedback rendering (rumble → CoreHaptics; lightbar /
player LEDs / adaptive triggers → `GCDeviceLight`/`playerIndex`/
`GCDualSenseAdaptiveTrigger` via the table-driven `DualSenseTriggerEffect` parser).
Loopback-tested end to end (`PUNKTFUNK_TEST_FEEDBACK=1` scripted burst); DualSense
motion sign/scale derived, not yet live-verified. Tests: `swift test` in
`clients/apple` (unit + real-codec round trip),
`test-loopback.sh` (Swift client vs synthetic punktfunk1-hosts on loopback — runs on macOS;
includes the pairing ceremony + `--require-pairing` gate),
`RemoteFirstLightTests` (full pipeline over the LAN). See
[`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter is now the DEFAULT**
(stage-1 is the Metal-unavailable / DEBUG fallback): explicit `VTDecompressionSession` decode →
`CAMetalLayer`, presented from the hosting view's **main-runloop `CADisplayLink`** (`renderTick` pops
the newest ready frame per vsync; macOS `displaySyncEnabled = false` is the real fullscreen-judder fix,
~11 ms p50). *(An off-main `CAMetalDisplayLink` and an off-main blocking-render present thread were
both tried and reverted — both measured slower on macOS and iPad.)* **HDR fixed**
(`design/apple-stage2-presenter.md`): the "too bright" bug was a missing reference-white anchor — the
fix keeps the PQ-passthrough shader and adds `CAEDRMetadata.hdr10(…, opticalOutputScale: 203)` +
`wantsExtendedDynamicRangeContent` on the layer (on all platforms; the old `#if os(macOS)` guard left
iOS/tvOS EDR half-engaged), routing the 0xCE mastering metadata to the layer (via `setHdrMeta`) instead
of a never-composited source buffer. **Mid-session SDR↔HDR** is handled: `render` reconciles the layer
per-frame from the decoded `frame.isHDR` (per-mode pixel format `bgra8`/`rgba16Float`), so a game
entering HDR mid-stream just reconfigures (last 0xCE grade cached + re-applied; pump drains 0xCE
unconditionally). **4:4:4 added**: decode format is a 2×2 `(chroma, HDR)` matrix
(`420v/x420/444v/x444`, all biplanar so the shaders are unchanged), advertised (`VIDEO_CAP_444`) only
behind a **hardware-required `VTDecompressionSession` probe** (`Stage444Probe`, validated on M3) with a
Settings opt-out + a bounded pump backstop for an undecodable 4:4:4 session. *HDR brightness + 4:4:4
still need on-glass validation (Windows-HDR / `PUNKTFUNK_444` host).* Next: glass-to-glass numbers via
`tools/latency-probe`.
**Linux stage 1 done, first light 2026-06-12** (`clients/linux`, binary
`punktfunk-client`): GTK4/libadwaita shell linking `punktfunk-core` directly (no C ABI;
`NativeClient` is now `Sync` — mutexed plane receivers), mDNS host list, TOFU + SPAKE2
PIN dialogs (identity shared with client-rs), FFmpeg software HEVC decode (LOW_DELAY,
slice threads) → `GtkGraphicsOffload`-wrapped picture, PipeWire playback (mic-player
jitter ring inverted), SDL3 gamepad capture + rumble/lightbar feedback, keyboard via
exact inverse of the host VK table, absolute mouse + 120-unit scroll. Validated live
against `serve` on this box: 1080p60, steady 60 fps, capture→decoded p50
≈6.4 ms (debug build). `--connect host[:port]` for scripting. **Swift-parity batch +
stage 1.5 (2026-06-12 evening)**: capture state machine (click-to-capture,
Ctrl+Alt+Shift+Q / focus-loss release, held-state flush), app-lifetime SDL gamepad
service (pad pin UI, auto type from the physical pad, DualSense touchpad/motion 0xCC +
raw-DS5-effects trigger/player-LED replay — needs a physical pad to live-verify), mic
uplink (validated live), per-host speed test, compositor pref, native-display mode
default, saved-hosts list, .deb + RPM-subpackage CI (deb.yml/rpm.yml). **VAAPI decode
→ DRM-PRIME dmabuf → `GdkDmabufTexture`** (BT.709 color state; Tier-1 zero-copy on
Intel/AMD, `PUNKTFUNK_DECODER=software|vaapi` override) with a proven fallback ladder —
no VAAPI device (NVIDIA) or mid-session VAAPI error → software decode. **First AMD test
(Steam Deck) hit a green-screen bug, fixed:** FFmpeg's VAAPI export uses
`SEPARATE_LAYERS`, so NV12 arrives as two single-plane layers (R8 luma + GR88 chroma,
one shared fd); the mapper took `layers[0]` only → GTK got a luma-only R8 texture, chroma
read as 0 → green field / red whites. Fix derives the combined fourcc from the decoder
`sw_format` (→ `DRM_FORMAT_NV12`) and flattens all planes across all layers (mpv's
pattern); a first-frame descriptor dump logs the real layout. Awaiting Steam Deck
reconfirm. Next: the stage-2 raw-Wayland
presenter (wp_presentation feedback, tearing-control, Vulkan Video on NVIDIA) —
**wgpu/winit rejected** (no dmabuf import / presentation feedback / shortcuts-inhibit).
**Windows stage 1 done 2026-06-15** (`clients/windows`, binary
`punktfunk-client`): pure-Rust **WinUI 3** UI via **windows-reactor** (a declarative React-like
framework backed by WinUI; PR #4499 added the `SwapChainPanel` widget + `set_swap_chain`). The
video is a **`SwapChainPanel`** bound to a **D3D11 composition swapchain** (WARP fallback for
the GPU-less dev box; runtime-compiled fullscreen-triangle shaders, Contain-fit letterbox),
driven by reactor's per-frame `on_rendering`. **FFmpeg HEVC decode with a D3D11VA
zero-copy hardware path** (`gpu.rs` shares one D3D11 device — hardware+`VIDEO_SUPPORT`, WARP
fallback, multithread-protected — between the decoder and presenter; the decoder outputs
NV12/P010 `ID3D11Texture2D` array slices with `BIND_SHADER_RESOURCE` and the presenter samples
them via per-plane SRVs + YUV→RGB shaders — NV12/BT.709, P010/BT.2020-PQ; **software CPU decode
stays as the robust fallback**, auto-selected with a `DecoderPref` override). **HDR10**: the
client advertises 10-bit/HDR (Settings toggle), detects PQ in-band (`transfer == SMPTE2084`),
and flips the swapchain to `R10G10B10A2` + ST.2084 with HDR10 metadata. **WASAPI** render + mic
capture, **SDL3** gamepads (rumble/lightbar/DualSense), `mdns-sd` discovery, and the full trust
surface — all **in-app**: a polished WinUI shell (host cards w/ monogram + status pills,
`InfoBar` errors/hints, `ToggleSwitch` settings, status-chip stream HUD showing GPU/CPU decode +
HDR), host list (live mDNS + saved + manual), settings (resolution/refresh/decoder/bitrate/HDR/
mic), SPAKE2 PIN pairing screen, TOFU, pinned-fp-mismatch re-pair. **(D3D11VA + HDR present + the
GUI polish are written against the windows-rs/reactor APIs but not yet on-glass validated — the
dev VM is headless/WARP; needs the RTX box.)** **Stream input** is Win32 low-level hooks (`WH_KEYBOARD_LL`/`WH_MOUSE_LL`) — reactor
exposes no raw key/pointer events; native Windows VK + absolute mouse (client-rect Contain-fit) +
wheel, Ctrl+Alt+Shift+Q capture toggle. `--headless`/`--discover` keep CLI paths. Builds + clippy
+ fmt green on **`x86_64-pc-windows-msvc` and `aarch64-pc-windows-msvc`** — the latter
**cross-compiled off the one x64 runner** (no ARM64 runner; the x64 MSVC toolset's ARM64 cross
compiler + a per-arch `FFMPEG_DIR` ARM64 tree, SDL3/libopus build-from-source cross-compile
cleanly), and both ship as signed MSIX (`windows-msix.yml` matrix → `..._x64.msix`/`..._arm64.msix`,
verified: ARM64 binaries + manifest arch). **windows-reactor is unpublished** (git
dep pinned to commit `b4129fcc`; `windows` pinned to the SAME commit so `IDXGISwapChain1` unifies
with `set_swap_chain`); its `build.rs` downloads the Win App SDK NuGets + needs `CARGO_WORKSPACE_DIR`
set (in the VM build env; `/temp`+`/winmd` gitignored). Gotcha: `CARGO_HOME` must be an ASCII path
— the `ü` in the dev box's username breaks SDL3's MSVC precompiled-header build. Next: **on-glass
validation** of the D3D11VA decode + HDR present + GUI on the RTX box (the dev VM is
headless/Session-0/WARP → the WinUI window + hardware decode need a real display+GPU: RDP or the
RTX box), then RAWINPUT relative-mouse pointer-lock and a per-host speed test in the UI.
**Android stage 1 done** (`clients/android`, Kotlin app + `native/` Rust JNI core linking
`punktfunk-core`; phone + Android TV): NDK `AMediaCodec` hardware HEVC decode → `SurfaceView` incl.
**HDR10** (Main10/BT.2020 PQ) with low-latency tuning + a live stats HUD (`decode.rs`/`stats.rs`),
Opus/Oboe audio + mic uplink (`audio.rs`/`mic.rs`), gamepad input with rumble/HID feedback
(`feedback.rs`), **native `mdns-sd` mDNS discovery** (`discovery.rs`, polled over JNI — the same
browse the Linux/Windows clients use, replacing the flaky per-OEM `NsdManager`; Kotlin keeps only
the `MulticastLock` + permission UX), SPAKE2 PIN pairing + TOFU (Keystore identity +
known-host store), Compose UI (Connect/Settings/Stream) with D-pad/controller focus nav. Built for
`arm64-v8a` + `x86_64`; published to Google Play (Internal Testing) via `android.yml`
(`ci/play-upload.py`). Next: real-device gamepad/HDR live-verify, presenter/latency polish.
2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct
NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~24 ms
at high res).
3. **punktfunk/1 protocol growth.** **Done:** unified host (`serve --gamestream` runs GameStream + the
punktfunk/1 QUIC host in one process; bare `serve` is the secure native-only default — GameStream is
opt-in, trusted-LAN only, security-review #5/#9) with native pairing driven over the mgmt API /
web console (`mod native_pairing`: arm-on-demand → display PIN, paired-device list).
**Done:** PIN pairing is the default, host-gated — the host requires pairing and advertises
`pair=required` unless opted out with `--allow-tofu`/`--open` (then `pair=optional`, accepts
unpaired clients); clients render TOFU only for a `pair=optional` host and force re-pairing on a
fingerprint change. **Done:** concurrent sessions — the accept loop spawns each session
(`--max-concurrent`, default 4, an NVENC bound), each with its own virtual output + encoder, sharing
the host-lifetime input/audio/mic services (shared-desktop multi-view on kwin/mutter/wlroots).
**Done:** delegated pairing approval (§8b-1) — an unpaired device shows up as a pending request in
the web console, one click approves + pins it. Next (see roadmap): gamescope multi-user isolation
(per-session input/audio = independent desktops); §8b-2 peer-push approval from a paired device's
own app.
4. **GameStream host polish**: HDR/10-bit (needs HDR capture + metadata plumbing; `av1_nvenc
-highbitdepth 1` already encodes Main10 from 8-bit input on this box),
reconnect-at-new-mode robustness. AV1 negotiation and surround audio are implemented
and unit/live-capture tested — both still need a live Moonlight confirmation (select
AV1 in a stock client; a real 5.1/7.1 listen incl. FEC under loss).
Box one-time setup is complete: udev rule + `input` group (gamepads validated live),
gamescope 3.16.22 installed system-wide (no PATH override), gnome-shell installed (Mutter
backend validated live). All three compositor backends are live-validated.
## Build / test / run
```sh
cargo build --workspace # green on Linux and macOS
cargo test --workspace # unit + loopback + proptest + C ABI harness (~100 tests)
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
```
Generated artifacts are **checked in** and CI fails on drift: `include/punktfunk_core.h`
(cbindgen from `punktfunk-core/src/abi.rs`) and `api/openapi.json` (regenerate with
`cargo run -p punktfunk-host -- openapi > api/openapi.json`; spec lives in `mgmt.rs`).
CI is Gitea Actions (`.gitea/workflows/`, guide: docs-site `ci.md`): `ci.yml` runs the
workspace checks inside the `git.unom.io/unom/punktfunk-rust-ci` image plus web/docs-site
build+typecheck; `docker.yml` builds+pushes the web/docs/rust-ci images (host and native
clients are deliberately NOT containerized); `apple.yml` builds the xcframework and runs
`swift build`/`swift test` on the `macos-arm64` host-mode runner (home-mac-mini-1,
provisioned by `scripts/ci/setup-macos-runner.sh`). Per-client/host release workflows:
`deb.yml`/`rpm.yml`/`flatpak.yml` (Linux client), `android.yml` (Google Play), `windows-msix.yml`
(Windows client), `windows-host.yml` (Windows host installer), `release.yml` (Apple notarized DMG +
TestFlight), `decky.yml` (Steam Deck plugin); Windows builds run on a self-hosted Windows runner
(`home-windows-runner-1`, vmid 210, `windows-amd64:host` label). The runner is reproducible and
**owned by `unom/infra`**, not this repo, since it's shared across unom Windows projects going
forward: `unom/infra`'s `windows-runner/` Packer template bakes a generic Windows 11 template (OS
install + OpenSSH Server + VS Build Tools/NASM/CMake/LLVM + the act_runner/Node/rustup base, no
registration) on Proxmox once; `proxmox/windows-runner/` (Terraform, `bpg/proxmox`) full-clones it
(agent-based IP discovery, no pre-provisioned DHCP reservation needed) and registers the instance
over SSH remote-exec — the same bake-once/clone-fast split `proxmox/unom-1` uses for the Linux CI
host, just without a native Windows cloud-init (registration goes over `remote-exec`/SSH instead of
`initialization{}`; WinRM was tried first but is deprecated in OpenTofu, so this moved to SSH via
Windows' in-box OpenSSH Server). punktfunk layers its own extras on top of that generic runner:
`scripts/ci/provision-windows-wdk.ps1` (WDK + cargo-wdk for the UMDF drivers) and
`scripts/ci/provision-windows-punktfunk-extras.ps1` (FFmpeg x64/ARM64 trees, Inno Setup, the
`aarch64-pc-windows-msvc` rustup target) — both idempotent, and both run automatically at the start
of every Windows CI job via the shared `scripts/ci/ensure-windows-toolchain.ps1` step (a fast no-op
once already provisioned), rather than a separate manually-dispatched provisioning workflow — that
avoided a real footgun once there could be more than one `windows-amd64` runner: a manually
dispatched provisioning workflow has no way to target a *specific* runner instance, so it could
land on an already-provisioned box instead of the one that actually needed it.
## Layout
```
crates/punktfunk-core/ protocol · FEC · crypto · quic (punktfunk/1 control plane, feature-gated)
crates/punktfunk-host/
gamestream/ Moonlight compat: nvhttp · pairing · rtsp · control · stream · gamepad · apps
vdisplay/{kwin,gamescope,mutter,wlroots}.rs per-compositor client-sized virtual outputs
zerocopy/{egl,cuda,vulkan}.rs dmabuf → CUDA → NVENC (tiled via EGL/GL, LINEAR via Vulkan)
inject/{libei,wlr,gamepad,dualsense}.rs input backends (uinput xpad + UHID DualSense)
encode/{nvenc,linux,vaapi,ffmpeg_win,sw}.rs per-GPU encoders (NVENC · Linux NVENC/CUDA · VAAPI · AMF/QSV · openh264)
capture.rs · encode.rs · audio.rs · spike.rs · punktfunk1.rs · mgmt.rs · native_pairing.rs · stats_recorder.rs
clients/probe/ punktfunk/1 reference/probe client (headless test/measurement tool)
clients/linux/ native Linux client (GTK4/libadwaita · FFmpeg · PipeWire · SDL3)
clients/windows/ native Windows client (WinUI 3 via windows-reactor · D3D11 · WASAPI · SDL3)
clients/apple/ native macOS/iOS/tvOS client (Swift · VideoToolbox · GameController)
clients/android/ native Android client (Kotlin app + native/ Rust JNI core over punktfunk-core)
clients/decky/ Steam Deck Decky plugin
crates/punktfunk-host/src/{capture/dxgi,vdisplay/sudovda,encode/ffmpeg_win,inject/gamepad_windows,audio/wasapi_*,service}.rs Windows host backends
web/ TanStack web console over the mgmt API (status · devices · pairing · performance graphs)
packaging/ apt(deb) · RPM/COPR · Arch/sysext · Flatpak · Bazzite bootc · Windows host installer (per-dir READMEs)
tools/{loss-harness,latency-probe}/ measurement (plan §10)
scripts/ 60-punktfunk.rules · punktfunk-host.service · host.env.example · headless/
include/punktfunk_core.h generated C header
```
## Design invariants — do not regress
- **One core, linked everywhere.** Protocol/FEC/crypto live only in `punktfunk-core`, behind a
stable, versioned C ABI. `tokio`/`quinn` exist only behind the `quic` feature (control
plane); **no async on the per-frame path** — native threads only.
- **Native client resolution, no scaling.** A session gets a virtual output at exactly the
client's WxH@Hz via the `VirtualDisplay` trait (`create(mode) → VirtualOutput { node_id,
remote_fd, preferred_mode, keepalive }`, RAII teardown). There is no cross-compositor
protocol for this — each compositor keeps its own backend.
- **FEC is the wall-breaker.** GF(2⁸) (≤255 shards/block, Moonlight-compatible) and GF(2¹⁶)
Leopard (≤65535 shards/block) — punktfunk/1 negotiates the latter, removing the ~1 Gbps
ceiling.
- **Core security hardening stays intact**: reassembler bounds attacker-controlled fields
before allocating (`ReassemblerLimits`); AES-GCM per-direction nonce salts + seq-as-AAD;
ABI `struct_size` checks. Regression tests exist — keep them green.
- **PipeWire consumer discipline**: our capture streams set `node.dont-reconnect` and tear
down promptly on negotiation timeout — one wedged link head-blocks the daemon's shared
work queue system-wide.
## Running on this box
Headless QEMU VM (Ubuntu 26.04, kernel 7.0), passthrough RTX 5070 Ti (driver 595 **open**
module — a kernel update silently drops it; reinstall `nvidia-driver-595-open`), no KMS
scanout → KWin `--drm` impossible; everything renders offscreen via `renderD128`.
```sh
# compositor session (shell 1, or the systemd unit in scripts/): full headless Plasma.
# The script sets XDG_MENU_PREFIX=plasma- & co. — without it plasmashell runs but the
# launcher menu is EMPTY (no apps, no System Settings).
bash scripts/headless/run-headless-kde.sh 1920x1080
# host (shell 2): bare `serve` is native-only (secure default); add --gamestream for Moonlight compat.
WAYLAND_DISPLAY=wayland-kde XDG_CURRENT_DESKTOP=KDE PUNKTFUNK_VIDEO_SOURCE=virtual \
PUNKTFUNK_ZEROCOPY=1 cargo run -rp punktfunk-host -- serve --gamestream
# punktfunk/1 native loopback test (no Moonlight needed; same env as serve, listener persists
# across sessions — bound it with --max-sessions):
cargo run -rp punktfunk-host -- punktfunk1-host --source virtual --seconds 10 --max-sessions 1
cargo run -rp punktfunk-probe -- --mode 1280x720x120 --out /tmp/a.h265 --input-test # + --pin HEX
```
Pinned crate facts: `ashpd` 0.13 + `pipewire` 0.9 (must match ashpd's) + `ffmpeg-next` 8.x
(`ffmpeg-sys-next` auto-detects the system FFmpeg, so it builds against **FFmpeg 7.x/libavcodec 61
or 8.x/libavcodec 62** — validated live on Ubuntu 26.04 (8) and Bazzite F43 (7.1); the zero-copy
FFI also link-needs `libGL`/`libgbm`/`libcuda` at build time). Env knobs: `PUNKTFUNK_VIDEO_SOURCE=virtual|portal`,
`PUNKTFUNK_COMPOSITOR=kwin|gamescope|mutter`, `PUNKTFUNK_ZEROCOPY=1`, `PUNKTFUNK_GAMESCOPE_APP=...`,
`PUNKTFUNK_INPUT_BACKEND=...`, `PUNKTFUNK_PERF=1` (per-stage timing), `PUNKTFUNK_VIDEO_DROP=N` (FEC
test), `PUNKTFUNK_FEC_PCT=N`, `PUNKTFUNK_DSCP=1` (opt-in DSCP/SO_PRIORITY media QoS on the data +
GameStream video/audio sockets; no-op on the wire on Windows without a qWAVE policy),
`PUNKTFUNK_444=1` (full-chroma HEVC 4:4:4, see below).
**HEVC 4:4:4 (full chroma, Range Extensions)**: opt-in via `PUNKTFUNK_444`, negotiated like 10-bit —
the host emits 4:4:4 only when the client advertised `VIDEO_CAP_444` (wire bit `0x04` + ABI
`PUNKTFUNK_VIDEO_CAP_444`), the codec is HEVC, the session is single-process, **and** a GPU probe
(`encode::can_encode_444`, run before the Welcome) confirms support — else it resolves to 4:2:0 and
`Welcome::chroma_format` reflects the real value (honest downgrade; the client reads it via
`punktfunk_connection_chroma_format`). **punktfunk/1-native only** — GameStream/Moonlight stays 4:2:0
(stock clients can't decode 4:4:4). **NVENC is the implemented path**: Linux `hevc_nvenc` feeds a
swscale'd `yuv444p` (RGB-in is always 4:2:0 — verified on the RTX 5070 Ti — so the session forces CPU
RGB capture for 4:4:4); Windows NVENC keeps ARGB input + FREXT profile + `chromaFormatIDC=3` and the
DDA capturer delivers RGB. VAAPI / AMF / QSV **decline** (probe returns false — no validated 4:4:4
hardware in the lab; they'd produce 4:2:0). Software (openh264) is 4:2:0-only. Test with
`PUNKTFUNK_CLIENT_444=1 punktfunk-probe --out x.h265` then `ffprobe x.h265` (expect `pix_fmt yuv444p`).
*Linux NVENC mechanism validated on the RTX 5070 Ti (ffmpeg CLI); Windows NVENC + 10-bit-4:4:4 not yet
on-glass validated.*
## Conventions
- Rust 2021, `rustfmt` + `clippy -D warnings` clean before commit.
- Match the surrounding code's comment density and naming.
- Commit messages end with the Co-Authored-By trailer (see `git log`).
- `pkill` caution on this box: match exact comm names (`pkill -x gamescope-wl`,
`pkill -x punktfunk-host`) — `pkill -f` self-matches the invoking shell.
Generated
+250 -48
View File
@@ -228,6 +228,67 @@ dependencies = [
"pin-project-lite",
]
[[package]]
name = "async-executor"
version = "1.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c96bf972d85afc50bf5ab8fe2d54d1586b4e0b46c97c50a0c9e71e2f7bcd812a"
dependencies = [
"async-task",
"concurrent-queue",
"fastrand",
"futures-lite",
"pin-project-lite",
"slab",
]
[[package]]
name = "async-io"
version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc"
dependencies = [
"autocfg",
"cfg-if",
"concurrent-queue",
"futures-io",
"futures-lite",
"parking",
"polling",
"rustix",
"slab",
"windows-sys 0.61.2",
]
[[package]]
name = "async-lock"
version = "3.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "290f7f2596bd5b78a9fec8088ccd89180d7f9f55b94b0576823bbbdc72ee8311"
dependencies = [
"event-listener",
"event-listener-strategy",
"pin-project-lite",
]
[[package]]
name = "async-process"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc50921ec0055cdd8a16de48773bfeec5c972598674347252c0399676be7da75"
dependencies = [
"async-channel",
"async-io",
"async-lock",
"async-signal",
"async-task",
"blocking",
"cfg-if",
"event-listener",
"futures-lite",
"rustix",
]
[[package]]
name = "async-recursion"
version = "1.1.1"
@@ -239,6 +300,30 @@ dependencies = [
"syn",
]
[[package]]
name = "async-signal"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52b5aaafa020cf5053a01f2a60e8ff5dccf550f0f77ec54a4e47285ac2bab485"
dependencies = [
"async-io",
"async-lock",
"atomic-waker",
"cfg-if",
"futures-core",
"futures-io",
"rustix",
"signal-hook-registry",
"slab",
"windows-sys 0.61.2",
]
[[package]]
name = "async-task"
version = "4.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de"
[[package]]
name = "async-trait"
version = "0.1.89"
@@ -434,6 +519,19 @@ dependencies = [
"generic-array",
]
[[package]]
name = "blocking"
version = "1.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21"
dependencies = [
"async-channel",
"async-task",
"futures-io",
"futures-lite",
"piper",
]
[[package]]
name = "bumpalo"
version = "3.20.3"
@@ -770,6 +868,15 @@ dependencies = [
"itertools 0.10.5",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.6"
@@ -1993,9 +2100,26 @@ dependencies = [
"libloading",
]
[[package]]
name = "ksni"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da9eeb3f510b6148ae68f963af2c1fbb0de4d9e4e05f82813cfb319837c3ad2b"
dependencies = [
"async-executor",
"async-io",
"async-lock",
"futures-channel",
"futures-lite",
"futures-util",
"pastey",
"serde",
"zbus",
]
[[package]]
name = "latency-probe"
version = "0.4.2"
version = "0.7.0"
[[package]]
name = "lazy_static"
@@ -2127,7 +2251,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.4.2"
version = "0.7.0"
dependencies = [
"punktfunk-core",
]
@@ -2552,6 +2676,12 @@ version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
[[package]]
name = "pastey"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4"
[[package]]
name = "pem"
version = "3.0.6"
@@ -2590,6 +2720,17 @@ version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "piper"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1"
dependencies = [
"atomic-waker",
"fastrand",
"futures-io",
]
[[package]]
name = "pipewire"
version = "0.9.2"
@@ -2645,6 +2786,20 @@ version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "polling"
version = "3.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218"
dependencies = [
"cfg-if",
"concurrent-queue",
"hermit-abi",
"pin-project-lite",
"rustix",
"windows-sys 0.61.2",
]
[[package]]
name = "polyval"
version = "0.6.2"
@@ -2720,7 +2875,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.4.2"
version = "0.7.0"
dependencies = [
"android_logger",
"jni",
@@ -2734,7 +2889,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.4.2"
version = "0.7.0"
dependencies = [
"anyhow",
"async-channel",
@@ -2745,19 +2900,22 @@ dependencies = [
"opus",
"pipewire",
"punktfunk-core",
"rustls",
"sdl3",
"serde",
"serde_json",
"tracing",
"tracing-subscriber",
"ureq",
]
[[package]]
name = "punktfunk-client-windows"
version = "0.4.2"
version = "0.7.0"
dependencies = [
"anyhow",
"async-channel",
"crossbeam-channel",
"ffmpeg-next",
"mdns-sd",
"opus",
@@ -2768,13 +2926,15 @@ dependencies = [
"tracing",
"tracing-subscriber",
"wasapi",
"windows 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-reactor",
"windows-reactor-setup",
"winresource",
]
[[package]]
name = "punktfunk-core"
version = "0.4.2"
version = "0.7.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -2804,7 +2964,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.4.2"
version = "0.7.0"
dependencies = [
"aes",
"aes-gcm",
@@ -2825,6 +2985,7 @@ dependencies = [
"khronos-egl",
"libc",
"libloading",
"log",
"mdns-sd",
"nvidia-video-codec-sdk",
"openh264",
@@ -2849,6 +3010,7 @@ dependencies = [
"tokio-rustls",
"tower",
"tracing",
"tracing-log",
"tracing-subscriber",
"ureq",
"usbip-sim",
@@ -2865,13 +3027,14 @@ dependencies = [
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service",
"winreg",
"winresource",
"x509-parser",
"xkbcommon",
]
[[package]]
name = "punktfunk-probe"
version = "0.4.2"
version = "0.7.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -2883,6 +3046,23 @@ dependencies = [
"tracing-subscriber",
]
[[package]]
name = "punktfunk-tray"
version = "0.7.0"
dependencies = [
"anyhow",
"ksni",
"libc",
"rustls",
"serde",
"serde_json",
"sha2",
"ureq",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service",
"winresource",
]
[[package]]
name = "quick-error"
version = "1.2.3"
@@ -4609,12 +4789,12 @@ dependencies = [
[[package]]
name = "windows"
version = "0.62.2"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-collections 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-future 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-numerics 0.3.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-collections 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-future 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-numerics 0.3.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-reference",
"windows-time",
]
@@ -4631,9 +4811,9 @@ dependencies = [
[[package]]
name = "windows-collections"
version = "0.3.2"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
@@ -4652,13 +4832,13 @@ dependencies = [
[[package]]
name = "windows-core"
version = "0.62.2"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-implement 0.60.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-interface 0.59.3 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-result 0.4.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-strings 0.5.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-implement 0.60.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-interface 0.59.3 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-result 0.4.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-strings 0.5.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
@@ -4675,11 +4855,11 @@ dependencies = [
[[package]]
name = "windows-future"
version = "0.3.2"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-threading 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-threading 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
@@ -4696,7 +4876,7 @@ dependencies = [
[[package]]
name = "windows-implement"
version = "0.60.2"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"proc-macro2",
"quote",
@@ -4717,7 +4897,7 @@ dependencies = [
[[package]]
name = "windows-interface"
version = "0.59.3"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"proc-macro2",
"quote",
@@ -4733,7 +4913,7 @@ checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
[[package]]
name = "windows-numerics"
@@ -4748,33 +4928,38 @@ dependencies = [
[[package]]
name = "windows-numerics"
version = "0.3.1"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
name = "windows-reactor"
version = "0.0.0"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"rustc-hash",
"windows-collections 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-future 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-numerics 0.3.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-collections 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-future 0.3.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-numerics 0.3.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-reference",
"windows-threading 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-threading 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-time",
]
[[package]]
name = "windows-reactor-setup"
version = "0.0.0"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
[[package]]
name = "windows-reference"
version = "0.1.0"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-time",
]
@@ -4790,9 +4975,9 @@ dependencies = [
[[package]]
name = "windows-result"
version = "0.4.1"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
@@ -4818,9 +5003,9 @@ dependencies = [
[[package]]
name = "windows-strings"
version = "0.5.1"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
@@ -4928,17 +5113,18 @@ dependencies = [
[[package]]
name = "windows-threading"
version = "0.2.1"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
name = "windows-time"
version = "0.1.0"
source = "git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1#b4129fcc1ae81eec8bf1217539883db821bca3a1"
source = "git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f#a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f"
dependencies = [
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=b4129fcc1ae81eec8bf1217539883db821bca3a1)",
"windows-core 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
"windows-link 0.2.1 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
@@ -5104,6 +5290,16 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "winresource"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0986a8b1d586b7d3e4fe3d9ea39fb451ae22869dcea4aa109d287a374d866087"
dependencies = [
"toml 1.1.2+spec-1.1.0",
"version_check",
]
[[package]]
name = "wit-bindgen"
version = "0.57.1"
@@ -5189,8 +5385,14 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eee682d202a77e4a9f3b2c2bdf48a7b28af5c08c34ddf66f98c93e5e39464285"
dependencies = [
"async-broadcast",
"async-executor",
"async-io",
"async-lock",
"async-process",
"async-recursion",
"async-task",
"async-trait",
"blocking",
"enumflags2",
"event-listener",
"futures-core",
+2 -1
View File
@@ -4,6 +4,7 @@ members = [
"crates/punktfunk-core",
"crates/punktfunk-host",
"crates/punktfunk-host/vendor/usbip-sim",
"crates/punktfunk-tray",
"crates/pf-driver-proto",
"clients/probe",
"clients/linux",
@@ -16,7 +17,7 @@ members = [
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package]
version = "0.4.2"
version = "0.7.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+14 -10
View File
@@ -15,6 +15,9 @@ your local network.
💬 **Community: [Discord](https://discord.gg/kaPNvzMuGU)** — chat, support, and **Android beta
access** · **[r/Punktfunk](https://www.reddit.com/r/Punktfunk/)**.
🔒 **Security:** found a vulnerability? Report it privately to **security@punktfunk.com** — see
[SECURITY.md](SECURITY.md). Please don't open a public issue.
punktfunk pairs a **virtual-display streaming host** with native clients on every platform. It speaks
the existing **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) client works
day one — and adds its own faster **`punktfunk/1`** protocol that breaks the ~1 Gbps FEC wall with a
@@ -33,7 +36,9 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
a screen — tight, push-based integration that's unusual for a Windows streaming host.
- **Low latency, GPU end to end.** Frames go straight from the compositor to the NVENC encoder with
zero CPU copies (dmabuf → CUDA/Vulkan → NVENC), over a transport tuned for responsiveness rather
than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on a LAN.
than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on-box,
~1.3 ms cross-machine on a LAN. (AMD/Intel encode via VAAPI, and a GPU-less software H.264
encoder exists as a fallback.)
- **Works with what you already have.** Any Moonlight/Artemis client connects over GameStream — and
native apps for macOS, Linux, Windows, and Android use the lower-latency `punktfunk/1` protocol.
- **Secure by default.** Hosts require a one-time SPAKE2 **PIN pairing**; after that, devices
@@ -47,19 +52,19 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
| **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened |
| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
| **Native protocol**`punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| **Windows host** (x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **Windows host** (Windows 11 22H2+, x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test |
| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch |
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, Oboe audio, controllers, discovery, pairing |
| **Windows client** (`clients/windows`, WinUI 3) | 🟡 Stage 1 complete, ships as signed MSIX (x64 + ARM64); D3D11VA decode + HDR present pending on-glass validation |
| **Web console + management API** (`web/`) | ✅ TanStack console over the OpenAPI mgmt API: host status, paired devices, on-demand PIN pairing |
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, AAudio audio, controllers, discovery, pairing |
| **Windows client** (`clients/windows`, WinUI 3) | Streaming live: D3D11VA hardware decode on all GPU vendors (NVIDIA + Intel validated on glass) with software fallback, WASAPI audio, SDL3 controllers, discovery, pairing; ships as signed MSIX (x64 + ARM64). HDR10 implemented, on-glass validation pending |
| **Web console + management API** (`web/`) | ✅ TanStack console over the OpenAPI mgmt API: host status, paired devices, on-demand PIN pairing, GPU selection, performance capture graphs, live host logs |
The **GameStream host works with a stock Moonlight client** — validated live on NVIDIA hardware
(RTX 5070 Ti, RTX 4090): PIN pairing that persists across restarts, an app catalog, RTSP/ENet/audio,
and **video at the client's exact resolution and refresh** via a per-session virtual output (KWin,
gamescope, Mutter, and Sway/wlroots backends), encoded with GPU **zero-copy** (dmabuf → CUDA/Vulkan →
NVENC) up to 5120×1440@240. The native **`punktfunk/1`** protocol adds a QUIC control plane and a
GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→reassembled at 720p120), with
GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→received at 720p120), with
mid-stream mode renegotiation and a wall-clock skew handshake so latency stays valid across machines.
Both run from **one process**: bare `punktfunk-host serve` is the **secure native-only default**
(`punktfunk/1` + the management API/web console), and `serve --gamestream` additionally enables the
@@ -80,7 +85,7 @@ Windows host also ships as a signed installer (all-vendor: NVIDIA, AMD, Intel).
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [Ubuntu — GNOME](https://docs.punktfunk.unom.io/docs/ubuntu-gnome) · [KDE](https://docs.punktfunk.unom.io/docs/ubuntu-kde) |
| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(or the bootc image)* | [Fedora — KDE](https://docs.punktfunk.unom.io/docs/fedora-kde) · [Bazzite](https://docs.punktfunk.unom.io/docs/bazzite) |
| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS)* | [packaging/arch](packaging/arch/README.md) |
| **Windows** (x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
| **Windows** (11 22H2+, x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status).
After install, run `punktfunk-host serve` inside your desktop session (the secure native default;
@@ -130,13 +135,12 @@ clients/
apple/ macOS / iOS / tvOS app (Swift · VideoToolbox · Metal · GameController)
linux/ Linux desktop app (Rust · GTK4/libadwaita · FFmpeg/VAAPI · PipeWire · SDL3)
windows/ Windows desktop app (Rust · WinUI 3 · D3D11 · WASAPI · SDL3)
android/ Android phone + TV app (Kotlin · Rust JNI core · AMediaCodec · Oboe)
android/ Android phone + TV app (Kotlin · Rust JNI core · AMediaCodec · AAudio)
probe/ headless reference / measurement client for punktfunk/1
decky/ Steam Deck Decky plugin
web/ web console (TanStack) over the management API — status · devices · pairing
web/ web console (TanStack) over the management API — status · devices · pairing · GPUs · performance · logs
packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image
docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io
design/ design notes & deep-dive plans (index: design/README.md)
include/punktfunk_core.h cbindgen-generated C header (checked in)
tools/ latency-probe · loss-harness (measurement)
```
+69
View File
@@ -0,0 +1,69 @@
# Security Policy
punktfunk is a low-latency desktop/game streaming stack. A host is effectively remote control of a
machine, so we take security reports seriously and appreciate responsible disclosure.
## Reporting a vulnerability
**Please report security issues privately by email to security@punktfunk.com.**
Do **not** open a public issue, pull request, or chat/forum post for a suspected vulnerability — that
exposes other users before a fix exists.
### What to include
The more of this you can give us, the faster we can act:
- The component and version (e.g. `punktfunk-host 0.6.0`, Windows or Linux, which client).
- The impact — what an attacker can do, and from what position (same LAN, a local service account,
admin, a paired client, …).
- Steps to reproduce, a proof-of-concept, or a crash/log if you have one.
- Any suggested fix or mitigation (optional).
## What to expect
We're a small team, so timelines are best-effort, but we commit to:
- **Acknowledge** your report within **3 business days**.
- Give an **initial assessment** (severity + whether we can reproduce) within about **7 days**.
- Keep you updated, and tell you when a fix ships.
- **Credit** you in the advisory / release notes when the fix is public — unless you'd rather stay
anonymous.
We practice **coordinated disclosure**: please give us reasonable time to release a fix before
publishing details. We aim to resolve valid issues within **90 days** and will agree a disclosure
date with you.
## Scope
In scope — the code in this repository:
- The host (`punktfunk-host`), its Windows drivers, and the protocol/crypto core (`punktfunk-core`).
- The native clients (Apple, Linux, Windows, Android), the web management console, and the management
API.
Known limits — documented behavior, not vulnerabilities (see
https://docs.punktfunk.unom.io/docs/security):
- **Admin/SYSTEM already on the host = out of scope.** An attacker who is already administrator or
SYSTEM on the host owns the machine regardless of punktfunk.
- **The virtual display is a real monitor** — any process already in the interactive desktop session
can capture it via the normal OS screen-capture APIs, exactly as it could a physical monitor.
- **GameStream/Moonlight compatibility** (`--gamestream`) uses legacy encryption and is documented as
opt-in, trusted-LAN-only.
- **Public-internet exposure is unsupported** — issues that only arise from exposing the host to the
WAN are expected; keep the host on a trusted LAN or a VPN.
If you're unsure whether something is in scope, report it anyway — we'd rather hear about it.
## Safe harbor
We consider good-faith security research that follows this policy to be authorized, and we won't
pursue legal action against researchers who:
- make a good-faith effort to avoid privacy violations, data loss, and service disruption,
- only test systems they own or have explicit permission to test,
- give us reasonable time to remediate before public disclosure,
- don't exfiltrate more data than needed to demonstrate the issue.
Thank you for helping keep punktfunk and its users safe.
+501 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.4.2"
"version": "0.6.0"
},
"paths": {
"/api/v1/clients": {
@@ -138,6 +138,100 @@
}
}
},
"/api/v1/gpus": {
"get": {
"tags": [
"gpu"
],
"summary": "GPU inventory and selection",
"description": "Lists the host's hardware GPUs, the persisted auto/manual preference, the GPU the next session\nwill use (and why), and the GPU live sessions encode on right now.",
"operationId": "listGpus",
"responses": {
"200": {
"description": "GPU inventory + selection state",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/GpuState"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/gpus/preference": {
"put": {
"tags": [
"gpu"
],
"summary": "Set the GPU preference",
"description": "`auto` restores automatic selection (`PUNKTFUNK_RENDER_ADAPTER` pin, else max dedicated VRAM);\n`manual` pins capture + encode to the given GPU. Persisted across restarts; applies to the\n**next** session (a running session keeps its GPU). If the preferred GPU is absent at session\nstart the host falls back to automatic selection rather than failing.",
"operationId": "setGpuPreference",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SetGpuPreference"
}
}
},
"required": true
},
"responses": {
"200": {
"description": "Preference stored; the new selection state",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/GpuState"
}
}
}
},
"400": {
"description": "Unknown mode, or `gpu_id` missing / not a listed GPU",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Preference could not be persisted",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/health": {
"get": {
"tags": [
@@ -484,6 +578,97 @@
}
}
},
"/api/v1/local/summary": {
"get": {
"tags": [
"host"
],
"summary": "Local status summary for the tray icon",
"description": "Non-sensitive status (counts and booleans only — no PIN values, no fingerprints, no device\nnames). Unauthenticated, but served to loopback peers only.",
"operationId": "getLocalSummary",
"responses": {
"200": {
"description": "Non-sensitive local host status (loopback peers only)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/LocalSummary"
}
}
}
},
"401": {
"description": "Non-loopback peer",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
},
"security": [
{}
]
}
},
"/api/v1/logs": {
"get": {
"tags": [
"logs"
],
"summary": "Host logs",
"description": "The host's recent log entries — an in-memory ring of the newest few thousand, captured at\nDEBUG and above regardless of `RUST_LOG`. Follow live by polling with `after` set to the last\nresponse's `next` cursor; a `dropped: true` means entries were evicted between polls (the ring\nwrapped). Bearer-only: logs can reference client identities and host paths, so this is part of\nthe loopback-only admin surface, never the LAN-readable mTLS one.",
"operationId": "logsGet",
"parameters": [
{
"name": "after",
"in": "query",
"description": "Return entries with seq greater than this (omitted/0 = oldest retained)",
"required": false,
"schema": {
"type": "integer",
"format": "int64",
"minimum": 0
}
},
{
"name": "limit",
"in": "query",
"description": "Max entries per response (default and cap 1000)",
"required": false,
"schema": {
"type": "integer",
"format": "int32",
"minimum": 0
}
}
],
"responses": {
"200": {
"description": "Entries after the cursor, oldest first",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/LogPage"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/native/clients": {
"get": {
"tags": [
@@ -1373,6 +1558,40 @@
},
"components": {
"schemas": {
"ApiActiveGpu": {
"type": "object",
"description": "The GPU live sessions are encoding on right now.",
"required": [
"id",
"name",
"vendor",
"backend",
"sessions"
],
"properties": {
"backend": {
"type": "string",
"description": "The encode backend in use (`nvenc` | `amf` | `qsv` | `vaapi` | `software`)."
},
"id": {
"type": "string",
"description": "Stable id matching an entry of `gpus` (empty for the CPU/software encoder)."
},
"name": {
"type": "string"
},
"sessions": {
"type": "integer",
"format": "int32",
"description": "Number of live encode sessions on it.",
"minimum": 0
},
"vendor": {
"type": "string",
"description": "`nvidia` | `amd` | `intel` | `other`."
}
}
},
"ApiCodec": {
"type": "string",
"description": "Video codec identifier.",
@@ -1394,6 +1613,64 @@
}
}
},
"ApiGpu": {
"type": "object",
"description": "One hardware GPU on the host (software/WARP adapters are never listed).",
"required": [
"id",
"name",
"vendor",
"vram_mb"
],
"properties": {
"id": {
"type": "string",
"description": "Stable identifier (`vendorid-deviceid-occurrence`, hex PCI ids) — pass to `setGpuPreference`.\nStable across reboots and driver updates, unlike an adapter index or LUID.",
"example": "10de-2c05-0"
},
"name": {
"type": "string",
"description": "Adapter/marketing name.",
"example": "NVIDIA GeForce RTX 5070 Ti"
},
"vendor": {
"type": "string",
"description": "`nvidia` | `amd` | `intel` | `other`."
},
"vram_mb": {
"type": "integer",
"format": "int64",
"description": "Dedicated VRAM in MiB (0 where the platform doesn't expose it).",
"minimum": 0
}
}
},
"ApiSelectedGpu": {
"type": "object",
"description": "The GPU the **next** session's pipeline will be created on, and why. (A preference change\napplies to the next session; a running session keeps the GPU it opened on.)",
"required": [
"id",
"name",
"vendor",
"source"
],
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
},
"source": {
"type": "string",
"description": "Why this GPU was selected: `preference` (the manual choice), `env`\n(`PUNKTFUNK_RENDER_ADAPTER`), `auto` (max dedicated VRAM / platform default), or\n`preference_missing` (a manual choice is set but that GPU is absent — auto-selected\ninstead so the host keeps streaming)."
},
"vendor": {
"type": "string",
"description": "`nvidia` | `amd` | `intel` | `other`."
}
}
},
"ApprovePending": {
"type": "object",
"description": "Approve-pending-device request body. Send `{}` to keep the device's own name.",
@@ -1671,6 +1948,75 @@
}
}
},
"GpuState": {
"type": "object",
"description": "Full GPU-selection state for the console: inventory, the persisted preference, what the next\nsession will use, and what is in use right now.",
"required": [
"gpus",
"mode",
"preferred_available"
],
"properties": {
"active": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/ApiActiveGpu",
"description": "The GPU live sessions use right now (absent while nothing is streaming)."
}
]
},
"env_override": {
"type": [
"string",
"null"
],
"description": "`PUNKTFUNK_RENDER_ADAPTER` (the host.env pin), when set — it applies while `mode` is\n`auto`; a manual preference overrides it."
},
"gpus": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ApiGpu"
},
"description": "The host's hardware GPUs."
},
"mode": {
"type": "string",
"description": "`auto` or `manual`."
},
"preferred_available": {
"type": "boolean",
"description": "Whether the preferred GPU is currently present."
},
"preferred_id": {
"type": [
"string",
"null"
],
"description": "The manually preferred GPU's stable id, when one is stored (kept while `mode` is `auto` so\na console can offer returning to it). May reference a GPU that is currently absent."
},
"preferred_name": {
"type": [
"string",
"null"
],
"description": "The stored name of the preferred GPU (a usable label even when it is absent)."
},
"selected": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/ApiSelectedGpu",
"description": "The GPU the next session will use."
}
]
}
}
},
"Health": {
"type": "object",
"description": "Liveness + version probe.",
@@ -1772,6 +2118,130 @@
}
}
},
"LocalSummary": {
"type": "object",
"description": "Non-sensitive host status for the local tray icon: counts and booleans only — no PIN values,\nno fingerprints, no device names. Served unauthenticated to LOOPBACK peers only (see\n`require_auth`): the bearer-token file is SYSTEM/Administrators-DACL'd on Windows, so the\nper-user tray process cannot authenticate — this narrow read-only route is its status source.",
"required": [
"version",
"video_streaming",
"audio_streaming",
"paired_clients",
"native_paired_clients",
"pin_pending",
"pending_approvals"
],
"properties": {
"audio_streaming": {
"type": "boolean",
"description": "True while the audio stream thread is running."
},
"native_paired_clients": {
"type": "integer",
"format": "int32",
"description": "Number of paired native (punktfunk/1) devices.",
"minimum": 0
},
"paired_clients": {
"type": "integer",
"format": "int32",
"description": "Number of pinned (paired) GameStream client certificates.",
"minimum": 0
},
"pending_approvals": {
"type": "integer",
"format": "int32",
"description": "Native pairing knocks awaiting the operator's approval (count only).",
"minimum": 0
},
"pin_pending": {
"type": "boolean",
"description": "True while a GameStream pairing handshake is parked waiting for the user's PIN."
},
"session": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/SessionInfo",
"description": "The active launch session (set by Moonlight's `/launch`, cleared on cancel/stop)."
}
]
},
"version": {
"type": "string",
"description": "Host version (mirrors `/health`)."
},
"video_streaming": {
"type": "boolean",
"description": "True while the video stream thread is running."
}
}
},
"LogEntry": {
"type": "object",
"description": "One captured log event.",
"required": [
"seq",
"ts_ms",
"level",
"target",
"msg"
],
"properties": {
"level": {
"type": "string",
"description": "`ERROR` | `WARN` | `INFO` | `DEBUG` | `TRACE`."
},
"msg": {
"type": "string",
"description": "The formatted message, structured fields appended as `key=value`."
},
"seq": {
"type": "integer",
"format": "int64",
"description": "Monotonic sequence number (1-based) — pass the last one back as the `after` cursor.",
"minimum": 0
},
"target": {
"type": "string",
"description": "The emitting module path (tracing target)."
},
"ts_ms": {
"type": "integer",
"format": "int64",
"description": "Unix timestamp in milliseconds.",
"minimum": 0
}
}
},
"LogPage": {
"type": "object",
"description": "One poll's worth of log entries.",
"required": [
"entries",
"next",
"dropped"
],
"properties": {
"dropped": {
"type": "boolean",
"description": "True when entries between `after` and the first returned one were already evicted."
},
"entries": {
"type": "array",
"items": {
"$ref": "#/components/schemas/LogEntry"
}
},
"next": {
"type": "integer",
"format": "int64",
"description": "Cursor for the next poll (the last returned seq, or the request's `after` when empty).",
"minimum": 0
}
}
},
"NativeClient": {
"type": "object",
"description": "A paired native (punktfunk/1) client.",
@@ -2047,6 +2517,28 @@
}
}
},
"SetGpuPreference": {
"type": "object",
"description": "Request body for `setGpuPreference`.",
"required": [
"mode"
],
"properties": {
"gpu_id": {
"type": [
"string",
"null"
],
"description": "Required when `mode` is `manual`: the stable `id` of a currently listed GPU\n(see `listGpus`).",
"example": "10de-2c05-0"
},
"mode": {
"type": "string",
"description": "`auto` (env pin, else max dedicated VRAM — the default) or `manual`.",
"example": "manual"
}
}
},
"StageTiming": {
"type": "object",
"description": "One pipeline stage's latency in an aggregation window (microseconds).",
@@ -2267,6 +2759,10 @@
"name": "host",
"description": "Host identity, capabilities, and liveness"
},
{
"name": "gpu",
"description": "GPU inventory and selection: list the host's GPUs, choose automatic or a preferred GPU, see the one in use"
},
{
"name": "clients",
"description": "Paired Moonlight client management"
@@ -2290,6 +2786,10 @@
{
"name": "stats",
"description": "Streaming performance-stats capture: arm/stop a recording, read the live + saved time-series for graphing"
},
{
"name": "logs",
"description": "Host log stream: the newest in-memory log entries, cursor-paged for live following"
}
]
}
+55 -59
View File
@@ -1,83 +1,79 @@
# punktfunk Android client
# punktfunk Android client (phone & TV)
Native Android client for **punktfunk/1**, targeting **phone + TV** (Compose, D-pad + touch).
The native **Android** app for streaming a punktfunk host to your phone, tablet, or Android TV. A
Compose app that finds hosts on your network, pairs with a PIN, and streams at the display's own
resolution — with hardware HEVC decode, HDR10, and controller support, built for both touch and the
couch (D-pad / gamepad focus navigation).
## Architecture — Rust-heavy (like the Linux client, not thin-native like Apple)
## Features
Kotlin cannot `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable.
We write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux
- **Hardware decode** — NDK `AMediaCodec` HEVC → `SurfaceView`, including **HDR10** (Main10 /
BT.2020 PQ), with low-latency tuning and a live stats HUD.
- **Audio both ways** — Opus + AAudio playback with a jitter ring, plus mic uplink to the host.
- **Controller support** — buttons + axes with rumble and HID feedback (lightbar / adaptive
triggers); D-pad / gamepad focus navigation for TV and phone.
- **Find hosts automatically** — native mDNS discovery; first connect does a one-time **SPAKE2 PIN
pairing** (or TOFU on trusted LANs), then reconnects on a Keystore-wrapped, pinned identity.
- **Compose UI** — Connect / Settings / Stream screens with Material You theming.
Built for `arm64-v8a` + `x86_64`.
## Get it
Published to **Google Play (Internal Testing)** — join the beta via the
[Discord](https://discord.gg/kaPNvzMuGU). Per-device setup and pairing:
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
## How it's built — Rust-heavy
Kotlin can't `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable. We
write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux
client's orchestration (audio jitter ring, VK keymap inverse, latency/skew math, capture state
machine, trust logic) instead of re-porting it into Kotlin.
| Side | Owns |
|------|------|
| **Rust** (`clients/android/native``libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB`AMediaCodec` decode, Opus+Oboe audio, VK keymap, latency math, trust/pairing, **mDNS discovery** (`mdns-sd`, the same browse the Linux/Windows clients use) |
| **Kotlin** (`clients/android`) | Compose UI (host grid / settings / stream), `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity, permissions |
| **Rust** (`native/``libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB`AMediaCodec` decode (incl. HDR10), Opus + AAudio audio + mic, controller feedback, latency math, trust/pairing, `mdns-sd` discovery |
| **Kotlin** (`app/`, `kit/`) | Compose UI, `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity |
The single seam is `io.unom.punktfunk.kit.NativeBridge``Java_io_unom_punktfunk_kit_NativeBridge_*`.
## Layout
```
clients/android/native/ Rust cdylib (workspace member) — links punktfunk-core directly
src/lib.rs JNI seam (connect/pair, input, plane getters, abi/core version)
src/session.rs session lifecycle + plane pumps
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink (jitter ring)
src/feedback.rs rumble + HID output (lightbar / adaptive triggers)
src/stats.rs live video stats
clients/android/ Gradle project (this dir)
settings.gradle.kts · build.gradle.kts · gradle.properties · gradlew
app/ :app — Compose UI: Connect / Settings / Stream screens (phone + TV)
kit/ :kit — NativeBridge · discovery (native mdns-sd, polled) · Gamepad · Keymap ·
security (Keystore identity + known-host store) · cargo-ndk build
native/ Rust cdylib (workspace member) — links punktfunk-core directly
src/lib.rs crate doc · JNI_OnLoad · version probes
src/session/ session lifecycle: connect/pair + trust, plane start/stop, input shims
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
src/audio.rs · src/mic.rs Opus + AAudio playback / mic uplink
src/feedback.rs · src/stats.rs rumble + HID feedback; live video stats
src/discovery.rs native mdns-sd browse of the host's _punktfunk._udp advert
app/ :app — Compose UI: Connect / Settings / Stream (phone + TV)
kit/ :kit — NativeBridge · native mDNS discovery · Gamepad · Keymap · Keystore identity
```
## Prerequisites
- Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`, `build-tools;37.0.0`,
**`cmake;3.22.1`** (`sdkmanager "cmake;3.22.1"` — the `cmake` crate builds libopus with it)
- **JDK 21** for Gradle/AGP (AGP 9.2 runs on JDK 1721, *not* a newer default JDK like 25)
- Rust + `rustup target add aarch64-linux-android x86_64-linux-android` + `cargo install cargo-ndk`
Toolchain pinned: AGP 9.2.0 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM 2026.05.01 ·
compileSdk 37 · targetSdk 36 · minSdk 31 · ABIs arm64-v8a + x86_64.
## Build & run
**Android Studio:** open `clients/android` — it uses its bundled JBR 21 automatically. The
`cargoNdk*` task builds the `.so` as part of the normal build.
**Prerequisites:** Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`,
`build-tools;37.0.0`, **`cmake;3.22.1`** (builds libopus); **JDK 21** (AGP 9.2 runs on JDK 1721, not
a newer default); Rust with `rustup target add aarch64-linux-android x86_64-linux-android` and
`cargo install cargo-ndk`. Toolchain is pinned (AGP 9.2 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM
2026.05.01 · compileSdk 37 · minSdk 31).
**CLI** (point Gradle at a JDK 21 if your machine default is newer, e.g. JDK 25):
**Android Studio:** open `clients/android` — it uses its bundled JBR 21, and the `cargoNdk*` task
builds the `.so` as part of the normal build.
**CLI** (point Gradle at JDK 21 if your machine default is newer):
```sh
# Adoptium/Temurin 21 (installed by the Android Studio setup, or `brew install temurin@21`):
export JAVA_HOME="$(/usr/libexec/java_home -v 21)"
export JAVA_HOME="$(/usr/libexec/java_home -v 21)" # or your Temurin 21 path
cd clients/android
./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first
./gradlew :app:installDebug # onto a running emulator/device
# Emulators (created during env setup): emulator -avd pf_phone | emulator -avd pf_tv
./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first
./gradlew :app:installDebug # onto a running emulator/device
# emulators from env setup: emulator -avd pf_phone | emulator -avd pf_tv
```
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host from the list, pair,
and stream.
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host, pair, and stream.
## Status
## Related
A working native client (phone + Android TV), at parity with the Linux and Apple apps for the core
streaming experience:
- **Video** — `AMediaCodec` hardware HEVC decode → `SurfaceView`, including **HDR10** (Main10 /
BT.2020 PQ), with low-latency decode tuning and a live stats HUD.
- **Audio** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
- **Input** — game controllers (buttons + axes) with rumble and HID feedback; D-pad /
game-controller focus navigation for the couch (TV + phone).
- **Discovery & trust** — native `mdns-sd` mDNS host list (polled over JNI; the same browse the
Linux/Windows clients use, not `NsdManager`), SPAKE2 PIN pairing and TOFU, with a
Keystore-wrapped client identity and a known-host store.
- **UI** — Compose host list / settings / stream screens, Material You theming.
- **Shipping** — built for `arm64-v8a` + `x86_64`; published to Google Play (Internal Testing).
`crates/punktfunk-core` uses the `ring` `rcgen` backend so the client `.so` is aws-lc-free.
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
@@ -27,8 +27,15 @@
<uses-feature android:name="android.software.leanback" android:required="false" />
<uses-feature android:name="android.hardware.gamepad" android:required="false" />
<!-- appCategory="game": a game-streaming client IS a game as far as the SoC is concerned.
On Snapdragon devices (and other OEMs with a Game Mode / Game Dashboard) this makes the app
eligible for the vendor's game performance profile — the aggressive CPU/GPU governor and
scheduler treatment games get — which, together with the ADPF hints in the native decode
path, is what keeps clocks up for low, consistent decode latency. Also groups it correctly
under Games in battery/data usage. Advisory: devices without Game Mode ignore it. -->
<application
android:allowBackup="false"
android:appCategory="game"
android:icon="@mipmap/ic_launcher"
android:roundIcon="@mipmap/ic_launcher_round"
android:label="@string/app_name"
@@ -0,0 +1,355 @@
package io.unom.punktfunk
import android.os.Build
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.ModalBottomSheet
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.material3.rememberModalBottomSheetState
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.security.ClientIdentity
import io.unom.punktfunk.kit.security.KnownHost
import io.unom.punktfunk.models.PendingTrust
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
/**
* The "Add host" bottom sheet: optional name + address + port, then connect at [modeLabel]. Field
* state stays hoisted in ConnectScreen so a dismissed sheet keeps its half-typed values.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
internal fun AddHostSheet(
hostName: String,
onHostNameChange: (String) -> Unit,
host: String,
onHostChange: (String) -> Unit,
port: String,
onPortChange: (String) -> Unit,
connecting: Boolean,
modeLabel: String,
onDismiss: () -> Unit,
onConnect: (host: String, port: Int, name: String) -> Unit,
) {
val scope = rememberCoroutineScope()
val sheetState = rememberModalBottomSheetState()
ModalBottomSheet(
onDismissRequest = onDismiss,
sheetState = sheetState,
) {
Column(
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 24.dp)
.padding(bottom = 32.dp),
) {
Text("Add a host", style = MaterialTheme.typography.titleLarge)
Spacer(Modifier.height(4.dp))
Text(
"Enter its address. You'll pair with the host's PIN on first connect.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(20.dp))
OutlinedTextField(
value = hostName,
onValueChange = onHostNameChange,
label = { Text("Name (optional)") },
placeholder = { Text("e.g. Living Room") },
singleLine = true,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(16.dp))
OutlinedTextField(
value = host,
onValueChange = onHostChange,
label = { Text("Host") },
singleLine = true,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(16.dp))
OutlinedTextField(
value = port,
onValueChange = { v -> onPortChange(v.filter { it.isDigit() }.take(5)) },
label = { Text("Port") },
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(20.dp))
Button(
enabled = !connecting && host.isNotBlank() && port.isNotBlank(),
onClick = {
val h = host.trim()
val p = port.toIntOrNull() ?: 9777
val n = hostName
scope.launch { sheetState.hide() }.invokeOnCompletion {
onDismiss()
onConnect(h, p, n)
}
},
modifier = Modifier.fillMaxWidth(),
) { Text("Connect ($modeLabel)") }
}
}
}
/** First connection to a host that advertised pair=optional: offer TOFU, but pitch PIN pairing. */
@Composable
internal fun TrustNewHostDialog(
pt: PendingTrust,
onTrust: () -> Unit,
onPairInstead: () -> Unit,
onDismiss: () -> Unit,
) {
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Trust this host?") },
text = {
Column {
Text("First connection to ${pt.host}:${pt.port}.")
pt.advertisedFp?.let { Text("Fingerprint ${it.take(16)}") }
Text(
"This host allows trust-on-first-use, but that can't tell an impostor " +
"from the real host. Pairing with a PIN is stronger — it proves both sides.",
)
}
},
confirmButton = {
TextButton(onClick = onTrust) { Text("Trust (TOFU)") }
},
dismissButton = {
Row {
TextButton(onClick = onPairInstead) { Text("Pair with PIN…") }
TextButton(onClick = onDismiss) { Text("Cancel") }
}
},
)
}
/** The pinned fingerprint no longer matches — force re-pairing (never a silent re-trust). */
@Composable
internal fun FingerprintChangedDialog(
pt: PendingTrust,
onRepair: () -> Unit,
onDismiss: () -> Unit,
) {
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Host identity changed") },
text = {
Text(
"The pinned fingerprint for ${pt.host} no longer matches what it now " +
"advertises. This can mean a host reinstall — or an impostor. Re-pair " +
"with the host's PIN to continue.",
)
},
confirmButton = {
TextButton(onClick = onRepair) { Text("Re-pair") }
},
dismissButton = {
TextButton(onClick = onDismiss) { Text("Cancel") }
},
)
}
/**
* A fresh pair=required (or manual/unknown-policy) host: offer the two ways in. "Request access" is
* the no-PIN path — connect and wait for the operator to click Approve in the host's console;
* "Use a PIN…" switches to the SPAKE2 ceremony.
*/
@Composable
internal fun RequestAccessDialog(
pt: PendingTrust,
onRequestAccess: () -> Unit,
onUsePin: () -> Unit,
onDismiss: () -> Unit,
) {
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Pairing required") },
text = {
Column {
Text("${pt.host}:${pt.port} requires pairing before it will stream.")
Text(
"Request access and approve this device in the host's console (or web " +
"UI) — no PIN needed. Or pair with the 4-digit PIN the host displays.",
)
}
},
confirmButton = {
TextButton(onClick = onRequestAccess) { Text("Request access") }
},
dismissButton = {
Row {
TextButton(onClick = onUsePin) { Text("Use a PIN…") }
TextButton(onClick = onDismiss) { Text("Cancel") }
}
},
)
}
/**
* The SPAKE2 PIN ceremony dialog. Runs [NativeBridge.nativePair] off the UI thread itself (the
* pin/name/error state is dialog-local); on success hands the host's verified fingerprint to
* [onPaired], which saves + connects. Dismissal is blocked while a pair attempt is in flight.
*/
@Composable
internal fun PairPinDialog(
pt: PendingTrust,
identity: ClientIdentity?,
onPaired: (fpHex: String) -> Unit,
onDismiss: () -> Unit,
) {
val scope = rememberCoroutineScope()
var pin by remember(pt) { mutableStateOf("") }
var name by remember(pt) { mutableStateOf(Build.MODEL ?: "Android") }
var pairing by remember(pt) { mutableStateOf(false) }
var err by remember(pt) { mutableStateOf<String?>(null) }
AlertDialog(
onDismissRequest = { if (!pairing) onDismiss() },
title = { Text("Pair with PIN") },
text = {
Column {
Text("Enter the 4-digit PIN shown on the host.")
OutlinedTextField(
value = pin,
onValueChange = { v -> pin = v.filter { it.isDigit() }.take(4) },
label = { Text("PIN") },
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
)
OutlinedTextField(
value = name,
onValueChange = { name = it },
label = { Text("This device") },
singleLine = true,
)
err?.let { Text(it, color = MaterialTheme.colorScheme.error) }
}
},
confirmButton = {
TextButton(
enabled = !pairing && pin.length == 4 && identity != null,
onClick = {
val id = identity
if (id != null) {
pairing = true
err = null
scope.launch {
val fp = withContext(Dispatchers.IO) {
NativeBridge.nativePair(
pt.host, pt.port, id.certPem, id.privateKeyPem, pin, name,
)
}
pairing = false
if (fp.isNotEmpty()) {
onPaired(fp) // verified host fp — caller saves + connects
} else {
err = "Pairing failed — wrong PIN, or the host isn't armed."
}
}
}
},
) { Text(if (pairing) "Pairing…" else "Pair") }
},
dismissButton = {
TextButton(enabled = !pairing, onClick = onDismiss) { Text("Cancel") }
},
)
}
/**
* The no-PIN "request access" wait: the connect is parked on the host until the operator approves
* this device. Cancel returns the UI immediately — the caller trips the per-attempt flag so a late
* approval is torn down silently (see ConnectScreen.requestAccess) and resumes discovery.
*/
@Composable
internal fun AwaitingApprovalDialog(hostLabel: String, onCancel: () -> Unit) {
AlertDialog(
onDismissRequest = onCancel,
title = { Text("Waiting for approval") },
text = {
val deviceName = Build.MODEL ?: "this device"
Column(verticalArrangement = Arrangement.spacedBy(12.dp)) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
CircularProgressIndicator(modifier = Modifier.size(20.dp), strokeWidth = 2.dp)
Text("Approve this device on $hostLabel.")
}
Text(
"Open the host's console (or web UI) and approve “$deviceName”. It connects " +
"automatically once you approve — no PIN needed.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
},
confirmButton = {},
dismissButton = {
TextButton(onClick = onCancel) { Text("Cancel") }
},
)
}
/**
* Rename a saved host's label (discovered hosts are named by mDNS; this is how you give one a
* friendly name like "Living Room" after pairing). Keyed by the host so reopening resets the field.
*/
@Composable
internal fun RenameHostDialog(
target: KnownHost,
onRename: (String) -> Unit,
onDismiss: () -> Unit,
) {
var newName by remember(target) { mutableStateOf(target.name) }
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Rename host") },
text = {
OutlinedTextField(
value = newName,
onValueChange = { newName = it },
label = { Text("Name") },
placeholder = { Text(target.address) },
singleLine = true,
)
},
confirmButton = {
TextButton(
enabled = newName.isNotBlank(),
onClick = { onRename(newName.trim()) },
) { Text("Save") }
},
dismissButton = {
TextButton(onClick = onDismiss) { Text("Cancel") }
},
)
}
@@ -6,11 +6,6 @@ import android.content.pm.PackageManager
import android.os.Build
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.scaleIn
import androidx.compose.animation.scaleOut
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
@@ -27,24 +22,14 @@ import androidx.compose.foundation.lazy.grid.GridCells
import androidx.compose.foundation.lazy.grid.GridItemSpan
import androidx.compose.foundation.lazy.grid.LazyVerticalGrid
import androidx.compose.foundation.lazy.grid.items
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Add
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.ExtendedFloatingActionButton
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.ModalBottomSheet
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.material3.rememberModalBottomSheetState
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
@@ -56,7 +41,6 @@ import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
@@ -99,7 +83,6 @@ private class RequestAccessState(val target: PendingTrust) {
val cancelled = AtomicBoolean(false)
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
val scope = rememberCoroutineScope()
@@ -162,6 +145,26 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
// it survives a DHCP address change; else by address:port). Mirrors the Apple client.
val discoveredUnsaved = discovered.filter { dh -> savedHosts.none { it.matches(dh) } }
// The one place the full nativeConnect is issued (shared by the normal connect and the
// request-access path), including the HDR/gamepad derivation both need.
suspend fun connectNative(id: ClientIdentity, targetHost: String, targetPort: Int, pinHex: String, timeoutMs: Int): Long {
// Advertise HDR only when the user enabled it AND this device's display can present it
// (else the host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID
// (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An
// explicit choice is passed through unchanged.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
return withContext(Dispatchers.IO) {
NativeBridge.nativeConnect(
targetHost, targetPort, w, h, hz,
id.certPem, id.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
)
}
}
// Issue the actual connect with identity + (optional) pin. On a TOFU connect (pinHex null),
// pin the fingerprint the host presented (as an unpaired known host) so the next connect goes
// straight through and it appears in the saved-hosts list.
@@ -175,21 +178,7 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
status = "Connecting to $targetHost:$targetPort"
discovery.stop() // free the Wi-Fi radio before the stream session
scope.launch {
// Advertise HDR only when the user enabled it AND this device's display can present it
// (else the host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID
// (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An
// explicit choice is passed through unchanged.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
val handle = withContext(Dispatchers.IO) {
NativeBridge.nativeConnect(
targetHost, targetPort, w, h, hz,
id.certPem, id.privateKeyPem, pinHex ?: "",
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, CONNECT_TIMEOUT_MS,
)
}
val handle = connectNative(id, targetHost, targetPort, pinHex ?: "", CONNECT_TIMEOUT_MS)
connecting = false
if (handle != 0L) {
if (pinHex == null) { // TOFU: pin what we observed (unpaired)
@@ -224,19 +213,10 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
status = null
discovery.stop() // free the Wi-Fi radio before the (parked) stream session
scope.launch {
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
// Pin the advertised fingerprint for a discovered host (defence against an impostor while
// we wait); a manually-typed host has none, so trust-on-first-use.
val pinHex = target.advertisedFp ?: ""
val handle = withContext(Dispatchers.IO) {
NativeBridge.nativeConnect(
target.host, target.port, w, h, hz,
id.certPem, id.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, REQUEST_ACCESS_TIMEOUT_MS,
)
}
val handle = connectNative(id, target.host, target.port, pinHex, REQUEST_ACCESS_TIMEOUT_MS)
// Cancelled while we were parked: tear the (possibly just-approved) session down and
// don't touch UI a fresh action may now own.
if (req.cancelled.get()) {
@@ -295,7 +275,6 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
}
}
val sheetState = rememberModalBottomSheetState()
var showManualSheet by remember { mutableStateOf(false) }
Box(Modifier.fillMaxSize()) {
@@ -427,291 +406,87 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
}
}
AnimatedVisibility(
visible = true, // Static for now, could be based on scroll if needed
enter = scaleIn() + fadeIn(),
exit = scaleOut() + fadeOut(),
ExtendedFloatingActionButton(
onClick = { showManualSheet = true },
icon = { Icon(Icons.Filled.Add, contentDescription = null) },
text = { Text("Add host") },
expanded = !connecting,
modifier = Modifier
.align(Alignment.BottomEnd)
.padding(20.dp)
) {
ExtendedFloatingActionButton(
onClick = { showManualSheet = true },
icon = { Icon(Icons.Filled.Add, contentDescription = null) },
text = { Text("Add host") },
expanded = !connecting,
)
}
.padding(20.dp),
)
}
if (showManualSheet) {
ModalBottomSheet(
onDismissRequest = { showManualSheet = false },
sheetState = sheetState,
) {
Column(
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 24.dp)
.padding(bottom = 32.dp),
) {
Text("Add a host", style = MaterialTheme.typography.titleLarge)
Spacer(Modifier.height(4.dp))
Text(
"Enter its address. You'll pair with the host's PIN on first connect.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(20.dp))
OutlinedTextField(
value = hostName,
onValueChange = { hostName = it },
label = { Text("Name (optional)") },
placeholder = { Text("e.g. Living Room") },
singleLine = true,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(16.dp))
OutlinedTextField(
value = host,
onValueChange = { host = it },
label = { Text("Host") },
singleLine = true,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(16.dp))
OutlinedTextField(
value = port,
onValueChange = { v -> port = v.filter { it.isDigit() }.take(5) },
label = { Text("Port") },
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(20.dp))
Button(
enabled = !connecting && host.isNotBlank() && port.isNotBlank(),
onClick = {
val h = host.trim()
val p = port.toIntOrNull() ?: 9777
val n = hostName
scope.launch { sheetState.hide() }.invokeOnCompletion {
showManualSheet = false
connect(h, p, manualName = n)
}
},
modifier = Modifier.fillMaxWidth(),
) { Text("Connect ($w×$h@$hz)") }
}
}
AddHostSheet(
hostName = hostName,
onHostNameChange = { hostName = it },
host = host,
onHostChange = { host = it },
port = port,
onPortChange = { port = it },
connecting = connecting,
modeLabel = "$w×$h@$hz",
onDismiss = { showManualSheet = false },
onConnect = { h2, p, n -> connect(h2, p, manualName = n) },
)
}
pendingTrust?.let { pt ->
when (pt.kind) {
PendingTrust.Kind.TRUST_NEW -> AlertDialog(
onDismissRequest = { pendingTrust = null },
title = { Text("Trust this host?") },
text = {
Column {
Text("First connection to ${pt.host}:${pt.port}.")
pt.advertisedFp?.let { Text("Fingerprint ${it.take(16)}") }
Text(
"This host allows trust-on-first-use, but that can't tell an impostor " +
"from the real host. Pairing with a PIN is stronger — it proves both sides.",
)
}
},
confirmButton = {
TextButton({ pendingTrust = null; doConnect(pt.host, pt.port, pt.name, null) }) {
Text("Trust (TOFU)")
}
},
dismissButton = {
Row {
TextButton({ pendingTrust = pt.copy(kind = PendingTrust.Kind.PAIR) }) {
Text("Pair with PIN…")
}
TextButton({ pendingTrust = null }) { Text("Cancel") }
}
},
PendingTrust.Kind.TRUST_NEW -> TrustNewHostDialog(
pt = pt,
onTrust = { pendingTrust = null; doConnect(pt.host, pt.port, pt.name, null) },
onPairInstead = { pendingTrust = pt.copy(kind = PendingTrust.Kind.PAIR) },
onDismiss = { pendingTrust = null },
)
PendingTrust.Kind.FP_CHANGED -> AlertDialog(
onDismissRequest = { pendingTrust = null },
title = { Text("Host identity changed") },
text = {
Text(
"The pinned fingerprint for ${pt.host} no longer matches what it now " +
"advertises. This can mean a host reinstall — or an impostor. Re-pair " +
"with the host's PIN to continue.",
)
},
confirmButton = {
TextButton({ pendingTrust = pt.copy(kind = PendingTrust.Kind.PAIR) }) { Text("Re-pair") }
},
dismissButton = {
TextButton({ pendingTrust = null }) { Text("Cancel") }
},
PendingTrust.Kind.FP_CHANGED -> FingerprintChangedDialog(
pt = pt,
onRepair = { pendingTrust = pt.copy(kind = PendingTrust.Kind.PAIR) },
onDismiss = { pendingTrust = null },
)
// A fresh pair=required (or manual/unknown-policy) host: offer the two ways in. "Request
// access" is the no-PIN path — connect and wait for the operator to click Approve in the
// host's console; "Use a PIN…" switches to the SPAKE2 ceremony.
PendingTrust.Kind.REQUEST_ACCESS -> AlertDialog(
onDismissRequest = { pendingTrust = null },
title = { Text("Pairing required") },
text = {
Column {
Text("${pt.host}:${pt.port} requires pairing before it will stream.")
Text(
"Request access and approve this device in the host's console (or web " +
"UI) — no PIN needed. Or pair with the 4-digit PIN the host displays.",
)
}
},
confirmButton = {
TextButton({ pendingTrust = null; requestAccess(pt) }) { Text("Request access") }
},
dismissButton = {
Row {
TextButton({ pendingTrust = pt.copy(kind = PendingTrust.Kind.PAIR) }) {
Text("Use a PIN…")
}
TextButton({ pendingTrust = null }) { Text("Cancel") }
}
},
PendingTrust.Kind.REQUEST_ACCESS -> RequestAccessDialog(
pt = pt,
onRequestAccess = { pendingTrust = null; requestAccess(pt) },
onUsePin = { pendingTrust = pt.copy(kind = PendingTrust.Kind.PAIR) },
onDismiss = { pendingTrust = null },
)
PendingTrust.Kind.PAIR -> PairPinDialog(
pt = pt,
identity = identity,
onPaired = { fp ->
// Verified host fp — save as a paired known host, then connect pinned.
knownHostStore.save(KnownHost(pt.host, pt.port, pt.name, fp, paired = true))
savedHosts = knownHostStore.all()
pendingTrust = null
doConnect(pt.host, pt.port, pt.name, fp)
},
onDismiss = { pendingTrust = null },
)
PendingTrust.Kind.PAIR -> {
var pin by remember(pt) { mutableStateOf("") }
var name by remember(pt) { mutableStateOf(Build.MODEL ?: "Android") }
var pairing by remember(pt) { mutableStateOf(false) }
var err by remember(pt) { mutableStateOf<String?>(null) }
AlertDialog(
onDismissRequest = { if (!pairing) pendingTrust = null },
title = { Text("Pair with PIN") },
text = {
Column {
Text("Enter the 4-digit PIN shown on the host.")
OutlinedTextField(
value = pin,
onValueChange = { v -> pin = v.filter { it.isDigit() }.take(4) },
label = { Text("PIN") },
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number),
)
OutlinedTextField(
value = name,
onValueChange = { name = it },
label = { Text("This device") },
singleLine = true,
)
err?.let { Text(it, color = MaterialTheme.colorScheme.error) }
}
},
confirmButton = {
TextButton(
enabled = !pairing && pin.length == 4 && identity != null,
onClick = {
val id = identity
if (id != null) {
pairing = true
err = null
scope.launch {
val fp = withContext(Dispatchers.IO) {
NativeBridge.nativePair(
pt.host, pt.port, id.certPem, id.privateKeyPem, pin, name,
)
}
pairing = false
if (fp.isNotEmpty()) {
// Verified host fp — save as a paired known host.
knownHostStore.save(
KnownHost(pt.host, pt.port, pt.name, fp, paired = true),
)
savedHosts = knownHostStore.all()
pendingTrust = null
doConnect(pt.host, pt.port, pt.name, fp)
} else {
err = "Pairing failed — wrong PIN, or the host isn't armed."
}
}
}
},
) { Text(if (pairing) "Pairing…" else "Pair") }
},
dismissButton = {
TextButton(enabled = !pairing, onClick = { pendingTrust = null }) { Text("Cancel") }
},
)
}
}
}
// The no-PIN "request access" wait: the connect is parked on the host until the operator
// approves this device. Cancel returns the UI immediately — it trips the per-attempt flag so a
// late approval is torn down silently (see requestAccess) and resumes discovery.
awaiting?.let { req ->
fun cancel() {
req.cancelled.set(true)
awaiting = null
connecting = false
discovery.start() // the request may still be pending on the host; keep scanning
}
AlertDialog(
onDismissRequest = { cancel() },
title = { Text("Waiting for approval") },
text = {
val deviceName = Build.MODEL ?: "this device"
Column(verticalArrangement = Arrangement.spacedBy(12.dp)) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
CircularProgressIndicator(modifier = Modifier.size(20.dp), strokeWidth = 2.dp)
Text("Approve this device on ${req.target.name}.")
}
Text(
"Open the host's console (or web UI) and approve “$deviceName”. It connects " +
"automatically once you approve — no PIN needed.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
},
confirmButton = {},
dismissButton = {
TextButton(onClick = { cancel() }) { Text("Cancel") }
AwaitingApprovalDialog(
hostLabel = req.target.name,
onCancel = {
req.cancelled.set(true)
awaiting = null
connecting = false
discovery.start() // the request may still be pending on the host; keep scanning
},
)
}
// Rename a saved host's label (discovered hosts are named by mDNS; this is how you give one a
// friendly name like "Living Room" after pairing). Keyed by the host so reopening resets the field.
renameTarget?.let { kh ->
var newName by remember(kh) { mutableStateOf(kh.name) }
AlertDialog(
onDismissRequest = { renameTarget = null },
title = { Text("Rename host") },
text = {
OutlinedTextField(
value = newName,
onValueChange = { newName = it },
label = { Text("Name") },
placeholder = { Text(kh.address) },
singleLine = true,
)
},
confirmButton = {
TextButton(
enabled = newName.isNotBlank(),
onClick = {
knownHostStore.rename(kh.address, kh.port, newName.trim())
savedHosts = knownHostStore.all()
renameTarget = null
},
) { Text("Save") }
},
dismissButton = {
TextButton(onClick = { renameTarget = null }) { Text("Cancel") }
RenameHostDialog(
target = kh,
onRename = { newName ->
knownHostStore.rename(kh.address, kh.port, newName)
savedHosts = knownHostStore.all()
renameTarget = null
},
onDismiss = { renameTarget = null },
)
}
}
@@ -0,0 +1,382 @@
package io.unom.punktfunk
import android.hardware.input.InputManager
import android.os.CombinedVibration
import android.os.Handler
import android.os.Looper
import android.os.VibrationEffect
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ColumnScope
import androidx.compose.foundation.layout.ExperimentalLayoutApi
import androidx.compose.foundation.layout.FlowRow
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.OutlinedCard
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableIntStateOf
import androidx.compose.runtime.mutableStateMapOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import io.unom.punktfunk.kit.Gamepad
import kotlinx.coroutines.delay
/**
* Connected-controllers debug view (Settings → Host → Connected controllers): everything the app
* can see about attached input devices, plus a live input test. This exists for exactly the support
* case where a pad "doesn't work" — adapters and BT-to-USB dongles often enumerate with a different
* identity than the physical pad, or not as a gamepad at all, and punktfunk only forwards devices
* Android classifies as gamepad/joystick. This screen makes that visible on the device itself.
*/
@Composable
fun ControllersScreen(gamepadSetting: Int, onBack: () -> Unit) {
BackHandler(onBack = onBack)
val context = LocalContext.current
val activity = context as? MainActivity
// Device list, re-read on every hot-plug event.
var generation by remember { mutableIntStateOf(0) }
val pads = remember(generation) { Gamepad.pads() }
val others = remember(generation) {
InputDevice.getDeviceIds()
.toList()
.mapNotNull { InputDevice.getDevice(it) }
.filter { !it.isVirtual && !Gamepad.isPad(it) }
}
DisposableEffect(Unit) {
val im = context.getSystemService(InputManager::class.java)
val listener = object : InputManager.InputDeviceListener {
override fun onInputDeviceAdded(deviceId: Int) { generation++ }
override fun onInputDeviceRemoved(deviceId: Int) { generation++ }
override fun onInputDeviceChanged(deviceId: Int) { generation++ }
}
im.registerInputDeviceListener(listener, Handler(Looper.getMainLooper()))
onDispose { im.unregisterInputDeviceListener(listener) }
}
// Live input test. While `testing`, the MainActivity probes consume pad events (so they show up
// here instead of driving focus navigation); holding B releases, since the pad can no longer
// reach the Switch. Events are observed (not consumed) even when the test is off, so the
// "last input" line works while browsing.
var testing by remember { mutableStateOf(false) }
val held = remember { mutableStateMapOf<Int, Boolean>() }
val axes = remember { mutableStateMapOf<String, Float>() }
var lastInput by remember { mutableStateOf<String?>(null) }
var bHeld by remember { mutableStateOf(false) }
DisposableEffect(Unit) {
activity?.padKeyProbe = probe@{ event ->
if (!Gamepad.isPad(event.device)) return@probe false
when (event.action) {
KeyEvent.ACTION_DOWN -> {
held[event.keyCode] = true
if (event.keyCode == KeyEvent.KEYCODE_BUTTON_B) bHeld = true
}
KeyEvent.ACTION_UP -> {
held[event.keyCode] = false
if (event.keyCode == KeyEvent.KEYCODE_BUTTON_B) bHeld = false
}
}
lastInput = "${event.device?.name}: ${KeyEvent.keyCodeToString(event.keyCode)}"
testing
}
activity?.padMotionProbe = probe@{ event ->
if (!Gamepad.isPad(event.device)) return@probe false
axes["LX"] = event.getAxisValue(MotionEvent.AXIS_X)
axes["LY"] = event.getAxisValue(MotionEvent.AXIS_Y)
axes["RX"] = event.getAxisValue(MotionEvent.AXIS_Z)
axes["RY"] = event.getAxisValue(MotionEvent.AXIS_RZ)
axes["LT"] = maxOf(
event.getAxisValue(MotionEvent.AXIS_LTRIGGER),
event.getAxisValue(MotionEvent.AXIS_BRAKE),
)
axes["RT"] = maxOf(
event.getAxisValue(MotionEvent.AXIS_RTRIGGER),
event.getAxisValue(MotionEvent.AXIS_GAS),
)
axes["HX"] = event.getAxisValue(MotionEvent.AXIS_HAT_X)
axes["HY"] = event.getAxisValue(MotionEvent.AXIS_HAT_Y)
testing
}
onDispose {
activity?.padKeyProbe = null
activity?.padMotionProbe = null
}
}
// Hold-B-to-exit: with events consumed, the pad can't reach the Switch — a 1.2 s hold ends the
// test instead (touch still works). A short tap cancels the effect before the delay fires.
LaunchedEffect(bHeld) {
if (bHeld && testing) {
delay(1_200)
testing = false
held.clear()
}
}
Column(
modifier = Modifier
.fillMaxSize()
.verticalScroll(rememberScrollState())
.padding(horizontal = 20.dp, vertical = 24.dp),
verticalArrangement = Arrangement.spacedBy(24.dp),
) {
Text("Controllers", style = MaterialTheme.typography.headlineMedium)
Group("Gamepads") {
if (pads.isEmpty()) {
Text(
"No controller detected. punktfunk can only forward devices Android " +
"classifies as a gamepad or joystick — a pad connected through an adapter " +
"or hub may show up under \"Other input devices\" below with the adapter's " +
"identity, or not at all.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
pads.forEachIndexed { i, dev ->
PadRow(dev, forwarded = i == 0, gamepadSetting = gamepadSetting)
}
}
Group("Input test") {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Column(Modifier.weight(1f)) {
Text("Test inputs", style = MaterialTheme.typography.bodyLarge)
Text(
if (testing) "Controller input stays on this screen — hold B to finish"
else "Show button presses and stick motion live",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Switch(checked = testing, onCheckedChange = { testing = it; if (!it) held.clear() })
}
if (testing) {
ButtonGrid(held)
AXIS_LABELS.forEach { label -> AxisBar(label, axes[label] ?: 0f) }
}
lastInput?.let {
Text(
"Last input — $it",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
Group("Other input devices") {
if (others.isEmpty()) {
Text(
"None",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
others.forEach { dev ->
Column {
Text(dev.name, style = MaterialTheme.typography.bodyMedium)
Text(
deviceDetail(dev),
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
}
}
/** One detected gamepad: identity, what it streams as, and a rumble test. */
@Composable
private fun PadRow(dev: InputDevice, forwarded: Boolean, gamepadSetting: Int) {
OutlinedCard(modifier = Modifier.fillMaxWidth()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(6.dp),
) {
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Text(dev.name, style = MaterialTheme.typography.bodyLarge, modifier = Modifier.weight(1f))
if (forwarded) {
Text(
"forwarded to host",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary,
)
}
}
Text(
deviceDetail(dev),
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
val resolved = Gamepad.prefFor(dev)
Text(
if (gamepadSetting == Gamepad.PREF_AUTO) {
"Streams as: ${prefLabel(resolved)} (automatic)"
} else {
"Streams as: ${prefLabel(gamepadSetting)} (set in Settings; " +
"automatic would pick ${prefLabel(resolved)})"
},
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
val canRumble = dev.vibratorManager.vibratorIds.isNotEmpty()
if (canRumble) {
OutlinedButton(onClick = { testRumble(dev) }) { Text("Test rumble") }
} else {
Text(
"No rumble motors reported — host rumble will be silent",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
}
/** The forwarded buttons as chips that light up while held. */
@OptIn(ExperimentalLayoutApi::class)
@Composable
private fun ButtonGrid(held: Map<Int, Boolean>) {
FlowRow(
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalArrangement = Arrangement.spacedBy(6.dp),
) {
TEST_BUTTONS.forEach { (label, keyCode) ->
val active = held[keyCode] == true
Text(
label,
style = MaterialTheme.typography.labelMedium,
color = if (active) MaterialTheme.colorScheme.onPrimary
else MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier
.background(
if (active) MaterialTheme.colorScheme.primary
else MaterialTheme.colorScheme.surfaceVariant,
RoundedCornerShape(6.dp),
)
.padding(horizontal = 10.dp, vertical = 6.dp),
)
}
}
}
/** A labelled live axis bar; sticks/HAT are 1..1 (centre = half), triggers 0..1. */
@Composable
private fun AxisBar(label: String, value: Float) {
val progress = if (label == "LT" || label == "RT") value else (value + 1f) / 2f
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Text(label, style = MaterialTheme.typography.labelMedium, modifier = Modifier.width(32.dp))
LinearProgressIndicator(
progress = { progress.coerceIn(0f, 1f) },
modifier = Modifier.weight(1f),
)
Text(
"%+.2f".format(value),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.padding(start = 8.dp),
)
}
}
/** A titled section — same look as the Settings groups. */
@Composable
private fun Group(title: String, content: @Composable ColumnScope.() -> Unit) {
Column(verticalArrangement = Arrangement.spacedBy(10.dp)) {
Text(
title,
style = MaterialTheme.typography.titleSmall,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.padding(start = 4.dp),
)
Column(verticalArrangement = Arrangement.spacedBy(12.dp), content = content)
}
}
private fun testRumble(dev: InputDevice) {
val vm = dev.vibratorManager
if (vm.vibratorIds.isEmpty()) return
runCatching {
vm.vibrate(CombinedVibration.createParallel(VibrationEffect.createOneShot(300, 200)))
}
}
/** Identity line: VID:PID + the source classes Android assigned. */
private fun deviceDetail(dev: InputDevice): String =
"%04X:%04X · %s".format(dev.vendorId, dev.productId, sourcesLabel(dev.sources))
private fun sourcesLabel(sources: Int): String {
fun has(flag: Int) = sources and flag == flag
val names = buildList {
if (has(InputDevice.SOURCE_GAMEPAD)) add("gamepad")
if (has(InputDevice.SOURCE_JOYSTICK)) add("joystick")
if (has(InputDevice.SOURCE_DPAD)) add("dpad")
if (has(InputDevice.SOURCE_KEYBOARD)) add("keyboard")
if (has(InputDevice.SOURCE_MOUSE)) add("mouse")
if (has(InputDevice.SOURCE_TOUCHSCREEN)) add("touchscreen")
if (has(InputDevice.SOURCE_TOUCHPAD)) add("touchpad")
if (has(InputDevice.SOURCE_STYLUS)) add("stylus")
if (has(InputDevice.SOURCE_ROTARY_ENCODER)) add("rotary")
}
return if (names.isEmpty()) "sources 0x%08X".format(sources) else names.joinToString(" · ")
}
/** [Gamepad] PREF_* wire byte → user-facing label (mirrors GAMEPAD_OPTIONS, plus the Steam types). */
private fun prefLabel(pref: Int): String = when (pref) {
Gamepad.PREF_XBOX360 -> "Xbox 360"
Gamepad.PREF_DUALSENSE -> "DualSense"
Gamepad.PREF_XBOXONE -> "Xbox One"
Gamepad.PREF_DUALSHOCK4 -> "DualShock 4"
Gamepad.PREF_STEAMCONTROLLER -> "Steam Controller"
Gamepad.PREF_STEAMDECK -> "Steam Deck"
else -> "Automatic"
}
/** Buttons shown in the test grid (label → Android keycode). */
private val TEST_BUTTONS = listOf(
"A" to KeyEvent.KEYCODE_BUTTON_A,
"B" to KeyEvent.KEYCODE_BUTTON_B,
"X" to KeyEvent.KEYCODE_BUTTON_X,
"Y" to KeyEvent.KEYCODE_BUTTON_Y,
"LB" to KeyEvent.KEYCODE_BUTTON_L1,
"RB" to KeyEvent.KEYCODE_BUTTON_R1,
"L2" to KeyEvent.KEYCODE_BUTTON_L2,
"R2" to KeyEvent.KEYCODE_BUTTON_R2,
"LS" to KeyEvent.KEYCODE_BUTTON_THUMBL,
"RS" to KeyEvent.KEYCODE_BUTTON_THUMBR,
"Select" to KeyEvent.KEYCODE_BUTTON_SELECT,
"Start" to KeyEvent.KEYCODE_BUTTON_START,
"Guide" to KeyEvent.KEYCODE_BUTTON_MODE,
"" to KeyEvent.KEYCODE_DPAD_UP,
"" to KeyEvent.KEYCODE_DPAD_DOWN,
"" to KeyEvent.KEYCODE_DPAD_LEFT,
"" to KeyEvent.KEYCODE_DPAD_RIGHT,
)
/** Axis bars shown in the test view, in display order. */
private val AXIS_LABELS = listOf("LX", "LY", "RX", "RY", "LT", "RT", "HX", "HY")
@@ -26,6 +26,14 @@ class MainActivity : ComponentActivity() {
/** Joystick-axis state mapper for the active session (built/reset by StreamScreen). */
var axisMapper: Gamepad.AxisMapper? = null
/**
* Input observers for the Controllers debug screen (set while it is shown, like [streamHandle]).
* Called for every key/motion event while not streaming; a `true` return consumes the event —
* the screen's "test inputs" mode uses that to keep pad input from also driving focus navigation.
*/
var padKeyProbe: ((KeyEvent) -> Boolean)? = null
var padMotionProbe: ((MotionEvent) -> Boolean)? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
// Dark, transparent system bars regardless of the system theme — our UI is always dark, so
@@ -72,23 +80,29 @@ class MainActivity : ComponentActivity() {
KeyEvent.ACTION_UP -> false
else -> return super.dispatchKeyEvent(event)
}
val vk = Keymap.toVk(event.keyCode)
// Full-event overload: evdev scancode first (positional under ANY selected
// physical-keyboard layout), keycode fallback — see Keymap docs.
val vk = Keymap.toVk(event)
if (vk != 0) {
NativeBridge.nativeSendKey(handle, vk, down, 0)
return true // consumed — don't let the system also act on it
}
}
}
} else if (event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
// Not streaming: a game controller drives the Compose UI (TV + phone). Map the face
// buttons to the navigation keys the focus system understands; D-pad *keys* already move
// focus on their own, so they fall through to super untouched.
val mapped = when (event.keyCode) {
KeyEvent.KEYCODE_BUTTON_A -> KeyEvent.KEYCODE_DPAD_CENTER // activate focused element
KeyEvent.KEYCODE_BUTTON_B -> KeyEvent.KEYCODE_BACK // back / dismiss
else -> 0
} else {
// The Controllers debug screen sees pad events before the navigation remap below.
padKeyProbe?.let { if (it(event)) return true }
if (event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
// Not streaming: a game controller drives the Compose UI (TV + phone). Map the face
// buttons to the navigation keys the focus system understands; D-pad *keys* already
// move focus on their own, so they fall through to super untouched.
val mapped = when (event.keyCode) {
KeyEvent.KEYCODE_BUTTON_A -> KeyEvent.KEYCODE_DPAD_CENTER // activate focused element
KeyEvent.KEYCODE_BUTTON_B -> KeyEvent.KEYCODE_BACK // back / dismiss
else -> 0
}
if (mapped != 0) return super.dispatchKeyEvent(KeyEvent(event.action, mapped))
}
if (mapped != 0) return super.dispatchKeyEvent(KeyEvent(event.action, mapped))
}
return super.dispatchKeyEvent(event)
}
@@ -101,6 +115,8 @@ class MainActivity : ComponentActivity() {
if (axisMapper?.onMotion(event) == true) return true
return super.dispatchGenericMotionEvent(event)
}
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
padMotionProbe?.let { if (it(event)) return true }
// Not streaming: turn the gamepad HAT / left stick into discrete D-pad focus moves, so a
// controller navigates the menus even when its D-pad reports as axes (not key events) and
// for stick-based navigation. Edge-detected so a held direction moves focus exactly once.
@@ -26,17 +26,26 @@ data class Settings(
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, or `"h264"`. A soft preference — the
* host emits it when it can, else falls back. AMediaCodec decodes whichever the host resolves. */
val codec: String = "auto",
val micEnabled: Boolean = false,
/** Show the live stats overlay (FPS / throughput / latency) during a stream. */
val statsHudEnabled: Boolean = true,
/**
* Touch input model. `true` (default) = trackpad: the cursor stays put on touch-down and moves
* by the finger's relative delta (swipe to nudge, lift and re-swipe to walk it across), tap to
* click where it is. `false` = direct pointing: the cursor jumps to the finger (the old behaviour).
* Touch input model — how touchscreen fingers drive the host. [TouchMode.TRACKPAD] (default):
* the cursor stays put on touch-down and moves by the finger's relative delta (swipe to nudge,
* lift and re-swipe to walk it across), tap to click where it is. [TouchMode.POINTER]: the
* cursor jumps to the finger (direct pointing). [TouchMode.TOUCH]: real multi-touch
* passthrough — every finger reaches the host as a touchscreen contact, for apps/games that
* understand touch. Mirrors the Apple client's TouchInputMode.
*/
val trackpadMode: Boolean = true,
val touchMode: TouchMode = TouchMode.TRACKPAD,
)
/** [Settings.touchMode] values; persisted by name. */
enum class TouchMode { TRACKPAD, POINTER, TOUCH }
/** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */
class SettingsStore(context: Context) {
private val prefs =
@@ -51,9 +60,13 @@ class SettingsStore(context: Context) {
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
codec = prefs.getString(K_CODEC, "auto") ?: "auto",
micEnabled = prefs.getBoolean(K_MIC, false),
statsHudEnabled = prefs.getBoolean(K_HUD, true),
trackpadMode = prefs.getBoolean(K_TRACKPAD, true),
touchMode = prefs.getString(K_TOUCH_MODE, null)
?.let { name -> TouchMode.entries.firstOrNull { it.name == name } }
// Migration: the pre-enum Boolean "trackpad_mode" (true = trackpad, false = direct).
?: if (prefs.getBoolean(K_TRACKPAD, true)) TouchMode.TRACKPAD else TouchMode.POINTER,
)
fun save(s: Settings) {
@@ -66,9 +79,10 @@ class SettingsStore(context: Context) {
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
.putInt(K_AUDIO_CH, s.audioChannels)
.putString(K_CODEC, s.codec)
.putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_HUD, s.statsHudEnabled)
.putBoolean(K_TRACKPAD, s.trackpadMode)
.putString(K_TOUCH_MODE, s.touchMode.name)
.apply()
}
@@ -81,8 +95,12 @@ class SettingsStore(context: Context) {
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
const val K_AUDIO_CH = "audio_channels"
const val K_CODEC = "codec"
const val K_MIC = "mic_enabled"
const val K_HUD = "stats_hud_enabled"
const val K_TOUCH_MODE = "touch_mode"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode"
}
}
@@ -156,6 +174,21 @@ val AUDIO_CHANNEL_OPTIONS = listOf(
8 to "7.1 Surround",
)
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. */
val CODEC_OPTIONS = listOf(
"auto" to "Automatic",
"hevc" to "HEVC (H.265)",
"h264" to "H.264 (AVC)",
)
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2. */
fun Settings.preferredCodec(): Int = when (codec) {
"h264" -> 1
"hevc" -> 2
"av1" -> 4
else -> 0
}
/** (kbps, label). `0` = host default. */
val BITRATE_OPTIONS = listOf(
0 to "Automatic",
@@ -174,6 +207,13 @@ val COMPOSITOR_OPTIONS = listOf(
"gamescope",
)
/** (mode, label) for the touch-input model. */
val TOUCH_MODE_OPTIONS = listOf(
TouchMode.TRACKPAD to "Trackpad",
TouchMode.POINTER to "Direct pointer",
TouchMode.TOUCH to "Touch passthrough",
)
/** index = GamepadPref wire byte (0=Auto 1=Xbox360 2=DualSense 3=XboxOne 4=DualShock4). */
val GAMEPAD_OPTIONS = listOf(
"Automatic",
@@ -46,6 +46,7 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
var s by remember { mutableStateOf(initial) }
val context = LocalContext.current
var showLicenses by remember { mutableStateOf(false) }
var showControllers by remember { mutableStateOf(false) }
fun update(next: Settings) {
s = next
onChange(next)
@@ -62,6 +63,10 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
LicensesScreen(onBack = { showLicenses = false })
return
}
if (showControllers) {
ControllersScreen(gamepadSetting = s.gamepad, onBack = { showControllers = false })
return
}
Column(
modifier = Modifier
@@ -95,6 +100,12 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
selected = s.bitrateKbps,
) { kbps -> update(s.copy(bitrateKbps = kbps)) }
SettingDropdown(
label = "Video codec",
options = CODEC_OPTIONS,
selected = s.codec,
) { c -> update(s.copy(codec = c)) }
// HDR is only meaningful on a panel that can present HDR10; on an SDR display the toggle
// is disabled (and HDR is never advertised regardless) so the host doesn't send PQ the
// panel would mis-tone-map. The capability is fixed for the device, so read it once.
@@ -124,6 +135,12 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
options = GAMEPAD_OPTIONS.mapIndexed { i, lbl -> i to lbl },
selected = s.gamepad,
) { g -> update(s.copy(gamepad = g)) }
ClickableRow(
title = "Connected controllers",
subtitle = "What the app detects, with a live input test",
onClick = { showControllers = true },
)
}
SettingsGroup("Audio") {
@@ -148,13 +165,21 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
)
}
SettingsGroup("Pointer") {
ToggleRow(
title = "Trackpad mode",
subtitle = "Relative cursor like a laptop touchpad — swipe to nudge, tap to click. " +
"Off = the cursor jumps to your finger.",
checked = s.trackpadMode,
onCheckedChange = { on -> update(s.copy(trackpadMode = on)) },
SettingsGroup("Touch input") {
SettingDropdown(
label = "Touch input",
options = TOUCH_MODE_OPTIONS,
selected = s.touchMode,
onSelect = { mode -> update(s.copy(touchMode = mode)) },
)
Text(
"Trackpad: relative cursor like a laptop touchpad — tap to click, two-finger " +
"tap right-clicks, two fingers scroll, tap-then-drag holds the button. " +
"Direct pointer: the cursor jumps to your finger. Touch passthrough: real " +
"multi-touch reaches the host, for apps that understand touch.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.padding(top = 6.dp),
)
}
@@ -0,0 +1,119 @@
package io.unom.punktfunk
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt
/**
* The live stats overlay — the unified HUD (`design/stats-unification.md`, Android v1: headline is
* `capture→decoded`, tiled by `host+network` + `decode`). Reads the 18-double layout from
* [NativeBridge.nativeVideoStats]:
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lost, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms]`. Indexes 1013
* (present on a current native lib) describe the negotiated video feed and render as a
* codec/depth/colour/chroma line; 14/15 render as the stage equation — split into
* `host + network + decode` when the Phase-2 terms at 16/17 are nonzero (a current host sends
* per-AU 0xCF timings; an old host leaves them 0 and the combined `host+network` term stands);
* older layouts just omit those lines.
*/
@Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
if (s.size < 10) return
val w = s[6].toInt()
val h = s[7].toInt()
val hz = s[8].toInt()
val latValid = s[4] != 0.0
val skew = s[5] != 0.0
val lost = s[9].toLong()
Column(
modifier = modifier
.background(Color.Black.copy(alpha = 0.45f), RoundedCornerShape(6.dp))
.padding(horizontal = 8.dp, vertical = 4.dp),
) {
Text(
"$w×$h@$hz ${s[0].roundToInt()} fps ${"%.1f".format(s[1])} Mb/s",
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
videoFeedLine(s)?.let { feed ->
Text(
feed,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
if (latValid) {
val tag = if (skew) "" else " (same-host clock)"
Text(
"end-to-end ${"%.1f".format(s[2])} ms p50 · ${"%.1f".format(s[3])} p95 · capture→decoded$tag",
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
if (s.size >= 16) {
// Phase-2 split (s[16]/s[17]): render `host + network` separately when the host
// reported its share this window; otherwise the combined term (old host / no
// matched 0xCF timing).
val equation = if (s.size >= 18 && s[16] > 0) {
"= host ${"%.1f".format(s[16])} + network ${"%.1f".format(s[17])} + decode ${"%.1f".format(s[15])}"
} else {
"= host+network ${"%.1f".format(s[14])} + decode ${"%.1f".format(s[15])}"
}
Text(
equation,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
}
if (lost > 0) {
Text(
"lost $lost",
color = Color(0xFFFFB0B0),
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
}
}
/**
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
*/
private fun videoFeedLine(s: DoubleArray): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
val transfer = s[12].toInt()
val chromaIdc = s[13].toInt()
val depthLabel = if (bitDepth > 0) "$bitDepth-bit" else "8-bit"
val (dynamicRange, colorSpace) = when (transfer) {
16 -> "HDR" to "BT.2020 PQ"
18 -> "HDR" to "BT.2020 HLG"
else -> "SDR" to if (primaries == 9) "BT.2020" else "BT.709"
}
val chromaLabel = when (chromaIdc) {
3 -> "4:4:4"
2 -> "4:2:2"
else -> "4:2:0"
}
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -7,15 +7,9 @@ import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.WindowManager
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
@@ -25,12 +19,9 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.core.content.ContextCompat
import androidx.core.view.WindowCompat
@@ -41,25 +32,6 @@ import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.NativeBridge
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.delay
import kotlin.math.abs
import kotlin.math.hypot
import kotlin.math.roundToInt
// Touch-gesture tuning (px / ms). TAP_SLOP: movement under this still counts as a tap, not a drag.
// TAP_DRAG_MS: a new touch within this long after a tap starts a left-button drag. SCROLL_DIV: px of
// two-finger pan per wheel notch (smaller = faster scroll).
private const val TAP_SLOP = 12f
private const val TAP_DRAG_MS = 250L
private const val SCROLL_DIV = 4f
// Trackpad-mode pointer ballistics (relative one-finger motion). POINTER_SENS: base finger-px →
// host-px gain (~1:1, never twitchy). The rest is mild acceleration so a flick crosses the screen
// while a slow drag stays precise: above ACCEL_SPEED_FLOOR px/ms the gain ramps by ACCEL_GAIN per
// px/ms, capped at ACCEL_MAX (so a fast swipe can't fling the cursor uncontrollably).
private const val POINTER_SENS = 1.3f
private const val ACCEL_GAIN = 0.6f
private const val ACCEL_SPEED_FLOOR = 0.3f
private const val ACCEL_MAX = 3.0f
@Composable
fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
@@ -76,18 +48,25 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
Manifest.permission.RECORD_AUDIO,
) == PackageManager.PERMISSION_GRANTED
// Live decode stats for the HUD. Poll once a second for the whole stream (cheap, and each call
// drains+resets the native window so it never grows unbounded even while the overlay is hidden);
// `showStats` only gates rendering. A 3-finger tap toggles it live; the default comes from Settings.
// Live decode stats for the HUD. `showStats` gates the whole pipeline: the native per-frame
// sampling (nativeSetVideoStatsEnabled — hidden HUD costs one atomic load per frame) AND the
// 1 s poll loop, which only runs while the overlay is visible. Enabling resets the native
// window, so re-showing never renders stale data. A 3-finger tap toggles it live; the default
// comes from Settings.
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var showStats by remember { mutableStateOf(initialSettings.statsHudEnabled) }
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
val trackpad = initialSettings.trackpadMode
LaunchedEffect(handle) {
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
val touchMode = initialSettings.touchMode
LaunchedEffect(handle, showStats) {
NativeBridge.nativeSetVideoStatsEnabled(handle, showStats)
if (showStats) {
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
}
} else {
stats = null // drop the last snapshot so a re-show never flashes stale numbers
}
}
@@ -169,240 +148,19 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (showStats) {
stats?.let { StatsOverlay(it, Modifier.align(Alignment.TopStart).padding(12.dp)) }
}
// Touch → mouse. Two models, chosen by the Trackpad-mode setting:
// • trackpad (default): the cursor STAYS where it is on touch-down and moves by the finger's
// relative delta (MouseMove) with mild pointer acceleration — swipe to nudge, lift and
// re-swipe to walk it across, tap to click where it is. This is what makes the cursor
// reachable on a small screen.
// • direct (opt-out): the cursor jumps to the finger and follows it (MouseMoveAbs,
// host-normalized against the overlay size), the old "direct pointing" behaviour.
// Both share the same gesture vocabulary: tap = left click; two-finger tap = right click;
// two-finger drag = scroll; tap-then-press-and-drag = left-drag (text selection / moving
// windows); three-finger tap = toggle the stats HUD.
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt.
Box(
Modifier.fillMaxSize().pointerInput(handle, trackpad) {
var lastTapUp = 0L
var lastTapX = 0f
var lastTapY = 0f
fun moveAbs(x: Float, y: Float) {
val sw = size.width
val sh = size.height
if (sw <= 0 || sh <= 0) return
NativeBridge.nativeSendPointerAbs(
Modifier.fillMaxSize().pointerInput(handle, touchMode) {
when (touchMode) {
TouchMode.TOUCH -> streamTouchPassthrough(handle)
else -> streamTouchInput(
handle,
x.coerceIn(0f, (sw - 1).toFloat()).roundToInt(),
y.coerceIn(0f, (sh - 1).toFloat()).roundToInt(),
sw,
sh,
trackpad = touchMode == TouchMode.TRACKPAD,
onToggleStats = { showStats = !showStats },
)
}
awaitEachGesture {
val down = awaitFirstDown(requireUnconsumed = false)
val startX = down.position.x
val startY = down.position.y
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left
// button for this whole gesture (laptop-trackpad convention).
val isDrag = down.uptimeMillis - lastTapUp < TAP_DRAG_MS &&
abs(startX - lastTapX) < TAP_SLOP && abs(startY - lastTapY) < TAP_SLOP
lastTapUp = 0L // consume the arming either way
// Direct mode jumps the cursor to the finger; trackpad mode leaves it put (the
// whole point — you nudge it with swipes instead).
if (!trackpad) moveAbs(startX, startY)
if (isDrag) NativeBridge.nativeSendPointerButton(handle, 1, true)
var moved = false
var maxFingers = 1
var scrolling = false
var prevCx = startX
var prevCy = startY
var upTime = down.uptimeMillis
// Trackpad relative-motion state: the tracked finger, its last position/time, and
// the sub-pixel remainder so a slow drag isn't lost to Int truncation.
var trackId = down.id
var prevX = startX
var prevY = startY
var prevT = down.uptimeMillis
var accX = 0f
var accY = 0f
while (true) {
val ev = awaitPointerEvent()
val pressed = ev.changes.filter { it.pressed }
if (pressed.isEmpty()) {
upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime
break
}
if (pressed.size > maxFingers) maxFingers = pressed.size
if (pressed.size >= 2) {
// Two fingers → scroll by the centroid delta; never move the cursor.
val cx = (pressed.sumOf { it.position.x.toDouble() } / pressed.size).toFloat()
val cy = (pressed.sumOf { it.position.y.toDouble() } / pressed.size).toFloat()
if (!scrolling) {
scrolling = true
prevCx = cx
prevCy = cy
}
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120)
prevCy = cy
moved = true
}
if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120)
prevCx = cx
moved = true
}
} else if (!scrolling) {
// One finger (skipped once a gesture turned into a scroll, so dropping
// back to one finger doesn't jerk the cursor).
val p = pressed.firstOrNull { it.id == down.id } ?: pressed.first()
if (abs(p.position.x - startX) > TAP_SLOP ||
abs(p.position.y - startY) > TAP_SLOP
) {
moved = true
}
if (trackpad) {
// Relative: move by the finger delta × (sensitivity × acceleration),
// carrying the sub-pixel remainder. Re-anchor (zero delta this frame)
// if the tracked finger changed, so lifting one of several fingers
// never jumps the cursor.
if (p.id != trackId) {
trackId = p.id
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
}
val dx = p.position.x - prevX
val dy = p.position.y - prevY
val dt = (p.uptimeMillis - prevT).coerceAtLeast(1L)
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
val speed = hypot(dx, dy) / dt // finger px per ms
val accel = (1f + ACCEL_GAIN * (speed - ACCEL_SPEED_FLOOR).coerceAtLeast(0f))
.coerceAtMost(ACCEL_MAX)
accX += dx * POINTER_SENS * accel
accY += dy * POINTER_SENS * accel
val outX = accX.toInt() // truncates toward zero → remainder kept w/ sign
val outY = accY.toInt()
if (outX != 0 || outY != 0) {
NativeBridge.nativeSendPointerMove(handle, outX, outY)
accX -= outX
accY -= outY
}
} else {
moveAbs(p.position.x, p.position.y) // direct: cursor follows the finger
}
}
ev.changes.forEach { it.consume() }
}
if (isDrag) {
NativeBridge.nativeSendPointerButton(handle, 1, false) // end the drag
} else if (!moved) {
when {
maxFingers >= 3 -> showStats = !showStats // in-stream HUD toggle
maxFingers == 2 -> { // two-finger tap → right click
NativeBridge.nativeSendPointerButton(handle, 3, true)
NativeBridge.nativeSendPointerButton(handle, 3, false)
}
else -> { // tap → left click (at the cursor's current spot), arm tap-drag
NativeBridge.nativeSendPointerButton(handle, 1, true)
NativeBridge.nativeSendPointerButton(handle, 1, false)
lastTapUp = upTime
lastTapX = startX
lastTapY = startY
}
}
}
}
},
)
}
}
/**
* The live stats overlay — mirrors the Apple client's HUD. Reads the 14-double layout from
* [NativeBridge.nativeVideoStats]:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc]`. The trailing four (present on a current native lib) describe the
* negotiated video feed and render as a codec/depth/colour/chroma line; older layouts just omit it.
*/
@Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
if (s.size < 10) return
val w = s[6].toInt()
val h = s[7].toInt()
val hz = s[8].toInt()
val latValid = s[4] != 0.0
val skew = s[5] != 0.0
val dropped = s[9].toLong()
Column(
modifier = modifier
.background(Color.Black.copy(alpha = 0.45f), RoundedCornerShape(6.dp))
.padding(horizontal = 8.dp, vertical = 4.dp),
) {
Text(
"$w×$h@$hz ${s[0].roundToInt()} fps ${"%.1f".format(s[1])} Mb/s",
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
videoFeedLine(s)?.let { feed ->
Text(
feed,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
if (latValid) {
val tag = if (skew) "" else " (same-host)"
Text(
"capture→client ${"%.1f".format(s[2])}/${"%.1f".format(s[3])} ms p50/p95$tag",
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
if (dropped > 0) {
Text(
"dropped $dropped",
color = Color(0xFFFFB0B0),
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
}
}
/**
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
*/
private fun videoFeedLine(s: DoubleArray): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
val transfer = s[12].toInt()
val chromaIdc = s[13].toInt()
val depthLabel = if (bitDepth > 0) "$bitDepth-bit" else "8-bit"
val (dynamicRange, colorSpace) = when (transfer) {
16 -> "HDR" to "BT.2020 PQ"
18 -> "HDR" to "BT.2020 HLG"
else -> "SDR" to if (primaries == 9) "BT.2020" else "BT.709"
}
val chromaLabel = when (chromaIdc) {
3 -> "4:4:4"
2 -> "4:2:2"
else -> "4:2:0"
}
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -0,0 +1,236 @@
package io.unom.punktfunk
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.ui.input.pointer.PointerId
import androidx.compose.ui.input.pointer.PointerInputScope
import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed
import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed
import androidx.compose.ui.input.pointer.positionChanged
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.abs
import kotlin.math.hypot
import kotlin.math.roundToInt
// Touch-gesture tuning (px / ms). TAP_SLOP: movement under this still counts as a tap, not a drag.
// TAP_DRAG_MS: a new touch within this long after a tap starts a left-button drag. SCROLL_DIV: px of
// two-finger pan per wheel notch (smaller = faster scroll).
private const val TAP_SLOP = 12f
private const val TAP_DRAG_MS = 250L
private const val SCROLL_DIV = 4f
// Trackpad-mode pointer ballistics (relative one-finger motion). POINTER_SENS: base finger-px →
// host-px gain (~1:1, never twitchy). The rest is mild acceleration so a flick crosses the screen
// while a slow drag stays precise: above ACCEL_SPEED_FLOOR px/ms the gain ramps by ACCEL_GAIN per
// px/ms, capped at ACCEL_MAX (so a fast swipe can't fling the cursor uncontrollably).
private const val POINTER_SENS = 1.3f
private const val ACCEL_GAIN = 0.6f
private const val ACCEL_SPEED_FLOOR = 0.3f
private const val ACCEL_MAX = 3.0f
/**
* Touch → mouse, run inside the stream overlay's `pointerInput`. Two models, chosen by the
* Trackpad-mode setting:
* * trackpad (default): the cursor STAYS where it is on touch-down and moves by the finger's
* relative delta (MouseMove) with mild pointer acceleration — swipe to nudge, lift and
* re-swipe to walk it across, tap to click where it is. This is what makes the cursor
* reachable on a small screen.
* * direct (opt-out): the cursor jumps to the finger and follows it (MouseMoveAbs,
* host-normalized against the overlay size), the old "direct pointing" behaviour.
*
* Both share the same gesture vocabulary: tap = left click; two-finger tap = right click;
* two-finger drag = scroll; tap-then-press-and-drag = left-drag (text selection / moving
* windows); three-finger tap = [onToggleStats] (the stats HUD).
*/
/**
* Real multi-touch passthrough ([TouchMode.TOUCH]): every finger forwards as a host touchscreen
* contact (down/move/up with a stable per-finger id), with NO gesture interpretation — taps,
* drags and multi-finger input mean whatever the remote app decides. Coordinates are overlay
* pixels with the overlay size as the surface, exactly like the absolute-mouse path (the host
* normalizes and maps into the output). On teardown (stream leaves composition) every still-held
* contact is lifted so nothing stays stuck on the host.
*/
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
val ids = mutableMapOf<PointerId, Int>()
fun alloc(p: PointerId): Int {
var id = 0
while (ids.containsValue(id)) id++
ids[p] = id
return id
}
try {
awaitPointerEventScope {
while (true) {
val ev = awaitPointerEvent()
val sw = size.width
val sh = size.height
if (sw <= 0 || sh <= 0) continue
for (c in ev.changes) {
val x = c.position.x.roundToInt().coerceIn(0, sw - 1)
val y = c.position.y.roundToInt().coerceIn(0, sh - 1)
when {
c.changedToDownIgnoreConsumed() ->
NativeBridge.nativeSendTouch(handle, alloc(c.id), 0, x, y, sw, sh)
c.changedToUpIgnoreConsumed() ->
ids.remove(c.id)?.let {
NativeBridge.nativeSendTouch(handle, it, 2, 0, 0, sw, sh)
}
c.positionChanged() ->
ids[c.id]?.let {
NativeBridge.nativeSendTouch(handle, it, 1, x, y, sw, sh)
}
}
c.consume()
}
}
}
} finally {
// Lift anything still down (composition/session teardown mid-touch).
ids.values.forEach { NativeBridge.nativeSendTouch(handle, it, 2, 0, 0, 1, 1) }
}
}
internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
trackpad: Boolean,
onToggleStats: () -> Unit,
) {
var lastTapUp = 0L
var lastTapX = 0f
var lastTapY = 0f
fun moveAbs(x: Float, y: Float) {
val sw = size.width
val sh = size.height
if (sw <= 0 || sh <= 0) return
NativeBridge.nativeSendPointerAbs(
handle,
x.coerceIn(0f, (sw - 1).toFloat()).roundToInt(),
y.coerceIn(0f, (sh - 1).toFloat()).roundToInt(),
sw,
sh,
)
}
awaitEachGesture {
val down = awaitFirstDown(requireUnconsumed = false)
val startX = down.position.x
val startY = down.position.y
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left
// button for this whole gesture (laptop-trackpad convention).
val isDrag = down.uptimeMillis - lastTapUp < TAP_DRAG_MS &&
abs(startX - lastTapX) < TAP_SLOP && abs(startY - lastTapY) < TAP_SLOP
lastTapUp = 0L // consume the arming either way
// Direct mode jumps the cursor to the finger; trackpad mode leaves it put (the
// whole point — you nudge it with swipes instead).
if (!trackpad) moveAbs(startX, startY)
if (isDrag) NativeBridge.nativeSendPointerButton(handle, 1, true)
var moved = false
var maxFingers = 1
var scrolling = false
var prevCx = startX
var prevCy = startY
var upTime = down.uptimeMillis
// Trackpad relative-motion state: the tracked finger, its last position/time, and
// the sub-pixel remainder so a slow drag isn't lost to Int truncation.
var trackId = down.id
var prevX = startX
var prevY = startY
var prevT = down.uptimeMillis
var accX = 0f
var accY = 0f
while (true) {
val ev = awaitPointerEvent()
val pressed = ev.changes.filter { it.pressed }
if (pressed.isEmpty()) {
upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime
break
}
if (pressed.size > maxFingers) maxFingers = pressed.size
if (pressed.size >= 2) {
// Two fingers → scroll by the centroid delta; never move the cursor.
val cx = (pressed.sumOf { it.position.x.toDouble() } / pressed.size).toFloat()
val cy = (pressed.sumOf { it.position.y.toDouble() } / pressed.size).toFloat()
if (!scrolling) {
scrolling = true
prevCx = cx
prevCy = cy
}
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120)
prevCy = cy
moved = true
}
if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120)
prevCx = cx
moved = true
}
} else if (!scrolling) {
// One finger (skipped once a gesture turned into a scroll, so dropping
// back to one finger doesn't jerk the cursor).
val p = pressed.firstOrNull { it.id == down.id } ?: pressed.first()
if (abs(p.position.x - startX) > TAP_SLOP ||
abs(p.position.y - startY) > TAP_SLOP
) {
moved = true
}
if (trackpad) {
// Relative: move by the finger delta × (sensitivity × acceleration),
// carrying the sub-pixel remainder. Re-anchor (zero delta this frame)
// if the tracked finger changed, so lifting one of several fingers
// never jumps the cursor.
if (p.id != trackId) {
trackId = p.id
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
}
val dx = p.position.x - prevX
val dy = p.position.y - prevY
val dt = (p.uptimeMillis - prevT).coerceAtLeast(1L)
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
val speed = hypot(dx, dy) / dt // finger px per ms
val accel = (1f + ACCEL_GAIN * (speed - ACCEL_SPEED_FLOOR).coerceAtLeast(0f))
.coerceAtMost(ACCEL_MAX)
accX += dx * POINTER_SENS * accel
accY += dy * POINTER_SENS * accel
val outX = accX.toInt() // truncates toward zero → remainder kept w/ sign
val outY = accY.toInt()
if (outX != 0 || outY != 0) {
NativeBridge.nativeSendPointerMove(handle, outX, outY)
accX -= outX
accY -= outY
}
} else {
moveAbs(p.position.x, p.position.y) // direct: cursor follows the finger
}
}
ev.changes.forEach { it.consume() }
}
if (isDrag) {
NativeBridge.nativeSendPointerButton(handle, 1, false) // end the drag
} else if (!moved) {
when {
maxFingers >= 3 -> onToggleStats() // in-stream HUD toggle
maxFingers == 2 -> { // two-finger tap → right click
NativeBridge.nativeSendPointerButton(handle, 3, true)
NativeBridge.nativeSendPointerButton(handle, 3, false)
}
else -> { // tap → left click (at the cursor's current spot), arm tap-drag
NativeBridge.nativeSendPointerButton(handle, 1, true)
NativeBridge.nativeSendPointerButton(handle, 1, false)
lastTapUp = upTime
lastTapX = startX
lastTapY = startY
}
}
}
}
}
@@ -27,6 +27,7 @@ import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import io.unom.punktfunk.BrandDark
import io.unom.punktfunk.Settings
import io.unom.punktfunk.TouchMode
import io.unom.punktfunk.SettingsScreen
import io.unom.punktfunk.StatsOverlay
import io.unom.punktfunk.components.HostCard
@@ -109,7 +110,7 @@ internal fun SettingsScene() {
gamepad = 2,
micEnabled = true,
statsHudEnabled = true,
trackpadMode = true,
touchMode = TouchMode.TRACKPAD,
),
onChange = {},
onBack = {},
+24 -7
View File
@@ -37,13 +37,30 @@ def call(method, url, token=None, data=None, content_type=None, want_json=True):
headers["Authorization"] = f"Bearer {token}"
if content_type:
headers["Content-Type"] = content_type
req = urllib.request.Request(url, data=data, method=method, headers=headers)
try:
with urllib.request.urlopen(req, timeout=300) as r:
body = r.read()
except urllib.error.HTTPError as e:
raise ApiError(e.code, method, url, e.read().decode("utf-8", "replace"))
return json.loads(body) if (want_json and body) else body
# Transient-fault retries: googleapis.com occasionally drops the TLS session ("EOF
# occurred in violation of protocol" — failed two release uploads on 2026-07-02) or
# answers 5xx. Retry those with backoff; 4xx raises immediately (a real API error).
# The edits API is transactional until commit, so re-sending any of these is safe.
last = None
for attempt in range(4):
if attempt:
delay = 3**attempt
print(f"transient Play API failure ({last}); retry {attempt}/3 in {delay}s")
time.sleep(delay)
req = urllib.request.Request(url, data=data, method=method, headers=headers)
try:
with urllib.request.urlopen(req, timeout=300) as r:
body = r.read()
return json.loads(body) if (want_json and body) else body
except urllib.error.HTTPError as e:
if e.code >= 500:
last = f"HTTP {e.code}"
continue
raise ApiError(e.code, method, url, e.read().decode("utf-8", "replace"))
except urllib.error.URLError as e:
last = str(getattr(e, "reason", e))
continue
sys.exit(f"ERROR: {method} {url} still failing after retries: {last}")
def load_sa():
@@ -98,20 +98,20 @@ object Gamepad {
}
}
/** First connected gamepad/joystick [InputDevice], or null when none is attached. */
fun firstPad(): InputDevice? {
for (id in InputDevice.getDeviceIds()) {
val d = InputDevice.getDevice(id) ?: continue
val s = d.sources
if (s and InputDevice.SOURCE_GAMEPAD == InputDevice.SOURCE_GAMEPAD ||
s and InputDevice.SOURCE_JOYSTICK == InputDevice.SOURCE_JOYSTICK
) {
return d
}
}
return null
/** True when [dev]'s source classes include gamepad or joystick. */
fun isPad(dev: InputDevice?): Boolean {
val s = dev?.sources ?: return false
return s and InputDevice.SOURCE_GAMEPAD == InputDevice.SOURCE_GAMEPAD ||
s and InputDevice.SOURCE_JOYSTICK == InputDevice.SOURCE_JOYSTICK
}
/** All connected gamepad/joystick [InputDevice]s, in system enumeration order. */
fun pads(): List<InputDevice> =
InputDevice.getDeviceIds().toList().mapNotNull { InputDevice.getDevice(it) }.filter { isPad(it) }
/** First connected gamepad/joystick [InputDevice], or null when none is attached. */
fun firstPad(): InputDevice? = pads().firstOrNull()
/**
* The [GamepadPref] wire byte to send for the user's [setting] (the persisted gamepad index). A
* non-Auto setting is passed through unchanged; "Automatic" ([PREF_AUTO]) resolves to a concrete
@@ -3,13 +3,79 @@ package io.unom.punktfunk.kit
import android.view.KeyEvent
/**
* Android `KEYCODE_*` → Windows Virtual-Key code (the punktfunk wire contract; the host maps VK →
* evdev via `inject::vk_to_evdev`). The Android analogue of the Linux client's evdev→VK table
* (`punktfunk-client-linux/src/keymap.rs`) and the Apple client's `hidToVK`. Positional/US-layout —
* we forward the physical key, not the typed character. Unmapped keys → 0 (the Rust side drops them).
* Extend this alongside `punktfunk-host/src/inject.rs::vk_to_evdev` (emit only VKs the host knows).
* Hardware key → Windows Virtual-Key code (the punktfunk wire contract: **US-positional** — we
* forward the physical key, not the typed character; the host maps VK → evdev via
* `inject::vk_to_evdev`). The Android analogue of the Linux client's evdev→VK table
* (`punktfunk-client-linux/src/keymap.rs`) and the Apple client's `hidToVK`.
*
* Prefer [toVk] with the full [KeyEvent]: it reads the raw evdev scancode first, because
* `KeyEvent.keyCode` is only positional under the stock US key layout — a user-selected physical
* keyboard layout (Settings → Physical keyboard) remaps keycodes semantically (AOSP's German .kcm
* carries `map key 21 Z` / `map key 44 Y`), which would apply the layout twice: once here, once on
* the host (the y↔z / ü-on-ö scramble). Unmapped keys → 0 (the Rust side drops them). Extend this
* alongside `punktfunk-host/src/inject.rs::vk_to_evdev` (emit only VKs the host knows).
*/
object Keymap {
/**
* Positional wire VK for a hardware key event: the evdev scancode table first (immune to the
* selected physical-keyboard layout), falling back to the keycode table for events without a
* scancode (soft keyboards, synthetic events) and for everything outside the typing area
* (layout-invariant there, incl. gamepad buttons whose scancodes lie outside the table).
*/
fun toVk(event: KeyEvent): Int {
val positional = evdevToVk(event.scanCode)
return if (positional != 0) positional else toVk(event.keyCode)
}
/**
* Linux evdev keycode (`KeyEvent.scanCode`) → US-positional VK for the layout-**variant**
* typing area — the same 48-key table as the Linux client's `evdev_to_vk` and the hosts'
* fixed tables. Everything else → 0 (the keycode path is already positional for those).
*/
fun evdevToVk(scan: Int): Int = when (scan) {
in 2..10 -> 0x31 + (scan - 2) // KEY_1..KEY_9
11 -> 0x30 // KEY_0
12 -> 0xBD // KEY_MINUS -_ VK_OEM_MINUS (DE: ß)
13 -> 0xBB // KEY_EQUAL =+ VK_OEM_PLUS
16 -> 0x51 // Q
17 -> 0x57 // W
18 -> 0x45 // E
19 -> 0x52 // R
20 -> 0x54 // T
21 -> 0x59 // KEY_Y — US-Y position (QWERTZ: the Z key)
22 -> 0x55 // U
23 -> 0x49 // I
24 -> 0x4F // O
25 -> 0x50 // P
26 -> 0xDB // KEY_LEFTBRACE [{ VK_OEM_4 (DE: ü)
27 -> 0xDD // KEY_RIGHTBRACE ]} VK_OEM_6
30 -> 0x41 // A
31 -> 0x53 // S
32 -> 0x44 // D
33 -> 0x46 // F
34 -> 0x47 // G
35 -> 0x48 // H
36 -> 0x4A // J
37 -> 0x4B // K
38 -> 0x4C // L
39 -> 0xBA // KEY_SEMICOLON ;: VK_OEM_1 (DE: ö)
40 -> 0xDE // KEY_APOSTROPHE '" VK_OEM_7 (DE: ä)
41 -> 0xC0 // KEY_GRAVE `~ VK_OEM_3 (DE: ^)
43 -> 0xDC // KEY_BACKSLASH \| VK_OEM_5
44 -> 0x5A // KEY_Z — US-Z position (QWERTZ: the Y key)
45 -> 0x58 // X
46 -> 0x43 // C
47 -> 0x56 // V
48 -> 0x42 // B
49 -> 0x4E // N
50 -> 0x4D // M
51 -> 0xBC // KEY_COMMA ,< VK_OEM_COMMA
52 -> 0xBE // KEY_DOT .> VK_OEM_PERIOD
53 -> 0xBF // KEY_SLASH /? VK_OEM_2
86 -> 0xE2 // KEY_102ND <>| VK_OEM_102 (ISO)
else -> 0
}
fun toVk(keyCode: Int): Int = when (keyCode) {
in KeyEvent.KEYCODE_A..KeyEvent.KEYCODE_Z -> 0x41 + (keyCode - KeyEvent.KEYCODE_A) // AZ
in KeyEvent.KEYCODE_0..KeyEvent.KEYCODE_9 -> 0x30 + (keyCode - KeyEvent.KEYCODE_0) // 09 row
@@ -48,6 +48,8 @@ object NativeBridge {
gamepadPref: Int,
hdrEnabled: Boolean,
audioChannels: Int,
/** Preferred video codec as a `quic::CODEC_*` bit (`0` = auto). Soft — the host falls back. */
preferredCodec: Int,
timeoutMs: Int,
): Long
@@ -103,15 +105,28 @@ object NativeBridge {
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 14 doubles:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
* (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — bit depth
* 8/10, CICP primaries/transfer, and the HEVC chroma_format_idc 1=4:2:0 / 3=4:4:4). Poll ~1 Hz;
* each call resets the measurement window.
* Returns 18 doubles (unified stats spec, `design/stats-unification.md`):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
* netP50Ms]`
* (the two flags are 1.0/0.0; indexes 2/3 are the end-to-end capture→decoded headline; 1013
* describe the negotiated video feed — bit depth 8/10, CICP primaries/transfer, and the HEVC
* chroma_format_idc 1=4:2:0 / 3=4:4:4; 14/15 are the stage p50s tiling the headline —
* `host+network` = capture→received, `decode` = received→decoded; 16/17 split the
* `host+network` term via the host's per-AU 0xCF timings — `host` = the host's capture→sent,
* `network` = the remainder — both 0.0 when no timing matched this window, i.e. an old host).
* Poll ~1 Hz; each call resets the measurement window.
*/
external fun nativeVideoStats(handle: Long): DoubleArray?
/**
* Gate per-frame stats sampling on the HUD being visible: while disabled the decode thread
* skips the per-AU clock read + lock, so toggle this with the overlay (and only poll
* [nativeVideoStats] while it's on). Enabling resets the measurement window — no stale data.
* Sticky for the session (survives video stop/start). No-op on `0`.
*/
external fun nativeSetVideoStatsEnabled(handle: Long, enabled: Boolean)
/**
* Start host→client audio: Opus decode → jitter ring → AAudio (LowLatency), all in Rust. No-op
* if already started. Best-effort — a failure leaves video streaming.
@@ -149,6 +164,22 @@ object NativeBridge {
/** One scroll step. axis: 0=vertical 1=horizontal. delta: signed, 120-scaled, +=up/right. */
external fun nativeSendScroll(handle: Long, axis: Int, delta: Int)
/**
* One REAL touchscreen transition (the touch-passthrough input mode). [kind]: 0=down 1=move
* 2=up. [id] distinguishes fingers and is reusable after up; coordinates are pixels on the
* client's touch surface — the host rescales against [surfaceWidth]×[surfaceHeight] and
* injects a real touch contact. On up only [id] matters.
*/
external fun nativeSendTouch(
handle: Long,
id: Int,
kind: Int,
x: Int,
y: Int,
surfaceWidth: Int,
surfaceHeight: Int,
)
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
@@ -0,0 +1,43 @@
package io.unom.punktfunk.kit
import org.junit.Assert.assertEquals
import org.junit.Test
/**
* Pure JVM test of the positional scancode table (`Keymap.evdevToVk`) — no Android runtime types
* (the `KeyEvent` constants in the keycode table are compile-time-inlined ints). Run:
* `./gradlew :kit:testDebugUnitTest`.
*/
class KeymapTest {
/**
* The German-scramble regression pins: the physical keys a QWERTZ board labels Z/Y/ö/ü/ä/ß
* must leave this client as their US-position VKs, regardless of the user-selected physical
* keyboard layout (which remaps `keyCode`, not `scanCode`).
*/
@Test
fun positionalPinsForTheQwertzScramble() {
assertEquals(0x59, Keymap.evdevToVk(21)) // KEY_Y (QWERTZ: Z key) → VK_Y
assertEquals(0x5A, Keymap.evdevToVk(44)) // KEY_Z (QWERTZ: Y key) → VK_Z
assertEquals(0xBA, Keymap.evdevToVk(39)) // KEY_SEMICOLON (QWERTZ: ö) → VK_OEM_1
assertEquals(0xDB, Keymap.evdevToVk(26)) // KEY_LEFTBRACE (QWERTZ: ü) → VK_OEM_4
assertEquals(0xDE, Keymap.evdevToVk(40)) // KEY_APOSTROPHE (QWERTZ: ä) → VK_OEM_7
assertEquals(0xBD, Keymap.evdevToVk(12)) // KEY_MINUS (QWERTZ: ß) → VK_OEM_MINUS
}
/**
* Exactly the 48 typing-area keys are covered (10 digits + 26 letters + 12 OEM) with unique
* VKs; everything else (nav, F-row, modifiers, gamepad buttons at 0x100+) falls through to
* the keycode table.
*/
@Test
fun tableCoversTheTypingAreaBijectively() {
val mapped = (0..0x200).mapNotNull { sc ->
Keymap.evdevToVk(sc).takeIf { it != 0 }?.let { sc to it }
}
assertEquals(48, mapped.size)
assertEquals(48, mapped.map { it.second }.toSet().size)
assertEquals(0, Keymap.evdevToVk(1)) // KEY_ESC — layout-invariant, keycode path
assertEquals(0, Keymap.evdevToVk(59)) // KEY_F1
assertEquals(0, Keymap.evdevToVk(304)) // BTN_SOUTH — gamepad, never a typing key
}
}
+2 -2
View File
@@ -27,8 +27,8 @@ log = "0.4"
mdns-sd = "0.20"
# Android-only deps. Gated so `cargo build --workspace` on the Linux/macOS dev boxes + CI still
# compiles this crate (as a host cdylib) — the Android-framework glue (logging now; AMediaCodec via
# `ndk` and Oboe/Opus audio later) is only pulled in for the real `*-linux-android` targets.
# compiles this crate (as a host cdylib) — the Android-framework glue (logging, AMediaCodec + AAudio
# via `ndk`, the Opus codec) is only pulled in for the real `*-linux-android` targets.
[target.'cfg(target_os = "android")'.dependencies]
android_logger = "0.14"
# NDK bindings. "media" = AMediaCodec/ANativeWindow (video); "audio" = AAudio (audio playback).
+137
View File
@@ -0,0 +1,137 @@
//! Android Adaptive Performance Framework (ADPF) — CPU performance hints for the decode thread.
//!
//! ADPF lets a latency-critical app tell the platform "these threads run a repeating workload with
//! this per-cycle deadline, and here's how long they *actually* took." The kernel's CPU governor
//! (on Qualcomm Snapdragon in particular — its ADPF backend is among the most responsive) then keeps
//! those threads on the fast cores at high clocks instead of migrating them to a little core or
//! down-clocking between frames. For a stream client the win is on the in-process hot path we
//! control — the `pf-decode` feed/drain/present loop — *not* the hardware codec itself (that decodes
//! in the mediacodec service, a separate process we can't hint); keeping our loop from being
//! scheduled late directly trims the jitter between "AU received" and "buffer released to the
//! Surface." It complements the codec-side `operating-rate`/`priority` hints, which push the codec's
//! own clocks.
//!
//! The `APerformanceHint_*` API arrived in NDK **API level 33**. minSdk is 31, so we CANNOT link the
//! symbols directly: a `libpunktfunk_android.so` carrying an unresolved
//! `APerformanceHint_createSession` import fails to load on API 31/32 devices
//! (`System.loadLibrary` throws) even if the code path is never taken. Instead we resolve the
//! entry points from `libandroid.so` with `dlsym` at runtime — absent on < 33 ⇒
//! [`HintSession::create`] returns `None` and the decode loop simply runs without hints.
use std::ffi::c_void;
use std::os::raw::c_int;
// `APerformanceHint_*` function-pointer types. The manager/session handles are opaque, so we treat
// them as `*mut c_void`.
type GetManagerFn = unsafe extern "C" fn() -> *mut c_void;
type CreateSessionFn = unsafe extern "C" fn(*mut c_void, *const i32, usize, i64) -> *mut c_void;
type ReportFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type UpdateTargetFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type CloseFn = unsafe extern "C" fn(*mut c_void);
/// The entry points we use, resolved once from `libandroid.so`, plus the process-wide manager.
struct Api {
create_session: CreateSessionFn,
report: ReportFn,
update_target: UpdateTargetFn,
close: CloseFn,
manager: *mut c_void,
}
/// Resolve the ADPF entry points + the process manager, or `None` on API < 33 (symbols absent) or if
/// the manager is unavailable.
fn resolve_api() -> Option<Api> {
// SAFETY: `dlopen` of an always-present system library with a NUL-terminated name; it returns
// null on failure (checked below). `libandroid.so` is already mapped into every app process, so
// this only bumps its refcount — we intentionally never `dlclose` (process-lifetime handle).
let lib = unsafe { libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW) };
if lib.is_null() {
return None;
}
// SAFETY: `dlsym` on the valid handle above with NUL-terminated symbol names; each returns null
// when the symbol is absent (device API < 33), which we check before transmuting the non-null
// pointer to its fn-pointer type (layout-compatible; a resolved symbol is a valid code address).
unsafe {
let get_manager = libc::dlsym(lib, c"APerformanceHint_getManager".as_ptr());
let create_session = libc::dlsym(lib, c"APerformanceHint_createSession".as_ptr());
let report = libc::dlsym(lib, c"APerformanceHint_reportActualWorkDuration".as_ptr());
let update_target = libc::dlsym(lib, c"APerformanceHint_updateTargetWorkDuration".as_ptr());
let close = libc::dlsym(lib, c"APerformanceHint_closeSession".as_ptr());
if get_manager.is_null()
|| create_session.is_null()
|| report.is_null()
|| update_target.is_null()
|| close.is_null()
{
return None; // device API < 33 — no ADPF
}
let get_manager = std::mem::transmute::<*mut c_void, GetManagerFn>(get_manager);
let manager = get_manager();
if manager.is_null() {
return None;
}
Some(Api {
create_session: std::mem::transmute::<*mut c_void, CreateSessionFn>(create_session),
report: std::mem::transmute::<*mut c_void, ReportFn>(report),
update_target: std::mem::transmute::<*mut c_void, UpdateTargetFn>(update_target),
close: std::mem::transmute::<*mut c_void, CloseFn>(close),
manager,
})
}
}
/// A live ADPF hint session bound to a set of thread ids. Dropping it closes the session. Holds raw
/// handles, so it is `!Send`/`!Sync` — created and used only on the `pf-decode` thread.
pub struct HintSession {
api: Api,
session: *mut c_void,
}
impl HintSession {
/// Open a session hinting `tids` with an initial per-frame target of `target_ns` nanoseconds.
/// `None` when ADPF is unavailable (device API < 33) or the platform declines — the caller then
/// runs unhinted (a no-op, not an error).
pub fn create(target_ns: i64, tids: &[i32]) -> Option<Self> {
if target_ns <= 0 || tids.is_empty() {
return None;
}
let api = resolve_api()?;
// SAFETY: `api.manager` is the live process manager returned above; `tids` is a valid slice
// of `len` i32s that `createSession` copies; it returns null on failure (checked).
let session =
unsafe { (api.create_session)(api.manager, tids.as_ptr(), tids.len(), target_ns) };
if session.is_null() {
return None;
}
Some(Self { api, session })
}
/// Report the wall-clock time the hinted thread spent producing the last displayed frame. When
/// it exceeds the session target the governor boosts the cores running the thread; when it
/// stays under, clocks may relax. No-op on a non-positive duration (the API rejects it).
pub fn report_actual(&self, actual_ns: i64) {
if actual_ns <= 0 {
return;
}
// SAFETY: `self.session` is a live session for `self`'s lifetime.
unsafe { (self.api.report)(self.session, actual_ns) };
}
/// Update the per-frame target (e.g. after a mid-session refresh-rate change). Unused today —
/// the decode thread restarts on renegotiation — but kept for that path.
#[allow(dead_code)]
pub fn update_target(&self, target_ns: i64) {
if target_ns <= 0 {
return;
}
// SAFETY: `self.session` is a live session for `self`'s lifetime.
unsafe { (self.api.update_target)(self.session, target_ns) };
}
}
impl Drop for HintSession {
fn drop(&mut self) {
// SAFETY: `self.session` was created by `createSession` and is closed exactly once, here.
unsafe { (self.api.close)(self.session) };
}
}
+132 -97
View File
@@ -129,109 +129,140 @@ impl AudioPlayback {
let jitter_headroom = JITTER_HEADROOM_MS * ms;
let hard_cap_max = HARD_CAP_MS * ms;
let counters = Arc::new(Counters::default());
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list: drained PCM buffers go BACK to the decode thread to be refilled, so the
// realtime callback never frees heap (Android's Scudo allocator has unbounded free() tail
// latency — a free on the audio thread is an XRun = a click) and the decode thread rarely
// allocates. Same depth as the data channel.
let (free_tx, free_rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from a
// single high-priority thread, and the decode thread only touches `tx`/`free_rx`.
let cb_counters = counters.clone();
// Pre-reserve the ring so `extend` never reallocates on the realtime thread. Worst transient
// before the trim below = the hard cap plus one full channel of 5 ms (480-f32) frames — the
// punktfunk protocol always sends 5 ms Opus frames (host `audio_thread`); a larger frame
// would force a one-time realloc, asserted (not silently corrupted) in `decode_loop`.
let mut ring: VecDeque<f32> = VecDeque::with_capacity(hard_cap_max + RING_CHUNKS * 5 * ms);
let mut primed = false;
let mut empties: u32 = 0; // consecutive empty callbacks (de-prime hysteresis)
let mut cb_count: u32 = 0; // callbacks since open (throttles the XRun grow check)
let mut last_xrun: i32 = 0; // last AAudio XRun count we grew the buffer for
let callback = move |s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * channels;
// SAFETY: AAudio provides `num_frames * channel_count` F32 slots at `data`.
let out = unsafe { std::slice::from_raw_parts_mut(data as *mut f32, want) };
// Drain decoded chunks into the ring WITHOUT freeing on the RT thread: `drain(..)` empties
// each Vec but keeps its capacity, then the empty buffer is handed back for reuse. The
// only RT-thread free is the rare case where the recycle channel is momentarily full.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
// Jitter buffer: prime to ~40 ms (prime_floor) before playing and after a sustained drain;
// drop-oldest only above a wide ~120 ms band. Decoupled from the AAudio burst `want` (tiny
// on the LowLatency MMAP path) so the depth doesn't collapse to a single quantum.
let target = (3 * want).clamp(prime_floor, prime_ceil);
let hard_cap = (target + jitter_headroom).min(hard_cap_max);
while ring.len() > hard_cap {
ring.pop_front();
}
if !primed && ring.len() >= target {
primed = true;
}
if primed {
for slot in out.iter_mut() {
*slot = ring.pop_front().unwrap_or(0.0);
// One open attempt at a given sharing mode. Everything the realtime callback captures
// (channels, ring, prime state) is rebuilt per attempt — `open_stream` consumes the builder
// AND the callback, so nothing survives a failed try to reuse.
let try_open = |sharing: AudioSharingMode| -> ndk::audio::Result<(
AudioStream,
SyncSender<Vec<f32>>,
Receiver<Vec<f32>>,
)> {
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list: drained PCM buffers go BACK to the decode thread to be refilled, so
// the realtime callback never frees heap (Android's Scudo allocator has unbounded free()
// tail latency — a free on the audio thread is an XRun = a click) and the decode thread
// rarely allocates. Same depth as the data channel.
let (free_tx, free_rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from
// a single high-priority thread, and the decode thread only touches `tx`/`free_rx`.
let cb_counters = counters.clone();
// Pre-reserve the ring so `extend` never reallocates on the realtime thread. Worst
// transient before the trim below = the hard cap plus one full channel of 5 ms (480-f32)
// frames — the punktfunk protocol always sends 5 ms Opus frames (host `audio_thread`); a
// larger frame would force a one-time realloc, asserted (not silently corrupted) in
// `decode_loop`.
let mut ring: VecDeque<f32> =
VecDeque::with_capacity(hard_cap_max + RING_CHUNKS * 5 * ms);
let mut primed = false;
let mut empties: u32 = 0; // consecutive empty callbacks (de-prime hysteresis)
let mut cb_count: u32 = 0; // callbacks since open (throttles the XRun grow check)
let mut last_xrun: i32 = 0; // last AAudio XRun count we grew the buffer for
let callback = move |s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * channels;
// SAFETY: AAudio provides `num_frames * channel_count` F32 slots at `data`.
let out = unsafe { std::slice::from_raw_parts_mut(data as *mut f32, want) };
// Drain decoded chunks into the ring WITHOUT freeing on the RT thread: `drain(..)`
// empties each Vec but keeps its capacity, then the empty buffer is handed back for
// reuse. The only RT-thread free is the rare case where the recycle channel is
// momentarily full.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
// Jitter buffer: prime to ~40 ms (prime_floor) before playing and after a sustained
// drain; drop-oldest only above a wide ~120 ms band. Decoupled from the AAudio burst
// `want` (tiny on the LowLatency MMAP path) so the depth doesn't collapse to a single
// quantum.
let target = (3 * want).clamp(prime_floor, prime_ceil);
let hard_cap = (target + jitter_headroom).min(hard_cap_max);
while ring.len() > hard_cap {
ring.pop_front();
}
if !primed && ring.len() >= target {
primed = true;
}
if primed {
for slot in out.iter_mut() {
*slot = ring.pop_front().unwrap_or(0.0);
}
cb_counters
.pcm_written
.fetch_add(num_frames as u64, Ordering::Relaxed);
} else {
out.fill(0.0);
cb_counters.underruns.fetch_add(1, Ordering::Relaxed);
}
// Re-prime only after a RUN of empty callbacks, not a single transient one —
// otherwise every momentary drain costs a fresh 40 ms silence (the old behaviour,
// self-inflicted crackle on any jitter spike).
if ring.is_empty() {
empties += 1;
if empties >= DEPRIME_AFTER_CALLBACKS {
primed = false;
}
} else {
empties = 0;
}
cb_counters
.pcm_written
.fetch_add(num_frames as u64, Ordering::Relaxed);
} else {
out.fill(0.0);
cb_counters.underruns.fetch_add(1, Ordering::Relaxed);
}
// Re-prime only after a RUN of empty callbacks, not a single transient one — otherwise
// every momentary drain costs a fresh 40 ms silence (the old behaviour, self-inflicted
// crackle on any jitter spike).
if ring.is_empty() {
empties += 1;
if empties >= DEPRIME_AFTER_CALLBACKS {
primed = false;
.ring_depth
.store(ring.len() as u64, Ordering::Relaxed);
// Google's AAudio anti-glitch technique: when the device reports new XRuns, grow the
// HW buffer by one burst (up to capacity). getXRunCount + setBufferSizeInFrames are
// both callback-safe / non-blocking, and set clamps to capacity so it self-limits.
// Throttled.
cb_count = cb_count.wrapping_add(1);
if cb_count % XRUN_CHECK_EVERY == 0 {
let xr = s.x_run_count();
if xr > last_xrun {
last_xrun = xr;
let burst = s.frames_per_burst().max(1);
let grown =
(s.buffer_size_in_frames() + burst).min(s.buffer_capacity_in_frames());
let _ = s.set_buffer_size_in_frames(grown);
}
}
} else {
empties = 0;
}
cb_counters
.ring_depth
.store(ring.len() as u64, Ordering::Relaxed);
// Google's AAudio anti-glitch technique: when the device reports new XRuns, grow the HW
// buffer by one burst (up to capacity). getXRunCount + setBufferSizeInFrames are both
// callback-safe / non-blocking, and set clamps to capacity so it self-limits. Throttled.
cb_count = cb_count.wrapping_add(1);
if cb_count % XRUN_CHECK_EVERY == 0 {
let xr = s.x_run_count();
if xr > last_xrun {
last_xrun = xr;
let burst = s.frames_per_burst().max(1);
let grown =
(s.buffer_size_in_frames() + burst).min(s.buffer_capacity_in_frames());
let _ = s.set_buffer_size_in_frames(grown);
}
}
AudioCallbackResult::Continue
AudioCallbackResult::Continue
};
let stream = AudioStreamBuilder::new()?
.direction(AudioDirection::Output)
.sample_rate(SAMPLE_RATE)
// The wire order (FL FR FC LFE RL RR SL SR) is the standard AAudio/Android channel
// order, so this is an IDENTITY mapping — no permute. AAudio infers the 5.1/7.1 mask
// from `channel_count` (the ndk crate's builder exposes no setChannelMask); the host
// captures + Opus-encodes in exactly this order.
.channel_count(channels as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(sharing)
.data_callback(Box::new(callback))
.error_callback(Box::new(|_s, e| {
log::warn!("audio: AAudio error (device reroute/disconnect?): {e:?}");
}))
.open_stream()?;
Ok((stream, tx, free_rx))
};
let stream = AudioStreamBuilder::new()
.map_err(|e| log::error!("audio: AudioStreamBuilder::new: {e}"))
.ok()?
.direction(AudioDirection::Output)
.sample_rate(SAMPLE_RATE)
// The wire order (FL FR FC LFE RL RR SL SR) is the standard AAudio/Android channel
// order, so this is an IDENTITY mapping — no permute. AAudio infers the 5.1/7.1 mask
// from `channel_count` (the ndk crate's builder exposes no setChannelMask); the host
// captures + Opus-encodes in exactly this order.
.channel_count(channels as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(AudioSharingMode::Shared)
.data_callback(Box::new(callback))
.error_callback(Box::new(|_s, e| {
log::warn!("audio: AAudio error (device reroute/disconnect?): {e:?}");
}))
.open_stream()
.map_err(|e| log::error!("audio: open_stream: {e}"))
.ok()?;
// Exclusive first — MMAP-exclusive is AAudio's lowest-latency path (once proven on-device it
// may also allow lowering the jitter-ring depths above; those stay put pending crackle
// testing) — and fall back to Shared when the device refuses (no MMAP, output claimed, …).
// The started-log below prints the mode the device actually GRANTED (`share=`): AAudio may
// still resolve an Exclusive request to Shared.
let (stream, tx, free_rx) = match try_open(AudioSharingMode::Exclusive) {
Ok(opened) => opened,
Err(e) => {
log::info!("audio: Exclusive open failed ({e}) — retrying Shared");
match try_open(AudioSharingMode::Shared) {
Ok(opened) => opened,
Err(e) => {
log::error!("audio: open_stream: {e}");
return None;
}
}
}
};
if let Err(e) = stream.request_start() {
log::error!("audio: request_start: {e}");
@@ -293,6 +324,10 @@ fn decode_loop(
counters: Arc<Counters>,
channels: usize,
) {
// Fold this Opus→AAudio thread into the client's hot-thread set so the ADPF session the decode
// thread opens also keeps audio decode on a fast core (registered before the video pump's first
// frame arrives, so it's captured when that session is created). No-op below API 33.
client.register_hot_thread();
// Interleaved f32 samples per millisecond at this layout — the ring's 5 ms reserve check below.
let ms = (SAMPLE_RATE as usize / 1000) * channels;
// Opus decode scratch: worst-case 120 ms frame (5760 samples/ch) × channels.
+286 -50
View File
@@ -9,15 +9,27 @@
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{
DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection,
OutputBuffer,
};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::{FrameRateCompatibility, NativeWindow};
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
/// Cap on the pts→received-timestamp map below: MediaCodec holds only a handful of frames in
/// flight, so anything beyond this is stale (codec flushed / HUD toggled) and gets evicted.
const IN_FLIGHT_CAP: usize = 64;
/// Cap on received AUs awaiting their 0xCF host timing (Phase 2 host/network split): the timing
/// datagram trails its AU by at most the wire, so a match lands within a frame or two — anything
/// this deep is a lost datagram (or an old host that never sends any) and gets evicted.
const PENDING_SPLIT_CAP: usize = 256;
/// The decode loop. Runs on the `pf-decode` thread until `shutdown` is set or the session closes.
pub fn run(
client: Arc<NativeClient>,
@@ -27,16 +39,23 @@ pub fn run(
) {
boost_thread_priority();
let mode = client.mode();
let codec = match MediaCodec::from_decoder_type("video/hevc") {
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`): HEVC or H.264. AMediaCodec
// needs no out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = match client.codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
_ => "video/hevc",
};
let codec = match MediaCodec::from_decoder_type(mime) {
Some(c) => c,
None => {
log::error!("decode: no HEVC decoder on this device");
log::error!("decode: no {mime} decoder on this device");
return;
}
};
log::info!("decode: codec mime = {mime}");
let mut format = MediaFormat::new();
format.set_str("mime", "video/hevc");
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
// Generous input buffer so a large keyframe AU is never truncated.
@@ -46,11 +65,21 @@ pub fn run(
);
// Ask for the low-latency decode path where the decoder supports it (no reordering buffer).
format.set_i32("low-latency", 1);
// Best-effort vendor twin of the standard key: older Qualcomm decoders only honor their own
// extension. Unknown keys are ignored by other vendors' codecs, so this is safe to set blind.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
// Advisory low-latency hints (KEY_PRIORITY / KEY_OPERATING_RATE), ignored where unsupported:
// realtime priority + the target frame rate, so vendor decoders (e.g. Qualcomm) run at full
// clocks instead of a power-saving cadence that adds dequeue latency.
format.set_i32("priority", 0); // 0 = realtime
format.set_i32("operating-rate", mode.refresh_hz as i32);
// Operating rate = the codec's clock hint. Setting it to the display rate merely asks the
// decoder to *sustain* that cadence — a Qualcomm decoder can meet 60/120 fps at a power-saving
// clock that adds a millisecond-plus of decode latency per frame. Setting it to the AOSP
// "unbounded" sentinel (Short.MAX) instead asks the decoder to run each frame at max clocks and
// finish ASAP, minimising per-frame decode latency — the right trade for a real-time stream
// (costs power/heat; the dial to lower if a device thermally throttles over a long session).
// Ignored where unsupported.
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
// HDR static metadata (ST.2086 mastering + content light level): when an HDR session was
// negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade.
@@ -93,41 +122,174 @@ pub fn run(
);
}
// ADPF: hint the platform that the whole video pipeline — this pf-decode feed/drain/present
// loop, the core's data-plane pump (UDP receive + FEC reassembly), and the audio thread — runs a
// per-frame real-time workload, so the CPU governor keeps those threads on fast cores at high
// clocks instead of down-clocking between frames or parking them on a little core. Snapdragon's
// ADPF backend responds well to this. We register this thread now but create the session lazily
// on the first presented frame: by then the pump + audio threads have registered their ids too,
// and ADPF `createSession` rejects a set with any not-yet-live/dead tid. No-op below API 33.
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread(); // this decode thread → the pipeline's hot-thread set
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
// Accumulates the loop's productive (feed+drain) time between displayed frames; reported to ADPF
// once per rendered frame against the frame-period target.
let mut work_accum_ns: i64 = 0;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
// The AU waiting for a free codec input buffer. `feed` is non-blocking; on transient input
// pressure the AU stays parked here instead of being dropped (a drop forces a keyframe
// round-trip) and we only pop the next one once it's queued.
let mut pending: Option<Frame> = None;
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it
// climbs.
let mut last_dropped = client.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
// Capture→client-receipt latency uses the negotiated host-minus-client clock offset (0 if the
// host didn't answer the skew handshake — then the HUD flags it "same-host").
// Skew-corrected latency stats (spec: design/stats-unification.md) use the negotiated
// host-minus-client clock offset (0 if the host didn't answer the skew handshake — then the
// HUD flags it "(same-host clock)").
let clock_offset = client.clock_offset_ns;
// HUD stage split: receipt timestamps keyed by the pts we queue into the codec, so the decoded
// point (output-buffer dequeue — MediaCodec round-trips presentationTimeUs) can be paired back
// to its receipt for the `decode` stage. Only fed while the HUD is visible.
let mut in_flight: VecDeque<(u64, i128)> = VecDeque::new();
// Phase-2 host/network split (design/stats-unification.md): received AUs awaiting their 0xCF
// host timing, as (pts_ns, capture→received µs). The timings are drained non-blockingly right
// where receipts are recorded and matched by pts; `network = hostnet host` (saturating).
// Only fed while the HUD is visible; an old host never sends a 0xCF, so entries just age out.
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
// The dataspace we've signalled on the Surface so far (None = default/SDR). Set reactively once
// the decoder reports an HDR stream (see `drain`); avoids re-applying every format event.
let mut applied_ds: Option<DataSpace> = None;
// One thread feeds AND drains: the NDK AMediaCodec wrapper isn't documented thread-safe for
// cross-thread feed/drain, so instead of splitting threads the loop decouples the two — input
// dequeue is non-blocking (never stalls presentation of already-decoded frames) and the only
// blocking wait is a short output dequeue while input is backed up (decoder progress is exactly
// what frees the next input buffer).
while !shutdown.load(Ordering::Relaxed) {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
if fed == 0 {
let p = &frame.data;
log::info!(
"decode: first AU {} bytes, head {:02x?}",
p.len(),
&p[..p.len().min(6)]
);
if pending.is_none() {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
if fed == 0 {
let p = &frame.data;
log::info!(
"decode: first AU {} bytes, head {:02x?}",
p.len(),
&p[..p.len().min(6)]
);
}
// HUD stat, `received` point: host+network = client_now + (hostclient)
// capture_pts. Gated on the HUD being visible — `enabled` first so the hidden
// steady state skips the wall-clock read and the lock entirely. The receipt
// stamp is also parked in `in_flight` (keyed by the pts the codec will echo on
// the output buffer) for the decoded-point pairing in `drain`.
if stats.enabled() {
let received_ns = now_realtime_ns();
let lat_ns = received_ns + clock_offset as i128 - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
in_flight.push_back((frame.pts_ns / 1000, received_ns));
if in_flight.len() > IN_FLIGHT_CAP {
in_flight.pop_front(); // stale — codec never echoed it back
}
// Phase-2 split: park this AU's capture→received sample, then match any
// 0xCF host timings that have arrived — host = the host's own
// capture→sent, network = our capture→received minus it (per-frame
// tiling; saturating in case of clock jitter).
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front(); // 0xCF lost / old host — evict
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
pending = Some(frame);
}
fed += 1;
// HUD stat: capture→client-receipt latency = client_now + (hostclient) capture_pts.
let lat_ns = now_realtime_ns() + clock_offset as i128 - frame.pts_ns as i128;
let lat_us =
(lat_ns > 0 && lat_ns < 10_000_000_000).then_some((lat_ns / 1000) as u64);
stats.note(frame.data.len(), lat_us, clock_offset != 0);
feed(&codec, &frame.data, frame.pts_ns / 1000);
Err(PunktfunkError::NoFrame) => {} // timeout — still drain output below
Err(_) => break, // session closed
}
Err(PunktfunkError::NoFrame) => {} // timeout — still drain output below
Err(_) => break, // session closed
}
rendered += drain(&codec, &window, &mut applied_ds);
// Time the productive work (feed + drain) only — the `next_frame` poll wait above is idle
// and excluded, so ADPF sees this thread's real per-frame CPU cost, not the poll timeout.
let work_t0 = Instant::now();
if let Some(frame) = pending.take() {
if feed(&codec, &frame.data, frame.pts_ns / 1000) {
fed += 1;
if fed % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
} else {
// No input buffer free — transient back-pressure. Keep the AU and let `drain` block
// briefly below; a released output buffer is what recycles an input slot.
pending = Some(frame);
}
}
// Drain every iteration. When input is blocked, wait ~2 ms on output so the loop rides
// decoder progress instead of busy-spinning against a full input queue.
let wait = if pending.is_some() {
Duration::from_millis(2)
} else {
Duration::ZERO
};
let (r, d) = drain(
&codec,
&window,
&mut applied_ds,
wait,
&stats,
&mut in_flight,
clock_offset,
);
rendered += r;
discarded += d;
// ADPF: attribute this iteration's feed+drain time to the frame being produced, and report
// the accumulated per-frame work once one is actually presented (r > 0). Under back-pressure
// the short output-dequeue wait is included in the tally — for a latency-first client,
// biasing the governor toward "boost" is the desired behaviour. Cheap when `hint` is None
// (one `Instant` diff, no report).
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if r > 0 {
if !hint_tried {
// First presented frame: the pump + audio threads have registered their ids by now.
// Build one ADPF session over the whole pipeline's thread set (empty below API 33,
// or where the platform declines → `None`, and the loop runs unhinted).
hint_tried = true;
let tids = client.hot_thread_ids();
hint = crate::adpf::HintSession::create(frame_period_ns, &tids);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
}
// Loss recovery: under infinite GOP the only recovery keyframe is one we request. The
// reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the
@@ -145,14 +307,10 @@ pub fn run(
log::debug!("decode: requested keyframe (loss recovery, dropped={dropped})");
}
}
if fed > 0 && fed % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered}");
}
}
let _ = codec.stop();
log::info!("decode: stopped (fed={fed} rendered={rendered})");
log::info!("decode: stopped (fed={fed} rendered={rendered} discarded={discarded})");
}
/// Wall-clock now in nanoseconds (CLOCK_REALTIME basis), to compare against the host-stamped
@@ -182,9 +340,12 @@ fn boost_thread_priority() {
}
}
/// Copy one access unit into a codec input buffer and queue it.
fn feed(codec: &MediaCodec, au: &[u8], pts_us: u64) {
match codec.dequeue_input_buffer(Duration::from_millis(10)) {
/// Try to copy one access unit into a codec input buffer and queue it, without blocking. Returns
/// `false` only on `TryAgainLater` (no input buffer free) — the caller keeps the AU pending and
/// retries; a hard dequeue/queue error counts as consumed (retrying can't salvage the AU, and
/// parking it forever would wedge the loop on a broken codec).
fn feed(codec: &MediaCodec, au: &[u8], pts_us: u64) -> bool {
match codec.dequeue_input_buffer(Duration::ZERO) {
Ok(DequeuedInputBufferResult::Buffer(mut buf)) => {
let n = {
let dst = buf.buffer_mut();
@@ -196,41 +357,74 @@ fn feed(codec: &MediaCodec, au: &[u8], pts_us: u64) {
dst.len()
);
}
for (slot, &b) in dst.iter_mut().zip(&au[..n]) {
slot.write(b);
// SAFETY: `au` and `dst` are distinct allocations (wire AU vs. codec buffer), both
// valid for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so the cast
// write initializes exactly `dst[..n]`.
unsafe {
std::ptr::copy_nonoverlapping(au.as_ptr(), dst.as_mut_ptr().cast::<u8>(), n);
}
n
};
if let Err(e) = codec.queue_input_buffer(buf, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer: {e}");
}
true
}
Ok(DequeuedInputBufferResult::TryAgainLater) => {
// No input buffer free right now; the AU is dropped (FEC/keyframes recover).
Ok(DequeuedInputBufferResult::TryAgainLater) => false, // caller keeps the AU pending
Err(e) => {
log::warn!("decode: dequeue_input_buffer: {e}");
true
}
Err(e) => log::warn!("decode: dequeue_input_buffer: {e}"),
}
}
/// Release any ready output buffers to the surface (render = true), latency-first. Returns the
/// number of frames presented. Also reacts to `OutputFormatChanged` to signal HDR on the Surface.
fn drain(codec: &MediaCodec, window: &NativeWindow, applied_ds: &mut Option<DataSpace>) -> u64 {
let mut n = 0;
/// Dequeue every ready output buffer and present only the NEWEST (render = true), discarding the
/// rest (render = false) — when decode falls behind, a back-to-back burst of stale frames on glass
/// is worse than skipping straight to the freshest one (the Apple client's 1-slot newest-ready
/// ring, ported). `first_wait` is the timeout for the first dequeue only: zero normally, ~2 ms when
/// the caller's input is blocked so the loop waits on decoder progress instead of busy-spinning.
/// Returns `(rendered, discarded)`. Also reacts to `OutputFormatChanged` (which can interleave
/// between buffers — handled without losing the held buffer) to signal HDR on the Surface.
///
/// Each dequeued buffer is also the HUD's `decoded` measurement point (rendered or not — the frame
/// finished decoding either way): end-to-end = decoded + clock_offset capture pts, and the
/// `decode` stage pairs the buffer's echoed presentationTimeUs back to the receipt stamp in
/// `in_flight` (single-clock local difference, no skew involved).
fn drain(
codec: &MediaCodec,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
first_wait: Duration,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
) -> (u64, u64) {
let mut held = None; // newest ready buffer so far, presented after the loop
let mut discarded: u64 = 0;
let mut wait = first_wait;
loop {
match codec.dequeue_output_buffer(Duration::from_millis(0)) {
match codec.dequeue_output_buffer(wait) {
Ok(DequeuedOutputBufferInfoResult::Buffer(buf)) => {
if let Err(e) = codec.release_output_buffer(buf, true) {
log::warn!("decode: release_output_buffer: {e}");
break;
wait = Duration::ZERO; // only the first dequeue may block
if stats.enabled() {
note_decoded(stats, in_flight, clock_offset, &buf);
}
if let Some(stale) = held.replace(buf) {
// A newer frame is ready — drop the held one without rendering.
if let Err(e) = codec.release_output_buffer(stale, false) {
log::warn!("decode: release_output_buffer(discard): {e}");
}
discarded += 1;
}
n += 1;
}
Ok(DequeuedOutputBufferInfoResult::OutputFormatChanged) => {
// The decoder has parsed the SPS and now reports the stream's real colour signalling
// (the AMediaCodec analogue of VideoToolbox's format description on the Apple client).
// If it's HDR (BT.2020 PQ/HLG), tell the Surface so the compositor/display switch to
// HDR; SDR streams leave the default dataspace alone. The decoder itself picks a
// Main10 path from the SPS — no profile override needed. Keep looping (buffers follow).
// Main10 path from the SPS — no profile override needed. Keep looping (buffers
// follow, and any held buffer stays held across this event).
wait = Duration::ZERO;
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
@@ -245,7 +439,7 @@ fn drain(codec: &MediaCodec, window: &NativeWindow, applied_ds: &mut Option<Data
}
}
}
// TryAgainLater / OutputBuffersChanged — nothing to render now.
// TryAgainLater / OutputBuffersChanged — nothing more to dequeue now.
Ok(_) => break,
Err(e) => {
log::warn!("decode: dequeue_output_buffer: {e}");
@@ -253,7 +447,49 @@ fn drain(codec: &MediaCodec, window: &NativeWindow, applied_ds: &mut Option<Data
}
}
}
n
// Present the newest ready frame, if any.
let mut rendered = 0;
if let Some(buf) = held {
match codec.release_output_buffer(buf, true) {
Ok(()) => rendered = 1,
Err(e) => log::warn!("decode: release_output_buffer: {e}"),
}
}
(rendered, discarded)
}
/// HUD `decoded` point for one dequeued output buffer: build the end-to-end (capture→decoded,
/// skew-corrected, clamped to (0, 10 s)) and `decode` (received→decoded, single-clock local, ≥ 0)
/// samples and hand them to [`crate::stats::VideoStats::note_decoded`]. The codec echoes the input
/// `presentationTimeUs` on the output buffer, which keys the receipt stamp in `in_flight`; entries
/// older than the echoed pts are evicted (decode order == input order here — low-latency, no
/// B-frames — so anything before it was dropped inside the codec or stamped before a flush).
fn note_decoded(
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
buf: &OutputBuffer<'_>,
) {
let pts_us = buf.info().presentation_time_us().max(0) as u64;
let decoded_ns = now_realtime_ns();
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
in_flight.pop_front();
if p == pts_us {
received_ns = Some(r);
break;
}
}
// pts_us is the truncated frame.pts_ns/1000 we queued, so ×1000 re-approximates capture time
// to < 1 µs — negligible against the ms-scale figures shown.
let e2e_ns = decoded_ns + clock_offset as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
let decode_us = received_ns.map(|r| ((decoded_ns - r).max(0) / 1000) as u64);
stats.note_decoded(e2e_us, decode_us);
}
/// Map the decoder's reported output colour to a BT.2020 HDR dataspace, or `None` for SDR. The
+6 -4
View File
@@ -16,15 +16,17 @@
//! Wi-Fi `MulticastLock` + permission UX, Keystore identity).
//!
//! JNI symbols map to `io.unom.punktfunk.kit.NativeBridge` in the `:kit` Gradle module
//! (`clients/android`). The current surface is the scaffold's native-link proof
//! (`abiVersion`/`coreVersion`) plus the session handle lifecycle in [`session`]; the per-plane
//! pumps (video → AMediaCodec, audio → Oboe), input, audio, pairing and mode renegotiation are
//! the next milestone (see the TODOs in [`session`]).
//! (`clients/android`). The surface: the native-link proof (`abiVersion`/`coreVersion`), mDNS host
//! discovery ([`discovery`]), and the session lifecycle in [`session`] — connect/pair + the trust
//! surface, the per-plane pumps (video → AMediaCodec, audio ↔ AAudio, mic uplink), input, and
//! rumble/HID feedback ([`feedback`]). Mode renegotiation is still TODO (see [`session`]).
use jni::objects::JObject;
use jni::sys::jint;
use jni::JNIEnv;
#[cfg(target_os = "android")]
mod adpf;
#[cfg(target_os = "android")]
mod audio;
#[cfg(target_os = "android")]
+112 -38
View File
@@ -1,9 +1,12 @@
//! Android microphone uplink (android-only): capture mic PCM via AAudio (LowLatency **input**),
//! Opus-encode 20 ms stereo frames, and push them to the host over the connector's mic plane
//! (`send_mic` → 0xCB datagram). The mirror of [`crate::audio`] in reverse: AAudio's realtime input
//! callback hands captured interleaved f32 to a channel; a worker thread we own does the Opus encode
//! + send (encoding is too heavy for the realtime callback, exactly as decode is on the playback
//! side). Format matches the host decoder + the Linux client: 48 kHz **stereo**, 20 ms, Opus VOIP.
//! callback hands captured interleaved f32 to a channel; a worker thread we own does the Opus
//! encode + send (encoding is too heavy for the realtime callback, exactly as decode is on the
//! playback side). Like the playback path, the realtime callback is allocation-free: captured
//! bursts are copied into pre-allocated buffers from a recycle free-list (pool empty = drop the
//! chunk, never allocate on the capture thread). Format matches the host decoder + the Linux
//! client: 48 kHz **stereo**, 20 ms, Opus VOIP.
use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
@@ -13,7 +16,7 @@ use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TrySendError};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
@@ -23,6 +26,10 @@ const SAMPLE_RATE: i32 = 48_000;
const FRAME_SAMPLES: usize = 960;
/// Captured-chunk hand-off depth (each ~ one burst); drops on overflow (best-effort uplink).
const RING_CHUNKS: usize = 64;
/// Free-list buffer capacity, in interleaved f32 samples: comfortably above a LowLatency input
/// burst (typically ≤ ~480 frames). A device with larger bursts costs each buffer a one-time grow
/// on the capture thread, after which the steady state is allocation-free again.
const CHUNK_CAP_SAMPLES: usize = 1920; // 20 ms stereo
/// Opus VOIP target bitrate (speech; tunable).
const MIC_BITRATE: i32 = 64_000;
@@ -38,56 +45,109 @@ impl MicCapture {
/// forwards captured PCM to a channel, then spawn the Opus encode + uplink thread. `None` on
/// failure (the caller leaves the rest of the session streaming).
pub fn start(client: Arc<NativeClient>) -> Option<MicCapture> {
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
let captured = Arc::new(AtomicU64::new(0));
let cb_captured = captured.clone();
// Chunks discarded on the capture thread (free-list empty / encoder lagging); logged
// throttled from the encode worker.
let dropped = Arc::new(AtomicU64::new(0));
let callback = move |_s: &AudioStream, data: *mut c_void, num_frames: i32| {
let n = num_frames as usize * CHANNELS;
// SAFETY: for an input stream AAudio provides `num_frames * channel_count` captured F32
// samples at `data` (read-only for us).
let inp = unsafe { std::slice::from_raw_parts(data as *const f32, n) };
match tx.try_send(inp.to_vec()) {
Ok(()) | Err(TrySendError::Full(_)) => {} // drop-newest if the encoder lags
Err(TrySendError::Disconnected(_)) => return AudioCallbackResult::Stop,
// One open attempt at a given sharing mode (same pattern as [`crate::audio`]: `open_stream`
// consumes the builder AND the callback, so each try rebuilds the channels it captures).
let try_open = |sharing: AudioSharingMode| -> ndk::audio::Result<(
AudioStream,
Receiver<Vec<f32>>,
SyncSender<Vec<f32>>,
)> {
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list, mirroring the playback path: the realtime capture callback must
// not touch the allocator (Android's Scudo has unbounded malloc/free tail latency — an
// allocation here is a missed burst), so it pops a pre-allocated buffer, copies the
// burst in and sends it; the encode worker returns drained buffers. Pool empty = DROP
// the chunk (counted) rather than allocate.
let (free_tx, free_rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
for _ in 0..RING_CHUNKS {
let _ = free_tx.try_send(Vec::with_capacity(CHUNK_CAP_SAMPLES));
}
cb_captured.fetch_add(num_frames as u64, Ordering::Relaxed);
AudioCallbackResult::Continue
let cb_captured = captured.clone();
let cb_dropped = dropped.clone();
let cb_free_tx = free_tx.clone(); // returns the buffer when the data channel is full
let callback = move |_s: &AudioStream, data: *mut c_void, num_frames: i32| {
let n = num_frames as usize * CHANNELS;
// SAFETY: for an input stream AAudio provides `num_frames * channel_count` captured
// F32 samples at `data` (read-only for us).
let inp = unsafe { std::slice::from_raw_parts(data as *const f32, n) };
cb_captured.fetch_add(num_frames as u64, Ordering::Relaxed);
match free_rx.try_recv() {
Ok(mut buf) => {
buf.clear();
buf.extend_from_slice(inp); // retained capacity — no realloc past the first
match tx.try_send(buf) {
Ok(()) => {}
Err(TrySendError::Full(buf)) => {
// Encoder lagging: drop the chunk, hand the buffer straight back.
let _ = cb_free_tx.try_send(buf);
cb_dropped.fetch_add(1, Ordering::Relaxed);
}
Err(TrySendError::Disconnected(_)) => return AudioCallbackResult::Stop,
}
}
// Pool empty (every buffer in flight): drop, never allocate on this thread.
Err(_) => {
cb_dropped.fetch_add(1, Ordering::Relaxed);
}
}
AudioCallbackResult::Continue
};
let stream = AudioStreamBuilder::new()?
.direction(AudioDirection::Input)
.sample_rate(SAMPLE_RATE)
.channel_count(CHANNELS as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(sharing)
.data_callback(Box::new(callback))
.error_callback(Box::new(|_s, e| {
log::warn!("mic: AAudio error (device reroute/disconnect?): {e:?}");
}))
.open_stream()?;
Ok((stream, rx, free_tx))
};
let stream = AudioStreamBuilder::new()
.map_err(|e| log::error!("mic: AudioStreamBuilder::new: {e}"))
.ok()?
.direction(AudioDirection::Input)
.sample_rate(SAMPLE_RATE)
.channel_count(CHANNELS as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(AudioSharingMode::Shared)
.data_callback(Box::new(callback))
.error_callback(Box::new(|_s, e| {
log::warn!("mic: AAudio error (device reroute/disconnect?): {e:?}");
}))
.open_stream()
.map_err(|e| log::error!("mic: open_stream (RECORD_AUDIO granted?): {e}"))
.ok()?;
// Exclusive first — MMAP-exclusive is AAudio's lowest-latency path — falling back to Shared
// when the device refuses (no MMAP, mic claimed, …). The started-log below prints the mode
// the device actually GRANTED (`share=`).
let (stream, rx, free_tx) = match try_open(AudioSharingMode::Exclusive) {
Ok(opened) => opened,
Err(e) => {
log::info!("mic: Exclusive open failed ({e}) — retrying Shared");
match try_open(AudioSharingMode::Shared) {
Ok(opened) => opened,
Err(e) => {
log::error!("mic: open_stream (RECORD_AUDIO granted?): {e}");
return None;
}
}
}
};
if let Err(e) = stream.request_start() {
log::error!("mic: request_start: {e}");
return None;
}
log::info!(
"mic: AAudio input started rate={} ch={} fmt={:?}",
"mic: AAudio input started rate={} ch={} fmt={:?} share={:?}",
stream.sample_rate(),
stream.channel_count(),
stream.format(),
stream.sharing_mode(),
);
let shutdown = Arc::new(AtomicBool::new(false));
let sd = shutdown.clone();
let join = std::thread::Builder::new()
.name("pf-mic".into())
.spawn(move || encode_loop(client, rx, sd, captured))
.spawn(move || encode_loop(client, rx, free_tx, sd, captured, dropped))
.ok();
Some(MicCapture {
@@ -109,11 +169,15 @@ impl Drop for MicCapture {
}
/// Consumer: drain captured f32 → accumulate → Opus `encode_float` 20 ms stereo frames → `send_mic`.
/// Drained chunk buffers go back to the callback's free-list; the encode scratch is reused across
/// frames (only the packet Vec handed to `send_mic` is allocated per frame — it's sent away owned).
fn encode_loop(
client: Arc<NativeClient>,
rx: Receiver<Vec<f32>>,
free_tx: SyncSender<Vec<f32>>,
shutdown: Arc<AtomicBool>,
captured: Arc<AtomicU64>,
dropped: Arc<AtomicU64>,
) {
let mut enc = match opus::Encoder::new(
SAMPLE_RATE as u32,
@@ -130,6 +194,7 @@ fn encode_loop(
let frame = FRAME_SAMPLES * CHANNELS;
let mut ring: VecDeque<f32> = VecDeque::with_capacity(frame * 4);
let mut pcm = vec![0f32; frame]; // reusable encode scratch (one 20 ms frame)
let mut out = vec![0u8; 4000]; // max Opus packet for a 20 ms frame fits easily
let mut seq: u32 = 0;
let mut sent: u64 = 0;
@@ -137,12 +202,19 @@ fn encode_loop(
while !shutdown.load(Ordering::Relaxed) {
match rx.recv_timeout(Duration::from_millis(100)) {
Ok(chunk) => ring.extend(chunk),
Ok(mut chunk) => {
// `drain(..)` keeps the Vec's capacity; hand the emptied buffer back to the
// callback's free-list (dropped only if the pool is momentarily full).
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
Err(RecvTimeoutError::Timeout) => continue, // wake to re-check shutdown
Err(RecvTimeoutError::Disconnected) => break,
}
while ring.len() >= frame {
let pcm: Vec<f32> = ring.drain(..frame).collect();
for (dst, src) in pcm.iter_mut().zip(ring.drain(..frame)) {
*dst = src;
}
for &s in &pcm {
peak = peak.max(s.abs());
}
@@ -157,8 +229,9 @@ fn encode_loop(
sent += 1;
if sent % 250 == 0 {
log::info!(
"mic: sent={sent} captured_frames={} peak={peak:.3}",
"mic: sent={sent} captured_frames={} dropped_chunks={} peak={peak:.3}",
captured.load(Ordering::Relaxed),
dropped.load(Ordering::Relaxed),
);
peak = 0.0;
}
@@ -168,7 +241,8 @@ fn encode_loop(
}
}
log::info!(
"mic: stopped (sent={sent} captured_frames={})",
"mic: stopped (sent={sent} captured_frames={} dropped_chunks={})",
captured.load(Ordering::Relaxed),
dropped.load(Ordering::Relaxed),
);
}
-756
View File
@@ -1,756 +0,0 @@
//! Session lifecycle + plane wiring over JNI.
//!
//! A connected session is a [`SessionHandle`] — an `Arc<NativeClient>` plus the decode thread it
//! feeds — boxed and handed to Kotlin as an opaque `jlong`. The connector is `Sync`, so the decode
//! thread pulls the video plane (`next_frame`) directly while Kotlin still holds the handle.
//!
//! Wired: connect/close, the video plane (HEVC `next_frame` → NDK AMediaCodec → the SurfaceView's
//! `ANativeWindow`, see [`crate::decode`]), host→client audio ([`crate::audio`]), input
//! (`send_input` — mouse/keyboard/gamepad), rumble/DualSense HID feedback ([`crate::feedback`]),
//! and the trust surface: `nativeGenerateIdentity` (persistent identity, Keystore-wrapped on the
//! Kotlin side), `nativeConnect` with identity + pin (TOFU / pinned), and `nativePair` (SPAKE2 PIN).
//!
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`.
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jdoubleArray, jint, jlong, jsize};
use jni::JNIEnv;
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use punktfunk_core::input::{InputEvent, InputKind};
use std::panic::AssertUnwindSafe;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
use std::time::Duration;
/// Run a JNI body, catching any panic at the FFI boundary and returning `default` instead.
///
/// A panic unwinding out of an `extern "system"` function aborts the whole process on Rust ≥ 1.81 —
/// a hard crash of the embedding Android app with no logcat trace. This mirrors the discipline the C
/// ABI already enforces (`punktfunk_core::abi` wraps every entry point in `catch_unwind`); the
/// `panic = "unwind"` profile in the workspace `Cargo.toml` exists precisely so these guards work.
/// We apply it to the teardown + background-thread shims (the "leaving a stream" path), where an
/// unexpected panic (e.g. a poisoned `Mutex` during concurrent teardown) must degrade to a logged
/// no-op rather than kill the app.
pub(crate) fn jni_guard<T>(default: T, f: impl FnOnce() -> T) -> T {
std::panic::catch_unwind(AssertUnwindSafe(f)).unwrap_or_else(|_| {
log::error!("punktfunk JNI: caught a panic at the FFI boundary (returning default)");
default
})
}
/// A live session behind the `jlong` handle: the connector + the decode thread it feeds.
pub(crate) struct SessionHandle {
// Read only by the android decode path (`nativeStartVideo` → `crate::decode`); on the host
// build (CI's workspace clippy/build) those readers are cfg'd out, so it's intentionally unused.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub client: Arc<NativeClient>,
video: Mutex<Option<VideoThread>>,
#[cfg(target_os = "android")]
audio: Mutex<Option<crate::audio::AudioPlayback>>,
#[cfg(target_os = "android")]
mic: Mutex<Option<crate::mic::MicCapture>>,
}
struct VideoThread {
shutdown: Arc<AtomicBool>,
join: Option<JoinHandle<()>>,
/// Live decode stats, written by the decode thread and drained ~1 Hz by `nativeVideoStats`.
stats: Arc<crate::stats::VideoStats>,
}
impl SessionHandle {
/// Signal the decode thread to stop and join it. Idempotent.
fn stop_video(&self) {
if let Some(mut vt) = self.video.lock().unwrap().take() {
vt.shutdown.store(true, Ordering::SeqCst);
if let Some(j) = vt.join.take() {
let _ = j.join();
}
}
}
/// Stop + close audio playback. Dropping the [`crate::audio::AudioPlayback`] joins its decode
/// thread and closes the AAudio stream. Idempotent.
#[cfg(target_os = "android")]
fn stop_audio(&self) {
let _ = self.audio.lock().unwrap().take();
}
/// Stop mic uplink. Dropping the [`crate::mic::MicCapture`] joins its encode thread and closes
/// the AAudio input stream. Idempotent.
#[cfg(target_os = "android")]
fn stop_mic(&self) {
let _ = self.mic.lock().unwrap().take();
}
}
impl Drop for SessionHandle {
fn drop(&mut self) {
self.stop_video();
#[cfg(target_os = "android")]
self.stop_audio();
#[cfg(target_os = "android")]
self.stop_mic();
}
}
/// SHA-256 fingerprint → 64 lowercase hex chars (matches the host log + client-rs).
fn hex32(fp: &[u8; 32]) -> String {
use std::fmt::Write;
fp.iter().fold(String::with_capacity(64), |mut s, b| {
let _ = write!(s, "{b:02x}");
s
})
}
/// 64-hex → [u8; 32]; `None` on bad length/char.
fn parse_hex32(s: &str) -> Option<[u8; 32]> {
if s.len() != 64 {
return None;
}
let mut out = [0u8; 32];
for (i, b) in out.iter_mut().enumerate() {
*b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?;
}
Some(out)
}
/// `NativeBridge.nativeGenerateIdentity(): String` — mint a fresh persistent self-signed identity.
/// Returns `"<certPem>\n-----PUNKTFUNK-KEY-----\n<keyPem>"`, or `""` on failure (logged). Kotlin
/// persists it (Keystore-wrapped) and only calls this again when the store is genuinely empty.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeGenerateIdentity<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
) -> jni::sys::jstring {
let out = match punktfunk_core::quic::endpoint::generate_identity() {
Ok((cert, key)) => format!("{cert}\n-----PUNKTFUNK-KEY-----\n{key}"),
Err(e) => {
log::error!("nativeGenerateIdentity failed: {e}");
String::new()
}
};
match env.new_string(out) {
Ok(s) => s.into_raw(),
Err(_) => JObject::null().into_raw(),
}
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, timeoutMs): Long`. `certPem`/`keyPem`
/// empty = anonymous, else presented as the persistent identity. `pinHex` empty = TOFU (read
/// `nativeHostFingerprint` after), else 64-hex SHA-256 to pin the host (mismatch → 0). `bitrateKbps`
/// 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref` wire bytes
/// (0 = Auto; unknown → Auto). `audioChannels` is the requested surround layout (2/6/8; normalized,
/// anything else → stereo) — the host clamps it and the resolved count drives playback. `timeoutMs`
/// is the handshake budget: the normal path passes a short value, the no-PIN "request access" path a
/// long one (≥ the host's approval-park window) so a slow operator approval lands on this same parked
/// connection rather than timing the client out first. Returns an opaque handle, or 0 on failure.
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'local>(
mut env: JNIEnv<'local>,
_this: JObject<'local>,
host: JString<'local>,
port: jint,
width: jint,
height: jint,
refresh_hz: jint,
cert_pem: JString<'local>,
key_pem: JString<'local>,
pin_hex: JString<'local>,
bitrate_kbps: jint,
compositor_pref: jint,
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
timeout_ms: jint,
) -> jlong {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
Err(_) => return 0,
};
let cert: String = env
.get_string(&cert_pem)
.map(Into::into)
.unwrap_or_default();
let key: String = env.get_string(&key_pem).map(Into::into).unwrap_or_default();
let pin_hex: String = env.get_string(&pin_hex).map(Into::into).unwrap_or_default();
let identity: Option<(String, String)> = if cert.is_empty() || key.is_empty() {
None
} else {
Some((cert, key))
};
let pin: Option<[u8; 32]> = if pin_hex.is_empty() {
None
} else {
match parse_hex32(&pin_hex) {
Some(fp) => Some(fp),
None => {
log::error!("nativeConnect: bad pin hex (len {})", pin_hex.len());
return 0;
}
}
};
let mode = Mode {
width: width as u32,
height: height as u32,
refresh_hz: refresh_hz as u32,
};
match NativeClient::connect(
&host,
port as u16,
mode,
CompositorPref::from_u8(compositor_pref.clamp(0, u8::MAX as jint) as u8),
GamepadPref::from_u8(gamepad_pref.clamp(0, u8::MAX as jint) as u8),
bitrate_kbps.max(0) as u32, // 0 = host default
// Advertise 10-bit + HDR ONLY when this device's display can actually present it (Kotlin
// checks Display.getHdrCapabilities() and passes the result): the host (e.g. Windows) then
// upgrades to a Main10 / BT.2020 PQ encode. On an SDR display we advertise 0 so the host
// sends a proper 8-bit BT.709 stream rather than PQ the panel would mis-tone-map. AMediaCodec
// decodes Main10 from the SPS and the decode loop signals the Surface HDR dataspace + static
// metadata (see crate::decode).
if hdr_enabled != 0 {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
} else {
0
},
// Requested surround layout (2 = stereo / 6 = 5.1 / 8 = 7.1). The host clamps to what it can
// capture and echoes the resolved count in `connector.audio_channels`, which drives the
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
None, // launch: default app
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
// Handshake budget from Kotlin: ~10 s for a normal connect, ~185 s for "request access"
// (the host parks the connection until the operator approves the device — see ConnectScreen).
Duration::from_millis(timeout_ms.max(0) as u64),
) {
Ok(client) => {
let handle = SessionHandle {
client: Arc::new(client),
video: Mutex::new(None),
#[cfg(target_os = "android")]
audio: Mutex::new(None),
#[cfg(target_os = "android")]
mic: Mutex::new(None),
};
Box::into_raw(Box::new(handle)) as jlong
}
Err(e) => {
log::error!("nativeConnect to {host}:{port} failed: {e}");
0
}
}
}
/// `NativeBridge.nativeClose(handle)` — drop the session (stops the decode thread, then RAII-tears
/// down the connector). No-op on `0`.
///
/// # Safety contract
/// `handle` must be `0` or a live handle from [`Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect`],
/// closed exactly once and not concurrently with other calls on the same handle (Kotlin owns this).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClose(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: per the contract, `handle` is a live `Box<SessionHandle>` pointer.
unsafe { drop(Box::from_raw(handle as *mut SessionHandle)) };
}
})
}
/// `NativeBridge.nativeHostFingerprint(handle): String` — the SHA-256 (64-hex) of the cert the host
/// presented on this connection. Valid after a successful `nativeConnect`; Kotlin pins it on a TOFU
/// connect. `""` on a `0` handle.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostFingerprint<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jni::sys::jstring {
let out = if handle == 0 {
String::new()
} else {
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
hex32(&h.client.host_fingerprint)
};
match env.new_string(out) {
Ok(s) => s.into_raw(),
Err(_) => JObject::null().into_raw(),
}
}
/// `NativeBridge.nativePair(host, port, certPem, keyPem, pin, name): String` — run the SPAKE2 PIN
/// ceremony, presenting our persistent identity. On success returns the host's verified fingerprint
/// (64-hex) to persist + pin; on any failure (wrong PIN / MITM / host reject / unreachable) returns
/// `""` (logged). Blocking — Kotlin calls it off the UI thread.
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativePair<'local>(
mut env: JNIEnv<'local>,
_this: JObject<'local>,
host: JString<'local>,
port: jint,
cert_pem: JString<'local>,
key_pem: JString<'local>,
pin: JString<'local>,
name: JString<'local>,
) -> jni::sys::jstring {
let g = |e: &mut JNIEnv<'local>, j: &JString<'local>| -> String {
e.get_string(j).map(Into::into).unwrap_or_default()
};
let host = g(&mut env, &host);
let cert = g(&mut env, &cert_pem);
let key = g(&mut env, &key_pem);
let pin = g(&mut env, &pin);
let name = g(&mut env, &name);
let out = if host.is_empty() || cert.is_empty() || key.is_empty() {
log::error!("nativePair: missing host/identity");
String::new()
} else {
match NativeClient::pair(
&host,
port as u16,
(&cert, &key), // borrowed identity
&pin,
&name,
Duration::from_secs(60),
) {
Ok(host_fp) => hex32(&host_fp),
Err(e) => {
// Crypto error == wrong PIN / MITM; anything else == transport/host reject.
log::error!("nativePair to {host}:{port} failed: {e}");
String::new()
}
}
};
match env.new_string(out) {
Ok(s) => s.into_raw(),
Err(_) => JObject::null().into_raw(),
}
}
/// `NativeBridge.nativeStartVideo(handle, surface)` — wrap the SurfaceView's `Surface` as an
/// `ANativeWindow` and start the HEVC decode thread rendering onto it. No-op if already started.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
env: JNIEnv,
_this: JObject,
handle: jlong,
surface: JObject,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.video.lock().unwrap();
if guard.is_some() {
return; // already streaming
}
// SAFETY: `env`/`surface` are valid JNI pointers for this call. `as *mut _` bridges any
// jni-sys version skew between the `jni` and `ndk` crates (both are raw `*mut _` pointers).
let window = match unsafe {
ndk::native_window::NativeWindow::from_surface(
env.get_native_interface() as *mut _,
surface.as_raw() as *mut _,
)
} {
Some(w) => w,
None => {
log::error!("nativeStartVideo: no ANativeWindow from Surface");
return;
}
};
let shutdown = Arc::new(AtomicBool::new(false));
let stats = Arc::new(crate::stats::VideoStats::new());
let client = h.client.clone();
let sd = shutdown.clone();
let st = stats.clone();
let join = std::thread::Builder::new()
.name("pf-decode".into())
.spawn(move || crate::decode::run(client, window, sd, st))
.ok();
*guard = Some(VideoThread {
shutdown,
join,
stats,
});
}
/// `NativeBridge.nativeStopVideo(handle)` — stop + join the decode thread (without closing the
/// session). No-op on `0`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stop_video();
}
})
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD.
/// Returns 14 doubles
/// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
/// (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — see below), or
/// `null` when no decode thread is running. Poll ~1 Hz from the UI; each call resets the measurement
/// window. Not android-gated — pure `jni` + connector reads, so it links on the host build too
/// (Kotlin only ever calls it on device).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jdoubleArray {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let snap = match h.video.lock().unwrap().as_ref() {
Some(vt) => vt.stats.drain(),
None => return std::ptr::null_mut(), // not streaming → no stats
};
let mode = h.client.mode();
let color = h.client.color;
let buf: [f64; 14] = [
snap.fps,
snap.mbps,
snap.lat_p50_ms,
snap.lat_p95_ms,
if snap.lat_valid { 1.0 } else { 0.0 },
if snap.skew_corrected { 1.0 } else { 0.0 },
mode.width as f64,
mode.height as f64,
mode.refresh_hz as f64,
h.client.frames_dropped() as f64,
// Video-feed properties the host resolved at the handshake (Welcome): encode bit depth
// (8 / 10), the CICP colour primaries + transfer code points (Kotlin maps these to a
// colour-space / HDR label — transfer 16 = PQ, 18 = HLG ⇒ HDR), and the HEVC
// chroma_format_idc (1 = 4:2:0, 3 = 4:4:4). Static for the session unless renegotiated.
h.client.bit_depth as f64,
color.primaries as f64,
color.transfer as f64,
h.client.chroma_format as f64,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
Err(_) => return std::ptr::null_mut(),
};
if env.set_double_array_region(&arr, 0, &buf).is_err() {
return std::ptr::null_mut();
}
arr.into_raw()
})
}
/// `NativeBridge.nativeStartAudio(handle)` — start the Opus→AAudio playback thread. No-op if already
/// started or on a `0` handle. Best-effort: a failure leaves video streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.audio.lock().unwrap();
if guard.is_some() {
return; // already playing
}
match crate::audio::AudioPlayback::start(h.client.clone()) {
Some(p) => *guard = Some(p),
None => log::error!("nativeStartAudio: playback init failed (video unaffected)"),
}
}
/// `NativeBridge.nativeStopAudio(handle)` — stop + join the audio thread and close AAudio (without
/// closing the session). No-op on `0`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stop_audio();
}
})
}
/// `NativeBridge.nativeStartMic(handle)` — start mic capture (AAudio input → Opus → host `send_mic`).
/// No-op if already running or on a `0` handle. Caller MUST hold RECORD_AUDIO; a failure (e.g. no
/// permission) leaves the rest of the session streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartMic(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.mic.lock().unwrap();
if guard.is_some() {
return; // already capturing
}
match crate::mic::MicCapture::start(h.client.clone()) {
Some(m) => *guard = Some(m),
None => log::error!("nativeStartMic: mic init failed (RECORD_AUDIO? — session unaffected)"),
}
}
/// `NativeBridge.nativeStopMic(handle)` — stop + join the mic thread and close the AAudio input
/// stream (without closing the session). No-op on `0`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopMic(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stop_mic();
}
})
}
// ---- Input plane: Kotlin capture → NativeClient::send_input ----------------------------------
// All four are `&self` on the `Sync` connector (send_input is a non-blocking datagram push), safe
// from the Kotlin UI thread. NOT android-gated — send_input exists on the host build too, so these
// compile everywhere (parity with nativeConnect/nativeClose). The wire codes are the GameStream
// conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
// signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
/// `NativeBridge.nativeSendPointerMove(handle, dx, dy)` — relative mouse motion (screen +y down).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerMove(
_env: JNIEnv,
_this: JObject,
handle: jlong,
dx: jint,
dy: jint,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_input is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: InputKind::MouseMove,
_pad: [0; 3],
code: 0,
x: dx,
y: dy,
flags: 0,
});
}
/// `NativeBridge.nativeSendPointerAbs(handle, x, y, surfaceWidth, surfaceHeight)` — absolute cursor
/// position: the host moves the pointer to `x`/`y` in a `surfaceWidth`×`surfaceHeight` pixel space,
/// normalizing against the size packed into `flags` as `(w << 16) | h` and mapping into the output
/// region (it drops the event if that size is zero). This is the touch "direct pointing" path — the
/// cursor jumps to the finger — and matches the Apple client's absolute touch forwarding.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerAbs(
_env: JNIEnv,
_this: JObject,
handle: jlong,
x: jint,
y: jint,
surface_width: jint,
surface_height: jint,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let w = (surface_width.max(0) as u32) & 0xffff;
let ht = (surface_height.max(0) as u32) & 0xffff;
let _ = h.client.send_input(&InputEvent {
kind: InputKind::MouseMoveAbs,
_pad: [0; 3],
code: 0,
x,
y,
flags: (w << 16) | ht,
});
}
/// `NativeBridge.nativeSendPointerButton(handle, button, down)` — one button transition.
/// `button`: GameStream id (1=left, 2=middle, 3=right, 4=X1, 5=X2). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerButton(
_env: JNIEnv,
_this: JObject,
handle: jlong,
button: jint,
down: jboolean,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: if down != 0 {
InputKind::MouseButtonDown
} else {
InputKind::MouseButtonUp
},
_pad: [0; 3],
code: button as u32,
x: 0,
y: 0,
flags: 0,
});
}
/// `NativeBridge.nativeSendScroll(handle, axis, delta)` — one scroll step. `axis`: 0=vertical,
/// 1=horizontal. `delta`: signed, WHEEL_DELTA(120)-scaled, +=up/right.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendScroll(
_env: JNIEnv,
_this: JObject,
handle: jlong,
axis: jint,
delta: jint,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: InputKind::MouseScroll,
_pad: [0; 3],
code: axis as u32,
x: delta,
y: 0,
flags: 0,
});
}
/// `NativeBridge.nativeSendKey(handle, vk, down, mods)` — one key transition. `vk`: Windows
/// Virtual-Key code (0 = unmapped → dropped). `down`: 1=press, 0=release. `mods`: VK modifier
/// bitmask (0 for now — the host folds modifiers from the L/R modifier key events themselves).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
_env: JNIEnv,
_this: JObject,
handle: jlong,
vk: jint,
down: jboolean,
mods: jint,
) {
if handle == 0 || vk == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: if down != 0 {
InputKind::KeyDown
} else {
InputKind::KeyUp
},
_pad: [0; 3],
code: vk as u32,
x: 0,
y: 0,
flags: mods as u32,
});
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Single-pad model: exactly one controller, forwarded as pad 0 (flags = 0). Buttons carry the
// gamepad::BTN_* bit in `code` and pressed/released in `x` (1/0); axes carry the gamepad::AXIS_* id
// in `code` and the value in `x` (sticks i16 32768..32767, +y = up; triggers 0..255). The host
// accumulates the incremental events into its virtual xpad. Wire contract: input.rs::gamepad.
/// `NativeBridge.nativeSendGamepadButton(handle, bit, down)` — one gamepad button transition.
/// `bit`: a `gamepad::BTN_*` bit (e.g. BTN_A = 0x1000). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadButton(
_env: JNIEnv,
_this: JObject,
handle: jlong,
bit: jint,
down: jboolean,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: InputKind::GamepadButton,
_pad: [0; 3],
code: bit as u32,
x: i32::from(down != 0),
y: 0,
flags: 0, // pad index 0 — single-pad model
});
}
/// `NativeBridge.nativeSendGamepadAxis(handle, axisId, value)` — one gamepad axis update.
/// `axisId`: a `gamepad::AXIS_*` id (LS_X=0..RT=5). `value`: stick i16 (32768..32767, +y=up) or
/// trigger 0..255.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadAxis(
_env: JNIEnv,
_this: JObject,
handle: jlong,
axis_id: jint,
value: jint,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind: InputKind::GamepadAxis,
_pad: [0; 3],
code: axis_id as u32,
x: value,
y: 0,
flags: 0, // pad index 0 — single-pad model
});
}
@@ -0,0 +1,244 @@
//! Connect lifecycle + the trust surface: identity mint, connect (TOFU / pinned), close,
//! host-fingerprint read, and the SPAKE2 PIN pairing ceremony.
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use super::{hex32, jni_guard, parse_hex32, SessionHandle};
/// `NativeBridge.nativeGenerateIdentity(): String` — mint a fresh persistent self-signed identity.
/// Returns `"<certPem>\n-----PUNKTFUNK-KEY-----\n<keyPem>"`, or `""` on failure (logged). Kotlin
/// persists it (Keystore-wrapped) and only calls this again when the store is genuinely empty.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeGenerateIdentity<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
) -> jni::sys::jstring {
let out = match punktfunk_core::quic::endpoint::generate_identity() {
Ok((cert, key)) => format!("{cert}\n-----PUNKTFUNK-KEY-----\n{key}"),
Err(e) => {
log::error!("nativeGenerateIdentity failed: {e}");
String::new()
}
};
match env.new_string(out) {
Ok(s) => s.into_raw(),
Err(_) => JObject::null().into_raw(),
}
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs): Long`.
/// `certPem`/`keyPem` empty = anonymous, else presented as the persistent identity. `pinHex` empty
/// = TOFU (read `nativeHostFingerprint` after), else 64-hex SHA-256 to pin the host (mismatch → 0).
/// `bitrateKbps` 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref`
/// wire bytes (0 = Auto; unknown → Auto). `audioChannels` is the requested surround layout (2/6/8;
/// normalized, anything else → stereo) — the host clamps it and the resolved count drives playback.
/// `preferredCodec` is the soft codec preference wire byte (0 = Auto). `timeoutMs` is the handshake
/// budget: the normal path passes a short value, the no-PIN "request access" path a long one (≥ the
/// host's approval-park window) so a slow operator approval lands on this same parked connection
/// rather than timing the client out first. Returns an opaque handle, or 0 on failure.
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'local>(
mut env: JNIEnv<'local>,
_this: JObject<'local>,
host: JString<'local>,
port: jint,
width: jint,
height: jint,
refresh_hz: jint,
cert_pem: JString<'local>,
key_pem: JString<'local>,
pin_hex: JString<'local>,
bitrate_kbps: jint,
compositor_pref: jint,
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
preferred_codec: jint,
timeout_ms: jint,
) -> jlong {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
Err(_) => return 0,
};
let cert: String = env
.get_string(&cert_pem)
.map(Into::into)
.unwrap_or_default();
let key: String = env.get_string(&key_pem).map(Into::into).unwrap_or_default();
let pin_hex: String = env.get_string(&pin_hex).map(Into::into).unwrap_or_default();
let identity: Option<(String, String)> = if cert.is_empty() || key.is_empty() {
None
} else {
Some((cert, key))
};
let pin: Option<[u8; 32]> = if pin_hex.is_empty() {
None
} else {
match parse_hex32(&pin_hex) {
Some(fp) => Some(fp),
None => {
log::error!("nativeConnect: bad pin hex (len {})", pin_hex.len());
return 0;
}
}
};
let mode = Mode {
width: width as u32,
height: height as u32,
refresh_hz: refresh_hz as u32,
};
match NativeClient::connect(
&host,
port as u16,
mode,
CompositorPref::from_u8(compositor_pref.clamp(0, u8::MAX as jint) as u8),
GamepadPref::from_u8(gamepad_pref.clamp(0, u8::MAX as jint) as u8),
bitrate_kbps.max(0) as u32, // 0 = host default
// Advertise 10-bit + HDR ONLY when this device's display can actually present it (Kotlin
// checks Display.getHdrCapabilities() and passes the result): the host (e.g. Windows) then
// upgrades to a Main10 / BT.2020 PQ encode. On an SDR display we advertise 0 so the host
// sends a proper 8-bit BT.709 stream rather than PQ the panel would mis-tone-map. AMediaCodec
// decodes Main10 from the SPS and the decode loop signals the Surface HDR dataspace + static
// metadata (see crate::decode).
if hdr_enabled != 0 {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
} else {
0
},
// Requested surround layout (2 = stereo / 6 = 5.1 / 8 = 7.1). The host clamps to what it can
// capture and echoes the resolved count in `connector.audio_channels`, which drives the
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
// Codecs this device can decode — AMediaCodec decodes both HEVC and H.264 (AV1 isn't wired;
// hosts don't emit it on the native path yet). The host resolves the emitted codec from these
// + the soft `preferred_codec` and echoes it in `connector.codec`, which drives the mime below.
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC,
preferred_codec.clamp(0, u8::MAX as jint) as u8,
None, // launch: default app
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
// Handshake budget from Kotlin: ~10 s for a normal connect, ~185 s for "request access"
// (the host parks the connection until the operator approves the device — see ConnectScreen).
Duration::from_millis(timeout_ms.max(0) as u64),
) {
Ok(client) => {
let handle = SessionHandle {
client: Arc::new(client),
stats: Arc::new(crate::stats::VideoStats::new()),
video: Mutex::new(None),
#[cfg(target_os = "android")]
audio: Mutex::new(None),
#[cfg(target_os = "android")]
mic: Mutex::new(None),
};
Box::into_raw(Box::new(handle)) as jlong
}
Err(e) => {
log::error!("nativeConnect to {host}:{port} failed: {e}");
0
}
}
}
/// `NativeBridge.nativeClose(handle)` — drop the session (stops the decode thread, then RAII-tears
/// down the connector). No-op on `0`.
///
/// # Safety contract
/// `handle` must be `0` or a live handle from [`Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect`],
/// closed exactly once and not concurrently with other calls on the same handle (Kotlin owns this).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClose(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: per the contract, `handle` is a live `Box<SessionHandle>` pointer.
unsafe { drop(Box::from_raw(handle as *mut SessionHandle)) };
}
})
}
/// `NativeBridge.nativeHostFingerprint(handle): String` — the SHA-256 (64-hex) of the cert the host
/// presented on this connection. Valid after a successful `nativeConnect`; Kotlin pins it on a TOFU
/// connect. `""` on a `0` handle.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostFingerprint<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jni::sys::jstring {
let out = if handle == 0 {
String::new()
} else {
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
hex32(&h.client.host_fingerprint)
};
match env.new_string(out) {
Ok(s) => s.into_raw(),
Err(_) => JObject::null().into_raw(),
}
}
/// `NativeBridge.nativePair(host, port, certPem, keyPem, pin, name): String` — run the SPAKE2 PIN
/// ceremony, presenting our persistent identity. On success returns the host's verified fingerprint
/// (64-hex) to persist + pin; on any failure (wrong PIN / MITM / host reject / unreachable) returns
/// `""` (logged). Blocking — Kotlin calls it off the UI thread.
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativePair<'local>(
mut env: JNIEnv<'local>,
_this: JObject<'local>,
host: JString<'local>,
port: jint,
cert_pem: JString<'local>,
key_pem: JString<'local>,
pin: JString<'local>,
name: JString<'local>,
) -> jni::sys::jstring {
let g = |e: &mut JNIEnv<'local>, j: &JString<'local>| -> String {
e.get_string(j).map(Into::into).unwrap_or_default()
};
let host = g(&mut env, &host);
let cert = g(&mut env, &cert_pem);
let key = g(&mut env, &key_pem);
let pin = g(&mut env, &pin);
let name = g(&mut env, &name);
let out = if host.is_empty() || cert.is_empty() || key.is_empty() {
log::error!("nativePair: missing host/identity");
String::new()
} else {
match NativeClient::pair(
&host,
port as u16,
(&cert, &key), // borrowed identity
&pin,
&name,
Duration::from_secs(60),
) {
Ok(host_fp) => hex32(&host_fp),
Err(e) => {
// Crypto error == wrong PIN / MITM; anything else == transport/host reject.
log::error!("nativePair to {host}:{port} failed: {e}");
String::new()
}
}
};
match env.new_string(out) {
Ok(s) => s.into_raw(),
Err(_) => JObject::null().into_raw(),
}
}
+187
View File
@@ -0,0 +1,187 @@
//! Input plane: Kotlin capture → `NativeClient::send_input`.
//!
//! All shims are `&self` on the `Sync` connector (send_input is a non-blocking datagram push), safe
//! from the Kotlin UI thread. NOT android-gated — send_input exists on the host build too, so these
//! compile everywhere (parity with nativeConnect/nativeClose). The wire codes are the GameStream
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::JObject;
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use super::SessionHandle;
/// Shared shim body: guard against a `0` handle, deref, and push one [`InputEvent`].
fn send_event(handle: jlong, kind: InputKind, code: u32, x: i32, y: i32, flags: u32) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_input is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_input(&InputEvent {
kind,
_pad: [0; 3],
code,
x,
y,
flags,
});
}
/// `NativeBridge.nativeSendPointerMove(handle, dx, dy)` — relative mouse motion (screen +y down).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerMove(
_env: JNIEnv,
_this: JObject,
handle: jlong,
dx: jint,
dy: jint,
) {
send_event(handle, InputKind::MouseMove, 0, dx, dy, 0);
}
/// `NativeBridge.nativeSendPointerAbs(handle, x, y, surfaceWidth, surfaceHeight)` — absolute cursor
/// position: the host moves the pointer to `x`/`y` in a `surfaceWidth`×`surfaceHeight` pixel space,
/// normalizing against the size packed into `flags` as `(w << 16) | h` and mapping into the output
/// region (it drops the event if that size is zero). This is the touch "direct pointing" path — the
/// cursor jumps to the finger — and matches the Apple client's absolute touch forwarding.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerAbs(
_env: JNIEnv,
_this: JObject,
handle: jlong,
x: jint,
y: jint,
surface_width: jint,
surface_height: jint,
) {
let w = (surface_width.max(0) as u32) & 0xffff;
let ht = (surface_height.max(0) as u32) & 0xffff;
send_event(handle, InputKind::MouseMoveAbs, 0, x, y, (w << 16) | ht);
}
/// `NativeBridge.nativeSendPointerButton(handle, button, down)` — one button transition.
/// `button`: GameStream id (1=left, 2=middle, 3=right, 4=X1, 5=X2). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPointerButton(
_env: JNIEnv,
_this: JObject,
handle: jlong,
button: jint,
down: jboolean,
) {
let kind = if down != 0 {
InputKind::MouseButtonDown
} else {
InputKind::MouseButtonUp
};
send_event(handle, kind, button as u32, 0, 0, 0);
}
/// `NativeBridge.nativeSendScroll(handle, axis, delta)` — one scroll step. `axis`: 0=vertical,
/// 1=horizontal. `delta`: signed, WHEEL_DELTA(120)-scaled, +=up/right.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendScroll(
_env: JNIEnv,
_this: JObject,
handle: jlong,
axis: jint,
delta: jint,
) {
send_event(handle, InputKind::MouseScroll, axis as u32, delta, 0, 0);
}
/// `NativeBridge.nativeSendTouch(handle, id, kind, x, y, surfaceWidth, surfaceHeight)` — one REAL
/// touchscreen transition (`kind`: 0=down 1=move 2=up), for the touch-passthrough input mode. `id`
/// distinguishes fingers (reusable after up); coordinates are pixels on the client's touch
/// surface, whose size rides in `flags` so the host can rescale into the output (identical
/// packing to MouseMoveAbs). On up only the id matters. The host injects a real touch contact
/// (libei touchscreen / wlroots / SendInput).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendTouch(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
kind: jint,
x: jint,
y: jint,
surface_width: jint,
surface_height: jint,
) {
let kind = match kind {
0 => InputKind::TouchDown,
1 => InputKind::TouchMove,
_ => InputKind::TouchUp,
};
let w = (surface_width.max(0) as u32) & 0xffff;
let h = (surface_height.max(0) as u32) & 0xffff;
send_event(handle, kind, id as u32, x, y, (w << 16) | h);
}
/// `NativeBridge.nativeSendKey(handle, vk, down, mods)` — one key transition. `vk`: Windows
/// Virtual-Key code (0 = unmapped → dropped). `down`: 1=press, 0=release. `mods`: VK modifier
/// bitmask (0 for now — the host folds modifiers from the L/R modifier key events themselves).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
_env: JNIEnv,
_this: JObject,
handle: jlong,
vk: jint,
down: jboolean,
mods: jint,
) {
if vk == 0 {
return;
}
let kind = if down != 0 {
InputKind::KeyDown
} else {
InputKind::KeyUp
};
send_event(handle, kind, vk as u32, 0, 0, mods as u32);
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Single-pad model: exactly one controller, forwarded as pad 0 (flags = 0). Buttons carry the
// gamepad::BTN_* bit in `code` and pressed/released in `x` (1/0); axes carry the gamepad::AXIS_* id
// in `code` and the value in `x` (sticks i16 32768..32767, +y = up; triggers 0..255). The host
// accumulates the incremental events into its virtual xpad. Wire contract: input.rs::gamepad.
/// `NativeBridge.nativeSendGamepadButton(handle, bit, down)` — one gamepad button transition.
/// `bit`: a `gamepad::BTN_*` bit (e.g. BTN_A = 0x1000). `down`: 1=press, 0=release.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadButton(
_env: JNIEnv,
_this: JObject,
handle: jlong,
bit: jint,
down: jboolean,
) {
// flags = 0: pad index 0 — single-pad model.
send_event(
handle,
InputKind::GamepadButton,
bit as u32,
i32::from(down != 0),
0,
0,
);
}
/// `NativeBridge.nativeSendGamepadAxis(handle, axisId, value)` — one gamepad axis update.
/// `axisId`: a `gamepad::AXIS_*` id (LS_X=0..RT=5). `value`: stick i16 (32768..32767, +y=up) or
/// trigger 0..255.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendGamepadAxis(
_env: JNIEnv,
_this: JObject,
handle: jlong,
axis_id: jint,
value: jint,
) {
// flags = 0: pad index 0 — single-pad model.
send_event(handle, InputKind::GamepadAxis, axis_id as u32, value, 0, 0);
}
+124
View File
@@ -0,0 +1,124 @@
//! Session lifecycle + plane wiring over JNI.
//!
//! A connected session is a [`SessionHandle`] — an `Arc<NativeClient>` plus the decode thread it
//! feeds — boxed and handed to Kotlin as an opaque `jlong`. The connector is `Sync`, so the decode
//! thread pulls the video plane (`next_frame`) directly while Kotlin still holds the handle.
//!
//! Wired: connect/close, the video plane (HEVC `next_frame` → NDK AMediaCodec → the SurfaceView's
//! `ANativeWindow`, see [`crate::decode`]), host→client audio ([`crate::audio`]), input
//! (`send_input` — mouse/keyboard/gamepad), rumble/DualSense HID feedback ([`crate::feedback`]),
//! and the trust surface: `nativeGenerateIdentity` (persistent identity, Keystore-wrapped on the
//! Kotlin side), `nativeConnect` with identity + pin (TOFU / pinned), and `nativePair` (SPAKE2 PIN).
//!
//! Split by concern: [`connect`] (identity + connect/close + the trust surface), [`planes`]
//! (video/audio/mic start/stop + the stats drain), [`input`] (the input-plane shims). This module
//! keeps the shared infrastructure they all deref through.
//!
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`.
mod connect;
mod input;
mod planes;
use punktfunk_core::client::NativeClient;
use std::panic::AssertUnwindSafe;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
/// Run a JNI body, catching any panic at the FFI boundary and returning `default` instead.
///
/// A panic unwinding out of an `extern "system"` function aborts the whole process on Rust ≥ 1.81 —
/// a hard crash of the embedding Android app with no logcat trace. This mirrors the discipline the C
/// ABI already enforces (`punktfunk_core::abi` wraps every entry point in `catch_unwind`); the
/// `panic = "unwind"` profile in the workspace `Cargo.toml` exists precisely so these guards work.
/// We apply it to the teardown + background-thread shims (the "leaving a stream" path), where an
/// unexpected panic (e.g. a poisoned `Mutex` during concurrent teardown) must degrade to a logged
/// no-op rather than kill the app.
pub(crate) fn jni_guard<T>(default: T, f: impl FnOnce() -> T) -> T {
std::panic::catch_unwind(AssertUnwindSafe(f)).unwrap_or_else(|_| {
log::error!("punktfunk JNI: caught a panic at the FFI boundary (returning default)");
default
})
}
/// A live session behind the `jlong` handle: the connector + the decode thread it feeds.
pub(crate) struct SessionHandle {
// Read only by the android decode path (`nativeStartVideo` → `crate::decode`); on the host
// build (CI's workspace clippy/build) those readers are cfg'd out, so it's intentionally unused.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub client: Arc<NativeClient>,
/// Live decode stats, written by the decode thread and drained ~1 Hz by `nativeVideoStats`.
/// Session-lifetime (not per `VideoThread`) so the HUD's enable gate set via
/// `nativeSetVideoStatsEnabled` survives surface teardown/recreate and can land before
/// `nativeStartVideo` — enabling resets the window, so no stale data leaks across restarts.
pub stats: Arc<crate::stats::VideoStats>,
video: Mutex<Option<VideoThread>>,
#[cfg(target_os = "android")]
audio: Mutex<Option<crate::audio::AudioPlayback>>,
#[cfg(target_os = "android")]
mic: Mutex<Option<crate::mic::MicCapture>>,
}
struct VideoThread {
shutdown: Arc<AtomicBool>,
join: Option<JoinHandle<()>>,
}
impl SessionHandle {
/// Signal the decode thread to stop and join it. Idempotent.
fn stop_video(&self) {
if let Some(mut vt) = self.video.lock().unwrap().take() {
vt.shutdown.store(true, Ordering::SeqCst);
if let Some(j) = vt.join.take() {
let _ = j.join();
}
}
}
/// Stop + close audio playback. Dropping the [`crate::audio::AudioPlayback`] joins its decode
/// thread and closes the AAudio stream. Idempotent.
#[cfg(target_os = "android")]
fn stop_audio(&self) {
let _ = self.audio.lock().unwrap().take();
}
/// Stop mic uplink. Dropping the [`crate::mic::MicCapture`] joins its encode thread and closes
/// the AAudio input stream. Idempotent.
#[cfg(target_os = "android")]
fn stop_mic(&self) {
let _ = self.mic.lock().unwrap().take();
}
}
impl Drop for SessionHandle {
fn drop(&mut self) {
self.stop_video();
#[cfg(target_os = "android")]
self.stop_audio();
#[cfg(target_os = "android")]
self.stop_mic();
}
}
/// SHA-256 fingerprint → 64 lowercase hex chars (matches the host log + client-rs).
fn hex32(fp: &[u8; 32]) -> String {
use std::fmt::Write;
fp.iter().fold(String::with_capacity(64), |mut s, b| {
let _ = write!(s, "{b:02x}");
s
})
}
/// 64-hex → [u8; 32]; `None` on bad length/char.
fn parse_hex32(s: &str) -> Option<[u8; 32]> {
if s.len() != 64 {
return None;
}
let mut out = [0u8; 32];
for (i, b) in out.iter_mut().enumerate() {
*b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?;
}
Some(out)
}
@@ -0,0 +1,248 @@
//! Plane start/stop: video (HEVC decode → Surface), host→client audio, mic uplink — plus the
//! ~1 Hz decode-stats drain for the HUD.
use jni::objects::JObject;
use jni::sys::{jboolean, jdoubleArray, jlong, jsize};
use jni::JNIEnv;
use super::{jni_guard, SessionHandle};
/// `NativeBridge.nativeStartVideo(handle, surface)` — wrap the SurfaceView's `Surface` as an
/// `ANativeWindow` and start the HEVC decode thread rendering onto it. No-op if already started.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
env: JNIEnv,
_this: JObject,
handle: jlong,
surface: JObject,
) {
use super::VideoThread;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.video.lock().unwrap();
if guard.is_some() {
return; // already streaming
}
// SAFETY: `env`/`surface` are valid JNI pointers for this call. `as *mut _` bridges any
// jni-sys version skew between the `jni` and `ndk` crates (both are raw `*mut _` pointers).
let window = match unsafe {
ndk::native_window::NativeWindow::from_surface(
env.get_native_interface() as *mut _,
surface.as_raw() as *mut _,
)
} {
Some(w) => w,
None => {
log::error!("nativeStartVideo: no ANativeWindow from Surface");
return;
}
};
let shutdown = Arc::new(AtomicBool::new(false));
let client = h.client.clone();
let sd = shutdown.clone();
let st = h.stats.clone(); // session-lifetime stats (gate survives surface recreate)
let join = std::thread::Builder::new()
.name("pf-decode".into())
.spawn(move || crate::decode::run(client, window, sd, st))
.ok();
*guard = Some(VideoThread { shutdown, join });
}
/// `NativeBridge.nativeStopVideo(handle)` — stop + join the decode thread (without closing the
/// session). No-op on `0`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stop_video();
}
})
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD
/// (unified stats spec, `design/stats-unification.md`). Returns 18 doubles
/// `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
/// netP50Ms]`
/// (the two flags are 1.0/0.0; indexes 015 match the previous 16-double layout — 013 the original
/// 14-double one with the latency pair re-based to the end-to-end capture→decoded headline, 14/15
/// the stage p50s tiling it: `host+network` = capture→received, `decode` = received→decoded; 16/17
/// are the Phase-2 split of the `host+network` term from the per-AU 0xCF host timings — `host` =
/// the host's capture→sent, `network` = the remainder — both 0.0 when no timing matched this
/// window, i.e. an old host), or `null` when no decode thread is running. Poll ~1 Hz from the UI; each call
/// resets the measurement window. Not android-gated — pure `jni` + connector reads, so it links on
/// the host build too (Kotlin only ever calls it on device).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jdoubleArray {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
if h.video.lock().unwrap().is_none() {
return std::ptr::null_mut(); // not streaming → no stats
}
let snap = h.stats.drain();
let mode = h.client.mode();
let color = h.client.color;
let buf: [f64; 18] = [
snap.fps,
snap.mbps,
snap.e2e_p50_ms,
snap.e2e_p95_ms,
if snap.lat_valid { 1.0 } else { 0.0 },
if snap.skew_corrected { 1.0 } else { 0.0 },
mode.width as f64,
mode.height as f64,
mode.refresh_hz as f64,
h.client.frames_dropped() as f64,
// Video-feed properties the host resolved at the handshake (Welcome): encode bit depth
// (8 / 10), the CICP colour primaries + transfer code points (Kotlin maps these to a
// colour-space / HDR label — transfer 16 = PQ, 18 = HLG ⇒ HDR), and the HEVC
// chroma_format_idc (1 = 4:2:0, 3 = 4:4:4). Static for the session unless renegotiated.
h.client.bit_depth as f64,
color.primaries as f64,
color.transfer as f64,
h.client.chroma_format as f64,
// Stage p50s tiling the end-to-end headline (appended to keep 013 index-compatible).
snap.hostnet_p50_ms,
snap.decode_p50_ms,
// Phase-2 host/network split of the `host+network` stage (0xCF host timings): 0.0
// when no timing matched this window (old host) — the HUD keeps the combined term.
snap.host_p50_ms,
snap.net_p50_ms,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
Err(_) => return std::ptr::null_mut(),
};
if env.set_double_array_region(&arr, 0, &buf).is_err() {
return std::ptr::null_mut();
}
arr.into_raw()
})
}
/// `NativeBridge.nativeSetVideoStatsEnabled(handle, enabled)` — gate per-frame stats sampling on the
/// HUD actually being visible: while disabled the decode thread skips the clock read + lock per AU.
/// Enabling resets the measurement window so a later show never reports stale data. Sticky for the
/// session (survives video stop/start across surface recreation). No-op on `0`. Not android-gated —
/// pure `jni` + an atomic store, so it links on the host build too.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetVideoStatsEnabled(
_env: JNIEnv,
_this: JObject,
handle: jlong,
enabled: jboolean,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stats.set_enabled(enabled != 0);
}
})
}
/// `NativeBridge.nativeStartAudio(handle)` — start the Opus→AAudio playback thread. No-op if already
/// started or on a `0` handle. Best-effort: a failure leaves video streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.audio.lock().unwrap();
if guard.is_some() {
return; // already playing
}
match crate::audio::AudioPlayback::start(h.client.clone()) {
Some(p) => *guard = Some(p),
None => log::error!("nativeStartAudio: playback init failed (video unaffected)"),
}
}
/// `NativeBridge.nativeStopAudio(handle)` — stop + join the audio thread and close AAudio (without
/// closing the session). No-op on `0`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stop_audio();
}
})
}
/// `NativeBridge.nativeStartMic(handle)` — start mic capture (AAudio input → Opus → host `send_mic`).
/// No-op if already running or on a `0` handle. Caller MUST hold RECORD_AUDIO; a failure (e.g. no
/// permission) leaves the rest of the session streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartMic(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
if handle == 0 {
return;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.mic.lock().unwrap();
if guard.is_some() {
return; // already capturing
}
match crate::mic::MicCapture::start(h.client.clone()) {
Some(m) => *guard = Some(m),
None => log::error!("nativeStartMic: mic init failed (RECORD_AUDIO? — session unaffected)"),
}
}
/// `NativeBridge.nativeStopMic(handle)` — stop + join the mic thread and close the AAudio input
/// stream (without closing the session). No-op on `0`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopMic(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.stop_mic();
}
})
}
+171 -38
View File
@@ -1,15 +1,27 @@
//! Live decode stats for the on-stream HUD (mirrors the Apple client's stats overlay): FPS,
//! receive throughput, and capture→client-receipt latency (p50/p95). The decode thread is the sole
//! writer (`note` per access unit); the JNI accessor `nativeVideoStats` drains a snapshot ~1 Hz and
//! resets the window. Pure `std` so it compiles on the host build too (the decode thread is
//! android-only, but `VideoThread` holds the shared handle unconditionally).
//! Live decode stats for the on-stream HUD, following the unified stats spec
//! (`design/stats-unification.md`): FPS, receive throughput, and the Android v1 stage split —
//! headline `end-to-end` = capture→decoded (p50/p95) tiled by `host+network` = capture→received
//! and `decode` = received→decoded (stage p50s). When the host emits per-AU 0xCF host timings, the
//! `host+network` term further splits into `host` + `network` (Phase 2, `note_host_split`); an old
//! host emits none and the combined term stands. The decode thread is the sole writer
//! (`note_received` per access unit at receipt, `note_decoded` per decoder output buffer); the JNI
//! accessor `nativeVideoStats` drains a snapshot ~1 Hz and resets the window. Sampling is gated on
//! the HUD actually being visible (`set_enabled`, driven by `nativeSetVideoStatsEnabled`) so the
//! hidden steady state costs one relaxed atomic load per frame.
//! Pure `std` so it compiles on the host build too (the decode thread is android-only, but
//! `SessionHandle` holds the shared handle unconditionally).
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Mutex;
use std::time::Instant;
/// Rolling per-window accumulator. Rates are computed over the actual elapsed wall-time at drain
/// (robust to poll jitter), so a poll that lands at 0.9 s or 1.1 s still reports the right FPS.
pub struct VideoStats {
/// HUD gate: the samplers run on the per-frame decode path, so while the overlay is hidden
/// they (and the caller's latency computation — see `enabled`) early-out on this flag alone.
/// Off until Kotlin shows the HUD.
enabled: AtomicBool,
inner: Mutex<Inner>,
}
@@ -17,77 +29,198 @@ struct Inner {
window_start: Instant,
frames: u64,
bytes: u64,
/// capture→client-receipt latency samples for this window, in microseconds.
lat_us: Vec<u64>,
/// `end-to-end` = capture→decoded latency samples for this window, in microseconds
/// (skew-corrected clock base).
e2e_us: Vec<u64>,
/// `host+network` stage = capture→received samples, in microseconds (skew-corrected).
hostnet_us: Vec<u64>,
/// Phase-2 split of `host+network` (design/stats-unification.md Phase 2), fed only when the
/// host emits per-AU 0xCF timings: `host` = the host's own capture→sent duration, µs.
host_us: Vec<u64>,
/// The matching `network` term, µs: capture→received minus the host's capture→sent
/// (wire + reassembly). Always pushed in lockstep with `host_us`.
net_us: Vec<u64>,
/// `decode` stage = received→decoded samples, in microseconds (client-local, single clock).
decode_us: Vec<u64>,
/// Whether the host answered the clock-skew handshake (latency is cross-machine valid).
skew_corrected: bool,
}
/// A drained, computed view of one window. `lat_valid` is false when no in-range latency sample
/// landed (then p50/p95 are 0 and the HUD hides the latency line, exactly like the Apple client).
/// A drained, computed view of one window. `lat_valid` is false when no in-range end-to-end sample
/// landed (then the latency figures are 0 and the HUD hides the latency lines, exactly like the
/// Apple client).
pub struct Snapshot {
pub fps: f64,
pub mbps: f64,
pub lat_p50_ms: f64,
pub lat_p95_ms: f64,
/// Headline `end-to-end` (capture→decoded) percentiles, ms.
pub e2e_p50_ms: f64,
pub e2e_p95_ms: f64,
/// Stage p50s (ms): `host+network` (capture→received) and `decode` (received→decoded).
pub hostnet_p50_ms: f64,
pub decode_p50_ms: f64,
/// Phase-2 `host` / `network` split p50s (ms) — 0.0 when no 0xCF timing matched this window
/// (old host / no samples yet), in which case the HUD keeps the combined `host+network` term.
pub host_p50_ms: f64,
pub net_p50_ms: f64,
pub lat_valid: bool,
pub skew_corrected: bool,
}
/// Percentile over a sorted-in-place µs sample vec, in ms. 0.0 when empty.
fn pctl_ms(sorted_us: &[u64], p: f64) -> f64 {
if sorted_us.is_empty() {
return 0.0;
}
let n = sorted_us.len();
sorted_us[((n as f64 * p) as usize).min(n - 1)] as f64 / 1000.0
}
impl VideoStats {
// `new`/`note` are driven only by the android-only decode thread; `drain` (the JNI accessor) is
// ungated, so on the host build these two are unreferenced — that's expected, not dead code.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn new() -> VideoStats {
VideoStats {
enabled: AtomicBool::new(false),
inner: Mutex::new(Inner {
window_start: Instant::now(),
frames: 0,
bytes: 0,
lat_us: Vec::with_capacity(256),
e2e_us: Vec::with_capacity(256),
hostnet_us: Vec::with_capacity(256),
host_us: Vec::with_capacity(256),
net_us: Vec::with_capacity(256),
decode_us: Vec::with_capacity(256),
skew_corrected: false,
}),
}
}
/// Record one decoded access unit: its wire size and (if in range) its capture→client latency.
/// Whether the HUD wants samples. The decode thread checks this BEFORE building a latency
/// sample, so the per-frame wall-clock reads are skipped too while hidden.
// Read only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note(&self, bytes: usize, lat_us: Option<u64>, skew_corrected: bool) {
let mut g = self.inner.lock().unwrap();
pub fn enabled(&self) -> bool {
self.enabled.load(Ordering::Relaxed)
}
/// Toggle sampling. Enabling resets the window, so the first HUD poll after a show never mixes
/// in counters (or a window start) from before the overlay was visible.
pub fn set_enabled(&self, on: bool) {
let was = self.enabled.swap(on, Ordering::Relaxed);
if on && !was {
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.window_start = Instant::now();
g.frames = 0;
g.bytes = 0;
g.e2e_us.clear();
g.hostnet_us.clear();
g.host_us.clear();
g.net_us.clear();
g.decode_us.clear();
}
}
/// Record one received access unit: its wire size and (if in range) its capture→received
/// `host+network` stage sample. Receipt is the fps/goodput counting point per the spec.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_received(&self, bytes: usize, hostnet_us: Option<u64>, skew_corrected: bool) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock (the caller already skipped the clock read)
}
// Poison-proof: this runs per-frame on the decode thread, which has no catch_unwind —
// a panic elsewhere must not turn every later lock into a second panic (the counters
// stay consistent regardless).
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.frames += 1;
g.bytes += bytes as u64;
g.skew_corrected = skew_corrected;
if let Some(l) = lat_us {
g.lat_us.push(l);
if let Some(l) = hostnet_us {
g.hostnet_us.push(l);
}
}
/// Record one matched host/network split sample (Phase 2): the host's reported capture→sent
/// duration and our capture→received minus it, both µs — one pair per AU whose 0xCF host
/// timing arrived and matched by pts. An old host emits none, leaving the vecs empty and the
/// snapshot p50s at 0 (HUD keeps the combined `host+network` term).
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_host_split(&self, host_us: u64, net_us: u64) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.host_us.push(host_us);
g.net_us.push(net_us);
}
/// Record one decoded output frame: its capture→decoded `end-to-end` sample and its
/// received→decoded `decode` stage sample (either may be absent — e.g. the receipt stamp for
/// this pts predates the HUD being shown).
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_decoded(&self, e2e_us: Option<u64>, decode_us: Option<u64>) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock (the caller already skipped the clock read)
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(l) = e2e_us {
g.e2e_us.push(l);
}
if let Some(l) = decode_us {
g.decode_us.push(l);
}
}
/// Compute the window's rates + latency percentiles, then reset for the next window.
pub fn drain(&self) -> Snapshot {
let mut g = self.inner.lock().unwrap();
// Poison-proof for the same reason as `note_received` — a poisoned window still drains
// fine.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let elapsed = g.window_start.elapsed().as_secs_f64().max(1e-3);
let fps = g.frames as f64 / elapsed;
let mbps = g.bytes as f64 * 8.0 / 1_000_000.0 / elapsed;
let (p50, p95, valid) = if g.lat_us.is_empty() {
(0.0, 0.0, false)
} else {
g.lat_us.sort_unstable();
let n = g.lat_us.len();
let at = |p: f64| g.lat_us[((n as f64 * p) as usize).min(n - 1)] as f64 / 1000.0;
(at(0.50), at(0.95), true)
g.e2e_us.sort_unstable();
g.hostnet_us.sort_unstable();
g.host_us.sort_unstable();
g.net_us.sort_unstable();
g.decode_us.sort_unstable();
let snap = Snapshot {
fps,
mbps,
e2e_p50_ms: pctl_ms(&g.e2e_us, 0.50),
e2e_p95_ms: pctl_ms(&g.e2e_us, 0.95),
hostnet_p50_ms: pctl_ms(&g.hostnet_us, 0.50),
decode_p50_ms: pctl_ms(&g.decode_us, 0.50),
host_p50_ms: pctl_ms(&g.host_us, 0.50),
net_p50_ms: pctl_ms(&g.net_us, 0.50),
lat_valid: !g.e2e_us.is_empty(),
skew_corrected: g.skew_corrected,
};
let skew = g.skew_corrected;
g.window_start = Instant::now();
g.frames = 0;
g.bytes = 0;
g.lat_us.clear();
Snapshot {
fps,
mbps,
lat_p50_ms: p50,
lat_p95_ms: p95,
lat_valid: valid,
skew_corrected: skew,
}
g.e2e_us.clear();
g.hostnet_us.clear();
g.host_us.clear();
g.net_us.clear();
g.decode_us.clear();
snap
}
}
+5
View File
@@ -16,5 +16,10 @@
compliance question. -->
<key>ITSAppUsesNonExemptEncryption</key>
<false/>
<!-- Allow CADisplayLink above 60 Hz on ProMotion iPhones: without this key the system
silently caps the link at 60 even when SessionPresenter asks for the stream's rate
via preferredFrameRateRange, so a 120 fps stream would present at half rate. -->
<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>
</dict>
</plist>
+103 -341
View File
@@ -1,364 +1,126 @@
# punktfunk Apple client (SwiftUI)
# punktfunk Apple client (macOS · iOS · iPadOS · tvOS)
The native macOS/iOS client for **`punktfunk/1`** (the post-GameStream protocol). All
networking/protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
input datagrams, Opus audio, cert pinning — lives in the shared Rust core (statically
linked as `PunktfunkCore.xcframework`); this package is the Swift shell: decode
(VideoToolbox), present (SwiftUI), input capture.
The native **Apple** app for streaming a punktfunk host to your Mac, iPhone, iPad, or Apple TV. A
SwiftUI app that finds hosts on your network, pairs with a PIN, and streams at your display's own
resolution and refresh rate — with VideoToolbox hardware decode and full controller support.
## Status — working client (macOS, with iOS / tvOS in the shared build)
All the networking and protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (statically linked as
`PunktfunkCore.xcframework`). This package is the Swift shell: decode, present, input, and UI.
A full streaming client: VideoToolbox HEVC decode, controllers incl. DualSense feedback, host
discovery, PIN pairing, and a network speed test. The lower-latency **stage-2 presenter**
(`VTDecompressionSession``CAMetalLayer`) is built and opt-in (Settings → Presenter); see below.
## Features
First light was achieved 2026-06-10 — validated live, Mac ↔ a Linux host over the LAN: gamescope
virtual output → NVENC HEVC →
`punktfunk/1` (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox →
`AVSampleBufferDisplayLayer` on glass at 1280×720@60, with mouse/keyboard flowing back as
QUIC datagrams into the host's gamescope EIS injector (thousands of events injected during
the session). Headless variant of the same proof: `RemoteFirstLightTests` decoded 60/60
received AUs spanning 983 ms of host capture clock.
- **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
(`VTDecompressionSession``CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
default and an `AVSampleBufferDisplayLayer` fallback.
- **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
reconfiguration, and hardware-probed 4:4:4 support.
- **Your display's native mode** — the host builds a virtual output at exactly your WxH@Hz;
mid-stream resize renegotiates without reconnecting.
- **Audio both ways** — Opus playback (CoreAudio, no bundled libopus) with a jitter ring, plus mic
uplink; speaker/mic selectable in Settings.
- **Full controller support** — one selected controller forwarded as pad 0, including **DualSense**
feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The
virtual pad type auto-resolves from your physical controller.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌘⎋ release, plus
iPad pointer lock and touch input.
- **Find hosts automatically** — mDNS discovery (`NWBrowser` over `_punktfunk._udp`); first connect
does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on a pinned,
Keychain-stored identity.
- **Tune the stream** — a fps / Mb·s / **latency** HUD (skew-corrected across machines), a bitrate
control, a per-host **network speed test** with a recommended bitrate, and a host-compositor picker.
The connector underneath (`punktfunk_core::client::NativeClient` over the C ABI) carries the
full session: video AUs, **Opus audio** (`nextAudio()`), **rumble** (`nextRumble()`),
**DualSense feedback** (`nextHidOutput()` — lightbar, player LEDs, adaptive-trigger
effects), input incl. gamepads + DualSense touchpad/motion (`sendTouchpad`/`sendMotion`),
and **cert pinning + TOFU** (`pinSHA256:`/`hostFingerprint`) — see
`punktfunk1.rs::tests::c_abi_connection_roundtrip` (three sequential sessions: TOFU, pinned
reconnect, wrong-pin rejection). The host (`punktfunk-host punktfunk1-host`) is a persistent listener:
reconnect at will during development.
Runs from one shared codebase across **macOS, iOS, iPadOS, and tvOS**.
What's here, all compiled and tested on macOS (Xcode 26.5 / Swift 6.3):
## Get it
- **`PunktfunkKit`** (library)
- `PunktfunkConnection.swift` — wrapper over the C ABI. AUs/audio are copied into `Data`
(the C pointer is only valid until the next call of the same kind). `close()` is safe
from any thread: per-plane locks enforce the C contract ("never close with a
`next_au`/`next_audio` in flight") instead of leaving it to callers. Pinning + TOFU
via `pinSHA256:`/`hostFingerprint`.
- `AnnexB.swift` — in-band VPS/SPS/PPS → `CMVideoFormatDescription`; Annex-B → AVCC
`CMSampleBuffer` with `DisplayImmediately` set.
- `StreamView.swift` — SwiftUI `NSViewRepresentable` over `AVSampleBufferDisplayLayer`
(stage-1 presenter: the layer hardware-decodes compressed HEVC itself). One pump
thread per view, token-cancelled so reconnects can't double-pump.
- `InputCapture.swift``GCMouse` raw deltas + `GCKeyboard` HID→VK mapping (the host's
`vk_to_evdev` consumes Windows VKs), with fractional-delta accumulation so sub-pixel
motion isn't truncated away. Buttons use GameStream ids (1=left … 5=X2). Scroll is
WHEEL_DELTA(120)-scaled: macOS via the stream view's `scrollWheel` override, iPad via
GCMouse's scroll dpad when pointer-locked and a scroll-only `UIPanGestureRecognizer`
otherwise (trackpad gestures never reach GC's scroll dpad).
- `GamepadManager.swift` — app-lifetime controller discovery + selection (`.shared`):
watches `GCController` connect/disconnect, fingerprints each pad for the Settings UI
(name, capabilities, battery), and selects the ONE controller forwarded to the host
(user pin via "Use controller", else most recently connected extended gamepad).
- `GamepadCapture.swift` — the active controller → wire: snapshot-diff over
`GCExtendedGamepad` into incremental `gamepadButton`/`gamepadAxis` events (pad 0),
plus DualSense touchpad contacts and ~250 Hz motion samples on the rich-input plane
(the GC→DualSense unit conversions live in `GamepadWire`, one place). Held state is
released on the wire on controller switch / app deactivation / stop.
- `GamepadFeedback.swift` + `DualSenseTriggerEffect.swift` — host feedback → the real
controller: one drain thread for `nextRumble()` (→ `CHHapticEngine` per handle
locality) and `nextHidOutput()` (lightbar → `GCDeviceLight`, player LEDs →
`playerIndex`, adaptive-trigger effect blocks → a total, table-driven parser →
`GCDualSenseAdaptiveTrigger`, exact for the 10-zone positional modes).
- `HostDiscovery.swift` — LAN auto-discovery: an `NWBrowser` over `_punktfunk._udp`
(the host's `crate::discovery` mDNS advert), resolving each service to an IP:port via a
throwaway `NWConnection` and parsing the TXT (`fp` advisory cert fingerprint, `pair`,
stable `id`). iOS/tvOS need `NSBonjourServices` (`Config/Info.plist`) or the system
blocks the browse.
- **`PunktfunkClient`** (the app): hosts grid (saved in UserDefaults) with an **On this
network** section listing mDNS-discovered hosts (tap to save + connect, or pair if the
host requires it), "+" toolbar sheet to add hosts manually, stream mode in Settings (⌘,),
two trust flows — the
trust-on-first-use fingerprint prompt over the live-but-blurred stream, and SPAKE2 PIN
pairing (`PairSheet`, from a host card's context menu or the trust prompt;
`ClientIdentityStore` keeps the client identity in the Keychain and presents it on
every connect) — then pinned reconnects, fps/Mb-s HUD + a **capture→client-receipt latency**
line (`LatencyMeter`, p50/p95): the AU `pts_ns` (host capture clock) to the instant the client
received it, **skew-corrected** across machines via `PunktfunkConnection.clockOffsetNs` (the
connect-time wall-clock handshake, `punktfunk_connection_clock_offset_ns`). It excludes the
layer's decode+present (stage-1 `AVSampleBufferDisplayLayer` has no per-frame present callback);
the opt-in **stage-2 presenter** (Settings → Presenter) adds a **capture→present**
(glass-to-glass) line via explicit decode + a Metal/display-link present. Settings also picks the HOST
compositor (KWin/wlroots/Mutter/gamescope, default automatic — the host honors it
only if that backend is available there) and has a **Controllers** section: every
detected controller (capability glyphs, battery, "In use" badge), which one to forward
("Use controller", default automatic), and the virtual pad type the host creates
("Controller type": Automatic / Xbox 360 / DualSense — Automatic matches the physical
pad; resolved at connect time, the host pad is fixed per session). Gamepad capture +
feedback run with streaming (`SessionModel` owns them, same trust gate as audio).
Settings also sets the **Bitrate** (Automatic toggle = host default; manual is a
log-scale slider, 2 Mbps 3 Gbps, snapped to two significant figures — above 1 Gbps
an inline warning says to run a speed test first; tvOS uses a preset picker instead,
Slider doesn't exist there; negotiated via the Hello on every connect), and a host
card's context menu offers **"Test Network Speed…"** (`SpeedTestSheet`): connects, has
the host burst probe filler over the real data plane (up to the host's 3 Gbps probe
ceiling for 2 s, roadmap §9),
shows measured goodput · loss · a recommended bitrate (≈70% of measured), and applies
it in one tap. The streaming **statistics overlay** can be turned off and moved to any
corner (Settings → Display → Statistics, `DefaultsKey.hudEnabled`/`hudPlacement`), and
toggled live with **⌘⇧S** — a Scene-level **"Stream" menu** (`StreamCommands`) that also
carries **Disconnect ⌘D**, so disconnect survives the HUD being hidden (on iOS a small
exit chip appears instead; on tvOS the Siri-Remote Menu button still disconnects). The
macOS Settings window is a **tabbed preferences pane** (General / Display / Audio /
Controllers / Advanced) — the sections are shared with the iOS single-Form layout and the
tvOS pushed-picker layout, defined once each.
- **Tests** (`swift test`): byte-level Annex-B units; a real-codec round trip
(VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → `AnnexB`
VTDecompressionSession → pixels); table-driven DualSense trigger-effect parsing
(`DualSenseTriggerEffectTests`) and the gamepad wire conversions
(`GamepadWireTests`); loopback integration against real local hosts
(`test-loopback.sh` — stream round trip incl. gamepad/touchpad/motion sends, a
host-scripted feedback burst asserted on the rumble + HID-output planes
(`PUNKTFUNK_TEST_FEEDBACK=1`), the bitrate-negotiation echo and a real 20 Mbps
bandwidth probe, plus the PIN pairing ceremony and the `--require-pairing` gate
against a second, armed host); the remote first-light test above.
Install from the App Store / TestFlight, or build from source below. Per-device install steps and the
pairing walkthrough:
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
## Build / run / test (on a Mac)
Requires Xcode 26.5 / Swift 6.3. First build the Rust core into an xcframework, then build the app:
```sh
rustup target add aarch64-apple-darwin x86_64-apple-darwin
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
# + BUILD_IOS=1 for the iOS slices (rustup target add aarch64-apple-ios{,-sim} x86_64-apple-ios)
# + BUILD_TVOS=1 for tvOS — TIER-3 Rust targets, built from source:
# rustup toolchain install nightly && rustup component add rust-src --toolchain nightly
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
# BUILD_IOS=1 also builds the iOS slices (add the ios rustup targets)
# BUILD_TVOS=1 also builds tvOS (tier-3 targets, built from source — see below)
cd clients/apple
swift build && swift test # loopback/remote tests self-skip without a host
swift run PunktfunkClient # the unbundled dev shell (CLI)
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
swift run PunktfunkClient # or the unbundled dev shell (CLI)
swift build && swift test # unit + loopback/remote tests (self-skip w/o a host)
```
bash test-loopback.sh # full loopback proof: builds punktfunk-host
# (synthetic source — runs on macOS), streams
# byte-verified frames into the Swift client
tvOS slices are tier-3 Rust targets, built from source:
`rustup toolchain install nightly && rustup component add rust-src --toolchain nightly`.
# against the real host (Linux box, see CLAUDE.md "Running on this box") — punktfunk1-host is a
# persistent listener, reconnect at will:
# PUNKTFUNK_COMPOSITOR=gamescope PUNKTFUNK_GAMESCOPE_APP=vkcube PUNKTFUNK_ZEROCOPY=1 \
# cargo run -rp punktfunk-host -- punktfunk1-host --source virtual --seconds 60
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
# (+ PUNKTFUNK_REMOTE_PORT / PUNKTFUNK_REMOTE_COMPOSITOR=gamescope|kwin|… /
# PUNKTFUNK_REMOTE_PIN=<arming-pin> for the remote pairing test)
### Test against a host
```sh
# full loopback proof — builds punktfunk-host (synthetic source, runs on macOS) and streams
# byte-verified frames into the Swift client, incl. the PIN pairing ceremony:
bash test-loopback.sh
# against a real Linux host on the LAN (see the repo README "Running on this box"):
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
```
## Xcode project (`Punktfunk.xcodeproj`)
## Project layout
The app target **Punktfunk** wraps the same sources as the `swift run` shell
(`Sources/PunktfunkClient`, a synchronized folder — no duplication) plus `App/` (asset
catalog) and links `PunktfunkKit` from the local package. Generated Info.plist, ad-hoc
signing, bundle id `io.unom.punktfunk`. Notes:
- **`PunktfunkKit`** (library) — the reusable pieces:
- `PunktfunkConnection` — the wrapper over the C ABI (thread-safe `close()`, per-plane locks,
pinning + TOFU).
- `AnnexB` / `StreamView` / `VideoDecoder` / `MetalVideoPresenter` — format handling, the stage-1
(`AVSampleBufferDisplayLayer`) and stage-2 (`VTDecompressionSession``CAMetalLayer`) presenters.
- `InputCapture``GCMouse`/`GCKeyboard` → host VK/mouse, with fractional-delta accumulation.
- `GamepadManager` / `GamepadCapture` / `GamepadFeedback` / `DualSenseTriggerEffect` — controller
discovery + selection, capture (buttons/axes/touchpad/motion), and host-feedback rendering.
- `HostDiscovery``NWBrowser` over `_punktfunk._udp`.
- **`PunktfunkClient`** (the app) — hosts grid with an *On this network* section, add-host sheet,
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
On iOS/iPadOS **and macOS** a connected controller swaps the whole home for the **gamepad UI**
(`Home/Gamepad*`, `Settings/GamepadSettingsView`): a console-style host carousel (A connect · Y
library · X settings), a controller-navigable settings screen, an add-host flow with an
on-screen controller keyboard (no touch required anywhere), and the coverflow library browser —
all driven by the shared `GamepadMenuInput` poller + `GamepadCarousel`/`GamepadMenuList` focus
machinery, with dual-channel haptics (device Taptic + controller `MenuHaptics`), over an
animated "aurora" backdrop (`GamepadScreenBackground` — TimelineView-driven drifting color
blobs; deliberately pure SwiftUI, since a .metal library only reliably bundles in one of the
two build systems these sources compile under). macOS presents the settings/add-host screens as
sheets (no `fullScreenCover` there); `PUNKTFUNK_FORCE_GAMEPAD_UI=1` forces the mode without a
physical pad (dev/screenshots).
- **Tests** (`swift test`) — Annex-B units, a real-codec VideoToolbox round trip, DualSense
trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the
remote first-light test.
- **Entitlements (sandbox)**: the macOS target uses
`Config/Punktfunk-macOS.entitlements`; iOS/tvOS use the shared
`Config/Punktfunk.entitlements`. The macOS app is **App-Sandboxed** (mandatory for the Mac
App Store/TestFlight, and used for the Developer ID DMG too so the local build matches what
ships): `com.apple.security.app-sandbox`, `network.client` + **`network.server`** (the
sandbox gates `bind()`; quinn + the raw-UDP plane both bind, so receive breaks without it),
`device.audio-input` (mic), `device.bluetooth` + `device.usb` (GameController over BT/USB),
and the existing `keychain-access-groups`. `app-sandbox` is macOS-only — keep it OUT of the
shared iOS/tvOS file (it fails upload validation there). Verify a build is sandboxed with
`codesign -d --entitlements :- <built .app>`. Heads-up: `device.usb` draws some App Review
scrutiny — justify it in the review notes ("reads input from USB game controllers").
- **App icon**: `App/Assets.xcassets` ships an empty `AppIcon` slot. For an Icon Composer
`.icon`: add the file to the project (target Punktfunk), set it as the App Icon in the
target's General tab, and delete the placeholder `AppIcon.appiconset`. Heads-up: CLI
`actool` (Xcode 26.5) crashed compiling `punktfunk_Logo.icon` — if Xcode does the same,
suspect the icon bundle (it has a duplicate-named layer, "…Layer-3 2.svg"), not the
project.
- **Tests from Xcode**: the package tests run with `swift test`; to get them on ⌘U, add
`PunktfunkKitTests` once via Edit Scheme → Test → + (Xcode persists it into the shared
scheme — a hand-written package-test reference doesn't resolve headlessly).
- `xcodebuild -project Punktfunk.xcodeproj -scheme Punktfunk build` works headlessly;
same for `-scheme Punktfunk-iOS -destination 'generic/platform=iOS Simulator'` (run it
in a simulator via `xcrun simctl install/launch``SIMCTL_CHILD_PUNKTFUNK_AUTOCONNECT=…`
passes the dev autoconnect env through).
## Notes for contributors
## App Store screenshots
- **Xcode project** (`Punktfunk.xcodeproj`) wraps the same sources as the `swift run` shell (a
synchronized folder — no duplication). The macOS target is **App-Sandboxed** (needs
`network.server` — the raw-UDP plane and quinn both `bind()`); iOS/tvOS use the shared
entitlements file (keep `app-sandbox` **out** of it). Verify with
`codesign -d --entitlements :- <built .app>`.
- **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band, and recovery
keyframes re-send them — refresh the format description on every IDR; there is no out-of-band
extradata, ever.
- **ABI threading**: one video pump thread per connection, one optional audio drain thread, and one
optional feedback drain thread (rumble + HID-output). `send()` is enqueue-only and safe alongside
all of them. The wrapper's per-plane locks make `close()` safe from anywhere.
- **DualSense motion scale** (`GamepadWire`) is derived from hid-playstation's math, not yet
live-verified — if gyro/accel feel wrong in a game, correct sign/scale there and `evtest` the
host's virtual pad.
- **App Store screenshots** are automated — `tools/screenshots.sh all` renders the real UI at the
required pixel sizes via a DEBUG-only shot mode; the `apple` CI workflow captures the iOS sizes on
every main push. See the script header for details.
- Deeper design notes live in [`design/apple-stage2-presenter.md`](../../design/apple-stage2-presenter.md).
Automated, faithful screenshots of the real UI for App Store Connect — one set per platform at
exactly the accepted pixel sizes. Driver: **`tools/screenshots.sh`**.
## Related
```sh
tools/screenshots.sh all # macOS + (if full Xcode) iOS, iPadOS, tvOS → ./screenshots
tools/screenshots.sh macos # just macOS
OUT=~/Desktop/shots tools/screenshots.sh ios ipad tvos
PUNKTFUNK_SHOT_HERO=~/frame.png tools/screenshots.sh ios # real captured frame behind the hero
```
How it works: the app has a DEBUG-only **shot mode** (`Sources/PunktfunkClient/Screenshots/`).
Launched with `PUNKTFUNK_SHOT_SCENE=<name>` it renders **one** mock-populated screen full-bleed
(`ScreenshotHostView`) instead of `ContentView`, then the OS screenshots the *real, fully-rendered*
window — `screencapture` on macOS, `xcrun simctl io booted screenshot` on the Simulators. The five
scenes (`ShotScenes.all`): `01-stream` (the stream hero — a synthetic frame + the glass HUD, since
`StreamView` needs a live connection), `02-hosts`, `03-pair`, `04-trust`, `05-settings`. Mock data
is in `ShotMock`; nothing touches a host.
Output pixels are App Store Connect's required/largest sizes (Apple auto-derives the smaller ones):
`mac` 2880×1800 · `iphone-6.9` 1320×2868 (hero 2868×1320) · `ipad-13` 2064×2752 (hero 2752×2064) ·
`appletv` 1920×1080.
Why not `ImageRenderer` (the obvious offscreen route)? It can't rasterize this app's chrome —
`NavigationStack`, `Form`/`TabView`, and Liquid-Glass/`NSVisualEffect` materials all render black or
SwiftUI's "can't render" placeholder. Capturing the live window/Simulator avoids that entirely.
Requirements / gotchas:
- **macOS**: only the Swift toolchain is needed, **plus a one-time Screen Recording grant** for
your terminal (System Settings → Privacy & Security → Screen Recording) — without it
`screencapture -l` fails with "could not create image from window". (A no-permission fallback,
`PUNKTFUNK_SHOT_SELFCAPTURE=<dir>`, uses `cacheDisplay` — but it omits material blur and can't
read `ScrollView` content, so it's for quick checks, not submission.)
- **iOS/iPadOS/tvOS**: needs **full Xcode** (xcodebuild + Simulators), not just Command Line Tools,
and the matching device Simulators installed (iPhone 16 Pro Max, iPad Pro 13", Apple TV). Run it
on a full-Xcode Mac (e.g. the `macos-arm64` CI mini).
- The hero defaults to a synthetic synthwave frame — set `PUNKTFUNK_SHOT_HERO` to a real captured
frame for a production-quality lead screenshot.
**CI**: the `apple` workflow's **`screenshots`** job runs on the `macos-arm64` runner on every main
push + manual dispatch (skipped on PRs), and attaches the result as a single zip artifact,
**`punktfunk-appstore-screenshots`** (download it from the run's Artifacts; `upload-artifact@v3`
Gitea's backend rejects v4). It captures the two **required iOS sizes — iPhone 6.9" + iPad 13"**
on the Simulator (auto-creating the device if the runner lacks it), and is isolated from the
build/test job so a capture hiccup never reds the build.
**macOS and tvOS are NOT in CI**, by design: the self-hosted runner is **headless** (no
window-server session), so the macOS window capture can't run there, and tvOS needs the Tier-3
build-std slice. Generate those on a GUI Mac: `tools/screenshots.sh macos tvos`. (If the runner is
ever switched to a logged-in GUI session, re-adding macOS to the job's capture step is one line.)
## Notes for whoever picks this up next
1. **cbindgen import quirk** (the predicted "small compile fixes", now fixed): the
C17-compatible header spells `PunktfunkStatus`/`PunktfunkInputKind` as integer typedefs while
the enum *constants* import into Swift as a distinct same-named type — bridge with
`.rawValue` (see the top of `PunktfunkConnection.swift`). Don't fight the generated header.
2. **ABI contract**: one video pump thread per connection, plus optionally one *separate*
audio drain thread for `nextAudio()` and one feedback drain thread for
`nextRumble()`/`nextHidOutput()` (the core keeps per-plane borrow slots, so the planes
never alias; rumble + HID-output are two planes drained sequentially by the one
feedback thread); `send()` is enqueue-only and safe alongside all of them. The
wrapper's per-plane locks make `close()` safe from anywhere (it waits out in-flight
polls, ≤ their timeouts).
3. **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band
and recovery keyframes re-send them — "refresh the format description on every IDR"
(what `StreamView` does) is sufficient; there is no out-of-band extradata, ever.
4. **Stage 2 — built, opt-in (`punktfunk.presenter == "stage2"`, default stage 1).** Explicit
`VTDecompressionSession` decode (`VideoDecoder`) → a `CAMetalLayer` + display-link present
(`MetalVideoPresenter`/`Stage2Pipeline`), hosted as a sublayer by the same `StreamView`s with
input capture + HUD unchanged. It adds a **capture→present** (glass-to-glass, modulo the host
render→capture term) HUD line, skew-corrected via `PunktfunkConnection.clockOffsetNs`. The
decode half is unit-tested (`testVideoDecoderAsyncCallbackDeliversPixels`); the Metal present
is display-bound — **validate live** (flip the Settings "Presenter" picker, watch the HUD
number and that the image looks right) before making it the default. 10-bit/HDR + a smoothing
pacer are later. Plan: `docs-site/content/docs/apple-stage2-presenter.md`.
5. **Audio — wired, both directions.** Playback: `SessionAudio` drains `nextAudio()`
on its own thread, decodes through CoreAudio's built-in Opus codec (`OpusCodec.swift`
— kAudioFormatOpus, no bundled libopus; round-trip unit-tested) into a priming
jitter ring feeding an `AVAudioSourceNode`. Mic: a second engine taps the input
device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and `sendMic()`s them
(the host's virtual PipeWire source accepts any frame size ≤ 120 ms). Speaker/mic
are chosen in Settings (`AudioDevices.swift` — persisted by UID; "System default"
leaves the engines unpinned so they follow macOS device changes), mic on/off toggle
included; the app asks for mic permission on first use
(NSMicrophoneUsageDescription is in the Xcode target). A/V sync and packet-loss
concealment beyond silence-fill are still open (AudioPacket.seq/ptsNs carry what's
needed). Decode with libopus or `AVAudioConverter`/`kAudioFormatOpus` into an
`AVAudioEngine` source node; conceal gaps (drop/dup) rather than blocking — the Rust
side buffers 320 ms and drops the newest packet when the puller lags. Wall-clock
`ptsNs` shares the host clock with video AUs for A/V sync. Wiring this into
`PunktfunkClient` is the next app-side task.
6. **Gamepads — wired end to end.** Exactly ONE controller (the `GamepadManager`
selection) forwards as pad 0; the host accumulates the incremental events into a
virtual pad whose TYPE the client negotiates in the Hello (`gamepad:` connect
parameter, echoed resolved in `resolvedGamepad` — Automatic resolves from the physical
pad at connect time; host precedence: explicit client choice > host `PUNKTFUNK_GAMEPAD`
env > Xbox 360). A DualSense session carries the full feel: adaptive-trigger blocks
(`DualSenseTriggerEffect.parse` — mode bytes per the community convention
(Nielk1/ds5w/inputtino), total, unknown → `.off`), lightbar, player LEDs, touchpad,
motion. **Motion scale constants** (`GamepadWire.gyroLSBPerRadS` = 20 LSB per deg/s,
`accelLSBPerG` = 10000) are derived from hid-playstation's math over the host's fixed
calibration blob, not yet live-verified — if gyro/accel feel wrong in a real game,
correct sign/scale in `GamepadCapture.forwardMotion`/`GamepadWire` and `evtest` the
host's virtual pad. Twin identical controllers share a fingerprint base, so a manual
pin can swap between them across reconnects (documented in the Settings footer).
7. **Trust — the full ceremony exists now (SPAKE2).** `generateIdentity()` once (persist
both PEMs in the Keychain), then `pair(host:identity:pin:name:)` with the 4-digit PIN
the host prints when it ARMS pairing (`--allow-pairing`/`--require-pairing`; one PIN
per arming window, surfaced in the host's web console — port 3000 → Pairing — and
printed at startup; the user reads it before pairing). Returns the
host's VERIFIED fingerprint; persist it and pass `pinSHA256:` + `identity:` to every
connect. Pairing is a real PAKE: a wrong PIN gets ONE online guess (no offline
dictionary attack), throwing `.wrongPIN`; a wrong-size pin throws `.invalidPin`. `PunktfunkClient` implements both flows:
the TOFU fingerprint sheet keeps working against hosts not running
`--require-pairing`, and the PIN ceremony is wired in — `ClientIdentityStore`
(Keychain) on every connect, `PairSheet` from a host card's context menu or the trust
prompt's "Pair with PIN instead…" (the host's accept loop is sequential, so that path
drops the live session before pairing). With `--require-pairing` the host now
authorizes clients too (the "other direction" is no longer open, opt-in per host);
the whole gate is regression-tested in `testPairingCeremonyAndRequirePairingGate`.
7b. **Resize without reconnect**: `requestMode(width:height:refreshHz:)` mid-stream —
the host rebuilds at the new mode in ~90 ms; the first new-mode AU is an IDR with
fresh parameter sets (the refresh-on-IDR decode flow handles it untouched) and
`currentMode()` reflects the switch. Wire it to window-resize events.
8. **Input capture** (stage 1): capture is a deliberate, reversible STATE owned by
`StreamLayerView`, Moonlight-style. Engaged when the stream starts / trust is
confirmed and when the user clicks into the video (that click is suppressed toward
the host); released by ⌘⎋ (toggles) or focus loss; NEVER engaged by mere app
activation — activating clicks may be title-bar drags or resizes, which used to get
their cursor warped away mid-drag. While captured: the local cursor is hidden +
frozen mid-view (the host renders its own), all input is forwarded, and the view
consumes key events as first responder so unhandled keyDowns don't beep — ⌘-combos
still work locally (⌘D disconnect, ⌘Q) *and* reach the host via GC. While released:
nothing is forwarded (`InputCapture.forwarding` gates the GC handlers; held
keys/buttons are flushed host-side on release so nothing sticks down), the cursor is
free, and the HUD shows "Click the stream to capture input". GC handlers only fire
while the app has focus, and focus loss also auto-releases everything held. One live capture per process (the GC
mouse/keyboard singletons have a single handler slot — ownership is tracked so a stale
capture's stop() can't clobber a newer one).
9. **iOS/iPadOS — ported and first-lit** (iPad simulator ↔ the real host, 60 fps).
`BUILD_IOS=1 bash scripts/build-xcframework.sh` builds device + universal-simulator
slices; the Xcode project has a second target, **Punktfunk-iOS**, sharing the same
synchronized sources. The iOS `StreamView` (StreamViewIOS.swift — same name/signature
as the macOS one, so the SwiftUI shell is identical) hosts the shared `StreamPump` in
a view controller for `prefersPointerLocked`: with a hardware mouse/trackpad that is
the iPadOS cursor capture (system honors it fullscreen-and-frontmost; in Stage
Manager it degrades to absolute-mouse forwarding). Input is routed by kind: DIRECT
fingers / Pencil are touches (each gets a wire touch id, coordinates mapped through the
aspect-fit letterbox into host-mode pixels — surface == host mode, so the host rescale is
the identity), while a mouse/trackpad is a MOUSE — pointer-LOCKED it is GCMouse relative
deltas; unlocked it is absolute moves + buttons + scroll over the UIKit pointer path
(hover + `.indirectPointer` touches), the local cursor staying visible so you can aim. An
indirect pointer is never sent as a touch. Touch is gated on trust (not forwarded under
the TOFU prompt), and returning to the foreground restores the capture you had on leaving.
`InputCapture` is cross-platform (GC works the same on iPadOS; ⌘⎋ is detected from
the HID stream there); audio routes via `AVAudioSession` (the Settings device
pickers are macOS-only). For the iPad-with-external-display setup: the target
enables multiple scenes + indirect input events — on Stage Manager iPads, drag the
punktfunk window onto the external screen and the stream runs there with full
keyboard/mouse/touch. While streaming the session is immersive (edge-to-edge,
status bar + home indicator hidden) and the iPadOS cursor is hidden over the video only
while the scene is actually pointer-LOCKED (`UIPointerInteraction` `.hidden()`); when the
lock isn't held it stays visible and the mouse forwards as an absolute cursor instead; on
iOS first run the stream mode defaults to the device's native screen so the video
fills the display. **tvOS** runs the same app (target **Punktfunk-tvOS**, first-lit
in the Apple TV simulator at 720p60): playback-only audio (no mic on tvOS),
focus-driven UI (`.card` host tiles), no kb/mouse capture yet — input lands with
gamepad support, the natural tvOS input anyway. While streaming there is NO focusable
control (a focusable Disconnect button would let the focus engine eat the controller's A
before the host sees it); the Siri Remote's **Menu** button disconnects (`.onExitCommand`).
Core slices are tier-3 Rust targets (see Build above). Known gaps: true pointer LOCK (`prefersPointerLocked`) isn't
consulted through UIHostingController, so the hidden cursor can still drift onto a
second screen (fixing it means putting the controller into the UIKit presentation
chain); and
AVAudioSession interruptions (calls, Siri) don't auto-restart the audio engines yet
(reconnect recovers).
## Known limitations of the current host (relevant to client UX)
- One session **at a time** (the listener is persistent, but a second concurrent client
waits in the accept queue until the current session ends — the virtual output and
encoder are single-tenant).
- Mid-stream renegotiation (resolution change without reconnect) is designed-for but not
implemented (the Welcome is one-shot today).
- Host-side gamepad injection needs `/dev/uinput` access on the box (udev rule from
`design/linux-setup.md`).
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
@@ -28,10 +28,20 @@ struct ContentView: View {
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.codec) private var codec = "auto"
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
/// The `codec` setting as a `PUNKTFUNK_CODEC_*` soft-preference byte (`0` = auto).
private var preferredCodecByte: UInt8 {
switch codec {
case "h264": return PunktfunkConnection.codecH264
case "hevc": return PunktfunkConnection.codecHEVC
case "av1": return PunktfunkConnection.codecAV1
default: return 0
}
}
@State private var showAddHost = false
@State private var pairingTarget: StoredHost?
/// A fresh `pair=required`/unknown host the user tapped: drives the choice between no-PIN
@@ -45,9 +55,9 @@ struct ContentView: View {
#if !os(macOS)
@State private var showSettings = false
#endif
#if os(iOS)
#if os(iOS) || os(macOS)
// A connected controller (+ the Settings toggle) swaps the whole home screen for
// GamepadHomeView instead of retrofitting HomeView's touch UI see `home` below.
// GamepadHomeView instead of retrofitting HomeView's touch/desktop UI see `home` below.
@ObservedObject private var gamepadManager = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) private var gamepadUIEnabled = true
private var gamepadUIActive: Bool {
@@ -127,12 +137,16 @@ struct ContentView: View {
// The library is a full-screen presentation, not a sheet: on iPad a sheet is a centered page
// card, but the gamepad coverflow is meant to be an immersive, full-bleed screen (and the
// launcher behind it stops consuming the controller see GamepadHomeView's `isActive`).
// macOS has no `fullScreenCover`, so it keeps the sheet there.
// macOS has no `fullScreenCover`, so it keeps the sheet there with an explicit size: a
// macOS sheet takes its content's IDEAL size, and both library layouts are geometry-driven
// (the coverflow is a GeometryReader, ideal zero), so without a frame it collapses to a
// tiny panel.
#if os(macOS)
.sheet(item: $libraryTarget) { host in
NavigationStack {
LibraryView(store: store, host: host, onLaunch: { launchTitle(host, $0) })
}
.frame(minWidth: 940, minHeight: 620)
}
#else
.fullScreenCover(item: $libraryTarget) { host in
@@ -166,6 +180,18 @@ struct ContentView: View {
+ "device in the host's web console (port 3000 → Pairing) — no PIN needed. Or "
+ "pair with the 4-digit PIN it can display.")
}
// One "Connection failed" surface for every home screen (touch grid, gamepad launcher) and
// platform SessionModel funnels all connect/session errors into `errorMessage`.
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
set: { if !$0 { model.errorMessage = nil } })
) {
Button("OK", role: .cancel) {}
} message: {
Text(model.errorMessage ?? "")
}
// The delegated-approval wait: the host holds the connection open until the operator
// approves it. Cancel returns the UI at once; the in-flight connect is left to time out
// and its late result is discarded by SessionModel's connect guard (disconnect resets the
@@ -187,12 +213,21 @@ struct ContentView: View {
private var home: some View {
#if os(macOS)
HomeView(
store: store, model: model, discovery: discovery,
showAddHost: $showAddHost, pairingTarget: $pairingTarget,
speedTestTarget: $speedTestTarget, libraryTarget: $libraryTarget,
connect: { connect($0) }, connectDiscovered: connectDiscovered,
onPaired: handlePaired, onLaunchTitle: launchTitle)
Group {
if gamepadUIActive {
GamepadHomeView(
store: store, model: model, discovery: discovery,
libraryTarget: $libraryTarget,
connect: { connect($0) }, connectDiscovered: connectDiscovered)
} else {
HomeView(
store: store, model: model, discovery: discovery,
showAddHost: $showAddHost, pairingTarget: $pairingTarget,
speedTestTarget: $speedTestTarget, libraryTarget: $libraryTarget,
connect: { connect($0) }, connectDiscovered: connectDiscovered,
onPaired: handlePaired, onLaunchTitle: launchTitle)
}
}
#elseif os(iOS)
Group {
if gamepadUIActive {
@@ -291,14 +326,21 @@ struct ContentView: View {
onCaptureChange: { [weak model] captured in
model?.mouseCaptured = captured
},
onFrame: { [meter = model.meter, latency = model.latency, offset = conn.clockOffsetNs] au in
onFrame: { [meter = model.meter, latency = model.latency,
split = model.latencySplit, offset = conn.clockOffsetNs] au in
meter.note(byteCount: au.data.count)
latency.record(ptsNs: au.ptsNs, offsetNs: offset)
// The same receipt, keyed by pts, awaiting its 0xCF host timing (the
// host/network split drained by the 1 s stats tick).
split.recordReceipt(
ptsNs: au.ptsNs, receivedNs: au.receivedNs, offsetNs: offset)
},
onSessionEnd: { [weak model] in
Task { @MainActor in model?.sessionEnded() }
},
presentMeter: model.presentLatency
endToEndMeter: model.endToEnd,
decodeMeter: model.decodeStage,
displayMeter: model.displayStage
)
.overlay(alignment: placement.alignment) {
if captureEnabled && hudEnabled {
@@ -378,6 +420,7 @@ struct ContentView: View {
bitrateKbps: UInt32(clamping: bitrateKbps),
audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
preferredCodec: preferredCodecByte,
launchID: launchID,
allowTofu: allowTofu,
requestAccess: requestAccess)
@@ -521,6 +564,7 @@ struct ContentView: View {
bitrateKbps: bitrate,
audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
preferredCodec: preferredCodecByte,
autoTrust: true)
}
}
@@ -553,23 +597,3 @@ private struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
private extension Data {
/// Parse an even-length hex string into bytes; nil on any non-hex character or odd length.
/// Used to turn an mDNS-advertised cert fingerprint into a connect pin.
init?(hexString: String) {
let chars = Array(hexString)
guard chars.count.isMultiple(of: 2) else { return nil }
var bytes = [UInt8]()
bytes.reserveCapacity(chars.count / 2)
var i = 0
while i < chars.count {
guard let hi = chars[i].hexDigitValue, let lo = chars[i + 1].hexDigitValue else {
return nil
}
bytes.append(UInt8(hi << 4 | lo))
i += 2
}
self = Data(bytes)
}
}
@@ -0,0 +1,234 @@
// The gamepad-driven "Add Host" screen (iOS/iPadOS/macOS) the controller counterpart of
// AddHostSheet, reached from the launcher's Add Host tile. Three field rows (name / address /
// port) plus the Add action, navigated with the same vertical focus list as the gamepad settings;
// A on a field opens GamepadKeyboard in a bottom tray, so a host can be registered end to end
// without touching the screen. Field edits are live (the row shows every keystroke); B closes the
// keyboard first, then cancels the screen the same "back peels one layer" rule as a console UI.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
struct GamepadAddHostView: View {
@Environment(\.dismiss) private var dismiss
let onAdd: (StoredHost) -> Void
#if os(iOS)
/// `.compact` in a landscape phone window tighter chrome so the keyboard tray still fits.
@Environment(\.verticalSizeClass) private var vSizeClass
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the sheet is sized to fit the tray
#endif
@State private var name = ""
@State private var address = ""
@State private var port = "9777"
@State private var focusID: String?
/// The field row the keyboard tray is editing; nil the row list owns the controller.
@State private var editing: String?
var body: some View {
GamepadMenuList(
items: rows,
focusID: $focusID,
onActivate: { activate(id: $0.id) },
onBack: { dismiss() },
isActive: editing == nil
) { row, focused in
rowView(row, focused: focused)
.frame(maxWidth: 620)
.padding(.horizontal, 24)
}
.frame(maxWidth: .infinity)
.safeAreaInset(edge: .top, spacing: 0) {
VStack(spacing: 4) {
Text("Add Host")
.font(.geist(compact ? 20 : 30, .bold, relativeTo: .title))
.foregroundStyle(.white)
if !compact {
Text("Hosts on this network appear automatically — add one by address "
+ "for everything else.")
.font(.geist(13, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.55))
.multilineTextAlignment(.center)
.frame(maxWidth: 440)
}
}
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.frame(maxWidth: .infinity)
.overlay(alignment: .topTrailing) { closeButton.padding(.trailing, 20) }
.background { GamepadTrayScrim(edge: .top) }
}
.safeAreaInset(edge: .bottom, spacing: 0) {
bottomTray
.padding(.horizontal, 22)
.padding(.vertical, compact ? 6 : 10)
.background { GamepadTrayScrim(edge: .bottom) }
}
.background { GamepadScreenBackground() }
// A port can't exceed 5 digits cap while typing so the row can't grow absurd.
.onChange(of: port) { _, value in
if value.count > 5 { port = String(value.prefix(5)) }
}
}
/// The keyboard tray while editing, the controls legend otherwise.
@ViewBuilder private var bottomTray: some View {
if let editing {
VStack(spacing: 10) {
GamepadKeyboard(
text: editingBinding(editing),
allowed: allowedCharacters(editing),
onDone: { closeKeyboard() })
// Fresh keyboard per field: a touch user can retarget the tray by tapping
// another field row, and the keyboard's input wiring captured the previous
// binding on appear new identity forces a rewire to the new field.
.id(editing)
GamepadHintBar(hints: [
.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Type"),
.init(glyph: buttonGlyph(\.buttonX, fallback: "x.circle"), text: "Delete"),
.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Done"),
])
.frame(maxWidth: .infinity, alignment: .leading)
}
.transition(.move(edge: .bottom).combined(with: .opacity))
} else {
GamepadHintBar(hints: [
.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Select"),
.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Cancel"),
])
.frame(maxWidth: .infinity, alignment: .leading)
}
}
/// Touch/click fallback for closing the controller path is B, a hardware keyboard's Esc
/// rides the cancel action.
private var closeButton: some View {
Button { dismiss() } label: {
Image(systemName: "xmark")
.font(.system(size: 14, weight: .semibold))
.foregroundStyle(.white)
.frame(width: 34, height: 34)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
.buttonStyle(.plain)
.keyboardShortcut(.cancelAction)
.accessibilityLabel("Cancel")
}
// MARK: - Rows
private struct Row: Identifiable {
let id: String
let label: String
var value = ""
var placeholder = ""
var isAction = false
}
private var rows: [Row] {
[
Row(id: "name", label: "Name", value: name, placeholder: "Optional — e.g. Living Room"),
Row(id: "address", label: "Address", value: address, placeholder: "IP or hostname"),
Row(id: "port", label: "Port", value: port, placeholder: "9777"),
Row(id: "add", label: "Add Host", isAction: true),
]
}
private func rowView(_ row: Row, focused: Bool) -> some View {
HStack(spacing: 14) {
if row.isAction {
Label("Add Host", systemImage: "plus.circle.fill")
.font(.geist(16, .semibold, relativeTo: .body))
.foregroundStyle(canAdd ? Color.brand : .white.opacity(0.35))
.frame(maxWidth: .infinity)
} else {
Text(row.label)
.font(.geist(16, .semibold, relativeTo: .body))
.foregroundStyle(.white)
Spacer(minLength: 12)
Text(row.value.isEmpty ? row.placeholder : row.value)
.font(.geistFixed(15, .medium))
.foregroundStyle(row.value.isEmpty ? .white.opacity(0.35) : .white)
.lineLimit(1)
.truncationMode(.head) // keep the end of a long address visible while typing
if editing == row.id {
// The live-edit caret: this row is what the keyboard tray is typing into.
Rectangle()
.fill(Color.brand)
.frame(width: 2, height: 18)
}
}
}
.padding(.horizontal, 16)
.padding(.vertical, 13)
.background {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.fill(.white.opacity(focused || editing == row.id ? 0.1 : 0))
}
.overlay {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.strokeBorder(
editing == row.id ? Color.brand.opacity(0.7) : .white.opacity(focused ? 0.22 : 0),
lineWidth: 1)
}
.scaleEffect(focused ? 1.0 : 0.98)
.animation(.smooth(duration: 0.18), value: focused)
}
// MARK: - Actions
private func activate(id: String) {
switch id {
case "add":
guard canAdd else {
// Not addable yet jump straight to what's missing instead of a dead press.
focusID = "address"
openKeyboard("address")
return
}
onAdd(StoredHost(
name: name.trimmingCharacters(in: .whitespaces),
address: address.trimmingCharacters(in: .whitespaces),
port: UInt16(port) ?? 9777))
dismiss()
default:
openKeyboard(id)
}
}
private var canAdd: Bool {
!address.trimmingCharacters(in: .whitespaces).isEmpty
&& UInt16(port).map { $0 > 0 } == true
}
private func openKeyboard(_ id: String) {
withAnimation(.spring(response: 0.32, dampingFraction: 0.86)) { editing = id }
}
private func closeKeyboard() {
withAnimation(.spring(response: 0.32, dampingFraction: 0.86)) { editing = nil }
}
private func editingBinding(_ id: String) -> Binding<String> {
switch id {
case "name": return $name
case "port": return $port
default: return $address
}
}
/// What the keyboard may type per field: a port is digits, an address never contains spaces;
/// a name is free-form.
private func allowedCharacters(_ id: String) -> CharacterSet? {
switch id {
case "port": return CharacterSet(charactersIn: "0123456789")
case "address": return CharacterSet(charactersIn: " ").inverted
default: return nil
}
}
}
#endif
@@ -1,6 +1,6 @@
// The one piece of gamepad-menu machinery shared by the host launcher (GamepadHomeView) and the
// library coverflow (LibraryCoverflowView): a horizontal, center-snapping carousel driven entirely
// by a controller (iOS/iPadOS only).
// by a controller (iOS/iPadOS/macOS).
//
// The scrolling is pure native SwiftUI `.scrollTargetLayout()` + `.scrollTargetBehavior(.viewAligned)`
// snap exactly one item to center, and symmetric `.safeAreaPadding(.horizontal)` (sized off the live
@@ -24,8 +24,7 @@
import PunktfunkKit
import SwiftUI
#if os(iOS)
import UIKit
#if os(iOS) || os(macOS)
struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hashable {
let items: [Item]
@@ -40,6 +39,8 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
let onActivate: (Item) -> Void
/// Y the screen's secondary action (e.g. open a host's library); nil disables it.
var onSecondary: (() -> Void)?
/// X the screen's tertiary action (e.g. open settings); nil disables it.
var onTertiary: (() -> Void)?
/// B back/dismiss; nil disables it (e.g. the root launcher has nowhere to go back to).
var onBack: (() -> Void)?
/// L1/R1 jump this many items at once (clamped to the ends); 0 disables the shoulders.
@@ -94,7 +95,9 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
}
.scrollPosition(id: $scrolledID)
.scrollTargetBehavior(.viewAligned)
.scrollIndicators(.hidden)
// .never, not .hidden macOS's "always show scroll bars" setting overrides .hidden
// and paints a scroller across the console strip.
.scrollIndicators(.never)
.scrollClipDisabled() // let the focused card scale up past the strip bounds
.safeAreaPadding(.horizontal, inset)
.offset(x: bumpOffset)
@@ -147,6 +150,7 @@ struct GamepadCarousel<Item: Identifiable, Card: View>: View where Item.ID: Hash
input.onMove = { move($0) }
input.onConfirm = { activate() }
input.onSecondary = onSecondary
input.onTertiary = onTertiary
input.onBack = onBack
input.onShoulder = shoulderJump > 0 ? { shoulder(right: $0) } : nil
}
@@ -0,0 +1,232 @@
// Chrome shared by the gamepad-driven screens (GamepadHomeView, GamepadSettingsView,
// GamepadAddHostView, LibraryCoverflowView): the full-bleed console backdrop, the
// controller-glyph hint bar, and the connected-controller status chip. One look across every
// screen is what makes the gamepad UI read as a coherent mode rather than a set of themed pages.
// iOS/iPadOS and macOS (the couch Mac-mini case); tvOS keeps its native focus engine instead.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
import GameController
/// The active controller's real glyph for a button (Xbox "A", DualSense , ) via
/// `sfSymbolsName`; a generic fallback before a controller profile resolves.
/// @MainActor: GamepadManager is main-actor-bound (inside a View body this was implicit).
@MainActor
func buttonGlyph(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, fallback: String
) -> String {
GamepadManager.shared.active?.controller.extendedGamepad?[keyPath: button].sfSymbolsName
?? fallback
}
/// Top padding for a gamepad screen's pinned title. macOS gets extra clearance the launcher
/// title sits right under the window titlebar and the settings/add-host sheets have no titlebar
/// at all, so the iOS value hugs the top edge there.
func gamepadTitleTopPadding(compact: Bool) -> CGFloat {
#if os(macOS)
26
#else
compact ? 4 : 10
#endif
}
/// One glyph + label cell in a hint bar.
struct GamepadHint: Identifiable {
let glyph: String
let text: String
var id: String { glyph + text }
}
/// The pinned controls legend every gamepad screen shows bottom-leading (via `.safeAreaInset`).
/// Same font/spacing everywhere so the legend reads as system chrome, not per-screen decoration.
struct GamepadHintBar: View {
let hints: [GamepadHint]
var body: some View {
HStack(spacing: 18) {
ForEach(hints) { hint in
HStack(spacing: 7) {
Image(systemName: hint.glyph)
.font(.system(size: 19))
.foregroundStyle(.white)
Text(hint.text)
}
.fixedSize() // keep glyph + label together; never truncate a hint mid-word
}
}
.font(.geist(14, .semibold, relativeTo: .subheadline))
.foregroundStyle(.white.opacity(0.85))
}
}
/// The console backdrop: a living "aurora" field in the brand's violet family soft color blobs
/// drifting on slow Lissajous paths over black, in the direction of Apple Music's animated player
/// background but calmer (long 3090 s periods, muted opacities, a legibility scrim on top, so it
/// reads as ambience behind the cards, never as content). Deliberately pure SwiftUI rather than a
/// .metal shader: these sources are built both by SwiftPM (`swift run`/tests) and by the Xcode
/// project's synchronized folders, and a compiled metallib is only reliably bundled in one of the
/// two radial gradients driven by a TimelineView give the same look with none of that risk.
///
/// Applied via `.background { }` NOT as a ZStack sibling so the `.ignoresSafeArea()` here
/// can't inflate the caller's layout past the safe area (see the layout discipline note in
/// GamepadHomeView's header). Honors Reduce Motion by freezing the field at a fixed phase.
struct GamepadScreenBackground: View {
@Environment(\.accessibilityReduceMotion) private var reduceMotion
/// One drifting color blob: a base position + drift ellipse (unit coordinates), angular
/// speeds (rad/s periods of 3090 s), and a radius that slowly breathes.
private struct Blob {
let color: Color
let center: CGPoint
let drift: CGSize
let speed: (x: Double, y: Double)
let phase: (x: Double, y: Double)
/// Radius as a fraction of the view's larger dimension (+ breathing amplitude/speed).
let radius: CGFloat
let breathe: (amount: CGFloat, speed: Double)
let opacity: Double
}
/// The brand violet, a deeper indigo, a warmer plum, and a cool blue related hues so the
/// field shifts within one temperature instead of strobing through the rainbow.
private static let blobs: [Blob] = [
Blob(color: Color(red: 0.53, green: 0.47, blue: 0.96), // brand violet
center: CGPoint(x: 0.30, y: 0.24), drift: CGSize(width: 0.16, height: 0.10),
speed: (0.111, 0.083), phase: (0.0, 1.9),
radius: 0.52, breathe: (0.07, 0.061), opacity: 0.52),
Blob(color: Color(red: 0.24, green: 0.20, blue: 0.72), // deep indigo
center: CGPoint(x: 0.78, y: 0.66), drift: CGSize(width: 0.13, height: 0.14),
speed: (0.071, 0.096), phase: (2.4, 0.7),
radius: 0.58, breathe: (0.08, 0.049), opacity: 0.55),
Blob(color: Color(red: 0.62, green: 0.30, blue: 0.80), // plum
center: CGPoint(x: 0.16, y: 0.82), drift: CGSize(width: 0.12, height: 0.09),
speed: (0.089, 0.067), phase: (4.1, 3.2),
radius: 0.44, breathe: (0.09, 0.078), opacity: 0.42),
Blob(color: Color(red: 0.22, green: 0.38, blue: 0.86), // cool blue
center: CGPoint(x: 0.70, y: 0.12), drift: CGSize(width: 0.10, height: 0.08),
speed: (0.059, 0.104), phase: (1.2, 5.0),
radius: 0.40, breathe: (0.06, 0.055), opacity: 0.38),
]
var body: some View {
Group {
if reduceMotion {
field(at: 0)
} else {
// 30 Hz is plenty for centimeters-per-minute drift, and halves the redraw cost
// of a battery-fed couch device vs. the default display rate.
TimelineView(.animation(minimumInterval: 1.0 / 30.0)) { context in
field(at: context.date.timeIntervalSinceReferenceDate)
}
}
}
.ignoresSafeArea()
}
private func field(at t: TimeInterval) -> some View {
GeometryReader { geo in
let side = max(geo.size.width, geo.size.height)
ZStack {
Color.black
ZStack {
ForEach(Self.blobs.indices, id: \.self) { i in
blobView(Self.blobs[i], at: t, in: geo.size, side: side)
}
}
// ±10° over ~5 min the whole field very slowly warms and cools.
.hueRotation(.degrees(sin(t * 0.021) * 10))
// Composite the additive blobs offscreen once instead of per-layer.
.drawingGroup()
// Legibility scrim: the title (top) and detail/hints (bottom) always sit on
// near-black, whatever the blobs are doing behind them.
LinearGradient(
stops: [
.init(color: .black.opacity(0.55), location: 0),
.init(color: .black.opacity(0.15), location: 0.35),
.init(color: .black.opacity(0.20), location: 0.65),
.init(color: .black.opacity(0.60), location: 1),
],
startPoint: .top, endPoint: .bottom)
}
}
}
private func blobView(_ blob: Blob, at t: TimeInterval, in size: CGSize, side: CGFloat) -> some View {
let x = blob.center.x + blob.drift.width * CGFloat(sin(t * blob.speed.x + blob.phase.x))
let y = blob.center.y + blob.drift.height * CGFloat(cos(t * blob.speed.y + blob.phase.y))
let r = side * blob.radius
* (1 + blob.breathe.amount * CGFloat(sin(t * blob.breathe.speed + blob.phase.x)))
return Circle()
.fill(RadialGradient(
colors: [blob.color, blob.color.opacity(0)],
center: .center, startRadius: 0, endRadius: r / 2))
.frame(width: r, height: r)
.position(x: x * size.width, y: y * size.height)
.opacity(blob.opacity)
.blendMode(.plusLighter)
}
}
/// A darkening scrim behind a pinned tray (a screen title, the hints/detail bar, the keyboard
/// tray): scrollable rows pass beneath those insets, so without this the tray text and the row
/// underneath render interleaved. Fades toward the content so it reads as depth, not a bar.
struct GamepadTrayScrim: View {
let edge: VerticalEdge
var body: some View {
LinearGradient(
stops: [
.init(color: .black.opacity(0.92), location: 0),
.init(color: .black.opacity(0.85), location: 0.55),
.init(color: .black.opacity(0), location: 1),
],
startPoint: edge == .top ? .top : .bottom,
endPoint: edge == .top ? .bottom : .top)
// Grow past the tray so the fade-to-clear happens OUTSIDE its bounds the tray's own
// text always sits on the near-opaque part, rows dim before they reach it.
.padding(edge == .top ? .bottom : .top, -32)
.ignoresSafeArea()
}
}
/// "Which pad is driving this UI" the active controller's name and battery, worn as a quiet
/// chip in the launcher's top bar. Callers observe GamepadManager already, so this re-renders
/// when the pad or its battery state changes.
struct ControllerStatusChip: View {
let controller: GamepadManager.DiscoveredController
var body: some View {
HStack(spacing: 7) {
Image(systemName: controller.hasTouchpadAndMotion
? "playstation.logo" : "gamecontroller.fill")
.font(.system(size: 12))
Text(controller.name)
.lineLimit(1)
if let level = controller.batteryLevel {
Image(systemName: batterySymbol(level))
.font(.system(size: 12))
.foregroundStyle(level <= 0.2 && !controller.isCharging
? AnyShapeStyle(.red) : AnyShapeStyle(.white.opacity(0.7)))
}
}
.font(.geist(12, .medium, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.7))
.padding(.horizontal, 12)
.padding(.vertical, 7)
.background(Capsule().fill(.white.opacity(0.08)))
.overlay(Capsule().strokeBorder(.white.opacity(0.12), lineWidth: 1))
}
private func batterySymbol(_ level: Float) -> String {
if controller.isCharging { return "battery.100.bolt" }
switch level {
case ..<0.125: return "battery.0"
case ..<0.375: return "battery.25"
case ..<0.625: return "battery.50"
case ..<0.875: return "battery.75"
default: return "battery.100"
}
}
}
#endif
@@ -1,8 +1,9 @@
// The gamepad-driven home screen (iOS/iPadOS only): a distinct, "10-foot" console-style host
// launcher shown INSTEAD of HomeView while GamepadUIEnvironment is active a separate screen built
// around a center-snapping carousel of hosts, driven from the couch with a controller. No touch is
// required (a tap still works as a fallback). Scope: browse saved + discovered hosts, connect, and
// when the library flag is on jump into a saved host's library (Y).
// required anywhere: A connects, Y opens a saved host's library (when the flag is on), X opens the
// gamepad settings screen, and the carousel always ends in an Add Host tile that opens the
// controller-keyboard add flow. (A tap still works as a fallback for all of it.)
//
// All the scrolling/snapping/navigation/haptics live in GamepadCarousel; this file is the launcher's
// chrome. Layout discipline (so nothing is EVER clipped, portrait or landscape): the gradient is a
@@ -11,18 +12,21 @@
// status bar / home indicator. As a background it draws behind without affecting layout, so the
// GeometryReader is sized to the safe area. The title and the controller-glyph hints are pinned with
// `.safeAreaInset` (top / bottom-leading) guaranteed inside the safe area and out of the carousel's
// vertical budget and the card is sized off the remaining height. tvOS/macOS never mount this view.
// vertical budget and the card is sized off the remaining height. macOS mounts it too (the
// couch Mac-mini case) same screen, with the settings/add-host covers presented as sheets
// (macOS has no fullScreenCover). tvOS never mounts this view (native focus engine instead).
import PunktfunkKit
import SwiftUI
#if os(iOS)
#if os(iOS) || os(macOS)
import GameController
/// One navigable tile: a saved host or a discovered-but-unsaved one. Hashable so it can be the
/// carousel's scroll-position identity.
/// One navigable tile: a saved host, a discovered-but-unsaved one, or the trailing Add Host
/// action. Hashable so it can be the carousel's scroll-position identity.
private enum GamepadHomeTarget: Hashable {
case saved(UUID)
case discovered(String)
case addHost
}
/// A fully-resolved launcher tile display fields + the activate action, built fresh each render
@@ -31,13 +35,17 @@ private struct HomeTile: Identifiable {
let id: GamepadHomeTarget
let title: String
let subtitle: String
let isOnline: Bool
let isPaired: Bool
let isConnecting: Bool
/// Saved (solid monogram) vs. discovered-but-unsaved (tinted outline).
let filled: Bool
var isOnline = false
var isPaired = false
var isConnecting = false
/// Saved (solid monogram) vs. discovered-but-unsaved / action (tinted outline).
var filled = false
/// Only saved hosts have a library (matches the touch grid's context-menu gate).
let hasLibrary: Bool
var hasLibrary = false
/// Shows this SF symbol in the badge instead of the title monogram (the Add Host tile).
var icon: String?
/// Whether the detail panel shows the online/paired pill (hosts yes, actions no).
var showsStatus = true
let activate: () -> Void
}
@@ -51,12 +59,18 @@ struct GamepadHomeView: View {
/// Same experimental gate the touch grid's "Browse Library" context-menu item uses.
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
#if os(iOS)
/// `.compact` in a landscape phone window drives tighter chrome so everything still fits.
@Environment(\.verticalSizeClass) private var vSizeClass
@State private var selection: GamepadHomeTarget?
@State private var breathe = false
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the window minimum keeps room
#endif
@ObservedObject private var gamepads = GamepadManager.shared
@State private var selection: GamepadHomeTarget?
@State private var showSettings = false
@State private var showAddHost = false
var body: some View {
GeometryReader { geo in
@@ -64,97 +78,70 @@ struct GamepadHomeView: View {
}
// Pinned inside the safe area, out of the carousel's vertical budget never clipped.
.safeAreaInset(edge: .top, spacing: 0) {
titleView
.padding(.top, compact ? 4 : 10)
titleBar
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.frame(maxWidth: .infinity)
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
if !tiles.isEmpty {
hintBar
.padding(.leading, 22)
.padding(.vertical, compact ? 6 : 10)
}
}
.background { background }
.onAppear {
discovery.start()
withAnimation(.easeInOut(duration: 4).repeatForever(autoreverses: true)) { breathe = true }
GamepadHintBar(hints: hints)
.padding(.leading, 22)
.padding(.vertical, compact ? 6 : 10)
}
.background { GamepadScreenBackground() }
.onAppear { discovery.start() }
.onDisappear { discovery.stop() }
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
set: { if !$0 { model.errorMessage = nil } })
) {
Button("OK", role: .cancel) {}
} message: {
Text(model.errorMessage ?? "")
// The settings / add-host screens take over the controller (the carousel's `isActive`
// gate above). iOS presents them full screen the immersive console feel; macOS has no
// fullScreenCover, so they become generously sized sheets over the dimmed launcher.
#if os(macOS)
.sheet(isPresented: $showSettings) {
GamepadSettingsView()
.frame(width: 720, height: 640)
}
.sheet(isPresented: $showAddHost) {
GamepadAddHostView { store.add($0) }
.frame(width: 660, height: 620)
}
.frame(minWidth: 640, minHeight: 420)
#else
.fullScreenCover(isPresented: $showSettings) { GamepadSettingsView() }
.fullScreenCover(isPresented: $showAddHost) {
GamepadAddHostView { store.add($0) }
}
#endif
}
// MARK: - Hero (carousel + detail), sized to fit the space between the pinned title and hints
@ViewBuilder private func hero(for size: CGSize) -> some View {
if tiles.isEmpty {
emptyState.frame(maxWidth: .infinity, maxHeight: .infinity)
} else {
let cardWidth = min(340, size.width * 0.84)
// 96 the carousel's own vertical breathing (+40) plus the detail line (~54); clamp so
// the strip + detail always fit the region the safe-area insets leave.
let cardHeight = min(compact ? 170 : 210, max(118, size.height - 96))
VStack(spacing: compact ? 8 : 10) {
Spacer(minLength: 0)
carousel(cardWidth: cardWidth, cardHeight: cardHeight)
detailPanel
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
let cardWidth = min(340, size.width * 0.84)
// 96 the carousel's own vertical breathing (+40) plus the detail line (~54); clamp so
// the strip + detail always fit the region the safe-area insets leave.
let cardHeight = min(compact ? 170 : 210, max(118, size.height - 96))
VStack(spacing: compact ? 8 : 10) {
Spacer(minLength: 0)
carousel(cardWidth: cardWidth, cardHeight: cardHeight)
detailPanel
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
// MARK: - Chrome
private var background: some View {
ZStack {
LinearGradient(
colors: [.black, Color.brand.opacity(0.22), .black],
startPoint: .top, endPoint: .bottom)
// A soft brand orb behind the strip gives the flat gradient depth; it breathes slowly.
Circle()
.fill(RadialGradient(
colors: [Color.brand.opacity(0.55), .clear],
center: .center, startRadius: 0, endRadius: 300))
.frame(width: 560, height: 560)
.blur(radius: 70)
.scaleEffect(breathe ? 1.08 : 0.92)
.opacity(breathe ? 0.5 : 0.32)
.offset(y: -20)
}
.ignoresSafeArea()
}
private var titleView: some View {
private var titleBar: some View {
Text("Select a Host")
.font(.geist(compact ? 20 : 30, .bold, relativeTo: .title))
.foregroundStyle(.white)
}
private var emptyState: some View {
VStack(spacing: 14) {
Image(systemName: "gamecontroller")
.font(.system(size: 46, weight: .light))
.foregroundStyle(Color.brand)
Text("No hosts yet")
.font(.geist(20, .semibold, relativeTo: .title3))
.foregroundStyle(.white)
Text("Add one with touch first — it'll show up here for the controller.")
.font(.geist(15, relativeTo: .body))
.foregroundStyle(.white.opacity(0.6))
.multilineTextAlignment(.center)
.frame(maxWidth: 320)
}
.frame(maxWidth: .infinity)
.overlay(alignment: .trailing) {
// Which pad is driving this UI (name + battery) quiet, and only where there's
// room; a compact-height phone gives the pixels to the carousel instead.
if !compact, let active = gamepads.active {
ControllerStatusChip(controller: active)
.padding(.trailing, 20)
}
}
}
// MARK: - Carousel
@@ -167,9 +154,10 @@ struct GamepadHomeView: View {
spacing: 30,
onActivate: { $0.activate() },
onSecondary: { openLibraryForSelected() },
// Stop consuming the controller while the library is presented on top otherwise the
// launcher navigates behind it (invisibly on iPhone, visibly on iPad's page sheet).
isActive: libraryTarget == nil
onTertiary: { showSettings = true },
// Stop consuming the controller while another screen is presented on top otherwise
// the launcher navigates behind it (invisibly on iPhone, visibly on iPad's page sheet).
isActive: libraryTarget == nil && !showSettings && !showAddHost
) { tile in
hostCard(tile, size: CGSize(width: cardWidth, height: cardHeight))
}
@@ -211,7 +199,7 @@ struct GamepadHomeView: View {
Text(tile?.subtitle ?? " ")
.font(.geist(13, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.6))
if let tile {
if let tile, tile.showsStatus {
statusPill(online: tile.isOnline, paired: tile.isPaired)
}
}
@@ -236,71 +224,52 @@ struct GamepadHomeView: View {
// MARK: - Hint bar (pinned bottom-leading via safeAreaInset)
private var hintBar: some View {
HStack(spacing: 18) {
hint(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Connect")
if showsLibraryHint {
hint(glyph: buttonGlyph(\.buttonY, fallback: "y.circle"), text: "Library")
}
private var hints: [GamepadHint] {
let selected = tiles.first { $0.id == selection }
var hints = [GamepadHint(
glyph: buttonGlyph(\.buttonA, fallback: "a.circle"),
text: selected?.id == .addHost ? "Add Host" : "Connect")]
if libraryEnabled, selected?.hasLibrary == true {
hints.append(.init(glyph: buttonGlyph(\.buttonY, fallback: "y.circle"), text: "Library"))
}
.font(.geist(14, .semibold, relativeTo: .subheadline))
.foregroundStyle(.white.opacity(0.85))
}
private func hint(glyph: String, text: String) -> some View {
HStack(spacing: 7) {
Image(systemName: glyph)
.font(.system(size: 19))
.foregroundStyle(.white)
Text(text)
}
.fixedSize() // keep glyph + label together; never truncate a hint mid-word
}
private var showsLibraryHint: Bool {
guard libraryEnabled else { return false }
return tiles.first { $0.id == selection }?.hasLibrary ?? false
}
/// The active controller's real glyph for a button (Xbox "A", DualSense , ) via
/// `sfSymbolsName`; a generic fallback before a controller profile resolves.
private func buttonGlyph(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, fallback: String
) -> String {
GamepadManager.shared.active?.controller.extendedGamepad?[keyPath: button].sfSymbolsName
?? fallback
hints.append(.init(glyph: buttonGlyph(\.buttonX, fallback: "x.circle"), text: "Settings"))
return hints
}
// MARK: - Data + actions
/// Built fresh each render from the live stores (no stale value capture) saved hosts first,
/// then discovered-but-unsaved ones.
/// then discovered-but-unsaved ones, then the Add Host action tile (so the strip is never
/// empty and manual entry is always one press away).
private var tiles: [HomeTile] {
let saved = store.hosts.map { host in
HomeTile(
id: .saved(host.id),
title: host.displayName,
subtitle: "\(host.address):\(String(host.port))",
isOnline: isOnline(host),
isOnline: discovery.advertises(host),
isPaired: host.pinnedSHA256 != nil,
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
filled: true,
hasLibrary: true,
activate: { connect(host) })
}
let discovered = discoveredUnsaved.map { d in
let discovered = discovery.unsaved(among: store.hosts).map { d in
HomeTile(
id: .discovered(d.id),
title: d.name,
subtitle: "\(d.host):\(String(d.port))",
isOnline: true,
isPaired: false,
isConnecting: false,
filled: false,
hasLibrary: false,
activate: { connectDiscovered(d) })
}
return saved + discovered
let add = HomeTile(
id: .addHost,
title: "Add Host",
subtitle: "Register a host by address",
icon: "plus",
showsStatus: false,
activate: { showAddHost = true })
return saved + discovered + [add]
}
/// Only saved hosts have a library matches the touch grid, where "Browse Library" is a
@@ -311,14 +280,6 @@ struct GamepadHomeView: View {
else { return }
libraryTarget = host
}
private func isOnline(_ host: StoredHost) -> Bool {
discovery.hosts.contains { host.matches($0) }
}
private var discoveredUnsaved: [DiscoveredHost] {
discovery.hosts.filter { d in !store.hosts.contains { $0.matches(d) } }
}
}
/// One "console tile" in the host carousel a dark-glass landscape card, bigger and bolder than the
@@ -381,6 +342,10 @@ private struct GamepadHostTile: View {
: AnyShapeStyle(Color.brand.opacity(0.16)))
if tile.isConnecting {
ProgressView().tint(.white)
} else if let icon = tile.icon {
Image(systemName: icon)
.font(.system(size: 24, weight: .semibold))
.foregroundStyle(Color.brand)
} else {
Text(monogram(tile.title))
.font(.geistFixed(25, .bold))
@@ -0,0 +1,182 @@
// A controller-driven on-screen keyboard for the gamepad UI's text fields (iOS/iPadOS only)
// iOS has no system keyboard a game controller can drive (the tvOS fullscreen entry doesn't
// exist here), so without this, adding a host from the couch would end with "now touch the
// screen". Dpad/stick moves a key cursor over a fixed grid, A types, X backspaces, B/Y confirms.
// Lowercase + digits + the hostname/address punctuation is deliberately the whole character set:
// these fields hold names, addresses and ports, not prose.
//
// Edits are applied to the binding live (the caller's field row shows every keystroke), so
// closing the keyboard is always "done" there is no separate cancel/commit step to get wrong.
// Touch stays a fallback: every keycap is tappable.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
struct GamepadKeyboard: View {
@Binding var text: String
/// Restricts typed characters (e.g. digits for a port field); backspace always works.
var allowed: CharacterSet?
/// B / Y / the Done key the binding already holds the final text.
let onDone: () -> Void
@State private var input = GamepadMenuInput(manager: .shared)
@State private var haptics = MenuHaptics(manager: .shared)
@State private var cursor = GridPos(row: 1, col: 0) // opens on "q"
@State private var pressTick = 0
@State private var boundaryTick = 0
#if os(iOS)
/// `.compact` (landscape phone): shorter keycaps so the tray leaves room for the field rows.
@Environment(\.verticalSizeClass) private var vSizeClass
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the sheet is sized generously
#endif
private struct GridPos: Hashable {
var row: Int
var col: Int
}
private enum Key: Hashable {
case char(Character)
case space
case backspace
case done
}
/// Digits first (addresses/ports), then letters; the last char column carries the
/// hostname/address punctuation.
private static let rows: [[Key]] = [
Array("1234567890").map(Key.char),
Array("qwertyuiop").map(Key.char),
Array("asdfghjkl-").map(Key.char),
Array("zxcvbnm._:").map(Key.char),
[.space, .backspace, .done],
]
var body: some View {
VStack(spacing: compact ? 5 : 7) {
ForEach(Self.rows.indices, id: \.self) { r in
HStack(spacing: compact ? 5 : 7) {
ForEach(Self.rows[r].indices, id: \.self) { c in
keycap(Self.rows[r][c], focused: cursor == GridPos(row: r, col: c))
.onTapGesture {
cursor = GridPos(row: r, col: c)
press(Self.rows[r][c])
}
}
}
}
}
.frame(maxWidth: 560)
.padding(compact ? 10 : 14)
.background {
RoundedRectangle(cornerRadius: 22, style: .continuous)
.fill(.ultraThinMaterial)
.environment(\.colorScheme, .dark)
}
.overlay {
RoundedRectangle(cornerRadius: 22, style: .continuous)
.strokeBorder(.white.opacity(0.12), lineWidth: 1)
}
.sensoryFeedback(.selection, trigger: cursor)
.sensoryFeedback(.impact(weight: .light), trigger: pressTick)
.sensoryFeedback(.impact(flexibility: .rigid, intensity: 0.7), trigger: boundaryTick)
.onAppear {
wire()
input.start()
}
.onDisappear {
input.stop()
haptics.stop()
}
}
// MARK: - Keycaps
@ViewBuilder private func keycap(_ key: Key, focused: Bool) -> some View {
Group {
switch key {
case .char(let c):
Text(String(c)).font(.geistFixed(18, .medium))
case .space:
Image(systemName: "space")
case .backspace:
Image(systemName: "delete.left")
case .done:
Label("Done", systemImage: "checkmark")
.font(.geist(15, .semibold, relativeTo: .callout))
}
}
.foregroundStyle(focused ? Color.black : .white)
.frame(maxWidth: .infinity, minHeight: compact ? 34 : 42)
.background {
RoundedRectangle(cornerRadius: 9, style: .continuous)
.fill(focused ? AnyShapeStyle(Color.brand) : AnyShapeStyle(.white.opacity(0.08)))
}
.animation(.smooth(duration: 0.12), value: focused)
.contentShape(Rectangle())
}
// MARK: - Input
private func wire() {
input.onMove = { move($0) }
input.onConfirm = { press(Self.rows[cursor.row][cursor.col]) }
input.onTertiary = { press(.backspace) }
input.onSecondary = onDone
input.onBack = onDone
}
private func move(_ direction: GamepadMenuInput.Direction) {
var next = cursor
switch direction {
case .left: next.col -= 1
case .right: next.col += 1
case .up, .down:
let row = cursor.row + (direction == .down ? 1 : -1)
guard row >= 0, row < Self.rows.count else { return refuse() }
// Map the column proportionally between rows of different widths, so e.g. Done
// (rightmost of 3) goes up to the rightmost letters, not to "e".
let from = max(1, Self.rows[cursor.row].count - 1)
let to = Self.rows[row].count - 1
next = GridPos(
row: row,
col: Int((Double(cursor.col) * Double(to) / Double(from)).rounded()))
}
guard next.row >= 0, next.row < Self.rows.count,
next.col >= 0, next.col < Self.rows[next.row].count
else { return refuse() }
cursor = next
haptics.move()
}
private func press(_ key: Key) {
switch key {
case .char(let c):
if let allowed, !c.unicodeScalars.allSatisfy(allowed.contains) { return refuse() }
text.append(c)
case .space:
if let allowed, !allowed.contains(" ") { return refuse() }
text.append(" ")
case .backspace:
guard !text.isEmpty else { return refuse() }
text.removeLast()
case .done:
haptics.confirm()
onDone()
return
}
pressTick &+= 1
haptics.move()
}
/// Refused input (edge of the grid, a disallowed character, deleting nothing).
private func refuse() {
boundaryTick &+= 1
haptics.boundary()
}
}
#endif
@@ -0,0 +1,178 @@
// The vertical sibling of GamepadCarousel (iOS/iPadOS/macOS): a controller-driven focus list for
// the gamepad UI's form-like screens (GamepadSettingsView, GamepadAddHostView). Up/down moves a
// focus bar through the rows, left/right adjusts the focused row's value, A activates it, B backs
// out. The CALLER owns each row's look (it gets the focused flag); this component owns the focus
// cursor, controller polling, haptics, and keeping the focused row scrolled into view.
//
// Unlike the carousel there is no snapping and no `.scrollPosition` two-way binding to fight: the
// cursor is plainly authoritative, the scroll view just chases it with `scrollTo`. Touch stays a
// first-class fallback tapping a row focuses AND activates it (rows are always fully visible, so
// the carousel's "first tap re-centers" step would only add friction here), and free finger
// scrolling is never hijacked back to the focused row until the next controller move.
//
// Feedback is dual-channel like the carousel: `.sensoryFeedback` ticks the DEVICE Taptic engine,
// `MenuHaptics` ticks the CONTROLLER. Moves and value changes get the crisp detent; a refused
// move at either end gets the dull boundary thud plus a short vertical recoil.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
struct GamepadMenuList<Item: Identifiable, Row: View>: View where Item.ID: Hashable {
let items: [Item]
/// Output only: the list WRITES the focused item's id here (e.g. for a caller's hint bar).
@Binding var focusID: Item.ID?
/// Left/right on the focused row. Return whether the value actually changed true plays the
/// move detent, false the boundary thud (end of a clamped range, or nothing to adjust).
var onAdjust: ((Item, Int) -> Bool)?
/// A activate the focused row (toggle it, open it, run it the caller decides).
let onActivate: (Item) -> Void
/// B back/dismiss; nil disables it.
var onBack: (() -> Void)?
/// Whether this list currently owns controller input same handoff contract as
/// GamepadCarousel's `isActive` (a covered screen must stop polling the shared pad).
var isActive: Bool = true
@ViewBuilder let row: (Item, _ focused: Bool) -> Row
@State private var input = GamepadMenuInput(manager: .shared)
@State private var haptics = MenuHaptics(manager: .shared)
/// Authoritative focus cursor (index into `items`).
@State private var cursor = 0
/// A short vertical recoil when a move is refused at a list end.
@State private var bumpOffset: CGFloat = 0
/// `.sensoryFeedback` counters (see GamepadCarousel): device ticks for activate / value-change
/// / end-stop events; moves trigger on `cursor` itself.
@State private var activateTick = 0
@State private var adjustTick = 0
@State private var boundaryTick = 0
var body: some View {
ScrollViewReader { proxy in
ScrollView(.vertical) {
LazyVStack(spacing: 6) {
ForEach(Array(items.enumerated()), id: \.element.id) { idx, item in
row(item, idx == cursor && isActive)
.contentShape(Rectangle())
.onTapGesture { tap(idx) }
.id(item.id)
}
}
.padding(.vertical, 10)
}
// .never, not .hidden macOS's "always show scroll bars" setting overrides .hidden.
.scrollIndicators(.never)
.offset(y: bumpOffset)
.onChange(of: cursor) { _, newValue in
guard newValue >= 0, newValue < items.count else { return }
withAnimation(.easeOut(duration: 0.2)) {
proxy.scrollTo(items[newValue].id)
}
}
}
.sensoryFeedback(.selection, trigger: cursor)
.sensoryFeedback(.selection, trigger: adjustTick)
.sensoryFeedback(.impact(weight: .medium), trigger: activateTick)
.sensoryFeedback(.impact(flexibility: .rigid, intensity: 0.7), trigger: boundaryTick)
.onAppear {
reconcile()
wire()
if isActive { input.start() }
}
.onDisappear {
input.stop()
haptics.stop()
}
.onChange(of: isActive) { _, active in
if active {
wire()
input.start()
} else {
input.stop()
haptics.stop()
}
}
// Re-seed a dropped focus AND re-wire the input callbacks so they capture the current
// `items` value (a plain array it would otherwise go stale in the stored closures).
.onChange(of: items.map(\.id)) { _, _ in
reconcile()
wire()
}
}
// MARK: - Input wiring
private func wire() {
input.onMove = { direction in
switch direction {
case .up: step(by: -1)
case .down: step(by: 1)
case .left: adjust(by: -1)
case .right: adjust(by: 1)
}
}
input.onConfirm = { activate() }
input.onBack = onBack
}
private func step(by delta: Int) {
guard !items.isEmpty else { return }
let target = cursor + delta
guard target >= 0, target < items.count else { return boundaryBump(forward: delta > 0) }
cursor = target
focusID = items[target].id
haptics.move()
}
private func adjust(by delta: Int) {
guard let onAdjust, cursor >= 0, cursor < items.count else { return }
if onAdjust(items[cursor], delta) {
adjustTick &+= 1
haptics.move()
} else {
boundaryTick &+= 1
haptics.boundary()
}
}
private func activate() {
guard cursor >= 0, cursor < items.count else { return }
activateTick &+= 1
haptics.confirm()
onActivate(items[cursor])
}
/// Touch fallback: a tap focuses the row and activates it in one go.
private func tap(_ idx: Int) {
guard idx >= 0, idx < items.count else { return }
if cursor != idx {
cursor = idx
focusID = items[idx].id
}
activate()
}
/// Keep `cursor`/`focusID` consistent with `items`: seed on appear; on a list change keep the
/// same focused item when it survives, else clamp the cursor into range.
private func reconcile() {
guard !items.isEmpty else {
cursor = 0
if focusID != nil { focusID = nil }
return
}
if let id = focusID, let idx = items.firstIndex(where: { $0.id == id }) {
cursor = idx
} else {
cursor = min(max(cursor, 0), items.count - 1)
focusID = items[cursor].id
}
}
private func boundaryBump(forward: Bool) {
boundaryTick &+= 1
haptics.boundary()
let recoil: CGFloat = forward ? -14 : 14
withAnimation(.spring(response: 0.16, dampingFraction: 0.42)) { bumpOffset = recoil }
withAnimation(.spring(response: 0.34, dampingFraction: 0.7).delay(0.1)) { bumpOffset = 0 }
}
}
#endif
@@ -137,17 +137,6 @@ struct HomeView: View {
}
#endif
#endif
.alert(
"Connection failed",
isPresented: Binding(
get: { model.errorMessage != nil },
set: { if !$0 { model.errorMessage = nil } }
)
) {
Button("OK", role: .cancel) {}
} message: {
Text(model.errorMessage ?? "")
}
}
// MARK: - Cards
@@ -156,7 +145,7 @@ struct HomeView: View {
let onBrowseLibrary: (() -> Void)? = libraryEnabled ? { libraryTarget = host } : nil
return HostCardView(
host: host,
isOnline: isOnline(host),
isOnline: discovery.advertises(host),
isConnecting: model.phase == .connecting && model.activeHost?.id == host.id,
isMostRecent: host.id == mostRecentHostID,
isBusy: model.isBusy,
@@ -186,18 +175,10 @@ struct HomeView: View {
.padding(.top, store.hosts.isEmpty ? 0 : 8)
}
/// A saved host is "online" iff a live mDNS advert currently matches it (see
/// `StoredHost.matches`). Recomputed on every discovery change (the @Published set), so the
/// dot tracks hosts appearing/leaving the network live.
private func isOnline(_ host: StoredHost) -> Bool {
discovery.hosts.contains { host.matches($0) }
}
/// Discovered hosts not already saved the saved grid shows the rest, so this section only
/// surfaces genuinely-new hosts on the network. Same match as the online dot, so a saved host
/// whose IP changed (still fingerprint-matched) doesn't also appear here as a stranger.
/// Discovered hosts not already saved (see `HostDiscovery.unsaved` shared with the gamepad
/// launcher so both screens classify hosts identically).
private var discoveredUnsaved: [DiscoveredHost] {
discovery.hosts.filter { d in !store.hosts.contains { $0.matches(d) } }
discovery.unsaved(among: store.hosts)
}
/// The host of the most recent session its card carries the accent ring.
@@ -1,4 +1,4 @@
// The gamepad-driven presentation of the game library (iOS/iPadOS only see LibraryView's
// The gamepad-driven presentation of the game library (iOS/iPadOS/macOS see LibraryView's
// `gamepadUIActive` branch): a classic coverflow instead of the touch grid. All the
// scrolling/snapping/navigation/haptics live in GamepadCarousel; this file is the coverflow card
// (poster + the 3D recede treatment via `.scrollTransition`), the "now focused" detail panel, and
@@ -15,9 +15,8 @@
import PunktfunkKit
import SwiftUI
#if os(iOS)
#if os(iOS) || os(macOS)
import GameController
import UIKit
struct LibraryCoverflowView: View {
let games: [GameEntry]
@@ -27,27 +26,26 @@ struct LibraryCoverflowView: View {
/// Close button already covers that); this is what makes gamepad-only exit possible.
var onDismiss: (() -> Void)?
#if os(iOS)
/// `.compact` in a landscape phone window drives a tighter poster so everything still fits.
@Environment(\.verticalSizeClass) private var vSizeClass
@State private var selection: String?
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS
#endif
@State private var selection: String?
var body: some View {
GeometryReader { geo in
content(for: geo.size)
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
hintBar
GamepadHintBar(hints: hints)
.padding(.leading, 22)
.padding(.vertical, compact ? 6 : 10)
}
.background {
LinearGradient(
colors: [.black, Color.brand.opacity(0.16), .black],
startPoint: .top, endPoint: .bottom)
.ignoresSafeArea()
}
.background { GamepadScreenBackground() }
}
@ViewBuilder private func content(for size: CGSize) -> some View {
@@ -138,34 +136,13 @@ struct LibraryCoverflowView: View {
// MARK: - Hint bar (pinned bottom-leading via safeAreaInset)
private var hintBar: some View {
HStack(spacing: 18) {
if onLaunch != nil {
hint(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Launch")
}
hint(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Close")
private var hints: [GamepadHint] {
var hints: [GamepadHint] = []
if onLaunch != nil {
hints.append(.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Launch"))
}
.font(.geist(14, .semibold, relativeTo: .subheadline))
.foregroundStyle(.white.opacity(0.85))
}
private func hint(glyph: String, text: String) -> some View {
HStack(spacing: 7) {
Image(systemName: glyph)
.font(.system(size: 19))
.foregroundStyle(.white)
Text(text)
}
.fixedSize() // keep glyph + label together; never truncate a hint mid-word
}
/// The active controller's real glyph for a button (Xbox "B", DualSense , ) via
/// `sfSymbolsName`; a generic fallback before a controller profile resolves.
private func buttonGlyph(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, fallback: String
) -> String {
GamepadManager.shared.active?.controller.extendedGamepad?[keyPath: button].sfSymbolsName
?? fallback
hints.append(.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Close"))
return hints
}
}
#endif
@@ -5,11 +5,6 @@
import PunktfunkKit
import SwiftUI
#if canImport(UIKit)
import UIKit
#elseif canImport(AppKit)
import AppKit
#endif
struct LibraryView: View {
@ObservedObject var store: HostStore
@@ -26,9 +21,9 @@ struct LibraryView: View {
/// list fetch, reused across every poster in the grid). Built alongside `games` in `load()`;
/// torn down on disappear since it isn't one-shot like `LibraryClient.fetch`'s own session.
@State private var imageSession: URLSession?
#if os(iOS)
// Gamepad-driven browsing is iOS/iPadOS-only see HomeView's identical gate. tvOS keeps its
// existing plain-grid presentation of this same view unchanged.
#if os(iOS) || os(macOS)
// Gamepad-driven browsing (iOS/iPadOS/macOS) see ContentView's identical gate. tvOS keeps
// its existing plain-grid presentation of this same view unchanged.
@ObservedObject private var gamepadManager = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) private var gamepadUIEnabled = true
private var gamepadUIActive: Bool {
@@ -74,7 +69,7 @@ struct LibraryView: View {
} else if games.isEmpty {
emptyState
} else {
#if os(iOS)
#if os(iOS) || os(macOS)
if gamepadUIActive {
LibraryCoverflowView(
games: games, imageSession: imageSession, onLaunch: onLaunch,
@@ -202,88 +197,3 @@ private struct GameCard: View {
}
}
}
/// The store-provenance badge (Steam vs. a user-curated custom entry) overlaid on a poster
/// shared by the touch grid's `GameCard` and the gamepad coverflow's cover cell.
struct StoreBadge: View {
let isCustom: Bool
var body: some View {
Text(isCustom ? "Custom" : "Steam")
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 6)
.padding(.vertical, 3)
.background(.ultraThinMaterial, in: Capsule())
.padding(6)
}
}
#if canImport(UIKit)
private typealias PlatformImage = UIImage
#elseif canImport(AppKit)
private typealias PlatformImage = NSImage
#endif
private extension Image {
init(platformImage: PlatformImage) {
#if canImport(UIKit)
self.init(uiImage: platformImage)
#elseif canImport(AppKit)
self.init(nsImage: platformImage)
#endif
}
}
/// Sequentially tries cover-art URLs over `session` (so a paired client can reach the host's own
/// art proxy, not just public CDNs see `LibraryImageLoader`), advancing past any that fail to
/// load, then a placeholder. The loaded image is hard-clipped to fill the card's actual frame
/// regardless of its own aspect ratio: a portrait capsule fills it as intended, but a fallback
/// banner (wide hero/header art, used when a title has no portrait capsule) would otherwise report
/// a much wider intrinsic size than the card and overflow into neighboring cards. Not `private`
/// the gamepad coverflow (`LibraryCoverflowView`) reuses it directly rather than re-fetching art.
struct PosterImage: View {
let candidates: [URL]
let title: String
let session: URLSession?
@State private var index = 0
@State private var image: PlatformImage?
var body: some View {
Group {
if let image {
Image(platformImage: image)
.resizable()
.scaledToFill()
} else if index < candidates.count {
ZStack { placeholder; ProgressView() }
} else {
placeholder
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.clipped()
.task(id: index) { await loadCurrent() }
}
private func loadCurrent() async {
guard index < candidates.count else { return }
guard let session, let data = try? await session.data(from: candidates[index]).0,
let loaded = PlatformImage(data: data)
else {
index += 1 // advance to the next candidate (or past the end placeholder)
return
}
image = loaded
}
private var placeholder: some View {
ZStack {
Rectangle().fill(.quaternary)
Text(title)
.font(.geist(17, .semibold, relativeTo: .headline))
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
.padding(8)
}
}
}
@@ -0,0 +1,95 @@
// Reusable library widgets, shared by the touch grid (LibraryView's `GameCard`) and the gamepad
// coverflow (LibraryCoverflowView's cover cell).
import PunktfunkKit
import SwiftUI
#if canImport(UIKit)
import UIKit
#elseif canImport(AppKit)
import AppKit
#endif
/// The store-provenance badge (Steam vs. a user-curated custom entry) overlaid on a poster
/// shared by the touch grid's `GameCard` and the gamepad coverflow's cover cell.
struct StoreBadge: View {
let isCustom: Bool
var body: some View {
Text(isCustom ? "Custom" : "Steam")
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 6)
.padding(.vertical, 3)
.background(.ultraThinMaterial, in: Capsule())
.padding(6)
}
}
#if canImport(UIKit)
private typealias PlatformImage = UIImage
#elseif canImport(AppKit)
private typealias PlatformImage = NSImage
#endif
private extension Image {
init(platformImage: PlatformImage) {
#if canImport(UIKit)
self.init(uiImage: platformImage)
#elseif canImport(AppKit)
self.init(nsImage: platformImage)
#endif
}
}
/// Sequentially tries cover-art URLs over `session` (so a paired client can reach the host's own
/// art proxy, not just public CDNs see `LibraryImageLoader`), advancing past any that fail to
/// load, then a placeholder. The loaded image is hard-clipped to fill the card's actual frame
/// regardless of its own aspect ratio: a portrait capsule fills it as intended, but a fallback
/// banner (wide hero/header art, used when a title has no portrait capsule) would otherwise report
/// a much wider intrinsic size than the card and overflow into neighboring cards. Not `private`
/// the gamepad coverflow (`LibraryCoverflowView`) reuses it directly rather than re-fetching art.
struct PosterImage: View {
let candidates: [URL]
let title: String
let session: URLSession?
@State private var index = 0
@State private var image: PlatformImage?
var body: some View {
Group {
if let image {
Image(platformImage: image)
.resizable()
.scaledToFill()
} else if index < candidates.count {
ZStack { placeholder; ProgressView() }
} else {
placeholder
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.clipped()
.task(id: index) { await loadCurrent() }
}
private func loadCurrent() async {
guard index < candidates.count else { return }
guard let session, let data = try? await session.data(from: candidates[index]).0,
let loaded = PlatformImage(data: data)
else {
index += 1 // advance to the next candidate (or past the end placeholder)
return
}
image = loaded
}
private var placeholder: some View {
ZStack {
Rectangle().fill(.quaternary)
Text(title)
.font(.geist(17, .semibold, relativeTo: .headline))
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
.padding(8)
}
}
}
@@ -170,7 +170,10 @@ private struct ShotHUD: View {
Text("5120×1440@240 240 fps 812.4 Mb/s")
.font(.system(.caption, design: .monospaced))
}
Text("capture→client 1.3/2.1 ms p50/p95")
Text("end-to-end 2.9 ms p50 · 3.8 p95 · capture→on-glass")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
Text("= host+network 1.3 + decode 0.7 + display 0.9")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
#if os(macOS)
@@ -0,0 +1,35 @@
// The HUD-corner model persisted by Settings and read wherever the overlay is placed
// (ContentView, StreamHUDView).
import SwiftUI
/// Which corner the HUD overlay occupies (persisted as `DefaultsKey.hudPlacement`). The raw
/// values are stable on disk rename the cases freely, never the strings.
enum HUDPlacement: String, CaseIterable, Identifiable {
case topLeading, topTrailing, bottomLeading, bottomTrailing
var id: String { rawValue }
/// SwiftUI overlay alignment for `.overlay(alignment:)`.
var alignment: Alignment {
switch self {
case .topLeading: return .topLeading
case .topTrailing: return .topTrailing
case .bottomLeading: return .bottomLeading
case .bottomTrailing: return .bottomTrailing
}
}
/// The HUD's own stack hugs the screen edge it sits against, so its text aligns outward.
var isTrailing: Bool { self == .topTrailing || self == .bottomTrailing }
/// User-facing corner label.
var label: String {
switch self {
case .topLeading: return "Top Left"
case .topTrailing: return "Top Right"
case .bottomLeading: return "Bottom Left"
case .bottomTrailing: return "Bottom Right"
}
}
}
@@ -59,29 +59,62 @@ final class SessionModel: ObservableObject {
@Published var fps = 0
@Published var mbps = 0.0
@Published var totalFrames = 0
/// Captureclient-receipt latency (ms), skew-corrected across machines via the connect-time
/// clock offset p50/p95 for the HUD. `latencyValid` is false until the first sample drains
/// (and whenever no host frames arrived in the last interval). `latencySkewCorrected` = the host
/// The unified latency stages (design/stats-unification.md), ms per 1 s window. `host+network`
/// = capturereceived, skew-corrected across machines via the connect-time clock offset: the
/// stage-2 HUD shows its p50 in the equation line; the stage-1 fallback shows p50/p95 as its
/// `capturereceived` headline. `hostNetworkValid` is false until the first sample drains (and
/// whenever no host frames arrived in the last interval). `hostNetworkSkewCorrected` = the host
/// answered the skew handshake (the number is cross-machine valid, not just same-host).
@Published var latencyP50Ms = 0.0
@Published var latencyP95Ms = 0.0
@Published var latencyValid = false
@Published var latencySkewCorrected = false
/// Capturepresent (glass-to-glass, modulo the host rendercapture term) only the stage-2
/// presenter can stamp this (it owns decode + a CAMetalLayer/display-link present). Stays
/// invalid under stage-1, where the layer presents internally with no per-frame callback.
@Published var presentLatencyP50Ms = 0.0
@Published var presentLatencyP95Ms = 0.0
@Published var presentLatencyValid = false
@Published var presentLatencySkewCorrected = false
@Published var hostNetworkP50Ms = 0.0
@Published var hostNetworkP95Ms = 0.0
@Published var hostNetworkValid = false
@Published var hostNetworkSkewCorrected = false
/// Phase 2 of the same stage: `host+network` split into its two terms via the host's per-AU
/// 0xCF timing reports (host = capturefully-sent as the host measured it, network = the
/// remainder), matched to receipts by pts in `latencySplit`. `splitValid` is false whenever
/// no timing matched in the window an old host that never emits the plane, or heavy 0xCF
/// loss and the HUD then falls back to the combined `host+network` term.
@Published var hostP50Ms = 0.0
@Published var networkP50Ms = 0.0
@Published var splitValid = false
/// End-to-end = captureon-glass, measured directly per frame (never summed from the stages)
/// the HUD headline. Only the stage-2 presenter can stamp it (it owns decode + a
/// CAMetalLayer/display-link present); stays invalid under stage-1, where the layer presents
/// internally with no per-frame callback.
@Published var endToEndP50Ms = 0.0
@Published var endToEndP95Ms = 0.0
@Published var endToEndValid = false
@Published var endToEndSkewCorrected = false
/// The client-local stage terms of the HUD's equation line (single clock, no skew; p50 only):
/// decode = receiveddecoded, display = decodedon-glass (ring wait + render + vsync the
/// term the stage-2 presenter exists to shorten).
@Published var decodeP50Ms = 0.0
@Published var decodeValid = false
@Published var displayP50Ms = 0.0
@Published var displayValid = false
/// Unrecoverable network frame drops in the last window (FEC couldn't rebuild them) and their
/// share of frames offered, `lost/(received+lost)`. The HUD hides the line while zero.
@Published var lostFrames = 0
@Published var lostPct = 0.0
/// Mirrors StreamView's capture state (it owns the input capture; this drives the
/// HUD's "click to capture" / " releases" hint).
@Published var mouseCaptured = false
let meter = FrameMeter()
/// Capturereceived (the host+network stage), fed per AU at receipt by the stream view's
/// onFrame under both presenters.
let latency = LatencyMeter()
/// Fed by the stage-2 presenter's display link (capturepresent). Passed to StreamView.
let presentLatency = LatencyMeter()
/// The host/network split of that same stage: onFrame also records (pts, interval) receipts
/// here, and the 1 s stats tick drains the connection's 0xCF host timings into it under
/// both presenters (the receipt path is presenter-independent).
let latencySplit = HostNetworkSplitter()
/// The stage-2 meters, passed to StreamView: end-to-end (captureon-glass, stamped at
/// present), decode (receiveddecoded), display (decodedon-glass).
let endToEnd = LatencyMeter()
let decodeStage = LatencyMeter()
let displayStage = LatencyMeter()
/// Cumulative reassembler-drop counter at the last stats drain (per-window `lost` delta).
private var lastFramesDropped: UInt64 = 0
private var statsTimer: Timer?
private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture?
@@ -108,6 +141,7 @@ final class SessionModel: ObservableObject {
bitrateKbps: UInt32 = 0,
audioChannels: UInt8 = 2,
hdrEnabled: Bool = true,
preferredCodec: UInt8 = 0,
launchID: String? = nil,
allowTofu: Bool = false,
autoTrust: Bool = false,
@@ -155,12 +189,17 @@ final class SessionModel: ObservableObject {
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
// This client's VideoToolbox path decodes H.264 and HEVC (AV1 isn't wired hosts don't
// emit it on the native path yet). The host resolves the emitted codec from these + the
// soft `preferredCodec`; `resolvedCodec` reflects what it chose.
let videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
audioChannels: audioChannels, launchID: launchID,
audioChannels: audioChannels,
videoCodecs: videoCodecs, preferredCodec: preferredCodec, launchID: launchID,
// Delegated approval: the host holds this connect open until the operator approves
// it (~180 s) outwait that window so a slow approval still lands here. Normal
// connects keep the snappy default.
@@ -268,7 +307,13 @@ final class SessionModel: ObservableObject {
phase = .idle
fps = 0
mbps = 0
latencyValid = false
hostNetworkValid = false
splitValid = false
endToEndValid = false
decodeValid = false
displayValid = false
lostFrames = 0
lostPct = 0
mouseCaptured = false
}
@@ -293,6 +338,7 @@ final class SessionModel: ObservableObject {
audio.start(
speakerUID: defaults.string(forKey: DefaultsKey.speakerUID) ?? "",
micUID: defaults.string(forKey: DefaultsKey.micUID) ?? "",
micChannel: defaults.integer(forKey: DefaultsKey.micChannel),
micEnabled: defaults.object(forKey: DefaultsKey.micEnabled) as? Bool ?? true)
self.audio = audio
// Gamepads: forward GamepadManager's active controller as pad 0 and render the
@@ -308,6 +354,8 @@ final class SessionModel: ObservableObject {
}
private func startStatsTimer() {
lastFramesDropped = 0 // a fresh connection's cumulative drop counter starts at 0
latencySplit.reset() // no stale receipts/samples from a previous session
let timer = Timer(timeInterval: 1.0, repeats: true) { [weak self] _ in
guard let self else { return }
Task { @MainActor in
@@ -315,21 +363,60 @@ final class SessionModel: ObservableObject {
self.fps = frames
self.mbps = Double(bytes) * 8 / 1_000_000
self.totalFrames = total
// Per-window `lost` = the delta of the connector's cumulative reassembler-drop
// counter (0 after close treat a rewind as no loss rather than underflowing).
let dropped = self.connection?.framesDropped() ?? 0
let lost = dropped >= self.lastFramesDropped
? Int(dropped - self.lastFramesDropped) : 0
self.lastFramesDropped = dropped
self.lostFrames = lost
self.lostPct = lost > 0 ? Double(lost) / Double(frames + lost) * 100 : 0
if let lat = self.latency.drain() {
self.latencyP50Ms = lat.p50Ms
self.latencyP95Ms = lat.p95Ms
self.latencySkewCorrected = lat.skewCorrected
self.latencyValid = true
self.hostNetworkP50Ms = lat.p50Ms
self.hostNetworkP95Ms = lat.p95Ms
self.hostNetworkSkewCorrected = lat.skewCorrected
self.hostNetworkValid = true
} else {
self.latencyValid = false
self.hostNetworkValid = false
}
if let p = self.presentLatency.drain() {
self.presentLatencyP50Ms = p.p50Ms
self.presentLatencyP95Ms = p.p95Ms
self.presentLatencySkewCorrected = p.skewCorrected
self.presentLatencyValid = true
// Phase 2: drain the window's per-AU host timings (0xCF) into the splitter
// non-blocking, bounded (a 240 fps window is ~240 reports; the cap only guards
// a pathological burst). `try?` flattens (SE-0230); a throw (.closed during
// teardown) just ends the drain. An old host never emits any splitValid stays
// false and the HUD keeps the combined host+network term.
if let conn = self.connection {
var burst = 0
while burst < 1024, let t = try? conn.nextHostTiming(timeoutMs: 0) {
self.latencySplit.noteHostTiming(ptsNs: t.ptsNs, hostUs: t.hostUs)
burst += 1
}
}
if let s = self.latencySplit.drain() {
self.hostP50Ms = s.hostP50Ms
self.networkP50Ms = s.networkP50Ms
self.splitValid = true
} else {
self.presentLatencyValid = false
self.splitValid = false
}
if let e = self.endToEnd.drain() {
self.endToEndP50Ms = e.p50Ms
self.endToEndP95Ms = e.p95Ms
self.endToEndSkewCorrected = e.skewCorrected
self.endToEndValid = true
} else {
self.endToEndValid = false
}
if let d = self.decodeStage.drain() {
self.decodeP50Ms = d.p50Ms
self.decodeValid = true
} else {
self.decodeValid = false
}
if let d = self.displayStage.drain() {
self.displayP50Ms = d.p50Ms
self.displayValid = true
} else {
self.displayValid = false
}
}
}
@@ -0,0 +1,102 @@
// The streaming overlay HUD: mode + fps/throughput, the unified latency lines
// (design/stats-unification.md end-to-end headline + the stage equation under stage-2, the
// capturereceived headline under the stage-1 fallback), the loss counter, the platform input
// hint, and disconnect.
import PunktfunkKit
import SwiftUI
struct StreamHUDView: View {
@ObservedObject var model: SessionModel
let connection: PunktfunkConnection
var placement: HUDPlacement = .topTrailing
var body: some View {
VStack(alignment: placement.isTrailing ? .trailing : .leading, spacing: 4) {
HStack(spacing: 6) {
Circle()
.fill(Color.accentColor)
.frame(width: 7, height: 7)
Text("\(connection.width)×\(connection.height)@\(connection.refreshHz) \(model.fps) fps \(model.mbps, specifier: "%.1f") Mb/s")
.font(.system(.caption, design: .monospaced))
}
if model.endToEndValid {
// Stage-2: the end-to-end headline (captureon-glass, measured directly, skew-
// corrected) "(same-host clock)" when the host didn't answer the skew handshake.
Text("end-to-end \(model.endToEndP50Ms, specifier: "%.1f") ms p50 · \(model.endToEndP95Ms, specifier: "%.1f") p95 · capture→on-glass\(model.endToEndSkewCorrected ? "" : " (same-host clock)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
// The equation: the stages tiling the headline interval (per-window p50s
// they only approximately sum to the directly-measured total). With a host
// that reports per-AU timings (0xCF) the first term splits into host + network
// (phase 2); an old host keeps the combined term.
if model.hostNetworkValid && model.decodeValid && model.displayValid {
if model.splitValid {
Text("= host \(model.hostP50Ms, specifier: "%.1f") + network \(model.networkP50Ms, specifier: "%.1f") + decode \(model.decodeP50Ms, specifier: "%.1f") + display \(model.displayP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
} else {
Text("= host+network \(model.hostNetworkP50Ms, specifier: "%.1f") + decode \(model.decodeP50Ms, specifier: "%.1f") + display \(model.displayP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
}
} else if model.hostNetworkValid {
// Stage-1 fallback presenter: the layer decodes + presents internally with no
// per-frame stamp, so the honest headline ends at receipt. The host/network
// split still applies there (receipt is presenter-independent) it becomes the
// only equation line; without it, host+network IS the whole measured interval.
Text("capture→received \(model.hostNetworkP50Ms, specifier: "%.1f") ms p50 · \(model.hostNetworkP95Ms, specifier: "%.1f") p95\(model.hostNetworkSkewCorrected ? "" : " (same-host clock)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
if model.splitValid {
Text("= host \(model.hostP50Ms, specifier: "%.1f") + network \(model.networkP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
}
if model.lostFrames > 0 {
// Unrecoverable network drops this window; hidden while the link is clean.
// String(format:) rather than specifier interpolation: the literal % would
// otherwise land in the LocalizedStringKey's format string as a bogus conversion.
Text(String(format: "lost %d (%.1f%%)", model.lostFrames, model.lostPct))
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
// While captured the cursor is hidden+frozen, so the button is keyboard-only
// ( or Cmd+Tab release the cursor; released, it's clickable again).
#if os(macOS)
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
: "Click the stream to capture input")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#elseif os(iOS)
// Touch always plays directly; (hardware keyboard) toggles kb/mouse.
Text(model.mouseCaptured
? "⌘⎋ releases keyboard & mouse"
: "⌘⎋ captures keyboard & mouse")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#endif
#if os(tvOS)
// No focusable control during play: a focusable button steals the controller's
// A press (the focus engine consumes it before the host sees it). Disconnect is
// the Siri Remote's Menu button (.onExitCommand on the stream) just hint it.
Text("Press Menu to disconnect")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
#else
// D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#endif
}
.padding(10)
// Floating HUD over live video the canonical Liquid-Glass overlay surface (26+);
// falls back to .regularMaterial below 26 (see GlassStyle).
.glassBackground(RoundedRectangle(cornerRadius: 10))
.padding(10)
}
}
@@ -255,6 +255,10 @@ struct ControllerTestView: View {
Toggle("Light motor (right)", isOn: $lightOn)
Label("Backend: \(tester.rumbleBackend)", systemImage: "waveform")
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
if let problem = tester.rumbleHealth {
Label(problem, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.orange)
}
Text("Toggle a motor to feel it. The host maps a game's low/high-frequency "
+ "rumble onto these two. A DualSense is driven over raw HID (CoreHaptics "
+ "can't reach its motors on macOS).")
@@ -0,0 +1,357 @@
// The gamepad-driven settings screen (iOS/iPadOS/macOS): the couch-relevant subset of SettingsView,
// restyled as a console settings page and fully navigable with a controller up/down moves the
// focus bar, left/right steps the focused value, A cycles/toggles it, B closes. Shown from the
// gamepad home launcher (X); the touch SettingsView remains the full-fidelity editor (custom
// resolutions, the log bitrate slider, debug tools), and both write the same DefaultsKey storage,
// so values round-trip freely between the two.
//
// Rows are rebuilt from live @AppStorage on every render; the focus list dispatches adjust/
// activate back here BY ROW ID (see `adjust`/`activate`), so a stored input callback can never act
// on stale captured state. Left/right CLAMPS at a choice list's ends (the dull boundary thud tells
// the thumb it's the last option); A always cycles forward, wrapping, so every option is reachable
// with one button. Toggles read left = off, right = on refusing a no-op with the same thud.
import PunktfunkKit
import SwiftUI
#if os(iOS) || os(macOS)
import GameController
struct GamepadSettingsView: View {
@Environment(\.dismiss) private var dismiss
@AppStorage(DefaultsKey.streamWidth) private var width = 1920
@AppStorage(DefaultsKey.streamHeight) private var height = 1080
@AppStorage(DefaultsKey.streamHz) private var hz = 60
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) private var enable444 = true
@AppStorage(DefaultsKey.codec) private var codec = "auto"
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
@AppStorage(DefaultsKey.gamepadUIEnabled) private var gamepadUIEnabled = true
@ObservedObject private var gamepads = GamepadManager.shared
#if os(iOS)
/// `.compact` in a landscape phone window tighter chrome so more rows fit.
@Environment(\.verticalSizeClass) private var vSizeClass
private var compact: Bool { vSizeClass == .compact }
#else
private let compact = false // no size classes on macOS; the sheet is sized generously
#endif
@State private var focusID: String?
var body: some View {
GamepadMenuList(
items: rows,
focusID: $focusID,
onAdjust: { row, delta in adjust(id: row.id, by: delta) },
onActivate: { activate(id: $0.id) },
onBack: { dismiss() }
) { row, focused in
rowView(row, focused: focused)
.frame(maxWidth: 620)
.padding(.horizontal, 24)
}
.frame(maxWidth: .infinity)
.safeAreaInset(edge: .top, spacing: 0) {
Text("Settings")
.font(.geist(compact ? 20 : 30, .bold, relativeTo: .title))
.foregroundStyle(.white)
.padding(.top, gamepadTitleTopPadding(compact: compact))
.padding(.bottom, compact ? 4 : 8)
.frame(maxWidth: .infinity)
.overlay(alignment: .trailing) { closeButton.padding(.trailing, 20) }
.background { GamepadTrayScrim(edge: .top) }
}
.safeAreaInset(edge: .bottom, alignment: .leading, spacing: 0) {
VStack(alignment: .leading, spacing: 8) {
Text(focusedDetail)
.font(.geist(13, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.55))
.lineLimit(2, reservesSpace: true)
.animation(.smooth(duration: 0.2), value: focusID)
GamepadHintBar(hints: [
.init(glyph: "arrow.left.and.right", text: "Adjust"),
.init(glyph: buttonGlyph(\.buttonA, fallback: "a.circle"), text: "Change"),
.init(glyph: buttonGlyph(\.buttonB, fallback: "b.circle"), text: "Done"),
])
}
.padding(.leading, 22)
.padding(.trailing, 22)
.padding(.vertical, compact ? 6 : 10)
.frame(maxWidth: .infinity, alignment: .leading)
.background { GamepadTrayScrim(edge: .bottom) }
}
.background { GamepadScreenBackground() }
.onAppear {
gamepads.refresh()
gamepads.startDiscovery()
}
.onDisappear { gamepads.stopDiscovery() }
}
/// Touch/click fallback for closing the controller path is B, a hardware keyboard's Esc
/// rides the cancel action.
private var closeButton: some View {
Button { dismiss() } label: {
Image(systemName: "xmark")
.font(.system(size: 14, weight: .semibold))
.foregroundStyle(.white)
.frame(width: 34, height: 34)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
.buttonStyle(.plain)
.keyboardShortcut(.cancelAction)
.accessibilityLabel("Close settings")
}
// MARK: - Row rendering
private func rowView(_ row: Row, focused: Bool) -> some View {
VStack(alignment: .leading, spacing: 6) {
if let header = row.header {
Text(header)
.font(.geist(12, .semibold, relativeTo: .caption))
.tracking(1.4)
.foregroundStyle(.white.opacity(0.45))
.padding(.leading, 16)
.padding(.top, 14)
}
HStack(spacing: 14) {
Image(systemName: row.icon)
.font(.system(size: 17))
.foregroundStyle(focused ? Color.brand : .white.opacity(0.55))
.frame(width: 28)
Text(row.label)
.font(.geist(16, .semibold, relativeTo: .body))
.foregroundStyle(.white)
.lineLimit(1)
Spacer(minLength: 12)
HStack(spacing: 9) {
Image(systemName: "chevron.left")
.font(.system(size: 12, weight: .semibold))
.foregroundStyle(.white.opacity(focused ? 0.6 : 0))
Text(row.value)
.font(.geist(15, .medium, relativeTo: .callout))
.foregroundStyle(focused ? .white : .white.opacity(0.6))
.lineLimit(1)
Image(systemName: "chevron.right")
.font(.system(size: 12, weight: .semibold))
.foregroundStyle(.white.opacity(focused ? 0.6 : 0))
}
}
.padding(.horizontal, 16)
.padding(.vertical, 13)
.background {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.fill(.white.opacity(focused ? 0.1 : 0))
}
.overlay {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.strokeBorder(.white.opacity(focused ? 0.22 : 0), lineWidth: 1)
}
.scaleEffect(focused ? 1.0 : 0.98)
.animation(.smooth(duration: 0.18), value: focused)
}
}
private var focusedDetail: String {
rows.first { $0.id == focusID }?.detail ?? " "
}
// MARK: - Row model
private struct Row: Identifiable {
let id: String
/// Section header drawn above this row (the first row of each group carries it).
var header: String?
let icon: String
let label: String
let value: String
/// One-line explanation shown near the hint bar while this row is focused.
let detail: String
/// Left/right step; returns whether the value actually changed (false boundary thud).
let adjust: (Int) -> Bool
/// A cycle forward (wrapping) / flip.
let activate: () -> Void
}
/// Dispatch by id so the focus list's stored input callbacks always act on freshly built rows
/// (never on state captured at wire time).
private func adjust(id: String, by delta: Int) -> Bool {
rows.first { $0.id == id }?.adjust(delta) ?? false
}
private func activate(id: String) {
rows.first { $0.id == id }?.activate()
}
private var rows: [Row] {
let resolution = resolutionOptions
let refresh = SettingsOptions.refreshRates(including: hz)
.map { (label: "\($0) Hz", tag: $0) }
let bitrate = SettingsOptions.bitrateOptions(current: bitrateKbps)
let controllers = SettingsOptions.controllerOptions(gamepads)
return [
choiceRow(
id: "resolution", header: "Stream", icon: "aspectratio",
label: "Resolution",
detail: "The host creates a virtual display at exactly this size — no scaling.",
options: resolution, current: "\(width)x\(height)"
) { tag in
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 2 else { return }
width = parts[0]
height = parts[1]
},
choiceRow(
id: "refresh", icon: "gauge.with.needle", label: "Refresh rate",
detail: "Rates this display can actually show.",
options: refresh, current: hz
) { hz = $0 },
choiceRow(
id: "bitrate", icon: "speedometer", label: "Bitrate",
detail: "Automatic uses the host's default (20 Mbps). "
+ "Run a speed test from the touch UI for an informed value.",
options: bitrate, current: bitrateKbps
) { bitrateKbps = $0 },
choiceRow(
id: "compositor", icon: "macwindow", label: "Compositor",
detail: "Which compositor drives the virtual output — honored only if "
+ "available on the host.",
options: SettingsOptions.compositors, current: compositor
) { compositor = $0 },
choiceRow(
id: "codec", header: "Video", icon: "film", label: "Video codec",
detail: "A preference — the host falls back if it can't encode this one "
+ "(10-bit and 4:4:4 are HEVC-only).",
options: SettingsOptions.codecs, current: codec
) { codec = $0 },
toggleRow(
id: "hdr", icon: "sun.max", label: "10-bit HDR",
detail: "HDR10 — engages when the host sends HDR content and this display "
+ "supports it.",
value: $hdrEnabled),
toggleRow(
id: "chroma", icon: "textformat", label: "Full chroma (4:4:4)",
detail: "Sharper text and UI at more bandwidth — needs host opt-in and "
+ "hardware decode.",
value: $enable444),
choiceRow(
id: "audio", header: "Audio", icon: "speaker.wave.2", label: "Audio channels",
detail: "The speaker layout requested from the host.",
options: SettingsOptions.audioChannels, current: audioChannels
) { audioChannels = $0 },
toggleRow(
id: "mic", icon: "mic", label: "Microphone",
detail: "Send this device's microphone to the host's virtual mic.",
value: $micEnabled),
choiceRow(
id: "pad", header: "Controller", icon: "gamecontroller", label: "Use controller",
detail: "Which pad is forwarded to the host, as player 1.",
options: controllers, current: gamepads.preferredID
) { gamepads.preferredID = $0 },
choiceRow(
id: "padType", icon: "dpad", label: "Controller type",
detail: "The virtual pad the host creates — Automatic matches this controller.",
options: SettingsOptions.padTypes, current: gamepadType
) { gamepadType = $0 },
toggleRow(
id: "hud", header: "Interface", icon: "chart.bar", label: "Statistics overlay",
detail: "Resolution, frame rate, throughput and latency while streaming.",
value: $hudEnabled),
choiceRow(
id: "hudPlacement", icon: "rectangle.inset.topright.filled", label: "Overlay position",
detail: "Which corner the statistics overlay sits in.",
options: SettingsOptions.hudPlacements, current: hudPlacement
) { hudPlacement = $0 },
toggleRow(
id: "library", icon: "square.grid.2x2", label: "Game library",
detail: "Browse and launch the host's games with \(buttonName(\.buttonY, "Y")) "
+ "(experimental).",
value: $libraryEnabled),
toggleRow(
id: "gamepadUI", icon: "hand.tap", label: "Controller-optimized UI",
detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled),
]
}
/// Resolution choices as "WxH" tags the current size is inserted when it's a custom mode
/// (set via the touch settings), so cycling starts from it instead of jumping.
private var resolutionOptions: [(label: String, tag: String)] {
var options = SettingsOptions.resolutionModes()
.map { (label: "\($0.name) · \($0.w) × \($0.h)", tag: "\($0.w)x\($0.h)") }
let current = "\(width)x\(height)"
if !options.contains(where: { $0.tag == current }) {
options.insert((label: "Custom · \(width) × \(height)", tag: current), at: 0)
}
return options
}
/// The active controller's user-facing name for a button (for detail strings).
private func buttonName(
_ button: KeyPath<GCExtendedGamepad, GCControllerButtonInput>, _ fallback: String
) -> String {
gamepads.active?.controller.extendedGamepad?[keyPath: button].localizedName ?? fallback
}
// MARK: - Row builders
private func choiceRow<T: Equatable>(
id: String, header: String? = nil, icon: String, label: String, detail: String,
options: [(label: String, tag: T)], current: T, write: @escaping (T) -> Void
) -> Row {
let index = options.firstIndex { $0.tag == current }
return Row(
id: id, header: header, icon: icon, label: label,
value: index.map { options[$0].label } ?? "",
detail: detail,
adjust: { delta in
// Unknown current value: snap to the first option on any step.
guard let index else {
guard let first = options.first else { return false }
write(first.tag)
return true
}
let target = index + delta
guard target >= 0, target < options.count else { return false }
write(options[target].tag)
return true
},
activate: {
guard let index else { return write(options.first?.tag ?? current) }
write(options[(index + 1) % options.count].tag)
})
}
private func toggleRow(
id: String, header: String? = nil, icon: String, label: String, detail: String,
value: Binding<Bool>
) -> Row {
Row(
id: id, header: header, icon: icon, label: label,
value: value.wrappedValue ? "On" : "Off",
detail: detail,
adjust: { delta in
// Directional semantics: left = off, right = on; a no-op reads as a boundary.
let target = delta > 0
guard value.wrappedValue != target else { return false }
value.wrappedValue = target
return true
},
activate: { value.wrappedValue.toggle() })
}
}
#endif
@@ -0,0 +1,60 @@
// SettingsView's navigation and presentation helpers: the iOS settings categories, the iPad
// sheet sizing, and the bounded-slider clamp.
import SwiftUI
#if os(iOS)
/// The settings groups, mirroring the macOS preference tabs. On iPad each is a sidebar row that
/// drives the detail pane; on iPhone the same list collapses to pushed sub-pages. Internal (not
/// private) so the screenshot harness can open SettingsView on a specific category.
enum SettingsCategory: String, CaseIterable, Identifiable {
case general, display, audio, controllers, advanced, about
var id: Self { self }
var title: String {
switch self {
case .general: return "General"
case .display: return "Display"
case .audio: return "Audio"
case .controllers: return "Controllers"
case .advanced: return "Advanced"
case .about: return "About"
}
}
var symbol: String {
switch self {
case .general: return "gearshape"
case .display: return "display"
case .audio: return "speaker.wave.2"
case .controllers: return "gamecontroller"
case .advanced: return "slider.horizontal.3"
case .about: return "info.circle"
}
}
}
extension View {
/// Present the settings sheet large on iPad so the NavigationSplitView has room for its
/// sidebar + detail a default form sheet is too narrow and the split view would collapse to
/// the iPhone push list. No-op on iPhone (the standard sheet is already right) and on iOS 17
/// (no `presentationSizing` it falls back to the default sheet, which still degrades cleanly
/// to the push list).
@ViewBuilder
func settingsSheetSizing() -> some View {
if UIDevice.current.userInterfaceIdiom == .pad, #available(iOS 18, *) {
presentationSizing(.page)
} else {
self
}
}
}
#endif
extension Double {
/// The log-scale slider mapping needs a bounded input (Automatic stores 0).
func clamped(_ lo: Double, _ hi: Double) -> Double {
Swift.min(Swift.max(self, lo), hi)
}
}
@@ -0,0 +1,147 @@
// The option lists every settings surface renders from one source of truth shared by the
// touch/desktop SettingsView (Pickers), the tvOS pushed selection rows, and the gamepad settings
// screen (GamepadSettingsView's left/right cycling). Pure data + small pure helpers; anything that
// reads live view state (e.g. the bitrate slider mapping) stays on SettingsView.
#if os(macOS)
import AppKit
#endif
import PunktfunkKit
import SwiftUI
enum SettingsOptions {
/// Compositor choices the `tag` is the wire value (`PunktfunkConnection.Compositor` raw).
static let compositors: [(label: String, tag: Int)] = [
("Automatic", 0),
("KWin (KDE Plasma)", 1),
("wlroots (Sway / Hyprland)", 2),
("Mutter (GNOME)", 3),
("gamescope", 4),
]
static let audioChannels: [(label: String, tag: Int)] = [
("Stereo", 2),
("5.1 Surround", 6),
("7.1 Surround", 8),
]
/// Virtual-pad types the `tag` is the wire value (`PunktfunkConnection.GamepadType` raw).
static let padTypes: [(label: String, tag: Int)] = [
("Automatic", 0),
("Xbox 360", 1),
("Xbox One", 3),
("DualSense", 2),
("DualShock 4", 4),
]
static let hudPlacements: [(label: String, tag: String)] =
HUDPlacement.allCases.map { ($0.label, $0.rawValue) }
/// Video-codec preference (`DefaultsKey.codec`) a soft preference the host falls back from.
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (hosts don't emit AV1 on
/// the native path yet).
static let codecs: [(label: String, tag: String)] = [
("Automatic", "auto"),
("HEVC (H.265)", "hevc"),
("H.264 (AVC)", "h264"),
]
// MARK: - Bitrate
/// Discrete bitrate steps for the surfaces with no Slider (tvOS pushed pickers, the gamepad
/// settings' left/right cycling), up to the same 3 Gbps ceiling the slider has.
static let bitratePresets: [(label: String, tag: Int)] = [
("Automatic", 0),
("10 Mbps", 10_000),
("20 Mbps", 20_000),
("40 Mbps", 40_000),
("80 Mbps", 80_000),
("150 Mbps", 150_000),
("300 Mbps", 300_000),
("500 Mbps", 500_000),
("1 Gbps", 1_000_000),
("1.5 Gbps", 1_500_000),
("2 Gbps", 2_000_000),
("3 Gbps", 3_000_000),
]
/// The presets plus the currently stored value when it isn't one of them (set via the touch
/// slider or a synced device) so the current choice stays visible/selectable.
static func bitrateOptions(current: Int) -> [(label: String, tag: Int)] {
var options = bitratePresets
if !options.contains(where: { $0.tag == current }) {
options.insert(
(SpeedTestSheet.mbpsLabel(kbps: current) + " (custom)", current), at: 1)
}
return options
}
// MARK: - Controllers
/// "Use controller" choices: Automatic, every forwardable controller, and so a stale pin
/// stays visible instead of leaving the selection tag-less any pinned id that is NOT among
/// the selectable (extended) entries, present-but-unusable included.
@MainActor
static func controllerOptions(_ gamepads: GamepadManager) -> [(label: String, tag: String)] {
let selectable = gamepads.controllers.filter(\.isExtended)
var options: [(label: String, tag: String)] = [("Automatic", "")]
options += selectable.map { ($0.name, $0.id) }
if !gamepads.preferredID.isEmpty,
!selectable.contains(where: { $0.id == gamepads.preferredID }) {
options.append(("Unavailable controller", gamepads.preferredID))
}
return options
}
#if os(iOS) || os(macOS)
// MARK: - Stream mode (iOS + macOS pickers; tvOS builds its own preset list)
/// 16:9 then ultrawide presets; the device's native mode is prepended by `resolutionModes`.
static let resolutionPresets: [(name: String, w: Int, h: Int)] = [
("720p", 1280, 720),
("1080p", 1920, 1080),
("1440p", 2560, 1440),
("4K", 3840, 2160),
("Ultrawide 1080p", 2560, 1080),
("Ultrawide 1440p", 3440, 1440),
("Super ultrawide", 5120, 1440),
]
/// This device's native mode first, then the presets, deduped by dimensions (native wins a
/// tie).
@MainActor
static func resolutionModes() -> [(name: String, w: Int, h: Int)] {
var native: [(name: String, w: Int, h: Int)] = []
#if os(iOS)
let bounds = UIScreen.main.nativeBounds // portrait-oriented pixels
native = [("This device",
Int(max(bounds.width, bounds.height)),
Int(min(bounds.width, bounds.height)))]
#else
if let screen = NSScreen.main {
let scale = screen.backingScaleFactor
native = [("This display",
Int(screen.frame.width * scale),
Int(screen.frame.height * scale))]
}
#endif
var seen = Set<String>()
return (native + resolutionPresets).filter { seen.insert("\($0.w)x\($0.h)").inserted }
}
/// Refresh rates the device can actually display (no point asking the host to render frames
/// the screen can't show), plus any stored custom value so it stays selectable.
@MainActor
static func refreshRates(including current: Int) -> [Int] {
#if os(iOS)
let maxHz = UIScreen.main.maximumFramesPerSecond
#else
let maxHz = NSScreen.main?.maximumFramesPerSecond ?? 60
#endif
var rates = [60, 120, 240].filter { $0 <= maxHz }
if rates.isEmpty { rates = [maxHz] }
if !rates.contains(current) { rates.append(current) }
return rates.sorted()
}
#endif
}
@@ -0,0 +1,435 @@
// SettingsView's shared sections each setting's Section is defined exactly once here and
// composed by the per-platform bodies in SettingsView.swift.
import PunktfunkKit
import SwiftUI
extension SettingsView {
// MARK: - Sections (shared)
// NOTE: the Section content is deliberately split into the small named builders below as one
// inline expression the iOS branch (wheel + 3-way refresh + bitrate rows) blew Swift's
// type-checker budget ("unable to type-check this expression in reasonable time"), which
// failed exactly one slice: the iOS archive (macOS/tvOS never compile that branch).
@ViewBuilder var streamModeSection: some View {
Section {
#if os(iOS)
iosResolutionWheel
iosRefreshRows
Button("Use this display's mode") { fillFromMainScreen() }
#elseif os(macOS)
HStack {
TextField("Resolution", value: $width, format: .number.grouping(.never))
Text("×")
TextField("", value: $height, format: .number.grouping(.never))
.labelsHidden()
}
TextField("Refresh rate (Hz)", value: $hz, format: .number.grouping(.never))
LabeledContent("") {
Button("Use this display's mode") { fillFromMainScreen() }
}
#endif
#if !os(tvOS)
bitrateRows
#endif
} header: {
Text("Stream mode")
} footer: {
Text("The host creates a virtual output at exactly this mode — "
+ "native resolution, no scaling. \(Self.bitrateFooter)")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
// MARK: - Stream mode (iOS wheel)
/// Touch-first: a rotating wheel of common resolutions (this device's own mode first) the
/// same family as the Clock/Timer pickers. The host renders a virtual output at exactly the
/// chosen mode, so these are real pixel sizes. The last wheel row, "Custom", reveals
/// width/height/refresh fields for an arbitrary mode (see `iosRefreshRows`).
@ViewBuilder private var iosResolutionWheel: some View {
VStack(alignment: .leading, spacing: 4) {
Text("Resolution")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Resolution", selection: resolutionSelection) {
ForEach(resolutionChoices, id: \.tag) { choice in
Text(choice.label).tag(choice.tag)
}
}
.labelsHidden()
.pickerStyle(.wheel)
.frame(maxHeight: 140)
}
}
/// Custom W×H(+Hz) fields, a segmented refresh picker, or a static single-rate row.
@ViewBuilder private var iosRefreshRows: some View {
if isCustomResolution {
// Arbitrary entry: type the exact width × height (and refresh) the host should drive.
HStack {
TextField("Width", value: $width, format: .number.grouping(.never))
.keyboardType(.numberPad)
Text("×")
TextField("Height", value: $height, format: .number.grouping(.never))
.labelsHidden()
.keyboardType(.numberPad)
}
// A row built from an HStack of TextFields otherwise insets its bottom separator to
// the inner content, clipping the hairline under "Width"; pin it to the cell edge.
.alignmentGuide(.listRowSeparatorLeading) { _ in 0 }
LabeledContent("Refresh rate") {
TextField("Hz", value: $hz, format: .number.grouping(.never))
.keyboardType(.numberPad)
.multilineTextAlignment(.trailing)
}
} else if refreshChoices.count > 1 {
VStack(alignment: .leading, spacing: 6) {
Text("Refresh rate")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Refresh rate", selection: $hz) {
ForEach(refreshChoices, id: \.self) { rate in
Text("\(rate) Hz").tag(rate)
}
}
.labelsHidden()
.pickerStyle(.segmented)
}
} else {
// A device with a single supported rate (e.g. 60 Hz) has nothing to pick.
LabeledContent("Refresh rate") {
Text("\(hz) Hz").foregroundStyle(.secondary)
}
}
}
/// Sentinel wheel tag for the "Custom" row. Real tags are "WxH" (digits + "x"), so this can't
/// collide with a resolution.
private static let customResolutionTag = "custom"
/// Wheel rows: the resolution modes (device native first see `SettingsOptions`), then a
/// "Custom" row that reveals the numeric fields.
private var resolutionChoices: [(label: String, tag: String)] {
SettingsOptions.resolutionModes()
.map { (label: "\($0.name) · \($0.w) × \($0.h)", tag: "\($0.w)x\($0.h)") }
+ [(label: "Custom…", tag: Self.customResolutionTag)]
}
private var presetResolutionTags: Set<String> {
Set(SettingsOptions.resolutionModes().map { "\($0.w)x\($0.h)" })
}
/// True when the editable custom fields should show: the wheel is parked on "Custom" (sticky),
/// or the stored size simply isn't one of the presets (e.g. a value synced from a Mac) so a
/// non-preset mode stays editable across relaunches without a persisted flag.
private var isCustomResolution: Bool {
customMode || !presetResolutionTags.contains("\(width)x\(height)")
}
/// The wheel works in "WxH" tags so one selection drives both width and height; the custom
/// sentinel toggles `customMode` instead of writing a size.
private var resolutionSelection: Binding<String> {
Binding(
get: { isCustomResolution ? Self.customResolutionTag : "\(width)x\(height)" },
set: { tag in
if tag == Self.customResolutionTag {
customMode = true
return
}
customMode = false
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 2 else { return }
width = parts[0]
height = parts[1]
})
}
/// Refresh rates this device can display, plus any stored custom value (see `SettingsOptions`).
private var refreshChoices: [Int] {
SettingsOptions.refreshRates(including: hz)
}
#endif
#if !os(tvOS)
/// The automatic-bitrate toggle + manual slider (and the >1 Gbps warning) rows.
@ViewBuilder private var bitrateRows: some View {
Toggle("Automatic bitrate", isOn: automaticBitrate)
if bitrateKbps != 0 {
HStack(spacing: 12) {
Slider(value: bitrateSlider, in: 0...1) {
Text("Bitrate")
}
Text(SpeedTestSheet.mbpsLabel(kbps: bitrateKbps))
.monospacedDigit()
.foregroundStyle(.secondary)
.frame(minWidth: 76, alignment: .trailing)
}
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
}
}
}
#endif
@ViewBuilder var audioSection: some View {
Section {
Picker("Audio channels", selection: $audioChannels) {
ForEach(SettingsOptions.audioChannels, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
#if os(macOS)
Picker("Speaker", selection: $speakerUID) {
Text("System default").tag("")
ForEach(outputDevices) { device in
Text(device.name).tag(device.uid)
}
if !speakerUID.isEmpty,
!outputDevices.contains(where: { $0.uid == speakerUID }) {
Text("Unavailable device").tag(speakerUID)
}
}
#endif
Toggle("Send microphone to the host", isOn: $micEnabled)
#if os(macOS)
Picker("Microphone", selection: $micUID) {
Text("System default").tag("")
ForEach(inputDevices) { device in
Text(device.name).tag(device.uid)
}
if !micUID.isEmpty,
!inputDevices.contains(where: { $0.uid == micUID }) {
Text("Unavailable device").tag(micUID)
}
}
.disabled(!micEnabled)
// Multi-channel interfaces only: the mic sits on ONE discrete input, so let the user
// pick it. Auto sums every channel (a lone hot mic still passes at full level).
if micChannelCount > 1 {
Picker("Microphone channel", selection: $micChannel) {
Text("Auto (all channels)").tag(0)
ForEach(1...micChannelCount, id: \.self) { ch in
Text("Channel \(ch)").tag(ch)
}
}
.disabled(!micEnabled)
}
#endif
} header: {
Text("Audio")
} footer: {
Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
/// Touch-input model (iPhone + iPad) plus the iPad-only pointer-capture toggle: lock the
/// mouse/trackpad for relative movement (games) vs forward an absolute cursor position.
@ViewBuilder var pointerSection: some View {
Section {
Picker("Touch input", selection: $touchMode) {
Text("Trackpad").tag(TouchInputMode.trackpad.rawValue)
Text("Direct pointer").tag(TouchInputMode.pointer.rawValue)
Text("Touch passthrough").tag(TouchInputMode.touch.rawValue)
}
if UIDevice.current.userInterfaceIdiom == .pad {
Toggle("Capture pointer for games", isOn: $pointerCapture)
}
} header: {
Text("Touch & pointer")
} footer: {
Text(pointerFooterText)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
/// Footer copy for `pointerSection`, built in plain `+=` statements. Deliberately NOT one big
/// `+` chain (with a ternary) inside the ViewBuilder that single expression blew Swift's
/// type-checker budget and was what actually broke the iOS archive.
private var pointerFooterText: String {
var text = "Trackpad: your finger nudges the host cursor like a laptop touchpad — tap to "
text += "click, two-finger tap for a right click, two-finger drag to scroll, "
text += "tap-then-drag to hold the button, three-finger tap for the stats overlay. "
text += "Direct pointer: the cursor jumps to your finger. Touch passthrough: real "
text += "multi-touch reaches the host, for apps that understand touch. Applies from "
text += "the next touch."
if UIDevice.current.userInterfaceIdiom == .pad {
text += " Pointer capture locks a hardware mouse/trackpad for relative movement "
text += "(mouse-look); off keeps the pointer free and sends absolute positions. "
text += "The lock needs the stream full-screen and frontmost, and falls back "
text += "automatically (Stage Manager, Slide Over)."
}
return text
}
#endif
@ViewBuilder var compositorSection: some View {
Section {
Picker("Compositor", selection: $compositor) {
ForEach(SettingsOptions.compositors, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
} header: {
Text("Host compositor")
} footer: {
Text("Which compositor drives the virtual output on the host. A specific "
+ "choice is honored only if that backend is available there — "
+ "otherwise the host falls back to auto-detection.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var windowSection: some View {
#if os(macOS)
Section {
Toggle("Fullscreen while streaming", isOn: $fullscreenWhileStreaming)
} header: {
Text("Window")
} footer: {
Text("Take the window fullscreen when a session starts and restore it on the host "
+ "list, so only the stream is fullscreen — not the picker.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
// the controller test. Empty in release builds (no presenter UI; stage-2 always).
@ViewBuilder var presenterSection: some View {
#if DEBUG
Section {
Picker("Presenter", selection: $presenter) {
Text("Stage 2 (default)").tag("stage2")
Text("Stage 1 (debug)").tag("stage1")
}
} header: {
Text("Video presenter · debug")
} footer: {
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it gives the HUD the end-to-end (capture→on-glass) headline with the "
+ "host+network/decode/display stage equation and self-recovers from decode "
+ "stalls. Stage 1 feeds compressed video straight to the system display layer; "
+ "it freezes on a lost HEVC reference frame, so it's a debug fallback only. "
+ "Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder var hdrSection: some View {
Section {
Picker("Video codec", selection: $codec) {
ForEach(SettingsOptions.codecs, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
Toggle("10-bit HDR", isOn: $hdrEnabled)
Toggle("Full chroma (4:4:4)", isOn: $enable444)
} header: {
Text("Video quality")
} footer: {
Text("Codec is a preference — the host falls back if it can't encode the one you pick "
+ "(and 10-bit/4:4:4 are HEVC-only). HDR requests a 10-bit BT.2020 PQ (HDR10) stream — "
+ "it only engages when the host is sending HDR content AND this display supports HDR. "
+ "4:4:4 requests full chroma (sharper text/UI, more bandwidth) — it only engages when "
+ "this device can hardware-decode it AND the host opted in. Otherwise the stream stays "
+ "8-bit 4:2:0 SDR. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var statisticsSection: some View {
Section {
Toggle("Show statistics overlay", isOn: $hudEnabled)
Picker("Position", selection: $hudPlacement) {
ForEach(HUDPlacement.allCases) { placement in
Text(placement.label).tag(placement.rawValue)
}
}
.disabled(!hudEnabled)
} header: {
Text("Statistics")
} footer: {
Text(Self.statisticsFooter)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var experimentalSection: some View {
Section {
Toggle("Show game library", isOn: $libraryEnabled)
} header: {
Text("Experimental")
} footer: {
Text("Adds a “Browse Library…” action to each host that lists its games "
+ "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "Works once you've paired with the host — the library is authorized by this "
+ "device's certificate, with no extra host setup.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@ViewBuilder var controllersSection: some View {
Section {
if gamepads.controllers.isEmpty {
Text("No controllers detected")
.foregroundStyle(.secondary)
} else {
ForEach(gamepads.controllers) { controller in
controllerRow(controller)
}
}
Picker("Use controller", selection: $gamepads.preferredID) {
ForEach(controllerOptions, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
Picker("Controller type", selection: $gamepadType) {
ForEach(SettingsOptions.padTypes, id: \.tag) { option in
Text(option.label).tag(option.tag)
}
}
#if !os(tvOS)
Toggle("Gamepad-optimized browsing", isOn: $gamepadUIEnabled)
#endif
#if DEBUG && !os(tvOS)
Button("Test Controller…") { showControllerTest = true }
.disabled(gamepads.active == nil)
.sheet(isPresented: $showControllerTest) { ControllerTestView() }
#endif
} header: {
Text("Controllers")
} footer: {
// The gamepad-UI blurb is appended here, not merged into the shared
// `controllersFooter` constant tvOS's `tvBody` reuses that exact string (line ~348)
// for its own footer and has no such toggle to describe.
VStack(alignment: .leading, spacing: 6) {
Text(Self.controllersFooter)
#if !os(tvOS)
Text(Self.gamepadUIFooter)
#endif
}
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
}
@@ -0,0 +1,153 @@
// SettingsView's footers and stateful helpers, used by both the section builders
// (SettingsView+Sections.swift) and the per-platform bodies (SettingsView.swift). The option
// LISTS live in SettingsOptions they're shared with the gamepad settings screen too.
#if os(macOS)
import AppKit
#endif
import PunktfunkKit
import SwiftUI
extension SettingsView {
// MARK: - Bitrate
/// Slider domain, log-scale: the useful range spans three orders of magnitude
/// (a few Mbps 3 Gbps) linear would cram everything below 100 Mbps into the
/// first pixels.
private static let minSliderKbps = 2_000.0
private static let maxSliderKbps = 3_000_000.0
static let bitrateFooter =
"Automatic uses the host's default bitrate (20 Mbps); the host clamps any choice "
+ "to its supported range. Run a speed test from a host card's context menu to "
+ "pick an informed value. Applies from the next session."
static let gigabitWarning =
"Above 1 Gbps — test the network speed first (a host card's context menu → "
+ "Test Network Speed…). A bitrate beyond what the link sustains causes loss "
+ "and stutter."
/// `bitrateKbps == 0` is Automatic; switching to manual lands on the host default.
var automaticBitrate: Binding<Bool> {
Binding(
get: { bitrateKbps == 0 },
set: { bitrateKbps = $0 ? 0 : 20_000 })
}
/// Slider position 0...1 kbps on the log scale, snapped to two significant figures
/// so the readout shows round numbers instead of 47_322.
var bitrateSlider: Binding<Double> {
Binding(
get: {
let v = Double(bitrateKbps).clamped(Self.minSliderKbps, Self.maxSliderKbps)
return log(v / Self.minSliderKbps)
/ log(Self.maxSliderKbps / Self.minSliderKbps)
},
set: { pos in
let raw = Self.minSliderKbps
* pow(Self.maxSliderKbps / Self.minSliderKbps, pos)
let mag = pow(10, floor(log10(raw)) - 1)
bitrateKbps = Int((raw / mag).rounded() * mag)
})
}
// MARK: - Statistics
static var statisticsFooter: String {
let base = "The overlay shows resolution, frame rate, throughput and latency while "
+ "streaming, in the chosen corner."
#if os(macOS) || os(iOS)
return base + " Toggle it any time with ⌘⇧S."
#else
return base
#endif
}
// MARK: - Controllers
static let controllersFooter =
"One controller is forwarded to the host, as player 1 — Automatic picks the most "
+ "recently connected one. The type is the virtual pad the host creates: Automatic "
+ "matches the controller (a DualSense gets adaptive triggers, lightbar, touchpad "
+ "and motion; a DualShock 4 the same minus adaptive triggers), and changes apply "
+ "from the next session. Two identical controllers may swap a manual selection "
+ "after reconnecting."
#if !os(tvOS)
static let gamepadUIFooter =
"When a controller is connected, the host list and game library switch to a "
+ "controller-friendly layout — larger focus targets, controller-navigable settings, "
+ "and a swipeable cover browser for the library. Turn this off to always use the "
+ "standard layout. (The system may still move basic focus with a controller "
+ "connected even with this off — that's outside the app's control.)"
#endif
/// "Use controller" choices for this view's manager (see `SettingsOptions.controllerOptions`).
var controllerOptions: [(label: String, tag: String)] {
SettingsOptions.controllerOptions(gamepads)
}
func controllerRow(_ controller: GamepadManager.DiscoveredController) -> some View {
HStack(spacing: 10) {
Image(systemName: controller.hasTouchpadAndMotion ? "playstation.logo" : "gamecontroller.fill")
.foregroundStyle(.secondary)
VStack(alignment: .leading, spacing: 2) {
Text(controller.name)
HStack(spacing: 8) {
if !controller.isExtended {
Text(controller.productCategory)
}
if controller.hasAdaptiveTriggers {
Image(systemName: "r2.button.roundedtop.horizontal")
}
if controller.hasLight {
Image(systemName: "lightbulb.fill")
}
if controller.hasMotion {
Image(systemName: "gyroscope")
}
if controller.hasHaptics {
Image(systemName: "waveform")
}
if let level = controller.batteryLevel {
Text("\(Int(level * 100))%")
if controller.isCharging {
Image(systemName: "bolt.fill")
}
}
}
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
}
Spacer()
if gamepads.active?.id == controller.id {
Text("In use")
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 8)
.padding(.vertical, 3)
.background(Capsule().fill(.green.opacity(0.2)))
.foregroundStyle(.green)
}
}
}
func fillFromMainScreen() {
#if os(macOS)
guard let screen = NSScreen.main else { return }
let scale = screen.backingScaleFactor
width = Int(screen.frame.width * scale)
height = Int(screen.frame.height * scale)
hz = screen.maximumFramesPerSecond
#else
// nativeBounds is portrait-oriented pixels streams are landscape.
let bounds = UIScreen.main.nativeBounds
width = Int(max(bounds.width, bounds.height))
height = Int(min(bounds.width, bounds.height))
hz = UIScreen.main.maximumFramesPerSecond
#if os(iOS)
// The native mode is the "This device" wheel row, so leave Custom mode if it was on.
customMode = false
#endif
#endif
}
}
@@ -0,0 +1,380 @@
// App settings. The host creates a native virtual output at exactly the chosen size/refresh
// there is no scaling anywhere in the pipeline.
//
// Navigation differs per platform, but all three group the same categories (General, Display,
// Audio, Controllers, Advanced, About): macOS uses a tabbed preferences window; iOS/iPadOS uses
// an adaptive NavigationSplitView a category sidebar + detail pane on iPad, auto-collapsing to
// a hierarchical push list on iPhone (the system Settings idiom on each); tvOS uses a
// focus-native pushed-picker layout. The individual sections (`streamModeSection`,
// `audioSection`, ) are shared across all three so a setting is defined exactly once they
// live in SettingsView+Sections.swift, with their helpers in SettingsView+Support.swift.
#if os(macOS)
import AppKit
#endif
import PunktfunkKit
import SwiftUI
@MainActor
struct SettingsView: View {
@Environment(\.dismiss) private var dismiss
@AppStorage(DefaultsKey.streamWidth) var width = 1920
@AppStorage(DefaultsKey.streamHeight) var height = 1080
@AppStorage(DefaultsKey.streamHz) var hz = 60
@AppStorage(DefaultsKey.compositor) var compositor = 0
@AppStorage(DefaultsKey.gamepadType) var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) var presenter = "stage2"
@AppStorage(DefaultsKey.hdrEnabled) var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) var libraryEnabled = false
@AppStorage(DefaultsKey.fullscreenWhileStreaming) var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) var micEnabled = true
@AppStorage(DefaultsKey.audioChannels) var audioChannels = 2
@AppStorage(DefaultsKey.codec) var codec = "auto"
@AppStorage(DefaultsKey.hudEnabled) var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) var hudPlacement = HUDPlacement.topTrailing.rawValue
@ObservedObject var gamepads = GamepadManager.shared
#if !os(tvOS)
@AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true
#endif
#if DEBUG && !os(tvOS)
@State var showControllerTest = false
#endif
#if os(iOS)
@AppStorage(DefaultsKey.pointerCapture) var pointerCapture = true
@AppStorage(DefaultsKey.touchMode) var touchMode = TouchInputMode.trackpad.rawValue
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
@State private var settingsSelection: SettingsCategory?
// Tracked so the detail can show its own Done whenever the sidebar (and its Done) is off screen
// not just on iPhone, but on any iPad layout that collapses the sidebar to an overlay. Starts
// .doubleColumn so iPad reliably opens with the sidebar (and its Done) visible.
@State private var columnVisibility: NavigationSplitViewVisibility = .doubleColumn
// Sticky once the wheel lands on "Custom", so editing a width/height that briefly equals a
// preset doesn't snap the wheel back off Custom. A stored non-preset value reads as custom even
// when this is false (see `isCustomResolution`), so it survives relaunches without persisting.
@State var customMode = false
#endif
#if os(macOS)
@AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0
@State var outputDevices: [AudioDevice] = []
@State var inputDevices: [AudioDevice] = []
// Input channels of the selected mic drives the "Microphone channel" picker, which only
// appears for a multi-channel interface (>1). 0 until the Audio tab loads it.
@State var micChannelCount = 0
#endif
#if os(iOS)
/// `initialCategory` is nil in the app (the list opens un-selected on iPhone; iPad lands on
/// General via `onAppear`). The screenshot harness passes an explicit category so the captured
/// shot opens on a real settings page (a populated detail) rather than the bare category list.
init(initialCategory: SettingsCategory? = nil) {
_settingsSelection = State(initialValue: initialCategory)
}
#endif
var body: some View {
#if os(tvOS)
// Native tv pattern: no inline text entry (typing numbers with a remote is
// miserable and the inline field chrome fights the focus system). Modes are
// preset pickers that push selection lists like the system Settings app.
tvBody
#elseif os(macOS)
macBody
#else
iosBody
#endif
}
// MARK: - macOS: tabbed preferences
#if os(macOS)
private var macBody: some View {
TabView {
Form {
streamModeSection
compositorSection
}
.formStyle(.grouped)
.tabItem { Label("General", systemImage: "gearshape") }
Form {
presenterSection
hdrSection
windowSection
statisticsSection
}
.formStyle(.grouped)
.tabItem { Label("Display", systemImage: "display") }
Form {
audioSection
}
.formStyle(.grouped)
.onAppear {
outputDevices = AudioDevices.outputs()
inputDevices = AudioDevices.inputs()
micChannelCount = AudioDevices.inputChannelCount(forUID: micUID)
}
.onChange(of: micUID) { _, newUID in
// A different mic different channel count; drop a now-out-of-range pin to Auto.
micChannelCount = AudioDevices.inputChannelCount(forUID: newUID)
if micChannel > micChannelCount { micChannel = 0 }
}
.tabItem { Label("Audio", systemImage: "speaker.wave.2") }
Form {
controllersSection
}
.formStyle(.grouped)
.onAppear {
gamepads.refresh()
gamepads.startDiscovery()
}
.onDisappear { gamepads.stopDiscovery() }
.tabItem { Label("Controllers", systemImage: "gamecontroller") }
Form {
experimentalSection
}
.formStyle(.grouped)
.tabItem { Label("Advanced", systemImage: "slider.horizontal.3") }
AcknowledgementsView()
.tabItem { Label("About", systemImage: "info.circle") }
}
.frame(width: 480, height: 460)
}
#endif
// MARK: - iOS / iPadOS: adaptive split view
#if os(iOS)
private var iosBody: some View {
NavigationSplitView(columnVisibility: $columnVisibility) {
List(selection: $settingsSelection) {
ForEach(SettingsCategory.allCases) { category in
// On iPhone the split view collapses to a push list, but a selection List
// draws no disclosure indicator of its own add one in compact width for the
// expected drill-in affordance. On iPad the selected row highlights instead, so
// the chevron is omitted there.
HStack {
Label(category.title, systemImage: category.symbol)
if horizontalSizeClass == .compact {
Spacer()
Image(systemName: "chevron.forward")
.font(.footnote.weight(.semibold))
.foregroundStyle(.tertiary)
// Purely a drill-in affordance the row's button trait already
// conveys "opens"; keep it out of the VoiceOver announcement.
.accessibilityHidden(true)
}
}
.tag(category)
}
}
.navigationTitle("Settings")
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
} detail: {
// NavigationSplitView hosts the detail in its own navigation context (its title bar),
// so no inner NavigationStack that would double the bar on iPad. On iPhone the split
// view collapses to one stack and pushes this when a row is tapped. `?? .general` only
// backs the brief pre-selection window; the list never auto-pushes on a nil selection.
settingsDetail(settingsSelection ?? .general)
// Keep a Done on the detail whenever the sidebar (and its Done) isn't on screen: the
// iPhone push, or any iPad layout that collapsed the sidebar to an overlay. When the
// sidebar is showing, its Done is the only one so this stays hidden to avoid two.
.toolbar {
if horizontalSizeClass == .compact || columnVisibility == .detailOnly {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
}
}
.onAppear {
if horizontalSizeClass == .regular, settingsSelection == nil {
settingsSelection = .general
}
gamepads.refresh()
gamepads.startDiscovery()
}
// A regularregular launch sets the default above; this catches a compactregular change
// (e.g. an iPad leaving narrow split-screen multitasking) so the detail pane fills in.
.onChange(of: horizontalSizeClass) { _, newValue in
if newValue == .regular, settingsSelection == nil {
settingsSelection = .general
}
}
.onDisappear { gamepads.stopDiscovery() }
}
@ViewBuilder
private func settingsDetail(_ category: SettingsCategory) -> some View {
switch category {
case .general:
Form {
streamModeSection
pointerSection
compositorSection
}
.formStyle(.grouped)
.navigationTitle("General")
.navigationBarTitleDisplayMode(.inline)
case .display:
Form {
presenterSection
hdrSection
statisticsSection
}
.formStyle(.grouped)
.navigationTitle("Display")
.navigationBarTitleDisplayMode(.inline)
case .audio:
Form { audioSection }
.formStyle(.grouped)
.navigationTitle("Audio")
.navigationBarTitleDisplayMode(.inline)
case .controllers:
Form { controllersSection }
.formStyle(.grouped)
.navigationTitle("Controllers")
.navigationBarTitleDisplayMode(.inline)
case .advanced:
Form { experimentalSection }
.formStyle(.grouped)
.navigationTitle("Advanced")
.navigationBarTitleDisplayMode(.inline)
case .about:
// Already a full scrollable view that sets its own "Acknowledgements" title; pin the
// display mode inline to match the five sibling detail pages (it would otherwise inherit
// the large title from the "Settings" sidebar root).
AcknowledgementsView()
.navigationBarTitleDisplayMode(.inline)
}
}
#endif
// MARK: - tvOS
#if os(tvOS)
private static let presets: [(label: String, tag: String)] = [
("720p @ 60", "1280x720x60"),
("1080p @ 60", "1920x1080x60"),
("4K @ 60", "3840x2160x60"),
]
private var modeTag: Binding<String> {
Binding(
get: { "\(width)x\(height)x\(hz)" },
set: { tag in
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 3 else { return }
width = parts[0]
height = parts[1]
hz = parts[2]
})
}
private var hudEnabledTag: Binding<String> {
Binding(get: { hudEnabled ? "on" : "off" }, set: { hudEnabled = $0 == "on" })
}
private var hdrEnabledTag: Binding<String> {
Binding(get: { hdrEnabled ? "on" : "off" }, set: { hdrEnabled = $0 == "on" })
}
private var tvBody: some View {
let currentTag = "\(width)x\(height)x\(hz)"
let bounds = UIScreen.main.nativeBounds
let nativeTag = "\(Int(max(bounds.width, bounds.height)))x"
+ "\(Int(min(bounds.width, bounds.height)))x\(UIScreen.main.maximumFramesPerSecond)"
var options = Self.presets
if !options.contains(where: { $0.tag == nativeTag }) {
options.insert(("This TV (native)", nativeTag), at: 0)
}
if !options.contains(where: { $0.tag == currentTag }) {
options.insert(("Custom (\(width)×\(height) @ \(hz))", currentTag), at: 0)
}
return ScrollView {
VStack(spacing: 16) {
TVSelectionRow(title: "Stream mode", options: options, selection: modeTag)
TVSelectionRow(
title: "Bitrate",
options: SettingsOptions.bitrateOptions(current: bitrateKbps),
selection: $bitrateKbps)
TVSelectionRow(
title: "Audio channels",
options: SettingsOptions.audioChannels,
selection: $audioChannels)
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
.multilineTextAlignment(.center)
}
TVSelectionRow(
title: "Compositor", options: SettingsOptions.compositors,
selection: $compositor)
#if DEBUG
TVSelectionRow(
title: "Presenter (debug)",
options: [("Stage 2 (default)", "stage2"), ("Stage 1 (debug)", "stage1")],
selection: $presenter)
#endif
TVSelectionRow(
title: "10-bit HDR",
options: [("On", "on"), ("Off", "off")], selection: hdrEnabledTag)
Text("The host creates a virtual output at exactly this mode — native "
+ "resolution, no scaling. \(Self.bitrateFooter) A specific compositor "
+ "is honored only if available on the host.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
TVSelectionRow(
title: "Statistics overlay",
options: [("On", "on"), ("Off", "off")], selection: hudEnabledTag)
TVSelectionRow(
title: "Statistics position", options: SettingsOptions.hudPlacements,
selection: $hudPlacement)
ForEach(gamepads.controllers) { controller in
controllerRow(controller)
.padding(.horizontal, 24)
}
TVSelectionRow(
title: "Use controller", options: controllerOptions,
selection: $gamepads.preferredID)
TVSelectionRow(
title: "Controller type", options: SettingsOptions.padTypes,
selection: $gamepadType)
Text(Self.controllersFooter)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
NavigationLink("Acknowledgements") { AcknowledgementsView() }
.padding(.top, 8)
}
.frame(maxWidth: 1000)
.frame(maxWidth: .infinity)
.padding(60)
}
.navigationTitle("Settings")
.onAppear {
gamepads.refresh()
gamepads.startDiscovery()
}
.onDisappear { gamepads.stopDiscovery() }
}
#endif
}
File diff suppressed because it is too large Load Diff
@@ -46,9 +46,24 @@ extension StoredHost {
}
}
private extension Data {
/// Lowercase hex, no separators to compare a pinned fingerprint against the mDNS `fp`.
var hexLower: String { map { String(format: "%02x", $0) }.joined() }
/// The two joins of live mDNS discovery against the saved-host store, shared by the touch grid
/// (HomeView) and the gamepad launcher (GamepadHomeView) so both screens classify hosts the same
/// way. LAN-scoped like the underlying match: a host that isn't advertising here is "not seen",
/// not proven off.
extension HostDiscovery {
/// A saved host is "online" iff a live advert currently matches it (see `StoredHost.matches`).
/// Recomputed on every discovery change (the @Published set), so it tracks hosts
/// appearing/leaving the network live.
func advertises(_ host: StoredHost) -> Bool {
hosts.contains { host.matches($0) }
}
/// Discovered hosts not already saved the saved list shows the rest, so this only surfaces
/// genuinely-new hosts on the network. Same match as `advertises`, so a saved host whose IP
/// changed (still fingerprint-matched) doesn't also appear as a stranger.
func unsaved(among saved: [StoredHost]) -> [DiscoveredHost] {
hosts.filter { d in !saved.contains { $0.matches(d) } }
}
}
@MainActor
@@ -1,107 +0,0 @@
// The streaming overlay HUD: mode + fps/throughput, the captureclient (and, under the stage-2
// presenter, capturepresent) latency lines, the platform input hint, and disconnect.
import PunktfunkKit
import SwiftUI
/// Which corner the HUD overlay occupies (persisted as `DefaultsKey.hudPlacement`). The raw
/// values are stable on disk rename the cases freely, never the strings.
enum HUDPlacement: String, CaseIterable, Identifiable {
case topLeading, topTrailing, bottomLeading, bottomTrailing
var id: String { rawValue }
/// SwiftUI overlay alignment for `.overlay(alignment:)`.
var alignment: Alignment {
switch self {
case .topLeading: return .topLeading
case .topTrailing: return .topTrailing
case .bottomLeading: return .bottomLeading
case .bottomTrailing: return .bottomTrailing
}
}
/// The HUD's own stack hugs the screen edge it sits against, so its text aligns outward.
var isTrailing: Bool { self == .topTrailing || self == .bottomTrailing }
/// User-facing corner label.
var label: String {
switch self {
case .topLeading: return "Top Left"
case .topTrailing: return "Top Right"
case .bottomLeading: return "Bottom Left"
case .bottomTrailing: return "Bottom Right"
}
}
}
struct StreamHUDView: View {
@ObservedObject var model: SessionModel
let connection: PunktfunkConnection
var placement: HUDPlacement = .topTrailing
var body: some View {
VStack(alignment: placement.isTrailing ? .trailing : .leading, spacing: 4) {
HStack(spacing: 6) {
Circle()
.fill(Color.accentColor)
.frame(width: 7, height: 7)
Text("\(connection.width)×\(connection.height)@\(connection.refreshHz) \(model.fps) fps \(model.mbps, specifier: "%.1f") Mb/s")
.font(.system(.caption, design: .monospaced))
}
if model.latencyValid {
// Captureclient-receipt (skew-corrected); excludes the layer's decode+present
// see LatencyMeter. "(same-host)" when the host didn't answer the skew handshake.
Text("capture→client \(model.latencyP50Ms, specifier: "%.1f")/\(model.latencyP95Ms, specifier: "%.1f") ms p50/p95\(model.latencySkewCorrected ? "" : " (same-host)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
if model.presentLatencyValid {
// Capturepresent (glass-to-glass, modulo host rendercapture) stage-2 presenter
// only; stage-1's layer presents internally with no per-frame stamp.
Text("capture→present \(model.presentLatencyP50Ms, specifier: "%.1f")/\(model.presentLatencyP95Ms, specifier: "%.1f") ms p50/p95\(model.presentLatencySkewCorrected ? "" : " (same-host)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
// While captured the cursor is hidden+frozen, so the button is keyboard-only
// ( or Cmd+Tab release the cursor; released, it's clickable again).
#if os(macOS)
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
: "Click the stream to capture input")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
// The client-side cursor (C) draws the local cursor over the stream instead of
// capturing it the only accurate cursor for gamescope, whose capture has none.
Text("⌘⇧C toggles the on-screen cursor")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#elseif os(iOS)
// Touch always plays directly; (hardware keyboard) toggles kb/mouse.
Text(model.mouseCaptured
? "⌘⎋ releases keyboard & mouse"
: "⌘⎋ captures keyboard & mouse")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#endif
#if os(tvOS)
// No focusable control during play: a focusable button steals the controller's
// A press (the focus engine consumes it before the host sees it). Disconnect is
// the Siri Remote's Menu button (.onExitCommand on the stream) just hint it.
Text("Press Menu to disconnect")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
#else
// D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#endif
}
.padding(10)
// Floating HUD over live video the canonical Liquid-Glass overlay surface (26+);
// falls back to .regularMaterial below 26 (see GlassStyle).
.glassBackground(RoundedRectangle(cornerRadius: 10))
.padding(10)
}
}
@@ -0,0 +1,27 @@
// Hex encode/decode for the trust surface pinned certificate fingerprints and the mDNS `fp`
// TXT value travel as lowercase hex.
import Foundation
extension Data {
/// Lowercase hex, no separators to compare a pinned fingerprint against the mDNS `fp`.
var hexLower: String { map { String(format: "%02x", $0) }.joined() }
/// Parse an even-length hex string into bytes; nil on any non-hex character or odd length.
/// Used to turn an mDNS-advertised cert fingerprint into a connect pin.
init?(hexString: String) {
let chars = Array(hexString)
guard chars.count.isMultiple(of: 2) else { return nil }
var bytes = [UInt8]()
bytes.reserveCapacity(chars.count / 2)
var i = 0
while i < chars.count {
guard let hi = chars[i].hexDigitValue, let lo = chars[i + 1].hexDigitValue else {
return nil
}
bytes.append(UInt8(hi << 4 | lo))
i += 2
}
self = Data(bytes)
}
}
@@ -70,7 +70,7 @@ struct TrustCardView: View {
/// 64 hex chars four groups per line, two lines easy to eyeball against the log.
private static func format(fingerprint: Data) -> String {
let hex = fingerprint.map { String(format: "%02x", $0) }.joined()
let hex = fingerprint.hexLower
let groups = stride(from: 0, to: hex.count, by: 8).map { i -> String in
let start = hex.index(hex.startIndex, offsetBy: i)
let end = hex.index(start, offsetBy: min(8, hex.count - i))
@@ -1,146 +0,0 @@
// Annex-B HEVC CoreMedia plumbing.
//
// The punktfunk host emits Annex-B access units with in-band VPS/SPS/PPS on every IDR
// (deliberately the client needs no out-of-band extradata). VideoToolbox wants the AVCC
// flavor instead: a CMVideoFormatDescription built from the parameter sets, and sample
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two.
//
// SCAFFOLD: written on the Linux host, not yet compiled against Xcode.
import CoreMedia
import Foundation
public enum AnnexB {
/// Split an Annex-B stream into NAL units (start codes 00 00 01 / 00 00 00 01 stripped).
/// All zeros immediately preceding a start code are dropped: they're either the
/// 4-byte-code prefix or `trailing_zero_8bits` padding, never NAL payload (emulation
/// prevention keeps 00 00 0x out of conforming NAL bytes) same policy as ffmpeg.
public static func nalUnits(in data: Data) -> [Data] {
var nals: [Data] = []
let bytes = [UInt8](data)
var i = 0
var start = -1
while i + 2 < bytes.count {
if bytes[i] == 0, bytes[i + 1] == 0, bytes[i + 2] == 1 {
var codeStart = i
while codeStart > 0, bytes[codeStart - 1] == 0 {
codeStart -= 1
}
if start >= 0, start < codeStart {
nals.append(Data(bytes[start..<codeStart]))
}
start = i + 3
i += 3
} else {
i += 1
}
}
if start >= 0, start < bytes.count {
nals.append(Data(bytes[start...]))
}
return nals
}
/// HEVC NAL unit type (bits 1..6 of the first byte).
public static func hevcNalType(_ nal: Data) -> UInt8 {
guard let first = nal.first else { return 0xFF }
return (first >> 1) & 0x3F
}
/// Build a format description from an IDR AU's in-band VPS(32)/SPS(33)/PPS(34).
/// Returns nil when the AU carries no parameter sets (non-IDR).
public static func formatDescription(fromIDR au: Data) -> CMVideoFormatDescription? {
var vps: Data?, sps: Data?, pps: Data?
for nal in nalUnits(in: au) {
switch hevcNalType(nal) {
case 32: vps = nal
case 33: sps = nal
case 34: pps = nal
default: break
}
}
guard let vps, let sps, let pps else { return nil }
var format: CMVideoFormatDescription?
let sets = [vps, sps, pps]
let status: OSStatus = sets[0].withUnsafeBytes { v in
sets[1].withUnsafeBytes { s in
sets[2].withUnsafeBytes { p in
let pointers: [UnsafePointer<UInt8>] = [
v.bindMemory(to: UInt8.self).baseAddress!,
s.bindMemory(to: UInt8.self).baseAddress!,
p.bindMemory(to: UInt8.self).baseAddress!,
]
let sizes = [vps.count, sps.count, pps.count]
return CMVideoFormatDescriptionCreateFromHEVCParameterSets(
allocator: kCFAllocatorDefault,
parameterSetCount: 3,
parameterSetPointers: pointers,
parameterSetSizes: sizes,
nalUnitHeaderLength: 4,
extensions: nil,
formatDescriptionOut: &format)
}
}
}
return status == noErr ? format : nil
}
/// Re-pack an Annex-B AU as AVCC (4-byte big-endian length before each NAL), dropping
/// the parameter-set NALs (they live in the format description).
public static func avcc(from au: Data) -> Data {
var out = Data(capacity: au.count + 16)
for nal in nalUnits(in: au) {
let t = hevcNalType(nal)
if t == 32 || t == 33 || t == 34 { continue } // VPS/SPS/PPS
var len = UInt32(nal.count).bigEndian
withUnsafeBytes(of: &len) { out.append(contentsOf: $0) }
out.append(nal)
}
return out
}
/// Wrap one AU as a decode-ready CMSampleBuffer.
public static func sampleBuffer(
au: AccessUnit, format: CMVideoFormatDescription
) -> CMSampleBuffer? {
let avccData = avcc(from: au.data)
var blockBuffer: CMBlockBuffer?
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault, memoryBlock: nil,
blockLength: avccData.count, blockAllocator: kCFAllocatorDefault,
customBlockSource: nil, offsetToData: 0, dataLength: avccData.count,
flags: 0, blockBufferOut: &blockBuffer) == noErr,
let block = blockBuffer
else { return nil }
let copied = avccData.withUnsafeBytes { raw in
CMBlockBufferReplaceDataBytes(
with: raw.baseAddress!, blockBuffer: block,
offsetIntoDestination: 0, dataLength: avccData.count)
}
guard copied == noErr else { return nil }
var timing = CMSampleTimingInfo(
duration: .invalid,
presentationTimeStamp: CMTime(value: Int64(au.ptsNs), timescale: 1_000_000_000),
decodeTimeStamp: .invalid)
var sampleSize = avccData.count
var sample: CMSampleBuffer?
guard CMSampleBufferCreate(
allocator: kCFAllocatorDefault, dataBuffer: block, dataReady: true,
makeDataReadyCallback: nil, refcon: nil, formatDescription: format,
sampleCount: 1, sampleTimingEntryCount: 1, sampleTimingArray: &timing,
sampleSizeEntryCount: 1, sampleSizeArray: &sampleSize,
sampleBufferOut: &sample) == noErr
else { return nil }
// Low-latency display: render on arrival, don't wait for a clock.
if let attachments = CMSampleBufferGetSampleAttachmentsArray(sample!, createIfNecessary: true) {
let dict = unsafeBitCast(CFArrayGetValueAtIndex(attachments, 0), to: CFMutableDictionary.self)
CFDictionarySetValue(
dict,
Unmanaged.passUnretained(kCMSampleAttachmentKey_DisplayImmediately).toOpaque(),
Unmanaged.passUnretained(kCFBooleanTrue).toOpaque())
}
return sample
}
}
@@ -33,6 +33,49 @@ public enum AudioDevices {
}
}
/// Input channel count of the mic the picker would use the device with this UID, or the
/// system default input when `uid` is empty. 0 when it can't be resolved. Drives the
/// "Microphone channel" picker (only shown for multi-channel interfaces).
public static func inputChannelCount(forUID uid: String) -> Int {
let id = uid.isEmpty ? defaultInputDevice() : deviceID(forUID: uid)
guard let id else { return 0 }
return channelCount(id, scope: kAudioObjectPropertyScopeInput)
}
private static func defaultInputDevice() -> AudioDeviceID? {
var address = AudioObjectPropertyAddress(
mSelector: kAudioHardwarePropertyDefaultInputDevice,
mScope: kAudioObjectPropertyScopeGlobal,
mElement: kAudioObjectPropertyElementMain)
var dev = AudioDeviceID(0)
var size = UInt32(MemoryLayout<AudioDeviceID>.size)
guard AudioObjectGetPropertyData(
AudioObjectID(kAudioObjectSystemObject), &address, 0, nil, &size, &dev) == noErr,
dev != 0
else { return nil }
return dev
}
/// Sum of channels across the device's streams in `scope` (its total input/output channels).
private static func channelCount(
_ id: AudioDeviceID, scope: AudioObjectPropertyScope
) -> Int {
var address = AudioObjectPropertyAddress(
mSelector: kAudioDevicePropertyStreamConfiguration,
mScope: scope,
mElement: kAudioObjectPropertyElementMain)
var size: UInt32 = 0
guard AudioObjectGetPropertyDataSize(id, &address, 0, nil, &size) == noErr, size > 0
else { return 0 }
let raw = UnsafeMutableRawPointer.allocate(
byteCount: Int(size), alignment: MemoryLayout<AudioBufferList>.alignment)
defer { raw.deallocate() }
guard AudioObjectGetPropertyData(id, &address, 0, nil, &size, raw) == noErr else { return 0 }
let abl = UnsafeMutableAudioBufferListPointer(
raw.assumingMemoryBound(to: AudioBufferList.self))
return abl.reduce(0) { $0 + Int($1.mNumberChannels) }
}
private static func all() -> [AudioDeviceID] {
var address = AudioObjectPropertyAddress(
mSelector: kAudioHardwarePropertyDevices,
@@ -62,7 +105,8 @@ public enum AudioDevices {
return AudioObjectGetPropertyDataSize(id, &address, 0, nil, &size) == noErr && size > 0
}
private static func describe(_ id: AudioDeviceID) -> AudioDevice? {
/// UID + human name for a live AudioDeviceID (nil if either property is unreadable).
static func describe(_ id: AudioDeviceID) -> AudioDevice? {
guard let uid = stringProperty(id, kAudioDevicePropertyDeviceUID),
let name = stringProperty(id, kAudioObjectPropertyName)
else { return nil }
@@ -0,0 +1,129 @@
import AVFoundation
import os
/// SPSC-ish jitter ring (interleaved float, `channels` per frame), drain thread render
/// callback. The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// reads return silence until enough is buffered (at least `prefill`, and at least one
/// packet more than the device's render quantum large-buffer devices would otherwise
/// chronically out-demand the prefill and oscillate prime dropout re-prime), and an
/// underrun re-primes, concealing jitter as one short dip instead of sustained crackle.
/// All counts stay whole frames (multiples of `channels`), so the interleave can never slip.
final class AudioRing: @unchecked Sendable {
private var buf: [Float]
private var readIdx = 0
private var writeIdx = 0
private var primed = false
private var renderQuantum = 0
private let prefill: Int
private let highWater: Int
private let channels: Int
private let lock = OSAllocatedUnfairLock()
/// `capacity`/`prefill` in samples (interleaved `channels` per frame, both whole frames).
init(capacity: Int, prefill: Int, channels: Int) {
buf = [Float](repeating: 0, count: capacity)
self.prefill = prefill
self.channels = channels
highWater = prefill * 4
}
func write(_ samples: UnsafePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
let capacity = buf.count
// A single write larger than the whole ring would push readIdx PAST writeIdx below
// (inverting the valid range corruption). It never happens (one decoded packet is far
// under capacity), but guard rather than corrupt.
guard count <= capacity else { return }
if writeIdx + count - readIdx > capacity {
readIdx = writeIdx + count - capacity // overflow: drop oldest
}
for i in 0..<count {
buf[(writeIdx + i) % capacity] = samples[i]
}
writeIdx += count
// Latency clamp: both ends run at 48 kHz, so backlog from a network stall (or
// creeping host-vs-DAC clock skew) never drains on its own without this, one
// 300 ms hiccup leaves audio 300 ms behind video for the rest of the session.
// Shedding down to 2× prefill costs one audible blip instead.
if writeIdx - readIdx > highWater {
readIdx = writeIdx - prefill * 2
}
}
/// Fills `out` completely (silence beyond what's buffered).
func read(into out: UnsafeMutablePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
renderQuantum = max(renderQuantum, count)
let available = writeIdx - readIdx
if !primed {
// One 5 ms host packet (240 frames × channels) of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 240 * channels) {
primed = true
} else {
for i in 0..<count { out[i] = 0 }
return
}
}
let n = min(available, count)
let capacity = buf.count
for i in 0..<n {
out[i] = buf[(readIdx + i) % capacity]
}
readIdx += n
if n < count {
for i in n..<count { out[i] = 0 }
primed = false // underrun re-prime before resuming
}
}
}
/// CoreAudio channel layout for the canonical wire order FL FR FC LFE RL RR [SL SR]. nil for
/// stereo (the standard layout is correct). For 5.1/7.1 we list explicit channel labels via
/// `kAudioChannelLayoutTag_UseChannelDescriptions` preset tags (DTS_5_1 etc.) don't reliably
/// match Moonlight's order. NB the 7.1 mapping (verified against the WASAPI 0x63F + SPA orderings):
/// wire idx 4-5 = RL/RR = the WAVE *back* pair LeftSurround/RightSurround; idx 6-7 = SL/SR = the
/// WAVE *side* pair LeftSurroundDirect/RightSurroundDirect. (Using RearSurround* for 6-7 would
/// swap side/back vs the Windows/Linux clients.)
func wireChannelLayout(channels: Int) -> AVAudioChannelLayout? {
let labels: [AudioChannelLabel]
switch channels {
case 6:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen, kAudioChannelLabel_LeftSurround,
kAudioChannelLabel_RightSurround,
]
case 8:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen,
kAudioChannelLabel_LeftSurround, kAudioChannelLabel_RightSurround, // wire RL/RR (back)
kAudioChannelLabel_LeftSurroundDirect, kAudioChannelLabel_RightSurroundDirect, // wire SL/SR (side)
]
default:
return nil
}
let size = MemoryLayout<AudioChannelLayout>.size
+ (labels.count - 1) * MemoryLayout<AudioChannelDescription>.stride
let raw = UnsafeMutableRawPointer.allocate(byteCount: size, alignment: 16)
defer { raw.deallocate() }
let layout = raw.bindMemory(to: AudioChannelLayout.self, capacity: 1)
layout.pointee.mChannelLayoutTag = kAudioChannelLayoutTag_UseChannelDescriptions
layout.pointee.mChannelBitmap = AudioChannelBitmap(rawValue: 0)
layout.pointee.mNumberChannelDescriptions = UInt32(labels.count)
// `mChannelDescriptions` is the C variable-length tail array (declared `[1]`, over-allocated
// above). Scope the pointer with `withUnsafeMutablePointer` taking `&mChannelDescriptions`
// inline yields a pointer valid only for that expression, so building a buffer from it that
// outlives the call is a dangling-pointer bug. Inside the closure it stays valid while we fill it.
withUnsafeMutablePointer(to: &layout.pointee.mChannelDescriptions) { tail in
let descs = UnsafeMutableBufferPointer(start: tail, count: labels.count)
for (i, lbl) in labels.enumerated() {
descs[i] = AudioChannelDescription(
mChannelLabel: lbl, mChannelFlags: AudioChannelFlags(rawValue: 0),
mCoordinates: (0, 0, 0))
}
}
return AVAudioChannelLayout(layout: layout)
}
@@ -5,9 +5,10 @@
// AVAudioSourceNode pulls from the ring (silence on underrun with re-priming, so a
// network gap costs one dip, not permanent crackle).
//
// mic host: a second AVAudioEngine taps the input device, resamples to 48 kHz
// stereo, slices 20 ms chunks, Opus-encodes, and sendMic()s each packet the host
// feeds them into a virtual PipeWire source.
// mic host: a second AVAudioEngine taps the input device, folds it to one mono bus (the
// chosen channel of a multi-channel interface, or a sum of all channels), resamples to 48 kHz
// stereo, slices 20 ms chunks, Opus-encodes, and sendMic()s each packet the host feeds them
// into a virtual PipeWire source.
//
// Devices are chosen by UID ("" = system default: the engine is then never pinned to a
// concrete device and follows default-device changes). Two engines, not one a single
@@ -19,99 +20,6 @@ import os
private let log = Logger(subsystem: "io.unom.punktfunk", category: "audio")
/// SPSC-ish jitter ring (interleaved float, `channels` per frame), drain thread render
/// callback. The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// reads return silence until enough is buffered (at least `prefill`, and at least one
/// packet more than the device's render quantum large-buffer devices would otherwise
/// chronically out-demand the prefill and oscillate prime dropout re-prime), and an
/// underrun re-primes, concealing jitter as one short dip instead of sustained crackle.
/// All counts stay whole frames (multiples of `channels`), so the interleave can never slip.
final class AudioRing: @unchecked Sendable {
private var buf: [Float]
private var readIdx = 0
private var writeIdx = 0
private var primed = false
private var renderQuantum = 0
private let prefill: Int
private let highWater: Int
private let channels: Int
private let lock = OSAllocatedUnfairLock()
/// `capacity`/`prefill` in samples (interleaved `channels` per frame, both whole frames).
init(capacity: Int, prefill: Int, channels: Int) {
buf = [Float](repeating: 0, count: capacity)
self.prefill = prefill
self.channels = channels
highWater = prefill * 4
}
func write(_ samples: UnsafePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
let capacity = buf.count
// A single write larger than the whole ring would push readIdx PAST writeIdx below
// (inverting the valid range corruption). It never happens (one decoded packet is far
// under capacity), but guard rather than corrupt.
guard count <= capacity else { return }
if writeIdx + count - readIdx > capacity {
readIdx = writeIdx + count - capacity // overflow: drop oldest
}
for i in 0..<count {
buf[(writeIdx + i) % capacity] = samples[i]
}
writeIdx += count
// Latency clamp: both ends run at 48 kHz, so backlog from a network stall (or
// creeping host-vs-DAC clock skew) never drains on its own without this, one
// 300 ms hiccup leaves audio 300 ms behind video for the rest of the session.
// Shedding down to 2× prefill costs one audible blip instead.
if writeIdx - readIdx > highWater {
readIdx = writeIdx - prefill * 2
}
}
/// Fills `out` completely (silence beyond what's buffered).
func read(into out: UnsafeMutablePointer<Float>, count: Int) {
lock.lock()
defer { lock.unlock() }
renderQuantum = max(renderQuantum, count)
let available = writeIdx - readIdx
if !primed {
// One 5 ms host packet (240 frames × channels) of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 240 * channels) {
primed = true
} else {
for i in 0..<count { out[i] = 0 }
return
}
}
let n = min(available, count)
let capacity = buf.count
for i in 0..<n {
out[i] = buf[(readIdx + i) % capacity]
}
readIdx += n
if n < count {
for i in n..<count { out[i] = 0 }
primed = false // underrun re-prime before resuming
}
}
}
private final class StopFlag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
}
/// Render-block-owned scratch storage: freed exactly when the closure (and thus the
/// last possible render call) is released never racing CoreAudio.
private final class ScratchBuffer {
@@ -120,55 +28,6 @@ private final class ScratchBuffer {
deinit { ptr.deallocate() }
}
/// CoreAudio channel layout for the canonical wire order FL FR FC LFE RL RR [SL SR]. nil for
/// stereo (the standard layout is correct). For 5.1/7.1 we list explicit channel labels via
/// `kAudioChannelLayoutTag_UseChannelDescriptions` preset tags (DTS_5_1 etc.) don't reliably
/// match Moonlight's order. NB the 7.1 mapping (verified against the WASAPI 0x63F + SPA orderings):
/// wire idx 4-5 = RL/RR = the WAVE *back* pair LeftSurround/RightSurround; idx 6-7 = SL/SR = the
/// WAVE *side* pair LeftSurroundDirect/RightSurroundDirect. (Using RearSurround* for 6-7 would
/// swap side/back vs the Windows/Linux clients.)
private func wireChannelLayout(channels: Int) -> AVAudioChannelLayout? {
let labels: [AudioChannelLabel]
switch channels {
case 6:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen, kAudioChannelLabel_LeftSurround,
kAudioChannelLabel_RightSurround,
]
case 8:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen,
kAudioChannelLabel_LeftSurround, kAudioChannelLabel_RightSurround, // wire RL/RR (back)
kAudioChannelLabel_LeftSurroundDirect, kAudioChannelLabel_RightSurroundDirect, // wire SL/SR (side)
]
default:
return nil
}
let size = MemoryLayout<AudioChannelLayout>.size
+ (labels.count - 1) * MemoryLayout<AudioChannelDescription>.stride
let raw = UnsafeMutableRawPointer.allocate(byteCount: size, alignment: 16)
defer { raw.deallocate() }
let layout = raw.bindMemory(to: AudioChannelLayout.self, capacity: 1)
layout.pointee.mChannelLayoutTag = kAudioChannelLayoutTag_UseChannelDescriptions
layout.pointee.mChannelBitmap = AudioChannelBitmap(rawValue: 0)
layout.pointee.mNumberChannelDescriptions = UInt32(labels.count)
// `mChannelDescriptions` is the C variable-length tail array (declared `[1]`, over-allocated
// above). Scope the pointer with `withUnsafeMutablePointer` taking `&mChannelDescriptions`
// inline yields a pointer valid only for that expression, so building a buffer from it that
// outlives the call is a dangling-pointer bug. Inside the closure it stays valid while we fill it.
withUnsafeMutablePointer(to: &layout.pointee.mChannelDescriptions) { tail in
let descs = UnsafeMutableBufferPointer(start: tail, count: labels.count)
for (i, lbl) in labels.enumerated() {
descs[i] = AudioChannelDescription(
mChannelLabel: lbl, mChannelFlags: AudioChannelFlags(rawValue: 0),
mCoordinates: (0, 0, 0))
}
}
return AVAudioChannelLayout(layout: layout)
}
public final class SessionAudio {
private let connection: PunktfunkConnection
private let flag = StopFlag()
@@ -210,10 +69,11 @@ public final class SessionAudio {
/// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending.
public func start(speakerUID: String, micUID: String, micEnabled: Bool) {
public func start(speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool) {
#if os(macOS)
// No AVAudioSession on macOS start the engines directly (caller's thread, as before).
startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel, micEnabled: micEnabled)
#else
// Configure + activate the session OFF the main thread (it blocks on the audio server),
// then start the engines back on the main thread once it's active engine routing/format
@@ -223,7 +83,9 @@ public final class SessionAudio {
self.activateAudioSession(micEnabled: micEnabled)
DispatchQueue.main.async { [weak self] in
guard let self, !self.flag.isStopped else { return }
self.startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
self.startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
micEnabled: micEnabled)
}
}
#endif
@@ -257,7 +119,9 @@ public final class SessionAudio {
/// Build + start the playback engine (and the mic uplink when enabled + authorized). Main
/// thread (engine setup); on iOS/tvOS the session is already active by the time this runs.
private func startEngines(speakerUID: String, micUID: String, micEnabled: Bool) {
private func startEngines(
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool
) {
startPlayback(speakerUID: speakerUID)
#if os(tvOS)
// No app-accessible microphone input on tvOS playback only.
@@ -265,12 +129,12 @@ public final class SessionAudio {
guard micEnabled else { return }
switch AVCaptureDevice.authorizationStatus(for: .audio) {
case .authorized:
startCapture(micUID: micUID)
startCapture(micUID: micUID, micChannel: micChannel)
case .notDetermined:
AVCaptureDevice.requestAccess(for: .audio) { [weak self] granted in
DispatchQueue.main.async {
guard let self, granted, !self.flag.isStopped else { return }
self.startCapture(micUID: micUID)
self.startCapture(micUID: micUID, micChannel: micChannel)
}
}
default:
@@ -422,7 +286,7 @@ public final class SessionAudio {
// MARK: - Mic (mic host)
#if !os(tvOS)
private func startCapture(micUID: String) {
private func startCapture(micUID: String, micChannel: Int) {
let engine = AVAudioEngine()
let input = engine.inputNode
#if os(macOS)
@@ -442,8 +306,63 @@ public final class SessionAudio {
log.error("no usable input device — mic uplink disabled")
return
}
guard let encoder = try? OpusEncoder(),
let resampler = AVAudioConverter(from: inFormat, to: encoder.pcmFormat),
// Multi-channel-interface handling. A pro interface exposes N discrete inputs with the mic
// on ONE of them, but AVAudioConverter's Nstereo downmix takes channels 0/1 dead
// silence when the mic sits higher up (the classic "host receives zeros"). So we fold the
// input to a single mono bus OURSELVES and resample that. micChannel: 0 = Auto (sum every
// channel a lone hot mic passes at full level), n1 pins 1-based input channel n.
let inChannels = Int(inFormat.channelCount)
let pinnedChannel: Int? = {
guard micChannel >= 1 else { return nil }
let idx = micChannel - 1
guard idx < inChannels else {
log.warning(
"mic channel \(micChannel) out of range (device has \(inChannels)) — mixing all")
return nil
}
return idx
}()
let channelPlan = pinnedChannel.map { "channel \($0 + 1)/\(inChannels)" }
?? (inChannels > 1 ? "mix \(inChannels)ch→mono" : "mono")
// Name the device we're ACTUALLY recording from + its format + how we fold it, once per
// session. This single line localizes the whole class of "host receives silence" failures
// that otherwise need a host-side tone injection to pin down: a UID that silently fell back
// to the default, the wrong device being live, or the wrong channel picked.
#if os(macOS)
if let unit = input.audioUnit, let live = Self.currentDevice(of: unit),
let dev = AudioDevices.describe(live) {
if !micUID.isEmpty, dev.uid != micUID {
log.warning("""
mic selection not honored requested \(micUID) but capturing from \
\(dev.name) [\(dev.uid)]; the device's UID likely changed (replug) \
reselect it in Settings
""")
}
log.info("""
mic capture: \(dev.name) [\(dev.uid)] \(Int(inFormat.sampleRate)) Hz, \
\(inChannels) ch, \(channelPlan)
""")
} else {
log.info("""
mic capture: <device unavailable> \(Int(inFormat.sampleRate)) Hz, \
\(inChannels) ch, \(channelPlan)
""")
}
#else
log.info(
"mic capture: \(Int(inFormat.sampleRate)) Hz, \(inChannels) ch, \(channelPlan)")
#endif
// Encode a single mono bus (folded from `inFormat` in the tap): the resampler goes
// mono@inputSR the encoder's 48 kHz stereo, so it handles both the rate change and the
// monostereo duplication, and the wrong-channel downmix never happens.
guard let monoFormat = AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: inFormat.sampleRate,
channels: 1, interleaved: false),
let encoder = try? OpusEncoder(),
let resampler = AVAudioConverter(from: monoFormat, to: encoder.pcmFormat),
let chunk = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: OpusEncoder.framesPerPacket)
else {
@@ -459,11 +378,59 @@ public final class SessionAudio {
let connection = connection
let flag = flag
// Silence tripwire (tap-confined): a "recording" app can be handed pure digital zeros
// a zeroed input-volume slider, a stale TCC grant, a muted device, OR the wrong channel
// picked and everything downstream looks alive while the host gets silence. Track the
// peak of the EXTRACTED mono bus over the first ~10 s (not the raw device a mic present
// on a channel we didn't grab must still read as silence) and emit exactly ONE verdict.
// This is the log line whose absence made the last occurrence take a host-side tone.
let silenceWindow = Int(inFormat.sampleRate * 10)
let deviceLabel = micUID.isEmpty ? "default input" : micUID
var framesInspected = 0
var inputPeak: Float = 0
var levelReported = false
input.installTap(onBus: 0, bufferSize: 2048, format: inFormat) { buffer, _ in
if flag.isStopped { return }
let frames = Int(buffer.frameLength)
guard frames > 0, let src = buffer.floatChannelData,
let mono = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: buffer.frameLength),
let dst = mono.floatChannelData?[0]
else { return }
mono.frameLength = buffer.frameLength
// Fold the multi-channel input down to the one mono bus we encode.
Self.foldToMono(
input: src, frames: frames, channels: Int(buffer.format.channelCount),
interleaved: buffer.format.isInterleaved, pinned: pinnedChannel, out: dst)
if !levelReported {
var localPeak: Float = 0
for i in 0..<frames where abs(dst[i]) > localPeak { localPeak = abs(dst[i]) }
if localPeak > inputPeak { inputPeak = localPeak }
framesInspected += frames
if framesInspected >= silenceWindow {
levelReported = true
if inputPeak == 0 {
log.warning("""
mic uplink has been pure digital SILENCE for 10 s (\(deviceLabel), \
\(channelPlan)) check the input level (System Settings Sound \
Input), Privacy & Security Microphone, and the Microphone channel in \
Settings; the host is receiving zeros
""")
} else {
let dbfs = 20 * log10(inputPeak)
log.info("""
mic uplink OK peak \(String(format: "%.1f", dbfs)) dBFS over first \
10 s (\(deviceLabel), \(channelPlan))
""")
}
}
}
let ratio = 48_000 / inFormat.sampleRate
let outCapacity = AVAudioFrameCount(
(Double(buffer.frameLength) * ratio).rounded(.up) + 64)
let outCapacity = AVAudioFrameCount((Double(frames) * ratio).rounded(.up) + 64)
guard let staging = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: outCapacity)
else { return }
@@ -476,7 +443,7 @@ public final class SessionAudio {
}
fed = true
outStatus.pointee = .haveData
return buffer
return mono
}
guard status != .error, let p = staging.floatChannelData?[0] else { return }
fifo.append(contentsOf: UnsafeBufferPointer(
@@ -520,6 +487,42 @@ public final class SessionAudio {
stateLock.unlock()
log.info("mic uplink started (\(micUID.isEmpty ? "default input" : micUID))")
}
/// Fold `channels` of input (`floatChannelData` layout: `interleaved` one buffer strided by
/// channel count; else one buffer per channel) down to a single mono bus in `out` (`frames`
/// long). `pinned` (0-based, must be `< channels`) copies exactly that channel the fix for a
/// mic on one input of a multi-channel interface; `nil` sums every channel, clamped to
/// [-1, 1], so a lone hot channel still passes at full level instead of the silent 0/1 the
/// default Nstereo downmix would grab. Pure + `internal` for unit testing the index math.
static func foldToMono(
input: UnsafePointer<UnsafeMutablePointer<Float>>, frames: Int, channels: Int,
interleaved: Bool, pinned: Int?, out: UnsafeMutablePointer<Float>
) {
if let ch = pinned, ch < channels {
if interleaved {
let d = input[0]
for i in 0..<frames { out[i] = d[i * channels + ch] }
} else {
let d = input[ch]
for i in 0..<frames { out[i] = d[i] }
}
} else if interleaved {
let d = input[0]
for i in 0..<frames {
var s: Float = 0
for c in 0..<channels { s += d[i * channels + c] }
out[i] = max(-1, min(1, s))
}
} else {
let d0 = input[0]
for i in 0..<frames { out[i] = d0[i] }
for c in 1..<channels {
let d = input[c]
for i in 0..<frames { out[i] += d[i] }
}
if channels > 1 { for i in 0..<frames { out[i] = max(-1, min(1, out[i])) } }
}
}
#endif
#if os(macOS)
@@ -529,5 +532,18 @@ public final class SessionAudio {
unit, kAudioOutputUnitProperty_CurrentDevice, kAudioUnitScope_Global, 0,
&dev, UInt32(MemoryLayout<AudioDeviceID>.size)) == noErr
}
/// Read back the AUHAL's live device the definitive "what are we actually capturing
/// from", which catches a selection that succeeded on paper but silently fell back to
/// the system default (a stale/changed UID, a device that vanished between resolve and
/// start). 0 / an error means we couldn't tell.
private static func currentDevice(of unit: AudioUnit) -> AudioDeviceID? {
var dev = AudioDeviceID(0)
var size = UInt32(MemoryLayout<AudioDeviceID>.size)
let status = AudioUnitGetProperty(
unit, kAudioOutputUnitProperty_CurrentDevice, kAudioUnitScope_Global, 0, &dev, &size)
guard status == noErr, dev != 0 else { return nil }
return dev
}
#endif
}
@@ -0,0 +1,59 @@
// The client's persistent identity + the SPAKE2 PIN pairing ceremony the trust
// bootstrap that precedes any pinned PunktfunkConnection.
import Foundation
import PunktfunkCore
/// This client's persistent self-signed identity. Generate ONCE with `generateIdentity()`,
/// store both PEMs (Keychain), present on every connect the certificate's fingerprint is
/// how hosts recognize this client after pairing.
public struct ClientIdentity: Sendable {
public let certPEM: String
public let keyPEM: String
public init(certPEM: String, keyPEM: String) {
self.certPEM = certPEM
self.keyPEM = keyPEM
}
}
/// Generate a fresh client identity (self-signed cert + key, PEM).
public func generateIdentity() throws -> ClientIdentity {
var cert = [CChar](repeating: 0, count: 4096)
var key = [CChar](repeating: 0, count: 4096)
let rc = punktfunk_generate_identity(&cert, UInt(cert.count), &key, UInt(key.count))
guard rc == PUNKTFUNK_STATUS_OK.rawValue else {
throw PunktfunkClientError.status(rc)
}
return ClientIdentity(certPEM: String(cString: cert), keyPEM: String(cString: key))
}
/// Run the PIN pairing ceremony: the host displays a 4-digit PIN (its log/UI), the user
/// types it here. On success the host stores this client's identity and the returned
/// fingerprint is the host's now-VERIFIED identity persist it and pass it as `pinSHA256`
/// to every later connect. Throws `.wrongPIN` when the proof is rejected.
public func pair(
host: String, port: UInt16 = 9777,
identity: ClientIdentity, pin: String, name: String,
timeoutMs: UInt32 = 90_000
) throws -> Data {
var observed = [UInt8](repeating: 0, count: 32)
// The C header types PunktfunkStatus as a bare int32 (C17, no enum import), so the ABI
// functions return Int32 directly compare against the enum constants' rawValue, the
// same bridging the connection methods use (statusOK etc.).
let rc = host.withCString { cs in
identity.certPEM.withCString { cert in
identity.keyPEM.withCString { key in
pin.withCString { p in
name.withCString { n in
punktfunk_pair(cs, port, cert, key, p, n, &observed, timeoutMs)
}
}
}
}
}
switch rc {
case PUNKTFUNK_STATUS_OK.rawValue: return Data(observed)
case PUNKTFUNK_STATUS_CRYPTO.rawValue: throw PunktfunkClientError.wrongPIN
default: throw PunktfunkClientError.status(rc)
}
}
@@ -0,0 +1,87 @@
// Convenience constructors for the wire input events (field semantics match
// punktfunk_core::input::InputEvent; see punktfunk_core.h).
import Foundation
import PunktfunkCore
public extension PunktfunkInputEvent {
private static func make(
_ kind: UInt32, code: UInt32, x: Int32, y: Int32, flags: UInt32 = 0
) -> PunktfunkInputEvent {
PunktfunkInputEvent(kind: UInt8(kind), _pad: (0, 0, 0), code: code, x: x, y: y, flags: flags)
}
static func mouseMove(dx: Int32, dy: Int32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_MOVE.rawValue, code: 0, x: dx, y: dy)
}
/// Absolute cursor position in client-surface pixels the host places its cursor
/// there (same letterbox mapping and `flags` surface-dims packing as the touch events).
/// Used by the iPad pointer fallback when the scene can't pointer-lock and GCMouse's
/// relative deltas aren't available; the surface dimensions must each fit in 16 bits.
static func mouseMoveAbs(
x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_MOUSE_MOVE_ABS.rawValue, code: 0, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
/// GameStream button ids: 1=left 2=middle 3=right 4=X1 5=X2 (host maps to evdev BTN_*).
static func mouseButton(_ button: UInt32, down: Bool) -> PunktfunkInputEvent {
make(
(down ? PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_DOWN : PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_UP).rawValue,
code: button, x: 0, y: 0)
}
/// `vk` is a Windows virtual-key code (the host's vk_to_evdev table consumes these).
static func key(_ vk: UInt32, down: Bool) -> PunktfunkInputEvent {
make((down ? PUNKTFUNK_INPUT_KIND_KEY_DOWN : PUNKTFUNK_INPUT_KIND_KEY_UP).rawValue, code: vk, x: 0, y: 0)
}
/// WHEEL_DELTA(120)-scaled; positive = up (vertical) / right (horizontal) the
/// convention Moonlight/SDL use; the host maps onto the ei/wl axes.
static func scroll(_ delta: Int32, horizontal: Bool = false) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_SCROLL.rawValue, code: horizontal ? 1 : 0, x: delta, y: 0)
}
// Gamepad (wire contract in punktfunk_core::input::gamepad): one transition per event,
// `pad` = controller index, accumulated host-side into a virtual Xbox 360 or DualSense
// pad (the session's negotiated `GamepadType`).
/// `button` is a GameStream buttonFlags bit (A=0x1000 B=0x2000 X=0x4000 Y=0x8000,
/// dpad=0x1/2/4/8, start=0x10 back=0x20 LS=0x40 RS=0x80 LB=0x100 RB=0x200 guide=0x400,
/// touchpad click=0x100000 DualSense sessions only, the xpad has no such button).
static func gamepadButton(_ button: UInt32, down: Bool, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_GAMEPAD_BUTTON.rawValue,
code: button, x: down ? 1 : 0, y: 0, flags: pad)
}
/// Axis ids: 0=LSX 1=LSY 2=RSX 3=RSY (32768...32767, XInput convention: +y = UP
/// `GCControllerDirectionPad.yAxis` already matches, no flip), 4=LT 5=RT (0...255).
static func gamepadAxis(_ axis: UInt32, value: Int32, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_GAMEPAD_AXIS.rawValue, code: axis, x: value, y: 0, flags: pad)
}
// Touch (host-side: libei ei_touchscreen on the virtual output). `id` distinguishes
// fingers and is reusable after touchUp; coordinates are absolute pixels on the
// client's touch surface, whose size rides in `flags` so the host can rescale
// the surface dimensions must each fit in 16 bits. Built for the iOS variant
// (UITouch these); nothing on macOS emits them yet.
static func touchDown(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_DOWN.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchMove(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_MOVE.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchUp(id: UInt32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_TOUCH_UP.rawValue, code: id, x: 0, y: 0)
}
}
@@ -35,6 +35,10 @@ public struct AccessUnit: Sendable {
public let ptsNs: UInt64
public let frameIndex: UInt32
public let flags: UInt32
/// Client `CLOCK_REALTIME` instant the AU was handed over by the core (post-FEC, decrypted)
/// the **received** measurement point of design/stats-unification.md. The decode stage is
/// `decodedNs - receivedNs`, both client-local (no skew offset applies).
public let receivedNs: Int64
}
/// One Opus audio packet (48 kHz stereo, 5 ms frames) decode with AVAudioConverter
@@ -57,60 +61,6 @@ public enum PunktfunkClientError: Error {
case status(Int32)
}
/// This client's persistent self-signed identity. Generate ONCE with `generateIdentity()`,
/// store both PEMs (Keychain), present on every connect the certificate's fingerprint is
/// how hosts recognize this client after pairing.
public struct ClientIdentity: Sendable {
public let certPEM: String
public let keyPEM: String
public init(certPEM: String, keyPEM: String) {
self.certPEM = certPEM
self.keyPEM = keyPEM
}
}
/// Generate a fresh client identity (self-signed cert + key, PEM).
public func generateIdentity() throws -> ClientIdentity {
var cert = [CChar](repeating: 0, count: 4096)
var key = [CChar](repeating: 0, count: 4096)
let rc = punktfunk_generate_identity(&cert, UInt(cert.count), &key, UInt(key.count))
guard rc == PUNKTFUNK_STATUS_OK.rawValue else {
throw PunktfunkClientError.status(rc)
}
return ClientIdentity(certPEM: String(cString: cert), keyPEM: String(cString: key))
}
/// Run the PIN pairing ceremony: the host displays a 4-digit PIN (its log/UI), the user
/// types it here. On success the host stores this client's identity and the returned
/// fingerprint is the host's now-VERIFIED identity persist it and pass it as `pinSHA256`
/// to every later connect. Throws `.wrongPIN` when the proof is rejected.
public func pair(
host: String, port: UInt16 = 9777,
identity: ClientIdentity, pin: String, name: String,
timeoutMs: UInt32 = 90_000
) throws -> Data {
var observed = [UInt8](repeating: 0, count: 32)
// The C header types PunktfunkStatus as a bare int32 (C17, no enum import), so the ABI
// functions return Int32 directly compare against the enum constants' rawValue, the
// same bridging the connection methods use (statusOK etc.).
let rc = host.withCString { cs in
identity.certPEM.withCString { cert in
identity.keyPEM.withCString { key in
pin.withCString { p in
name.withCString { n in
punktfunk_pair(cs, port, cert, key, p, n, &observed, timeoutMs)
}
}
}
}
}
switch rc {
case PUNKTFUNK_STATUS_OK.rawValue: return Data(observed)
case PUNKTFUNK_STATUS_CRYPTO.rawValue: throw PunktfunkClientError.wrongPIN
default: throw PunktfunkClientError.status(rc)
}
}
/// `withCString` over an optional nil maps to a NULL C pointer.
func withOptionalCString<R>(_ s: String?, _ body: (UnsafePointer<CChar>?) -> R) -> R {
guard let s else { return body(nil) }
@@ -133,6 +83,9 @@ public final class PunktfunkConnection {
/// Same role for the feedback drain thread (rumble + HID-output two core planes,
/// drained sequentially by one thread).
private let feedbackLock = NSLock()
/// Same role for the host-timing (0xCF) puller its own plane in the core, drained
/// non-blockingly by the app's 1 s stats tick (never contends with the blocking pullers).
private let statsLock = NSLock()
/// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0
@@ -255,6 +208,13 @@ public final class PunktfunkConnection {
/// PCM from `nextAudioPcm` is interleaved in the canonical wire order FL FR FC LFE RL RR SL SR.
public private(set) var resolvedAudioChannels: UInt8 = 2
/// The video codec the host resolved for this session (`Welcome.codec`, `PUNKTFUNK_CODEC_*`):
/// `2` = HEVC (default / older host), `1` = H.264, `4` = AV1. Build the decoder from THIS. The
/// resolved value honors the client's `preferredCodec` when the host could emit it.
public private(set) var resolvedCodec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
/// The resolved codec as an `AnnexB.VideoCodec` (H.264 vs HEVC) drives the NAL parsing.
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
/// Connect and start a session at the requested mode (the host creates a native virtual
/// output at exactly this size/refresh). Blocks up to `timeoutMs`.
///
@@ -285,6 +245,8 @@ public final class PunktfunkConnection {
bitrateKbps: UInt32 = 0,
videoCaps: UInt8 = 0,
audioChannels: UInt8 = 2,
videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode
preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference
launchID: String? = nil,
timeoutMs: UInt32 = 10_000
) throws {
@@ -300,16 +262,18 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in
if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in
punktfunk_connect_ex6(
punktfunk_connect_ex7(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, launch,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs)
}
}
return punktfunk_connect_ex6(
return punktfunk_connect_ex7(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, launch,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch,
nil, &observed, cert, key, timeoutMs)
}
}
@@ -347,6 +311,9 @@ public final class PunktfunkConnection {
var ac: UInt8 = 2
_ = punktfunk_connection_audio_channels(handle, &ac)
resolvedAudioChannels = ac
var codec: UInt8 = 2 // PUNKTFUNK_CODEC_HEVC
_ = punktfunk_connection_codec(handle, &codec)
resolvedCodec = codec
}
/// A bandwidth speed-test measurement (see `startSpeedTest`). Partial until `done`.
@@ -459,9 +426,13 @@ public final class PunktfunkConnection {
case statusOK:
guard let base = frame.data, frame.len > 0 else { return nil }
let data = Data(bytes: base, count: Int(frame.len)) // copy: ptr valid only until next call
var ts = timespec()
clock_gettime(CLOCK_REALTIME, &ts)
let receivedNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec)
return AccessUnit(
data: data, ptsNs: frame.pts_ns,
frameIndex: frame.frame_index, flags: frame.flags)
frameIndex: frame.frame_index, flags: frame.flags,
receivedNs: receivedNs)
case statusNoFrame:
return nil
case statusClosed:
@@ -620,6 +591,11 @@ public final class PunktfunkConnection {
/// the host then emits 4:4:4 only if it too opted in. `chromaFormat` reflects the real value.
public static let videoCap444: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_444)
/// Codec bits for `videoCodecs` / `preferredCodec` and the value `resolvedCodec` returns.
public static let codecH264: UInt8 = UInt8(PUNKTFUNK_CODEC_H264)
public static let codecHEVC: UInt8 = UInt8(PUNKTFUNK_CODEC_HEVC)
public static let codecAV1: UInt8 = UInt8(PUNKTFUNK_CODEC_AV1)
/// Static HDR mastering metadata (SMPTE ST.2086 + content light level) the host sent for an HDR
/// session. Mirrors the wire/ABI `PunktfunkHdrMeta`; primaries are in ST.2086 **G, B, R** order,
/// 1/50000 units; mastering luminance in 0.0001 cd/m²; MaxCLL/MaxFALL in nits.
@@ -692,6 +668,40 @@ public final class PunktfunkConnection {
}
}
/// One per-AU host-timing report (0xCF): the host's capturefully-sent duration for the
/// access unit whose `AccessUnit.ptsNs` equals `ptsNs` exactly. The stats consumer derives
/// `network = (receivedNs + clockOffsetNs ptsNs) hostUs` the host/network split of the
/// HUD's `host+network` stage (design/stats-unification.md Phase 2).
public struct HostTiming: Sendable, Equatable {
/// The AU's capture stamp (host capture clock matches the AU's `ptsNs`).
public let ptsNs: UInt64
/// Host capturesent duration, µs.
public let hostUs: UInt32
}
/// Pull the next per-AU host timing; nil on timeout, throws `.closed` once the session
/// ended. Best-effort plane: an older host never emits any keep showing the combined
/// `host+network` stage then. Drain non-blockingly (`timeoutMs: 0`) from ONE stats
/// consumer (its own core plane, safe alongside the other pullers).
public func nextHostTiming(timeoutMs: UInt32 = 0) throws -> HostTiming? {
statsLock.lock()
defer { statsLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkHostTiming()
let rc = punktfunk_connection_next_host_timing(h, &out, timeoutMs)
switch rc {
case statusOK:
return HostTiming(ptsNs: out.pts_ns, hostUs: out.host_us)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Send one input event (delivered to the host as a QUIC datagram). Thread-safe;
/// silently dropped after close.
public func send(_ event: PunktfunkInputEvent) {
@@ -711,10 +721,12 @@ public final class PunktfunkConnection {
pumpLock.lock() // pullers exit at their next poll boundary, releasing these
audioLock.lock()
feedbackLock.lock()
statsLock.lock()
abiLock.lock()
let h = handle
handle = nil
abiLock.unlock()
statsLock.unlock()
feedbackLock.unlock()
audioLock.unlock()
pumpLock.unlock()
@@ -784,87 +796,3 @@ public final class PunktfunkConnection {
return closeRequested ? nil : handle
}
}
// Convenience constructors for the wire input events (field semantics match
// punktfunk_core::input::InputEvent; see punktfunk_core.h).
public extension PunktfunkInputEvent {
private static func make(
_ kind: UInt32, code: UInt32, x: Int32, y: Int32, flags: UInt32 = 0
) -> PunktfunkInputEvent {
PunktfunkInputEvent(kind: UInt8(kind), _pad: (0, 0, 0), code: code, x: x, y: y, flags: flags)
}
static func mouseMove(dx: Int32, dy: Int32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_MOVE.rawValue, code: 0, x: dx, y: dy)
}
/// Absolute cursor position in client-surface pixels the host places its cursor
/// there (same letterbox mapping and `flags` surface-dims packing as the touch events).
/// Used by the iPad pointer fallback when the scene can't pointer-lock and GCMouse's
/// relative deltas aren't available; the surface dimensions must each fit in 16 bits.
static func mouseMoveAbs(
x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_MOUSE_MOVE_ABS.rawValue, code: 0, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
/// GameStream button ids: 1=left 2=middle 3=right 4=X1 5=X2 (host maps to evdev BTN_*).
static func mouseButton(_ button: UInt32, down: Bool) -> PunktfunkInputEvent {
make(
(down ? PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_DOWN : PUNKTFUNK_INPUT_KIND_MOUSE_BUTTON_UP).rawValue,
code: button, x: 0, y: 0)
}
/// `vk` is a Windows virtual-key code (the host's vk_to_evdev table consumes these).
static func key(_ vk: UInt32, down: Bool) -> PunktfunkInputEvent {
make((down ? PUNKTFUNK_INPUT_KIND_KEY_DOWN : PUNKTFUNK_INPUT_KIND_KEY_UP).rawValue, code: vk, x: 0, y: 0)
}
/// WHEEL_DELTA(120)-scaled; positive = up (vertical) / right (horizontal) the
/// convention Moonlight/SDL use; the host maps onto the ei/wl axes.
static func scroll(_ delta: Int32, horizontal: Bool = false) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_MOUSE_SCROLL.rawValue, code: horizontal ? 1 : 0, x: delta, y: 0)
}
// Gamepad (wire contract in punktfunk_core::input::gamepad): one transition per event,
// `pad` = controller index, accumulated host-side into a virtual Xbox 360 or DualSense
// pad (the session's negotiated `GamepadType`).
/// `button` is a GameStream buttonFlags bit (A=0x1000 B=0x2000 X=0x4000 Y=0x8000,
/// dpad=0x1/2/4/8, start=0x10 back=0x20 LS=0x40 RS=0x80 LB=0x100 RB=0x200 guide=0x400,
/// touchpad click=0x100000 DualSense sessions only, the xpad has no such button).
static func gamepadButton(_ button: UInt32, down: Bool, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_GAMEPAD_BUTTON.rawValue,
code: button, x: down ? 1 : 0, y: 0, flags: pad)
}
/// Axis ids: 0=LSX 1=LSY 2=RSX 3=RSY (32768...32767, XInput convention: +y = UP
/// `GCControllerDirectionPad.yAxis` already matches, no flip), 4=LT 5=RT (0...255).
static func gamepadAxis(_ axis: UInt32, value: Int32, pad: UInt32 = 0) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_GAMEPAD_AXIS.rawValue, code: axis, x: value, y: 0, flags: pad)
}
// Touch (host-side: libei ei_touchscreen on the virtual output). `id` distinguishes
// fingers and is reusable after touchUp; coordinates are absolute pixels on the
// client's touch surface, whose size rides in `flags` so the host can rescale
// the surface dimensions must each fit in 16 bits. Built for the iOS variant
// (UITouch these); nothing on macOS emits them yet.
static func touchDown(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_DOWN.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchMove(
id: UInt32, x: Int32, y: Int32, surfaceWidth: UInt32, surfaceHeight: UInt32
) -> PunktfunkInputEvent {
make(
PUNKTFUNK_INPUT_KIND_TOUCH_MOVE.rawValue, code: id, x: x, y: y,
flags: ((surfaceWidth & 0xFFFF) << 16) | (surfaceHeight & 0xFFFF))
}
static func touchUp(id: UInt32) -> PunktfunkInputEvent {
make(PUNKTFUNK_INPUT_KIND_TOUCH_UP.rawValue, code: id, x: 0, y: 0)
}
}
@@ -0,0 +1,85 @@
#if DEBUG
import Combine
import GameController
/// Local feedback driver for the Settings Controllers "Test Controller" panel (DEBUG builds
/// only). It drives the SAME CoreHaptics rumble renderer and `DualSenseTriggerEffect` path a
/// live session uses just aimed at the physically-connected controller instead of the
/// hostclient feedback planes so rumble, the adaptive triggers, the lightbar and the player
/// LEDs can be confirmed on-device without a host. Reusing the real renderers is the point:
/// a passing test exercises the exact code a session runs.
@MainActor
public final class ControllerTester: ObservableObject {
// `.manual`: the panel's toggles hold a level until changed no session wire refreshes
// exist here to keep the renderer's staleness watchdog fed.
private let renderer = RumbleRenderer(policy: .manual)
private weak var controller: GCController?
/// The rumble backend now in use "DualSense HID · USB/Bluetooth", "CoreHaptics", or ""
/// for the test panel to display so it's obvious which path a given pad takes.
@Published public private(set) var rumbleBackend = ""
/// Why rumble structurally cannot work right now (nil = healthy) e.g. the device's
/// haptics service refusing every connection, or a pad with no rumble engine. Shown by the
/// test panel so silence diagnoses itself instead of reading as an app bug.
@Published public private(set) var rumbleHealth: String?
public init() {}
/// Aim the feedback at a controller (nil releases it). Idempotent safe to call on every
/// active-controller change.
public func target(_ c: GCController?) {
guard c !== controller else { return }
controller = c
renderer.retarget(
c,
onBackend: { [weak self] note in
Task { @MainActor in self?.rumbleBackend = note }
},
onHealth: { [weak self] problem in
Task { @MainActor in self?.rumbleHealth = problem }
})
}
/// Drive both motors at 0...1 amplitudes low = left/heavy, high = right/light mapped to
/// the 0...0xFFFF wire range the session carries, through the real `RumbleRenderer`.
public func rumble(low: Float, high: Float) {
func u16(_ v: Float) -> UInt16 { UInt16((min(max(v, 0), 1) * 65535).rounded()) }
renderer.apply(low: u16(low), high: u16(high))
}
public func stopRumble() { renderer.apply(low: 0, high: 0) }
/// Replay an adaptive-trigger effect on a DualSense via the real `DualSenseTriggerEffect`
/// renderer. `right == false` L2, `true` R2. No-op on a non-DualSense pad.
public func applyTrigger(_ effect: DualSenseTriggerEffect, right: Bool) {
guard let ds = controller?.extendedGamepad as? GCDualSenseGamepad else { return }
effect.apply(to: right ? ds.rightTrigger : ds.leftTrigger)
}
public func resetTriggers() {
guard let ds = controller?.extendedGamepad as? GCDualSenseGamepad else { return }
ds.leftTrigger.setModeOff()
ds.rightTrigger.setModeOff()
}
/// Lightbar colour (DualSense / DualShock 4); nil turns it off. No-op without a light.
public func setLight(_ color: GCColor?) {
controller?.light?.color = color ?? GCColor(red: 0, green: 0, blue: 0)
}
/// Player-indicator LEDs (`.index1`...`.index4`, or `.indexUnset` to clear).
public func setPlayerIndex(_ index: GCControllerPlayerIndex) {
controller?.playerIndex = index
}
/// Silence every channel and release the controller call on the panel's disappear.
public func stop() {
resetTriggers()
setPlayerIndex(.indexUnset)
setLight(nil)
renderer.retarget(nil) // async teardown: stops the motors + drops the controller ref
controller = nil
}
}
#endif

Some files were not shown because too many files have changed in this diff Show More