51 Commits

Author SHA1 Message Date
enricobuehler caa7a1c735 chore(release): bump workspace version to 0.7.1
apple / swift (push) Successful in 1m5s
audit / cargo-audit (push) Successful in 16s
ci / web (push) Successful in 1m6s
ci / docs-site (push) Successful in 1m24s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 48s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 52s
ci / bench (push) Successful in 5m1s
android-screenshots / screenshots (push) Successful in 47s
ci / rust (push) Successful in 9m48s
windows-host / package (push) Successful in 7m10s
android / android (push) Successful in 3m53s
release / apple (push) Successful in 8m47s
decky / build-publish (push) Successful in 15s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m20s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m14s
deb / build-publish (push) Successful in 2m59s
apple / screenshots (push) Successful in 5m42s
flatpak / build-publish (push) Successful in 4m11s
linux-client-screenshots / screenshots (push) Successful in 1m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m19s
docker / deploy-docs (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
web-screenshots / screenshots (push) Successful in 2m34s
Cut 0.7.1 to test the GTK + Decky polish batch (57ae00a) on-device: the
host-click / disconnect-chord / gray-screen-recovery / leak fixes on the
Linux client, the Deck launcher perf profile, and the Decky pin/pairing
fixes. The [workspace.package] version (inherited by every crate via
version.workspace) is the release being cut; refresh the 9 workspace
entries in Cargo.lock to match (CI builds --locked). Canary derives from
the tag (scripts/ci/pf-version.sh), so cutting v0.7.1 auto-advances canary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 07:47:58 +00:00
enricobuehler 13dc7fc49f style(linux): rustfmt the keyframe-recovery throttle line
apple / swift (push) Successful in 1m9s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Wrap the `last_kf_req.is_none_or(...)` guard to satisfy `cargo fmt --all
--check` (CI Format step).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 07:47:16 +00:00
enricobuehler 57ae00a9c8 fix(clients): GTK + Decky polish batch from live Deck/Windows testing
ci / rust (push) Failing after 41s
apple / swift (push) Successful in 1m8s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 3m20s
deb / build-publish (push) Successful in 2m55s
decky / build-publish (push) Successful in 27s
apple / screenshots (push) Successful in 5m46s
ci / bench (push) Successful in 5m5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 3m20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m31s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m18s
docker / deploy-docs (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
GTK Linux client:
- hosts/library: clicking a card was dead — the handler was on
  FlowBoxChild::activate (never emitted on click); bridge child-activated
  → child.activate() on the FlowBox (ui_hosts, ui_library).
- stream: the Ctrl+Alt+Shift+D/Q/S chords (and all key forwarding) were
  dropped because the key controller sat on the overlay, which loses focus
  to the header back button after nav.push+fullscreen — move it to the
  window and remove it on teardown.
- video: a mid-session VAAPI decode error rebuilt a software decoder but
  never requested a keyframe, so under the infinite GOP the picture stayed
  gray/frozen forever. Request an IDR on any VAAPI error, keep the hardware
  decoder, and demote to software only after repeated failures.
- stream: fix a per-session Capture↔overlay reference cycle that leaked the
  overlay subtree + the Arc<NativeClient> on every session end — hold the
  overlay weakly.
- stream: accumulate the fractional wheel remainder so precision-scroll
  (Deck trackpad / hi-res wheels) sub-unit deltas aren't dropped.
- gamepad library: keep the launcher smooth on the Deck — freeze the aurora
  and trim the visible card range (fewer 3D offscreen passes) on low-power.
- gamepad: log full pad identity (vid:pid:name:type:virtual) on attach to
  diagnose an empty controller list on the Deck.
- cli: --connect host:<badport> silently did nothing; default to 9777 + warn.
- css: add the missing .pf-neutral pill rule; fix the clipped most-recent
  accent (inset outline instead of a corner-clipped box-shadow bar).

Decky plugin:
- surface the on-screen library browser: label the host-row Games button.
- fix silent pin data-loss — the detached Games modal captured a frozen
  pins array, so pinning a second game clobbered the first; mirror pins in
  a ref and track the modal's pinned ids locally for a live label.
- route pair-required hosts through the pairing modal from the fullscreen
  Stream button (parity with the QAM panel).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 07:37:04 +00:00
enricobuehler 882a3d57f6 chore(release): bump workspace version to 0.7.0
android-screenshots / screenshots (push) Successful in 2m12s
windows-host / package (push) Successful in 6m43s
android / android (push) Successful in 9m24s
release / apple (push) Successful in 9m35s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m8s
apple / swift (push) Successful in 1m10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 53s
linux-client-screenshots / screenshots (push) Successful in 1m41s
apple / screenshots (push) Successful in 5m37s
web-screenshots / screenshots (push) Successful in 2m37s
audit / cargo-audit (push) Successful in 1m12s
ci / web (push) Successful in 1m8s
ci / docs-site (push) Successful in 1m22s
ci / rust (push) Successful in 4m34s
deb / build-publish (push) Successful in 3m0s
ci / bench (push) Successful in 4m57s
decky / build-publish (push) Successful in 12s
flatpak / build-publish (push) Successful in 4m7s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m2s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m42s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 54s
docker / deploy-docs (push) Successful in 20s
The [workspace.package] version (inherited by every crate via
version.workspace) is the release being cut. Refresh the 9 workspace entries
in Cargo.lock to match (CI builds --locked). Canary derives from the tag now
(scripts/ci/pf-version.sh), so no canary-base edit is needed — cutting v0.7.0
auto-advances canary to 0.8.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:45:45 +00:00
enricobuehler fa28fa19a0 Merge remote-tracking branch 'origin/main'
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
2026-07-03 22:41:42 +00:00
enricobuehler 42595b5558 fix(flatpak): keep both channels in the OSTree summary (fixes stable "No such ref")
The surfaced install command
  flatpak install --user https://flatpak.unom.io/io.unom.Punktfunk.flatpakref
failed with "No such ref 'app/io.unom.Punktfunk/x86_64/stable'". The stable
commit's objects are on the server, but the repo *summary* (what flatpak reads
to resolve refs) listed only canary.

Root cause: each CI run builds a fresh SINGLE-branch local OSTree repo,
build-update-repo regenerates the summary from that one branch, and rsync
uploads it without --delete. Objects for both channels accumulate, but the
summary is overwritten every run and only names that run's branch. Canary runs
on every main push, stable only on tags — so a tag published stable, then the
next canary push clobbered the summary back to canary-only.

Fix: seed the local repo from the live server (rsync repo/ DOWN) before the
build, so it carries every published branch; the build only adds this run's
commit and the regenerated+signed summary keeps both channels. Single shared
repo kept (no URL/Caddyfile change; existing installs fixed transparently).
Adds a refs log after build-update-repo as a clobber tripwire. Also adopts
scripts/ci/pf-version.sh for the canary base (see previous commit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:40:35 +00:00
enricobuehler 4de543c146 ci(release): derive canary version from git tags (single source of truth)
Every release workflow hardcoded a canary base version (0.5.0 in
Apple/Android/rpm/flatpak/deb, 0.3 in windows-msix/windows-host/decky) that
had to be hand-bumped on each stable release and wasn't. With stable at
v0.6.0, every canary was a version *behind* stable — e.g. the Apple canary
showed up on TestFlight as 0.5.0 while 0.6.0 was already published.

Add scripts/ci/pf-version.{sh,ps1} (bash + pwsh twin) as the single source of
truth: stable = the vX.Y.Z tag; canary = latest stable tag with minor+1,
patch 0 (v0.6.0 -> 0.7.0), so canary is always exactly one minor ahead of the
newest release with zero maintenance. Falls back to the workspace Cargo.toml
version when no tag is fetchable. All workflows now eval/call it and format
their own channel suffix off $PF_BASE; only the canary branch changed, stable
branches and per-channel suffixes are untouched. channels.md drops the old
manual "bump the canary base" release step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:40:25 +00:00
enricobuehler 42d1c74663 fix(apple-client/audio): capture the right channel of a multi-channel mic + diagnostics
apple / swift (push) Successful in 1m5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
The mic uplink handed the host pure digital silence on a multi-channel
interface: AVAudioConverter's N→stereo downmix takes channels 0/1, but a
pro interface puts the mic on ONE higher discrete channel. Fold the input
to a mono bus ourselves instead — pick the mic's channel (or sum all) and
resample that to the encoder's 48 kHz stereo, so the silent 0/1 downmix
never happens.

- New "Microphone channel" setting (macOS): Auto (sum every channel — a
  lone hot mic passes at full level) or pin 1-based channel N. Picker
  appears only for multi-channel devices, driven by the device's input
  channel count.
- Diagnostics that make this class of failure self-naming next session:
  log the actual live capture device + format + fold mode, warn on a
  silent UID fallback, and a one-shot silence tripwire on the EXTRACTED
  signal (WARN on 10 s of zeros, else peak dBFS).
- foldToMono extracted as a pure, unit-tested helper (pin / sum-clamp x
  interleaved / deinterleaved / mono / out-of-range).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 00:36:35 +02:00
enricobuehler 136f6e8f0e feat(probe): --mic-burst — real-client mic pacing for jitter-buffer regression tests
apple / swift (push) Successful in 1m8s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
The steady 5 ms mic-test cadence never trips host-side buffering bugs:
the WASAPI crackle (fixed in the previous commit) only reproduced under
a real client's bursty input tap. --mic-burst paces the tone the same
way (two 20 ms Opus packets every 40 ms), so recording the host mic and
counting silence gaps regression-tests the jitter buffer headlessly.
Validated against the fixed Windows host on the lab box: 15 s of bursty
tone, zero mid-stream gaps >=3 ms (gaps confined to the first 40 ms
priming window).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 22:35:49 +00:00
enricobuehler 00acf5e44e fix(host/audio): WASAPI virtual mic — port the priming jitter buffer (crackling fix)
apple / swift (push) Successful in 1m8s
ci / rust (push) Successful in 1m56s
apple / screenshots (push) Successful in 5m17s
ci / bench (push) Successful in 4m41s
decky / build-publish (push) Successful in 24s
ci / web (push) Successful in 59s
android / android (push) Successful in 3m41s
ci / docs-site (push) Successful in 1m0s
windows-host / package (push) Successful in 7m6s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m39s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 1m7s
deb / build-publish (push) Successful in 9m15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Mac → Windows mic passthrough crackled heavily while the identical
stream was clean on the Linux host. Cause: clients push mic audio in
BURSTS on their own clock (the Mac input tap yields ~two 20 ms Opus
packets every ~42 ms) while the WASAPI render loop pulled a block every
~10 ms device period and greedily drained whatever was queued, padding
the rest with zeros — the queue sat near-empty and most periods
inserted mid-stream silence. The Linux backend has absorbed this since
day one with its priming jitter buffer; the WASAPI loop had none.

Port the same semantics: emit silence until ~48 ms is buffered (covers
the worst inter-burst gap), then play from the cushion (zero-filling
only a momentary shortfall), re-prime only after a genuine full drain
(client went quiet). Queue cap raised 80 → 120 ms for burst headroom;
steady-state added latency ≈ the 48 ms cushion.

Diagnosed live on .173: probe tone recording from CABLE Output proved
the endpoint wiring, then the burst-vs-period math explained the
crackle. Build-verified on Windows; on-glass listen pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 22:31:18 +00:00
enricobuehler 38078fe7ee feat(linux-client): gamepad library launcher — console-style coverflow (--browse)
ci / rust (push) Successful in 1m54s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
windows-host / package (push) Successful in 6m43s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m12s
ci / bench (push) Successful in 4m47s
apple / swift (push) Successful in 1m9s
android / android (push) Successful in 3m33s
deb / build-publish (push) Successful in 4m36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
decky / build-publish (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
release / apple (push) Successful in 8m30s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 48s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m15s
flatpak / build-publish (push) Successful in 4m4s
apple / screenshots (push) Successful in 5m31s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m48s
docker / deploy-docs (push) Successful in 24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m20s
A controller-driven, chrome-less library launcher for the Steam Deck flow
(the Decky plugin's "Open library on screen" + pinned games, 8470419):
`--browse host[:port]` opens a paired host's game library as a coverflow
over a drifting aurora — A streams the focused title (the id rides the
Hello), session end returns to the launcher, B quits back to Gaming Mode.
`--connect` gains `--launch <id>` for direct-to-game starts; `--mgmt`
overrides the library port. Scope is deliberately library-only: host
selection/settings stay in the touch UI, pairing stays in the plugin (no
dialog can map under gamescope — every state renders in-page).

- gamepad.rs menu mode: the worker holds the active pad open while idle
  (WITHOUT the Valve HIDAPI drivers — Deck lizard mode survives) and
  translates it through a pure MenuNav state machine: edge-triggered
  buttons, held-state snapshot on entry/detach (the escape chord that ends
  a stream can't ghost-fire in the menu), 380/160 ms stick/dpad repeat,
  menu rumble ticks. Keyboard fallback (arrows/Enter/Esc) drives the same
  handler — fully usable with no pad, no host (PUNKTFUNK_FAKE_LIBRARY).
- Coverflow: ±38° corridor-facing tilt under per-card perspective
  (gsk rotate_3d), dense overlapping side shelves with paint-order
  restacking (gtk::Fixed draws in child order), opaque card faces + a
  darkening veil for the recede (translucency would bleed the stack
  through). The strip lives in an External-policy ScrolledWindow because
  a bare gtk::Fixed measures its TRANSFORMED children and inflates the
  page min-width past the window.
- Spring-driven motion: semi-implicit Euler in ≤8 ms substeps (a raw
  50 ms frame leaves the stiff recoil spring ringing at ω·dt ≈ 1.2 —
  regression-tested), ζ≈0.85 cursor chase + ζ≈0.55 boundary wobble;
  velocity carries across retargets so held-repeat scrolling glides.
- Shot scene `gamepad-library` (GTK animations force-disabled in shot mode
  — nav transitions froze mid-slide in headless captures); shared poster
  fetch extracted to library::spawn_art_fetch.

Verified here: 21 unit tests (MenuNav, cursor stepping, spring
convergence/stability), clippy -D warnings clean, screenshot scene
pixel-checked, --browse smoke runs (fake-library + unpaired) on the
headless session. On-Deck validation pending (virtual-pad input, lizard
mode, rumble via Steam Input, full Decky→browse→stream→launcher loop).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:41:43 +00:00
enricobuehler 69609945a3 feat(clients): host/network split in every stats HUD (stats phase 2, client side)
Consumes the 0xCF host-timing plane (449a67c) on all four GUI clients: each
keeps a bounded pending ring of receipt samples keyed by pts, matches the
host's per-AU capture→sent reports against it, and the HUD equation becomes

  = host 3.1 + network 6.7 + decode 2.1 + display 2.3

falling back to the combined `= host+network …` term whenever no timing
matched the window (old host / datagram loss) — same total, one split
fewer, never a misleading zero. Apple additionally gains the split as the
only equation line under the stage-1 fallback presenter (receipt is
presenter-independent), a `nextHostTiming` wrapper with its own plane lock,
and a unit-tested `HostNetworkSplitter`; Android extends the JNI stats
array 16→18 doubles (0–15 unchanged); Windows/Linux thread the split
through `Stats` into the HUD and the headless/debug logs.

Docs updated: design/stats-unification.md Phase 2 → implemented (wire
format, fallback semantics), and the docs-site matrix's Sunshine "Host
processing latency" row is now a direct match (ours includes the paced
send; avg vs p50).

Verified here: linux client clippy -D warnings green on the live tree,
windows stub check + hand-verified diff, android cargo-ndk arm64 check
green, apple loopback test extended (needs the rebuilt xcframework + swift
test on the mac). On-glass: pending on all platforms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:31:49 +00:00
enricobuehler 8470419433 feat(decky): pinned-games library + self-update robustness; fix gamepad tab-nav
Decky client batch:
- Pinned games / library picker: per-host game grid (GamePickerModal),
  pin/unpin, one-tap streams surfaced on the Hosts tab and QAM
  (usePins/streamPin/resolvePinHost, new src/library.tsx).
- Self-update + client-update plumbing (main.py check_update, hooks.ts
  applyUpdate) with a CA-bundle-resolving SSL context and per-channel
  manifest polling; steam.ts / punktfunkrun.sh launch tweaks.
- scripts/test-backend.py harness for the backend RPCs; README refresh.

Fix: the fullscreen page wrapped <Tabs> in an overflow-visible box, so
Valve's L1/R1 tab slide + autoFocusContents scrollIntoView panned
#GamepadUI itself — the whole Steam UI slid left until a tab was clicked.
Clip the Tabs wrapper (overflow:hidden), matching Valve's own Tabs
containers. (On-glass verification pending — Deck offline this session.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 21:25:07 +00:00
enricobuehler 449a67ce8d feat(protocol): per-AU host-timing plane (0xCF) — split host+network latency (stats phase 2)
The unified-stats equation's host+network stage was one opaque number
because the wire carried nothing but pts_ns. Now the host reports its own
share per frame: when the client's Hello sets VIDEO_CAP_HOST_TIMING (0x08),
the send thread emits a 13-byte 0xCF datagram — [tag][pts_ns u64][host_us
u32] — right after the AU's last packet leaves the socket, so host_us =
capture→fully-sent (capture read/convert, encode, FEC+seal, paced send)
against the same anchor the wire pts carries. Clients correlate by pts_ns
and derive network = (received + clock_offset − pts) − host_us; the two
terms tile per frame by construction.

Back-compat is free in all four combinations: old clients ignore unknown
datagram tags, old hosts ignore unknown cap bits (client keeps the combined
stage). The hardened data-plane format is untouched — this rides the
established QUIC side-plane pattern (0xC8…0xCE). NativeClient ORs the bit
in unconditionally and exposes next_host_timing(); the C ABI gains
PunktfunkHostTiming + punktfunk_connection_next_host_timing (additive).
The synthetic host emits 0xCF too, so pure-loopback protocol tests cover
the plane.

The probe reports the split (host_p50/p95_us · net_p50/p95_us) and is our
direct analogue of Sunshine's "host processing latency" — ours additionally
includes the paced send.

Validated on loopback (synthetic host + probe, debug build): 240/240 AUs
matched, host_p50 6.5 ms + net_p50 6.4 ms ≈ capture→received p50 13.0 ms.
Core suite + new 0xCF roundtrip/truncation test green; host+core+probe
clippy clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:22:12 +00:00
enricobuehler 09a5957c6d feat(clients): unified stats vocabulary across every client + Moonlight comparison docs
One stat model everywhere (design/stats-unification.md): four measurement
points (capture/received/decoded/displayed), three stages that tile the
interval exactly, and a HUD that shows the addition explicitly —

  end-to-end 14.2 ms p50 · 19.8 p95 · capture→on-glass
  = host+network 9.8 + decode 2.1 + display 2.3

replacing each client's ad-hoc mix of overlapping absolutes (the Apple HUD's
three arrow lines that looked sequential but weren't), mean-vs-median decode
times (Windows/Linux), missing same-host-clock flags (Windows/Linux), and
three different names for the same capture→received measurement (probe's
"reassembled", Apple/Android's "client", Windows/Linux's post-decode "lat").

Per client: Apple threads receivedNs through the VT decode via the frame
refcon bit pattern so the decode stage exists at all (stage-1 fallback
honestly degrades to a capture→received headline); Windows carries
FrameTimes through the existing frame channel to the render thread and adds
e2e p50/p95 post-Present; Linux stamps received at AU pop and rides
decoded_ns on DecodedFrame to the paintable-set site; Android pairs receipt
stamps with MediaCodec output buffers via the codec's pts round-trip (JNI
stats array 14→16 doubles, indexes 0-13 unchanged). fps now uniformly counts
received AUs; lost/(received+lost) per window, hidden at zero.

docs-site gains "Understanding the Stats Overlay": what each line means, why
the equation only approximately sums (percentiles), and a line-by-line
Moonlight/Sunshine matrix — including that Moonlight has no end-to-end
number and its "network latency" is an ENet control RTT, so punktfunk's
headline must not be compared against any single Moonlight line.

Verified here: linux client + probe + core check/clippy/fmt green, android
native cargo-ndk arm64 check green. Pending: Windows CI + on-glass, swift
test on the mac, on-device Android.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:01:29 +00:00
enricobuehler c7630ff5dc fix(host/audio): mic pump — open handshake on Linux + rapid-death backoff
apple / swift (push) Successful in 1m8s
apple / screenshots (push) Successful in 5m18s
android / android (push) Successful in 3m21s
windows-host / package (push) Successful in 6m58s
ci / rust (push) Successful in 1m58s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m1s
ci / bench (push) Successful in 4m49s
deb / build-publish (push) Successful in 4m37s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m17s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m59s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m34s
Found by a live boot-order test (host started before the user session's
PipeWire): PwMicSource::open returned Ok before the daemon connection was
attempted, so a PipeWire that wasn't running surfaced as an instantly-dead
instance instead of an open failure — and the pump churned
open→die→reopen at heartbeat rate (1 Hz "virtual mic ready" log spam)
instead of backing off.

- PwMicSource::open now has a bring-up handshake (mirrors the Windows
  backend): ready only after connect + stream connect succeed, so a
  down daemon is an open ERROR and the pump's backoff engages.
- The pump triages deaths: an instance that lived >= 5 s (a one-off
  daemon restart) reopens immediately with the backoff reset; one that
  died right after opening counts as a failed open and backs off
  (2 s → 60 s cap). New pump test rapid_death_backs_off.

Re-validated live: host started with PipeWire stopped → throttled
"unavailable" warns, zero churn; daemon started → mic node up on the
next retry; exactly one pump + one loop thread (no leak).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:58:06 +00:00
enricobuehler 2c7ded0f3c fix(host/audio): rebuild mic passthrough — eager, self-healing virtual mic on both hosts
apple / swift (push) Successful in 1m7s
ci / rust (push) Successful in 1m57s
ci / web (push) Successful in 59s
android / android (push) Successful in 3m19s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m12s
windows-host / package (push) Successful in 7m2s
ci / bench (push) Successful in 4m52s
decky / build-publish (push) Successful in 14s
deb / build-publish (push) Successful in 4m37s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m14s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m40s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m28s
Mic passthrough silently died on real hosts. Root causes, all fixed:

- No liveness anywhere: a PipeWire restart (Linux) or any WASAPI device
  error (Windows) killed the backend worker; push() fed the dead queue
  for the rest of the host's life. VirtualMic now has a liveness
  contract (push -> bool, alive(), discard()) and the new shared
  audio::MicPump reopens with backoff, probing on an idle heartbeat so
  the mic heals BETWEEN sessions too. Validated live: systemctl restart
  pipewire -> node back in ~0.5 s, tone flows through the reopened
  backend.

- Lazy creation: the mic device didn't exist until the first 0xCB
  frame, but games bind their capture device at launch and never
  re-follow. The pump opens eagerly at host start (node exists with
  zero clients, elected default source).

- Windows headless dead-end: with VB-CABLE as the ONLY render endpoint
  (exactly what the installer ships), the anti-echo guard rejected the
  cable as the default render endpoint -> mic permanently dead. The new
  wiring_plan (pure, unit-tested on every platform) assigns the mic its
  endpoint FIRST (cable reserved for the mic), points the loopback at a
  DIFFERENT endpoint, and the capture side now yields (explicit
  endpoint or honest error) instead of the mic dying. Plan recomputed
  per (re)open — endpoints churn at boot/logon/driver installs.

- Stale bursts: buffered audio from a previous session played into a
  newly-attached recorder (observed live). Timestamped chunks + a
  consumer-gap check in the process callback age everything past 1 s.

The Linux node mechanism stays the stream-based Audio/Source with
RT_PROCESS + priority.session: the canonical null-audio-sink adapter
recipe was tested on this box (PipeWire 1.6.2) and never gets a clock
(QUANT 0 -> pure silence), and WirePlumber reroutes a feeder targeting
it to the default sink (echo). Decision documented in the module docs.

Live-validated on this box (synthetic host + probe --mic-test,
pw-record): eager node, both attach orderings, PipeWire-restart
self-heal, post-session silence. Windows side compile/CI + on-glass
validation pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:41:19 +00:00
enricobuehler b7048446c4 fix(windows-host): IDD-push compose kick — idle desktop no longer fails the attach gate
windows-drivers / probe-and-proto (push) Successful in 24s
apple / swift (push) Successful in 1m8s
ci / rust (push) Successful in 1m42s
windows-drivers / driver-build (push) Successful in 1m45s
ci / web (push) Successful in 54s
android / android (push) Successful in 3m39s
ci / docs-site (push) Successful in 1m8s
deb / build-publish (push) Successful in 4m40s
ci / bench (push) Successful in 4m58s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
decky / build-publish (push) Successful in 25s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
release / apple (push) Successful in 8m9s
windows-host / package (push) Successful in 7m35s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m27s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m11s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 54s
flatpak / build-publish (push) Successful in 4m26s
apple / screenshots (push) Successful in 5m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m46s
docker / deploy-docs (push) Successful in 24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m21s
DWM presents a display only when something dirties it. On an idle desktop a
perfectly healthy session sat at E_PENDING: the driver attached but no
first frame ever landed, so wait_for_attach's 4 s gate failed the open (and
a mid-session ring recreate hit the same stall against the 3 s
recover-or-drop). A real client escaped only because its own input soon
dirtied the desktop; a headless probe / input-less connect never did.

kick_dwm_compose() injects two net-zero 1 px relative mouse moves via
SendInput — pf-vdisplay has no hardware-cursor plane, so a cursor move is
composited into the frame, a guaranteed real present onto the IDD
swap-chain (the mechanism --input-test always relied on; the pointer ends
where it started). Wired into wait_for_attach (first kick at 600 ms, then
every 800 ms) and, rate-limited, into the GB1 recovery window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 19:24:57 +00:00
enricobuehler 3039626b87 design(host): add vrr design doc 2026-07-03 19:22:19 +00:00
enricobuehler 3f33ed30ae fix(windows-host): claim the vdisplay single-instance guard eagerly at serve startup
On-glass the lazy (first-session) claim let a second host started while the
freshly-restarted service sat idle win the mutex and ADD a monitor on the
real driver — priority backwards. The claim is now a process-global,
retryable slot (a failed claim is not memoized, so it heals once the other
instance exits), and `serve` claims it before any client can connect;
ensure_device keeps the lazy claim for standalone punktfunk1-host runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:57:47 +00:00
enricobuehler 7e31020c1c fix(windows-host): second-host guard — classify ACCESS_DENIED on the instance mutex as in-use
On-glass the SCM service creates Global\punktfunk-vdisplay-manager as
SYSTEM, so a second elevated-admin host's CreateMutexW fails ACCESS_DENIED
(the implicit open is checked against the SYSTEM DACL) before the
ALREADY_EXISTS branch can fire — right refusal, wrong message. Map it to
the same loud "another instance is live" error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:53:54 +00:00
enricobuehler fe54aff658 fix(windows-host): cross-plane IDD serialization, linger-expiry race, second-host guard
Batch C of the audit's medium tier (M7+M8+M9):

- M7: GameStream sessions now run the same begin_idd_setup dance as
  punktfunk/1 before creating the shared monitor. A GS connect could
  previously ADD/reconfigure the monitor while a native session was
  mid-build (and vice versa), and its sealed-channel delivery replaced the
  native ring (newest-wins) — each plane could freeze the other. GS has no
  cooperative stop plumbing, so it registers a flag nobody reads: a later
  session signals it, waits the 3 s grace, then force-preempts — the
  intended handover.
- M8: the linger-expiry teardown now runs UNDER the state lock. Running it
  outside let a concurrent acquire see Idle and ADD+isolate while the old
  monitor's pinger-join / CCD-restore / REMOVE were still in flight — a
  failed or de-isolated session exactly at the expiry boundary. A racing
  acquire now waits the few teardown seconds instead. Lock order stays
  state → device everywhere; the pinger takes only the device lock.
- M9: a named mutex (Global\punktfunk-vdisplay-manager) makes a SECOND host
  process fail its vdisplay open loudly instead of firing a startup
  CLEAR_ALL that razes the live host's monitors mid-stream (the admin
  footgun the shared watchdog then masked).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:28:22 +00:00
enricobuehler b46aa15afb fix(windows-drivers): pf-vdisplay robustness — AdapterInitStatus gate, pooled-device TDR check, MMCSS-optional worker
Batch B of the audit's medium tier (M4+M5+M6):

- M4: adapter_init_finished now reads AdapterInitStatus (was ignored) and
  only stashes the adapter on NT_SUCCESS, per the MS sample. A failed async
  init previously produced a HUSK adapter: monitors created on it arrive
  but the OS never assigns a swap-chain — every session black-screens with
  no visible cause (the exact signature live fault-injection produced after
  a WUDFHost kill). Unset adapter → ADD fails cleanly (host-retryable) and
  a re-entrant D0 retries the init; the status is now in the debug log.
- M5: pooled_device checks GetDeviceRemovedReason on a cache hit — a TDR'd
  device was returned for its LUID forever (SetDevice fail-loop, black
  virtual display until device teardown); now it falls through to a fresh
  create.
- M6: an AvSetMmThreadCharacteristicsW failure no longer aborts the worker
  before draining (which stalled the monitor and leaked the WDF swap-chain
  object) — continue unprioritized like the MS sample; revert only if MMCSS
  actually engaged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:20:48 +00:00
enricobuehler 058630f542 feat(decky): visible branded Steam shortcut, one-tap client updates, fullscreen-page polish
- The "Punktfunk" shortcut is no longer hidden: it now ships committed
  artwork (grid/wide/hero/logo/icon, generated by scripts/gen-steam-art.py
  — a pure-stdlib SDF renderer drawing the lens mark + a monoline
  "punktfunk" wordmark) applied via SetCustomArtworkForApp /
  SetShortcutIcon. Existing installs are unhidden and re-arted once per
  ART_VERSION; relaunching the library entry streams to the last host.
- Updates cover the flatpak CLIENT too: check_update compares the
  user-scope installed commit against its remote, applyUpdate runs
  `flatpak update --user` first (awaited) and the plugin reinstall —
  which reloads the panel — last; docs spell out the sudo-less --user
  update ("sudo flatpak update" silently skips per-user installs).
- Fullscreen page: DialogButton stretches to 100% width in the gamepad
  UI, so the Stream/Pair/Refresh/… actions filled whole rows — sized to
  content + right-aligned now; the header drops its Update button (About
  tab + QAM banner keep the flow) and the back button gets a real 40px
  hit target.
- Settings: the disable-Steam-Input note also shows for Automatic — on a
  Deck that now forwards the built-in controller as a Steam Deck pad
  (paddles/trackpads/gyro), which needs Steam Input off for the shortcut.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:16:40 +00:00
enricobuehler e9c1f4083a fix(client-linux): Deck Gaming Mode — auto pad type, real chrome-less fullscreen, leave-to-Gaming-Mode, colour bisect
- "Automatic" gamepad type resolves to the virtual Steam Deck pad on Deck
  hardware (env SteamDeck / DMI Jupiter|Galileo): the built-in 28DE:1205
  identity is invisible at Hello time — the Valve HIDAPI drivers run
  in-session only and Steam Input shadows the pad with its virtual X360 —
  so auto always fell through to Xbox 360. "steamdeck" is now also
  selectable in Settings.
- Chrome-less launches flatten the window CSS (border-radius/box-shadow)
  and fullscreen at startup: gamescope never ACKs the xdg fullscreen
  state, so adwaita kept the floating-CSD rounded corners + shadow
  visible over the stream.
- Gaming-Mode --connect launches quit on session end, so Steam ends the
  "game" and the Deck returns to Gaming Mode — previously the app popped
  to its own hosts page, stranding the user fullscreen and making the
  escape chord read as broken.
- The capture hint is controller-aware; the chromeless hint teaches the
  hold-chord ("hold L1+R1+Start+Select to leave") and a quick chord press
  re-flashes it.
- Colour bisect for the reported off-colours on the VAAPI dmabuf path:
  graphics offload defaults OFF under gamescope (a subsurface hands the
  NV12 CSC to the compositor), PUNKTFUNK_OFFLOAD=1|0 overrides, and each
  colour-signaling change logs whether GDK accepted the BT.709-narrow
  color state (fallback = GDK's BT.601 dmabuf default).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:16:26 +00:00
enricobuehler 20f0d2802f feat(client/android): Snapdragon latency tuning — ADPF pipeline hints, game mode, max-clock decode
Three levers to lower and steady decode latency on Snapdragon (Adreno) devices:

- ADPF (Adaptive Performance Framework): a new dlsym-resolved hint session
  (native/src/adpf.rs; API-33+, resolved at runtime so there's no build-time
  link dependency and libpunktfunk_android.so still loads on API 31/32) tells
  the CPU governor the video pipeline runs a per-frame real-time workload, so it
  keeps those threads on fast cores at high clocks. It now covers all three
  latency-critical threads — the pf-decode feed/drain/present loop, the core
  data-plane pump (UDP receive + FEC reassembly), and the audio thread — via a
  new generic hot-thread registry on NativeClient (register_hot_thread /
  hot_thread_ids; the pump self-registers). The session is built lazily on the
  first presented frame, since ADPF createSession rejects a set containing any
  not-yet-live tid.

- operating-rate -> Short.MAX ("as fast as possible"): pushes the Qualcomm
  decoder to run each frame at max clocks instead of merely sustaining the
  display rate at a power-saving clock that adds per-frame decode latency.

- appCategory="game": makes the app eligible for OEM Game Mode / Game Dashboard
  performance profiles.

The core registry is cross-platform (gettid on Linux/Android, a no-op
elsewhere) — no Android-specific pollution of the shared core. Host workspace +
64 core tests green; Android arm64-v8a + x86_64 (platform 31) build + clippy
clean. On-device Snapdragon validation pending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 17:16:11 +00:00
enricobuehler 6f8fb15c9b fix(windows-host): self-heal the hostless-zombie pf-vdisplay device (adapter cycle + re-probe)
Fault-injection on-glass showed a killed/crashed WUDFHost leaves the devnode
"started" but HOSTLESS: PnP Status OK, no WUDFHost process, zero device-
interface instances — is_available() then fails every future session at the
vdisplay::open gate (and a reopen inside VdisplayDriver::open finds nothing),
until something cycles the device. Port reset-pf-vdisplay.ps1's adapter
disable→enable step in-process (restart_vdisplay_device): the open gate now
uses ensure_available() (cycle once + bounded re-probe; a genuinely
uninstalled driver — no adapter devnode — still fails fast), and
VdisplayDriver::open retries open_device over a short arrival window after a
cycle, covering the manager's reopen path too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:12:43 +00:00
enricobuehler 89455032a0 fix(windows-host): IDD-push resilience — driver-death recovery, reopenable control device, full interface discovery
Batch A of the audit's medium tier (M1+M2+M3):

- M1 driver-death detection: a dead WUDFHost stops publishing, which at the
  ring is indistinguishable from an idle desktop — SDR sessions streamed a
  frozen frame forever (next_frame's 20 s bail is unreachable once anything
  presented). The ChannelBroker's process handle now doubles as a liveness
  probe (SYNCHRONIZE at OpenProcess); while no fresh frame arrives,
  try_consume polls it (rate-limited) and fails the capturer, landing in the
  session's bounded in-place rebuild.
- M2 reopenable control device: the manager's OnceLock-cached handle is now
  a retire/reopen DeviceSlot — a gone-classified IOCTL failure (driver
  upgrade / WUDFHost restart; pinger, create, or REMOVE) retires the handle
  and the next use reopens + re-handshakes. Retired handles are deliberately
  kept alive forever: bare-HANDLE holders (pinger, ChannelBroker) rely on
  never-closed, and a retired handle only fails IOCTLs. CLEAR_ALL runs on
  the FIRST open only (a reopen races live-ish sessions); acquire retries
  the monitor create once after a reopen. The JOIN path now probes the
  active monitor's WUDFHost pid and preempts a DEAD monitor instead of
  handing the rebuilding session its stale target — without this the whole
  recovery chain starved to the rebuild budget.
- M3 interface discovery: enumerate ALL interface instances with an
  SPINT_ACTIVE filter (a Code-10 devnode at index 0 no longer shadows the
  live interface), HDEVINFO behind RAII (error paths leaked one per probe),
  the raw device handle wrapped before GET_INFO (leaked on handshake
  failure), and the detail-sizing result guarded before the cbSize write.
- pf-driver-proto: SetFrameChannelRequest doc now states the real
  adopt-on-success contract (the old wording invited a driver-side
  close-on-error — a cross-process double-close against the host's reap).
- install: pf_vdisplay_present() passes /connected so a phantom devnode
  can't suppress creating a live ROOT node.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:04:19 +00:00
enricobuehler 0da9d8ec10 fix(windows): IDD-push audit highs — keyed-mutex timeout, two per-frame leaks, IDD_PUSH knob, pooled-device threading
Five verified findings from the IDD-push/pf-vdisplay deep audit:

- Keyed-mutex acquire (BOTH endpoints): AcquireSync returns WAIT_TIMEOUT
  (0x102) / WAIT_ABANDONED (0x80) as SUCCESS-severity HRESULTs, which the
  windows-rs Result wrapper erases — a busy slot read as "acquired", so
  driver and host could race the same ring texture (torn frames) and the
  designed busy-skip backpressure was dead code. Both sides now classify
  the raw vtable HRESULT; WAIT_ABANDONED counts as acquired (ownership
  transfers — refusing it would wedge the slot forever).
- Host SDR hot path leaked one ID3D11VideoProcessorInputView per converted
  frame: the D3D11_VIDEO_PROCESSOR_STREAM ManuallyDrop field suppressed the
  release after VideoProcessorBlt. Released by hand now, success or not.
- Driver leaked IddCx's per-acquire surface reference (from_raw_borrowed on
  a TRANSFERRED reference — the MS sample Attach/Reset's it): the swap-chain
  surface set survived swap-chain destruction, the likely true root cause of
  the ~50 MB-per-reconnect VRAM loss that device pooling only mitigated.
  Now adopted via from_raw (publisher or not) and dropped pre-Finished.
- PUNKTFUNK_IDD_PUSH removed: capture is unconditionally IDD-push, but the
  vdisplay manager still gated the lingering-monitor preempt (and render
  pin) on the knob, whose default was OFF — dev/CLI runs reused a lingering
  monitor whose IddCx swap-chain is dead (black reconnect). The preempt and
  the render-GPU pin are now unconditional; host.env comments no longer
  promise the removed DDA/WGC fallback.
- Driver D3D device: dropped D3D11_CREATE_DEVICE_SINGLETHREADED (unsound
  since DEVICE_POOL shares one device across processors) and the pooled
  immediate context is now SetMultithreadProtected — two concurrent
  monitors' workers otherwise race an unlocked context (UB in the UMD).

No wire-contract change (pf-driver-proto untouched); the driver fixes take
effect on the next pf-vdisplay redeploy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:27:13 +00:00
enricobuehler fbf3fea0c8 fix(packaging/windows): Inno [Code] comment brace-nesting trap broke ISCC
apple / swift (push) Successful in 1m17s
windows-host / package (push) Successful in 7m3s
apple / screenshots (push) Successful in 5m37s
android / android (push) Successful in 3m22s
ci / web (push) Successful in 48s
ci / rust (push) Successful in 1m18s
ci / bench (push) Successful in 4m59s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 4m35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m31s
docker / deploy-docs (push) Successful in 15s
The PublicFwParam doc comment contained a literal code-constant token; Inno's
{ } comments don't nest, so its closing brace ended the comment early and the
trailing text parsed as code ("'BEGIN' expected", compile aborted). Reworded to
avoid the literal braces + added a warning note. Verified: the [Code] section
has no other nested-brace-in-comment traps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:27:27 +00:00
enricobuehler c52ae119e1 fix: regenerate Cargo.lock for punktfunk-host's winresource build-dep
apple / swift (push) Successful in 1m10s
audit / cargo-audit (push) Successful in 1m12s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m8s
android / android (push) Successful in 3m49s
ci / web (push) Successful in 48s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 45s
ci / docs-site (push) Successful in 56s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 50s
release / apple (push) Successful in 8m51s
ci / rust (push) Successful in 5m51s
ci / bench (push) Successful in 4m51s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
deb / build-publish (push) Successful in 4m46s
apple / screenshots (push) Successful in 5m34s
flatpak / build-publish (push) Successful in 5m33s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m52s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m43s
docker / deploy-docs (push) Failing after 18s
windows-host / package (push) Has been cancelled
The "Punktfunk Host" identity work added winresource to the host crate but
didn't update the lock, so every --locked CI job failed to resolve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:17:31 +00:00
enricobuehler 5d7aabe8f0 fix(scripts/windows): keep deploy-host.ps1 pure ASCII (em-dash -> hyphen)
apple / swift (push) Successful in 1m8s
ci / rust (push) Failing after 50s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m8s
android / android (push) Successful in 3m24s
deb / build-publish (push) Failing after 45s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
windows-host / package (push) Failing after 6m18s
apple / screenshots (push) Successful in 5m35s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m26s
docker / deploy-docs (push) Successful in 17s
Two comment em-dashes I added tripped the installer-run ASCII guard (PS 5.1
mis-parses non-ASCII on non-UTF-8 locales).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:09:17 +00:00
enricobuehler f204a89cef perf(encode/windows): AMF quality=speed + bf=0; drop the useless poll spin
ci / rust (push) Failing after 48s
windows-host / package (push) Failing after 10s
apple / swift (push) Successful in 1m6s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m8s
android / android (push) Successful in 3m20s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
apple / screenshots (push) Successful in 5m10s
ci / bench (push) Successful in 4m43s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m25s
deb / build-publish (push) Failing after 44s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m21s
On-box A/B on the .173 Ryzen 7000 iGPU (720p60, real composition via input
injection — an idle virtual desktop composes ~1 fps and gives meaningless
encode timings): the encode-time-first `quality=speed` preset + explicit `bf=0`
cut host-side encode_us from ~36 ms to ~19.5 ms.

The blocking-poll idea from the prior commit was WRONG and is reverted to a
single non-blocking receive (default PUNKTFUNK_FFWIN_POLL_MS=0): libavcodec's
hevc_amf holds ~2 frames before releasing the oldest (needs frame N+2 to flush
N), so a spin between submits provably never yields the owed AU — verified with
a 150 ms cap pegging at exactly 150 ms across every usage preset and pipeline
depth. That ~2-frame buffer is inherent to the libavcodec wrapper, not host
scheduling; the real latency lever is a direct AMF SDK encoder (the AMF
analogue of the direct-NVENC path), tracked as the next AMD work item. The
env knob is retained for a future VCN/driver where a bounded spin can help.

Also measured and rejected: PUNKTFUNK_ZEROCOPY=1 on AMF is ~2x WORSE (68 ms vs
36 ms) — the D3D11 import path adds sync overhead beyond the readback it saves,
so the system-memory default stays. GPU-priority elevation is already
process-wide (dxgi.rs), so it covers the iGPU encode session with no change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:57:39 +00:00
enricobuehler 24fa018c70 chore(encode/windows): AMF forensics knobs — PUNKTFUNK_AMF_USAGE + PUNKTFUNK_FFWIN_POLL_MS
apple / swift (push) Successful in 1m6s
ci / web (push) Successful in 53s
deb / build-publish (push) Failing after 44s
windows-host / package (push) Failing after 10s
ci / rust (push) Failing after 49s
android / android (push) Successful in 3m33s
apple / screenshots (push) Successful in 5m18s
ci / docs-site (push) Successful in 57s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 5m0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m27s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m18s
The blocking poll landed but wait_us pegs at exactly the 2-frame-period cap:
AMF holds the AU ~2 frame periods regardless of retrieval. Field knobs to
bisect on-box (usage preset × poll cap) without rebuild cycles.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:39:36 +00:00
enricobuehler 51a6ca7e02 fix(encode/windows): AMF latency — honor the loop's blocking-poll contract + preset polish
apple / swift (push) Successful in 1m6s
windows-drivers / driver-build (push) Successful in 1m34s
windows-drivers / probe-and-proto (push) Successful in 20s
ci / rust (push) Failing after 47s
ci / web (push) Successful in 52s
windows-host / package (push) Failing after 11s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 3m20s
deb / build-publish (push) Failing after 46s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
apple / screenshots (push) Successful in 5m11s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m27s
ci / bench (push) Successful in 4m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m24s
The session loop's pipeline deferral was designed around direct NVENC, whose
poll() BLOCKS in lock_bitstream; libavcodec's AMF wrapper is truly async
(EAGAIN until the ASIC finishes), so a single non-blocking receive quantized AU
retrieval to the submit cadence: +1–2 frame periods flat (~43 ms p50 at 720p60
on the Ryzen iGPU vs ~3.5 ms of actual encode). FfmpegWinEncoder now tracks
in-flight frames and, while an AU is owed, spin-polls with short sleeps bounded
to ~2 frame periods (an overloaded encoder degrades to next-tick pickup instead
of stalling capture). Also: quality=speed (latency-first, iGPU-class VCN),
explicit bf=0 (h264_amf defaults >0 on RDNA3+), AMF low-latency submission
mode (FFmpeg ≥6.1, ignored on older).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:32:41 +00:00
enricobuehler b9fde03f1e feat(security): finish Windows firewall Public opt-in wiring + vuln-disclosure + doc cleanup
Firewall (the service.rs core landed in efb1ba2): scope the web-console rule
(TCP 47992) to Domain+Private by default with a `--allow-public-network` opt-in
that deletes-then-re-adds the rule, and add the installer "Allow connections on
Public networks" task (unchecked) forwarding the flag to `service install` and
`web setup`. Default is now trusted-networks-only; Public is explicit.

Vulnerability disclosure: SECURITY.md (report to security@punktfunk.com, scope,
SLAs, safe harbor), a Gitea issue-template contact link, a README security line,
and a Reporting section on the docs Security page.

Docs: the Security page now documents the Private/Domain firewall default (and
how to fix a misclassified-Public network / opt in); removed internal design-doc
and CLAUDE.md links from the user-facing docs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 14:08:17 +00:00
enricobuehler efb1ba26d7 fix(windows): opt-in pad-driver file logs + size-capped service log rotation
Two disk-write fixes:

- pf-xusb/pf-dualsense no longer write C:\Users\Public\pf*-driver.log
  unconditionally — the file log is now opt-in (debug builds, or the
  PFXUSB_DEBUG_LOG / PFDS_DEBUG_LOG system env var), mirroring the audit-§4.4
  fix pf-vdisplay already got: a release driver never writes the world-writable
  Public file (info-leak/DoS surface), and the per-report OUTPUT/SET_STATE hex
  dumps stop being a sustained per-rumble disk-write path during gameplay.
  OutputDebugStringA stays unconditional; the host's driver-silence WARN and
  the gamepad-driver-health failure-mode table now say the log is opt-in.

- service.log/host.log get one-generation rotation: at each (re)open a file
  over 10 MB is renamed to .old, so a crash-restart loop or a RUST_LOG=debug
  left in host.env can't grow the append-forever logs without bound. Rotation
  runs only before an open (never under a live appender — host.log's handle
  lacks FILE_SHARE_DELETE, so a racing rename harmlessly fails).

Windows CI compile/clippy pending (drivers workspace + host are not
Linux-cross-checkable); rides along with the next pad-driver redeploy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:03:32 +00:00
enricobuehler 1320e3dc66 fix(scripts/windows): deploy-host.ps1 builds all-vendor when an FFmpeg tree exists
windows-host / package (push) Failing after 20s
apple / swift (push) Successful in 1m9s
apple / screenshots (push) Successful in 5m26s
ci / rust (push) Failing after 48s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 3m22s
deb / build-publish (push) Failing after 43s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m40s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 3m27s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m18s
The dev deploy built --features nvenc only, so a web-console GPU preference
pointing at an AMD/Intel adapter made every session die at encoder open
(NV_ENC_ERR_NO_ENCODE_DEVICE) — the exact "can't connect" just hit on the RTX
box's Ryzen iGPU. The script now enables amf-qsv when FFMPEG_DIR (machine env,
process env, or C:\Users\Public\ffmpeg) has a dev tree, and copies the FFmpeg
runtime DLLs next to the exe after a successful build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:52:55 +00:00
enricobuehler 1be83575b6 feat(host/windows): "Punktfunk Host" identity in Task Manager (icon + version info)
punktfunk-host.exe embedded no icon or version resources, so Task Manager and
Explorer showed a bare lowercase exe name with a generic icon. build.rs now
embeds the branded .ico + FileDescription "Punktfunk Host" / ProductName
"Punktfunk" via winresource (same pattern as the Windows client and the tray;
Linux packaging builds skip the block). The tray gets a matching "Punktfunk
Tray" description, and the SCM display name moves off lowercase
"punktfunk streaming host" to "Punktfunk Host" (applied idempotently by
`service install` on upgrade).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:52:55 +00:00
enricobuehler 4d1d20f832 chore: untrack CLAUDE.md
apple / swift (push) Successful in 1m14s
apple / screenshots (push) Successful in 5m45s
ci / rust (push) Successful in 1m15s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m5s
android / android (push) Successful in 3m25s
deb / build-publish (push) Successful in 4m35s
ci / bench (push) Successful in 4m55s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m27s
docker / deploy-docs (push) Successful in 18s
Local per-box assistant instructions (incl. internal environment detail) don't
belong in the published tree; the file stays on disk, now gitignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:22:23 +00:00
enricobuehler 6e875fea44 fix(apple/ios): the ACTUAL type-checker bomb was pointerSection's footer ternary chain
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 8m36s
apple / screenshots (push) Successful in 5m44s
ci / rust (push) Successful in 1m31s
ci / web (push) Successful in 56s
android / android (push) Successful in 10m1s
deb / build-publish (push) Successful in 4m33s
ci / bench (push) Successful in 4m52s
ci / docs-site (push) Successful in 1m24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m20s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m19s
docker / deploy-docs (push) Failing after 14s
4f3cd24 split the wrong expression — act's log masking hid the real line number.
The unmasked retry pinpointed it: the pointerSection footer, a ten-segment
string + chain with an isPad ternary nesting four more, evaluated inside the
ViewBuilder. Moved the copy into a plain computed String built with +=
statements (linear to type-check); no text change. The two remaining 5-6
segment chains in Settings are compiled by the passing macOS slice, so they
are proven cheap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:17:01 +00:00
enricobuehler 4f3cd24036 fix(apple/ios): split streamModeSection — the inline iOS branch blew the type-checker budget
release / apple (push) Successful in 6m29s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m13s
apple / swift (push) Successful in 1m5s
apple / screenshots (push) Successful in 4m5s
ci / bench (push) Successful in 4m36s
ci / rust (push) Successful in 11m23s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 4m26s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m45s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m37s
android / android (push) Successful in 9m37s
docker / deploy-docs (push) Successful in 19s
The Section's iOS content (resolution wheel + 3-way refresh rows + bitrate
rows) as ONE ViewBuilder expression hit "the compiler is unable to type-check
this expression in reasonable time" — failing exactly one build slice, the iOS
archive, so swift test (macOS) and the tvOS/macOS archives never saw it and the
0.6.0 iOS TestFlight upload soft-failed. Extracted iosResolutionWheel /
iosRefreshRows / bitrateRows; no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 13:06:48 +00:00
enricobuehler af13f0b749 chore(release): 0.6.0
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
android-screenshots / screenshots (push) Successful in 2m18s
android / android (push) Successful in 4m13s
decky / build-publish (push) Successful in 26s
windows-host / package (push) Successful in 6m36s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m10s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m50s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 35s
release / apple (push) Successful in 7m53s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m32s
deb / build-publish (push) Successful in 9m52s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
web-screenshots / screenshots (push) Successful in 2m38s
ci / web (push) Successful in 48s
ci / rust (push) Successful in 11m43s
linux-client-screenshots / screenshots (push) Successful in 1m33s
flatpak / build-publish (push) Successful in 4m8s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m10s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m49s
docker / deploy-docs (push) Successful in 25s
ci / docs-site (push) Successful in 57s
ci / bench (push) Successful in 5m9s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:19:18 +00:00
enricobuehler d285d4a0b2 fix(tray): live-probe the web console instead of sniffing the install layout
windows-drivers / probe-and-proto (push) Successful in 29s
audit / cargo-audit (push) Successful in 1m31s
apple / swift (push) Successful in 1m8s
windows-drivers / driver-build (push) Successful in 1m35s
android / android (push) Successful in 4m45s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m0s
release / apple (push) Successful in 7m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
The "Open web console" entry was gated on {exe dir}\web\web-run.cmd (Windows)
/ the punktfunk-web unit file (Linux) — which misses consoles run from a repo
checkout (the RTX box, caught on-glass) and shows a dead entry while an
installed console is stopped. The poller now probes https://127.0.0.1:<web
port>/ each cycle (any HTTP response = up, transport failure = down) and the
menu follows live on both platforms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:17:01 +00:00
enricobuehler 04f370999c fix(web): pin the sidebar at viewport height
Sticky h-dvh sidebar: long pages scroll the content, not the nav — the flex
stretch was pushing the language switcher below the fold; overflow-y-auto keeps
the nav usable on short viewports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:52 +00:00
enricobuehler 2c937855b3 fix(packaging/windows): Windows 11 22H2 floor + tray install task + stale console-port fixes
The OS floor is now enforced at install time (MinVersion=10.0.22621 with an
explanatory [Messages] override): pf-vdisplay is built against IddCx 1.10, and
on Windows 10 (incl. LTSC) / Win11 21H2 the device fails start with Code 10
STATUS_DEVICE_POWER_FAILURE (field-reported). Docs (site requirements/install/
windows-host pages + README) state the floor; new docs-site Security page.

Installer also gains the trayicon task (punktfunk-tray.exe file + HKLM Run key,
post-install launch as the signed-in user, upgrade taskkill + uninstall
--quit/taskkill choreography before file deletion), and the wizard/cleanup
text/port sweeps move off the stale :3000 web-console references to :47992
(cleanups sweep both for upgrades from old installs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:52 +00:00
enricobuehler 8005b11faf feat(tray): system-tray status icon for the host (Windows + Linux)
New crates/punktfunk-tray — a small per-user companion showing the host service
state at a glance (running / stopped / starting / degraded / failed + the live
session in the tooltip) with one-click actions: open web console, approve a
pending pairing request, start/stop/restart, open logs. No more digging through
logs to learn whether the service came back after a reboot or an update.

Status is service-manager-FIRST (SCM / systemd user unit — a port squatter can
never fake Running), then the new loopback-only unauthenticated
GET /api/v1/local/summary (counts/booleans only; the mgmt token and cert.pem
are SYSTEM/Admins-DACL'd on Windows, so a non-elevated tray cannot bearer-auth).

Windows: windows_subsystem binary (a console exe in the Run key would flash a
terminal at sign-in), Shell_NotifyIcon + hidden window, per-session single
instance, TaskbarCreated re-add, --quit for the uninstaller; service actions
elevate per click via ShellExecuteW "runas" onto the new
`punktfunk-host service restart` (stop → wait Stopped → start).
Linux: ksni/StatusNotifierItem over zbus, systemctl --user actions (no polkit),
/etc/xdg/autostart entry whose --autostart self-gates to actual host users.
Icons: scripts/gen-tray-icons.py (pure stdlib) renders the brand lens + status
dot into committed .ico/hicolor assets; deb/rpm/arch ship binary+autostart+icons.

Live-validated: Linux on the headless KDE session (SNI registration, state
transitions, menu-driven start, dbusmenu layout); Windows on the RTX box
(session-1 launch with no NIM_ADD failure, single instance, --quit, restart
round-trip, summary loopback-200/LAN-401).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:35 +00:00
enricobuehler 01fcb01019 fix(encode/windows): resolve NVENC at runtime — AMD/Intel hosts no longer crash at start
The nvenc build linked nvEncodeAPI64.dll's entry points at load time, so a
--features nvenc binary hard-crashed on any box without the NVIDIA driver
(AMD/Intel). Entry points now come from a runtime LoadLibrary table
(encode/windows/nvenc.rs load_api); a missing DLL just falls through the
encoder auto-detect to AMF/QSV/software. The generated import lib and all its
plumbing (gen-nvenc-importlib.ps1, nvenc.def, PUNKTFUNK_NVENC_LIB_DIR,
setup-build-env wiring) are gone.

Live-validated on the RTX 4090 box (NVENC session, 7000+ frames).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:09:18 +00:00
enricobuehler 95a08e99c3 feat(host/windows): seal the host↔driver channels (frame + gamepad, proto v2)
Frame ring (pf-vdisplay) and both gamepad SHM channels move off named Global\
objects (openable by any sibling LocalService) to UNNAMED sections/events whose
handles the host DuplicateHandles into the driver's verified WUDFHost with least
access — frame delivery over the SYSTEM+admins-only IOCTL_SET_FRAME_CHANNEL,
pads over a 32-byte named bootstrap mailbox (pid + handle value only, DoS-bounded;
HID minidrivers have no control device). Driver-validated pad_index kills
cross-pad redirects; v1↔v2 mixes fail closed with diagnosis logs on both sides.
Sibling-LocalService denial proven empirically (design/idd-push-security.md,
design/gamepad-channel-sealing.md).

Driver-side raw ops now live behind pf-umdf-util (checked shm accessors, the
forbid(unsafe_code) ChannelClient state machine, WDF request tokens) — the pad
drivers' logic is 100% safe Rust; whole drivers workspace clippy-gated in CI.

driver install --gamepad now sweeps SWD\punktfunk phantom devnodes: a re-created
SwDevice REVIVES the old devnode with its previously-bound driver (never
re-ranks), so an upgrade otherwise leaves the old driver serving — or, across
the v1→v2 fence, a dead pad (found live on the RTX box).

On-glass validated on the RTX 4090 box: frame path 7007 frames p50 2.06 ms
cross-machine; DualSense + XUSB "sealed pad channel mapped"/proto=2 attach via
both the test harness and a real streaming session; phantom-sweep repro.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:08:56 +00:00
enricobuehler a3e1ea2b44 fix(android/ci): retry transient Play API failures in play-upload.py
apple / swift (push) Successful in 1m9s
apple / screenshots (push) Successful in 4m2s
android / android (push) Successful in 11m51s
ci / web (push) Successful in 1m0s
ci / docs-site (push) Successful in 1m13s
ci / rust (push) Successful in 4m30s
deb / build-publish (push) Successful in 3m35s
ci / bench (push) Successful in 4m47s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
decky / build-publish (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m3s
docker / deploy-docs (push) Successful in 20s
The uploader only caught HTTPError — a URLError (TLS "EOF occurred in
violation of protocol", the failure that dropped two release uploads on
2026-07-02) or a Google 5xx killed the job outright. Retry those with
3/9/27 s backoff; 4xx still fails fast. The edits API is transactional
until commit, so re-sending is safe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:05:27 +00:00
enricobuehler 6686fcdded fix(gamestream/tests): sender_delivers_batches flaked under CI load — burst overflowed the default socket buffer
apple / swift (push) Successful in 1m12s
apple / screenshots (push) Successful in 4m26s
windows-host / package (push) Successful in 6m25s
ci / rust (push) Successful in 5m5s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m4s
android / android (push) Failing after 10m7s
deb / build-publish (push) Successful in 3m35s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
ci / bench (push) Successful in 4m38s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m53s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m53s
docker / deploy-docs (push) Successful in 18s
The test burst 3×100 1200 B datagrams into an undrained loopback socket: at
~2.5 KB kernel truesize each, the default ~212 KB rmem holds only ~80, so on
a starved CI runner (parallel release builds) the kernel silently dropped the
overflow and the recv loop could never reach 300 — surfacing as WouldBlock
after the 3 s timeout. Size the burst (3×20) to fit the default buffer even
with zero concurrent draining, and give recv a starvation-tolerant 10 s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:35:23 +00:00
214 changed files with 15243 additions and 3556 deletions
+9
View File
@@ -0,0 +1,9 @@
# Shown on the "new issue" chooser so security reports go to the private channel, not a public issue.
blank_issues_enabled: true
contact_links:
- name: 🔒 Report a security vulnerability
url: https://git.unom.io/unom/punktfunk/src/branch/main/SECURITY.md
about: >-
Found a security issue? Please report it privately by email to security@punktfunk.com — do not
open a public issue, so other users aren't exposed before a fix ships. See SECURITY.md for the
full policy.
+2 -1
View File
@@ -78,9 +78,10 @@ jobs:
- name: Version + channel - name: Version + channel
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')) if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
run: | run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
case "$GITHUB_REF" in case "$GITHUB_REF" in
refs/tags/v*) VN="${GITHUB_REF_NAME#v}"; TRACK="alpha" ;; # alpha = built-in closed testing refs/tags/v*) VN="${GITHUB_REF_NAME#v}"; TRACK="alpha" ;; # alpha = built-in closed testing
*) VN="0.5.0-ci${GITHUB_RUN_NUMBER}"; TRACK="internal" ;; *) VN="${PF_BASE}-ci${GITHUB_RUN_NUMBER}"; TRACK="internal" ;;
esac esac
echo "VERSION_NAME=$VN" >> "$GITHUB_ENV" echo "VERSION_NAME=$VN" >> "$GITHUB_ENV"
echo "PLAY_TRACK=$TRACK" >> "$GITHUB_ENV" echo "PLAY_TRACK=$TRACK" >> "$GITHUB_ENV"
+6 -5
View File
@@ -36,16 +36,17 @@ jobs:
- name: Version + channel - name: Version + channel
# vX.Y.Z tag -> X.Y.Z, published to the `stable` apt distribution (a real release). # vX.Y.Z tag -> X.Y.Z, published to the `stable` apt distribution (a real release).
# A main push -> 0.5.0~ciN.g<sha>, published to the `canary` distribution: the '~' sorts # A main push -> <next-minor>~ciN.g<sha>, published to the `canary` distribution: the '~' sorts
# below the eventual 0.5.0 tag, it climbs monotonically by run number, and the canary base # below the eventual tag, it climbs monotonically by run number, and the canary base is
# stays one minor AHEAD of the latest stable so a stable->canary box re-point still moves # derived one minor AHEAD of the latest stable tag (scripts/ci/pf-version.sh) so a
# forward (see channels.md). Computed BEFORE the build so it's stamped into the binary # stable->canary box re-point still moves forward (see channels.md). Computed BEFORE the build so it's stamped into the binary
# (PUNKTFUNK_BUILD_VERSION -> build.rs -> --version). # (PUNKTFUNK_BUILD_VERSION -> build.rs -> --version).
run: | run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8) SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; DIST=stable ;; refs/tags/v*) V="${GITHUB_REF_NAME#v}"; DIST=stable ;;
*) V="0.5.0~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;; *) V="${PF_BASE}~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;;
esac esac
echo "VERSION=$V" >> "$GITHUB_ENV" echo "VERSION=$V" >> "$GITHUB_ENV"
echo "DISTRIBUTION=$DIST" >> "$GITHUB_ENV" echo "DISTRIBUTION=$DIST" >> "$GITHUB_ENV"
+6 -2
View File
@@ -63,7 +63,8 @@ jobs:
pnpm run build # rollup -> clients/decky/dist/index.js pnpm run build # rollup -> clients/decky/dist/index.js
- name: Version + channel + stamp - name: Version + channel + stamp
# Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> 0.3.<run> # Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> <next-minor>.<run>
# (base one minor ahead of the latest stable tag via scripts/ci/pf-version.sh)
# (`canary/` alias). Decky reads a plugin's INSTALLED version from package.json (NOT # (`canary/` alias). Decky reads a plugin's INSTALLED version from package.json (NOT
# plugin.json), and the plugin's own update check (clients/decky/main.py check_update) # plugin.json), and the plugin's own update check (clients/decky/main.py check_update)
# compares against it — so the build version is STAMPED into package.json here (mirrored # compares against it — so the build version is STAMPED into package.json here (mirrored
@@ -72,9 +73,12 @@ jobs:
# (ci10 < ci9), which would break update detection; the run number is monotonic. # (ci10 < ci9), which would break update detection; the run number is monotonic.
working-directory: ${{ gitea.workspace }} working-directory: ${{ gitea.workspace }}
run: | run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_MAJOR/PF_MINOR (base one minor ahead of latest stable)
case "$GITHUB_REF" in case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; ALIAS=latest ;; refs/tags/v*) V="${GITHUB_REF_NAME#v}"; ALIAS=latest ;;
*) V="0.3.${GITHUB_RUN_NUMBER}"; ALIAS=canary ;; # Canary MUST be a plain monotonic numeric semver (see the note above): <major>.<minor>.<run>,
# where major.minor track one minor ahead of the latest stable and the run number climbs.
*) V="${PF_MAJOR}.${PF_MINOR}.${GITHUB_RUN_NUMBER}"; ALIAS=canary ;;
esac esac
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE" BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
echo "VERSION=$V" >> "$GITHUB_ENV" echo "VERSION=$V" >> "$GITHUB_ENV"
+48 -5
View File
@@ -73,15 +73,17 @@ jobs:
- name: Version + channel - name: Version + channel
# Tag vX.Y.Z -> X.Y.Z on the OSTree `stable` branch (a real release); a main push -> # Tag vX.Y.Z -> X.Y.Z on the OSTree `stable` branch (a real release); a main push ->
# 0.5.0-ciN.g<sha> on the `canary` branch. The two branches live side-by-side in one repo # <next-minor>-ciN.g<sha> on the `canary` branch (base one minor ahead of the latest stable
# tag via scripts/ci/pf-version.sh). The two branches live side-by-side in one repo
# (rsync runs without --delete), each tracked by its own .flatpakref, so `flatpak update` # (rsync runs without --delete), each tracked by its own .flatpakref, so `flatpak update`
# on a stable box never jumps to a canary build. The generic-registry version string allows # on a stable box never jumps to a canary build. The generic-registry version string allows
# letters/dots/hyphens. # letters/dots/hyphens.
run: | run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8) SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; BRANCH=stable; ALIAS=latest ;; refs/tags/v*) V="${GITHUB_REF_NAME#v}"; BRANCH=stable; ALIAS=latest ;;
*) V="0.5.0-ci${GITHUB_RUN_NUMBER}.g${SHORT}"; BRANCH=canary; ALIAS=canary ;; *) V="${PF_BASE}-ci${GITHUB_RUN_NUMBER}.g${SHORT}"; BRANCH=canary; ALIAS=canary ;;
esac esac
echo "VERSION=$V" >> "$GITHUB_ENV" echo "VERSION=$V" >> "$GITHUB_ENV"
echo "BUNDLE=punktfunk-client-${V}.flatpak" >> "$GITHUB_ENV" echo "BUNDLE=punktfunk-client-${V}.flatpak" >> "$GITHUB_ENV"
@@ -106,6 +108,40 @@ jobs:
python3 /tmp/flatpak-cargo-generator.py /tmp/Cargo.flatpak.lock \ python3 /tmp/flatpak-cargo-generator.py /tmp/Cargo.flatpak.lock \
-o packaging/flatpak/cargo-sources.json -o packaging/flatpak/cargo-sources.json
- name: Seed the local OSTree repo from the live server (keep BOTH channels in the summary)
# Each CI run builds only ONE branch (canary on main, stable on a tag). The deploy step's
# `flatpak build-update-repo` regenerates the repo SUMMARY from whatever refs are in the
# LOCAL repo, and the rsync publishes it (without --delete). A fresh single-branch local
# repo therefore produces a single-branch summary that CLOBBERS the other channel on the
# server — the exact bug that made `app/io.unom.Punktfunk/x86_64/stable` unresolvable
# ("No such ref") after a canary main-push overwrote the post-release summary, even though
# the stable commit's objects were still on disk. Fix: mirror the published repo DOWN first,
# so the local repo carries every existing branch; the build below then only ADDS this run's
# commit and the regenerated+signed summary keeps both channels. No-op on a fresh repo (first
# publish) or when the deploy secrets aren't set (the build still produces a valid bundle).
env:
DEPLOY_HOST: ${{ secrets.DEPLOY_HOST }}
DEPLOY_USER: ${{ secrets.DEPLOY_USER }}
DEPLOY_PORT: ${{ secrets.DEPLOY_PORT }}
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
run: |
set -euo pipefail
if [ -z "${DEPLOY_HOST:-}" ] || [ -z "${DEPLOY_SSH_KEY:-}" ]; then
echo "::warning::DEPLOY_* not set — no seed; building a fresh single-branch repo."
exit 0
fi
install -d -m700 ~/.ssh
printf '%s\n' "$DEPLOY_SSH_KEY" > ~/.ssh/deploy; chmod 600 ~/.ssh/deploy
SSH="ssh -i $HOME/.ssh/deploy -p ${DEPLOY_PORT:-22} -o StrictHostKeyChecking=accept-new"
DEST="${DEPLOY_USER}@${DEPLOY_HOST}"
mkdir -p "$PWD/repo"
# Pull the currently-published repo (all channels' objects + refs) into the repo the build
# will extend. No --delete: the local repo starts empty, so this only ADDS. A missing
# server repo (very first publish) is fine — we continue with a fresh repo.
rsync -az --info=stats1 -e "$SSH" "$DEST:$DEPLOY_DIR/site/repo/" "$PWD/repo/" \
|| echo "::warning::no published repo to seed (first publish?) — continuing fresh"
echo "seeded refs:"; ls "$PWD/repo/refs/heads/app/$APP_ID/x86_64/" 2>/dev/null || echo " (none)"
- name: Build the flatpak (install deps from Flathub, offline build) - name: Build the flatpak (install deps from Flathub, offline build)
run: | run: |
# --install-deps-from=flathub pulls everything the manifest declares: the GNOME 50 # --install-deps-from=flathub pulls everything the manifest declares: the GNOME 50
@@ -177,6 +213,10 @@ jobs:
--gpg-sign="$KEYID" --gpg-homedir="$GNUPGHOME" --gpg-sign="$KEYID" --gpg-homedir="$GNUPGHOME"
flatpak build-update-repo --generate-static-deltas \ flatpak build-update-repo --generate-static-deltas \
--gpg-sign="$KEYID" --gpg-homedir="$GNUPGHOME" "$PWD/repo" --gpg-sign="$KEYID" --gpg-homedir="$GNUPGHOME" "$PWD/repo"
# The regenerated summary advertises exactly these refs — must include EVERY channel that
# has ever published (the seed step ensures the other channel's commit is present). If this
# ever shows only one branch on a repo that had two, the seed didn't run — investigate.
echo "published summary advertises:"; ls "$PWD/repo/refs/heads/app/$APP_ID/x86_64/" 2>/dev/null || echo " (none)"
# 2) Build the install descriptors (GPGKey = the committed public key, base64). # 2) Build the install descriptors (GPGKey = the committed public key, base64).
GPGKEY="$(base64 -w0 packaging/flatpak/unom-flatpak.gpg)" GPGKEY="$(base64 -w0 packaging/flatpak/unom-flatpak.gpg)"
rm -rf site && mkdir -p site rm -rf site && mkdir -p site
@@ -188,9 +228,12 @@ jobs:
Comment=unom Flatpak applications Comment=unom Flatpak applications
GPGKey=$GPGKEY GPGKey=$GPGKEY
EOF EOF
# Two refs, one per channel — both regenerated every run and rsync'd without --delete, so # Two refs, one per channel. Both descriptor files are regenerated every run and rsync'd
# the server always offers both (the stable ref only resolves once a release has built the # without --delete; the repo SUMMARY carries both branches because the build was seeded
# `stable` branch). A box installs ONE; `flatpak update` then tracks that channel's branch. # from the live repo above (so build-update-repo below re-signs a summary listing every
# published channel, not just this run's). The stable ref resolves for good once any
# release has built the `stable` branch. A box installs ONE; `flatpak update` then tracks
# that channel's branch.
write_ref() { # <filename> <branch> <title> write_ref() { # <filename> <branch> <title>
cat > "site/$1" <<EOF cat > "site/$1" <<EOF
[Flatpak Ref] [Flatpak Ref]
+3 -2
View File
@@ -99,13 +99,14 @@ jobs:
- name: Version from tag - name: Version from tag
run: | run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE, PF_CHANNEL, PF_STABLE_TAG (single source of truth)
case "$GITHUB_REF" in case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; V="${V%%-*}" ;; # App Store marketing version is numeric X.Y.Z (drop -rc) refs/tags/v*) V="${GITHUB_REF_NAME#v}"; V="${V%%-*}" ;; # App Store marketing version is numeric X.Y.Z (drop -rc)
*) V="0.5.0" ;; # canary marketing version; the build number disambiguates *) V="$PF_BASE" ;; # canary marketing version = one minor ahead of the latest stable tag; the build number disambiguates
esac esac
echo "VERSION=$V" >> "$GITHUB_ENV" echo "VERSION=$V" >> "$GITHUB_ENV"
echo "BUILD_NUM=$GITHUB_RUN_NUMBER" >> "$GITHUB_ENV" echo "BUILD_NUM=$GITHUB_RUN_NUMBER" >> "$GITHUB_ENV"
echo "version $V build $GITHUB_RUN_NUMBER" echo "version $V build $GITHUB_RUN_NUMBER (channel $PF_CHANNEL, latest stable ${PF_STABLE_TAG})"
- name: Rust toolchain (mac + iOS + tvOS slices) - name: Rust toolchain (mac + iOS + tvOS slices)
run: | run: |
+6 -5
View File
@@ -68,16 +68,17 @@ jobs:
restore-keys: cargo-home- restore-keys: cargo-home-
- name: Version + channel - name: Version + channel
# vX.Y.Z tag -> X.Y.Z-1 in the base group (a real release); main push -> 0.5.0-0.ciN.g<sha> # vX.Y.Z tag -> X.Y.Z-1 in the base group (a real release); main push -> <next-minor>-0.ciN.g<sha>
# in the `<base>-canary` group, whose "0." release sorts below the eventual 0.5.0-1 yet # in the `<base>-canary` group, whose "0." release sorts below the eventual <next-minor>-1 yet
# climbs by run number. The canary base stays one minor ahead of the latest stable so a # climbs by run number. The canary base is derived one minor ahead of the latest stable tag
# stable->canary box re-point still moves forward. The spec %build stamps # (scripts/ci/pf-version.sh) so a stable->canary box re-point still moves forward. The spec %build stamps
# PUNKTFUNK_BUILD_VERSION from these macros into the binary (--version provenance). # PUNKTFUNK_BUILD_VERSION from these macros into the binary (--version provenance).
run: | run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8) SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; R="1"; GROUP="${{ matrix.group }}" ;; refs/tags/v*) V="${GITHUB_REF_NAME#v}"; R="1"; GROUP="${{ matrix.group }}" ;;
*) V="0.5.0"; R="0.ci${GITHUB_RUN_NUMBER}.g${SHORT}"; GROUP="${{ matrix.group }}-canary" ;; *) V="$PF_BASE"; R="0.ci${GITHUB_RUN_NUMBER}.g${SHORT}"; GROUP="${{ matrix.group }}-canary" ;;
esac esac
echo "PF_VERSION=$V" >> "$GITHUB_ENV" echo "PF_VERSION=$V" >> "$GITHUB_ENV"
echo "PF_RELEASE=$R" >> "$GITHUB_ENV" echo "PF_RELEASE=$R" >> "$GITHUB_ENV"
+13 -3
View File
@@ -131,11 +131,21 @@ jobs:
# dispatched provisioning workflow landing on a different one. Path is relative to the job # dispatched provisioning workflow landing on a different one. Path is relative to the job
# working-directory (packaging/windows/drivers). Near-noop once the toolchain is present. # working-directory (packaging/windows/drivers). Near-noop once the toolchain is present.
run: ../../../scripts/ci/ensure-windows-toolchain.ps1 run: ../../../scripts/ci/ensure-windows-toolchain.ps1
- name: cargo build the driver workspace (wdk-probe + wdk-iddcx + pf-vdisplay) - name: cargo build the driver workspace (wdk-probe + wdk-iddcx + pf-vdisplay + gamepad drivers)
# Whole workspace: wdk-probe (toolchain/surface-assert probe) + wdk-iddcx (DDI wrappers) + # Whole workspace: wdk-probe (toolchain/surface-assert probe) + wdk-iddcx (DDI wrappers) +
# pf-vdisplay (the real IddCx driver). pf-vdisplay linking proves the IddCx call sites resolve # pf-vdisplay (the real IddCx driver) + pf-umdf-util (the safe UMDF primitive layer) + the two
# against IddCxStub end-to-end (M1 step 2 gate). # gamepad drivers. pf-vdisplay linking proves the IddCx call sites resolve against IddCxStub
# end-to-end (M1 step 2 gate); the gamepad drivers prove pf-umdf-util's WDF dispatch links.
run: cargo build -v run: cargo build -v
- name: cargo clippy the shipped drivers (-D warnings — enforces the unsafe-audit gates)
# The gamepad drivers' business logic is 100% safe (it moved onto pf-umdf-util, the audited
# unsafe layer); pf-vdisplay + wdk-iddcx are inherently FFI-bound but every `unsafe {}` carries a
# `// SAFETY:` proof. Both invariants are lint-gated (`unsafe_op_in_unsafe_fn` +
# `undocumented_unsafe_blocks`); this step keeps them from regressing. (wdk-probe is a
# toolchain-only probe crate and is excluded.)
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense --check
- name: Inspect /INTEGRITYCHECK (before) — expect FORCE_INTEGRITY set by wdk-build - name: Inspect /INTEGRITYCHECK (before) — expect FORCE_INTEGRITY set by wdk-build
run: | run: |
# explicit --target (.cargo/config.toml) -> output under the triple subdir. # explicit --target (.cargo/config.toml) -> output under the triple subdir.
+18 -12
View File
@@ -16,15 +16,17 @@
# Versioning (free-form; not MSIX's 4-part rule) — single project version: # Versioning (free-form; not MSIX's 4-part rule) — single project version:
# vX.Y.Z tag -> X.Y.Z (THE release; published + stable `latest/` alias + attached to the # vX.Y.Z tag -> X.Y.Z (THE release; published + stable `latest/` alias + attached to the
# unified Gitea Release). # unified Gitea Release).
# main push / dispatch -> 0.3.<run_number> (canary; `canary/` alias; climbs by run number). # main push / dispatch -> <next-minor>.<run_number> (canary; `canary/` alias; base one minor
# ahead of the latest stable tag via scripts/ci/pf-version.ps1, run climbs).
# #
# Signing reuses the client's MSIX_CERT_PFX_B64 / MSIX_CERT_PASSWORD secrets (CN=unom). Without them # Signing reuses the client's MSIX_CERT_PFX_B64 / MSIX_CERT_PASSWORD secrets (CN=unom). Without them
# an ephemeral self-signed cert is generated and its public .cer published next to the installer # an ephemeral self-signed cert is generated and its public .cer published next to the installer
# (import once to LocalMachine\TrustedPublisher). See packaging/windows/pack-host-installer.ps1. # (import once to LocalMachine\TrustedPublisher). See packaging/windows/pack-host-installer.ps1.
# #
# GPU backends: the host builds with --features nvenc,amf-qsv = all three vendors in one installer. # GPU backends: the host builds with --features nvenc,amf-qsv = all three vendors in one installer.
# - NVENC (NVIDIA, direct SDK): the only link need is nvencodeapi.lib, synthesised from a 2-export # - NVENC (NVIDIA, direct SDK): nothing needed at build time — the entry points are resolved at
# .def with llvm-dlltool (no GPU/SDK at build time). # RUNTIME from the driver's nvEncodeAPI64.dll (a link-time import would kill the binary on
# AMD/Intel-only boxes before main).
# - AMF/QSV (AMD/Intel, libavcodec): link-imports the FFmpeg libs from FFMPEG_DIR (the BtbN lgpl-shared # - AMF/QSV (AMD/Intel, libavcodec): link-imports the FFmpeg libs from FFMPEG_DIR (the BtbN lgpl-shared
# tree the client uses; includes the *_amf/*_qsv encoders) and bundles its DLLs into the installer. # tree the client uses; includes the *_amf/*_qsv encoders) and bundles its DLLs into the installer.
# lgpl-shared (not gpl-shared) keeps those bundled DLLs LGPL (we never use the GPL-only x264/x265). # lgpl-shared (not gpl-shared) keeps those bundled DLLs LGPL (we never use the GPL-only x264/x265).
@@ -37,6 +39,7 @@ on:
paths: paths:
- 'crates/punktfunk-host/**' - 'crates/punktfunk-host/**'
- 'crates/punktfunk-core/**' - 'crates/punktfunk-core/**'
- 'crates/punktfunk-tray/**'
- 'packaging/windows/**' - 'packaging/windows/**'
- 'scripts/windows/**' - 'scripts/windows/**'
- 'web/**' - 'web/**'
@@ -100,30 +103,33 @@ jobs:
if (-not $env:VBCABLE_DIR) { if (-not $env:VBCABLE_DIR) {
"VBCABLE_DIR=C:\Users\Public\vbcable" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 "VBCABLE_DIR=C:\Users\Public\vbcable" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
} }
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$v = if ($env:GITHUB_REF -like 'refs/tags/v*') { $v = if ($env:GITHUB_REF -like 'refs/tags/v*') {
$env:GITHUB_REF_NAME -replace '^v', '' $env:GITHUB_REF_NAME -replace '^v', ''
} else { } else {
"0.3.$($env:GITHUB_RUN_NUMBER)" # Canary: <major>.<minor>.<run> — major.minor track one minor ahead of stable, run climbs monotonically.
"$($pf.PF_MAJOR).$($pf.PF_MINOR).$($env:GITHUB_RUN_NUMBER)"
} }
"HOST_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 "HOST_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"PUNKTFUNK_BUILD_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 "PUNKTFUNK_BUILD_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
Write-Output "host version $v" Write-Output "host version $v"
- name: Generate NVENC import lib
shell: pwsh
run: |
& packaging/windows/nvenc/gen-nvenc-importlib.ps1 -OutDir C:\t\nvenc
"PUNKTFUNK_NVENC_LIB_DIR=C:\t\nvenc" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Build (release, nvenc + amf-qsv) - name: Build (release, nvenc + amf-qsv)
shell: pwsh shell: pwsh
# All-vendor host: NVENC (NVIDIA, direct SDK) + AMF/QSV (AMD/Intel, libavcodec via FFMPEG_DIR). # All-vendor host: NVENC (NVIDIA, direct SDK) + AMF/QSV (AMD/Intel, libavcodec via FFMPEG_DIR).
run: cargo build --release -p punktfunk-host --features nvenc,amf-qsv run: cargo build --release -p punktfunk-host --features nvenc,amf-qsv
- name: Clippy (host, Windows) - name: Build (release, status tray)
shell: pwsh
# The per-user notification-area companion the installer bundles (punktfunk-tray.exe).
run: cargo build --release -p punktfunk-tray
- name: Clippy (host + tray, Windows)
shell: pwsh shell: pwsh
# First-ever Windows lint coverage for the host (Linux CI never lints the windows-cfg code). # First-ever Windows lint coverage for the host (Linux CI never lints the windows-cfg code).
run: cargo clippy -p punktfunk-host --features nvenc,amf-qsv -- -D warnings run: |
cargo clippy -p punktfunk-host --features nvenc,amf-qsv -- -D warnings; if ($LASTEXITCODE) { throw "host clippy" }
cargo clippy -p punktfunk-tray -- -D warnings; if ($LASTEXITCODE) { throw "tray clippy" }
- name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer) - name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer)
shell: pwsh shell: pwsh
+5 -2
View File
@@ -16,7 +16,8 @@
# vX.Y.Z tag -> X.Y.Z.0 (THE release; any -rc/+meta pre-release suffix is dropped for MSIX). # vX.Y.Z tag -> X.Y.Z.0 (THE release; any -rc/+meta pre-release suffix is dropped for MSIX).
# Published to the generic registry + the stable `latest/` alias + attached to the # Published to the generic registry + the stable `latest/` alias + attached to the
# unified Gitea Release alongside every other platform's artifact. # unified Gitea Release alongside every other platform's artifact.
# main push / dispatch -> 0.3.<run_number>.0 (canary; climbs monotonically by run number). # main push / dispatch -> <next-minor>.<run_number>.0 (canary; base is one minor ahead of the
# latest stable tag via scripts/ci/pf-version.ps1, run number climbs monotonically).
# Published to the generic registry + the `canary/` alias. # Published to the generic registry + the `canary/` alias.
# Both arches share the version; artifacts are arch-suffixed (..._x64.msix / ..._arm64.msix). # Both arches share the version; artifacts are arch-suffixed (..._x64.msix / ..._arm64.msix).
# #
@@ -78,11 +79,13 @@ jobs:
"CARGO_TARGET_DIR=${{ matrix.td }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 "CARGO_TARGET_DIR=${{ matrix.td }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"FFMPEG_DIR=${{ matrix.ffmpeg }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 "FFMPEG_DIR=${{ matrix.ffmpeg }}" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
rustup target add ${{ matrix.target }} rustup target add ${{ matrix.target }}
$pf = & "$env:GITHUB_WORKSPACE/scripts/ci/pf-version.ps1" # single source of truth: base is one minor ahead of the latest stable tag
$parts = if ($env:GITHUB_REF -like 'refs/tags/v*') { $parts = if ($env:GITHUB_REF -like 'refs/tags/v*') {
# MSIX needs a purely-numeric 4-part version: drop any -rc/+meta pre-release suffix. # MSIX needs a purely-numeric 4-part version: drop any -rc/+meta pre-release suffix.
(($env:GITHUB_REF_NAME -replace '^v', '') -replace '[-+].*$', '').Split('.') (($env:GITHUB_REF_NAME -replace '^v', '') -replace '[-+].*$', '').Split('.')
} else { } else {
@('0', '3', $env:GITHUB_RUN_NUMBER) # Canary: <major>.<minor>.<run>.0 — major.minor track one minor ahead of stable, run climbs monotonically.
@($pf.PF_MAJOR, $pf.PF_MINOR, $env:GITHUB_RUN_NUMBER)
} }
while ($parts.Count -lt 4) { $parts += '0' } while ($parts.Count -lt 4) { $parts += '0' }
$v = ($parts[0..3] -join '.') $v = ($parts[0..3] -join '.')
+3
View File
@@ -31,3 +31,6 @@ xcuserdata/
# Python bytecode (e.g. clients/android/ci tooling) # Python bytecode (e.g. clients/android/ci tooling)
__pycache__/ __pycache__/
*.pyc *.pyc
# Claude Code project instructions — local to each dev box, not part of the repo.
CLAUDE.md
-541
View File
@@ -1,541 +0,0 @@
# CLAUDE.md — punktfunk
Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protocol core
(`punktfunk-core`) exposed over a C ABI and native clients per platform. Full design:
[`design/implementation-plan.md`](design/implementation-plan.md). Status table: `README.md`.
## Where the work stands
- **Core (`punktfunk-core` + C ABI): complete and hardened.** FEC recovery, loopback-under-loss,
proptests, C ABI harness all green; 13 adversarial-review findings fixed +
regression-tested (`a913042`).
- **GameStream host: working end-to-end with a stock Moonlight client.** Validated live
on this box: pairing (persists across restarts), serverinfo/applist (app catalog from
`~/.config/punktfunk/apps.json` → each entry picks a compositor + nested command), RTSP, ENet
control, audio, and video at the **client's native resolution and refresh** — the host
creates a per-session virtual output via per-compositor `VirtualDisplay` backends:
**KWin** (`zkde_screencast stream_virtual_output`, needs KWin ≥ 6.5.6 headless; >60 Hz via
custom modes), **gamescope** (spawned headless at WxH@Hz, its PipeWire node captured, needs
gamescope ≥ 3.16.22 — older deadlocks on PipeWire ≥ 1.6), **Mutter** (D-Bus
`RecordVirtual` virtual monitor; validated live on headless GNOME Shell 50, zero-copy),
**Sway/wlroots** (`swaymsg create_output` + custom mode, xdpw portal capture with a
managed chooser config; validated live on sway 1.11, zero-copy).
Performance work landed and measured: GPU **zero-copy** on all paths (tiled dmabuf →
EGL/GL → CUDA; LINEAR dmabuf → **Vulkan bridge** → CUDA → NVENC), auto 2-way NVENC
split-encode above ~1 Gpix/s (5K@240), infinite GOP + RFI keyframes (killed the periodic
freeze), encode|send thread split with `sendmmsg` batching. Stable 240 fps at 5120×1440.
Input: mouse/keyboard (libei via RemoteDesktop portal on KWin/GNOME, gamescope's own EIS
socket, wlr protocols on Sway) and **gamepads** (uinput X-Box-360 pads + rumble
back-channel; validated live — pad created/destroyed with the session). Management REST API +
checked-in OpenAPI doc (`mgmt.rs`). **Web-console performance capture** (`stats_recorder.rs`,
design: [`design/stats-capture-plan.md`](design/stats-capture-plan.md)): the operator arms stats
recording from the web console, plays, stops, and reviews the run as graphs (per-stage latency
breakdown · fps new/repeat · goodput · loss/FEC). A shared `Arc<StatsRecorder>` ring (the hot-path
gate is a runtime `AtomicBool`, replacing the startup-only `PUNKTFUNK_PERF`) is fed by **both** the
native `virtual_stream` and the GameStream encode loop at their existing ~2 s/~1 s aggregation
boundary, and finished captures are saved as on-disk recordings
(`~/.config/punktfunk/captures/*.json`) browsable/exportable from the console's **Performance** page
(recharts). Endpoints `/api/v1/stats/*` (bearer-only). *Implemented; not yet on-glass validated.*
**Web-console log view** (`log_capture.rs`): a `tracing` layer tees DEBUG-and-up (independent of
`RUST_LOG`) into a 4096-entry in-memory ring, served cursor-paged at `GET /api/v1/logs`
(bearer-only) → the console's **Logs** page (follow/pause · level filter · search). The Windows
gamepad drivers now stamp attach/heartbeat marks into their shm sections and the host's
`DriverAttach` watcher turns silence into a one-shot diagnosis WARN (driver-store check + CM
devnode problem code) — failure-mode table: [`design/gamepad-driver-health.md`](design/gamepad-driver-health.md).
The Android client gained Settings → **Connected controllers** (device list + VID/PID + resolved
pad type + live input test) for the client end of the same chain. *Log view + driver health:
Linux-tested; Windows/Android sides CI/device-validation pending.*
- **Native protocol (`punktfunk/1`): full session planes, validated live.** QUIC
control plane (`punktfunk-core` `quic` feature: Hello{mode}/Welcome{full Config}/Start), data
plane = the hardened core `Session` over raw UDP with **GF(2¹⁶) Leopard FEC + AES-GCM**
(inexpressible in GameStream), host creates the native virtual output at the client's
requested mode. `punktfunk1-host` is a **persistent listener** (sessions back to back;
`--max-sessions`). QUIC datagrams carry the side planes, demuxed by first byte: input
0xC8 (incl. **gamepads** — incremental events accumulated into the uinput xpad), **Opus
audio** 0xC9 (48 kHz stereo, 5 ms, host→client), **rumble** 0xCA (host→client). **Trust:**
host serves its persistent identity (`~/.config/punktfunk/cert.pem`, shared with GameStream
pairing) and logs the SHA-256 fingerprint; clients pin it, established by a **SPAKE2 PIN pairing
ceremony** (host arms pairing and displays a 4-digit PIN; a PAKE binds both cert fingerprints so an
attacker gets one online guess, no offline dictionary attack) — PIN pairing is the default for new
hosts. **TOFU on first connect** (`endpoint::client_pinned`) stays as an explicit host opt-in
(`punktfunk1-host --allow-tofu` / `serve --open`, advertised as `pair=optional`) for fully trusted LANs;
clients only offer the TOFU "Trust" path for a host that advertised `pair=optional`, route every
other new host straight to the PIN ceremony, and on a pinned-fingerprint change force re-pairing
(no re-TOFU shortcut). Clients present persistent identities via QUIC client auth, the host stores
paired fingerprints (`punktfunk1-paired.json`) and gates sessions with `--require-pairing` (the
default; `--allow-tofu`/`--open` accept unpaired clients).
**LAN auto-discovery**: both `serve` and `punktfunk1-host` advertise the native service over
mDNS (`_punktfunk._udp`, `crate::discovery`) with TXT `proto`/`fp`(cert fingerprint to
pin)/`pair`(required|optional)/`id`; `punktfunk-probe --discover` lists hosts, Apple clients
browse the same service via NWBrowser (validated cross-LAN 2026-06-12).
**Mid-stream mode renegotiation**: `Reconfigure` on the still-open control stream — the
host rebuilds output+encoder at the new mode in ~90 ms while the data plane runs on
(validated live: one .h265 with 720p and 1080p segments). Measured on-box at 720p120: 1680/1680 frames, **p50 0.83 ms**
capture→…→reassembled; audio measured live (~200 pkts/s). A **wall-clock skew handshake**
(`ClockProbe`/`ClockEcho`, 8 NTP rounds after `Start`, `clock_offset_ns`) aligns the client to the
host clock, so that latency is now valid **cross-machine** (`skew_corrected=true`) — measured GNOME
box → dev box over the LAN: **p50 1.30 ms** (the 1.57 ms inter-box clock offset removed).
`punktfunk-probe` is the
working reference client (`--pin`, datagram counters, `--input-test` incl. gamepad).
The embeddable connector (`NativeClient`) exposes it all over the C ABI: `punktfunk_connect`
(pin/TOFU) + `next_au`/`next_audio`/`next_rumble`/`next_hidout`/`send_input`/
`send_rich_input`. **Client-negotiated virtual pad type**: the Hello carries a gamepad
preference byte (same trailing-byte back-compat pattern as the compositor), the Welcome
echoes the resolved backend — precedence: explicit client choice > `PUNKTFUNK_GAMEPAD`
env > uinput Xbox 360. Backends: **Xbox 360** (uinput on Linux / the pf-xusb UMDF driver on
Windows), **Xbox One/Series** (the same
XInput backend with the One/Series USB identity for matching glyphs — no extra game-visible
capability; impulse-trigger rumble is unreachable through a virtual pad), and the UHID
`hid-playstation` pads — **DualSense** (adaptive triggers, lightbar, touchpad, motion) and
**DualShock 4** (lightbar, touchpad, motion, rumble; DualSense minus adaptive triggers / player
LEDs / mute). DualSense and DualShock 4 each have a Linux (UHID `hid-playstation`) **and a Windows
(UMDF minidriver)** backend — `inject/windows/dualsense_windows.rs` + `inject/windows/dualshock4_windows.rs`, one
driver serving either identity per a `device_type` byte the host stamps into shared memory (the DS4
reuses the same SwDeviceCreate game-detection identity fix as the DualSense). One/Series stays
Linux-only and folds into Xbox 360 off it. Clients auto-resolve the type from the physical controller
(DS5→DualSense, DS4→DualShock 4, Xbox One→Xbox One). **Windows uses ZERO external gamepad
dependencies — ViGEmBus is gone.** Xbox 360 (XInput) runs on a UMDF2 **XUSB companion** driver
(`packaging/windows/drivers/pf-xusb/`, `inject/windows/gamepad_windows.rs`) that registers `GUID_DEVINTERFACE_XUSB`
and answers the buffered XInput IOCTLs from a shared section, so classic `XInputGetState`/`SetState`
work with no kernel bus driver (validated live: slot connected, state + rumble round-trip; Xbox One
folds to this 360 path). All three UMDF drivers (DualSense/DS4 + XUSB) are built from source in CI
(`packaging/windows/drivers/`) and installed by the Inno Setup installer via
`punktfunk-host.exe driver install --gamepad`.
**Multi-pad ready**: the host stamps each pad's index into the device Location (`pszDeviceLocation`),
the driver reads it (`WdfDeviceAllocAndQueryProperty`) to map its own `*-shm-<index>`, and
`UmdfHostProcessSharing=ProcessSharingDisabled` gives each pad its own host (per-pad statics) —
validated live with 2 distinct XInput slots + 2 DualSense pads. (Client-side multi-pad forwarding is
the remaining piece.)
- **Windows host: implemented and shipping (all-vendor, x64-only).** `#[cfg(windows)]` backends
behind the same traits as Linux — **IDD-push capture** straight into the in-house all-Rust IddCx
**pf-vdisplay** virtual display (`capture/windows/idd_push.rs`, `vdisplay/windows/pf_vdisplay.rs`;
DXGI Desktop Duplication / WGC as fallbacks, `capture/windows/dxgi.rs`), GPU encode (NVENC
`--features nvenc`; AMD/Intel `--features amf-qsv`), SendInput + the in-house UMDF gamepad drivers
(`inject/windows/`), WASAPI loopback + virtual mic (`audio/windows/wasapi_*`). **Keyboard wire
convention: US-positional VKs** (every first-party client sends the physical key's US-layout VK;
the Windows client derives it from the scancode, NOT the layout-resolved `vkCode`) — the Windows
injector resolves them via a fixed table mirroring the Linux `vk_to_evdev` (never through a
keyboard layout: the SYSTEM service thread's layout re-reads positions as characters — the
German y↔z / ö→ü scramble), while GameStream/Moonlight VKs are layout-semantic
(`KEY_FLAG_SEMANTIC_VK`, resolved under the foreground app's layout, Sunshine's model). Linux
renders positions under the session compositor's layout (libei) or the virtual keyboard's
uploaded keymap (Sway/wlroots — honors `XKB_DEFAULT_LAYOUT` et al., default US); the Android
client reads `KeyEvent.scanCode` first so a user-selected physical-keyboard layout can't
re-map keycodes semantically. Ships as a **signed
Inno Setup installer** that registers a `LocalSystem` SCM service launching into the interactive
session for secure-desktop (UAC/lock-screen) capture (`windows/service.rs`), bundles the
pf-vdisplay driver + the FFmpeg DLLs (+ VB-CABLE for the virtual mic), and is published by
`windows-host.yml`. **Encoder is GPU-aware** (`encode.rs` `open_video` + `windows_resolved_backend`):
`PUNKTFUNK_ENCODER=auto` (the host.env default) reads the **selected render adapter's** vendor →
**NVENC** (NVIDIA, direct SDK, `encode/windows/nvenc.rs`), **AMF** (AMD) / **QSV** (Intel) via libavcodec
(`encode/windows/ffmpeg_win.rs`, the Windows analogue of the Linux VAAPI backend — `WinVendor{Amf,Qsv}`,
system-memory NV12/P010 readback default + opt-in zero-copy D3D11 behind `PUNKTFUNK_ZEROCOPY` with a
system fallback), or software H.264 (`encode/sw.rs`, GPU-less). GameStream codec advertisement is
probed per-GPU on AMF/QSV (`windows_codec_support``serverinfo`, AV1 gated; cached per selected
GPU). **Multi-GPU is first-class** (`gpu.rs`): GPU inventory + a persisted auto/manual preference
(`<config>/gpu-settings.json`, stored by stable PCI identity — LUIDs are per-boot) exposed over
`GET /api/v1/gpus` + `PUT /api/v1/gpus/preference` and a web-console GPU card (Host page: list,
Automatic/Prefer, "In use · backend" badge). One selection — precedence **console preference >
`PUNKTFUNK_RENDER_ADAPTER` > max VRAM**, graceful fallback when the preferred GPU is absent —
feeds `win_adapter::resolve_render_adapter_luid` (capture ring + IddCx render pin), the encoder
vendor auto-detect (previously DXGI adapter 0 — wrong on hybrid boxes like NVIDIA dGPU + Intel
Arc iGPU), and the NVENC 4:4:4 probe; a preference change applies to the next session. On Linux a
matched manual preference picks the VAAPI render node / NVENC-vs-VAAPI auto choice (auto mode
unchanged). *Implemented + unit-tested; not yet on-glass validated on the hybrid box.* **HDR (10-bit)**: WGC
captures the HDR desktop as FP16/Rgb10a2 (DDA FP16 for the secure desktop), the encoder forces HEVC
Main10 + BT.2020 PQ (NVENC ABGR10/P010; AMF/QSV P010 + a swscale Rgb10a2→P010 fallback), the client
auto-detects PQ from the HEVC VUI — gated by `PUNKTFUNK_10BIT` + client `VIDEO_CAP_10BIT`; **Windows
host only** (the Linux host stays 8-bit, blocked upstream). **Vulkan-game HDR over the virtual
display**: NVIDIA/AMD Vulkan ICDs refuse to *advertise* an HDR color space for a surface on an IddCx
indirect display (so Vulkan games — Doom: The Dark Ages, id Tech, etc. — say "device does not support
HDR"), even though the ICD happily *accepts + presents* a forced HDR swapchain there. A tiny always-on
Vulkan **implicit layer** (`packaging/windows/pf-vkhdr-layer/`, `VK_LAYER_PUNKTFUNK_hdr_inject`)
injects the `HDR10_ST2084`/scRGB surface formats into `vkGetPhysicalDeviceSurfaceFormats[2]KHR`,
self-gated on the display's actual advanced-color state (no-op on SDR / real monitors); bundled +
HKLM-registered by the installer. **Live-validated: Doom: The Dark Ages enables HDR over the virtual
display.** **AMF/QSV is CI-green but not yet on-glass validated** (no AMD/Intel Windows box in the
lab); NVENC is live-validated. Newer/less battle-tested than the Linux host. Packaging: `packaging/windows/`.
## What's left
1. **Native clients — decode + present: macOS stage 1 done, first light achieved
(2026-06-10).** PunktfunkKit compiles and is tested on macOS (AnnexB → VideoToolbox →
`AVSampleBufferDisplayLayer`, GCMouse/GCKeyboard capture, `PunktfunkClient` app shell);
validated live Mac ↔ this box at 720p60 — vkcube on glass, input injected via gamescope
EIS. The app speaks the full ABI v2 trust surface: Keychain-persisted client identity
presented on every connect, SPAKE2 PIN pairing UI (host-card context menu + the trust
prompt's "Pair with PIN instead…"), TOFU fingerprint prompt. **Gamepads (2026-06-11):**
controller discovery + selection in Settings (`GamepadManager` — exactly one pad
forwarded as pad 0, auto or pinned; pad TYPE auto-resolves from the physical
controller, user-overridable), capture incl. DualSense touchpad/motion
(`GamepadCapture`/`GamepadWire`; while streaming, EVERY element's
`preferredSystemGestureState` is claimed `.disabled` — share/create reaches the host as
select instead of screenshotting locally, PS/Home reaches the host as guide/`BTN_MODE` =
the Steam-overlay button — restored `.enabled` on unbind), feedback rendering (rumble → CoreHaptics; lightbar /
player LEDs / adaptive triggers → `GCDeviceLight`/`playerIndex`/
`GCDualSenseAdaptiveTrigger` via the table-driven `DualSenseTriggerEffect` parser).
Loopback-tested end to end (`PUNKTFUNK_TEST_FEEDBACK=1` scripted burst); DualSense
motion sign/scale derived, not yet live-verified. **Rumble renderer rewritten
(2026-07-02, `RumbleRenderer.swift`)** around "rumble is idempotent state, divergence
must be bounded": the old per-datagram infinite-duration CoreHaptics players could leak
one dropped async `stop` into a forever-buzzing motor (the stuck-rumble-after-menu bug)
— now finite self-expiring segments with seamless engine-timeline re-arm, newest-wins
dry drain of the 0xCA plane (was 1 datagram/8 ms), dedupe of the host's 500 ms state
refreshes, zero-immediate/ramp-throttled rebakes, escalation to `engine.stop()` on a
throwing player stop, and a 1.6 s staleness watchdog (`Policy.session`; the settings
test panel uses `.manual` = hold). Controller engines use **plain `makePlayer` — never
`makeAdvancedPlayer`**: the controller haptics server (gamecontrollerd) advertises
`adv players: 0`, and iOS 27 beta 2 hard-drops advanced-player loads (XPC decode fault →
CoreHaptics -4811/4097, rumble silently dead). Unit-tested (`RumbleTuningTests`);
stuck-rumble repro on-glass revalidation pending. **Gamepad UI (iOS/iPadOS + macOS,
2026-07-02 rework):** a connected pad swaps the home for a console-style launcher
(`Home/Gamepad*` + `Settings/GamepadSettingsView`) — host carousel with a trailing Add
Host tile (A connect · Y library · X settings · B back), a controller-navigable
settings screen (vertical `GamepadMenuList`, left/right steps values), an add-host
flow with an on-screen controller keyboard (`GamepadKeyboard` — no touch needed
anywhere), and the coverflow library, all over an animated aurora backdrop
(`GamepadScreenBackground`, TimelineView-driven drifting blobs — pure SwiftUI ON
PURPOSE: a .metal lib only reliably bundles in one of the two build systems (SPM vs
xcodeproj synced folders) these sources compile under). Input is the polled
`GamepadMenuInput` (handlers don't fire outside a stream; on (re)start it SNAPSHOTS
held buttons so a handoff press never double-fires), haptics dual-channel (device +
`MenuHaptics` on the pad). macOS: same screens, settings/add-host as sheets (no
fullScreenCover), NSScreen-based mode lists, scroll indicators `.never` (macOS
"always show scroll bars" overrides `.hidden`); launcher/settings/add-host/keyboard
render-verified live on this Mac via `PUNKTFUNK_FORCE_GAMEPAD_UI=1` (dev hook, forces
the mode without a pad). Controller-in-hand on-glass validation still pending on all
platforms. **Touch input (iOS/iPadOS, 2026-07-02):** a 3-way model in Settings —
**Trackpad** (default; the Android client's gesture vocabulary ported 1:1 in
`Input/TouchMouse.swift`: tap=click · two-finger tap=right-click · two-finger drag=scroll ·
tap-then-drag=held drag · three-finger tap=HUD toggle, relative ballistics with the same
px-based acceleration curve), **Direct pointer** (cursor jumps to the finger), **Touch
passthrough** (the previous always-on behavior — real wire touches). Latched per gesture
from `DefaultsKey.touchMode`; not yet on-glass validated. Tests: `swift test` in
`clients/apple` (unit + real-codec round trip),
`test-loopback.sh` (Swift client vs synthetic punktfunk1-hosts on loopback — runs on macOS;
includes the pairing ceremony + `--require-pairing` gate),
`RemoteFirstLightTests` (full pipeline over the LAN). See
[`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter is now the DEFAULT**
(stage-1 is the Metal-unavailable / DEBUG fallback): explicit `VTDecompressionSession` decode →
`CAMetalLayer`, presented from the hosting view's **main-runloop `CADisplayLink`** (`renderTick` pops
the newest ready frame per vsync; macOS `displaySyncEnabled = false` is the real fullscreen-judder fix,
~11 ms p50). *(An off-main `CAMetalDisplayLink` and an off-main blocking-render present thread were
both tried and reverted — both measured slower on macOS and iPad.)* **HDR fixed**
(`design/apple-stage2-presenter.md`): the "too bright" bug was a missing reference-white anchor — the
fix keeps the PQ-passthrough shader and adds `CAEDRMetadata.hdr10(…, opticalOutputScale: 203)` +
`wantsExtendedDynamicRangeContent` on the layer (on all platforms; the old `#if os(macOS)` guard left
iOS/tvOS EDR half-engaged), routing the 0xCE mastering metadata to the layer (via `setHdrMeta`) instead
of a never-composited source buffer. **Mid-session SDR↔HDR** is handled: `render` reconciles the layer
per-frame from the decoded `frame.isHDR` (per-mode pixel format `bgra8`/`rgba16Float`), so a game
entering HDR mid-stream just reconfigures (last 0xCE grade cached + re-applied; pump drains 0xCE
unconditionally). **4:4:4 added**: decode format is a 2×2 `(chroma, HDR)` matrix
(`420v/x420/444v/x444`, all biplanar so the shaders are unchanged), advertised (`VIDEO_CAP_444`) only
behind a **hardware-required `VTDecompressionSession` probe** (`Stage444Probe`, validated on M3) with a
Settings opt-out + a bounded pump backstop for an undecodable 4:4:4 session. *HDR brightness + 4:4:4
still need on-glass validation (Windows-HDR / `PUNKTFUNK_444` host).* Next: glass-to-glass numbers via
`tools/latency-probe`.
**Linux stage 1 done, first light 2026-06-12** (`clients/linux`, binary
`punktfunk-client`): GTK4/libadwaita shell linking `punktfunk-core` directly (no C ABI;
`NativeClient` is now `Sync` — mutexed plane receivers), mDNS host list, TOFU + SPAKE2
PIN dialogs (identity shared with client-rs), FFmpeg software HEVC decode (LOW_DELAY,
slice threads) → `GtkGraphicsOffload`-wrapped picture, PipeWire playback (mic-player
jitter ring inverted), SDL3 gamepad capture + rumble/lightbar feedback, keyboard via
exact inverse of the host VK table, absolute mouse + 120-unit scroll. Validated live
against `serve` on this box: 1080p60, steady 60 fps, capture→decoded p50
≈6.4 ms (debug build). `--connect host[:port]` for scripting. **Swift-parity batch +
stage 1.5 (2026-06-12 evening)**: capture state machine (click-to-capture,
Ctrl+Alt+Shift+Q / focus-loss release, held-state flush), app-lifetime SDL gamepad
service (pad pin UI, auto type from the physical pad, DualSense touchpad/motion 0xCC +
raw-DS5-effects trigger/player-LED replay — needs a physical pad to live-verify), mic
uplink (validated live), per-host speed test, compositor pref, native-display mode
default, saved-hosts list, .deb + RPM-subpackage CI (deb.yml/rpm.yml). **VAAPI decode
→ DRM-PRIME dmabuf → `GdkDmabufTexture`** (BT.709 color state; Tier-1 zero-copy on
Intel/AMD, `PUNKTFUNK_DECODER=software|vaapi` override) with a proven fallback ladder —
no VAAPI device (NVIDIA) or mid-session VAAPI error → software decode. **First AMD test
(Steam Deck) hit a green-screen bug, fixed:** FFmpeg's VAAPI export uses
`SEPARATE_LAYERS`, so NV12 arrives as two single-plane layers (R8 luma + GR88 chroma,
one shared fd); the mapper took `layers[0]` only → GTK got a luma-only R8 texture, chroma
read as 0 → green field / red whites. Fix derives the combined fourcc from the decoder
`sw_format` (→ `DRM_FORMAT_NV12`) and flattens all planes across all layers (mpv's
pattern); a first-frame descriptor dump logs the real layout. Awaiting Steam Deck
reconfirm. Next: the stage-2 raw-Wayland
presenter (wp_presentation feedback, tearing-control, Vulkan Video on NVIDIA) —
**wgpu/winit rejected** (no dmabuf import / presentation feedback / shortcuts-inhibit).
**Windows stage 1 done 2026-06-15** (`clients/windows`, binary
`punktfunk-client`): pure-Rust **WinUI 3** UI via **windows-reactor** (a declarative React-like
framework backed by WinUI; PR #4499 added the `SwapChainPanel` widget + `set_swap_chain`). The
video is a **`SwapChainPanel`** bound to a **D3D11 composition swapchain**, presented from a
**dedicated render thread** (`render.rs`, 2026-07-02 rewrite — presenting never touches or is
stalled by the XAML thread): frame-latency-waitable swapchain + `SetMaximumFrameLatency(1)`
(≤1 queued present, newest-wins drain after the wait, so a stream faster than the display drops
backlog before any GPU work), **HiDPI-correct** (pixel-sized buffers + `SetMatrixTransform`
96/DPI — DIP-sized buffers were blurry at 125/150 %), Contain-fit letterbox, WARP fallback.
**FFmpeg decode with a D3D11VA hardware path on all vendors** (`gpu.rs` shares one D3D11 device
between decoder + presenter, adapter picked by console pref `PUNKTFUNK_ADAPTER` > the window's
monitor's adapter > default; `PUNKTFUNK_D3D_DEBUG=1` adds the debug layer): the decode pool is
**decoder-only bind, sized/aligned by libavcodec itself** (get_format returns `AV_PIX_FMT_D3D11`
and lets `hw_device_ctx` drive — three hand-built-frames-context strikes are why: NVIDIA rejects
`DECODER|SHADER_RESOURCE` arrays, `BindFlags=0` fails texture creation, and Intel rejects
non-128-aligned HEVC surfaces at the first `SubmitDecoderBuffers`), a DXVA **profile probe**
before the hwdevice commits software-vs-hardware up front (no burned first IDR), and the
presenter copies the decoded slice with ONE display-size-boxed `CopySubresourceRegion` (a planar
slice is a single subresource in D3D11 — the old two-copy D3D12-style code silently no-opped =
the black screen) into its sampleable NV12/P010 texture → per-plane SRVs + YUV→RGB shaders
(NV12/BT.709, P010/BT.2020-PQ). **Software CPU decode is the fallback** (auto-selected,
`DecoderPref` override, mid-session demotion + keyframe re-request) and now feeds the SAME
shaders (swscale → NV12/P010 planes → two dynamic plane textures) so hw/sw colour math is
identical. **HDR10**: the client advertises 10-bit/HDR (Settings toggle, gated on an HDR
display), detects PQ in-band (`transfer == SMPTE2084`), and flips the swapchain to
`R10G10B10A2` + ST.2084 with HDR10 metadata (0xCE mastering metadata plumbed). **WASAPI** render
+ mic capture, **SDL3** gamepads (rumble/lightbar/DualSense), `mdns-sd` discovery, and the full
trust surface — all **in-app**: a polished WinUI shell (host tiles w/ monogram + status pills,
`InfoBar` errors/hints, `ToggleSwitch` settings, status-chip stream HUD showing GPU/CPU decode +
HDR), host list (live mDNS + saved + manual), settings (resolution/refresh/decoder/bitrate/HDR/
mic), SPAKE2 PIN pairing screen, TOFU, pinned-fp-mismatch re-pair. **Live-validated 2026-07-02
on the hybrid laptop (Intel Arc Pro iGPU + RTX 3500 Ada) against the local Windows host**:
D3D11VA hardware decode 60 fps on BOTH vendors (headless, `PUNKTFUNK_ADAPTER`-forced; NVIDIA
0.2 ms decode, Intel 0.2 ms), software path, and the GUI on glass (real decoded desktop pixels,
GPU-decode HUD chip, ~18 ms capture→decoded p50 over loopback — dominated by the host's 60 Hz
virtual-display capture cadence). HDR-on-glass still pending. **Stream input** is Win32 low-level hooks (`WH_KEYBOARD_LL`/`WH_MOUSE_LL`) — reactor
exposes no raw key/pointer events; native Windows VK + absolute mouse (client-rect Contain-fit) +
wheel, Ctrl+Alt+Shift+Q capture toggle. `--headless`/`--discover` keep CLI paths. Builds + clippy
+ fmt green on **`x86_64-pc-windows-msvc` and `aarch64-pc-windows-msvc`** — the latter
**cross-compiled off the one x64 runner** (no ARM64 runner; the x64 MSVC toolset's ARM64 cross
compiler + a per-arch `FFMPEG_DIR` ARM64 tree, SDL3/libopus build-from-source cross-compile
cleanly), and both ship as signed MSIX (`windows-msix.yml` matrix → `..._x64.msix`/`..._arm64.msix`,
verified: ARM64 binaries + manifest arch). **windows-reactor is unpublished** (git
dep pinned to commit `a4f7b2cb`, bumped 2026-07-02 from `b4129fcc` for `on_pointer_entered`/
`on_pointer_exited` hover events — mechanical renames only: `SymbolGlyph``Symbol`,
`placeholder``placeholder_text`, TextBox `on_changed``on_text_changed`, ToggleSwitch
`on_changed``on_toggled`, `on_menu_item_clicked``on_item_clicked`, SwapChainPanel
`on_ready``on_mounted`; `windows` pinned to the SAME commit so `IDXGISwapChain1` unifies with
`set_swap_chain`). New-model runtime staging: reactor has NO build.rs anymore — the app's own
`build.rs` calls `windows_reactor_setup::as_framework_dependent()` (same-rev build-dep, stages
the bootstrap DLL + resources.pri that pack-msix expects) and `main` calls
`windows_reactor::bootstrap()` before `App` (packaged MSIX: a no-op, the manifest's
`Microsoft.WindowsAppRuntime.2` dependency resolves the runtime). `CARGO_WORKSPACE_DIR` is no
longer required (harmless where still set). Gotcha: `CARGO_HOME` must be an ASCII path
— the `ü` in the dev box's username breaks SDL3's MSVC precompiled-header build. **Parity/cleanup
batch (2026-07-02)**: `app.rs` split into per-screen `app/` modules (mod=root/router · hosts ·
connect · pair · speed · settings · licenses · stream · style; thread-driven state lives in ROOT
`use_async_state` and flows down as props — a child's own async-state write does NOT re-render it);
"Native display" now resolves the real monitor mode at connect (`MonitorFromWindow`
`EnumDisplaySettingsW`, was hardcoded 1080p60); per-host **speed test** (saved-host card button +
`--headless --speed-test`, probe burst → recommended ≈70 % bitrate applied in one tap; bitrate
setting is now a free-form NumberBox); **forget host** (ContentDialog confirm →
`KnownHosts::remove_by_fp`); settings gained forwarded-controller picker + gamepad type + host
compositor + capture-system-shortcuts — the previously-dead `Settings.compositor`/
`inhibit_shortcuts` are now honored (off = Alt+Tab/Alt+Esc/Ctrl+Esc/Win act locally);
**click-to-recapture** after a Ctrl+Alt+Shift+Q release with the HUD hint tracking capture state;
input hook caches lock geometry (no per-move `GetClientRect`), audio jitter-ring trims via
`drain`. Validated on the bare-metal RTX box: `--discover` (3 live LAN hosts), synthetic-host
loopback E2E (TOFU connect → clock skew → HEVC negotiate → shared-D3D11 + D3D11VA init → WASAPI →
session end; synthetic payload isn't decodable so decode output stays unvalidated), speed-test
E2E. The WinUI window itself CANNOT be launched from SSH (session-0 → WinAppSDK 0x80070005,
pre-existing; needs the console session, e.g. PsExec -i 1). **UX batch (2026-07-02 evening,
UIA-smoke-tested on the hybrid laptop)**: host tiles get the WinUI pointer-over fill
(`on_pointer_entered`/`exited` → root hover state → `ControlFillSecondary`), Settings is a stock
**NavigationView** sidebar (Windows-Settings pattern: Display/Video/Input/Audio/About panes,
built-in back arrow, section in root state; the section card is **keyed by section** — an
in-place diff across sections re-sets a reused ComboBox's items, clearing WinUI's selection,
but skips `selected_index` when the values compare equal → blank selection; the key forces a
remount — and the content column rides its own section-switch slide-up tween), new
**"Show the stats overlay (HUD)"** toggle
(`Settings::show_hud`, applies mid-stream via the 400 ms HUD re-render), the Add-host modal
slides up + fades in (root margin/opacity tween, same pattern as screen navigation), and a
self-initiated disconnect (Ctrl+Alt+Shift+D → `Ended(None)`) returns to the host list silently
instead of raising the error banner.
Next: **HDR on-glass validation** (Windows host with `PUNKTFUNK_10BIT` → the HDR laptop
display), then RAWINPUT relative-mouse pointer-lock.
**Android stage 1 done** (`clients/android`, Kotlin app + `native/` Rust JNI core linking
`punktfunk-core`; phone + Android TV): NDK `AMediaCodec` hardware HEVC decode → `SurfaceView` incl.
**HDR10** (Main10/BT.2020 PQ) with low-latency tuning + a live stats HUD (`decode.rs`/`stats.rs`),
Opus/AAudio audio + mic uplink (`audio.rs`/`mic.rs`), gamepad input with rumble/HID feedback
(`feedback.rs`), **native `mdns-sd` mDNS discovery** (`discovery.rs`, polled over JNI — the same
browse the Linux/Windows clients use, replacing the flaky per-OEM `NsdManager`; Kotlin keeps only
the `MulticastLock` + permission UX), SPAKE2 PIN pairing + TOFU (Keystore identity +
known-host store), Compose UI (Connect/Settings/Stream) with D-pad/controller focus nav. Built for
`arm64-v8a` + `x86_64`; published to Google Play (Internal Testing) via `android.yml`
(`ci/play-upload.py`). Touch input is the same 3-way model as iOS (2026-07-02): the existing
Trackpad/Direct mouse modes plus new **real multi-touch passthrough**
(`streamTouchPassthrough``nativeSendTouch` → wire TouchDown/Move/Up), a `TouchMode`
Settings dropdown replacing the old trackpad Boolean (migrated on load); not yet
on-device validated. Next: real-device gamepad/HDR live-verify, presenter/latency polish.
2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct
NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~24 ms
at high res).
3. **punktfunk/1 protocol growth.** **Done:** unified host (`serve --gamestream` runs GameStream + the
punktfunk/1 QUIC host in one process; bare `serve` is the secure native-only default — GameStream is
opt-in, trusted-LAN only, security-review #5/#9) with native pairing driven over the mgmt API /
web console (`mod native_pairing`: arm-on-demand → display PIN, paired-device list).
**Done:** PIN pairing is the default, host-gated — the host requires pairing and advertises
`pair=required` unless opted out with `--allow-tofu`/`--open` (then `pair=optional`, accepts
unpaired clients); clients render TOFU only for a `pair=optional` host and force re-pairing on a
fingerprint change. **Done:** concurrent sessions — the accept loop spawns each session
(`--max-concurrent`, default 4, an NVENC bound), each with its own virtual output + encoder, sharing
the host-lifetime input/audio/mic services (shared-desktop multi-view on kwin/mutter/wlroots).
**Done:** delegated pairing approval (§8b-1) — an unpaired device shows up as a pending request in
the web console, one click approves + pins it. Next (see roadmap): gamescope multi-user isolation
(per-session input/audio = independent desktops); §8b-2 peer-push approval from a paired device's
own app.
4. **GameStream host polish**: HDR/10-bit (needs HDR capture + metadata plumbing; `av1_nvenc
-highbitdepth 1` already encodes Main10 from 8-bit input on this box),
reconnect-at-new-mode robustness. AV1 negotiation and surround audio are implemented
and unit/live-capture tested — both still need a live Moonlight confirmation (select
AV1 in a stock client; a real 5.1/7.1 listen incl. FEC under loss).
Box one-time setup is complete: udev rule + `input` group (gamepads validated live),
gamescope 3.16.22 installed system-wide (no PATH override), gnome-shell installed (Mutter
backend validated live). All three compositor backends are live-validated.
## Build / test / run
```sh
cargo build --workspace # green on Linux and macOS
cargo test --workspace # unit + loopback + proptest + C ABI harness (~100 tests)
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
```
Generated artifacts are **checked in** and CI fails on drift: `include/punktfunk_core.h`
(cbindgen from `punktfunk-core/src/abi.rs`) and `api/openapi.json` (regenerate with
`cargo run -p punktfunk-host -- openapi > api/openapi.json`; spec lives in `mgmt.rs`).
CI is Gitea Actions (`.gitea/workflows/`, guide: docs-site `ci.md`): `ci.yml` runs the
workspace checks inside the `git.unom.io/unom/punktfunk-rust-ci` image plus web/docs-site
build+typecheck; `docker.yml` builds+pushes the web/docs/rust-ci images (host and native
clients are deliberately NOT containerized); `apple.yml` builds the xcframework and runs
`swift build`/`swift test` on the `macos-arm64` host-mode runner (home-mac-mini-1,
provisioned by `scripts/ci/setup-macos-runner.sh`). Per-client/host release workflows:
`deb.yml`/`rpm.yml`/`flatpak.yml` (Linux client), `android.yml` (Google Play), `windows-msix.yml`
(Windows client), `windows-host.yml` (Windows host installer), `release.yml` (Apple notarized DMG +
TestFlight), `decky.yml` (Steam Deck plugin); Windows builds run on a self-hosted Windows runner
(`home-windows-runner-1`, vmid 210, `windows-amd64:host` label). The runner is reproducible and
**owned by `unom/infra`**, not this repo, since it's shared across unom Windows projects going
forward: `unom/infra`'s `windows-runner/` Packer template bakes a generic Windows 11 template (OS
install + OpenSSH Server + VS Build Tools/NASM/CMake/LLVM + the act_runner/Node/rustup base, no
registration) on Proxmox once; `proxmox/windows-runner/` (Terraform, `bpg/proxmox`) full-clones it
(agent-based IP discovery, no pre-provisioned DHCP reservation needed) and registers the instance
over SSH remote-exec — the same bake-once/clone-fast split `proxmox/unom-1` uses for the Linux CI
host, just without a native Windows cloud-init (registration goes over `remote-exec`/SSH instead of
`initialization{}`; WinRM was tried first but is deprecated in OpenTofu, so this moved to SSH via
Windows' in-box OpenSSH Server). punktfunk layers its own extras on top of that generic runner:
`scripts/ci/provision-windows-wdk.ps1` (WDK + cargo-wdk for the UMDF drivers) and
`scripts/ci/provision-windows-punktfunk-extras.ps1` (FFmpeg x64/ARM64 trees, Inno Setup, the
`aarch64-pc-windows-msvc` rustup target) — both idempotent, and both run automatically at the start
of every Windows CI job via the shared `scripts/ci/ensure-windows-toolchain.ps1` step (a fast no-op
once already provisioned), rather than a separate manually-dispatched provisioning workflow — that
avoided a real footgun once there could be more than one `windows-amd64` runner: a manually
dispatched provisioning workflow has no way to target a *specific* runner instance, so it could
land on an already-provisioned box instead of the one that actually needed it.
## Layout
```
crates/punktfunk-core/ protocol · FEC · crypto · quic (punktfunk/1 control plane, feature-gated)
crates/punktfunk-host/
gamestream/ Moonlight compat: nvhttp · pairing · rtsp · control · stream · gamepad · apps
vdisplay/linux/{kwin,gamescope,mutter,wlroots}.rs per-compositor client-sized virtual outputs
vdisplay/windows/{pf_vdisplay,manager,identity}.rs all-Rust IddCx virtual display (pf-vdisplay)
linux/zerocopy/{egl,cuda,vulkan}.rs dmabuf → CUDA → NVENC (tiled via EGL/GL, LINEAR via Vulkan)
inject/linux/{libei,wlr,gamepad,dualsense,dualshock4,steam_*}.rs Linux input (uinput xpad · UHID pads · virtual Deck)
inject/windows/{sendinput,gamepad_windows,dualsense_windows,dualshock4_windows}.rs Windows input (UMDF shared-mem pads)
encode/linux/{mod,vaapi}.rs · encode/windows/{nvenc,ffmpeg_win}.rs · encode/sw.rs per-GPU encoders (NVENC/CUDA · VAAPI · AMF/QSV) + GPU-less openh264
capture/{linux/,windows/{dxgi,idd_push}}.rs · audio/{linux/,windows/wasapi_*}.rs
windows/{service,install,interactive}.rs SCM service + in-binary driver/web install
capture.rs · encode.rs · audio.rs · gpu.rs · spike.rs · punktfunk1.rs · mgmt.rs · native_pairing.rs · stats_recorder.rs · library.rs
clients/probe/ punktfunk/1 reference/probe client (headless test/measurement tool)
clients/linux/ native Linux client (GTK4/libadwaita · FFmpeg · PipeWire · SDL3)
clients/windows/ native Windows client (WinUI 3 via windows-reactor · D3D11 · WASAPI · SDL3)
clients/apple/ native macOS/iOS/tvOS client (Swift · VideoToolbox · GameController)
clients/android/ native Android client (Kotlin app + native/ Rust JNI core over punktfunk-core)
clients/decky/ Steam Deck Decky plugin
packaging/windows/drivers/{pf-vdisplay,pf-dualsense,pf-xusb}/ in-house UMDF drivers (built from source in CI)
web/ TanStack web console over the mgmt API (status · devices · pairing · GPU selection · performance graphs)
packaging/ apt(deb) · RPM/COPR · Arch/sysext · Flatpak · Bazzite bootc · Windows host installer (per-dir READMEs)
tools/{loss-harness,latency-probe}/ measurement (plan §10)
scripts/ 60-punktfunk.rules · punktfunk-host.service · host.env.example · headless/
include/punktfunk_core.h generated C header
```
## Design invariants — do not regress
- **One core, linked everywhere.** Protocol/FEC/crypto live only in `punktfunk-core`, behind a
stable, versioned C ABI. `tokio`/`quinn` exist only behind the `quic` feature (control
plane); **no async on the per-frame path** — native threads only.
- **Native client resolution, no scaling.** A session gets a virtual output at exactly the
client's WxH@Hz via the `VirtualDisplay` trait (`create(mode) → VirtualOutput { node_id,
remote_fd, preferred_mode, keepalive }`, RAII teardown). There is no cross-compositor
protocol for this — each compositor keeps its own backend.
- **FEC is the wall-breaker.** GF(2⁸) (≤255 shards/block, Moonlight-compatible) and GF(2¹⁶)
Leopard (≤65535 shards/block) — punktfunk/1 negotiates the latter, removing the ~1 Gbps
ceiling.
- **Core security hardening stays intact**: reassembler bounds attacker-controlled fields
before allocating (`ReassemblerLimits`); AES-GCM per-direction nonce salts + seq-as-AAD;
ABI `struct_size` checks. Regression tests exist — keep them green.
- **PipeWire consumer discipline**: our capture streams set `node.dont-reconnect` and tear
down promptly on negotiation timeout — one wedged link head-blocks the daemon's shared
work queue system-wide.
## Running on this box
Headless QEMU VM (Ubuntu 26.04, kernel 7.0), passthrough RTX 5070 Ti (driver 595 **open**
module — a kernel update silently drops it; reinstall `nvidia-driver-595-open`), no KMS
scanout → KWin `--drm` impossible; everything renders offscreen via `renderD128`.
```sh
# compositor session (shell 1, or the systemd unit in scripts/): full headless Plasma.
# The script sets XDG_MENU_PREFIX=plasma- & co. — without it plasmashell runs but the
# launcher menu is EMPTY (no apps, no System Settings).
bash scripts/headless/run-headless-kde.sh 1920x1080
# host (shell 2): bare `serve` is native-only (secure default); add --gamestream for Moonlight compat.
WAYLAND_DISPLAY=wayland-kde XDG_CURRENT_DESKTOP=KDE PUNKTFUNK_VIDEO_SOURCE=virtual \
PUNKTFUNK_ZEROCOPY=1 cargo run -rp punktfunk-host -- serve --gamestream
# punktfunk/1 native loopback test (no Moonlight needed; same env as serve, listener persists
# across sessions — bound it with --max-sessions):
cargo run -rp punktfunk-host -- punktfunk1-host --source virtual --seconds 10 --max-sessions 1
cargo run -rp punktfunk-probe -- --mode 1280x720x120 --out /tmp/a.h265 --input-test # + --pin HEX
```
Pinned crate facts: `ashpd` 0.13 + `pipewire` 0.9 (must match ashpd's) + `ffmpeg-next` 8.x
(`ffmpeg-sys-next` auto-detects the system FFmpeg, so it builds against **FFmpeg 7.x/libavcodec 61
or 8.x/libavcodec 62** — validated live on Ubuntu 26.04 (8) and Bazzite F43 (7.1); the zero-copy
FFI also link-needs `libGL`/`libgbm`/`libcuda` at build time). Env knobs: `PUNKTFUNK_VIDEO_SOURCE=virtual|portal`,
`PUNKTFUNK_COMPOSITOR=kwin|gamescope|mutter`, `PUNKTFUNK_ZEROCOPY=1|0` (Linux default: ON for
VAAPI/AMD/Intel with a one-shot CPU downgrade if the dmabuf offer never negotiates, OFF/opt-in for
NVENC), `PUNKTFUNK_VAAPI_LOW_POWER=1|0` (pin the VAAPI entrypoint; auto = full-feature then VDEnc
fallback for modern Intel), `PUNKTFUNK_NV12=0` (opt OUT of the default GPU RGB→NV12 convert on the
NVIDIA tiled zero-copy path), `PUNKTFUNK_INTRA_REFRESH=1` (opt-in NVENC intra-refresh loss recovery),
`PUNKTFUNK_PIN_CLOCKS=1` (opt-in NVML GPU clock floor, root-gated), `PUNKTFUNK_GAMESCOPE_APP=...`,
`PUNKTFUNK_INPUT_BACKEND=...`, `PUNKTFUNK_PERF=1` (per-stage timing), `PUNKTFUNK_VIDEO_DROP=N` (FEC
test — injects N% wire-packet loss on BOTH the GameStream and native video paths, no netem needed), `PUNKTFUNK_FEC_PCT=N`, `PUNKTFUNK_DSCP=1` (opt-in DSCP/SO_PRIORITY media QoS on the data +
GameStream video/audio sockets; no-op on the wire on Windows without a qWAVE policy),
`PUNKTFUNK_444=1` (full-chroma HEVC 4:4:4, see below).
**HEVC 4:4:4 (full chroma, Range Extensions)**: opt-in via `PUNKTFUNK_444`, negotiated like 10-bit —
the host emits 4:4:4 only when the client advertised `VIDEO_CAP_444` (wire bit `0x04` + ABI
`PUNKTFUNK_VIDEO_CAP_444`), the codec is HEVC, the session is single-process, **and** a GPU probe
(`encode::can_encode_444`, run before the Welcome) confirms support — else it resolves to 4:2:0 and
`Welcome::chroma_format` reflects the real value (honest downgrade; the client reads it via
`punktfunk_connection_chroma_format`). **punktfunk/1-native only** — GameStream/Moonlight stays 4:2:0
(stock clients can't decode 4:4:4). **NVENC is the implemented path**: Linux `hevc_nvenc` feeds a
swscale'd `yuv444p` (RGB-in is always 4:2:0 — verified on the RTX 5070 Ti — so the session forces CPU
RGB capture for 4:4:4); Windows NVENC keeps ARGB input + FREXT profile + `chromaFormatIDC=3` and the
DDA capturer delivers RGB. VAAPI / AMF / QSV **decline** (probe returns false — no validated 4:4:4
hardware in the lab; they'd produce 4:2:0). Software (openh264) is 4:2:0-only. Test with
`PUNKTFUNK_CLIENT_444=1 punktfunk-probe --out x.h265` then `ffprobe x.h265` (expect `pix_fmt yuv444p`).
*Linux NVENC mechanism validated on the RTX 5070 Ti (ffmpeg CLI); Windows NVENC + 10-bit-4:4:4 not yet
on-glass validated.*
## Conventions
- Rust 2021, `rustfmt` + `clippy -D warnings` clean before commit.
- Match the surrounding code's comment density and naming.
- Commit messages end with the Co-Authored-By trailer (see `git log`).
- `pkill` caution on this box: match exact comm names (`pkill -x gamescope-wl`,
`pkill -x punktfunk-host`) — `pkill -f` self-matches the invoking shell.
Generated
+178 -8
View File
@@ -228,6 +228,67 @@ dependencies = [
"pin-project-lite", "pin-project-lite",
] ]
[[package]]
name = "async-executor"
version = "1.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c96bf972d85afc50bf5ab8fe2d54d1586b4e0b46c97c50a0c9e71e2f7bcd812a"
dependencies = [
"async-task",
"concurrent-queue",
"fastrand",
"futures-lite",
"pin-project-lite",
"slab",
]
[[package]]
name = "async-io"
version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc"
dependencies = [
"autocfg",
"cfg-if",
"concurrent-queue",
"futures-io",
"futures-lite",
"parking",
"polling",
"rustix",
"slab",
"windows-sys 0.61.2",
]
[[package]]
name = "async-lock"
version = "3.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "290f7f2596bd5b78a9fec8088ccd89180d7f9f55b94b0576823bbbdc72ee8311"
dependencies = [
"event-listener",
"event-listener-strategy",
"pin-project-lite",
]
[[package]]
name = "async-process"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc50921ec0055cdd8a16de48773bfeec5c972598674347252c0399676be7da75"
dependencies = [
"async-channel",
"async-io",
"async-lock",
"async-signal",
"async-task",
"blocking",
"cfg-if",
"event-listener",
"futures-lite",
"rustix",
]
[[package]] [[package]]
name = "async-recursion" name = "async-recursion"
version = "1.1.1" version = "1.1.1"
@@ -239,6 +300,30 @@ dependencies = [
"syn", "syn",
] ]
[[package]]
name = "async-signal"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52b5aaafa020cf5053a01f2a60e8ff5dccf550f0f77ec54a4e47285ac2bab485"
dependencies = [
"async-io",
"async-lock",
"atomic-waker",
"cfg-if",
"futures-core",
"futures-io",
"rustix",
"signal-hook-registry",
"slab",
"windows-sys 0.61.2",
]
[[package]]
name = "async-task"
version = "4.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de"
[[package]] [[package]]
name = "async-trait" name = "async-trait"
version = "0.1.89" version = "0.1.89"
@@ -434,6 +519,19 @@ dependencies = [
"generic-array", "generic-array",
] ]
[[package]]
name = "blocking"
version = "1.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21"
dependencies = [
"async-channel",
"async-task",
"futures-io",
"futures-lite",
"piper",
]
[[package]] [[package]]
name = "bumpalo" name = "bumpalo"
version = "3.20.3" version = "3.20.3"
@@ -2002,9 +2100,26 @@ dependencies = [
"libloading", "libloading",
] ]
[[package]]
name = "ksni"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da9eeb3f510b6148ae68f963af2c1fbb0de4d9e4e05f82813cfb319837c3ad2b"
dependencies = [
"async-executor",
"async-io",
"async-lock",
"futures-channel",
"futures-lite",
"futures-util",
"pastey",
"serde",
"zbus",
]
[[package]] [[package]]
name = "latency-probe" name = "latency-probe"
version = "0.5.1" version = "0.7.1"
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
@@ -2136,7 +2251,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]] [[package]]
name = "loss-harness" name = "loss-harness"
version = "0.5.1" version = "0.7.1"
dependencies = [ dependencies = [
"punktfunk-core", "punktfunk-core",
] ]
@@ -2561,6 +2676,12 @@ version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a" checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
[[package]]
name = "pastey"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4"
[[package]] [[package]]
name = "pem" name = "pem"
version = "3.0.6" version = "3.0.6"
@@ -2599,6 +2720,17 @@ version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "piper"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1"
dependencies = [
"atomic-waker",
"fastrand",
"futures-io",
]
[[package]] [[package]]
name = "pipewire" name = "pipewire"
version = "0.9.2" version = "0.9.2"
@@ -2654,6 +2786,20 @@ version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e" checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "polling"
version = "3.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218"
dependencies = [
"cfg-if",
"concurrent-queue",
"hermit-abi",
"pin-project-lite",
"rustix",
"windows-sys 0.61.2",
]
[[package]] [[package]]
name = "polyval" name = "polyval"
version = "0.6.2" version = "0.6.2"
@@ -2729,7 +2875,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-android" name = "punktfunk-client-android"
version = "0.5.1" version = "0.7.1"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"jni", "jni",
@@ -2743,7 +2889,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-linux" name = "punktfunk-client-linux"
version = "0.5.1" version = "0.7.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -2765,7 +2911,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-windows" name = "punktfunk-client-windows"
version = "0.5.1" version = "0.7.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -2788,7 +2934,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-core" name = "punktfunk-core"
version = "0.5.1" version = "0.7.1"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"bytes", "bytes",
@@ -2818,7 +2964,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-host" name = "punktfunk-host"
version = "0.5.1" version = "0.7.1"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
@@ -2881,13 +3027,14 @@ dependencies = [
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)", "windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service", "windows-service",
"winreg", "winreg",
"winresource",
"x509-parser", "x509-parser",
"xkbcommon", "xkbcommon",
] ]
[[package]] [[package]]
name = "punktfunk-probe" name = "punktfunk-probe"
version = "0.5.1" version = "0.7.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"mdns-sd", "mdns-sd",
@@ -2899,6 +3046,23 @@ dependencies = [
"tracing-subscriber", "tracing-subscriber",
] ]
[[package]]
name = "punktfunk-tray"
version = "0.7.1"
dependencies = [
"anyhow",
"ksni",
"libc",
"rustls",
"serde",
"serde_json",
"sha2",
"ureq",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service",
"winresource",
]
[[package]] [[package]]
name = "quick-error" name = "quick-error"
version = "1.2.3" version = "1.2.3"
@@ -5221,8 +5385,14 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eee682d202a77e4a9f3b2c2bdf48a7b28af5c08c34ddf66f98c93e5e39464285" checksum = "eee682d202a77e4a9f3b2c2bdf48a7b28af5c08c34ddf66f98c93e5e39464285"
dependencies = [ dependencies = [
"async-broadcast", "async-broadcast",
"async-executor",
"async-io",
"async-lock",
"async-process",
"async-recursion", "async-recursion",
"async-task",
"async-trait", "async-trait",
"blocking",
"enumflags2", "enumflags2",
"event-listener", "event-listener",
"futures-core", "futures-core",
+2 -1
View File
@@ -4,6 +4,7 @@ members = [
"crates/punktfunk-core", "crates/punktfunk-core",
"crates/punktfunk-host", "crates/punktfunk-host",
"crates/punktfunk-host/vendor/usbip-sim", "crates/punktfunk-host/vendor/usbip-sim",
"crates/punktfunk-tray",
"crates/pf-driver-proto", "crates/pf-driver-proto",
"clients/probe", "clients/probe",
"clients/linux", "clients/linux",
@@ -16,7 +17,7 @@ members = [
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"] exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package] [workspace.package]
version = "0.5.1" version = "0.7.1"
edition = "2021" edition = "2021"
rust-version = "1.82" rust-version = "1.82"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
+6 -4
View File
@@ -15,6 +15,9 @@ your local network.
💬 **Community: [Discord](https://discord.gg/kaPNvzMuGU)** — chat, support, and **Android beta 💬 **Community: [Discord](https://discord.gg/kaPNvzMuGU)** — chat, support, and **Android beta
access** · **[r/Punktfunk](https://www.reddit.com/r/Punktfunk/)**. access** · **[r/Punktfunk](https://www.reddit.com/r/Punktfunk/)**.
🔒 **Security:** found a vulnerability? Report it privately to **security@punktfunk.com** — see
[SECURITY.md](SECURITY.md). Please don't open a public issue.
punktfunk pairs a **virtual-display streaming host** with native clients on every platform. It speaks punktfunk pairs a **virtual-display streaming host** with native clients on every platform. It speaks
the existing **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) client works the existing **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) client works
day one — and adds its own faster **`punktfunk/1`** protocol that breaks the ~1 Gbps FEC wall with a day one — and adds its own faster **`punktfunk/1`** protocol that breaks the ~1 Gbps FEC wall with a
@@ -49,7 +52,7 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
| **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened | | **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened |
| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads | | **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
| **Native protocol**`punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation | | **Native protocol**`punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| **Windows host** (x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green | | **Windows host** (Windows 11 22H2+, x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test | | **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test |
| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch | | **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch |
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, AAudio audio, controllers, discovery, pairing | | **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, AAudio audio, controllers, discovery, pairing |
@@ -61,7 +64,7 @@ The **GameStream host works with a stock Moonlight client** — validated live o
and **video at the client's exact resolution and refresh** via a per-session virtual output (KWin, and **video at the client's exact resolution and refresh** via a per-session virtual output (KWin,
gamescope, Mutter, and Sway/wlroots backends), encoded with GPU **zero-copy** (dmabuf → CUDA/Vulkan → gamescope, Mutter, and Sway/wlroots backends), encoded with GPU **zero-copy** (dmabuf → CUDA/Vulkan →
NVENC) up to 5120×1440@240. The native **`punktfunk/1`** protocol adds a QUIC control plane and a NVENC) up to 5120×1440@240. The native **`punktfunk/1`** protocol adds a QUIC control plane and a
GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→reassembled at 720p120), with GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→received at 720p120), with
mid-stream mode renegotiation and a wall-clock skew handshake so latency stays valid across machines. mid-stream mode renegotiation and a wall-clock skew handshake so latency stays valid across machines.
Both run from **one process**: bare `punktfunk-host serve` is the **secure native-only default** Both run from **one process**: bare `punktfunk-host serve` is the **secure native-only default**
(`punktfunk/1` + the management API/web console), and `serve --gamestream` additionally enables the (`punktfunk/1` + the management API/web console), and `serve --gamestream` additionally enables the
@@ -82,7 +85,7 @@ Windows host also ships as a signed installer (all-vendor: NVIDIA, AMD, Intel).
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [Ubuntu — GNOME](https://docs.punktfunk.unom.io/docs/ubuntu-gnome) · [KDE](https://docs.punktfunk.unom.io/docs/ubuntu-kde) | | **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [Ubuntu — GNOME](https://docs.punktfunk.unom.io/docs/ubuntu-gnome) · [KDE](https://docs.punktfunk.unom.io/docs/ubuntu-kde) |
| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(or the bootc image)* | [Fedora — KDE](https://docs.punktfunk.unom.io/docs/fedora-kde) · [Bazzite](https://docs.punktfunk.unom.io/docs/bazzite) | | **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(or the bootc image)* | [Fedora — KDE](https://docs.punktfunk.unom.io/docs/fedora-kde) · [Bazzite](https://docs.punktfunk.unom.io/docs/bazzite) |
| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS)* | [packaging/arch](packaging/arch/README.md) | | **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS)* | [packaging/arch](packaging/arch/README.md) |
| **Windows** (x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) | | **Windows** (11 22H2+, x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status). `punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status).
After install, run `punktfunk-host serve` inside your desktop session (the secure native default; After install, run `punktfunk-host serve` inside your desktop session (the secure native default;
@@ -138,7 +141,6 @@ clients/
web/ web console (TanStack) over the management API — status · devices · pairing · GPUs · performance · logs web/ web console (TanStack) over the management API — status · devices · pairing · GPUs · performance · logs
packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image
docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io
design/ design notes & deep-dive plans (index: design/README.md)
include/punktfunk_core.h cbindgen-generated C header (checked in) include/punktfunk_core.h cbindgen-generated C header (checked in)
tools/ latency-probe · loss-harness (measurement) tools/ latency-probe · loss-harness (measurement)
``` ```
+69
View File
@@ -0,0 +1,69 @@
# Security Policy
punktfunk is a low-latency desktop/game streaming stack. A host is effectively remote control of a
machine, so we take security reports seriously and appreciate responsible disclosure.
## Reporting a vulnerability
**Please report security issues privately by email to security@punktfunk.com.**
Do **not** open a public issue, pull request, or chat/forum post for a suspected vulnerability — that
exposes other users before a fix exists.
### What to include
The more of this you can give us, the faster we can act:
- The component and version (e.g. `punktfunk-host 0.6.0`, Windows or Linux, which client).
- The impact — what an attacker can do, and from what position (same LAN, a local service account,
admin, a paired client, …).
- Steps to reproduce, a proof-of-concept, or a crash/log if you have one.
- Any suggested fix or mitigation (optional).
## What to expect
We're a small team, so timelines are best-effort, but we commit to:
- **Acknowledge** your report within **3 business days**.
- Give an **initial assessment** (severity + whether we can reproduce) within about **7 days**.
- Keep you updated, and tell you when a fix ships.
- **Credit** you in the advisory / release notes when the fix is public — unless you'd rather stay
anonymous.
We practice **coordinated disclosure**: please give us reasonable time to release a fix before
publishing details. We aim to resolve valid issues within **90 days** and will agree a disclosure
date with you.
## Scope
In scope — the code in this repository:
- The host (`punktfunk-host`), its Windows drivers, and the protocol/crypto core (`punktfunk-core`).
- The native clients (Apple, Linux, Windows, Android), the web management console, and the management
API.
Known limits — documented behavior, not vulnerabilities (see
https://docs.punktfunk.unom.io/docs/security):
- **Admin/SYSTEM already on the host = out of scope.** An attacker who is already administrator or
SYSTEM on the host owns the machine regardless of punktfunk.
- **The virtual display is a real monitor** — any process already in the interactive desktop session
can capture it via the normal OS screen-capture APIs, exactly as it could a physical monitor.
- **GameStream/Moonlight compatibility** (`--gamestream`) uses legacy encryption and is documented as
opt-in, trusted-LAN-only.
- **Public-internet exposure is unsupported** — issues that only arise from exposing the host to the
WAN are expected; keep the host on a trusted LAN or a VPN.
If you're unsure whether something is in scope, report it anyway — we'd rather hear about it.
## Safe harbor
We consider good-faith security research that follows this policy to be authorized, and we won't
pursue legal action against researchers who:
- make a good-faith effort to avoid privacy violations, data loss, and service disruption,
- only test systems they own or have explicit permission to test,
- give us reasonable time to remediate before public disclosure,
- don't exfiltrate more data than needed to demonstrate the issue.
Thank you for helping keep punktfunk and its users safe.
+96 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0", "name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0" "identifier": "MIT OR Apache-2.0"
}, },
"version": "0.5.1" "version": "0.6.0"
}, },
"paths": { "paths": {
"/api/v1/clients": { "/api/v1/clients": {
@@ -578,6 +578,41 @@
} }
} }
}, },
"/api/v1/local/summary": {
"get": {
"tags": [
"host"
],
"summary": "Local status summary for the tray icon",
"description": "Non-sensitive status (counts and booleans only — no PIN values, no fingerprints, no device\nnames). Unauthenticated, but served to loopback peers only.",
"operationId": "getLocalSummary",
"responses": {
"200": {
"description": "Non-sensitive local host status (loopback peers only)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/LocalSummary"
}
}
}
},
"401": {
"description": "Non-loopback peer",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
},
"security": [
{}
]
}
},
"/api/v1/logs": { "/api/v1/logs": {
"get": { "get": {
"tags": [ "tags": [
@@ -2083,6 +2118,66 @@
} }
} }
}, },
"LocalSummary": {
"type": "object",
"description": "Non-sensitive host status for the local tray icon: counts and booleans only — no PIN values,\nno fingerprints, no device names. Served unauthenticated to LOOPBACK peers only (see\n`require_auth`): the bearer-token file is SYSTEM/Administrators-DACL'd on Windows, so the\nper-user tray process cannot authenticate — this narrow read-only route is its status source.",
"required": [
"version",
"video_streaming",
"audio_streaming",
"paired_clients",
"native_paired_clients",
"pin_pending",
"pending_approvals"
],
"properties": {
"audio_streaming": {
"type": "boolean",
"description": "True while the audio stream thread is running."
},
"native_paired_clients": {
"type": "integer",
"format": "int32",
"description": "Number of paired native (punktfunk/1) devices.",
"minimum": 0
},
"paired_clients": {
"type": "integer",
"format": "int32",
"description": "Number of pinned (paired) GameStream client certificates.",
"minimum": 0
},
"pending_approvals": {
"type": "integer",
"format": "int32",
"description": "Native pairing knocks awaiting the operator's approval (count only).",
"minimum": 0
},
"pin_pending": {
"type": "boolean",
"description": "True while a GameStream pairing handshake is parked waiting for the user's PIN."
},
"session": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/SessionInfo",
"description": "The active launch session (set by Moonlight's `/launch`, cleared on cancel/stop)."
}
]
},
"version": {
"type": "string",
"description": "Host version (mirrors `/health`)."
},
"video_streaming": {
"type": "boolean",
"description": "True while the video stream thread is running."
}
}
},
"LogEntry": { "LogEntry": {
"type": "object", "type": "object",
"description": "One captured log event.", "description": "One captured log event.",
@@ -27,8 +27,15 @@
<uses-feature android:name="android.software.leanback" android:required="false" /> <uses-feature android:name="android.software.leanback" android:required="false" />
<uses-feature android:name="android.hardware.gamepad" android:required="false" /> <uses-feature android:name="android.hardware.gamepad" android:required="false" />
<!-- appCategory="game": a game-streaming client IS a game as far as the SoC is concerned.
On Snapdragon devices (and other OEMs with a Game Mode / Game Dashboard) this makes the app
eligible for the vendor's game performance profile — the aggressive CPU/GPU governor and
scheduler treatment games get — which, together with the ADPF hints in the native decode
path, is what keeps clocks up for low, consistent decode latency. Also groups it correctly
under Games in battery/data usage. Advisory: devices without Game Mode ignore it. -->
<application <application
android:allowBackup="false" android:allowBackup="false"
android:appCategory="game"
android:icon="@mipmap/ic_launcher" android:icon="@mipmap/ic_launcher"
android:roundIcon="@mipmap/ic_launcher_round" android:roundIcon="@mipmap/ic_launcher_round"
android:label="@string/app_name" android:label="@string/app_name"
@@ -15,11 +15,16 @@ import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt import kotlin.math.roundToInt
/** /**
* The live stats overlay — mirrors the Apple client's HUD. Reads the 14-double layout from * The live stats overlay — the unified HUD (`design/stats-unification.md`, Android v1: headline is
* `capture→decoded`, tiled by `host+network` + `decode`). Reads the 18-double layout from
* [NativeBridge.nativeVideoStats]: * [NativeBridge.nativeVideoStats]:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped, bitDepth, colorPrimaries, * `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lost, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc]`. The trailing four (present on a current native lib) describe the * colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms]`. Indexes 1013
* negotiated video feed and render as a codec/depth/colour/chroma line; older layouts just omit it. * (present on a current native lib) describe the negotiated video feed and render as a
* codec/depth/colour/chroma line; 14/15 render as the stage equation — split into
* `host + network + decode` when the Phase-2 terms at 16/17 are nonzero (a current host sends
* per-AU 0xCF timings; an old host leaves them 0 and the combined `host+network` term stands);
* older layouts just omit those lines.
*/ */
@Composable @Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) { internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
@@ -29,7 +34,7 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
val hz = s[8].toInt() val hz = s[8].toInt()
val latValid = s[4] != 0.0 val latValid = s[4] != 0.0
val skew = s[5] != 0.0 val skew = s[5] != 0.0
val dropped = s[9].toLong() val lost = s[9].toLong()
Column( Column(
modifier = modifier modifier = modifier
.background(Color.Black.copy(alpha = 0.45f), RoundedCornerShape(6.dp)) .background(Color.Black.copy(alpha = 0.45f), RoundedCornerShape(6.dp))
@@ -50,17 +55,33 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
) )
} }
if (latValid) { if (latValid) {
val tag = if (skew) "" else " (same-host)" val tag = if (skew) "" else " (same-host clock)"
Text( Text(
"capture→client ${"%.1f".format(s[2])}/${"%.1f".format(s[3])} ms p50/p95$tag", "end-to-end ${"%.1f".format(s[2])} ms p50 · ${"%.1f".format(s[3])} p95 · capture→decoded$tag",
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
if (s.size >= 16) {
// Phase-2 split (s[16]/s[17]): render `host + network` separately when the host
// reported its share this window; otherwise the combined term (old host / no
// matched 0xCF timing).
val equation = if (s.size >= 18 && s[16] > 0) {
"= host ${"%.1f".format(s[16])} + network ${"%.1f".format(s[17])} + decode ${"%.1f".format(s[15])}"
} else {
"= host+network ${"%.1f".format(s[14])} + decode ${"%.1f".format(s[15])}"
}
Text(
equation,
color = Color.White, color = Color.White,
fontFamily = FontFamily.Monospace, fontFamily = FontFamily.Monospace,
fontSize = 12.sp, fontSize = 12.sp,
) )
} }
if (dropped > 0) { }
if (lost > 0) {
Text( Text(
"dropped $dropped", "lost $lost",
color = Color(0xFFFFB0B0), color = Color(0xFFFFB0B0),
fontFamily = FontFamily.Monospace, fontFamily = FontFamily.Monospace,
fontSize = 12.sp, fontSize = 12.sp,
+19 -2
View File
@@ -37,13 +37,30 @@ def call(method, url, token=None, data=None, content_type=None, want_json=True):
headers["Authorization"] = f"Bearer {token}" headers["Authorization"] = f"Bearer {token}"
if content_type: if content_type:
headers["Content-Type"] = content_type headers["Content-Type"] = content_type
# Transient-fault retries: googleapis.com occasionally drops the TLS session ("EOF
# occurred in violation of protocol" — failed two release uploads on 2026-07-02) or
# answers 5xx. Retry those with backoff; 4xx raises immediately (a real API error).
# The edits API is transactional until commit, so re-sending any of these is safe.
last = None
for attempt in range(4):
if attempt:
delay = 3**attempt
print(f"transient Play API failure ({last}); retry {attempt}/3 in {delay}s")
time.sleep(delay)
req = urllib.request.Request(url, data=data, method=method, headers=headers) req = urllib.request.Request(url, data=data, method=method, headers=headers)
try: try:
with urllib.request.urlopen(req, timeout=300) as r: with urllib.request.urlopen(req, timeout=300) as r:
body = r.read() body = r.read()
except urllib.error.HTTPError as e:
raise ApiError(e.code, method, url, e.read().decode("utf-8", "replace"))
return json.loads(body) if (want_json and body) else body return json.loads(body) if (want_json and body) else body
except urllib.error.HTTPError as e:
if e.code >= 500:
last = f"HTTP {e.code}"
continue
raise ApiError(e.code, method, url, e.read().decode("utf-8", "replace"))
except urllib.error.URLError as e:
last = str(getattr(e, "reason", e))
continue
sys.exit(f"ERROR: {method} {url} still failing after retries: {last}")
def load_sa(): def load_sa():
@@ -105,12 +105,17 @@ object NativeBridge {
/** /**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs. * Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 14 doubles: * Returns 18 doubles (unified stats spec, `design/stats-unification.md`):
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped, * `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]` * bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
* (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — bit depth * netP50Ms]`
* 8/10, CICP primaries/transfer, and the HEVC chroma_format_idc 1=4:2:0 / 3=4:4:4). Poll ~1 Hz; * (the two flags are 1.0/0.0; indexes 2/3 are the end-to-end capture→decoded headline; 1013
* each call resets the measurement window. * describe the negotiated video feed — bit depth 8/10, CICP primaries/transfer, and the HEVC
* chroma_format_idc 1=4:2:0 / 3=4:4:4; 14/15 are the stage p50s tiling the headline —
* `host+network` = capture→received, `decode` = received→decoded; 16/17 split the
* `host+network` term via the host's per-AU 0xCF timings — `host` = the host's capture→sent,
* `network` = the remainder — both 0.0 when no timing matched this window, i.e. an old host).
* Poll ~1 Hz; each call resets the measurement window.
*/ */
external fun nativeVideoStats(handle: Long): DoubleArray? external fun nativeVideoStats(handle: Long): DoubleArray?
+137
View File
@@ -0,0 +1,137 @@
//! Android Adaptive Performance Framework (ADPF) — CPU performance hints for the decode thread.
//!
//! ADPF lets a latency-critical app tell the platform "these threads run a repeating workload with
//! this per-cycle deadline, and here's how long they *actually* took." The kernel's CPU governor
//! (on Qualcomm Snapdragon in particular — its ADPF backend is among the most responsive) then keeps
//! those threads on the fast cores at high clocks instead of migrating them to a little core or
//! down-clocking between frames. For a stream client the win is on the in-process hot path we
//! control — the `pf-decode` feed/drain/present loop — *not* the hardware codec itself (that decodes
//! in the mediacodec service, a separate process we can't hint); keeping our loop from being
//! scheduled late directly trims the jitter between "AU received" and "buffer released to the
//! Surface." It complements the codec-side `operating-rate`/`priority` hints, which push the codec's
//! own clocks.
//!
//! The `APerformanceHint_*` API arrived in NDK **API level 33**. minSdk is 31, so we CANNOT link the
//! symbols directly: a `libpunktfunk_android.so` carrying an unresolved
//! `APerformanceHint_createSession` import fails to load on API 31/32 devices
//! (`System.loadLibrary` throws) even if the code path is never taken. Instead we resolve the
//! entry points from `libandroid.so` with `dlsym` at runtime — absent on < 33 ⇒
//! [`HintSession::create`] returns `None` and the decode loop simply runs without hints.
use std::ffi::c_void;
use std::os::raw::c_int;
// `APerformanceHint_*` function-pointer types. The manager/session handles are opaque, so we treat
// them as `*mut c_void`.
type GetManagerFn = unsafe extern "C" fn() -> *mut c_void;
type CreateSessionFn = unsafe extern "C" fn(*mut c_void, *const i32, usize, i64) -> *mut c_void;
type ReportFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type UpdateTargetFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type CloseFn = unsafe extern "C" fn(*mut c_void);
/// The entry points we use, resolved once from `libandroid.so`, plus the process-wide manager.
struct Api {
create_session: CreateSessionFn,
report: ReportFn,
update_target: UpdateTargetFn,
close: CloseFn,
manager: *mut c_void,
}
/// Resolve the ADPF entry points + the process manager, or `None` on API < 33 (symbols absent) or if
/// the manager is unavailable.
fn resolve_api() -> Option<Api> {
// SAFETY: `dlopen` of an always-present system library with a NUL-terminated name; it returns
// null on failure (checked below). `libandroid.so` is already mapped into every app process, so
// this only bumps its refcount — we intentionally never `dlclose` (process-lifetime handle).
let lib = unsafe { libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW) };
if lib.is_null() {
return None;
}
// SAFETY: `dlsym` on the valid handle above with NUL-terminated symbol names; each returns null
// when the symbol is absent (device API < 33), which we check before transmuting the non-null
// pointer to its fn-pointer type (layout-compatible; a resolved symbol is a valid code address).
unsafe {
let get_manager = libc::dlsym(lib, c"APerformanceHint_getManager".as_ptr());
let create_session = libc::dlsym(lib, c"APerformanceHint_createSession".as_ptr());
let report = libc::dlsym(lib, c"APerformanceHint_reportActualWorkDuration".as_ptr());
let update_target = libc::dlsym(lib, c"APerformanceHint_updateTargetWorkDuration".as_ptr());
let close = libc::dlsym(lib, c"APerformanceHint_closeSession".as_ptr());
if get_manager.is_null()
|| create_session.is_null()
|| report.is_null()
|| update_target.is_null()
|| close.is_null()
{
return None; // device API < 33 — no ADPF
}
let get_manager = std::mem::transmute::<*mut c_void, GetManagerFn>(get_manager);
let manager = get_manager();
if manager.is_null() {
return None;
}
Some(Api {
create_session: std::mem::transmute::<*mut c_void, CreateSessionFn>(create_session),
report: std::mem::transmute::<*mut c_void, ReportFn>(report),
update_target: std::mem::transmute::<*mut c_void, UpdateTargetFn>(update_target),
close: std::mem::transmute::<*mut c_void, CloseFn>(close),
manager,
})
}
}
/// A live ADPF hint session bound to a set of thread ids. Dropping it closes the session. Holds raw
/// handles, so it is `!Send`/`!Sync` — created and used only on the `pf-decode` thread.
pub struct HintSession {
api: Api,
session: *mut c_void,
}
impl HintSession {
/// Open a session hinting `tids` with an initial per-frame target of `target_ns` nanoseconds.
/// `None` when ADPF is unavailable (device API < 33) or the platform declines — the caller then
/// runs unhinted (a no-op, not an error).
pub fn create(target_ns: i64, tids: &[i32]) -> Option<Self> {
if target_ns <= 0 || tids.is_empty() {
return None;
}
let api = resolve_api()?;
// SAFETY: `api.manager` is the live process manager returned above; `tids` is a valid slice
// of `len` i32s that `createSession` copies; it returns null on failure (checked).
let session =
unsafe { (api.create_session)(api.manager, tids.as_ptr(), tids.len(), target_ns) };
if session.is_null() {
return None;
}
Some(Self { api, session })
}
/// Report the wall-clock time the hinted thread spent producing the last displayed frame. When
/// it exceeds the session target the governor boosts the cores running the thread; when it
/// stays under, clocks may relax. No-op on a non-positive duration (the API rejects it).
pub fn report_actual(&self, actual_ns: i64) {
if actual_ns <= 0 {
return;
}
// SAFETY: `self.session` is a live session for `self`'s lifetime.
unsafe { (self.api.report)(self.session, actual_ns) };
}
/// Update the per-frame target (e.g. after a mid-session refresh-rate change). Unused today —
/// the decode thread restarts on renegotiation — but kept for that path.
#[allow(dead_code)]
pub fn update_target(&self, target_ns: i64) {
if target_ns <= 0 {
return;
}
// SAFETY: `self.session` is a live session for `self`'s lifetime.
unsafe { (self.api.update_target)(self.session, target_ns) };
}
}
impl Drop for HintSession {
fn drop(&mut self) {
// SAFETY: `self.session` was created by `createSession` and is closed exactly once, here.
unsafe { (self.api.close)(self.session) };
}
}
+4
View File
@@ -324,6 +324,10 @@ fn decode_loop(
counters: Arc<Counters>, counters: Arc<Counters>,
channels: usize, channels: usize,
) { ) {
// Fold this Opus→AAudio thread into the client's hot-thread set so the ADPF session the decode
// thread opens also keeps audio decode on a fast core (registered before the video pump's first
// frame arrives, so it's captured when that session is created). No-op below API 33.
client.register_hot_thread();
// Interleaved f32 samples per millisecond at this layout — the ring's 5 ms reserve check below. // Interleaved f32 samples per millisecond at this layout — the ring's 5 ms reserve check below.
let ms = (SAMPLE_RATE as usize / 1000) * channels; let ms = (SAMPLE_RATE as usize / 1000) * channels;
// Opus decode scratch: worst-case 120 ms frame (5760 samples/ch) × channels. // Opus decode scratch: worst-case 120 ms frame (5760 samples/ch) × channels.
+168 -9
View File
@@ -9,16 +9,27 @@
use ndk::data_space::DataSpace; use ndk::data_space::DataSpace;
use ndk::media::media_codec::{ use ndk::media::media_codec::{
DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection, DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection,
OutputBuffer,
}; };
use ndk::media::media_format::MediaFormat; use ndk::media::media_format::MediaFormat;
use ndk::native_window::{FrameRateCompatibility, NativeWindow}; use ndk::native_window::{FrameRateCompatibility, NativeWindow};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError; use punktfunk_core::error::PunktfunkError;
use punktfunk_core::session::Frame; use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc; use std::sync::Arc;
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
/// Cap on the pts→received-timestamp map below: MediaCodec holds only a handful of frames in
/// flight, so anything beyond this is stale (codec flushed / HUD toggled) and gets evicted.
const IN_FLIGHT_CAP: usize = 64;
/// Cap on received AUs awaiting their 0xCF host timing (Phase 2 host/network split): the timing
/// datagram trails its AU by at most the wire, so a match lands within a frame or two — anything
/// this deep is a lost datagram (or an old host that never sends any) and gets evicted.
const PENDING_SPLIT_CAP: usize = 256;
/// The decode loop. Runs on the `pf-decode` thread until `shutdown` is set or the session closes. /// The decode loop. Runs on the `pf-decode` thread until `shutdown` is set or the session closes.
pub fn run( pub fn run(
client: Arc<NativeClient>, client: Arc<NativeClient>,
@@ -61,7 +72,14 @@ pub fn run(
// realtime priority + the target frame rate, so vendor decoders (e.g. Qualcomm) run at full // realtime priority + the target frame rate, so vendor decoders (e.g. Qualcomm) run at full
// clocks instead of a power-saving cadence that adds dequeue latency. // clocks instead of a power-saving cadence that adds dequeue latency.
format.set_i32("priority", 0); // 0 = realtime format.set_i32("priority", 0); // 0 = realtime
format.set_i32("operating-rate", mode.refresh_hz as i32); // Operating rate = the codec's clock hint. Setting it to the display rate merely asks the
// decoder to *sustain* that cadence — a Qualcomm decoder can meet 60/120 fps at a power-saving
// clock that adds a millisecond-plus of decode latency per frame. Setting it to the AOSP
// "unbounded" sentinel (Short.MAX) instead asks the decoder to run each frame at max clocks and
// finish ASAP, minimising per-frame decode latency — the right trade for a real-time stream
// (costs power/heat; the dial to lower if a device thermally throttles over a long session).
// Ignored where unsupported.
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
// HDR static metadata (ST.2086 mastering + content light level): when an HDR session was // HDR static metadata (ST.2086 mastering + content light level): when an HDR session was
// negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade. // negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade.
@@ -104,6 +122,25 @@ pub fn run(
); );
} }
// ADPF: hint the platform that the whole video pipeline — this pf-decode feed/drain/present
// loop, the core's data-plane pump (UDP receive + FEC reassembly), and the audio thread — runs a
// per-frame real-time workload, so the CPU governor keeps those threads on fast cores at high
// clocks instead of down-clocking between frames or parking them on a little core. Snapdragon's
// ADPF backend responds well to this. We register this thread now but create the session lazily
// on the first presented frame: by then the pump + audio threads have registered their ids too,
// and ADPF `createSession` rejects a set with any not-yet-live/dead tid. No-op below API 33.
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread(); // this decode thread → the pipeline's hot-thread set
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
// Accumulates the loop's productive (feed+drain) time between displayed frames; reported to ADPF
// once per rendered frame against the frame-period target.
let mut work_accum_ns: i64 = 0;
let mut fed: u64 = 0; let mut fed: u64 = 0;
let mut rendered: u64 = 0; let mut rendered: u64 = 0;
let mut discarded: u64 = 0; let mut discarded: u64 = 0;
@@ -115,9 +152,19 @@ pub fn run(
// climbs. // climbs.
let mut last_dropped = client.frames_dropped(); let mut last_dropped = client.frames_dropped();
let mut last_kf_req: Option<Instant> = None; let mut last_kf_req: Option<Instant> = None;
// Capture→client-receipt latency uses the negotiated host-minus-client clock offset (0 if the // Skew-corrected latency stats (spec: design/stats-unification.md) use the negotiated
// host didn't answer the skew handshake — then the HUD flags it "same-host"). // host-minus-client clock offset (0 if the host didn't answer the skew handshake — then the
// HUD flags it "(same-host clock)").
let clock_offset = client.clock_offset_ns; let clock_offset = client.clock_offset_ns;
// HUD stage split: receipt timestamps keyed by the pts we queue into the codec, so the decoded
// point (output-buffer dequeue — MediaCodec round-trips presentationTimeUs) can be paired back
// to its receipt for the `decode` stage. Only fed while the HUD is visible.
let mut in_flight: VecDeque<(u64, i128)> = VecDeque::new();
// Phase-2 host/network split (design/stats-unification.md): received AUs awaiting their 0xCF
// host timing, as (pts_ns, capture→received µs). The timings are drained non-blockingly right
// where receipts are recorded and matched by pts; `network = hostnet host` (saturating).
// Only fed while the HUD is visible; an old host never sends a 0xCF, so entries just age out.
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
// The dataspace we've signalled on the Surface so far (None = default/SDR). Set reactively once // The dataspace we've signalled on the Surface so far (None = default/SDR). Set reactively once
// the decoder reports an HDR stream (see `drain`); avoids re-applying every format event. // the decoder reports an HDR stream (see `drain`); avoids re-applying every format event.
let mut applied_ds: Option<DataSpace> = None; let mut applied_ds: Option<DataSpace> = None;
@@ -138,15 +185,41 @@ pub fn run(
&p[..p.len().min(6)] &p[..p.len().min(6)]
); );
} }
// HUD stat: capture→client-receipt latency = client_now + (hostclient) // HUD stat, `received` point: host+network = client_now + (hostclient)
// capture_pts. Gated on the HUD being visible — `enabled` first so the hidden // capture_pts. Gated on the HUD being visible — `enabled` first so the hidden
// steady state skips the wall-clock read and the lock entirely. // steady state skips the wall-clock read and the lock entirely. The receipt
// stamp is also parked in `in_flight` (keyed by the pts the codec will echo on
// the output buffer) for the decoded-point pairing in `drain`.
if stats.enabled() { if stats.enabled() {
let lat_ns = let received_ns = now_realtime_ns();
now_realtime_ns() + clock_offset as i128 - frame.pts_ns as i128; let lat_ns = received_ns + clock_offset as i128 - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000) let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64); .then_some((lat_ns / 1000) as u64);
stats.note(frame.data.len(), lat_us, clock_offset != 0); stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
in_flight.push_back((frame.pts_ns / 1000, received_ns));
if in_flight.len() > IN_FLIGHT_CAP {
in_flight.pop_front(); // stale — codec never echoed it back
}
// Phase-2 split: park this AU's capture→received sample, then match any
// 0xCF host timings that have arrived — host = the host's own
// capture→sent, network = our capture→received minus it (per-frame
// tiling; saturating in case of clock jitter).
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front(); // 0xCF lost / old host — evict
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
} }
pending = Some(frame); pending = Some(frame);
} }
@@ -154,6 +227,9 @@ pub fn run(
Err(_) => break, // session closed Err(_) => break, // session closed
} }
} }
// Time the productive work (feed + drain) only — the `next_frame` poll wait above is idle
// and excluded, so ADPF sees this thread's real per-frame CPU cost, not the poll timeout.
let work_t0 = Instant::now();
if let Some(frame) = pending.take() { if let Some(frame) = pending.take() {
if feed(&codec, &frame.data, frame.pts_ns / 1000) { if feed(&codec, &frame.data, frame.pts_ns / 1000) {
fed += 1; fed += 1;
@@ -173,10 +249,48 @@ pub fn run(
} else { } else {
Duration::ZERO Duration::ZERO
}; };
let (r, d) = drain(&codec, &window, &mut applied_ds, wait); let (r, d) = drain(
&codec,
&window,
&mut applied_ds,
wait,
&stats,
&mut in_flight,
clock_offset,
);
rendered += r; rendered += r;
discarded += d; discarded += d;
// ADPF: attribute this iteration's feed+drain time to the frame being produced, and report
// the accumulated per-frame work once one is actually presented (r > 0). Under back-pressure
// the short output-dequeue wait is included in the tally — for a latency-first client,
// biasing the governor toward "boost" is the desired behaviour. Cheap when `hint` is None
// (one `Instant` diff, no report).
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if r > 0 {
if !hint_tried {
// First presented frame: the pump + audio threads have registered their ids by now.
// Build one ADPF session over the whole pipeline's thread set (empty below API 33,
// or where the platform declines → `None`, and the loop runs unhinted).
hint_tried = true;
let tids = client.hot_thread_ids();
hint = crate::adpf::HintSession::create(frame_period_ns, &tids);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
}
// Loss recovery: under infinite GOP the only recovery keyframe is one we request. The // Loss recovery: under infinite GOP the only recovery keyframe is one we request. The
// reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the // reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the
// reference-missing delta frames that follow and renders them without error, so keying off // reference-missing delta frames that follow and renders them without error, so keying off
@@ -271,11 +385,19 @@ fn feed(codec: &MediaCodec, au: &[u8], pts_us: u64) -> bool {
/// the caller's input is blocked so the loop waits on decoder progress instead of busy-spinning. /// the caller's input is blocked so the loop waits on decoder progress instead of busy-spinning.
/// Returns `(rendered, discarded)`. Also reacts to `OutputFormatChanged` (which can interleave /// Returns `(rendered, discarded)`. Also reacts to `OutputFormatChanged` (which can interleave
/// between buffers — handled without losing the held buffer) to signal HDR on the Surface. /// between buffers — handled without losing the held buffer) to signal HDR on the Surface.
///
/// Each dequeued buffer is also the HUD's `decoded` measurement point (rendered or not — the frame
/// finished decoding either way): end-to-end = decoded + clock_offset capture pts, and the
/// `decode` stage pairs the buffer's echoed presentationTimeUs back to the receipt stamp in
/// `in_flight` (single-clock local difference, no skew involved).
fn drain( fn drain(
codec: &MediaCodec, codec: &MediaCodec,
window: &NativeWindow, window: &NativeWindow,
applied_ds: &mut Option<DataSpace>, applied_ds: &mut Option<DataSpace>,
first_wait: Duration, first_wait: Duration,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
) -> (u64, u64) { ) -> (u64, u64) {
let mut held = None; // newest ready buffer so far, presented after the loop let mut held = None; // newest ready buffer so far, presented after the loop
let mut discarded: u64 = 0; let mut discarded: u64 = 0;
@@ -284,6 +406,9 @@ fn drain(
match codec.dequeue_output_buffer(wait) { match codec.dequeue_output_buffer(wait) {
Ok(DequeuedOutputBufferInfoResult::Buffer(buf)) => { Ok(DequeuedOutputBufferInfoResult::Buffer(buf)) => {
wait = Duration::ZERO; // only the first dequeue may block wait = Duration::ZERO; // only the first dequeue may block
if stats.enabled() {
note_decoded(stats, in_flight, clock_offset, &buf);
}
if let Some(stale) = held.replace(buf) { if let Some(stale) = held.replace(buf) {
// A newer frame is ready — drop the held one without rendering. // A newer frame is ready — drop the held one without rendering.
if let Err(e) = codec.release_output_buffer(stale, false) { if let Err(e) = codec.release_output_buffer(stale, false) {
@@ -333,6 +458,40 @@ fn drain(
(rendered, discarded) (rendered, discarded)
} }
/// HUD `decoded` point for one dequeued output buffer: build the end-to-end (capture→decoded,
/// skew-corrected, clamped to (0, 10 s)) and `decode` (received→decoded, single-clock local, ≥ 0)
/// samples and hand them to [`crate::stats::VideoStats::note_decoded`]. The codec echoes the input
/// `presentationTimeUs` on the output buffer, which keys the receipt stamp in `in_flight`; entries
/// older than the echoed pts are evicted (decode order == input order here — low-latency, no
/// B-frames — so anything before it was dropped inside the codec or stamped before a flush).
fn note_decoded(
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
buf: &OutputBuffer<'_>,
) {
let pts_us = buf.info().presentation_time_us().max(0) as u64;
let decoded_ns = now_realtime_ns();
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
in_flight.pop_front();
if p == pts_us {
received_ns = Some(r);
break;
}
}
// pts_us is the truncated frame.pts_ns/1000 we queued, so ×1000 re-approximates capture time
// to < 1 µs — negligible against the ms-scale figures shown.
let e2e_ns = decoded_ns + clock_offset as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
let decode_us = received_ns.map(|r| ((decoded_ns - r).max(0) / 1000) as u64);
stats.note_decoded(e2e_us, decode_us);
}
/// Map the decoder's reported output colour to a BT.2020 HDR dataspace, or `None` for SDR. The /// Map the decoder's reported output colour to a BT.2020 HDR dataspace, or `None` for SDR. The
/// integer values are the Android MediaFormat colour constants the NDK shares: COLOR_TRANSFER /// integer values are the Android MediaFormat colour constants the NDK shares: COLOR_TRANSFER
/// ST2084 = 6 (PQ/HDR10), HLG = 7; COLOR_RANGE FULL = 1, LIMITED = 2 (the host encodes limited). /// ST2084 = 6 (PQ/HDR10), HLG = 7; COLOR_RANGE FULL = 1, LIMITED = 2 (the host encodes limited).
+2
View File
@@ -25,6 +25,8 @@ use jni::objects::JObject;
use jni::sys::jint; use jni::sys::jint;
use jni::JNIEnv; use jni::JNIEnv;
#[cfg(target_os = "android")]
mod adpf;
#[cfg(target_os = "android")] #[cfg(target_os = "android")]
mod audio; mod audio;
#[cfg(target_os = "android")] #[cfg(target_os = "android")]
+23 -11
View File
@@ -72,14 +72,19 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}) })
} }
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD. /// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD
/// Returns 14 doubles /// (unified stats spec, `design/stats-unification.md`). Returns 18 doubles
/// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped, /// `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]` /// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
/// (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — see below), or /// netP50Ms]`
/// `null` when no decode thread is running. Poll ~1 Hz from the UI; each call resets the measurement /// (the two flags are 1.0/0.0; indexes 015 match the previous 16-double layout — 013 the original
/// window. Not android-gated — pure `jni` + connector reads, so it links on the host build too /// 14-double one with the latency pair re-based to the end-to-end capture→decoded headline, 14/15
/// (Kotlin only ever calls it on device). /// the stage p50s tiling it: `host+network` = capture→received, `decode` = received→decoded; 16/17
/// are the Phase-2 split of the `host+network` term from the per-AU 0xCF host timings — `host` =
/// the host's capture→sent, `network` = the remainder — both 0.0 when no timing matched this
/// window, i.e. an old host), or `null` when no decode thread is running. Poll ~1 Hz from the UI; each call
/// resets the measurement window. Not android-gated — pure `jni` + connector reads, so it links on
/// the host build too (Kotlin only ever calls it on device).
#[no_mangle] #[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats( pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
env: JNIEnv, env: JNIEnv,
@@ -98,11 +103,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
let snap = h.stats.drain(); let snap = h.stats.drain();
let mode = h.client.mode(); let mode = h.client.mode();
let color = h.client.color; let color = h.client.color;
let buf: [f64; 14] = [ let buf: [f64; 18] = [
snap.fps, snap.fps,
snap.mbps, snap.mbps,
snap.lat_p50_ms, snap.e2e_p50_ms,
snap.lat_p95_ms, snap.e2e_p95_ms,
if snap.lat_valid { 1.0 } else { 0.0 }, if snap.lat_valid { 1.0 } else { 0.0 },
if snap.skew_corrected { 1.0 } else { 0.0 }, if snap.skew_corrected { 1.0 } else { 0.0 },
mode.width as f64, mode.width as f64,
@@ -117,6 +122,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
color.primaries as f64, color.primaries as f64,
color.transfer as f64, color.transfer as f64,
h.client.chroma_format as f64, h.client.chroma_format as f64,
// Stage p50s tiling the end-to-end headline (appended to keep 013 index-compatible).
snap.hostnet_p50_ms,
snap.decode_p50_ms,
// Phase-2 host/network split of the `host+network` stage (0xCF host timings): 0.0
// when no timing matched this window (old host) — the HUD keeps the combined term.
snap.host_p50_ms,
snap.net_p50_ms,
]; ];
let arr = match env.new_double_array(buf.len() as jsize) { let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a, Ok(a) => a,
+130 -40
View File
@@ -1,8 +1,13 @@
//! Live decode stats for the on-stream HUD (mirrors the Apple client's stats overlay): FPS, //! Live decode stats for the on-stream HUD, following the unified stats spec
//! receive throughput, and capture→client-receipt latency (p50/p95). The decode thread is the sole //! (`design/stats-unification.md`): FPS, receive throughput, and the Android v1 stage split —
//! writer (`note` per access unit); the JNI accessor `nativeVideoStats` drains a snapshot ~1 Hz and //! headline `end-to-end` = capture→decoded (p50/p95) tiled by `host+network` = capture→received
//! resets the window. Sampling is gated on the HUD actually being visible (`set_enabled`, driven by //! and `decode` = received→decoded (stage p50s). When the host emits per-AU 0xCF host timings, the
//! `nativeSetVideoStatsEnabled`) so the hidden steady state costs one relaxed atomic load per frame. //! `host+network` term further splits into `host` + `network` (Phase 2, `note_host_split`); an old
//! host emits none and the combined term stands. The decode thread is the sole writer
//! (`note_received` per access unit at receipt, `note_decoded` per decoder output buffer); the JNI
//! accessor `nativeVideoStats` drains a snapshot ~1 Hz and resets the window. Sampling is gated on
//! the HUD actually being visible (`set_enabled`, driven by `nativeSetVideoStatsEnabled`) so the
//! hidden steady state costs one relaxed atomic load per frame.
//! Pure `std` so it compiles on the host build too (the decode thread is android-only, but //! Pure `std` so it compiles on the host build too (the decode thread is android-only, but
//! `SessionHandle` holds the shared handle unconditionally). //! `SessionHandle` holds the shared handle unconditionally).
@@ -13,9 +18,9 @@ use std::time::Instant;
/// Rolling per-window accumulator. Rates are computed over the actual elapsed wall-time at drain /// Rolling per-window accumulator. Rates are computed over the actual elapsed wall-time at drain
/// (robust to poll jitter), so a poll that lands at 0.9 s or 1.1 s still reports the right FPS. /// (robust to poll jitter), so a poll that lands at 0.9 s or 1.1 s still reports the right FPS.
pub struct VideoStats { pub struct VideoStats {
/// HUD gate: `note` runs on the per-frame decode path, so while the overlay is hidden it (and /// HUD gate: the samplers run on the per-frame decode path, so while the overlay is hidden
/// the caller's latency computation — see `enabled`) early-outs on this flag alone. Off until /// they (and the caller's latency computation — see `enabled`) early-out on this flag alone.
/// Kotlin shows the HUD. /// Off until Kotlin shows the HUD.
enabled: AtomicBool, enabled: AtomicBool,
inner: Mutex<Inner>, inner: Mutex<Inner>,
} }
@@ -24,23 +29,52 @@ struct Inner {
window_start: Instant, window_start: Instant,
frames: u64, frames: u64,
bytes: u64, bytes: u64,
/// capture→client-receipt latency samples for this window, in microseconds. /// `end-to-end` = capture→decoded latency samples for this window, in microseconds
lat_us: Vec<u64>, /// (skew-corrected clock base).
e2e_us: Vec<u64>,
/// `host+network` stage = capture→received samples, in microseconds (skew-corrected).
hostnet_us: Vec<u64>,
/// Phase-2 split of `host+network` (design/stats-unification.md Phase 2), fed only when the
/// host emits per-AU 0xCF timings: `host` = the host's own capture→sent duration, µs.
host_us: Vec<u64>,
/// The matching `network` term, µs: capture→received minus the host's capture→sent
/// (wire + reassembly). Always pushed in lockstep with `host_us`.
net_us: Vec<u64>,
/// `decode` stage = received→decoded samples, in microseconds (client-local, single clock).
decode_us: Vec<u64>,
/// Whether the host answered the clock-skew handshake (latency is cross-machine valid). /// Whether the host answered the clock-skew handshake (latency is cross-machine valid).
skew_corrected: bool, skew_corrected: bool,
} }
/// A drained, computed view of one window. `lat_valid` is false when no in-range latency sample /// A drained, computed view of one window. `lat_valid` is false when no in-range end-to-end sample
/// landed (then p50/p95 are 0 and the HUD hides the latency line, exactly like the Apple client). /// landed (then the latency figures are 0 and the HUD hides the latency lines, exactly like the
/// Apple client).
pub struct Snapshot { pub struct Snapshot {
pub fps: f64, pub fps: f64,
pub mbps: f64, pub mbps: f64,
pub lat_p50_ms: f64, /// Headline `end-to-end` (capture→decoded) percentiles, ms.
pub lat_p95_ms: f64, pub e2e_p50_ms: f64,
pub e2e_p95_ms: f64,
/// Stage p50s (ms): `host+network` (capture→received) and `decode` (received→decoded).
pub hostnet_p50_ms: f64,
pub decode_p50_ms: f64,
/// Phase-2 `host` / `network` split p50s (ms) — 0.0 when no 0xCF timing matched this window
/// (old host / no samples yet), in which case the HUD keeps the combined `host+network` term.
pub host_p50_ms: f64,
pub net_p50_ms: f64,
pub lat_valid: bool, pub lat_valid: bool,
pub skew_corrected: bool, pub skew_corrected: bool,
} }
/// Percentile over a sorted-in-place µs sample vec, in ms. 0.0 when empty.
fn pctl_ms(sorted_us: &[u64], p: f64) -> f64 {
if sorted_us.is_empty() {
return 0.0;
}
let n = sorted_us.len();
sorted_us[((n as f64 * p) as usize).min(n - 1)] as f64 / 1000.0
}
impl VideoStats { impl VideoStats {
pub fn new() -> VideoStats { pub fn new() -> VideoStats {
VideoStats { VideoStats {
@@ -49,14 +83,18 @@ impl VideoStats {
window_start: Instant::now(), window_start: Instant::now(),
frames: 0, frames: 0,
bytes: 0, bytes: 0,
lat_us: Vec::with_capacity(256), e2e_us: Vec::with_capacity(256),
hostnet_us: Vec::with_capacity(256),
host_us: Vec::with_capacity(256),
net_us: Vec::with_capacity(256),
decode_us: Vec::with_capacity(256),
skew_corrected: false, skew_corrected: false,
}), }),
} }
} }
/// Whether the HUD wants samples. The decode thread checks this BEFORE building a latency /// Whether the HUD wants samples. The decode thread checks this BEFORE building a latency
/// sample, so the per-frame wall-clock read is skipped too while hidden. /// sample, so the per-frame wall-clock reads are skipped too while hidden.
// Read only by the android-only decode thread; unreferenced on the host build — expected. // Read only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))] #[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn enabled(&self) -> bool { pub fn enabled(&self) -> bool {
@@ -75,18 +113,23 @@ impl VideoStats {
g.window_start = Instant::now(); g.window_start = Instant::now();
g.frames = 0; g.frames = 0;
g.bytes = 0; g.bytes = 0;
g.lat_us.clear(); g.e2e_us.clear();
g.hostnet_us.clear();
g.host_us.clear();
g.net_us.clear();
g.decode_us.clear();
} }
} }
/// Record one decoded access unit: its wire size and (if in range) its capture→client latency. /// Record one received access unit: its wire size and (if in range) its capture→received
/// `host+network` stage sample. Receipt is the fps/goodput counting point per the spec.
// Driven only by the android-only decode thread; unreferenced on the host build — expected. // Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))] #[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note(&self, bytes: usize, lat_us: Option<u64>, skew_corrected: bool) { pub fn note_received(&self, bytes: usize, hostnet_us: Option<u64>, skew_corrected: bool) {
if !self.enabled.load(Ordering::Relaxed) { if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock (the caller already skipped the clock read) return; // HUD hidden — skip the lock (the caller already skipped the clock read)
} }
// Poison-proof: `note` runs per-frame on the decode thread, which has no catch_unwind — // Poison-proof: this runs per-frame on the decode thread, which has no catch_unwind —
// a panic elsewhere must not turn every later lock into a second panic (the counters // a panic elsewhere must not turn every later lock into a second panic (the counters
// stay consistent regardless). // stay consistent regardless).
let mut g = self let mut g = self
@@ -96,14 +139,56 @@ impl VideoStats {
g.frames += 1; g.frames += 1;
g.bytes += bytes as u64; g.bytes += bytes as u64;
g.skew_corrected = skew_corrected; g.skew_corrected = skew_corrected;
if let Some(l) = lat_us { if let Some(l) = hostnet_us {
g.lat_us.push(l); g.hostnet_us.push(l);
}
}
/// Record one matched host/network split sample (Phase 2): the host's reported capture→sent
/// duration and our capture→received minus it, both µs — one pair per AU whose 0xCF host
/// timing arrived and matched by pts. An old host emits none, leaving the vecs empty and the
/// snapshot p50s at 0 (HUD keeps the combined `host+network` term).
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_host_split(&self, host_us: u64, net_us: u64) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.host_us.push(host_us);
g.net_us.push(net_us);
}
/// Record one decoded output frame: its capture→decoded `end-to-end` sample and its
/// received→decoded `decode` stage sample (either may be absent — e.g. the receipt stamp for
/// this pts predates the HUD being shown).
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_decoded(&self, e2e_us: Option<u64>, decode_us: Option<u64>) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock (the caller already skipped the clock read)
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(l) = e2e_us {
g.e2e_us.push(l);
}
if let Some(l) = decode_us {
g.decode_us.push(l);
} }
} }
/// Compute the window's rates + latency percentiles, then reset for the next window. /// Compute the window's rates + latency percentiles, then reset for the next window.
pub fn drain(&self) -> Snapshot { pub fn drain(&self) -> Snapshot {
// Poison-proof for the same reason as `note` — a poisoned window still drains fine. // Poison-proof for the same reason as `note_received` — a poisoned window still drains
// fine.
let mut g = self let mut g = self
.inner .inner
.lock() .lock()
@@ -111,26 +196,31 @@ impl VideoStats {
let elapsed = g.window_start.elapsed().as_secs_f64().max(1e-3); let elapsed = g.window_start.elapsed().as_secs_f64().max(1e-3);
let fps = g.frames as f64 / elapsed; let fps = g.frames as f64 / elapsed;
let mbps = g.bytes as f64 * 8.0 / 1_000_000.0 / elapsed; let mbps = g.bytes as f64 * 8.0 / 1_000_000.0 / elapsed;
let (p50, p95, valid) = if g.lat_us.is_empty() { g.e2e_us.sort_unstable();
(0.0, 0.0, false) g.hostnet_us.sort_unstable();
} else { g.host_us.sort_unstable();
g.lat_us.sort_unstable(); g.net_us.sort_unstable();
let n = g.lat_us.len(); g.decode_us.sort_unstable();
let at = |p: f64| g.lat_us[((n as f64 * p) as usize).min(n - 1)] as f64 / 1000.0; let snap = Snapshot {
(at(0.50), at(0.95), true) fps,
mbps,
e2e_p50_ms: pctl_ms(&g.e2e_us, 0.50),
e2e_p95_ms: pctl_ms(&g.e2e_us, 0.95),
hostnet_p50_ms: pctl_ms(&g.hostnet_us, 0.50),
decode_p50_ms: pctl_ms(&g.decode_us, 0.50),
host_p50_ms: pctl_ms(&g.host_us, 0.50),
net_p50_ms: pctl_ms(&g.net_us, 0.50),
lat_valid: !g.e2e_us.is_empty(),
skew_corrected: g.skew_corrected,
}; };
let skew = g.skew_corrected;
g.window_start = Instant::now(); g.window_start = Instant::now();
g.frames = 0; g.frames = 0;
g.bytes = 0; g.bytes = 0;
g.lat_us.clear(); g.e2e_us.clear();
Snapshot { g.hostnet_us.clear();
fps, g.host_us.clear();
mbps, g.net_us.clear();
lat_p50_ms: p50, g.decode_us.clear();
lat_p95_ms: p95, snap
lat_valid: valid,
skew_corrected: skew,
}
} }
} }
@@ -326,15 +326,21 @@ struct ContentView: View {
onCaptureChange: { [weak model] captured in onCaptureChange: { [weak model] captured in
model?.mouseCaptured = captured model?.mouseCaptured = captured
}, },
onFrame: { [meter = model.meter, latency = model.latency, offset = conn.clockOffsetNs] au in onFrame: { [meter = model.meter, latency = model.latency,
split = model.latencySplit, offset = conn.clockOffsetNs] au in
meter.note(byteCount: au.data.count) meter.note(byteCount: au.data.count)
latency.record(ptsNs: au.ptsNs, offsetNs: offset) latency.record(ptsNs: au.ptsNs, offsetNs: offset)
// The same receipt, keyed by pts, awaiting its 0xCF host timing (the
// host/network split drained by the 1 s stats tick).
split.recordReceipt(
ptsNs: au.ptsNs, receivedNs: au.receivedNs, offsetNs: offset)
}, },
onSessionEnd: { [weak model] in onSessionEnd: { [weak model] in
Task { @MainActor in model?.sessionEnded() } Task { @MainActor in model?.sessionEnded() }
}, },
presentMeter: model.presentLatency, endToEndMeter: model.endToEnd,
presentTailMeter: model.presentTail decodeMeter: model.decodeStage,
displayMeter: model.displayStage
) )
.overlay(alignment: placement.alignment) { .overlay(alignment: placement.alignment) {
if captureEnabled && hudEnabled { if captureEnabled && hudEnabled {
@@ -170,7 +170,10 @@ private struct ShotHUD: View {
Text("5120×1440@240 240 fps 812.4 Mb/s") Text("5120×1440@240 240 fps 812.4 Mb/s")
.font(.system(.caption, design: .monospaced)) .font(.system(.caption, design: .monospaced))
} }
Text("capture→client 1.3/2.1 ms p50/p95") Text("end-to-end 2.9 ms p50 · 3.8 p95 · capture→on-glass")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
Text("= host+network 1.3 + decode 0.7 + display 0.9")
.font(.system(.caption2, design: .monospaced)) .font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
#if os(macOS) #if os(macOS)
@@ -59,36 +59,62 @@ final class SessionModel: ObservableObject {
@Published var fps = 0 @Published var fps = 0
@Published var mbps = 0.0 @Published var mbps = 0.0
@Published var totalFrames = 0 @Published var totalFrames = 0
/// Captureclient-receipt latency (ms), skew-corrected across machines via the connect-time /// The unified latency stages (design/stats-unification.md), ms per 1 s window. `host+network`
/// clock offset p50/p95 for the HUD. `latencyValid` is false until the first sample drains /// = capturereceived, skew-corrected across machines via the connect-time clock offset: the
/// (and whenever no host frames arrived in the last interval). `latencySkewCorrected` = the host /// stage-2 HUD shows its p50 in the equation line; the stage-1 fallback shows p50/p95 as its
/// `capturereceived` headline. `hostNetworkValid` is false until the first sample drains (and
/// whenever no host frames arrived in the last interval). `hostNetworkSkewCorrected` = the host
/// answered the skew handshake (the number is cross-machine valid, not just same-host). /// answered the skew handshake (the number is cross-machine valid, not just same-host).
@Published var latencyP50Ms = 0.0 @Published var hostNetworkP50Ms = 0.0
@Published var latencyP95Ms = 0.0 @Published var hostNetworkP95Ms = 0.0
@Published var latencyValid = false @Published var hostNetworkValid = false
@Published var latencySkewCorrected = false @Published var hostNetworkSkewCorrected = false
/// Capturepresent (glass-to-glass, modulo the host rendercapture term) only the stage-2 /// Phase 2 of the same stage: `host+network` split into its two terms via the host's per-AU
/// presenter can stamp this (it owns decode + a CAMetalLayer/display-link present). Stays /// 0xCF timing reports (host = capturefully-sent as the host measured it, network = the
/// invalid under stage-1, where the layer presents internally with no per-frame callback. /// remainder), matched to receipts by pts in `latencySplit`. `splitValid` is false whenever
@Published var presentLatencyP50Ms = 0.0 /// no timing matched in the window an old host that never emits the plane, or heavy 0xCF
@Published var presentLatencyP95Ms = 0.0 /// loss and the HUD then falls back to the combined `host+network` term.
@Published var presentLatencyValid = false @Published var hostP50Ms = 0.0
@Published var presentLatencySkewCorrected = false @Published var networkP50Ms = 0.0
/// Decode-completionpresent (the "present tail": ring wait + render + vsync) the term the @Published var splitValid = false
/// stage-2 presenter exists to shorten. Both instants are client-side, so no skew applies. /// End-to-end = captureon-glass, measured directly per frame (never summed from the stages)
@Published var presentTailP50Ms = 0.0 /// the HUD headline. Only the stage-2 presenter can stamp it (it owns decode + a
@Published var presentTailP95Ms = 0.0 /// CAMetalLayer/display-link present); stays invalid under stage-1, where the layer presents
@Published var presentTailValid = false /// internally with no per-frame callback.
@Published var endToEndP50Ms = 0.0
@Published var endToEndP95Ms = 0.0
@Published var endToEndValid = false
@Published var endToEndSkewCorrected = false
/// The client-local stage terms of the HUD's equation line (single clock, no skew; p50 only):
/// decode = receiveddecoded, display = decodedon-glass (ring wait + render + vsync the
/// term the stage-2 presenter exists to shorten).
@Published var decodeP50Ms = 0.0
@Published var decodeValid = false
@Published var displayP50Ms = 0.0
@Published var displayValid = false
/// Unrecoverable network frame drops in the last window (FEC couldn't rebuild them) and their
/// share of frames offered, `lost/(received+lost)`. The HUD hides the line while zero.
@Published var lostFrames = 0
@Published var lostPct = 0.0
/// Mirrors StreamView's capture state (it owns the input capture; this drives the /// Mirrors StreamView's capture state (it owns the input capture; this drives the
/// HUD's "click to capture" / " releases" hint). /// HUD's "click to capture" / " releases" hint).
@Published var mouseCaptured = false @Published var mouseCaptured = false
let meter = FrameMeter() let meter = FrameMeter()
/// Capturereceived (the host+network stage), fed per AU at receipt by the stream view's
/// onFrame under both presenters.
let latency = LatencyMeter() let latency = LatencyMeter()
/// Fed by the stage-2 presenter's display link (capturepresent). Passed to StreamView. /// The host/network split of that same stage: onFrame also records (pts, interval) receipts
let presentLatency = LatencyMeter() /// here, and the 1 s stats tick drains the connection's 0xCF host timings into it under
/// Fed by the same present stamp (decode-completionpresent). Passed to StreamView. /// both presenters (the receipt path is presenter-independent).
let presentTail = LatencyMeter() let latencySplit = HostNetworkSplitter()
/// The stage-2 meters, passed to StreamView: end-to-end (captureon-glass, stamped at
/// present), decode (receiveddecoded), display (decodedon-glass).
let endToEnd = LatencyMeter()
let decodeStage = LatencyMeter()
let displayStage = LatencyMeter()
/// Cumulative reassembler-drop counter at the last stats drain (per-window `lost` delta).
private var lastFramesDropped: UInt64 = 0
private var statsTimer: Timer? private var statsTimer: Timer?
private var audio: SessionAudio? private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture? private var gamepadCapture: GamepadCapture?
@@ -281,7 +307,13 @@ final class SessionModel: ObservableObject {
phase = .idle phase = .idle
fps = 0 fps = 0
mbps = 0 mbps = 0
latencyValid = false hostNetworkValid = false
splitValid = false
endToEndValid = false
decodeValid = false
displayValid = false
lostFrames = 0
lostPct = 0
mouseCaptured = false mouseCaptured = false
} }
@@ -306,6 +338,7 @@ final class SessionModel: ObservableObject {
audio.start( audio.start(
speakerUID: defaults.string(forKey: DefaultsKey.speakerUID) ?? "", speakerUID: defaults.string(forKey: DefaultsKey.speakerUID) ?? "",
micUID: defaults.string(forKey: DefaultsKey.micUID) ?? "", micUID: defaults.string(forKey: DefaultsKey.micUID) ?? "",
micChannel: defaults.integer(forKey: DefaultsKey.micChannel),
micEnabled: defaults.object(forKey: DefaultsKey.micEnabled) as? Bool ?? true) micEnabled: defaults.object(forKey: DefaultsKey.micEnabled) as? Bool ?? true)
self.audio = audio self.audio = audio
// Gamepads: forward GamepadManager's active controller as pad 0 and render the // Gamepads: forward GamepadManager's active controller as pad 0 and render the
@@ -321,6 +354,8 @@ final class SessionModel: ObservableObject {
} }
private func startStatsTimer() { private func startStatsTimer() {
lastFramesDropped = 0 // a fresh connection's cumulative drop counter starts at 0
latencySplit.reset() // no stale receipts/samples from a previous session
let timer = Timer(timeInterval: 1.0, repeats: true) { [weak self] _ in let timer = Timer(timeInterval: 1.0, repeats: true) { [weak self] _ in
guard let self else { return } guard let self else { return }
Task { @MainActor in Task { @MainActor in
@@ -328,28 +363,60 @@ final class SessionModel: ObservableObject {
self.fps = frames self.fps = frames
self.mbps = Double(bytes) * 8 / 1_000_000 self.mbps = Double(bytes) * 8 / 1_000_000
self.totalFrames = total self.totalFrames = total
// Per-window `lost` = the delta of the connector's cumulative reassembler-drop
// counter (0 after close treat a rewind as no loss rather than underflowing).
let dropped = self.connection?.framesDropped() ?? 0
let lost = dropped >= self.lastFramesDropped
? Int(dropped - self.lastFramesDropped) : 0
self.lastFramesDropped = dropped
self.lostFrames = lost
self.lostPct = lost > 0 ? Double(lost) / Double(frames + lost) * 100 : 0
if let lat = self.latency.drain() { if let lat = self.latency.drain() {
self.latencyP50Ms = lat.p50Ms self.hostNetworkP50Ms = lat.p50Ms
self.latencyP95Ms = lat.p95Ms self.hostNetworkP95Ms = lat.p95Ms
self.latencySkewCorrected = lat.skewCorrected self.hostNetworkSkewCorrected = lat.skewCorrected
self.latencyValid = true self.hostNetworkValid = true
} else { } else {
self.latencyValid = false self.hostNetworkValid = false
} }
if let p = self.presentLatency.drain() { // Phase 2: drain the window's per-AU host timings (0xCF) into the splitter
self.presentLatencyP50Ms = p.p50Ms // non-blocking, bounded (a 240 fps window is ~240 reports; the cap only guards
self.presentLatencyP95Ms = p.p95Ms // a pathological burst). `try?` flattens (SE-0230); a throw (.closed during
self.presentLatencySkewCorrected = p.skewCorrected // teardown) just ends the drain. An old host never emits any splitValid stays
self.presentLatencyValid = true // false and the HUD keeps the combined host+network term.
} else { if let conn = self.connection {
self.presentLatencyValid = false var burst = 0
while burst < 1024, let t = try? conn.nextHostTiming(timeoutMs: 0) {
self.latencySplit.noteHostTiming(ptsNs: t.ptsNs, hostUs: t.hostUs)
burst += 1
} }
if let t = self.presentTail.drain() { }
self.presentTailP50Ms = t.p50Ms if let s = self.latencySplit.drain() {
self.presentTailP95Ms = t.p95Ms self.hostP50Ms = s.hostP50Ms
self.presentTailValid = true self.networkP50Ms = s.networkP50Ms
self.splitValid = true
} else { } else {
self.presentTailValid = false self.splitValid = false
}
if let e = self.endToEnd.drain() {
self.endToEndP50Ms = e.p50Ms
self.endToEndP95Ms = e.p95Ms
self.endToEndSkewCorrected = e.skewCorrected
self.endToEndValid = true
} else {
self.endToEndValid = false
}
if let d = self.decodeStage.drain() {
self.decodeP50Ms = d.p50Ms
self.decodeValid = true
} else {
self.decodeValid = false
}
if let d = self.displayStage.drain() {
self.displayP50Ms = d.p50Ms
self.displayValid = true
} else {
self.displayValid = false
} }
} }
} }
@@ -1,5 +1,7 @@
// The streaming overlay HUD: mode + fps/throughput, the captureclient (and, under the stage-2 // The streaming overlay HUD: mode + fps/throughput, the unified latency lines
// presenter, capturepresent) latency lines, the platform input hint, and disconnect. // (design/stats-unification.md end-to-end headline + the stage equation under stage-2, the
// capturereceived headline under the stage-1 fallback), the loss counter, the platform input
// hint, and disconnect.
import PunktfunkKit import PunktfunkKit
import SwiftUI import SwiftUI
@@ -18,24 +20,46 @@ struct StreamHUDView: View {
Text("\(connection.width)×\(connection.height)@\(connection.refreshHz) \(model.fps) fps \(model.mbps, specifier: "%.1f") Mb/s") Text("\(connection.width)×\(connection.height)@\(connection.refreshHz) \(model.fps) fps \(model.mbps, specifier: "%.1f") Mb/s")
.font(.system(.caption, design: .monospaced)) .font(.system(.caption, design: .monospaced))
} }
if model.latencyValid { if model.endToEndValid {
// Captureclient-receipt (skew-corrected); excludes the layer's decode+present // Stage-2: the end-to-end headline (captureon-glass, measured directly, skew-
// see LatencyMeter. "(same-host)" when the host didn't answer the skew handshake. // corrected) "(same-host clock)" when the host didn't answer the skew handshake.
Text("capture→client \(model.latencyP50Ms, specifier: "%.1f")/\(model.latencyP95Ms, specifier: "%.1f") ms p50/p95\(model.latencySkewCorrected ? "" : " (same-host)")") Text("end-to-end \(model.endToEndP50Ms, specifier: "%.1f") ms p50 · \(model.endToEndP95Ms, specifier: "%.1f") p95 · capture→on-glass\(model.endToEndSkewCorrected ? "" : " (same-host clock)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
// The equation: the stages tiling the headline interval (per-window p50s
// they only approximately sum to the directly-measured total). With a host
// that reports per-AU timings (0xCF) the first term splits into host + network
// (phase 2); an old host keeps the combined term.
if model.hostNetworkValid && model.decodeValid && model.displayValid {
if model.splitValid {
Text("= host \(model.hostP50Ms, specifier: "%.1f") + network \(model.networkP50Ms, specifier: "%.1f") + decode \(model.decodeP50Ms, specifier: "%.1f") + display \(model.displayP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
} else {
Text("= host+network \(model.hostNetworkP50Ms, specifier: "%.1f") + decode \(model.decodeP50Ms, specifier: "%.1f") + display \(model.displayP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced)) .font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
if model.presentLatencyValid { }
// Capturepresent (glass-to-glass, modulo host rendercapture) stage-2 presenter } else if model.hostNetworkValid {
// only; stage-1's layer presents internally with no per-frame stamp. // Stage-1 fallback presenter: the layer decodes + presents internally with no
Text("capture→present \(model.presentLatencyP50Ms, specifier: "%.1f")/\(model.presentLatencyP95Ms, specifier: "%.1f") ms p50/p95\(model.presentLatencySkewCorrected ? "" : " (same-host)")") // per-frame stamp, so the honest headline ends at receipt. The host/network
// split still applies there (receipt is presenter-independent) it becomes the
// only equation line; without it, host+network IS the whole measured interval.
Text("capture→received \(model.hostNetworkP50Ms, specifier: "%.1f") ms p50 · \(model.hostNetworkP95Ms, specifier: "%.1f") p95\(model.hostNetworkSkewCorrected ? "" : " (same-host clock)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
if model.splitValid {
Text("= host \(model.hostP50Ms, specifier: "%.1f") + network \(model.networkP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced)) .font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
if model.presentTailValid { }
// Decodepresent (the client-local "present tail": ring wait + render + vsync) if model.lostFrames > 0 {
// the term the stage-2 presenter shortens; no skew applies (one clock). // Unrecoverable network drops this window; hidden while the link is clean.
Text("decode→present \(model.presentTailP50Ms, specifier: "%.1f")/\(model.presentTailP95Ms, specifier: "%.1f") ms p50/p95") // String(format:) rather than specifier interpolation: the literal % would
// otherwise land in the LocalizedStringKey's format string as a bogus conversion.
Text(String(format: "lost %d (%.1f%%)", model.lostFrames, model.lostPct))
.font(.system(.caption2, design: .monospaced)) .font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
@@ -7,13 +7,49 @@ import SwiftUI
extension SettingsView { extension SettingsView {
// MARK: - Sections (shared) // MARK: - Sections (shared)
// NOTE: the Section content is deliberately split into the small named builders below as one
// inline expression the iOS branch (wheel + 3-way refresh + bitrate rows) blew Swift's
// type-checker budget ("unable to type-check this expression in reasonable time"), which
// failed exactly one slice: the iOS archive (macOS/tvOS never compile that branch).
@ViewBuilder var streamModeSection: some View { @ViewBuilder var streamModeSection: some View {
Section { Section {
#if os(iOS) #if os(iOS)
// Touch-first: a rotating wheel of common resolutions (this device's own mode first) and iosResolutionWheel
// a segmented refresh-rate control the same family as the Clock/Timer pickers. The host iosRefreshRows
// renders a virtual output at exactly the chosen mode, so these are real pixel sizes. The Button("Use this display's mode") { fillFromMainScreen() }
// last wheel row, "Custom", reveals width/height/refresh fields for an arbitrary mode. #elseif os(macOS)
HStack {
TextField("Resolution", value: $width, format: .number.grouping(.never))
Text("×")
TextField("", value: $height, format: .number.grouping(.never))
.labelsHidden()
}
TextField("Refresh rate (Hz)", value: $hz, format: .number.grouping(.never))
LabeledContent("") {
Button("Use this display's mode") { fillFromMainScreen() }
}
#endif
#if !os(tvOS)
bitrateRows
#endif
} header: {
Text("Stream mode")
} footer: {
Text("The host creates a virtual output at exactly this mode — "
+ "native resolution, no scaling. \(Self.bitrateFooter)")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
// MARK: - Stream mode (iOS wheel)
/// Touch-first: a rotating wheel of common resolutions (this device's own mode first) the
/// same family as the Clock/Timer pickers. The host renders a virtual output at exactly the
/// chosen mode, so these are real pixel sizes. The last wheel row, "Custom", reveals
/// width/height/refresh fields for an arbitrary mode (see `iosRefreshRows`).
@ViewBuilder private var iosResolutionWheel: some View {
VStack(alignment: .leading, spacing: 4) { VStack(alignment: .leading, spacing: 4) {
Text("Resolution") Text("Resolution")
.font(.geist(15, relativeTo: .subheadline)) .font(.geist(15, relativeTo: .subheadline))
@@ -27,6 +63,10 @@ extension SettingsView {
.pickerStyle(.wheel) .pickerStyle(.wheel)
.frame(maxHeight: 140) .frame(maxHeight: 140)
} }
}
/// Custom W×H(+Hz) fields, a segmented refresh picker, or a static single-rate row.
@ViewBuilder private var iosRefreshRows: some View {
if isCustomResolution { if isCustomResolution {
// Arbitrary entry: type the exact width × height (and refresh) the host should drive. // Arbitrary entry: type the exact width × height (and refresh) the host should drive.
HStack { HStack {
@@ -64,50 +104,7 @@ extension SettingsView {
Text("\(hz) Hz").foregroundStyle(.secondary) Text("\(hz) Hz").foregroundStyle(.secondary)
} }
} }
Button("Use this display's mode") { fillFromMainScreen() }
#elseif os(macOS)
HStack {
TextField("Resolution", value: $width, format: .number.grouping(.never))
Text("×")
TextField("", value: $height, format: .number.grouping(.never))
.labelsHidden()
} }
TextField("Refresh rate (Hz)", value: $hz, format: .number.grouping(.never))
LabeledContent("") {
Button("Use this display's mode") { fillFromMainScreen() }
}
#endif
#if !os(tvOS)
Toggle("Automatic bitrate", isOn: automaticBitrate)
if bitrateKbps != 0 {
HStack(spacing: 12) {
Slider(value: bitrateSlider, in: 0...1) {
Text("Bitrate")
}
Text(SpeedTestSheet.mbpsLabel(kbps: bitrateKbps))
.monospacedDigit()
.foregroundStyle(.secondary)
.frame(minWidth: 76, alignment: .trailing)
}
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
}
}
#endif
} header: {
Text("Stream mode")
} footer: {
Text("The host creates a virtual output at exactly this mode — "
+ "native resolution, no scaling. \(Self.bitrateFooter)")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
// MARK: - Stream mode (iOS wheel)
/// Sentinel wheel tag for the "Custom" row. Real tags are "WxH" (digits + "x"), so this can't /// Sentinel wheel tag for the "Custom" row. Real tags are "WxH" (digits + "x"), so this can't
/// collide with a resolution. /// collide with a resolution.
@@ -156,6 +153,29 @@ extension SettingsView {
} }
#endif #endif
#if !os(tvOS)
/// The automatic-bitrate toggle + manual slider (and the >1 Gbps warning) rows.
@ViewBuilder private var bitrateRows: some View {
Toggle("Automatic bitrate", isOn: automaticBitrate)
if bitrateKbps != 0 {
HStack(spacing: 12) {
Slider(value: bitrateSlider, in: 0...1) {
Text("Bitrate")
}
Text(SpeedTestSheet.mbpsLabel(kbps: bitrateKbps))
.monospacedDigit()
.foregroundStyle(.secondary)
.frame(minWidth: 76, alignment: .trailing)
}
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
}
}
}
#endif
@ViewBuilder var audioSection: some View { @ViewBuilder var audioSection: some View {
Section { Section {
Picker("Audio channels", selection: $audioChannels) { Picker("Audio channels", selection: $audioChannels) {
@@ -188,6 +208,17 @@ extension SettingsView {
} }
} }
.disabled(!micEnabled) .disabled(!micEnabled)
// Multi-channel interfaces only: the mic sits on ONE discrete input, so let the user
// pick it. Auto sums every channel (a lone hot mic still passes at full level).
if micChannelCount > 1 {
Picker("Microphone channel", selection: $micChannel) {
Text("Auto (all channels)").tag(0)
ForEach(1...micChannelCount, id: \.self) { ch in
Text("Channel \(ch)").tag(ch)
}
}
.disabled(!micEnabled)
}
#endif #endif
} header: { } header: {
Text("Audio") Text("Audio")
@@ -204,35 +235,42 @@ extension SettingsView {
/// Touch-input model (iPhone + iPad) plus the iPad-only pointer-capture toggle: lock the /// Touch-input model (iPhone + iPad) plus the iPad-only pointer-capture toggle: lock the
/// mouse/trackpad for relative movement (games) vs forward an absolute cursor position. /// mouse/trackpad for relative movement (games) vs forward an absolute cursor position.
@ViewBuilder var pointerSection: some View { @ViewBuilder var pointerSection: some View {
let isPad = UIDevice.current.userInterfaceIdiom == .pad
Section { Section {
Picker("Touch input", selection: $touchMode) { Picker("Touch input", selection: $touchMode) {
Text("Trackpad").tag(TouchInputMode.trackpad.rawValue) Text("Trackpad").tag(TouchInputMode.trackpad.rawValue)
Text("Direct pointer").tag(TouchInputMode.pointer.rawValue) Text("Direct pointer").tag(TouchInputMode.pointer.rawValue)
Text("Touch passthrough").tag(TouchInputMode.touch.rawValue) Text("Touch passthrough").tag(TouchInputMode.touch.rawValue)
} }
if isPad { if UIDevice.current.userInterfaceIdiom == .pad {
Toggle("Capture pointer for games", isOn: $pointerCapture) Toggle("Capture pointer for games", isOn: $pointerCapture)
} }
} header: { } header: {
Text("Touch & pointer") Text("Touch & pointer")
} footer: { } footer: {
Text("Trackpad: your finger nudges the host cursor like a laptop touchpad — tap to " Text(pointerFooterText)
+ "click, two-finger tap for a right click, two-finger drag to scroll, "
+ "tap-then-drag to hold the button, three-finger tap for the stats overlay. "
+ "Direct pointer: the cursor jumps to your finger. Touch passthrough: real "
+ "multi-touch reaches the host, for apps that understand touch. Applies from "
+ "the next touch."
+ (isPad
? " Pointer capture locks a hardware mouse/trackpad for relative movement "
+ "(mouse-look); off keeps the pointer free and sends absolute positions. "
+ "The lock needs the stream full-screen and frontmost, and falls back "
+ "automatically (Stage Manager, Slide Over)."
: ""))
.font(.geist(12, relativeTo: .caption)) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
/// Footer copy for `pointerSection`, built in plain `+=` statements. Deliberately NOT one big
/// `+` chain (with a ternary) inside the ViewBuilder that single expression blew Swift's
/// type-checker budget and was what actually broke the iOS archive.
private var pointerFooterText: String {
var text = "Trackpad: your finger nudges the host cursor like a laptop touchpad — tap to "
text += "click, two-finger tap for a right click, two-finger drag to scroll, "
text += "tap-then-drag to hold the button, three-finger tap for the stats overlay. "
text += "Direct pointer: the cursor jumps to your finger. Touch passthrough: real "
text += "multi-touch reaches the host, for apps that understand touch. Applies from "
text += "the next touch."
if UIDevice.current.userInterfaceIdiom == .pad {
text += " Pointer capture locks a hardware mouse/trackpad for relative movement "
text += "(mouse-look); off keeps the pointer free and sends absolute positions. "
text += "The lock needs the stream full-screen and frontmost, and falls back "
text += "automatically (Stage Manager, Slide Over)."
}
return text
}
#endif #endif
@ViewBuilder var compositorSection: some View { @ViewBuilder var compositorSection: some View {
@@ -283,10 +321,11 @@ extension SettingsView {
Text("Video presenter · debug") Text("Video presenter · debug")
} footer: { } footer: {
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display " Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD and " + "link — it gives the HUD the end-to-end (capture→on-glass) headline with the "
+ "self-recovers from decode stalls. Stage 1 feeds compressed video straight to the " + "host+network/decode/display stage equation and self-recovers from decode "
+ "system display layer; it freezes on a lost HEVC reference frame, so it's a debug " + "stalls. Stage 1 feeds compressed video straight to the system display layer; "
+ "fallback only. Applies from the next session.") + "it freezes on a lost HEVC reference frame, so it's a debug fallback only. "
+ "Applies from the next session.")
.font(.geist(12, relativeTo: .caption)) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
@@ -61,8 +61,12 @@ struct SettingsView: View {
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.speakerUID) var speakerUID = "" @AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = "" @AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0
@State var outputDevices: [AudioDevice] = [] @State var outputDevices: [AudioDevice] = []
@State var inputDevices: [AudioDevice] = [] @State var inputDevices: [AudioDevice] = []
// Input channels of the selected mic drives the "Microphone channel" picker, which only
// appears for a multi-channel interface (>1). 0 until the Audio tab loads it.
@State var micChannelCount = 0
#endif #endif
#if os(iOS) #if os(iOS)
@@ -115,6 +119,12 @@ struct SettingsView: View {
.onAppear { .onAppear {
outputDevices = AudioDevices.outputs() outputDevices = AudioDevices.outputs()
inputDevices = AudioDevices.inputs() inputDevices = AudioDevices.inputs()
micChannelCount = AudioDevices.inputChannelCount(forUID: micUID)
}
.onChange(of: micUID) { _, newUID in
// A different mic different channel count; drop a now-out-of-range pin to Auto.
micChannelCount = AudioDevices.inputChannelCount(forUID: newUID)
if micChannel > micChannelCount { micChannel = 0 }
} }
.tabItem { Label("Audio", systemImage: "speaker.wave.2") } .tabItem { Label("Audio", systemImage: "speaker.wave.2") }
@@ -33,6 +33,49 @@ public enum AudioDevices {
} }
} }
/// Input channel count of the mic the picker would use the device with this UID, or the
/// system default input when `uid` is empty. 0 when it can't be resolved. Drives the
/// "Microphone channel" picker (only shown for multi-channel interfaces).
public static func inputChannelCount(forUID uid: String) -> Int {
let id = uid.isEmpty ? defaultInputDevice() : deviceID(forUID: uid)
guard let id else { return 0 }
return channelCount(id, scope: kAudioObjectPropertyScopeInput)
}
private static func defaultInputDevice() -> AudioDeviceID? {
var address = AudioObjectPropertyAddress(
mSelector: kAudioHardwarePropertyDefaultInputDevice,
mScope: kAudioObjectPropertyScopeGlobal,
mElement: kAudioObjectPropertyElementMain)
var dev = AudioDeviceID(0)
var size = UInt32(MemoryLayout<AudioDeviceID>.size)
guard AudioObjectGetPropertyData(
AudioObjectID(kAudioObjectSystemObject), &address, 0, nil, &size, &dev) == noErr,
dev != 0
else { return nil }
return dev
}
/// Sum of channels across the device's streams in `scope` (its total input/output channels).
private static func channelCount(
_ id: AudioDeviceID, scope: AudioObjectPropertyScope
) -> Int {
var address = AudioObjectPropertyAddress(
mSelector: kAudioDevicePropertyStreamConfiguration,
mScope: scope,
mElement: kAudioObjectPropertyElementMain)
var size: UInt32 = 0
guard AudioObjectGetPropertyDataSize(id, &address, 0, nil, &size) == noErr, size > 0
else { return 0 }
let raw = UnsafeMutableRawPointer.allocate(
byteCount: Int(size), alignment: MemoryLayout<AudioBufferList>.alignment)
defer { raw.deallocate() }
guard AudioObjectGetPropertyData(id, &address, 0, nil, &size, raw) == noErr else { return 0 }
let abl = UnsafeMutableAudioBufferListPointer(
raw.assumingMemoryBound(to: AudioBufferList.self))
return abl.reduce(0) { $0 + Int($1.mNumberChannels) }
}
private static func all() -> [AudioDeviceID] { private static func all() -> [AudioDeviceID] {
var address = AudioObjectPropertyAddress( var address = AudioObjectPropertyAddress(
mSelector: kAudioHardwarePropertyDevices, mSelector: kAudioHardwarePropertyDevices,
@@ -62,7 +105,8 @@ public enum AudioDevices {
return AudioObjectGetPropertyDataSize(id, &address, 0, nil, &size) == noErr && size > 0 return AudioObjectGetPropertyDataSize(id, &address, 0, nil, &size) == noErr && size > 0
} }
private static func describe(_ id: AudioDeviceID) -> AudioDevice? { /// UID + human name for a live AudioDeviceID (nil if either property is unreadable).
static func describe(_ id: AudioDeviceID) -> AudioDevice? {
guard let uid = stringProperty(id, kAudioDevicePropertyDeviceUID), guard let uid = stringProperty(id, kAudioDevicePropertyDeviceUID),
let name = stringProperty(id, kAudioObjectPropertyName) let name = stringProperty(id, kAudioObjectPropertyName)
else { return nil } else { return nil }
@@ -5,9 +5,10 @@
// AVAudioSourceNode pulls from the ring (silence on underrun with re-priming, so a // AVAudioSourceNode pulls from the ring (silence on underrun with re-priming, so a
// network gap costs one dip, not permanent crackle). // network gap costs one dip, not permanent crackle).
// //
// mic host: a second AVAudioEngine taps the input device, resamples to 48 kHz // mic host: a second AVAudioEngine taps the input device, folds it to one mono bus (the
// stereo, slices 20 ms chunks, Opus-encodes, and sendMic()s each packet the host // chosen channel of a multi-channel interface, or a sum of all channels), resamples to 48 kHz
// feeds them into a virtual PipeWire source. // stereo, slices 20 ms chunks, Opus-encodes, and sendMic()s each packet the host feeds them
// into a virtual PipeWire source.
// //
// Devices are chosen by UID ("" = system default: the engine is then never pinned to a // Devices are chosen by UID ("" = system default: the engine is then never pinned to a
// concrete device and follows default-device changes). Two engines, not one a single // concrete device and follows default-device changes). Two engines, not one a single
@@ -68,10 +69,11 @@ public final class SessionAudio {
/// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on /// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not /// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending. /// on return. The mic may start later still if the permission prompt is pending.
public func start(speakerUID: String, micUID: String, micEnabled: Bool) { public func start(speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool) {
#if os(macOS) #if os(macOS)
// No AVAudioSession on macOS start the engines directly (caller's thread, as before). // No AVAudioSession on macOS start the engines directly (caller's thread, as before).
startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled) startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel, micEnabled: micEnabled)
#else #else
// Configure + activate the session OFF the main thread (it blocks on the audio server), // Configure + activate the session OFF the main thread (it blocks on the audio server),
// then start the engines back on the main thread once it's active engine routing/format // then start the engines back on the main thread once it's active engine routing/format
@@ -81,7 +83,9 @@ public final class SessionAudio {
self.activateAudioSession(micEnabled: micEnabled) self.activateAudioSession(micEnabled: micEnabled)
DispatchQueue.main.async { [weak self] in DispatchQueue.main.async { [weak self] in
guard let self, !self.flag.isStopped else { return } guard let self, !self.flag.isStopped else { return }
self.startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled) self.startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
micEnabled: micEnabled)
} }
} }
#endif #endif
@@ -115,7 +119,9 @@ public final class SessionAudio {
/// Build + start the playback engine (and the mic uplink when enabled + authorized). Main /// Build + start the playback engine (and the mic uplink when enabled + authorized). Main
/// thread (engine setup); on iOS/tvOS the session is already active by the time this runs. /// thread (engine setup); on iOS/tvOS the session is already active by the time this runs.
private func startEngines(speakerUID: String, micUID: String, micEnabled: Bool) { private func startEngines(
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool
) {
startPlayback(speakerUID: speakerUID) startPlayback(speakerUID: speakerUID)
#if os(tvOS) #if os(tvOS)
// No app-accessible microphone input on tvOS playback only. // No app-accessible microphone input on tvOS playback only.
@@ -123,12 +129,12 @@ public final class SessionAudio {
guard micEnabled else { return } guard micEnabled else { return }
switch AVCaptureDevice.authorizationStatus(for: .audio) { switch AVCaptureDevice.authorizationStatus(for: .audio) {
case .authorized: case .authorized:
startCapture(micUID: micUID) startCapture(micUID: micUID, micChannel: micChannel)
case .notDetermined: case .notDetermined:
AVCaptureDevice.requestAccess(for: .audio) { [weak self] granted in AVCaptureDevice.requestAccess(for: .audio) { [weak self] granted in
DispatchQueue.main.async { DispatchQueue.main.async {
guard let self, granted, !self.flag.isStopped else { return } guard let self, granted, !self.flag.isStopped else { return }
self.startCapture(micUID: micUID) self.startCapture(micUID: micUID, micChannel: micChannel)
} }
} }
default: default:
@@ -280,7 +286,7 @@ public final class SessionAudio {
// MARK: - Mic (mic host) // MARK: - Mic (mic host)
#if !os(tvOS) #if !os(tvOS)
private func startCapture(micUID: String) { private func startCapture(micUID: String, micChannel: Int) {
let engine = AVAudioEngine() let engine = AVAudioEngine()
let input = engine.inputNode let input = engine.inputNode
#if os(macOS) #if os(macOS)
@@ -300,8 +306,63 @@ public final class SessionAudio {
log.error("no usable input device — mic uplink disabled") log.error("no usable input device — mic uplink disabled")
return return
} }
guard let encoder = try? OpusEncoder(),
let resampler = AVAudioConverter(from: inFormat, to: encoder.pcmFormat), // Multi-channel-interface handling. A pro interface exposes N discrete inputs with the mic
// on ONE of them, but AVAudioConverter's Nstereo downmix takes channels 0/1 dead
// silence when the mic sits higher up (the classic "host receives zeros"). So we fold the
// input to a single mono bus OURSELVES and resample that. micChannel: 0 = Auto (sum every
// channel a lone hot mic passes at full level), n1 pins 1-based input channel n.
let inChannels = Int(inFormat.channelCount)
let pinnedChannel: Int? = {
guard micChannel >= 1 else { return nil }
let idx = micChannel - 1
guard idx < inChannels else {
log.warning(
"mic channel \(micChannel) out of range (device has \(inChannels)) — mixing all")
return nil
}
return idx
}()
let channelPlan = pinnedChannel.map { "channel \($0 + 1)/\(inChannels)" }
?? (inChannels > 1 ? "mix \(inChannels)ch→mono" : "mono")
// Name the device we're ACTUALLY recording from + its format + how we fold it, once per
// session. This single line localizes the whole class of "host receives silence" failures
// that otherwise need a host-side tone injection to pin down: a UID that silently fell back
// to the default, the wrong device being live, or the wrong channel picked.
#if os(macOS)
if let unit = input.audioUnit, let live = Self.currentDevice(of: unit),
let dev = AudioDevices.describe(live) {
if !micUID.isEmpty, dev.uid != micUID {
log.warning("""
mic selection not honored — requested \(micUID) but capturing from \
\(dev.name) [\(dev.uid)]; the device's UID likely changed (replug) — \
reselect it in Settings
""")
}
log.info("""
mic capture: \(dev.name) [\(dev.uid)] — \(Int(inFormat.sampleRate)) Hz, \
\(inChannels) ch, \(channelPlan)
""")
} else {
log.info("""
mic capture: <device unavailable> — \(Int(inFormat.sampleRate)) Hz, \
\(inChannels) ch, \(channelPlan)
""")
}
#else
log.info(
"mic capture: \(Int(inFormat.sampleRate)) Hz, \(inChannels) ch, \(channelPlan)")
#endif
// Encode a single mono bus (folded from `inFormat` in the tap): the resampler goes
// mono@inputSR the encoder's 48 kHz stereo, so it handles both the rate change and the
// monostereo duplication, and the wrong-channel downmix never happens.
guard let monoFormat = AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: inFormat.sampleRate,
channels: 1, interleaved: false),
let encoder = try? OpusEncoder(),
let resampler = AVAudioConverter(from: monoFormat, to: encoder.pcmFormat),
let chunk = AVAudioPCMBuffer( let chunk = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: OpusEncoder.framesPerPacket) pcmFormat: encoder.pcmFormat, frameCapacity: OpusEncoder.framesPerPacket)
else { else {
@@ -317,11 +378,59 @@ public final class SessionAudio {
let connection = connection let connection = connection
let flag = flag let flag = flag
// Silence tripwire (tap-confined): a "recording" app can be handed pure digital zeros
// a zeroed input-volume slider, a stale TCC grant, a muted device, OR the wrong channel
// picked and everything downstream looks alive while the host gets silence. Track the
// peak of the EXTRACTED mono bus over the first ~10 s (not the raw device a mic present
// on a channel we didn't grab must still read as silence) and emit exactly ONE verdict.
// This is the log line whose absence made the last occurrence take a host-side tone.
let silenceWindow = Int(inFormat.sampleRate * 10)
let deviceLabel = micUID.isEmpty ? "default input" : micUID
var framesInspected = 0
var inputPeak: Float = 0
var levelReported = false
input.installTap(onBus: 0, bufferSize: 2048, format: inFormat) { buffer, _ in input.installTap(onBus: 0, bufferSize: 2048, format: inFormat) { buffer, _ in
if flag.isStopped { return } if flag.isStopped { return }
let frames = Int(buffer.frameLength)
guard frames > 0, let src = buffer.floatChannelData,
let mono = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: buffer.frameLength),
let dst = mono.floatChannelData?[0]
else { return }
mono.frameLength = buffer.frameLength
// Fold the multi-channel input down to the one mono bus we encode.
Self.foldToMono(
input: src, frames: frames, channels: Int(buffer.format.channelCount),
interleaved: buffer.format.isInterleaved, pinned: pinnedChannel, out: dst)
if !levelReported {
var localPeak: Float = 0
for i in 0..<frames where abs(dst[i]) > localPeak { localPeak = abs(dst[i]) }
if localPeak > inputPeak { inputPeak = localPeak }
framesInspected += frames
if framesInspected >= silenceWindow {
levelReported = true
if inputPeak == 0 {
log.warning("""
mic uplink has been pure digital SILENCE for 10 s (\(deviceLabel), \
\(channelPlan)) — check the input level (System Settings → Sound → \
Input), Privacy & Security → Microphone, and the Microphone channel in \
Settings; the host is receiving zeros
""")
} else {
let dbfs = 20 * log10(inputPeak)
log.info("""
mic uplink OK — peak \(String(format: "%.1f", dbfs)) dBFS over first \
10 s (\(deviceLabel), \(channelPlan))
""")
}
}
}
let ratio = 48_000 / inFormat.sampleRate let ratio = 48_000 / inFormat.sampleRate
let outCapacity = AVAudioFrameCount( let outCapacity = AVAudioFrameCount((Double(frames) * ratio).rounded(.up) + 64)
(Double(buffer.frameLength) * ratio).rounded(.up) + 64)
guard let staging = AVAudioPCMBuffer( guard let staging = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: outCapacity) pcmFormat: encoder.pcmFormat, frameCapacity: outCapacity)
else { return } else { return }
@@ -334,7 +443,7 @@ public final class SessionAudio {
} }
fed = true fed = true
outStatus.pointee = .haveData outStatus.pointee = .haveData
return buffer return mono
} }
guard status != .error, let p = staging.floatChannelData?[0] else { return } guard status != .error, let p = staging.floatChannelData?[0] else { return }
fifo.append(contentsOf: UnsafeBufferPointer( fifo.append(contentsOf: UnsafeBufferPointer(
@@ -378,6 +487,42 @@ public final class SessionAudio {
stateLock.unlock() stateLock.unlock()
log.info("mic uplink started (\(micUID.isEmpty ? "default input" : micUID))") log.info("mic uplink started (\(micUID.isEmpty ? "default input" : micUID))")
} }
/// Fold `channels` of input (`floatChannelData` layout: `interleaved` one buffer strided by
/// channel count; else one buffer per channel) down to a single mono bus in `out` (`frames`
/// long). `pinned` (0-based, must be `< channels`) copies exactly that channel the fix for a
/// mic on one input of a multi-channel interface; `nil` sums every channel, clamped to
/// [-1, 1], so a lone hot channel still passes at full level instead of the silent 0/1 the
/// default Nstereo downmix would grab. Pure + `internal` for unit testing the index math.
static func foldToMono(
input: UnsafePointer<UnsafeMutablePointer<Float>>, frames: Int, channels: Int,
interleaved: Bool, pinned: Int?, out: UnsafeMutablePointer<Float>
) {
if let ch = pinned, ch < channels {
if interleaved {
let d = input[0]
for i in 0..<frames { out[i] = d[i * channels + ch] }
} else {
let d = input[ch]
for i in 0..<frames { out[i] = d[i] }
}
} else if interleaved {
let d = input[0]
for i in 0..<frames {
var s: Float = 0
for c in 0..<channels { s += d[i * channels + c] }
out[i] = max(-1, min(1, s))
}
} else {
let d0 = input[0]
for i in 0..<frames { out[i] = d0[i] }
for c in 1..<channels {
let d = input[c]
for i in 0..<frames { out[i] += d[i] }
}
if channels > 1 { for i in 0..<frames { out[i] = max(-1, min(1, out[i])) } }
}
}
#endif #endif
#if os(macOS) #if os(macOS)
@@ -387,5 +532,18 @@ public final class SessionAudio {
unit, kAudioOutputUnitProperty_CurrentDevice, kAudioUnitScope_Global, 0, unit, kAudioOutputUnitProperty_CurrentDevice, kAudioUnitScope_Global, 0,
&dev, UInt32(MemoryLayout<AudioDeviceID>.size)) == noErr &dev, UInt32(MemoryLayout<AudioDeviceID>.size)) == noErr
} }
/// Read back the AUHAL's live device the definitive "what are we actually capturing
/// from", which catches a selection that succeeded on paper but silently fell back to
/// the system default (a stale/changed UID, a device that vanished between resolve and
/// start). 0 / an error means we couldn't tell.
private static func currentDevice(of unit: AudioUnit) -> AudioDeviceID? {
var dev = AudioDeviceID(0)
var size = UInt32(MemoryLayout<AudioDeviceID>.size)
let status = AudioUnitGetProperty(
unit, kAudioOutputUnitProperty_CurrentDevice, kAudioUnitScope_Global, 0, &dev, &size)
guard status == noErr, dev != 0 else { return nil }
return dev
}
#endif #endif
} }
@@ -35,6 +35,10 @@ public struct AccessUnit: Sendable {
public let ptsNs: UInt64 public let ptsNs: UInt64
public let frameIndex: UInt32 public let frameIndex: UInt32
public let flags: UInt32 public let flags: UInt32
/// Client `CLOCK_REALTIME` instant the AU was handed over by the core (post-FEC, decrypted)
/// the **received** measurement point of design/stats-unification.md. The decode stage is
/// `decodedNs - receivedNs`, both client-local (no skew offset applies).
public let receivedNs: Int64
} }
/// One Opus audio packet (48 kHz stereo, 5 ms frames) decode with AVAudioConverter /// One Opus audio packet (48 kHz stereo, 5 ms frames) decode with AVAudioConverter
@@ -79,6 +83,9 @@ public final class PunktfunkConnection {
/// Same role for the feedback drain thread (rumble + HID-output two core planes, /// Same role for the feedback drain thread (rumble + HID-output two core planes,
/// drained sequentially by one thread). /// drained sequentially by one thread).
private let feedbackLock = NSLock() private let feedbackLock = NSLock()
/// Same role for the host-timing (0xCF) puller its own plane in the core, drained
/// non-blockingly by the app's 1 s stats tick (never contends with the blocking pullers).
private let statsLock = NSLock()
/// Negotiated session mode (host-confirmed). /// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0 public private(set) var width: UInt32 = 0
@@ -419,9 +426,13 @@ public final class PunktfunkConnection {
case statusOK: case statusOK:
guard let base = frame.data, frame.len > 0 else { return nil } guard let base = frame.data, frame.len > 0 else { return nil }
let data = Data(bytes: base, count: Int(frame.len)) // copy: ptr valid only until next call let data = Data(bytes: base, count: Int(frame.len)) // copy: ptr valid only until next call
var ts = timespec()
clock_gettime(CLOCK_REALTIME, &ts)
let receivedNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec)
return AccessUnit( return AccessUnit(
data: data, ptsNs: frame.pts_ns, data: data, ptsNs: frame.pts_ns,
frameIndex: frame.frame_index, flags: frame.flags) frameIndex: frame.frame_index, flags: frame.flags,
receivedNs: receivedNs)
case statusNoFrame: case statusNoFrame:
return nil return nil
case statusClosed: case statusClosed:
@@ -657,6 +668,40 @@ public final class PunktfunkConnection {
} }
} }
/// One per-AU host-timing report (0xCF): the host's capturefully-sent duration for the
/// access unit whose `AccessUnit.ptsNs` equals `ptsNs` exactly. The stats consumer derives
/// `network = (receivedNs + clockOffsetNs ptsNs) hostUs` the host/network split of the
/// HUD's `host+network` stage (design/stats-unification.md Phase 2).
public struct HostTiming: Sendable, Equatable {
/// The AU's capture stamp (host capture clock matches the AU's `ptsNs`).
public let ptsNs: UInt64
/// Host capturesent duration, µs.
public let hostUs: UInt32
}
/// Pull the next per-AU host timing; nil on timeout, throws `.closed` once the session
/// ended. Best-effort plane: an older host never emits any keep showing the combined
/// `host+network` stage then. Drain non-blockingly (`timeoutMs: 0`) from ONE stats
/// consumer (its own core plane, safe alongside the other pullers).
public func nextHostTiming(timeoutMs: UInt32 = 0) throws -> HostTiming? {
statsLock.lock()
defer { statsLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkHostTiming()
let rc = punktfunk_connection_next_host_timing(h, &out, timeoutMs)
switch rc {
case statusOK:
return HostTiming(ptsNs: out.pts_ns, hostUs: out.host_us)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Send one input event (delivered to the host as a QUIC datagram). Thread-safe; /// Send one input event (delivered to the host as a QUIC datagram). Thread-safe;
/// silently dropped after close. /// silently dropped after close.
public func send(_ event: PunktfunkInputEvent) { public func send(_ event: PunktfunkInputEvent) {
@@ -676,10 +721,12 @@ public final class PunktfunkConnection {
pumpLock.lock() // pullers exit at their next poll boundary, releasing these pumpLock.lock() // pullers exit at their next poll boundary, releasing these
audioLock.lock() audioLock.lock()
feedbackLock.lock() feedbackLock.lock()
statsLock.lock()
abiLock.lock() abiLock.lock()
let h = handle let h = handle
handle = nil handle = nil
abiLock.unlock() abiLock.unlock()
statsLock.unlock()
feedbackLock.unlock() feedbackLock.unlock()
audioLock.unlock() audioLock.unlock()
pumpLock.unlock() pumpLock.unlock()
@@ -24,6 +24,12 @@ public enum DefaultsKey {
public static let micEnabled = "punktfunk.micEnabled" public static let micEnabled = "punktfunk.micEnabled"
public static let speakerUID = "punktfunk.speakerUID" public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID" public static let micUID = "punktfunk.micUID"
/// macOS: which input channel of the chosen mic device feeds the host. 0 = "Auto" (sum every
/// channel to mono a mic on a single input of a multi-channel interface passes at full
/// level); n1 pins 1-based input channel n. Multi-channel interfaces expose the mic on ONE
/// discrete channel, and the default Nstereo downmix grabs channels 0/1 (silence when the mic
/// is higher up), so we fold to mono ourselves. Only meaningful for multi-channel devices.
public static let micChannel = "punktfunk.micChannel"
public static let presenter = "punktfunk.presenter" public static let presenter = "punktfunk.presenter"
/// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host /// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host
/// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR. /// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR.
@@ -0,0 +1,88 @@
// Splits the unified stats model's `host+network` stage (capturereceived) into its `host`
// (capturefully-sent, reported per AU by the host on the 0xCF plane) and `network`
// (the remainder) terms design/stats-unification.md Phase 2.
//
// Receipt samples are recorded per frame from the pump path; host timings are matched to them
// by exact pts (the 0xCF datagram carries the AU's own `pts_ns`). Best-effort by construction:
// a lost 0xCF datagram, an FEC-dropped AU, or an old host that never emits the plane simply
// contributes no split sample the HUD then keeps the combined `host+network` line. NSLock
// rather than an actor the receipt writer is the non-async pump path (same pattern as
// LatencyMeter/FrameMeter).
import Foundation
/// Per-frame `host` / `network` sampler: `recordReceipt` at AU receipt (pts + the combined
/// capturereceived interval), `noteHostTiming` per drained 0xCF report, `drain` the window's
/// p50s once a second. The pending ring is bounded (drop-oldest) so an old host receipts
/// forever, timings never costs a fixed ~4 KB, not growth.
public final class HostNetworkSplitter: @unchecked Sendable {
private let lock = NSLock()
/// Received AUs awaiting their 0xCF host timing: (pts, combined capturereceived µs).
private var pending: [(ptsNs: UInt64, combinedUs: Int64)] = []
private var hostUsSamples: [Int64] = []
private var networkUsSamples: [Int64] = []
/// ~1 s of frames at 240 fps; beyond it the oldest receipt can no longer expect a match.
private static let pendingCap = 256
public init() {}
/// Record one frame at receipt. `ptsNs` is the host capture clock (the AU's pts),
/// `receivedNs` the client `CLOCK_REALTIME` receipt instant (`AccessUnit.receivedNs`),
/// `offsetNs` the connect-time hostclient clock offset (0 = uncorrected). Same
/// absurd-value clamp as LatencyMeter a sample it would drop must not linger here.
public func recordReceipt(ptsNs: UInt64, receivedNs: Int64, offsetNs: Int64) {
let combinedNs = receivedNs &+ offsetNs &- Int64(bitPattern: ptsNs)
guard combinedNs > 0, combinedNs < 10_000_000_000 else { return }
lock.lock()
pending.append((ptsNs: ptsNs, combinedUs: combinedNs / 1000))
if pending.count > Self.pendingCap {
pending.removeFirst(pending.count - Self.pendingCap)
}
lock.unlock()
}
/// Match one host timing (0xCF) to its receipt: `host` = the reported capturesent,
/// `network` = the combined interval minus it, floored at 0 (the terms tile per frame; a
/// slightly-off skew offset must not produce a negative wire time). Unmatched timings
/// the AU was FEC-dropped, or its receipt raced this drain are simply skipped.
public func noteHostTiming(ptsNs: UInt64, hostUs: UInt32) {
lock.lock()
defer { lock.unlock() }
guard let i = pending.firstIndex(where: { $0.ptsNs == ptsNs }) else { return }
let combinedUs = pending.remove(at: i).combinedUs
hostUsSamples.append(Int64(hostUs))
networkUsSamples.append(max(0, combinedUs - Int64(hostUs)))
}
public struct Split: Sendable {
public let hostP50Ms: Double
public let networkP50Ms: Double
public let count: Int
}
/// The window's p50s since the last drain, then reset (matched samples only; the pending
/// ring survives a receipt may still match a timing drained next tick). `nil` when no
/// timing matched in the interval the caller falls back to the combined stage.
public func drain() -> Split? {
lock.lock()
let host = hostUsSamples.sorted()
let network = networkUsSamples.sorted()
hostUsSamples.removeAll(keepingCapacity: true)
networkUsSamples.removeAll(keepingCapacity: true)
lock.unlock()
guard !host.isEmpty else { return nil }
func p50(_ sorted: [Int64]) -> Double {
Double(sorted[min(sorted.count / 2, sorted.count - 1)]) / 1000.0 // µs ms
}
return Split(hostP50Ms: p50(host), networkP50Ms: p50(network), count: host.count)
}
/// Forget everything (pending receipts + window) a fresh connection starts clean.
public func reset() {
lock.lock()
pending.removeAll()
hostUsSamples.removeAll()
networkUsSamples.removeAll()
lock.unlock()
}
}
@@ -1,23 +1,25 @@
// Per-frame latency sampler for the live HUD: records capture->client-receipt latency and drains // Per-frame latency-stage sampler for the live HUD: records one interval per frame (an end
// percentiles on demand. NSLock rather than an actor the writer is the non-async pump/arrival // instant minus a start instant, both CLOCK_REALTIME ns) and drains percentiles on demand.
// path (same pattern as the app's FrameMeter). // NSLock rather than an actor the writers are the non-async pump/decode/present paths (same
// pattern as the app's FrameMeter).
import Foundation import Foundation
/// Samples the **capture->client-receipt** latency of each access unit and reports percentiles. /// Samples one **latency stage** per frame and reports percentiles. One instance per stage of the
/// unified stats model (design/stats-unification.md):
/// ///
/// The latency is `now - pts_ns`, where `pts_ns` is the host's capture wall clock (the AU's pts) and /// - `host+network` = capturereceived: `record(ptsNs:offsetNs:)` at AU receipt.
/// `now` is the client's `CLOCK_REALTIME` instant the AU was received, shifted by the connect-time /// - `decode` = receiveddecoded and `display` = decodeddisplayed: client-local single-clock
/// **clock-skew offset** (`PunktfunkConnection.clockOffsetNs`, host minus client) so the difference /// stages `record(ptsNs:atNs:offsetNs:)` with the start instant as `ptsNs` and `offsetNs: 0`.
/// is valid across machines. `offsetNs == 0` means an old host that didn't answer the skew handshake /// - `end-to-end` = capturedisplayed, measured directly (never summed from the stages):
/// (or genuinely synced clocks) the number is then only meaningful same-host. /// `record(ptsNs:atNs:offsetNs:)` at present.
/// ///
/// SCOPE (stage-1 presenter): this covers host capture -> encode -> FEC -> network -> reassembly -> /// For the host-anchored intervals (capture) the sample is `end + offset - pts_ns`, where
/// decrypt -> handed to the presenter. It does **not** include the on-device VideoToolbox decode or /// `pts_ns` is the host's capture wall clock (the AU's pts) and the connect-time **clock-skew
/// the `AVSampleBufferDisplayLayer` present that layer decodes and presents compressed samples /// offset** (`PunktfunkConnection.clockOffsetNs`, host minus client) makes the difference valid
/// internally with no per-frame callback. True decode->present (the full glass-to-glass) needs the /// across machines. `offsetNs == 0` means an old host that didn't answer the skew handshake (or
/// stage-2 presenter (`VTDecompressionSession` decode-completion + `CAMetalLayer`/display-link /// genuinely synced clocks) the number is then only meaningful same-host, and the HUD tags the
/// present); this meter is the substrate it will extend. /// end-to-end line `(same-host clock)`.
public final class LatencyMeter: @unchecked Sendable { public final class LatencyMeter: @unchecked Sendable {
private let lock = NSLock() private let lock = NSLock()
private var samplesUs: [Int64] = [] private var samplesUs: [Int64] = []
@@ -34,12 +36,16 @@ public final class LatencyMeter: @unchecked Sendable {
record(ptsNs: ptsNs, atNs: nowNs, offsetNs: offsetNs) record(ptsNs: ptsNs, atNs: nowNs, offsetNs: offsetNs)
} }
/// Record one frame whose latency is `atNs + offsetNs - ptsNs` an EXPLICIT client instant /// Record one frame whose sample is `atNs + offsetNs - ptsNs` an EXPLICIT end instant
/// rather than now. The stage-2 presenter uses this to stamp capturepresent at the display /// rather than now. `ptsNs` is the stage's start point: the AU pts for the host-anchored
/// link's target present time (not the moment the present call ran). All in `CLOCK_REALTIME`. /// intervals, or a client stamp (receivedNs / decodedNs, with `offsetNs: 0`) for the local
/// decode/display stages. The stage-2 presenter stamps its present-side samples at the
/// display link's target present time (not the moment the present call ran). All in
/// `CLOCK_REALTIME`.
public func record(ptsNs: UInt64, atNs: Int64, offsetNs: Int64) { public func record(ptsNs: UInt64, atNs: Int64, offsetNs: Int64) {
let latNs = atNs &+ offsetNs &- Int64(bitPattern: ptsNs) let latNs = atNs &+ offsetNs &- Int64(bitPattern: ptsNs)
// Drop absurd values (a clock step, a wildly wrong offset, or garbage pts). // Drop absurd values (a clock step, a wildly wrong offset, garbage pts, or a stage whose
// start stamp is missing/after its end) samples are clamped to (0, 10 s).
guard latNs > 0, latNs < 10_000_000_000 else { return } guard latNs > 0, latNs < 10_000_000_000 else { return }
lock.lock() lock.lock()
samplesUs.append(latNs / 1000) samplesUs.append(latNs / 1000)
@@ -38,8 +38,9 @@ final class SessionPresenter {
func start( func start(
connection: PunktfunkConnection, connection: PunktfunkConnection,
baseLayer: AVSampleBufferDisplayLayer, baseLayer: AVSampleBufferDisplayLayer,
presentMeter: LatencyMeter?, endToEndMeter: LatencyMeter?,
presentTailMeter: LatencyMeter? = nil, decodeMeter: LatencyMeter? = nil,
displayMeter: LatencyMeter? = nil,
makeDisplayLink: (AnyObject, Selector) -> CADisplayLink, makeDisplayLink: (AnyObject, Selector) -> CADisplayLink,
onFrame: (@Sendable (AccessUnit) -> Void)?, onFrame: (@Sendable (AccessUnit) -> Void)?,
onSessionEnd: (@Sendable () -> Void)? onSessionEnd: (@Sendable () -> Void)?
@@ -59,7 +60,8 @@ final class SessionPresenter {
#endif #endif
if !forceStage1, if !forceStage1,
let pipeline = Stage2Pipeline( let pipeline = Stage2Pipeline(
presentMeter: presentMeter, presentTailMeter: presentTailMeter) { endToEndMeter: endToEndMeter, decodeMeter: decodeMeter,
displayMeter: displayMeter) {
let metal = pipeline.layer let metal = pipeline.layer
// The opaque metal layer composites OVER the AVSampleBufferDisplayLayer base, which // The opaque metal layer composites OVER the AVSampleBufferDisplayLayer base, which
// sits idle (un-enqueued) in stage-2. contentsScale + frame are set in layout(). // sits idle (un-enqueued) in stage-2. contentsScale + frame are set in layout().
@@ -1,7 +1,8 @@
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output // Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output
// drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick` // drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick`
// once per vsync to draw + present the newest ready frame and stamp capturepresent. Mirrors // once per vsync to draw + present the newest ready frame and stamp the unified latency stages
// StreamPump's lifecycle (one per start; cancel is permanent). // (end-to-end captureon-glass, plus the decode and display stage terms
// design/stats-unification.md). Mirrors StreamPump's lifecycle (one per start; cancel is permanent).
// //
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` + // Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` +
// `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded) // `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded)
@@ -40,8 +41,8 @@ public final class Stage2Pipeline {
private let ring = ReadyRing() private let ring = ReadyRing()
private let presenter: MetalVideoPresenter private let presenter: MetalVideoPresenter
private let decoder: VideoDecoder private let decoder: VideoDecoder
private let presentMeter: LatencyMeter? private let endToEndMeter: LatencyMeter?
private let presentTailMeter: LatencyMeter? private let displayMeter: LatencyMeter?
private let recovery = KeyframeRecovery() private let recovery = KeyframeRecovery()
private var token = StopFlag() private var token = StopFlag()
private var offsetNs: Int64 = 0 private var offsetNs: Int64 = 0
@@ -56,28 +57,41 @@ public final class Stage2Pipeline {
/// The Metal layer the hosting view installs + sizes. /// The Metal layer the hosting view installs + sizes.
public var layer: CAMetalLayer { presenter.layer } public var layer: CAMetalLayer { presenter.layer }
/// `presentMeter` records capturepresent (the glass-to-glass term); `presentTailMeter` /// Unified-stats meters (design/stats-unification.md): `endToEndMeter` records the headline
/// records decode-completionpresent (the ring wait + render the tail stage-2 exists to /// end-to-end (captureon-glass, skew-corrected); `decodeMeter` the decode stage
/// shorten). Both optional: metering never gates the presenter choice. Returns nil if Metal /// (receiveddecoded); `displayMeter` the display stage (decodedon-glass, the ring wait +
/// can't be set up (headless / no GPU) caller falls back to the stage-1 presenter. /// render + vsync the tail stage-2 exists to shorten). All optional: metering never gates
public init?(presentMeter: LatencyMeter?, presentTailMeter: LatencyMeter? = nil) { /// the presenter choice. Returns nil if Metal can't be set up (headless / no GPU) caller
/// falls back to the stage-1 presenter.
public init?(
endToEndMeter: LatencyMeter?,
decodeMeter: LatencyMeter? = nil,
displayMeter: LatencyMeter? = nil
) {
guard let presenter = MetalVideoPresenter.make() else { return nil } guard let presenter = MetalVideoPresenter.make() else { return nil }
self.presenter = presenter self.presenter = presenter
self.presentMeter = presentMeter self.endToEndMeter = endToEndMeter
self.presentTailMeter = presentTailMeter self.displayMeter = displayMeter
let ring = ring let ring = ring
let recovery = recovery let recovery = recovery
self.decoder = VideoDecoder( self.decoder = VideoDecoder(
onDecoded: { ring.submit($0) }, onDecoded: { frame in
// Decode stage = receiveddecoded, both client CLOCK_REALTIME (offset 0 no
// skew applies). Stamped at decode completion, so it covers every decoded frame,
// including ones the newest-wins ring drops before present.
decodeMeter?.record(
ptsNs: UInt64(frame.receivedNs), atNs: frame.decodedNs, offsetNs: 0)
ring.submit(frame)
},
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to // Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to
// re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't // re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't
// otherwise come soon). Throttled in KeyframeRecovery. // otherwise come soon). Throttled in KeyframeRecovery.
onDecodeError: { _ in recovery.request() }) onDecodeError: { _ in recovery.request() })
} }
/// Start pulling AUs into the decoder. MAIN THREAD. `onFrame` fires per AU at receipt (captureclient /// Start pulling AUs into the decoder. MAIN THREAD. `onFrame` fires per AU at receipt (the
/// meter, exactly as stage-1); `onSessionEnd` on close. `clockOffsetNs` (host minus client) makes the /// host+network / capturereceived meter, exactly as stage-1); `onSessionEnd` on close.
/// present stamp cross-machine valid. /// `clockOffsetNs` (host minus client) makes the end-to-end stamp cross-machine valid.
public func start( public func start(
connection: PunktfunkConnection, connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?, onFrame: (@Sendable (AccessUnit) -> Void)?,
@@ -174,14 +188,16 @@ public final class Stage2Pipeline {
public func renderTick(targetPresentNs: Int64) { public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return } guard let frame = ring.take() else { return }
let offsetNs = offsetNs let offsetNs = offsetNs
let presentMeter = presentMeter let endToEndMeter = endToEndMeter
let presentTailMeter = presentTailMeter let displayMeter = displayMeter
let rendered = presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) { presentedNs in let rendered = presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) { presentedNs in
let atNs = presentedNs ?? targetPresentNs let atNs = presentedNs ?? targetPresentNs
presentMeter?.record(ptsNs: frame.ptsNs, atNs: atNs, offsetNs: offsetNs) // End-to-end = captureon-glass, measured directly (skew-corrected via the
// Present tail = decode-completion on-glass. Both instants are client // connect-time clock offset) the HUD headline.
// CLOCK_REALTIME, so no skew offset applies. endToEndMeter?.record(ptsNs: frame.ptsNs, atNs: atNs, offsetNs: offsetNs)
presentTailMeter?.record(ptsNs: UInt64(frame.decodedNs), atNs: atNs, offsetNs: 0) // Display stage = decoded on-glass. Both instants are client CLOCK_REALTIME,
// so no skew offset applies.
displayMeter?.record(ptsNs: UInt64(frame.decodedNs), atNs: atNs, offsetNs: 0)
} }
if !rendered { ring.putBack(frame) } if !rendered { ring.putBack(frame) }
} }
@@ -61,7 +61,7 @@ public enum Stage444Probe {
guard created == noErr, let session else { return false } guard created == noErr, let session else { return false }
defer { VTDecompressionSessionInvalidate(session) } defer { VTDecompressionSessionInvalidate(session) }
let au = AccessUnit(data: data, ptsNs: 0, frameIndex: 0, flags: 0) let au = AccessUnit(data: data, ptsNs: 0, frameIndex: 0, flags: 0, receivedNs: 0)
guard let sample = AnnexB.sampleBuffer(au: au, format: format, codec: .hevc) else { return false } guard let sample = AnnexB.sampleBuffer(au: au, format: format, codec: .hevc) else { return false }
var produced: OSType = 0 var produced: OSType = 0
@@ -15,6 +15,10 @@ import VideoToolbox
public struct ReadyFrame: @unchecked Sendable { public struct ReadyFrame: @unchecked Sendable {
/// Host capture clock (the AU's pts), in nanoseconds. /// Host capture clock (the AU's pts), in nanoseconds.
public let ptsNs: UInt64 public let ptsNs: UInt64
/// Client `CLOCK_REALTIME` instant the AU was received (`AccessUnit.receivedNs`, threaded
/// through the decode via the frame refcon), in nanoseconds. 0 when unknown (a caller that
/// didn't stamp receipt) the decode-stage meter then drops the sample via its sanity guard.
public let receivedNs: Int64
/// Client `CLOCK_REALTIME` instant decode completed, in nanoseconds. /// Client `CLOCK_REALTIME` instant decode completed, in nanoseconds.
public let decodedNs: Int64 public let decodedNs: Int64
/// The decoded image 8-bit NV12 biplanar (SDR) or 10-bit P010 biplanar (HDR), Metal-compatible. /// The decoded image 8-bit NV12 biplanar (SDR) or 10-bit P010 biplanar (HDR), Metal-compatible.
@@ -25,13 +29,16 @@ public struct ReadyFrame: @unchecked Sendable {
} }
/// The C output callback can't capture context, so VideoToolbox hands it the refcon we set at /// The C output callback can't capture context, so VideoToolbox hands it the refcon we set at
/// session creation a pointer back to the owning `VideoDecoder`. /// session creation a pointer back to the owning `VideoDecoder`. The per-frame refcon carries
/// the AU's `receivedNs` as a pointer bit pattern (a scalar smuggled through the C void*, never
/// dereferenced) so the decode stage can be computed against decode-completion.
private let decoderOutputCallback: VTDecompressionOutputCallback = { private let decoderOutputCallback: VTDecompressionOutputCallback = {
refcon, _, status, _, imageBuffer, pts, _ in refcon, frameRefcon, status, _, imageBuffer, pts, _ in
guard let refcon else { return } guard let refcon else { return }
let receivedNs = frameRefcon.map { Int64(Int(bitPattern: $0)) } ?? 0
Unmanaged<VideoDecoder>.fromOpaque(refcon) Unmanaged<VideoDecoder>.fromOpaque(refcon)
.takeUnretainedValue() .takeUnretainedValue()
.handleDecoded(status: status, imageBuffer: imageBuffer, pts: pts) .handleDecoded(status: status, imageBuffer: imageBuffer, pts: pts, receivedNs: receivedNs)
} }
/// Owns a `VTDecompressionSession` rebuilt whenever the format description changes (every IDR / /// Owns a `VTDecompressionSession` rebuilt whenever the format description changes (every IDR /
@@ -112,7 +119,9 @@ public final class VideoDecoder: @unchecked Sendable {
session, session,
sampleBuffer: sample, sampleBuffer: sample,
flags: [._EnableAsynchronousDecompression], flags: [._EnableAsynchronousDecompression],
frameRefcon: nil, // The AU's receipt instant rides through as a bit pattern (nil for 0 the output
// callback maps that back to 0); the callback needs it to stamp the decode stage.
frameRefcon: UnsafeMutableRawPointer(bitPattern: Int(au.receivedNs)),
infoFlagsOut: &infoOut) infoFlagsOut: &infoOut)
lock.unlock() lock.unlock()
if status != noErr { if status != noErr {
@@ -218,8 +227,11 @@ public final class VideoDecoder: @unchecked Sendable {
return true return true
} }
/// VT thread. Stamp decode-completion and enqueue, or report the error. /// VT thread. Stamp decode-completion and enqueue, or report the error. `receivedNs` is the
fileprivate func handleDecoded(status: OSStatus, imageBuffer: CVImageBuffer?, pts: CMTime) { /// AU's receipt instant threaded through the frame refcon (0 = unknown).
fileprivate func handleDecoded(
status: OSStatus, imageBuffer: CVImageBuffer?, pts: CMTime, receivedNs: Int64
) {
guard status == noErr, let imageBuffer else { guard status == noErr, let imageBuffer else {
onDecodeError(status) onDecodeError(status)
return return
@@ -242,6 +254,8 @@ public final class VideoDecoder: @unchecked Sendable {
|| fmt == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange || fmt == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange
|| fmt == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange || fmt == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange
onDecoded( onDecoded(
ReadyFrame(ptsNs: ptsNs, decodedNs: decodedNs, pixelBuffer: imageBuffer, isHDR: isHDR)) ReadyFrame(
ptsNs: ptsNs, receivedNs: receivedNs, decodedNs: decodedNs,
pixelBuffer: imageBuffer, isHDR: isHDR))
} }
} }
@@ -85,39 +85,45 @@ public struct StreamView: NSViewRepresentable {
private let onCaptureChange: ((Bool) -> Void)? private let onCaptureChange: ((Bool) -> Void)?
private let onFrame: (@Sendable (AccessUnit) -> Void)? private let onFrame: (@Sendable (AccessUnit) -> Void)?
private let onSessionEnd: (@Sendable () -> Void)? private let onSessionEnd: (@Sendable () -> Void)?
private let presentMeter: LatencyMeter? private let endToEndMeter: LatencyMeter?
private let presentTailMeter: LatencyMeter? private let decodeMeter: LatencyMeter?
private let displayMeter: LatencyMeter?
/// `onFrame`/`onSessionEnd` fire on the pump thread hop to the main actor for UI. /// `onFrame`/`onSessionEnd` fire on the pump thread hop to the main actor for UI.
/// `captureEnabled: false` disables input capture entirely while UI (e.g. a trust /// `captureEnabled: false` disables input capture entirely while UI (e.g. a trust
/// prompt) is layered over the stream; flipping it to true auto-engages capture /// prompt) is layered over the stream; flipping it to true auto-engages capture
/// once. `onCaptureChange` (main thread) reports engage/release drive the HUD's /// once. `onCaptureChange` (main thread) reports engage/release drive the HUD's
/// "click to capture" / " releases" hint with it. `presentMeter` records capturepresent /// "click to capture" / " releases" hint with it. The meters record the unified latency
/// and `presentTailMeter` decodepresent when the stage-2 presenter is active. /// stages when the stage-2 presenter is active (design/stats-unification.md):
/// `endToEndMeter` captureon-glass, `decodeMeter` receiveddecoded, `displayMeter`
/// decodedon-glass.
public init( public init(
connection: PunktfunkConnection, connection: PunktfunkConnection,
captureEnabled: Bool = true, captureEnabled: Bool = true,
onCaptureChange: ((Bool) -> Void)? = nil, onCaptureChange: ((Bool) -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil, onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil, onSessionEnd: (@Sendable () -> Void)? = nil,
presentMeter: LatencyMeter? = nil, endToEndMeter: LatencyMeter? = nil,
presentTailMeter: LatencyMeter? = nil decodeMeter: LatencyMeter? = nil,
displayMeter: LatencyMeter? = nil
) { ) {
self.connection = connection self.connection = connection
self.captureEnabled = captureEnabled self.captureEnabled = captureEnabled
self.onCaptureChange = onCaptureChange self.onCaptureChange = onCaptureChange
self.onFrame = onFrame self.onFrame = onFrame
self.onSessionEnd = onSessionEnd self.onSessionEnd = onSessionEnd
self.presentMeter = presentMeter self.endToEndMeter = endToEndMeter
self.presentTailMeter = presentTailMeter self.decodeMeter = decodeMeter
self.displayMeter = displayMeter
} }
public func makeNSView(context: Context) -> StreamLayerView { public func makeNSView(context: Context) -> StreamLayerView {
let view = StreamLayerView() let view = StreamLayerView()
view.onCaptureChange = onCaptureChange view.onCaptureChange = onCaptureChange
view.captureEnabled = captureEnabled view.captureEnabled = captureEnabled
view.presentMeter = presentMeter view.endToEndMeter = endToEndMeter
view.presentTailMeter = presentTailMeter view.decodeMeter = decodeMeter
view.displayMeter = displayMeter
view.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd) view.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
return view return view
} }
@@ -125,8 +131,9 @@ public struct StreamView: NSViewRepresentable {
public func updateNSView(_ view: StreamLayerView, context: Context) { public func updateNSView(_ view: StreamLayerView, context: Context) {
view.onCaptureChange = onCaptureChange view.onCaptureChange = onCaptureChange
view.captureEnabled = captureEnabled view.captureEnabled = captureEnabled
view.presentMeter = presentMeter view.endToEndMeter = endToEndMeter
view.presentTailMeter = presentTailMeter view.decodeMeter = decodeMeter
view.displayMeter = displayMeter
// SwiftUI reuses the NSView across state changes repoint the pump only when the // SwiftUI reuses the NSView across state changes repoint the pump only when the
// connection identity actually changed. // connection identity actually changed.
if view.connection !== connection { if view.connection !== connection {
@@ -141,10 +148,11 @@ public struct StreamView: NSViewRepresentable {
public final class StreamLayerView: NSView { public final class StreamLayerView: NSView {
private let displayLayer = AVSampleBufferDisplayLayer() private let displayLayer = AVSampleBufferDisplayLayer()
/// Record capturepresent / decodepresent when the stage-2 presenter is active. /// Record the unified latency stages (end-to-end / decode / display) when the stage-2
/// Consulted at start(). /// presenter is active. Consulted at start().
var presentMeter: LatencyMeter? var endToEndMeter: LatencyMeter?
var presentTailMeter: LatencyMeter? var decodeMeter: LatencyMeter?
var displayMeter: LatencyMeter?
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the /// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
/// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback. /// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback.
private let presenter = SessionPresenter() private let presenter = SessionPresenter()
@@ -571,8 +579,9 @@ public final class StreamLayerView: NSView {
presenter.start( presenter.start(
connection: connection, connection: connection,
baseLayer: displayLayer, baseLayer: displayLayer,
presentMeter: presentMeter, endToEndMeter: endToEndMeter,
presentTailMeter: presentTailMeter, decodeMeter: decodeMeter,
displayMeter: displayMeter,
makeDisplayLink: { displayLink(target: $0, selector: $1) }, makeDisplayLink: { displayLink(target: $0, selector: $1) },
onFrame: onFrame, onFrame: onFrame,
onSessionEnd: onSessionEnd) onSessionEnd: onSessionEnd)
@@ -50,8 +50,9 @@ public struct StreamView: UIViewControllerRepresentable {
private let onCaptureChange: ((Bool) -> Void)? private let onCaptureChange: ((Bool) -> Void)?
private let onFrame: (@Sendable (AccessUnit) -> Void)? private let onFrame: (@Sendable (AccessUnit) -> Void)?
private let onSessionEnd: (@Sendable () -> Void)? private let onSessionEnd: (@Sendable () -> Void)?
private let presentMeter: LatencyMeter? private let endToEndMeter: LatencyMeter?
private let presentTailMeter: LatencyMeter? private let decodeMeter: LatencyMeter?
private let displayMeter: LatencyMeter?
public init( public init(
connection: PunktfunkConnection, connection: PunktfunkConnection,
@@ -59,24 +60,27 @@ public struct StreamView: UIViewControllerRepresentable {
onCaptureChange: ((Bool) -> Void)? = nil, onCaptureChange: ((Bool) -> Void)? = nil,
onFrame: (@Sendable (AccessUnit) -> Void)? = nil, onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
onSessionEnd: (@Sendable () -> Void)? = nil, onSessionEnd: (@Sendable () -> Void)? = nil,
presentMeter: LatencyMeter? = nil, endToEndMeter: LatencyMeter? = nil,
presentTailMeter: LatencyMeter? = nil decodeMeter: LatencyMeter? = nil,
displayMeter: LatencyMeter? = nil
) { ) {
self.connection = connection self.connection = connection
self.captureEnabled = captureEnabled self.captureEnabled = captureEnabled
self.onCaptureChange = onCaptureChange self.onCaptureChange = onCaptureChange
self.onFrame = onFrame self.onFrame = onFrame
self.onSessionEnd = onSessionEnd self.onSessionEnd = onSessionEnd
self.presentMeter = presentMeter self.endToEndMeter = endToEndMeter
self.presentTailMeter = presentTailMeter self.decodeMeter = decodeMeter
self.displayMeter = displayMeter
} }
public func makeUIViewController(context: Context) -> StreamViewController { public func makeUIViewController(context: Context) -> StreamViewController {
let controller = StreamViewController() let controller = StreamViewController()
controller.onCaptureChange = onCaptureChange controller.onCaptureChange = onCaptureChange
controller.captureEnabled = captureEnabled controller.captureEnabled = captureEnabled
controller.presentMeter = presentMeter controller.endToEndMeter = endToEndMeter
controller.presentTailMeter = presentTailMeter controller.decodeMeter = decodeMeter
controller.displayMeter = displayMeter
controller.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd) controller.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
return controller return controller
} }
@@ -84,8 +88,9 @@ public struct StreamView: UIViewControllerRepresentable {
public func updateUIViewController(_ controller: StreamViewController, context: Context) { public func updateUIViewController(_ controller: StreamViewController, context: Context) {
controller.onCaptureChange = onCaptureChange controller.onCaptureChange = onCaptureChange
controller.captureEnabled = captureEnabled controller.captureEnabled = captureEnabled
controller.presentMeter = presentMeter controller.endToEndMeter = endToEndMeter
controller.presentTailMeter = presentTailMeter controller.decodeMeter = decodeMeter
controller.displayMeter = displayMeter
if controller.connection !== connection { if controller.connection !== connection {
controller.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd) controller.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
} }
@@ -101,10 +106,11 @@ public struct StreamView: UIViewControllerRepresentable {
public final class StreamViewController: UIViewController { public final class StreamViewController: UIViewController {
public private(set) var connection: PunktfunkConnection? public private(set) var connection: PunktfunkConnection?
private var observers: [NSObjectProtocol] = [] private var observers: [NSObjectProtocol] = []
/// Record capturepresent / decodepresent when the stage-2 presenter is active. /// Record the unified latency stages (end-to-end / decode / display) when the stage-2
/// Consulted at start(). /// presenter is active. Consulted at start().
var presentMeter: LatencyMeter? var endToEndMeter: LatencyMeter?
var presentTailMeter: LatencyMeter? var decodeMeter: LatencyMeter?
var displayMeter: LatencyMeter?
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the /// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
/// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback. /// stage-1 StreamPump displayLayer path as the Metal-unavailable / DEBUG fallback.
private let presenter = SessionPresenter() private let presenter = SessionPresenter()
@@ -285,8 +291,9 @@ public final class StreamViewController: UIViewController {
presenter.start( presenter.start(
connection: connection, connection: connection,
baseLayer: streamView.displayLayer, baseLayer: streamView.displayLayer,
presentMeter: presentMeter, endToEndMeter: endToEndMeter,
presentTailMeter: presentTailMeter, decodeMeter: decodeMeter,
displayMeter: displayMeter,
makeDisplayLink: { CADisplayLink(target: $0, selector: $1) }, makeDisplayLink: { CADisplayLink(target: $0, selector: $1) },
onFrame: onFrame, onFrame: onFrame,
onSessionEnd: onSessionEnd) onSessionEnd: onSessionEnd)
@@ -0,0 +1,93 @@
// Multi-channel input mono fold (SessionAudio.foldToMono): the fix for a mic on one channel of
// a multi-channel interface. AVAudioConverter's default Nstereo downmix grabs channels 0/1 dead
// silence when the mic sits higher up so we fold ourselves. This pins the fiddly bits (the
// interleaved stride, channel pinning, the sum-clamp) against regressions without needing hardware.
#if !os(tvOS)
import XCTest
@testable import PunktfunkKit
final class AudioChannelFoldTests: XCTestCase {
/// Drive `foldToMono` over channel data expressed as `[[Float]]`, mirroring the two
/// `floatChannelData` layouts:
/// - deinterleaved: each inner array is one channel (all `frames` long).
/// - interleaved: a single inner array already interleaved (c0f0, c1f0, ), with the real
/// channel count passed separately.
private func fold(
_ planes: [[Float]], frames: Int, channels: Int, interleaved: Bool, pinned: Int?
) -> [Float] {
// One C buffer per plane + a table of pointers to them the shape of floatChannelData.
let buffers: [UnsafeMutablePointer<Float>] = planes.map { plane in
let p = UnsafeMutablePointer<Float>.allocate(capacity: plane.count)
for i in 0..<plane.count { p[i] = plane[i] }
return p
}
let table = UnsafeMutablePointer<UnsafeMutablePointer<Float>>.allocate(
capacity: buffers.count)
for (i, b) in buffers.enumerated() { table[i] = b }
let out = UnsafeMutablePointer<Float>.allocate(capacity: frames)
defer {
buffers.forEach { $0.deallocate() }
table.deallocate()
out.deallocate()
}
SessionAudio.foldToMono(
input: table, frames: frames, channels: channels,
interleaved: interleaved, pinned: pinned, out: out)
return (0..<frames).map { out[$0] }
}
// A pinned channel is copied verbatim the exact fix: mic on a HIGH channel, not 0/1.
func testPinsHigherChannelDeinterleaved() {
let result = fold(
[[0, 0, 0], [0, 0, 0], [0.1, 0.2, 0.3], [0, 0, 0]],
frames: 3, channels: 4, interleaved: false, pinned: 2)
XCTAssertEqual(result, [0.1, 0.2, 0.3])
}
// Same signal, interleaved layout: [c0f0,c1f0,c2f0,c3f0, c0f1,]. Guards the `i*ch + c` stride.
func testPinsHigherChannelInterleaved() {
let interleaved: [Float] = [
0, 0, 0.1, 0,
0, 0, 0.2, 0,
0, 0, 0.3, 0,
]
let result = fold([interleaved], frames: 3, channels: 4, interleaved: true, pinned: 2)
XCTAssertEqual(result, [0.1, 0.2, 0.3])
}
// Auto (pinned: nil): a lone hot channel amid silence passes at FULL level, never attenuated.
func testAutoSumsAllChannelsSoALoneMicSurvives() {
let result = fold(
[[0, 0], [0.4, -0.4], [0, 0]],
frames: 2, channels: 3, interleaved: false, pinned: nil)
XCTAssertEqual(result, [0.4, -0.4])
}
// Two simultaneously-hot channels sum past the unit range clamped, never wraps/overflows.
func testAutoSumClampsToUnitRange() {
let result = fold(
[[0.8, -0.8], [0.9, -0.9]],
frames: 2, channels: 2, interleaved: false, pinned: nil)
XCTAssertEqual(result, [1.0, -1.0])
}
// A plain mono device is passed through untouched (no clamp, no attenuation).
func testMonoIsIdentity() {
let result = fold(
[[0.25, -0.5, 0.75]], frames: 3, channels: 1, interleaved: false, pinned: nil)
XCTAssertEqual(result, [0.25, -0.5, 0.75])
}
// Belt-and-suspenders: an out-of-range pin (the tap already guards, but the setting is
// persisted) is ignored by foldToMono's own `ch < channels` guard, which sums instead of
// reading past the buffer.
func testOutOfRangePinFallsBackToSum() {
let result = fold(
[[0, 0], [0.3, 0.3]],
frames: 2, channels: 2, interleaved: false, pinned: 2)
XCTAssertEqual(result, [0.3, 0.3])
}
}
#endif
@@ -0,0 +1,107 @@
// Unit tests for HostNetworkSplitter (the host/network split of the unified stats model's
// host+network stage design/stats-unification.md Phase 2): pts matching, the per-frame
// tiling arithmetic (network = combined host, floored at 0), drain/reset semantics, the
// bounded pending ring, and the absurd-receipt clamp. All samples use explicit instants, so
// the expectations are exact.
import Foundation
import XCTest
@testable import PunktfunkKit
final class HostNetworkSplitterTests: XCTestCase {
/// An arbitrary host-capture pts (ns) far from zero, like a real CLOCK_REALTIME stamp.
private let basePts: UInt64 = 1_000_000_000_000
private func receipt(_ s: HostNetworkSplitter, pts: UInt64, combinedMs: Int64,
offsetNs: Int64 = 0) {
s.recordReceipt(
ptsNs: pts, receivedNs: Int64(pts) + combinedMs * 1_000_000 - offsetNs,
offsetNs: offsetNs)
}
func testEmptyDrainIsNil() {
XCTAssertNil(HostNetworkSplitter().drain())
}
func testMatchSplitsCombinedIntoHostAndNetwork() {
let s = HostNetworkSplitter()
receipt(s, pts: basePts, combinedMs: 8) // capturereceived 8 ms
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000) // host says 3 ms of it was its own
guard let split = s.drain() else { return XCTFail("expected a matched sample") }
XCTAssertEqual(split.count, 1)
XCTAssertEqual(split.hostP50Ms, 3.0)
XCTAssertEqual(split.networkP50Ms, 5.0, "the two terms tile the combined interval")
XCTAssertNil(s.drain(), "drain resets the window")
}
func testSkewOffsetAppliesToTheCombinedInterval() {
let s = HostNetworkSplitter()
// Client clock 2 ms behind the host: the raw difference alone would read 6 ms.
receipt(s, pts: basePts, combinedMs: 8, offsetNs: 2_000_000)
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000)
XCTAssertEqual(s.drain()?.networkP50Ms, 5.0)
}
func testUnmatchedTimingIsSkipped() {
let s = HostNetworkSplitter()
receipt(s, pts: basePts, combinedMs: 8)
// A timing for an AU we never received (FEC-dropped) must not fabricate a sample.
s.noteHostTiming(ptsNs: basePts + 1, hostUs: 3_000)
XCTAssertNil(s.drain())
}
func testReceiptSurvivesADrainUntilItsTimingArrives() {
let s = HostNetworkSplitter()
receipt(s, pts: basePts, combinedMs: 8)
XCTAssertNil(s.drain(), "no timing matched yet")
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000) // arrives one tick late still matches
XCTAssertEqual(s.drain()?.hostP50Ms, 3.0)
}
func testEachReceiptMatchesOnce() {
let s = HostNetworkSplitter()
receipt(s, pts: basePts, combinedMs: 8)
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000)
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000) // duplicate 0xCF no second sample
XCTAssertEqual(s.drain()?.count, 1)
}
func testNetworkFlooredAtZero() {
let s = HostNetworkSplitter()
// A slightly-off skew offset can make host_us exceed the combined interval.
receipt(s, pts: basePts, combinedMs: 2)
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000)
guard let split = s.drain() else { return XCTFail("expected a sample") }
XCTAssertEqual(split.hostP50Ms, 3.0)
XCTAssertEqual(split.networkP50Ms, 0.0)
}
func testPendingRingDropsOldest() {
let s = HostNetworkSplitter()
for i in 0..<300 { // cap is 256 the first receipts fall out
receipt(s, pts: basePts + UInt64(i), combinedMs: 8)
}
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000) // evicted no match
XCTAssertNil(s.drain())
s.noteHostTiming(ptsNs: basePts + 299, hostUs: 3_000) // newest still pending
XCTAssertEqual(s.drain()?.count, 1)
}
func testAbsurdReceiptsAreDropped() {
let s = HostNetworkSplitter()
receipt(s, pts: basePts, combinedMs: -1) // received before capture clock step
receipt(s, pts: basePts + 1, combinedMs: 20_000) // > 10 s garbage pts/offset
s.noteHostTiming(ptsNs: basePts, hostUs: 1_000)
s.noteHostTiming(ptsNs: basePts + 1, hostUs: 1_000)
XCTAssertNil(s.drain())
}
func testResetForgetsPendingReceipts() {
let s = HostNetworkSplitter()
receipt(s, pts: basePts, combinedMs: 8)
s.reset()
s.noteHostTiming(ptsNs: basePts, hostUs: 3_000)
XCTAssertNil(s.drain(), "a fresh session must not match a previous session's receipts")
}
}
@@ -1,6 +1,10 @@
// Unit tests for LatencyMeter: percentiles, the skew-corrected flag, reset-on-drain, and the // Unit tests for LatencyMeter (one instance per unified-stats stage see
// absurd-value guard. Latencies are constructed by stamping a pts a known interval in the past, so // design/stats-unification.md): percentiles, the skew-corrected flag, reset-on-drain, the
// the result is that interval plus the (tiny) clock advance between reads asserted with tolerance. // absurd-value guard, and the explicit-instant stage form (record(ptsNs:atNs:offsetNs:), used for
// the client-local decode/display stages and the at-present end-to-end stamp). Receipt-path
// latencies are constructed by stamping a pts a known interval in the past, so the result is that
// interval plus the (tiny) clock advance between reads asserted with tolerance; the explicit
// form is exact.
import Foundation import Foundation
import XCTest import XCTest
@@ -38,6 +42,26 @@ final class LatencyMeterTests: XCTestCase {
XCTAssertEqual(m.drain()?.skewCorrected, true) XCTAssertEqual(m.drain()?.skewCorrected, true)
} }
func testExplicitStageRecordIsExact() {
let m = LatencyMeter()
// A client-local stage (decode: receiveddecoded) start instant as ptsNs, offset 0.
let receivedNs: Int64 = 1_000_000_000_000
m.record(ptsNs: UInt64(receivedNs), atNs: receivedNs + 3_000_000, offsetNs: 0)
guard let s = m.drain() else { return XCTFail("expected a sample") }
XCTAssertEqual(s.count, 1)
XCTAssertEqual(s.p50Ms, 3.0, "explicit instants make the sample exact")
XCTAssertFalse(s.skewCorrected, "local stages record with offset 0")
}
func testExplicitStageDropsNonPositiveInterval() {
let m = LatencyMeter()
// A stage whose start stamp is missing (0) or after its end must not pollute the window.
let decodedNs: Int64 = 1_000_000_000_000
m.record(ptsNs: 0, atNs: decodedNs, offsetNs: 0) // "start unknown" > 10 s dropped
m.record(ptsNs: UInt64(decodedNs + 1), atNs: decodedNs, offsetNs: 0) // negative dropped
XCTAssertNil(m.drain())
}
func testDropsAbsurdValues() { func testDropsAbsurdValues() {
let m = LatencyMeter() let m = LatencyMeter()
let now = nowRealtimeNs() let now = nowRealtimeNs()
@@ -25,12 +25,18 @@ final class LoopbackIntegrationTests: XCTestCase {
XCTAssertEqual(conn.resolvedBitrateKbps, 50_000) XCTAssertEqual(conn.resolvedBitrateKbps, 50_000)
// Pull 25 synthetic frames and byte-verify the documented pattern: // Pull 25 synthetic frames and byte-verify the documented pattern:
// u32 LE frame index, then data[i] = (idx as u8) &+ (i as u8). // u32 LE frame index, then data[i] = (idx as u8) &+ (i as u8). Alongside, drain the
// per-AU host-timing plane (0xCF) the way the app's stats tick does the connector
// ORs VIDEO_CAP_HOST_TIMING in unconditionally and the synthetic host stamps one
// report per AU, so the pts correlation must hold end to end through the xcframework.
var got = 0 var got = 0
var lastIndex: UInt32 = 0 var lastIndex: UInt32 = 0
var receivedPts = Set<UInt64>()
var timings: [PunktfunkConnection.HostTiming] = []
let deadline = Date().addingTimeInterval(30) let deadline = Date().addingTimeInterval(30)
while got < 25 { while got < 25 {
XCTAssertLessThan(Date(), deadline, "timed out after \(got) frames") XCTAssertLessThan(Date(), deadline, "timed out after \(got) frames")
while let t = try conn.nextHostTiming(timeoutMs: 0) { timings.append(t) }
guard let au = try conn.nextAU(timeoutMs: 2000) else { continue } guard let au = try conn.nextAU(timeoutMs: 2000) else { continue }
let idx = au.data.prefix(4).reversed().reduce(UInt32(0)) { ($0 << 8) | UInt32($1) } let idx = au.data.prefix(4).reversed().reduce(UInt32(0)) { ($0 << 8) | UInt32($1) }
for (i, byte) in au.data.enumerated().dropFirst(4) { for (i, byte) in au.data.enumerated().dropFirst(4) {
@@ -41,10 +47,22 @@ final class LoopbackIntegrationTests: XCTestCase {
} }
} }
XCTAssertGreaterThan(au.ptsNs, 0) XCTAssertGreaterThan(au.ptsNs, 0)
receivedPts.insert(au.ptsNs)
lastIndex = idx lastIndex = idx
got += 1 got += 1
} }
XCTAssertGreaterThanOrEqual(lastIndex, 24) XCTAssertGreaterThanOrEqual(lastIndex, 24)
// Belt-and-braces: the last frame's timing lands just after its AU give it a bounded
// grace drain (the stream keeps running, so this must not loop on fresh timings).
var grace = 0
while grace < 64, !timings.contains(where: { receivedPts.contains($0.ptsNs) }),
let t = try conn.nextHostTiming(timeoutMs: 100) {
timings.append(t)
grace += 1
}
XCTAssertTrue(
timings.contains { receivedPts.contains($0.ptsNs) },
"no 0xCF host timing matched a received AU's pts (got \(timings.count) timings)")
// Input goes the other way (enqueue-only; the host logs the count on close) // Input goes the other way (enqueue-only; the host logs the count on close)
// including the touch kinds, gamepad events, the rich-input plane (DualSense // including the touch kinds, gamepad events, the rich-input plane (DualSense
@@ -31,7 +31,7 @@ final class Stage444Tests: XCTestCase {
let data = Data(Probe444Blobs.au444_8bit) let data = Data(Probe444Blobs.au444_8bit)
let format = try XCTUnwrap( let format = try XCTUnwrap(
AnnexB.formatDescription(fromIDR: data, codec: .hevc), "the 4:4:4 blob must yield a format description") AnnexB.formatDescription(fromIDR: data, codec: .hevc), "the 4:4:4 blob must yield a format description")
let au = AccessUnit(data: data, ptsNs: 7_000_000, frameIndex: 0, flags: 0) let au = AccessUnit(data: data, ptsNs: 7_000_000, frameIndex: 0, flags: 0, receivedNs: 0)
let box = FrameBox() let box = FrameBox()
let done = DispatchSemaphore(value: 0) let done = DispatchSemaphore(value: 0)
@@ -38,7 +38,8 @@ final class VideoToolboxRoundTripTests: XCTestCase {
XCTAssertEqual(AnnexB.avcc(from: annexB, codec: .hevc), avccSample) XCTAssertEqual(AnnexB.avcc(from: annexB, codec: .hevc), avccSample)
// 3) Sample buffer real decoder pixels. // 3) Sample buffer real decoder pixels.
let au = AccessUnit(data: annexB, ptsNs: 1_000_000, frameIndex: 0, flags: 0) let au = AccessUnit(
data: annexB, ptsNs: 1_000_000, frameIndex: 0, flags: 0, receivedNs: 0)
let sample = try XCTUnwrap(AnnexB.sampleBuffer(au: au, format: rebuilt, codec: .hevc)) let sample = try XCTUnwrap(AnnexB.sampleBuffer(au: au, format: rebuilt, codec: .hevc))
var session: VTDecompressionSession? var session: VTDecompressionSession?
@@ -67,13 +68,14 @@ final class VideoToolboxRoundTripTests: XCTestCase {
} }
/// Stage-2 decode half: the same known IDR through `VideoDecoder` assert its async output /// Stage-2 decode half: the same known IDR through `VideoDecoder` assert its async output
/// callback fires with a CVPixelBuffer of the right dimensions, the pts round-trips, and /// callback fires with a CVPixelBuffer of the right dimensions, the pts and the receipt stamp
/// decode-completion is stamped. /// round-trip (the latter rides the frame refcon), and decode-completion is stamped.
func testVideoDecoderAsyncCallbackDeliversPixels() throws { func testVideoDecoderAsyncCallbackDeliversPixels() throws {
let (formatDesc, avccSample) = try encodeOneHEVCKeyframe() let (formatDesc, avccSample) = try encodeOneHEVCKeyframe()
let annexB = try annexBAU(formatDesc: formatDesc, avccSample: avccSample) let annexB = try annexBAU(formatDesc: formatDesc, avccSample: avccSample)
let format = try XCTUnwrap(AnnexB.formatDescription(fromIDR: annexB, codec: .hevc)) let format = try XCTUnwrap(AnnexB.formatDescription(fromIDR: annexB, codec: .hevc))
let au = AccessUnit(data: annexB, ptsNs: 42_000_000, frameIndex: 0, flags: 0) let au = AccessUnit(
data: annexB, ptsNs: 42_000_000, frameIndex: 0, flags: 0, receivedNs: 41_000_000)
let box = FrameBox() let box = FrameBox()
let done = DispatchSemaphore(value: 0) let done = DispatchSemaphore(value: 0)
@@ -100,6 +102,8 @@ final class VideoToolboxRoundTripTests: XCTestCase {
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), width) XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), width)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), height) XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), height)
XCTAssertEqual(ready.ptsNs, 42_000_000, "pts round-trips through the decoder") XCTAssertEqual(ready.ptsNs, 42_000_000, "pts round-trips through the decoder")
XCTAssertEqual(
ready.receivedNs, 41_000_000, "receivedNs round-trips through the frame refcon")
XCTAssertGreaterThan(ready.decodedNs, 0, "decode-completion is stamped") XCTAssertGreaterThan(ready.decodedNs, 0, "decode-completion is stamped")
} }
+15 -7
View File
@@ -17,10 +17,16 @@ the panel looks and feels native to Gaming Mode.
fingerprint to cross-check against the host's log). fingerprint to cross-check against the host's log).
2. **Pair** — for a host that requires it, a gamepad-navigable PIN keypad runs the SPAKE2 pairing 2. **Pair** — for a host that requires it, a gamepad-navigable PIN keypad runs the SPAKE2 pairing
ceremony headlessly, then remembers the host so future streams connect silently. ceremony headlessly, then remembers the host so future streams connect silently.
3. **Stream** — launches fullscreen via a hidden Steam shortcut so gamescope focuses it. 3. **Stream** — launches fullscreen via a branded "Punktfunk" Steam shortcut so gamescope focuses it.
4. **Settings**resolution / refresh / bitrate / gamepad type / host compositor / mic, written 4. **Games**each host row has a games button that opens its **library picker**: pin titles as
one-tap "Stream <Game>" rows in the QAM (jump straight into e.g. Playnite on the host), or
**"Open library on screen"** to launch the client's controller-driven, console-style library
browser (aurora backdrop + poster coverflow; A plays, B returns to Gaming Mode). Pins survive
plugin reinstalls (stored next to the client's config) and follow a host across IP changes
(matched by certificate fingerprint).
5. **Settings** — resolution / refresh / bitrate / gamepad type / host compositor / mic, written
to the client's config. to the client's config.
5. **About** — plugin version, an explicit "Check for updates" button, the setup-guide link, and 6. **About** — plugin version, an explicit "Check for updates" button, the setup-guide link, and
a force-stop for a wedged stream client. a force-stop for a wedged stream client.
To leave a stream: the in-client controller chord (**L1 + R1 + Start + Select**), or close the To leave a stream: the in-client controller chord (**L1 + R1 + Start + Select**), or close the
@@ -67,11 +73,13 @@ restart is required for an out-of-band install to appear.
| `src/index.tsx` | Plugin entry: the QAM panel + route registration. | | `src/index.tsx` | Plugin entry: the QAM panel + route registration. |
| `src/page.tsx` | The `/punktfunk` fullscreen page — Hosts (with per-host details) / Settings / About tabs. | | `src/page.tsx` | The `/punktfunk` fullscreen page — Hosts (with per-host details) / Settings / About tabs. |
| `src/settings.tsx` · `src/pair.tsx` | Stream-settings section; the gamepad-navigable PIN-pairing modal. | | `src/settings.tsx` · `src/pair.tsx` | Stream-settings section; the gamepad-navigable PIN-pairing modal. |
| `src/hooks.ts` · `src/boundary.tsx` | Shared discovery/update hooks + actions; the render error boundary. | | `src/library.tsx` | The per-host game picker (pin/unpin, "Open library on screen") + the pinned-game launch helper. |
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. The shortcut's exe is `/bin/sh` with the wrapper passed as an argument, so the script never needs an exec bit (Decky's zip extraction drops it and the root-owned plugins dir can't be chmodded by the unprivileged backend). | | `src/hooks.ts` · `src/boundary.tsx` | Shared discovery/update/pins hooks + actions; the render error boundary. |
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. The shortcut's exe is `/bin/sh` with the wrapper passed as an argument, so the script never needs an exec bit (Decky's zip extraction drops it and the root-owned plugins dir can't be chmodded by the unprivileged backend). Launch extras ride env-prefix tokens: `PF_LAUNCH=<id>` (pinned game) / `PF_BROWSE=1` + `PF_MGMT=<port>` (on-screen library); ids are validated space/quote-free at pin AND launch time. |
| `src/backend.ts` | Typed `callable` bridges to `main.py`. | | `src/backend.ts` | Typed `callable` bridges to `main.py`. |
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut runs (so the window is focusable). | | `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut runs (so the window is focusable); maps `PF_LAUNCH`/`PF_BROWSE`/`PF_MGMT` to `--launch`/`--browse`/`--mgmt`. An older flatpak ignores the flags harmlessly (plain stream / hosts page). |
| `main.py` | Backend: `discover` (via `avahi-browse`) / `pair` / settings / `kill_stream` / `check_update` (with an explicit CA-bundle search — Decky's embedded Python has no usable default TLS roots on SteamOS). | | `main.py` | Backend: `discover` (via `avahi-browse`) / `pair` / `library` (headless flatpak `--library`, TSV) / pins store (`decky-pinned.json`) / settings / `kill_stream` / `check_update` (with an explicit CA-bundle search — Decky's embedded Python has no usable default TLS roots on SteamOS). |
| `scripts/test-backend.py` | Stdlib-only checks for the backend's pure parsers (TSV, error classes, avahi TXT) + the pins round trip. |
| `plugin.json` · `update.json` | Decky manifest; CI-baked update channel. | | `plugin.json` · `update.json` | Decky manifest; CI-baked update channel. |
## Limitations / next steps ## Limitations / next steps
Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

+24 -2
View File
@@ -11,11 +11,19 @@
# #
# Per-session parameters arrive as environment variables, set as the shortcut's Steam launch # Per-session parameters arrive as environment variables, set as the shortcut's Steam launch
# options by the plugin (SteamClient.Apps.SetAppLaunchOptions), so ONE generic shortcut serves # options by the plugin (SteamClient.Apps.SetAppLaunchOptions), so ONE generic shortcut serves
# every host: # every host (and every pinned game):
# PF_HOST host[:port] to connect to (required) # PF_HOST host[:port] to connect to (required)
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_APPID flatpak app id (default io.unom.Punktfunk) # PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH) # PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
# #
# Values are plain tokens (the plugin validates launch ids to space/quote-free ASCII before
# they ever reach Steam launch options). An older flatpak without --launch/--browse ignores
# the unknown flags harmlessly (hand-scanned argv): PF_LAUNCH degrades to the plain desktop
# session, PF_BROWSE to the client's hosts page.
#
# Runs as the `deck` user (Steam launched it), so the --user flatpak install is visible and # Runs as the `deck` user (Steam launched it), so the --user flatpak install is visible and
# WAYLAND_DISPLAY / XDG_RUNTIME_DIR are already correct for gamescope. # WAYLAND_DISPLAY / XDG_RUNTIME_DIR are already correct for gamescope.
# #
@@ -33,9 +41,23 @@ if [ -z "${PF_HOST:-}" ]; then
exit 2 exit 2
fi fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
# exec so the flatpak client IS the game process — when it exits, Steam ends the "game" and # exec so the flatpak client IS the game process — when it exits, Steam ends the "game" and
# Gaming Mode reclaims focus automatically (no manual refocus needed). # Gaming Mode reclaims focus automatically (no manual refocus needed).
# --fullscreen: present the stream chrome-less and fullscreen (the client also auto-detects the # --fullscreen: present the stream chrome-less and fullscreen (the client also auto-detects the
# Deck/gamescope env, and ignores the flag harmlessly on older builds that predate it). # Deck/gamescope env, and ignores the flag harmlessly on older builds that predate it).
if [ -n "${PF_BROWSE:-}" ]; then
# The gamepad library launcher: browse the host's games on-screen, A streams one,
# session end returns to the launcher, B quits back to Gaming Mode.
echo "punktfunkrun: library $APPID --browse $PF_HOST" >&2
if [ -n "${PF_MGMT:-}" ]; then
exec "$FLATPAK" run --arch=x86_64 "$APPID" --browse "$PF_HOST" --mgmt "$PF_MGMT" --fullscreen
fi
exec "$FLATPAK" run --arch=x86_64 "$APPID" --browse "$PF_HOST" --fullscreen
fi
if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen
fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen
+306 -14
View File
@@ -12,6 +12,11 @@ The backend's jobs are the things Steam can't do:
* **pair(host, port, pin, name)** — run the SPAKE2 PIN ceremony headlessly via the flatpak * **pair(host, port, pin, name)** — run the SPAKE2 PIN ceremony headlessly via the flatpak
client's ``--pair`` mode, capturing the result. Pairing uses the SAME flatpak (so the same client's ``--pair`` mode, capturing the result. Pairing uses the SAME flatpak (so the same
identity store the stream uses), so once paired the stream connects silently. identity store the stream uses), so once paired the stream connects silently.
* **library(host, mgmt_port, fp)** — fetch a paired host's game library headlessly via the
flatpak client's ``--library`` mode (mTLS with the client's own identity; TSV on stdout),
so the picker UI can offer games to pin.
* **get_pins() / set_pins()** — the pinned-games store (``decky-pinned.json`` next to the
client's config, so pins survive plugin reinstalls), annotated with live pairing state.
* **runner_info()** — the absolute path to the launch wrapper + the flatpak app id, handed to * **runner_info()** — the absolute path to the launch wrapper + the flatpak app id, handed to
the frontend so it can create/point the Steam shortcut. the frontend so it can create/point the Steam shortcut.
* **get_settings() / set_settings()** — read/write the flatpak client's stream settings JSON * **get_settings() / set_settings()** — read/write the flatpak client's stream settings JSON
@@ -20,11 +25,12 @@ The backend's jobs are the things Steam can't do:
* **check_update()** — poll the registry's per-channel ``manifest.json`` and report whether a * **check_update()** — poll the registry's per-channel ``manifest.json`` and report whether a
newer build is available (the frontend then drives Decky's own install RPC to apply it). newer build is available (the frontend then drives Decky's own install RPC to apply it).
The TXT-record keys parsed (``proto`` / ``fp`` / ``pair`` / ``id``) are defined by the host The TXT-record keys parsed (``proto`` / ``fp`` / ``pair`` / ``id`` / ``mgmt``) are defined by
advert in ``crates/punktfunk-host/src/discovery.rs``. the host advert in ``crates/punktfunk-host/src/discovery.rs``.
""" """
import asyncio import asyncio
import base64
import json import json
import os import os
import shutil import shutil
@@ -76,6 +82,46 @@ def _runner_path() -> str:
return str(Path(decky.DECKY_PLUGIN_DIR) / "bin" / "punktfunkrun.sh") return str(Path(decky.DECKY_PLUGIN_DIR) / "bin" / "punktfunkrun.sh")
def _pins_path() -> Path:
"""The pinned-games store — plugin-owned, but deliberately in the CLIENT's config dir
(like everything else we persist): the plugins dir is root-owned and wiped on
reinstall, while ``~/.config/punktfunk`` survives both."""
return _client_config_dir() / "decky-pinned.json"
def _parse_library_tsv(stdout: str) -> list[dict]:
"""Parse the flatpak client's ``--library`` output: one ``id\\tstore\\ttitle`` line per
game plus a trailing ``N game(s)`` count line (no tabs — it self-skips here). A title
may itself contain tabs, so split at most twice."""
games: list[dict] = []
for line in stdout.splitlines():
parts = line.split("\t", 2)
if len(parts) == 3:
games.append({"id": parts[0], "store": parts[1], "title": parts[2]})
return games
def _classify_library_error(stderr: str) -> str:
"""Map the client's ``library: <LibraryError Display>`` stderr line to a stable error
code for the UI. Substring-matched against the Display strings in
``clients/linux/src/library.rs`` — a wording change degrades to ``client-error``
(generic copy), never a crash."""
s = stderr.lower()
if "didn't recognize this device" in s:
return "not-paired"
if "pinned fingerprint" in s:
return "pin-mismatch"
if "couldn't reach the host" in s:
return "unreachable"
if "management api returned http" in s:
return "http"
if "display" in s or "gtk" in s:
# A flatpak so old it predates --library falls through to GTK init, which fails
# headless from this backend.
return "client-outdated"
return "client-error"
# ---------------------------------------------------------------------------------------- # ----------------------------------------------------------------------------------------
# Self-update check (no Decky store). The plugin is distributed via "Install Plugin from # Self-update check (no Decky store). The plugin is distributed via "Install Plugin from
# URL" pointing at our Gitea generic registry, so the official store never sees it and # URL" pointing at our Gitea generic registry, so the official store never sees it and
@@ -224,6 +270,71 @@ def _flatpak_env() -> dict:
return env return env
async def _flatpak_capture(args: list[str], timeout: float = 20.0) -> tuple[int, str]:
"""Run ``flatpak <args>`` with the user-session env, merging stderr into stdout. Returns
``(returncode, output)``; ``(-1, "")`` if the binary is missing or the call errors/times out.
Best-effort by design — every caller here treats a failure as "no update / can't tell"."""
flatpak = _flatpak()
if not flatpak:
return -1, ""
proc = None
try:
proc = await asyncio.create_subprocess_exec(
flatpak, *args,
stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.STDOUT,
env=_flatpak_env(),
)
out, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout)
rc = proc.returncode if proc.returncode is not None else -1
return rc, (out or b"").decode("utf-8", "replace")
except asyncio.TimeoutError:
decky.logger.warning("flatpak %s timed out", " ".join(args))
if proc:
try:
proc.kill()
except ProcessLookupError:
pass
return -1, ""
except Exception: # noqa: BLE001
decky.logger.exception("flatpak %s failed", " ".join(args))
return -1, ""
def _field_from(text: str, name: str) -> str:
"""Pull ``<name>: value`` out of ``flatpak info`` / ``remote-info`` output (e.g. ``Commit``,
``Origin``)."""
prefix = f"{name}:"
for line in text.splitlines():
s = line.strip()
if s.startswith(prefix):
return s.split(":", 1)[1].strip()
return ""
async def _client_update_state() -> dict:
"""Is a newer commit of the flatpak client available in the remote it tracks? The client is a
**per-user** install (so ``sudo flatpak update``, which is system-scope, never touches it), and
it versions independently of this plugin — so we compare the installed commit against the
remote's here and let the QAM offer a user-scope update. Best-effort; all-``False`` on any error
(not installed, no flatpak, offline)."""
state = {"available": False, "installed": "", "remote": ""}
rc, info = await _flatpak_capture(["info", "--user", APP_ID], timeout=10.0)
if rc != 0:
return state # client not installed as a user app / no flatpak
state["installed"] = _field_from(info, "Commit")
origin = _field_from(info, "Origin")
if not origin:
return state
rc, rinfo = await _flatpak_capture(["remote-info", "--user", origin, APP_ID], timeout=25.0)
if rc != 0:
return state # remote unreachable — treat as "up to date", retry next check
state["remote"] = _field_from(rinfo, "Commit")
state["available"] = bool(
state["installed"] and state["remote"] and state["installed"] != state["remote"]
)
return state
def _split_txt(txt: str) -> list[str]: def _split_txt(txt: str) -> list[str]:
"""Split an avahi TXT column into tokens, honouring the ``"key=value"`` quoting.""" """Split an avahi TXT column into tokens, honouring the ``"key=value"`` quoting."""
tokens: list[str] = [] tokens: list[str] = []
@@ -273,6 +384,11 @@ def _parse_avahi_browse(stdout: str) -> list[dict]:
if props.get("proto") and not props["proto"].startswith("punktfunk/"): if props.get("proto") and not props["proto"].startswith("punktfunk/"):
continue continue
try:
mgmt = int(props.get("mgmt", ""))
except ValueError:
mgmt = 0 # not advertised (standalone punktfunk1-host) — callers default 47990
entry = { entry = {
"name": name, "name": name,
"host": address, "host": address,
@@ -280,6 +396,8 @@ def _parse_avahi_browse(stdout: str) -> list[dict]:
"pair": props.get("pair", "optional"), "pair": props.get("pair", "optional"),
"fp": props.get("fp", ""), "fp": props.get("fp", ""),
"proto": props.get("proto", ""), "proto": props.get("proto", ""),
"id": props.get("id", ""),
"mgmt": mgmt,
} }
key = props.get("id") or f"{address}:{port}" key = props.get("id") or f"{address}:{port}"
existing = out.get(key) existing = out.get(key)
@@ -371,6 +489,136 @@ class Plugin:
reason = (err.strip().splitlines() or out.strip().splitlines() or ["pairing failed"])[-1] reason = (err.strip().splitlines() or out.strip().splitlines() or ["pairing failed"])[-1]
return {"ok": False, "error": reason} return {"ok": False, "error": reason}
async def library(self, host: str, mgmt_port: int = 0, fp: str = "") -> dict:
"""Fetch a paired host's game library via the flatpak client's headless
``--library`` mode (the client's own mTLS identity + pinned-fingerprint transport —
no trust logic reimplemented here). ``fp`` is passed through whenever the caller
knows the host's cert fingerprint so an IP change can never degrade the pin to a
TOFU accept. Returns ``{ok, games: [{id, store, title}]}`` or
``{ok: False, error: <code>, detail}`` (codes: ``flatpak-not-found`` / ``timeout`` /
``not-paired`` / ``pin-mismatch`` / ``unreachable`` / ``http`` /
``client-outdated`` / ``client-error``)."""
flatpak = _flatpak()
if not flatpak:
return {"ok": False, "error": "flatpak-not-found", "detail": ""}
target = f"{host}:{int(mgmt_port) or 47990}"
argv = [flatpak, "run", "--arch=x86_64", APP_ID, "--library", target]
if fp:
argv += ["--fp", fp]
decky.logger.info("library: fetching %s", target)
proc = None
try:
# Separate pipes (unlike _flatpak_capture): the TSV comes on stdout, the
# client's one-line error reason on stderr. Cold flatpak start on a Deck can
# take seconds — generous timeout, spinner in the UI.
proc = await asyncio.create_subprocess_exec(
*argv,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=_flatpak_env(),
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=45.0)
except asyncio.TimeoutError:
if proc:
try:
proc.kill()
except ProcessLookupError:
pass
return {"ok": False, "error": "timeout", "detail": ""}
except Exception as exc: # noqa: BLE001
decky.logger.exception("library fetch failed to launch")
return {"ok": False, "error": "client-error", "detail": str(exc)}
err = stderr.decode(errors="replace")
if proc.returncode != 0:
detail = (err.strip().splitlines() or ["library fetch failed"])[-1]
code = _classify_library_error(err)
decky.logger.warning("library fetch failed (%s): %s", code, detail)
return {"ok": False, "error": code, "detail": detail}
games = _parse_library_tsv(stdout.decode(errors="replace"))
decky.logger.info("library: %d game(s) from %s", len(games), target)
return {"ok": True, "games": games}
async def get_pins(self) -> dict:
"""The pinned games, each annotated with the LIVE ``paired`` state of its host (by
cert fingerprint — an unpaired-since host renders "pairing required" in the QAM)."""
try:
data = json.loads(_pins_path().read_text())
except (OSError, json.JSONDecodeError):
return {"pins": []}
pins = data.get("pins", []) if isinstance(data, dict) else []
paired = _paired_fingerprints()
out = []
for p in pins:
if not isinstance(p, dict) or not p.get("game_id"):
continue
p = dict(p)
p["paired"] = str(p.get("host_fp", "")).lower() in paired
out.append(p)
return {"pins": out}
async def set_pins(self, pins: list) -> dict:
"""Persist the pinned-games list (the frontend sends the whole list — add, remove,
and address-refresh all funnel through here). Validated + deduped on
``(host_fp, game_id)``; written atomically (tmp + rename) — pins are long-lived
user data."""
clean: list[dict] = []
seen: set[tuple[str, str]] = set()
for p in pins if isinstance(pins, list) else []:
if not isinstance(p, dict):
continue
game_id = str(p.get("game_id", ""))
host_fp = str(p.get("host_fp", ""))
if not game_id or not (host_fp or p.get("host")):
continue
key = (host_fp, game_id)
if key in seen:
continue
seen.add(key)
clean.append({
"game_id": game_id,
"title": str(p.get("title", game_id)),
"store": str(p.get("store", "")),
"host_fp": host_fp,
"host_id": str(p.get("host_id", "")),
"host_name": str(p.get("host_name", p.get("host", ""))),
"host": str(p.get("host", "")),
"port": int(p.get("port", 9777) or 9777),
"mgmt": int(p.get("mgmt", 0) or 0),
"added_at": int(p.get("added_at", 0) or 0),
})
try:
d = _client_config_dir()
d.mkdir(parents=True, exist_ok=True)
tmp = _pins_path().with_suffix(".json.tmp")
tmp.write_text(json.dumps({"version": 1, "pins": clean}, indent=2))
os.replace(tmp, _pins_path())
return {"ok": True}
except OSError as exc:
decky.logger.exception("could not write pins")
return {"ok": False, "error": str(exc)}
async def shortcut_art(self) -> dict:
"""The Steam-shortcut artwork shipped with the plugin (``assets/``, generated by
``scripts/gen-steam-art.py``): base64 PNGs for SetCustomArtworkForApp plus the
icon's absolute path for SetShortcutIcon (which wants a file, not bytes). Missing
files are simply omitted — artwork is cosmetic and must never block a launch."""
art: dict = {}
base = Path(decky.DECKY_PLUGIN_DIR) / "assets"
for key, fname in (
("grid", "grid.png"),
("gridwide", "gridwide.png"),
("hero", "hero.png"),
("logo", "logo.png"),
):
try:
art[key] = base64.b64encode((base / fname).read_bytes()).decode()
except OSError:
pass
icon = base / "icon.png"
art["icon_path"] = str(icon) if icon.exists() else ""
return art
async def runner_info(self) -> dict: async def runner_info(self) -> dict:
"""The wrapper-script path + flatpak app id the frontend needs to create the Steam """The wrapper-script path + flatpak app id the frontend needs to create the Steam
shortcut. The shortcut invokes the script through ``/bin/sh`` (see steam.ts), so no shortcut. The shortcut invokes the script through ``/bin/sh`` (see steam.ts), so no
@@ -419,11 +667,37 @@ class Plugin:
return {"ok": False} return {"ok": False}
return {"ok": True} return {"ok": True}
async def update_client(self) -> dict:
"""Update the flatpak **client** (io.unom.Punktfunk) in the USER installation — the scope a
Steam Deck install lives in, which ``sudo flatpak update`` (system-scope) never reaches.
Returns whether a new commit was actually pulled. Best-effort; non-fatal."""
flatpak = _flatpak()
if not flatpak:
return {"ok": False, "updated": False, "error": "flatpak-not-found"}
_, before = await _flatpak_capture(["info", "--user", APP_ID], timeout=10.0)
before_commit = _field_from(before, "Commit")
rc, out = await _flatpak_capture(["update", "--user", "-y", APP_ID], timeout=300.0)
if rc != 0:
decky.logger.warning("flatpak client update failed (rc=%s): %s", rc, out[-400:])
return {"ok": False, "updated": False, "error": "update-failed"}
_, after = await _flatpak_capture(["info", "--user", APP_ID], timeout=10.0)
after_commit = _field_from(after, "Commit")
updated = bool(before_commit and after_commit and before_commit != after_commit)
decky.logger.info(
"flatpak client update: %s -> %s (updated=%s)",
before_commit[:10], after_commit[:10], updated,
)
_update_cache["data"] = None # invalidate the cached "update available" snapshot
return {"ok": True, "updated": updated}
async def check_update(self, force: bool = False) -> dict: async def check_update(self, force: bool = False) -> dict:
"""Is a newer build available in our registry? Compares the installed version """Report pending updates for BOTH the plugin and the flatpak client.
(``package.json``) against the per-channel ``manifest.json`` the CI publishes, and
returns everything the frontend needs to drive Decky's install RPC. Non-fatal: any The plugin updates via Decky's install RPC (the per-channel ``manifest.json`` the CI
failure (no channel baked in, network down) returns ``update_available: False``. publishes); the **client** updates via ``flatpak update --user`` (a per-user install, so
``sudo flatpak update`` — system-scope — never touches it) and versions independently, so
it's checked here too and applied through :meth:`update_client`. Non-fatal: any failure
leaves the respective ``*_update_available`` ``False``.
""" """
current = _installed_version() current = _installed_version()
cfg = _update_config() cfg = _update_config()
@@ -434,23 +708,37 @@ class Plugin:
"hash": "", "hash": "",
"channel": str(cfg.get("channel", "")), "channel": str(cfg.get("channel", "")),
"update_available": False, "update_available": False,
"client_update_available": False,
"client_current": "",
"client_latest": "",
} }
manifest_url = cfg.get("manifest")
if not manifest_url:
result["error"] = "update-channel-unknown" # dev / sideloaded build
return result
now = time.monotonic() now = time.monotonic()
cached = _update_cache["data"] cached = _update_cache["data"]
if not force and cached and (now - _update_cache["at"]) < _UPDATE_TTL_S: if not force and cached and (now - _update_cache["at"]) < _UPDATE_TTL_S:
return cached return cached
# Client (flatpak) update — checked ALWAYS, even on a dev/sideloaded plugin build.
try:
cu = await _client_update_state()
result["client_update_available"] = bool(cu["available"])
result["client_current"] = (cu["installed"] or "")[:10]
result["client_latest"] = (cu["remote"] or "")[:10]
except Exception: # noqa: BLE001
decky.logger.warning("client update check failed", exc_info=True)
manifest_url = cfg.get("manifest")
if not manifest_url:
result["error"] = "update-channel-unknown" # dev / sideloaded plugin build
_update_cache["at"] = now
_update_cache["data"] = result # the client info is still valid to cache
return result
try: try:
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
manifest = await loop.run_in_executor(None, _fetch_json, manifest_url) manifest = await loop.run_in_executor(None, _fetch_json, manifest_url)
except Exception as exc: # noqa: BLE001 except Exception as exc: # noqa: BLE001
decky.logger.warning("update check failed: %s", exc) decky.logger.warning("plugin update check failed: %s", exc)
result["error"] = "fetch-failed" result["error"] = "fetch-failed"
return result # transient — don't cache, retry next open return result # transient — don't cache, retry next open
@@ -461,8 +749,12 @@ class Plugin:
result["update_available"] = bool(result["artifact"]) and ( result["update_available"] = bool(result["artifact"]) and (
_semver_tuple(latest) > _semver_tuple(current) _semver_tuple(latest) > _semver_tuple(current)
) )
if result["update_available"]: if result["update_available"] or result["client_update_available"]:
decky.logger.info("update available: %s -> %s (%s)", current, latest, result["channel"]) decky.logger.info(
"updates: plugin %s->%s (avail=%s), client->%s (avail=%s)",
current, latest, result["update_available"],
result["client_latest"], result["client_update_available"],
)
_update_cache["at"] = now _update_cache["at"] = now
_update_cache["data"] = result _update_cache["data"] = result
return result return result
+297
View File
@@ -0,0 +1,297 @@
#!/usr/bin/env python3
"""Generate the Steam-shortcut artwork for the Decky plugin (committed, like the tray icons).
The plugin registers a non-Steam shortcut ("Punktfunk") whose grid/hero/logo/icon Steam
would otherwise render as a gray placeholder tile. These assets brand it: the lens mark
(same geometry as scripts/gen-tray-icons.py / web's brand-mark.tsx) over the brand-navy
gradient, plus a monoline "punktfunk" wordmark built from stroke segments ("punktfunk"
needs only p·u·n·k·t·f). The frontend applies them via
SteamClient.Apps.SetCustomArtworkForApp / SetShortcutIcon (src/steam.ts).
Outputs (checked in; re-run only when the brand changes):
clients/decky/assets/grid.png 600 x 900 library capsule (portrait)
clients/decky/assets/gridwide.png 920 x 430 wide capsule (recent games / search)
clients/decky/assets/hero.png 1920 x 620 game-page banner
clients/decky/assets/logo.png transparent overlaid on the hero by Steam
clients/decky/assets/icon.png 256 x 256 list icon (SetShortcutIcon)
Pure stdlib. Unlike the tiny tray icons this rasterizes big surfaces, so edges are
antialiased analytically from signed distances (one sample per pixel) instead of 4x4
supersampling.
"""
import math
import struct
import zlib
from pathlib import Path
HERE = Path(__file__).resolve().parent.parent # clients/decky
OUT = HERE / "assets"
# Brand-mark geometry in its 1000-unit viewbox (identical to gen-tray-icons.py).
R = 194.41
C1 = (403.037, 597.262) # light circle, behind
C2 = (597.8075, 402.8525) # deep circle, in front
BB_MIN = (C1[0] - R, C2[1] - R)
BB_MAX = (C2[0] + R, C1[1] + R)
MARK_CENTER = ((BB_MIN[0] + BB_MAX[0]) / 2, (BB_MIN[1] + BB_MAX[1]) / 2)
MARK_SPAN = BB_MAX[0] - BB_MIN[0]
COL_LIGHT = (0xA7, 0x9F, 0xF8)
COL_DEEP = (0x6C, 0x5B, 0xF3)
COL_HI = (0xD2, 0xC9, 0xFB)
WORD = (0xEF, 0xEC, 0xFD) # wordmark: near-white lavender
BG_TOP = (0x28, 0x1E, 0x46)
BG_BOT = (0x12, 0x0D, 0x22)
# ------------------------------------------------------------------------------------------
# Wordmark: monoline glyphs as polylines in a unit box (y down; x-height top y=0, baseline
# y=1, ascender to -0.5, descender to +1.5). Arcs are sampled into the polylines, so the
# rasterizer only ever measures distance-to-segment; round caps/joins fall out of that.
# ------------------------------------------------------------------------------------------
def _arc(cx, cy, r, a0, a1, n=24):
"""Polyline along a circle arc; degrees, 0 = +x, angles grow clockwise on screen."""
pts = []
for i in range(n + 1):
a = math.radians(a0 + (a1 - a0) * i / n)
pts.append((cx + r * math.cos(a), cy + r * math.sin(a)))
return pts
GLYPHS = {
# letter: (advance, [polyline, ...])
"p": (1.05, [[(0, 0), (0, 1.5)], _arc(0.5, 0.5, 0.5, 0, 360)]),
"u": (1.05, [[(0, 0), (0, 0.5)], _arc(0.5, 0.5, 0.5, 0, 180), [(1, 0), (1, 0.5)]]),
"n": (1.05, [[(0, 0), (0, 1)], _arc(0.5, 0.5, 0.5, 180, 360), [(1, 0.5), (1, 1)]]),
"k": (1.0, [[(0, -0.5), (0, 1)], [(0, 0.62), (0.78, 0)], [(0.30, 0.38), (0.85, 1)]]),
"t": (0.85, [[(0.42, -0.42), (0.42, 1)], [(0, 0), (0.84, 0)]]),
"f": (
0.85,
[[(0.42, 1), (0.42, -0.15)] + _arc(0.75, -0.15, 0.33, 180, 270, 12), [(0, 0), (0.78, 0)]],
),
}
GAP = 0.34 # inter-letter gap, in glyph units
STROKE = 0.26 # stroke thickness, in glyph units
ASCENT, DESCENT = -0.5, 1.5 # glyph-space vertical extent
def word_segments(text):
"""The word's stroke segments [(x1,y1,x2,y2)] in glyph units, plus its unit width."""
segs = []
x = 0.0
for ch in text:
adv, lines = GLYPHS[ch]
for line in lines:
for (x1, y1), (x2, y2) in zip(line, line[1:]):
segs.append((x + x1, y1, x + x2, y2))
x += adv + GAP
return segs, x - GAP
def render_word_alpha(text, unit_px):
"""Coverage (0..255) buffer of the word at `unit_px` pixels per glyph unit."""
segs, width_u = word_segments(text)
half = STROKE / 2 * unit_px
pad = half + 1.5
w = math.ceil(width_u * unit_px + 2 * pad)
h = math.ceil((DESCENT - ASCENT) * unit_px + 2 * pad)
ox, oy = pad, pad - ASCENT * unit_px
px_segs = [(ox + a * unit_px, oy + b * unit_px, ox + c * unit_px, oy + d * unit_px) for a, b, c, d in segs]
# Bucket segments per pixel column range so each pixel tests only nearby strokes.
buf = bytearray(w * h)
for x1, y1, x2, y2 in px_segs:
lo_x = max(0, math.floor(min(x1, x2) - pad))
hi_x = min(w, math.ceil(max(x1, x2) + pad))
lo_y = max(0, math.floor(min(y1, y2) - pad))
hi_y = min(h, math.ceil(max(y1, y2) + pad))
dx, dy = x2 - x1, y2 - y1
len2 = dx * dx + dy * dy
for py in range(lo_y, hi_y):
row = py * w
fy = py + 0.5
for px in range(lo_x, hi_x):
fx = px + 0.5
if len2 > 0:
t = max(0.0, min(1.0, ((fx - x1) * dx + (fy - y1) * dy) / len2))
else:
t = 0.0
d = math.hypot(fx - (x1 + t * dx), fy - (y1 + t * dy))
cov = 0.5 + (half - d)
if cov > 0:
v = min(255, round(min(1.0, cov) * 255))
if v > buf[row + px]:
buf[row + px] = v
return buf, w, h
# ------------------------------------------------------------------------------------------
# Canvas: RGBA bytearray, straight alpha, painted back to front.
# ------------------------------------------------------------------------------------------
class Canvas:
def __init__(self, w, h):
self.w, self.h = w, h
self.buf = bytearray(w * h * 4)
def fill_gradient(self, top, bottom):
for y in range(self.h):
t = y / max(1, self.h - 1)
c = bytes(
(
round(top[0] + (bottom[0] - top[0]) * t),
round(top[1] + (bottom[1] - top[1]) * t),
round(top[2] + (bottom[2] - top[2]) * t),
255,
)
)
self.buf[y * self.w * 4 : (y + 1) * self.w * 4] = c * self.w
def _blend(self, i, rgb, a):
"""`rgb` over the pixel at byte offset i with coverage a (0..1)."""
if a <= 0:
return
b = self.buf
ia = 1.0 - a
da = b[i + 3] / 255.0
oa = a + da * ia
if oa <= 0:
return
for k in range(3):
b[i + k] = round((rgb[k] * a + b[i + k] * da * ia) / oa)
b[i + 3] = round(oa * 255)
def glow(self, cx, cy, radius, rgb, strength):
"""Soft gaussian-ish radial glow (for the mark's halo on the big surfaces)."""
lo_x = max(0, math.floor(cx - 2.2 * radius))
hi_x = min(self.w, math.ceil(cx + 2.2 * radius))
lo_y = max(0, math.floor(cy - 2.2 * radius))
hi_y = min(self.h, math.ceil(cy + 2.2 * radius))
for y in range(lo_y, hi_y):
for x in range(lo_x, hi_x):
d2 = ((x + 0.5 - cx) ** 2 + (y + 0.5 - cy) ** 2) / (radius * radius)
a = strength * math.exp(-2.5 * d2)
if a > 1 / 255:
self._blend((y * self.w + x) * 4, rgb, a)
def mark(self, cx, cy, span):
"""The lens mark centered at (cx, cy) with the given pixel span."""
scale = span / MARK_SPAN
c1 = (cx + (C1[0] - MARK_CENTER[0]) * scale, cy + (C1[1] - MARK_CENTER[1]) * scale)
c2 = (cx + (C2[0] - MARK_CENTER[0]) * scale, cy + (C2[1] - MARK_CENTER[1]) * scale)
r = R * scale
lo_x = max(0, math.floor(min(c1[0], c2[0]) - r - 2))
hi_x = min(self.w, math.ceil(max(c1[0], c2[0]) + r + 2))
lo_y = max(0, math.floor(min(c1[1], c2[1]) - r - 2))
hi_y = min(self.h, math.ceil(max(c1[1], c2[1]) + r + 2))
for y in range(lo_y, hi_y):
for x in range(lo_x, hi_x):
fx, fy = x + 0.5, y + 0.5
cov1 = min(1.0, max(0.0, 0.5 + r - math.hypot(fx - c1[0], fy - c1[1])))
cov2 = min(1.0, max(0.0, 0.5 + r - math.hypot(fx - c2[0], fy - c2[1])))
if cov1 <= 0 and cov2 <= 0:
continue
i = (y * self.w + x) * 4
self._blend(i, COL_LIGHT, cov1)
self._blend(i, COL_DEEP, cov2)
self._blend(i, COL_HI, min(cov1, cov2))
def word(self, text, unit_px, cx, cy):
"""The wordmark centered at (cx, cy); `unit_px` = pixels per glyph unit."""
alpha, w, h = render_word_alpha(text, unit_px)
ox = round(cx - w / 2)
# Optical vertical centering on the x-height band (0..1 in glyph units), not the
# ascender/descender box — the word reads centered that way.
pad = STROKE / 2 * unit_px + 1.5
band_mid = pad - ASCENT * unit_px + 0.5 * unit_px
oy = round(cy - band_mid)
for y in range(h):
ty = y + oy
if not 0 <= ty < self.h:
continue
for x in range(w):
a = alpha[y * w + x]
if a:
tx = x + ox
if 0 <= tx < self.w:
self._blend((ty * self.w + tx) * 4, WORD, a / 255.0)
def round_corners(self, radius):
"""Multiply alpha with a rounded-rect mask (icon)."""
for y in range(self.h):
for x in range(self.w):
dx = max(0.0, max(radius - (x + 0.5), (x + 0.5) - (self.w - radius)))
dy = max(0.0, max(radius - (y + 0.5), (y + 0.5) - (self.h - radius)))
if dx > 0 and dy > 0:
cov = min(1.0, max(0.0, 0.5 + radius - math.hypot(dx, dy)))
i = (y * self.w + x) * 4
self.buf[i + 3] = round(self.buf[i + 3] * cov)
def png(self):
def chunk(tag, data):
return (
struct.pack(">I", len(data))
+ tag
+ data
+ struct.pack(">I", zlib.crc32(tag + data) & 0xFFFFFFFF)
)
ihdr = struct.pack(">IIBBBBB", self.w, self.h, 8, 6, 0, 0, 0)
raw = b"".join(
b"\x00" + bytes(self.buf[y * self.w * 4 : (y + 1) * self.w * 4]) for y in range(self.h)
)
return (
b"\x89PNG\r\n\x1a\n"
+ chunk(b"IHDR", ihdr)
+ chunk(b"IDAT", zlib.compress(raw, 9))
+ chunk(b"IEND", b"")
)
def save(name, canvas):
OUT.mkdir(parents=True, exist_ok=True)
out = OUT / name
out.write_bytes(canvas.png())
print(f"wrote {out.relative_to(HERE.parent.parent)} ({canvas.w}x{canvas.h})")
def main():
# Portrait capsule: mark in the upper half, wordmark beneath.
c = Canvas(600, 900)
c.fill_gradient(BG_TOP, BG_BOT)
c.glow(300, 340, 260, COL_DEEP, 0.35)
c.mark(300, 340, 320)
c.word("punktfunk", 44, 300, 640)
save("grid.png", c)
# Wide capsule: mark left, wordmark right of it.
c = Canvas(920, 430)
c.fill_gradient(BG_TOP, BG_BOT)
c.glow(230, 215, 200, COL_DEEP, 0.35)
c.mark(230, 215, 240)
c.word("punktfunk", 40, 620, 220)
save("gridwide.png", c)
# Hero: ambient banner — the mark rides the right third; Steam overlays logo.png itself.
c = Canvas(1920, 620)
c.fill_gradient(BG_TOP, BG_BOT)
c.glow(1500, 310, 330, COL_DEEP, 0.4)
c.mark(1500, 310, 400)
save("hero.png", c)
# Logo (transparent): mark + wordmark side by side, overlaid on the hero by Steam.
c = Canvas(1120, 300)
c.mark(150, 150, 240)
c.word("punktfunk", 62, 660, 155)
save("logo.png", c)
# Icon: brand tile, rounded corners, mark only.
c = Canvas(256, 256)
c.fill_gradient(BG_TOP, BG_BOT)
c.glow(128, 128, 110, COL_DEEP, 0.3)
c.mark(128, 128, 190)
c.round_corners(36)
save("icon.png", c)
if __name__ == "__main__":
main()
+3 -1
View File
@@ -20,12 +20,14 @@ VER="$(python3 -c 'import json;print(json.load(open("package.json"))["version"])
STAGE="$(mktemp -d)" STAGE="$(mktemp -d)"
DEST="$STAGE/$NAME" DEST="$STAGE/$NAME"
mkdir -p "$DEST/dist" "$DEST/bin" mkdir -p "$DEST/dist" "$DEST/bin" "$DEST/assets"
cp dist/index.js "$DEST/dist/index.js" # ship the bundle only, not the sourcemap cp dist/index.js "$DEST/dist/index.js" # ship the bundle only, not the sourcemap
cp main.py plugin.json package.json LICENSE "$DEST/" cp main.py plugin.json package.json LICENSE "$DEST/"
# The stream-launch wrapper (target of the Steam shortcut) — must stay executable. # The stream-launch wrapper (target of the Steam shortcut) — must stay executable.
cp bin/punktfunkrun.sh "$DEST/bin/punktfunkrun.sh" cp bin/punktfunkrun.sh "$DEST/bin/punktfunkrun.sh"
chmod 0755 "$DEST/bin/punktfunkrun.sh" chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Steam-shortcut artwork (grid/hero/logo/icon — scripts/gen-steam-art.py, committed).
cp assets/*.png "$DEST/assets/"
[ -f decky.pyi ] && cp decky.pyi "$DEST/" [ -f decky.pyi ] && cp decky.pyi "$DEST/"
[ -f README.md ] && cp README.md "$DEST/" [ -f README.md ] && cp README.md "$DEST/"
+151
View File
@@ -0,0 +1,151 @@
#!/usr/bin/env python3
"""Unit checks for main.py's pure helpers — stdlib only, no Decky runtime needed.
Stubs the ``decky`` module (main.py imports it at module level), then asserts the
avahi/TSV/error parsers against fixture strings. The LibraryError fixtures are pinned to
the REAL Display strings in clients/linux/src/library.rs — if those are reworded, the
classifier degrades to ``client-error`` and the matching assertion here fails on purpose.
python3 clients/decky/scripts/test-backend.py
"""
import sys
import types
from pathlib import Path
# ---- stub the decky module before importing main.py ------------------------------------
decky = types.ModuleType("decky")
decky.DECKY_USER_HOME = "/tmp/pf-test-home"
decky.DECKY_PLUGIN_DIR = "/tmp/pf-test-plugin"
class _Log:
def __getattr__(self, _name):
return lambda *a, **k: None
decky.logger = _Log()
sys.modules["decky"] = decky
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
import main # noqa: E402 (the plugin backend)
failures = 0
def check(name: str, cond: bool):
global failures
print(("ok " if cond else "FAIL") + " " + name)
if not cond:
failures += 1
# ---- _parse_library_tsv -----------------------------------------------------------------
tsv = (
"steam:570\tsteam\tDota 2\n"
"custom:abc\tcustom\tTabs\tin\ttitle\n" # tabs inside the title survive (split max 2)
"2 game(s)\n" # the count trailer has no tabs — self-skips
)
games = main._parse_library_tsv(tsv)
check("tsv: two games parsed", len(games) == 2)
check("tsv: fields", games[0] == {"id": "steam:570", "store": "steam", "title": "Dota 2"})
check("tsv: tabs in title preserved", games[1]["title"] == "Tabs\tin\ttitle")
check("tsv: empty input", main._parse_library_tsv("0 game(s)\n") == [])
# ---- _classify_library_error (fixtures = library.rs Display strings) --------------------
check(
"err: not-paired",
main._classify_library_error(
"library: The host didn't recognize this device. Pair with the host first — the "
"library is authorized by this device's certificate (no token needed)."
)
== "not-paired",
)
check(
"err: pin-mismatch",
main._classify_library_error(
"library: The host's certificate doesn't match the pinned fingerprint. "
"Re-pair with a PIN to re-establish trust."
)
== "pin-mismatch",
)
check(
"err: unreachable",
main._classify_library_error(
"library: Couldn't reach the host's management API: connection refused. Check the "
"host is updated and reachable."
)
== "unreachable",
)
check(
"err: http",
main._classify_library_error("library: The management API returned HTTP 500.") == "http",
)
check(
"err: outdated client (GTK init noise)",
main._classify_library_error("cannot open display: \nGtk-WARNING: init failed")
== "client-outdated",
)
check("err: generic fallback", main._classify_library_error("boom") == "client-error")
# ---- _parse_avahi_browse (incl. the new id/mgmt TXT keys) --------------------------------
avahi = (
"+;eth0;IPv4;living-room;_punktfunk._udp;local\n"
"=;eth0;IPv4;living-room;_punktfunk._udp;local;lr.local;192.168.1.42;9777;"
'"proto=punktfunk/1" "fp=aabbcc" "pair=required" "id=abc123" "mgmt=47990"\n'
"=;eth0;IPv6;living-room;_punktfunk._udp;local;lr.local;fe80::1;9777;"
'"proto=punktfunk/1" "fp=aabbcc" "pair=required" "id=abc123" "mgmt=47990"\n'
"=;eth0;IPv4;bare-host;_punktfunk._udp;local;bh.local;192.168.1.77;9777;"
'"proto=punktfunk/1" "fp=ddeeff" "pair=optional"\n'
)
hosts = main._parse_avahi_browse(avahi)
check("avahi: two hosts (id-dedup, IPv4 preferred)", len(hosts) == 2)
lr = next(h for h in hosts if h["name"] == "living-room")
check("avahi: ipv4 wins", lr["host"] == "192.168.1.42")
check("avahi: mgmt parsed", lr["mgmt"] == 47990)
check("avahi: id parsed", lr["id"] == "abc123")
bare = next(h for h in hosts if h["name"] == "bare-host")
check("avahi: mgmt absent -> 0", bare["mgmt"] == 0)
check("avahi: id absent -> empty", bare["id"] == "")
# ---- pins store (round-trip through the real methods, isolated HOME) --------------------
import asyncio # noqa: E402
import shutil # noqa: E402
shutil.rmtree(decky.DECKY_USER_HOME, ignore_errors=True)
plugin = main.Plugin()
pin = {
"game_id": "steam:570",
"title": "Dota 2",
"store": "steam",
"host_fp": "AABBCC",
"host_id": "abc123",
"host_name": "living-room",
"host": "192.168.1.42",
"port": 9777,
"mgmt": 47990,
"added_at": 1780000000,
}
dupe = dict(pin, title="Dota 2 again")
junk = {"title": "no game id"}
res = asyncio.run(plugin.set_pins([pin, dupe, junk]))
check("pins: write ok", res.get("ok") is True)
got = asyncio.run(plugin.get_pins())["pins"]
check("pins: dedup + junk dropped", len(got) == 1)
check("pins: unpaired without known-hosts", got[0]["paired"] is False)
# Mark the host paired in the client's known-hosts store — get_pins must pick it up.
cfg = main._client_config_dir()
cfg.mkdir(parents=True, exist_ok=True)
(cfg / "client-known-hosts.json").write_text(
'{"hosts": [{"name": "living-room", "addr": "192.168.1.42", "port": 9777, '
'"fp_hex": "aabbcc", "paired": true}]}'
)
got = asyncio.run(plugin.get_pins())["pins"]
check("pins: paired via known-hosts fp (case-insensitive)", got[0]["paired"] is True)
shutil.rmtree(decky.DECKY_USER_HOME, ignore_errors=True)
print()
if failures:
print(f"{failures} check(s) FAILED")
sys.exit(1)
print("all checks passed")
+72 -3
View File
@@ -9,6 +9,43 @@ export interface Host {
fp: string; // host cert SHA-256 fingerprint (lowercase hex) from the mDNS advert fp: string; // host cert SHA-256 fingerprint (lowercase hex) from the mDNS advert
proto: string; // advertised protocol, e.g. "punktfunk/1" proto: string; // advertised protocol, e.g. "punktfunk/1"
paired: boolean; // whether THIS device has already PIN-paired this host (by fingerprint) paired: boolean; // whether THIS device has already PIN-paired this host (by fingerprint)
id: string; // the host's stable instance id (mDNS TXT `id`; "" when not advertised)
mgmt: number; // management-API port (mDNS TXT `mgmt`; 0 = not advertised → default 47990)
}
// One title from a host's game library (the flatpak client's --library TSV, parsed by the
// backend). `id` is store-qualified (steam:<appid> / custom:<id>) and doubles as the
// launch handle (PF_LAUNCH → the session Hello).
export interface GameEntry {
id: string;
store: string; // "steam" | "custom" | "heroic" | "lutris" | …
title: string;
}
export interface LibraryResult {
ok: boolean;
games?: GameEntry[];
// "flatpak-not-found" | "timeout" | "not-paired" | "pin-mismatch" | "unreachable" |
// "http" | "client-outdated" | "client-error"
error?: string;
detail?: string; // the client's own one-line reason, for the generic error copy
}
// A pinned game — a one-tap stream row in the QAM. The host is identified primarily by
// cert fingerprint (survives IP changes; pairing is fp-keyed too), with the stored
// address as the launch fallback when the host isn't currently advertising.
export interface PinnedGame {
game_id: string;
title: string;
store: string;
host_fp: string;
host_id: string;
host_name: string;
host: string;
port: number;
mgmt: number;
added_at: number; // unix seconds
paired?: boolean; // annotated by get_pins from the client's known-hosts store
} }
export interface PairResult { export interface PairResult {
@@ -38,24 +75,56 @@ export interface StreamSettings {
} }
export interface UpdateInfo { export interface UpdateInfo {
current: string; // installed version (package.json) current: string; // installed PLUGIN version (package.json)
latest: string; // newest version in our registry for this channel latest: string; // newest plugin version in our registry for this channel
artifact: string; // immutable zip URL Decky should install artifact: string; // immutable zip URL Decky should install
hash: string; // sha256 of that zip (Decky verifies it) hash: string; // sha256 of that zip (Decky verifies it)
channel: string; // "latest" (stable) | "canary" channel: string; // "latest" (stable) | "canary"
update_available: boolean; update_available: boolean; // a newer PLUGIN build is available
// The flatpak CLIENT (io.unom.Punktfunk) versions independently and is a per-user install, so
// `sudo flatpak update` never touches it — the plugin offers a user-scope update instead.
client_update_available: boolean;
client_current: string; // installed client commit (short) — informational
client_latest: string; // remote client commit (short) — informational
error?: string; // "update-channel-unknown" (dev build) | "fetch-failed" error?: string; // "update-channel-unknown" (dev build) | "fetch-failed"
} }
// Steam-shortcut artwork (assets/ in the plugin dir): base64 PNGs keyed grid / gridwide /
// hero / logo, plus the icon's absolute path (SetShortcutIcon wants a file). Keys for
// missing files are absent.
export interface ShortcutArt {
grid?: string;
gridwide?: string;
hero?: string;
logo?: string;
icon_path: string;
}
export const discover = callable<[], Host[]>("discover"); export const discover = callable<[], Host[]>("discover");
export const pair = callable< export const pair = callable<
[host: string, port: number, pin: string, name: string], [host: string, port: number, pin: string, name: string],
PairResult PairResult
>("pair"); >("pair");
// Fetch a paired host's game library (headless flatpak --library; can take seconds on a
// cold client start — show a spinner). Pass fp whenever known so the pin can't degrade.
export const library = callable<
[host: string, mgmt_port: number, fp: string],
LibraryResult
>("library");
export const getPins = callable<[], { pins: PinnedGame[] }>("get_pins");
export const setPins = callable<[pins: PinnedGame[]], { ok: boolean; error?: string }>(
"set_pins",
);
export const runnerInfo = callable<[], RunnerInfo>("runner_info"); export const runnerInfo = callable<[], RunnerInfo>("runner_info");
export const shortcutArt = callable<[], ShortcutArt>("shortcut_art");
export const getSettings = callable<[], StreamSettings>("get_settings"); export const getSettings = callable<[], StreamSettings>("get_settings");
export const setSettings = callable<[settings: StreamSettings], { ok: boolean }>( export const setSettings = callable<[settings: StreamSettings], { ok: boolean }>(
"set_settings", "set_settings",
); );
export const killStream = callable<[], { ok: boolean }>("kill_stream"); export const killStream = callable<[], { ok: boolean }>("kill_stream");
export const checkUpdate = callable<[force: boolean], UpdateInfo>("check_update"); export const checkUpdate = callable<[force: boolean], UpdateInfo>("check_update");
// Update the flatpak client in the user installation (`flatpak update --user -y io.unom.Punktfunk`).
export const updateClient = callable<
[],
{ ok: boolean; updated: boolean; error?: string }
>("update_client");
+216 -13
View File
@@ -1,9 +1,19 @@
// Shared state hooks + user actions for the QAM panel and the fullscreen page. // Shared state hooks + user actions for the QAM panel and the fullscreen page.
import { toaster } from "@decky/api"; import { toaster } from "@decky/api";
import { Navigation } from "@decky/ui"; import { Navigation } from "@decky/ui";
import { useCallback, useEffect, useState } from "react"; import { useCallback, useEffect, useRef, useState } from "react";
import { checkUpdate, discover, Host, UpdateInfo } from "./backend"; import {
import { launchStream } from "./steam"; checkUpdate,
discover,
GameEntry,
getPins,
Host,
PinnedGame,
setPins as setPinsBackend,
updateClient,
UpdateInfo,
} from "./backend";
import { LaunchOpts, launchStream } from "./steam";
export const DOCS_URL = "https://docs.punktfunk.unom.io/docs/steam-deck"; export const DOCS_URL = "https://docs.punktfunk.unom.io/docs/steam-deck";
@@ -77,6 +87,11 @@ export function useUpdate() {
return { info, checking, check }; return { info, checking, check };
} }
/** True when EITHER the plugin or the flatpak client has a pending update. */
export function hasUpdate(info: UpdateInfo | null | undefined): boolean {
return !!info && (info.update_available || info.client_update_available);
}
/** The explicit "Check for updates" action — always ends in a toast so the tap has feedback. */ /** The explicit "Check for updates" action — always ends in a toast so the tap has feedback. */
export async function checkForUpdatesNow( export async function checkForUpdatesNow(
check: (force: boolean) => Promise<UpdateInfo | null>, check: (force: boolean) => Promise<UpdateInfo | null>,
@@ -85,17 +100,48 @@ export async function checkForUpdatesNow(
let body: string; let body: string;
if (!res || res.error === "fetch-failed") { if (!res || res.error === "fetch-failed") {
body = "Couldnt reach the update server — are you online?"; body = "Couldnt reach the update server — are you online?";
} else if (hasUpdate(res)) {
const parts: string[] = [];
if (res.update_available) parts.push(`plugin v${res.current} → v${res.latest}`);
if (res.client_update_available) parts.push("client");
body = `Update available: ${parts.join(" + ")}.`;
} else if (res.error === "update-channel-unknown") { } else if (res.error === "update-channel-unknown") {
body = "Development build — update checks are disabled."; body = "Development build — plugin updates are disabled; the client is up to date.";
} else if (res.update_available) {
body = `Update available: v${res.current} → v${res.latest}.`;
} else { } else {
body = `Youre up to date (v${res.current}).`; body = `Youre up to date (plugin v${res.current}).`;
} }
toaster.toast({ title: "Punktfunk", body }); toaster.toast({ title: "Punktfunk", body });
} }
export async function applyUpdate(info: UpdateInfo): Promise<void> { /**
* Apply whichever updates are pending. The flatpak CLIENT is updated first (a user-scope
* `flatpak update`, awaited); then, if the PLUGIN itself has an update, Decky's install RPC
* reinstalls it — which reloads the plugin and tears this panel down, so it goes last and is
* fire-and-forget. `check` (when passed) refreshes the panel state after a client-only update so
* the "Update available" button clears.
*/
export async function applyUpdate(
info: UpdateInfo,
check?: (force: boolean) => Promise<UpdateInfo | null>,
): Promise<void> {
if (info.client_update_available) {
toaster.toast({ title: "Punktfunk", body: "Updating the client…" });
try {
const r = await updateClient();
toaster.toast({
title: "Punktfunk",
body: !r.ok
? `Client update failed${r.error ? ` (${r.error})` : ""}.`
: r.updated
? "Client updated to the latest version."
: "Client is already up to date.",
});
} catch {
toaster.toast({ title: "Punktfunk", body: "Client update failed." });
}
}
if (info.update_available) {
try { try {
const backend = window.DeckyBackend; const backend = window.DeckyBackend;
if (backend?.callable) { if (backend?.callable) {
@@ -112,7 +158,7 @@ export async function applyUpdate(info: UpdateInfo): Promise<void> {
title: "Punktfunk", title: "Punktfunk",
// Decky's installer also phones the plugin store first, which can hang on some // Decky's installer also phones the plugin store first, which can hang on some
// networks before the actual install proceeds — set expectations. // networks before the actual install proceeds — set expectations.
body: `Updating to v${info.latest} — confirm Deckys prompt. This can take a couple of minutes.`, body: `Updating the plugin to v${info.latest} — confirm Deckys prompt. This can take a couple of minutes.`,
}); });
return; return;
} }
@@ -121,19 +167,176 @@ export async function applyUpdate(info: UpdateInfo): Promise<void> {
} }
toaster.toast({ toaster.toast({
title: "Punktfunk", title: "Punktfunk",
body: "Update from Decky → Developer → Install Plugin from URL.", body: "Update the plugin from Decky → Developer → Install Plugin from URL.",
}); });
return;
}
// Client-only update (no plugin reinstall): refresh so the button clears.
if (check) void check(true);
} }
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
// Stream launch — via the hidden Steam shortcut (see steam.ts for why). // Stream launch — via the hidden Steam shortcut (see steam.ts for why).
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
export async function startStream(h: Host): Promise<void> { export async function startStream(
h: Host,
opts: LaunchOpts = {},
label?: string,
): Promise<void> {
try { try {
await launchStream(h.host, h.port); await launchStream(h.host, h.port, opts);
Navigation.CloseSideMenus(); Navigation.CloseSideMenus();
toaster.toast({ title: "Punktfunk", body: `Starting stream${h.name}` }); toaster.toast({ title: "Punktfunk", body: `Starting ${label ?? "stream"}${h.name}` });
} catch (e) { } catch (e) {
toaster.toast({ title: "Punktfunk", body: `Launch failed: ${e}` }); toaster.toast({ title: "Punktfunk", body: `Launch failed: ${e}` });
} }
} }
/** Open the GTK client's gamepad library launcher for a host (`--browse` via PF_BROWSE). */
export async function startBrowse(h: Host): Promise<void> {
try {
await launchStream(h.host, h.port, { browse: true, mgmt: h.mgmt });
Navigation.CloseSideMenus();
toaster.toast({ title: "Punktfunk", body: `Opening library — ${h.name}` });
} catch (e) {
toaster.toast({ title: "Punktfunk", body: `Launch failed: ${e}` });
}
}
// ----------------------------------------------------------------------------------------
// Pinned games — the QAM's one-tap game rows, persisted by the backend next to the
// client's config (survives plugin reinstalls).
// ----------------------------------------------------------------------------------------
export interface PinsApi {
pins: PinnedGame[];
addPin: (h: Host, g: GameEntry) => void;
removePin: (hostFp: string, gameId: string) => void;
isPinned: (hostFp: string, gameId: string) => boolean;
/** Refresh a pin's stored address from a live advert (hosts change IPs). */
updatePinHost: (pin: PinnedGame, h: Host) => void;
refresh: () => Promise<void>;
}
export function usePins(): PinsApi {
const [pins, setPins] = useState<PinnedGame[]>([]);
// A live mirror of `pins`. The Games picker is mounted by Decky's `showModal` into a
// detached portal that captures this hook's callbacks ONCE and never re-renders with fresh
// props, so a mutator closing over the `pins` array reads a frozen base — pinning a second
// game in the same session would compute from the stale `[]` and clobber the first (silent
// data loss). Reading the ref keeps every mutation based on the current set, and lets the
// callbacks keep a stable identity (deps free of `pins`).
const pinsRef = useRef<PinnedGame[]>([]);
pinsRef.current = pins;
const refresh = useCallback(async () => {
try {
setPins((await getPins()).pins);
} catch {
/* backend unavailable — keep the current view */
}
}, []);
useEffect(() => {
void refresh();
}, [refresh]);
// Optimistic local state; the backend validates/dedups and is re-read on failure.
const save = useCallback(
(next: PinnedGame[]) => {
pinsRef.current = next;
setPins(next);
setPinsBackend(next).catch(() => void refresh());
},
[refresh],
);
const addPin = useCallback(
(h: Host, g: GameEntry) => {
const pin: PinnedGame = {
game_id: g.id,
title: g.title,
store: g.store,
host_fp: h.fp,
host_id: h.id,
host_name: h.name,
host: h.host,
port: h.port,
mgmt: h.mgmt,
added_at: Math.floor(Date.now() / 1000),
paired: h.paired,
};
save([
...pinsRef.current.filter(
(p) => !(p.host_fp === pin.host_fp && p.game_id === pin.game_id),
),
pin,
]);
},
[save],
);
const removePin = useCallback(
(hostFp: string, gameId: string) => {
save(pinsRef.current.filter((p) => !(p.host_fp === hostFp && p.game_id === gameId)));
},
[save],
);
const isPinned = useCallback(
(hostFp: string, gameId: string) =>
pins.some((p) => p.host_fp === hostFp && p.game_id === gameId),
[pins],
);
const updatePinHost = useCallback(
(pin: PinnedGame, h: Host) => {
if (pin.host === h.host && pin.port === h.port && pin.mgmt === h.mgmt) {
return;
}
save(
pinsRef.current.map((p) =>
p.host_fp === pin.host_fp && p.game_id === pin.game_id
? { ...p, host: h.host, port: h.port, mgmt: h.mgmt, host_name: h.name }
: p,
),
);
},
[save],
);
return { pins, addPin, removePin, isPinned, updatePinHost, refresh };
}
/**
* The host a pin should launch against right now: match the live mDNS scan by cert
* fingerprint first (pairing is fp-keyed, survives IP changes), then by the host's stable
* id, else fall back to the stored address (host offline or scan flaky — still launch).
*/
export function resolvePinHost(
pin: PinnedGame,
live: Host[],
): { host: Host; online: boolean } {
const fp = pin.host_fp.toLowerCase();
const match =
(fp && live.find((h) => h.fp && h.fp.toLowerCase() === fp)) ||
(pin.host_id && live.find((h) => h.id && h.id === pin.host_id)) ||
undefined;
if (match) {
return { host: match, online: true };
}
return {
host: {
name: pin.host_name || pin.host,
host: pin.host,
port: pin.port,
pair: pin.paired ? "optional" : "required",
fp: pin.host_fp,
proto: "",
paired: !!pin.paired,
id: pin.host_id,
mgmt: pin.mgmt,
},
online: false,
};
}
+49 -6
View File
@@ -12,28 +12,46 @@ import {
} from "@decky/ui"; } from "@decky/ui";
import { definePlugin, routerHook } from "@decky/api"; import { definePlugin, routerHook } from "@decky/api";
import { FC } from "react"; import { FC } from "react";
import { FaDownload, FaLock, FaLockOpen, FaSyncAlt, FaTv } from "react-icons/fa"; import { FaDownload, FaLock, FaLockOpen, FaPlay, FaSyncAlt, FaTv } from "react-icons/fa";
import { PluginErrorBoundary } from "./boundary"; import { PluginErrorBoundary } from "./boundary";
import { applyUpdate, checkForUpdatesNow, startStream, useHosts, useUpdate } from "./hooks"; import {
applyUpdate,
checkForUpdatesNow,
hasUpdate,
resolvePinHost,
startStream,
useHosts,
usePins,
useUpdate,
} from "./hooks";
import { streamPin } from "./library";
import { PunktfunkRoute, ROUTE } from "./page"; import { PunktfunkRoute, ROUTE } from "./page";
import { PairModal } from "./pair"; import { PairModal } from "./pair";
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
// QAM panel — quick status + entry into the full page + one-tap stream for known hosts. // QAM panel — quick status + entry into the full page + one-tap stream for known hosts
// and pinned games.
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
const QamPanel: FC = () => { const QamPanel: FC = () => {
const { hosts, scanning, refresh } = useHosts(); const { hosts, scanning, refresh } = useHosts();
const { info: update, checking, check } = useUpdate(); const { info: update, checking, check } = useUpdate();
const pins = usePins();
return ( return (
<> <>
{update?.update_available && ( {hasUpdate(update) && (
<PanelSection title="Update available"> <PanelSection title="Update available">
<PanelSectionRow> <PanelSectionRow>
<ButtonItem <ButtonItem
layout="below" layout="below"
onClick={() => applyUpdate(update)} onClick={() => applyUpdate(update!, check)}
label={`v${update.current} → v${update.latest}`} label={
update!.update_available
? `Plugin v${update!.current} → v${update!.latest}${
update!.client_update_available ? " + client" : ""
}`
: "New client version"
}
description="Installing can take a couple of minutes" description="Installing can take a couple of minutes"
> >
<FaDownload style={{ marginRight: "0.5em" }} /> <FaDownload style={{ marginRight: "0.5em" }} />
@@ -59,6 +77,31 @@ const QamPanel: FC = () => {
</PanelSectionRow> </PanelSectionRow>
</PanelSection> </PanelSection>
{/* Pinned games — the "jump straight into Playnite" rows. Pin games from a host's
picker (fullscreen page → host row → games button). */}
{pins.pins.length > 0 && (
<PanelSection title="Games">
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
return (
<PanelSectionRow key={`${pin.host_fp}:${pin.game_id}`}>
<ButtonItem
layout="below"
onClick={() => streamPin(pin, hosts, pins)}
label={pin.title}
description={`${pin.host_name}${online ? "" : " · offline?"}${
pin.paired ? "" : " · pairing required"
}`}
>
<FaPlay style={{ marginRight: "0.5em" }} />
Stream
</ButtonItem>
</PanelSectionRow>
);
})}
</PanelSection>
)}
<PanelSection title="Hosts"> <PanelSection title="Hosts">
<PanelSectionRow> <PanelSectionRow>
<ButtonItem layout="below" onClick={refresh} disabled={scanning}> <ButtonItem layout="below" onClick={refresh} disabled={scanning}>
+231
View File
@@ -0,0 +1,231 @@
// The per-host game picker + pinned-game launch helper. The picker fetches a paired
// host's library through the backend (headless flatpak --library — a cold client start
// can take seconds, hence the explicit spinner copy) and pins titles as one-tap rows in
// the QAM's Games section; its header also launches the GTK client's on-screen gamepad
// library (`--browse`).
import { DialogButton, Field, Focusable, ModalRoot, Spinner, showModal } from "@decky/ui";
import { CSSProperties, FC, useEffect, useState } from "react";
import { FaThLarge, FaTv } from "react-icons/fa";
import { GameEntry, Host, library, LibraryResult, PinnedGame } from "./backend";
import { PinsApi, resolvePinHost, startBrowse, startStream } from "./hooks";
import { isSafeLaunchId } from "./steam";
import { PairModal } from "./pair";
/** Human store tag (mirrors the GTK client's `store_label`). */
export function storeLabel(store: string): string {
switch (store) {
case "steam":
return "Steam";
case "custom":
return "Custom";
case "heroic":
return "Heroic";
case "lutris":
return "Lutris";
case "epic":
return "Epic";
case "gog":
return "GOG";
case "xbox":
return "Xbox";
default:
return "Game";
}
}
/**
* Stream a pinned game: resolve the host from the live scan (fp → id → stored address),
* opportunistically refresh a drifted stored address, and route through pairing first if
* this device is no longer paired with the host.
*/
export function streamPin(pin: PinnedGame, live: Host[], pins: PinsApi): void {
const { host, online } = resolvePinHost(pin, live);
if (online) {
pins.updatePinHost(pin, host); // no-op unless the address actually drifted
}
if (!pin.paired) {
showModal(
<PairModal
host={host}
onPaired={() => {
void pins.refresh(); // pick up the now-paired annotation
void startStream(host, { launchId: pin.game_id }, pin.title);
}}
/>,
);
return;
}
void startStream(host, { launchId: pin.game_id }, pin.title);
}
const pickButton: CSSProperties = {
width: "fit-content",
minWidth: "5em",
flexShrink: 0,
};
// Copy per backend error code (LibraryResult.error); `detail` covers the generic case.
function errorCopy(res: LibraryResult): string {
switch (res.error) {
case "not-paired":
return "This Deck isn't paired with the host — pair first, then browse its library.";
case "pin-mismatch":
return "The host's identity changed — re-pair to re-establish trust.";
case "unreachable":
return "Couldn't reach the host's management API. Is the host online and up to date?";
case "timeout":
return "Timed out talking to the host — try again.";
case "flatpak-not-found":
return "The Punktfunk client isn't installed (flatpak io.unom.Punktfunk).";
case "client-outdated":
return "The installed client is too old for library browsing — update it from the About tab.";
default:
return res.detail || "Couldn't fetch the library.";
}
}
// ----------------------------------------------------------------------------------------
// The picker modal: "open on screen" + a pin-toggle list of the host's games.
// ----------------------------------------------------------------------------------------
export const GamePickerModal: FC<{
host: Host;
pins: PinsApi;
clientUpdatePending?: boolean;
closeModal?: () => void;
}> = ({ host, pins, clientUpdatePending, closeModal }) => {
const [result, setResult] = useState<LibraryResult | null>(null);
const [attempt, setAttempt] = useState(0); // bump to refetch (retry / after pairing)
// The modal is a detached `showModal` portal that never re-renders from the page's pin
// state, so `pins.isPinned` would read a frozen snapshot and the Pin/Unpin label would
// never flip within a session. Track this host's pinned ids locally, seeded once from the
// snapshot at open; persistence still goes through the (stale-closure-safe) pins API.
const [pinnedIds, setPinnedIds] = useState<Set<string>>(
() => new Set(pins.pins.filter((p) => p.host_fp === host.fp).map((p) => p.game_id)),
);
const togglePin = (g: GameEntry) => {
const wasPinned = pinnedIds.has(g.id);
setPinnedIds((prev) => {
const next = new Set(prev);
if (wasPinned) next.delete(g.id);
else next.add(g.id);
return next;
});
if (wasPinned) pins.removePin(host.fp, g.id);
else pins.addPin(host, g);
};
useEffect(() => {
let stale = false;
setResult(null);
library(host.host, host.mgmt, host.fp)
.then((res) => {
if (!stale) setResult(res);
})
.catch((e) => {
if (!stale) setResult({ ok: false, error: "client-error", detail: String(e) });
});
return () => {
stale = true;
};
}, [host.host, host.mgmt, host.fp, attempt]);
const games = (result?.ok && result.games) || [];
const sorted = [...games].sort((a, b) => a.title.localeCompare(b.title));
return (
<ModalRoot closeModal={closeModal}>
<div style={{ fontWeight: "bold", fontSize: "1.3em", marginBottom: "0.4em" }}>
{host.name} Games
</div>
<Field
label="Open library on screen"
description="Browse this host's games with the controller, full screen"
childrenContainerWidth="max"
>
<DialogButton
style={pickButton}
onClick={() => {
closeModal?.();
void startBrowse(host);
}}
>
<FaTv style={{ marginRight: "0.4em" }} />
Open
</DialogButton>
</Field>
{clientUpdatePending && (
<Field
focusable={false}
description="A client update is available — direct game launch and on-screen browsing need the latest client."
/>
)}
{result === null && (
<Field
focusable={false}
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.6em" }}>
<Spinner style={{ height: "1em" }} />
Fetching the library
</span>
}
description="This starts the client headlessly — a cold start can take a few seconds."
/>
)}
{result !== null && !result.ok && (
<Field label="Couldn't fetch the library" description={errorCopy(result)} childrenContainerWidth="max">
<Focusable style={{ display: "flex", gap: "0.5em", justifyContent: "flex-end" }}>
{result.error === "not-paired" && (
<DialogButton
style={pickButton}
onClick={() =>
showModal(<PairModal host={host} onPaired={() => setAttempt((n) => n + 1)} />)
}
>
Pair
</DialogButton>
)}
<DialogButton style={pickButton} onClick={() => setAttempt((n) => n + 1)}>
Retry
</DialogButton>
</Focusable>
</Field>
)}
{result?.ok && sorted.length === 0 && (
<Field
focusable={false}
label="No games found"
description="Install Steam titles or add custom entries in the host's web console."
/>
)}
{sorted.length > 0 && (
<div style={{ maxHeight: "55vh", overflowY: "auto" }}>
{sorted.map((g: GameEntry) => {
const pinned = pinnedIds.has(g.id);
const safe = isSafeLaunchId(g.id);
return (
<Field
key={g.id}
label={g.title}
description={
storeLabel(g.store) + (safe ? "" : " · unsupported id — can't be pinned")
}
childrenContainerWidth="max"
>
<DialogButton style={pickButton} disabled={!safe} onClick={() => togglePin(g)}>
<FaThLarge style={{ marginRight: "0.4em" }} />
{pinned ? "Unpin" : "Pin"}
</DialogButton>
</Field>
);
})}
</div>
)}
</ModalRoot>
);
};
+136 -26
View File
@@ -21,17 +21,23 @@ import {
FaLockOpen, FaLockOpen,
FaPlay, FaPlay,
FaSyncAlt, FaSyncAlt,
FaThLarge,
} from "react-icons/fa"; } from "react-icons/fa";
import { Host, UpdateInfo, killStream } from "./backend"; import { Host, UpdateInfo, killStream } from "./backend";
import { PluginErrorBoundary } from "./boundary"; import { PluginErrorBoundary } from "./boundary";
import { import {
DOCS_URL, DOCS_URL,
PinsApi,
applyUpdate, applyUpdate,
checkForUpdatesNow, checkForUpdatesNow,
hasUpdate,
resolvePinHost,
startStream, startStream,
useHosts, useHosts,
usePins,
useUpdate, useUpdate,
} from "./hooks"; } from "./hooks";
import { GamePickerModal, storeLabel, streamPin } from "./library";
import { PairModal } from "./pair"; import { PairModal } from "./pair";
import { SettingsSection } from "./settings"; import { SettingsSection } from "./settings";
import { stopStream } from "./steam"; import { stopStream } from "./steam";
@@ -52,6 +58,27 @@ const tabScroll: CSSProperties = {
boxSizing: "border-box", boxSizing: "border-box",
}; };
// DialogButton stretches to 100% width in the gamepad UI — on a fullscreen row that means a
// screen-wide button. Size action buttons to their content instead (right-aligned by the
// Field's children container).
const actionButton: CSSProperties = {
width: "fit-content",
minWidth: "6em",
flexShrink: 0,
};
// Square icon-only button (details ⓘ, header back arrow) — needs an explicit height too, or
// the zero padding collapses it to the icon's line height.
const iconButton: CSSProperties = {
width: "40px",
minWidth: "40px",
height: "40px",
padding: 0,
flexShrink: 0,
display: "flex",
alignItems: "center",
justifyContent: "center",
};
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
// Host details — everything the mDNS advert told us, incl. the fingerprint to cross-check // Host details — everything the mDNS advert told us, incl. the fingerprint to cross-check
// against the host's own log / web console before trusting it. // against the host's own log / web console before trusting it.
@@ -96,7 +123,11 @@ const HostDetailsModal: FC<{ host: Host; closeModal?: () => void }> = ({
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
// One host row: status icon + address, details / pair / stream actions. // One host row: status icon + address, details / pair / stream actions.
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
const HostRow: FC<{ host: Host; onPaired: () => void }> = ({ host, onPaired }) => { const HostRow: FC<{ host: Host; onPaired: () => void; onGames: () => void }> = ({
host,
onPaired,
onGames,
}) => {
// The host's policy is `pair=required`, but if THIS device is already paired we don't need to // The host's policy is `pair=required`, but if THIS device is already paired we don't need to
// pair again — show it as trusted and go straight to Stream. // pair again — show it as trusted and go straight to Stream.
const needsPair = host.pair === "required" && !host.paired; const needsPair = host.pair === "required" && !host.paired;
@@ -113,22 +144,37 @@ const HostRow: FC<{ host: Host; onPaired: () => void }> = ({ host, onPaired }) =
}`} }`}
childrenContainerWidth="max" childrenContainerWidth="max"
> >
<Focusable style={{ display: "flex", gap: "0.5em" }}> <Focusable style={{ display: "flex", gap: "0.5em", justifyContent: "flex-end" }}>
<DialogButton <DialogButton
style={{ width: "3em", minWidth: "3em", padding: 0 }} style={iconButton}
onClick={() => showModal(<HostDetailsModal host={host} />)} onClick={() => showModal(<HostDetailsModal host={host} />)}
> >
<FaInfoCircle /> <FaInfoCircle />
</DialogButton> </DialogButton>
{/* Labeled, not icon-only: this is the entry to the game picker AND the on-screen
library browser, and controller nav has no hover tooltip to explain a bare icon. */}
<DialogButton style={{ ...actionButton, minWidth: "6em" }} onClick={onGames}>
<FaThLarge style={{ marginRight: "0.4em" }} />
Games
</DialogButton>
{needsPair && ( {needsPair && (
<DialogButton <DialogButton
style={{ minWidth: "5em" }} style={{ ...actionButton, minWidth: "5em" }}
onClick={() => showModal(<PairModal host={host} onPaired={onPaired} />)} onClick={() => showModal(<PairModal host={host} onPaired={onPaired} />)}
> >
Pair Pair
</DialogButton> </DialogButton>
)} )}
<DialogButton style={{ minWidth: "6em" }} onClick={() => startStream(host)}> <DialogButton
style={actionButton}
onClick={() =>
needsPair
? showModal(
<PairModal host={host} onPaired={() => startStream(host)} />,
)
: startStream(host)
}
>
<FaPlay style={{ marginRight: "0.4em" }} /> <FaPlay style={{ marginRight: "0.4em" }} />
Stream Stream
</DialogButton> </DialogButton>
@@ -141,7 +187,9 @@ const HostsTab: FC<{
hosts: Host[]; hosts: Host[];
scanning: boolean; scanning: boolean;
refresh: () => void; refresh: () => void;
}> = ({ hosts, scanning, refresh }) => ( pins: PinsApi;
clientUpdatePending: boolean;
}> = ({ hosts, scanning, refresh, pins, clientUpdatePending }) => (
<div style={tabScroll}> <div style={tabScroll}>
<Field <Field
label="Discover" label="Discover"
@@ -153,7 +201,7 @@ const HostsTab: FC<{
childrenContainerWidth="max" childrenContainerWidth="max"
bottomSeparator={hosts.length ? "standard" : "none"} bottomSeparator={hosts.length ? "standard" : "none"}
> >
<DialogButton style={{ minWidth: "8em" }} disabled={scanning} onClick={refresh}> <DialogButton style={{ ...actionButton, minWidth: "8em" }} disabled={scanning} onClick={refresh}>
{scanning ? ( {scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} /> <Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : ( ) : (
@@ -171,8 +219,55 @@ const HostsTab: FC<{
/> />
)} )}
{hosts.map((h) => ( {hosts.map((h) => (
<HostRow key={h.fp || `${h.host}:${h.port}`} host={h} onPaired={refresh} /> <HostRow
key={h.fp || `${h.host}:${h.port}`}
host={h}
onPaired={refresh}
onGames={() =>
showModal(
<GamePickerModal host={h} pins={pins} clientUpdatePending={clientUpdatePending} />,
)
}
/>
))} ))}
{/* Pinned games — also the cleanup surface for pins whose host is gone from the scan. */}
{pins.pins.length > 0 && (
<>
<Field
focusable={false}
label="Pinned games"
description="One-tap streams — they also live in the quick-access menu"
bottomSeparator="standard"
/>
{pins.pins.map((pin) => {
const { online } = resolvePinHost(pin, hosts);
return (
<Field
key={`${pin.host_fp}:${pin.game_id}`}
label={pin.title}
description={`${storeLabel(pin.store)} · ${pin.host_name}${
online ? "" : " · offline?"
}${pin.paired ? "" : " · pairing required"}`}
childrenContainerWidth="max"
>
<Focusable style={{ display: "flex", gap: "0.5em", justifyContent: "flex-end" }}>
<DialogButton style={actionButton} onClick={() => streamPin(pin, hosts, pins)}>
<FaPlay style={{ marginRight: "0.4em" }} />
Play
</DialogButton>
<DialogButton
style={{ ...actionButton, minWidth: "5em" }}
onClick={() => pins.removePin(pin.host_fp, pin.game_id)}
>
Remove
</DialogButton>
</Focusable>
</Field>
);
})}
</>
)}
</div> </div>
); );
@@ -212,20 +307,29 @@ const AboutTab: FC<{
childrenContainerWidth="max" childrenContainerWidth="max"
> >
<DialogButton <DialogButton
style={{ minWidth: "11em" }} style={{ ...actionButton, minWidth: "11em" }}
disabled={checking} disabled={checking}
onClick={() => void checkForUpdatesNow(check)} onClick={() => void checkForUpdatesNow(check)}
> >
{checking ? <Spinner style={{ height: "1em" }} /> : "Check for updates"} {checking ? <Spinner style={{ height: "1em" }} /> : "Check for updates"}
</DialogButton> </DialogButton>
</Field> </Field>
{update?.update_available && ( {hasUpdate(update) && (
<Field <Field
label={`Update available — v${update.latest}`} label={
update!.update_available
? `Plugin update — v${update!.latest}${
update!.client_update_available ? " + client" : ""
}`
: "Client update available"
}
description="Installing can take a couple of minutes; Decky reloads the plugin when done" description="Installing can take a couple of minutes; Decky reloads the plugin when done"
childrenContainerWidth="max" childrenContainerWidth="max"
> >
<DialogButton style={{ minWidth: "9em" }} onClick={() => applyUpdate(update)}> <DialogButton
style={{ ...actionButton, minWidth: "9em" }}
onClick={() => applyUpdate(update!, check)}
>
<FaDownload style={{ marginRight: "0.4em" }} /> <FaDownload style={{ marginRight: "0.4em" }} />
Update Update
</DialogButton> </DialogButton>
@@ -237,7 +341,7 @@ const AboutTab: FC<{
childrenContainerWidth="max" childrenContainerWidth="max"
> >
<DialogButton <DialogButton
style={{ minWidth: "8em" }} style={{ ...actionButton, minWidth: "8em" }}
onClick={() => Navigation.NavigateToExternalWeb(DOCS_URL)} onClick={() => Navigation.NavigateToExternalWeb(DOCS_URL)}
> >
<FaExternalLinkAlt style={{ marginRight: "0.4em" }} /> <FaExternalLinkAlt style={{ marginRight: "0.4em" }} />
@@ -254,7 +358,7 @@ const AboutTab: FC<{
description="Force-stop the stream client if a session wedges" description="Force-stop the stream client if a session wedges"
childrenContainerWidth="max" childrenContainerWidth="max"
> >
<DialogButton style={{ minWidth: "8em" }} onClick={() => void forceStopStream()}> <DialogButton style={{ ...actionButton, minWidth: "8em" }} onClick={() => void forceStopStream()}>
Force-stop Force-stop
</DialogButton> </DialogButton>
</Field> </Field>
@@ -264,6 +368,7 @@ const AboutTab: FC<{
const PunktfunkPage: FC = () => { const PunktfunkPage: FC = () => {
const { hosts, scanning, refresh } = useHosts(); const { hosts, scanning, refresh } = useHosts();
const { info: update, checking, check } = useUpdate(); const { info: update, checking, check } = useUpdate();
const pins = usePins();
const [tab, setTab] = useState("hosts"); const [tab, setTab] = useState("hosts");
return ( return (
@@ -275,6 +380,7 @@ const PunktfunkPage: FC = () => {
flexDirection: "column", flexDirection: "column",
}} }}
> >
{/* Header is title + back only — updates live on the About tab (and the QAM banner). */}
<Focusable <Focusable
style={{ style={{
display: "flex", display: "flex",
@@ -285,24 +391,20 @@ const PunktfunkPage: FC = () => {
flexShrink: 0, flexShrink: 0,
}} }}
> >
<DialogButton <DialogButton style={iconButton} onClick={() => Navigation.NavigateBack()}>
style={{ width: "3em", minWidth: "3em", padding: 0 }}
onClick={() => Navigation.NavigateBack()}
>
<FaArrowLeft /> <FaArrowLeft />
</DialogButton> </DialogButton>
<div className={staticClasses?.Title} style={{ flex: 1, margin: 0 }}> <div className={staticClasses?.Title} style={{ flex: 1, margin: 0 }}>
Punktfunk Punktfunk
</div> </div>
{update?.update_available && (
<DialogButton style={{ minWidth: "9em" }} onClick={() => applyUpdate(update)}>
<FaDownload style={{ marginRight: "0.4em" }} />
Update v{update.latest}
</DialogButton>
)}
</Focusable> </Focusable>
<div style={{ flex: 1, minHeight: 0 }}> {/* overflow:hidden is load-bearing: Valve's Tabs slides the incoming panel in from the
right on L1/R1, and with autoFocusContents it scrollIntoViews a control inside that
still-offscreen panel. Without a clip here the scroll pans #GamepadUI itself — the whole
Steam UI (top bar included) slides left until you click a tab. Valve's own Tabs always
live in a clipped flex box; match that. */}
<div style={{ flex: 1, minHeight: 0, overflow: "hidden" }}>
<Tabs <Tabs
activeTab={tab} activeTab={tab}
onShowTab={(id: string) => setTab(id)} onShowTab={(id: string) => setTab(id)}
@@ -311,7 +413,15 @@ const PunktfunkPage: FC = () => {
{ {
id: "hosts", id: "hosts",
title: "Hosts", title: "Hosts",
content: <HostsTab hosts={hosts} scanning={scanning} refresh={refresh} />, content: (
<HostsTab
hosts={hosts}
scanning={scanning}
refresh={refresh}
pins={pins}
clientUpdatePending={!!update?.client_update_available}
/>
),
}, },
{ {
id: "settings", id: "settings",
+2 -2
View File
@@ -99,10 +99,10 @@ export const SettingsSection: FC = () => {
onChange={(o) => patch({ gamepad: o.data as string })} onChange={(o) => patch({ gamepad: o.data as string })}
/> />
</Field> </Field>
{s.gamepad === "steamdeck" && ( {(s.gamepad === "steamdeck" || s.gamepad === "auto") && (
<Field <Field
label="⚠ Disable Steam Input" label="⚠ Disable Steam Input"
description="Steam Deck mode forwards the paddles, both trackpads, and gyro to the host. For that, Steam Input must be OFF for Punktfunk: on the game page tap ⚙ → Controller Settings → set Steam Input to Off. Otherwise Steam keeps the Deck's controls and only the sticks + buttons reach the host." description="On a Deck, Automatic forwards the built-in controller as a Steam Deck pad — paddles, both trackpads, and gyro included. For that, Steam Input must be OFF for Punktfunk: on the game page tap ⚙ → Controller Settings → set Steam Input to Off. Otherwise Steam keeps the Deck's controls and only the sticks + buttons reach the host."
/> />
)} )}
<Field <Field
+109 -19
View File
@@ -8,7 +8,7 @@
// and start it with RunGame. The wrapper then execs // and start it with RunGame. The wrapper then execs
// `flatpak run io.unom.Punktfunk --connect <host>` as a reaper descendant. // `flatpak run io.unom.Punktfunk --connect <host>` as a reaper descendant.
import { runnerInfo } from "./backend"; import { runnerInfo, shortcutArt } from "./backend";
// SteamClient is a Steam-internal global injected into the CEF context; it is not fully typed // SteamClient is a Steam-internal global injected into the CEF context; it is not fully typed
// by @decky/ui, so declare the surface we use. Signatures verified against MoonDeck + the // by @decky/ui, so declare the surface we use. Signatures verified against MoonDeck + the
@@ -24,24 +24,35 @@ declare const SteamClient: {
SetShortcutName(appId: number, name: string): void; SetShortcutName(appId: number, name: string): void;
SetShortcutExe(appId: number, exe: string): void; SetShortcutExe(appId: number, exe: string): void;
SetShortcutStartDir(appId: number, dir: string): void; SetShortcutStartDir(appId: number, dir: string): void;
SetShortcutIcon(appId: number, iconPath: string): void;
SetAppLaunchOptions(appId: number, options: string): void; SetAppLaunchOptions(appId: number, options: string): void;
// assetType: 0 = grid (portrait capsule), 1 = hero, 2 = logo, 3 = wide grid.
SetCustomArtworkForApp(
appId: number,
base64Image: string,
imageType: string,
assetType: number,
): Promise<unknown>;
RunGame(gameId: string, _unused: string, _i: number, _j: number): void; RunGame(gameId: string, _unused: string, _i: number, _j: number): void;
TerminateApp(gameId: string, _b: boolean): void; TerminateApp(gameId: string, _b: boolean): void;
}; };
}; };
// Steam removed `SteamClient.Apps.SetAppHidden`. Hiding a non-Steam shortcut now goes through // Steam removed `SteamClient.Apps.SetAppHidden`; visibility goes through
// `collectionStore.SetAppsAsHidden([appId], true)` — but that looks the app up in appStore, which // `collectionStore.SetAppsAsHidden` — but that looks the app up in appStore, which only
// only registers a freshly-created shortcut a moment later (calling it immediately throws on a // registers a freshly-created shortcut a moment later (calling it immediately throws on a
// null overview). So hiding is BEST-EFFORT + DEFERRED and must NEVER block the launch. // null overview). So visibility changes are BEST-EFFORT + DEFERRED, never launch-blocking.
declare const collectionStore: declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void } | { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined; | undefined;
function hideShortcut(appId: number): void { // The shortcut used to be hidden ("implementation detail"); it is user-visible now — it
// carries proper artwork and living in the library is how users relaunch their last host.
// Existing installs still have theirs hidden, so unhide is applied every ensure (idempotent).
function unhideShortcut(appId: number): void {
const attempt = () => { const attempt = () => {
try { try {
collectionStore?.SetAppsAsHidden?.([appId], true); collectionStore?.SetAppsAsHidden?.([appId], false);
} catch { } catch {
/* overview not registered yet, or the API changed — cosmetic, ignore */ /* overview not registered yet, or the API changed — cosmetic, ignore */
} }
@@ -50,6 +61,40 @@ function hideShortcut(appId: number): void {
setTimeout(attempt, 2500); // fresh shortcut: retry once its app overview lands setTimeout(attempt, 2500); // fresh shortcut: retry once its app overview lands
} }
// Bump when the shipped artwork changes so existing shortcuts re-apply it once.
const ART_VERSION = 1;
const ART_KEY = "punktfunk:shortcutArt";
/**
* Apply the plugin's grid/hero/logo/icon to the shortcut (idempotent, once per ART_VERSION).
* Cosmetic and fully best-effort: any failure is swallowed and retried on the next launch.
*/
async function applyArtwork(appId: number): Promise<void> {
try {
if (localStorage.getItem(ART_KEY) === `${appId}:${ART_VERSION}`) {
return;
}
const art = await shortcutArt();
const assets: [string | undefined, number][] = [
[art.grid, 0],
[art.hero, 1],
[art.logo, 2],
[art.gridwide, 3],
];
for (const [data, assetType] of assets) {
if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
}
}
if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
}
localStorage.setItem(ART_KEY, `${appId}:${ART_VERSION}`);
} catch (e) {
console.warn("punktfunk: shortcut artwork not applied", e);
}
}
// The shortcut name is user-visible (Steam overlay + library while streaming) — brand-case it. // The shortcut name is user-visible (Steam overlay + library while streaming) — brand-case it.
const SHORTCUT_NAME = "Punktfunk"; const SHORTCUT_NAME = "Punktfunk";
@@ -87,10 +132,11 @@ function recallAppId(): number | null {
} }
/** /**
* Ensure exactly one hidden "Punktfunk" shortcut exists (exe = /bin/sh; the wrapper script is * Ensure exactly one "Punktfunk" shortcut exists (exe = /bin/sh; the wrapper script is
* appended per-launch via the launch options), and return its appId + the current runner path. * appended per-launch via the launch options), branded and visible in the library, and
* Reuses the remembered shortcut, re-pointing it each time — the plugin dir can change across * return its appId + the current runner path. Reuses the remembered shortcut, re-pointing
* reinstalls, and pre-0.4 shortcuts pointed at the script directly and relied on its exec bit. * it each time — the plugin dir can change across reinstalls, pre-0.4 shortcuts pointed at
* the script directly, and pre-0.7 shortcuts were hidden and artless.
*/ */
async function ensureShortcut(): Promise<{ appId: number; runner: string }> { async function ensureShortcut(): Promise<{ appId: number; runner: string }> {
const info = await runnerInfo(); const info = await runnerInfo();
@@ -105,14 +151,15 @@ async function ensureShortcut(): Promise<{ appId: number; runner: string }> {
SteamClient.Apps.SetShortcutExe(remembered, SHELL); SteamClient.Apps.SetShortcutExe(remembered, SHELL);
SteamClient.Apps.SetShortcutStartDir(remembered, startDir); SteamClient.Apps.SetShortcutStartDir(remembered, startDir);
SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME);
unhideShortcut(remembered); // pre-0.7 installs hid it
void applyArtwork(remembered); // fire-and-forget — cosmetic, never blocks the launch
return { appId: remembered, runner: info.runner }; return { appId: remembered, runner: info.runner };
} }
const appId = await SteamClient.Apps.AddShortcut(SHORTCUT_NAME, SHELL, startDir, ""); const appId = await SteamClient.Apps.AddShortcut(SHORTCUT_NAME, SHELL, startDir, "");
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
// Hide it from the library — it's an implementation detail, launched programmatically. unhideShortcut(appId);
// Best-effort + deferred (see hideShortcut); never let it block the launch. void applyArtwork(appId); // fire-and-forget — cosmetic, never blocks the launch
hideShortcut(appId);
rememberAppId(appId); rememberAppId(appId);
return { appId, runner: info.runner }; return { appId, runner: info.runner };
} }
@@ -137,19 +184,62 @@ function disableSteamInputForShortcut(appId: number): void {
} }
} }
/** Per-launch extras beyond the host target (all optional — {} is the plain stream). */
export interface LaunchOpts {
/** Library id to launch on connect (a pinned game) — rides PF_LAUNCH → `--launch`. */
launchId?: string;
/** Open the gamepad library launcher instead of streaming (PF_BROWSE → `--browse`). */
browse?: boolean;
/** Management-API port for the launcher's library fetch (PF_MGMT; 0/absent = default). */
mgmt?: number;
}
// Launch ids ride Steam launch options as an env-prefix token (`PF_LAUNCH=<id>`), so they
// must be space/quote-free — Steam's tokenizer and the wrapper's env both break otherwise.
// Real ids are `steam:<digits>` / `custom:<slug>`, so this rejects nothing in practice;
// it's VALIDATION, never encoding (the host must match the opaque token verbatim).
const UNSAFE_LAUNCH_ID = /["'\\$`\s]/;
export function isSafeLaunchId(id: string): boolean {
return (
id.length > 0 &&
id.length <= 128 &&
UNSAFE_LAUNCH_ID.exec(id) === null &&
/^[\x21-\x7e]+$/.test(id)
);
}
/** /**
* Launch a stream to `host:port` fullscreen in Gaming Mode. Encodes the target into the * Launch a stream to `host:port` fullscreen in Gaming Mode (optionally straight into a
* shortcut's launch options (so one generic shortcut serves every host), then RunGame. * library title, or into the gamepad library launcher). Encodes the target into the
* shortcut's launch options (so one generic shortcut serves every host and every pinned
* game), then RunGame.
*/ */
export async function launchStream(host: string, port: number): Promise<void> { export async function launchStream(
host: string,
port: number,
opts: LaunchOpts = {},
): Promise<void> {
const { appId, runner } = await ensureShortcut(); const { appId, runner } = await ensureShortcut();
// Best-effort so the Deck's rich controls reach the client; no-op if the API is absent (the user // Best-effort so the Deck's rich controls reach the client; no-op if the API is absent (the user
// disables Steam Input manually — see the Settings instruction). // disables Steam Input manually — see the Settings instruction).
disableSteamInputForShortcut(appId); disableSteamInputForShortcut(appId);
const target = port && port !== 9777 ? `${host}:${port}` : host; const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`];
if (opts.browse) {
env.push("PF_BROWSE=1");
if (opts.mgmt) {
env.push(`PF_MGMT=${Math.floor(opts.mgmt)}`);
}
} else if (opts.launchId) {
if (!isSafeLaunchId(opts.launchId)) {
// Enforced at pin time too (the picker disables Pin) — this is the backstop.
throw new Error(`unsupported launch id: ${opts.launchId}`);
}
env.push(`PF_LAUNCH=${opts.launchId}`);
}
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper // KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_HOST from the environment. // script rides behind it as an argument and reads PF_* from the environment.
SteamClient.Apps.SetAppLaunchOptions(appId, `PF_HOST=${target} %command% "${runner}"`); SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100); SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
} }
+12 -4
View File
@@ -26,6 +26,10 @@ Built in Rust, it links the shared **`punktfunk-core`** directly (no C ABI) and
shows its games (Steam + custom) as a poster grid; click one to launch it in the session. shows its games (Steam + custom) as a poster grid; click one to launch it in the session.
Fetched from the host's management API over mTLS — paired devices are authorized by their Fetched from the host's management API over mTLS — paired devices are authorized by their
certificate, no extra host setup. certificate, no extra host setup.
- **Gamepad library launcher** (`--browse host`) — a console-style, controller-driven coverflow of
a paired host's library (drifting aurora backdrop, center-focus posters, button hints): A plays
the focused title, B quits, L1/R1 jump. Built for the Steam Deck plugin's "Open library" launch;
session end returns to the launcher. Arrow keys/Enter/Esc drive it too (no pad needed).
## Get it ## Get it
@@ -49,24 +53,28 @@ and SDL3 (with hidapi) development packages.
```sh ```sh
# from the repo root # from the repo root
cargo run -p punktfunk-client-linux # launch the app cargo run -p punktfunk-client-linux # launch the app
cargo run -p punktfunk-client-linux -- --discover # list hosts on the LAN, then exit
cargo run -p punktfunk-client-linux -- --connect HOST[:PORT] # skip the host list and connect cargo run -p punktfunk-client-linux -- --connect HOST[:PORT] # skip the host list and connect
cargo run -p punktfunk-client-linux -- --browse HOST # the gamepad library launcher
``` ```
The binary is named **`punktfunk-client`**. Handy flags: `--connect host[:port]` (start a session The binary is named **`punktfunk-client`**. Handy flags: `--connect host[:port]` (start a session
immediately — for scripting and the Steam Deck launcher), `--discover [secs]`, and immediately — for scripting and the Steam Deck launcher) with optional `--launch <id>` (ask the
host to launch that library title, id from `--library`), `--browse host[:port]` (the gamepad
library launcher; `--mgmt <port>` overrides the management port it fetches from),
`--pair <PIN> --connect host[:port]` (run the pairing ceremony headlessly), and `--pair <PIN> --connect host[:port]` (run the pairing ceremony headlessly), and
`--library host[:mgmt_port]` (print a host's game library headlessly). Force a decoder with `--library host[:mgmt_port]` (print a host's game library headlessly). Force a decoder with
`PUNKTFUNK_DECODER=software|vaapi`. `PUNKTFUNK_DECODER=software|vaapi`; `PUNKTFUNK_FAKE_LIBRARY=<file.json>` feeds the launcher
canned entries for UI work with no host.
## Layout ## Layout
``` ```
src/ src/
main.rs · app.rs entry point, GTK application, primary menu, CSS main.rs · app.rs entry point, GTK application, primary menu, CSS
cli.rs CLI paths (--connect, headless --pair, screenshot scenes) cli.rs CLI paths (--connect/--launch, --browse, headless --pair, screenshot scenes)
ui_hosts.rs host card grids (saved + discovered) · add-host dialog · banner ui_hosts.rs host card grids (saved + discovered) · add-host dialog · banner
ui_library.rs game-library poster grid (per-host, launches titles) ui_library.rs game-library poster grid (per-host, launches titles)
ui_gamepad_library.rs the --browse gamepad launcher (aurora · coverflow · hint bar)
ui_trust.rs TOFU / PIN-pairing / request-access dialogs ui_trust.rs TOFU / PIN-pairing / request-access dialogs
ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic
ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture
+81 -6
View File
@@ -22,14 +22,44 @@ const CSS: &str = "
color: alpha(currentColor, 0.8); background: alpha(currentColor, 0.1); } color: alpha(currentColor, 0.8); background: alpha(currentColor, 0.1); }
.pf-pill.pf-green { color: @success_color; background: alpha(@success_color, 0.15); } .pf-pill.pf-green { color: @success_color; background: alpha(@success_color, 0.15); }
.pf-pill.pf-accent { color: @accent_color; background: alpha(@accent_color, 0.15); } .pf-pill.pf-accent { color: @accent_color; background: alpha(@accent_color, 0.15); }
.pf-pill.pf-neutral { color: alpha(currentColor, 0.75); background: alpha(currentColor, 0.12); }
.pf-pip { min-width: 8px; min-height: 8px; border-radius: 999px; .pf-pip { min-width: 8px; min-height: 8px; border-radius: 999px;
background: alpha(currentColor, 0.35); } background: alpha(currentColor, 0.35); }
.pf-pip.pf-online { background: @success_color; } .pf-pip.pf-online { background: @success_color; }
.pf-recent { box-shadow: inset 3px 0 0 0 @accent_bg_color; } /* Most-recent host: a full accent ring drawn as an inset outline so it follows the card's
rounded corners (an `inset` box-shadow bar gets eaten by the 12px corner clip) and leaves
the card's own elevation shadow intact. */
.pf-recent { outline: 2px solid @accent_color; outline-offset: -2px; }
.pf-discovered { border: 1px dashed alpha(currentColor, 0.35); } .pf-discovered { border: 1px dashed alpha(currentColor, 0.35); }
.pf-poster { border-radius: 10px; background: alpha(currentColor, 0.08); } .pf-poster { border-radius: 10px; background: alpha(currentColor, 0.08); }
.pf-poster-monogram { font-size: 2.4em; font-weight: bold; color: alpha(currentColor, 0.45); } .pf-poster-monogram { font-size: 2.4em; font-weight: bold; color: alpha(currentColor, 0.45); }
.pf-store-badge { color: white; background: rgba(0, 0, 0, 0.55); } .pf-store-badge { color: white; background: rgba(0, 0, 0, 0.55); }
/* Gaming-Mode launches: gamescope displays the window fullscreen but never ACKs the
xdg_toplevel fullscreen state, so GTK keeps the floating-CSD styling — libadwaita's
rounded corners + shadow margin stay visible over the stream. Flatten them outright. */
window.pf-chromeless { border-radius: 0; box-shadow: none; }
/* The gamepad library launcher (`--browse`, ui_gamepad_library) — always-dark console
chrome over the aurora, independent of the desktop theme. */
.pf-gl-page { background: black; color: white; }
.pf-gl-host { font-size: 1.15em; font-weight: bold; color: rgba(255, 255, 255, 0.9); }
.pf-gl-chip { font-size: 0.8em; color: rgba(255, 255, 255, 0.7);
background: rgba(255, 255, 255, 0.08);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: 999px; padding: 4px 12px; }
/* Solid face, not glass: coverflow side cards OVERLAP — a translucent card would bleed
the stack through the one on top. */
.pf-gl-poster { border-radius: 16px; background: rgb(30, 30, 37);
border: 1px solid rgba(255, 255, 255, 0.07); }
.pf-gl-dim { background: black; border-radius: 16px; }
.pf-gl-detail-title { font-size: 1.7em; font-weight: bold; color: white; }
.pf-gl-detail-store { font-size: 0.75em; font-weight: 600; letter-spacing: 2px;
color: rgba(255, 255, 255, 0.5); }
.pf-gl-glyph { font-size: 0.85em; font-weight: bold; color: white;
background: rgba(255, 255, 255, 0.14);
border-radius: 999px; min-width: 26px; min-height: 26px; padding: 2px 8px; }
.pf-gl-hint { color: rgba(255, 255, 255, 0.85); }
.pf-gl-status { font-size: 0.85em; color: #ff938a; }
.pf-gl-error-title { font-size: 1.4em; font-weight: bold; color: white; }
"; ";
pub struct App { pub struct App {
@@ -44,9 +74,16 @@ pub struct App {
pub busy: std::cell::Cell<bool>, pub busy: std::cell::Cell<bool>,
/// Steam Deck / Gaming-Mode launch: fullscreen the window (chrome-less) when a stream starts. /// Steam Deck / Gaming-Mode launch: fullscreen the window (chrome-less) when a stream starts.
pub fullscreen: bool, pub fullscreen: bool,
/// Quit when the session ends (Gaming-Mode `--connect` launch): the app IS the stream —
/// exiting ends the Steam "game" so the Deck returns to Gaming Mode instead of stranding
/// the user on the client's own hosts page.
pub quit_on_session_end: bool,
/// The hosts page handle (banner + per-card connecting spinner), set right after the /// The hosts page handle (banner + per-card connecting spinner), set right after the
/// page is built — `None` only during construction. /// page is built — `None` only during construction.
pub hosts: RefCell<Option<Rc<HostsUi>>>, pub hosts: RefCell<Option<Rc<HostsUi>>>,
/// The gamepad library launcher — `Some` only under `--browse`, where it replaces the
/// hosts page as the root (and session end returns here instead of quitting).
pub browse: RefCell<Option<Rc<crate::ui_gamepad_library::LauncherUi>>>,
} }
impl App { impl App {
@@ -58,11 +95,17 @@ impl App {
self.hosts.borrow().clone() self.hosts.borrow().clone()
} }
/// Surface a connect failure on the hosts page banner (toast fallback pre-build). pub fn browse_ui(&self) -> Option<Rc<crate::ui_gamepad_library::LauncherUi>> {
self.browse.borrow().clone()
}
/// Surface a connect failure: the launcher in browse mode, else the hosts page banner
/// (toast fallback pre-build).
pub fn connect_error(&self, msg: &str) { pub fn connect_error(&self, msg: &str) {
match self.hosts_ui() { match (self.browse_ui(), self.hosts_ui()) {
Some(h) => h.show_error(msg), (Some(l), _) => l.show_error(msg),
None => self.toast(msg), (_, Some(h)) => h.show_error(msg),
_ => self.toast(msg),
} }
} }
} }
@@ -104,6 +147,14 @@ fn build_ui(gtk_app: &adw::Application) {
} }
}; };
load_css(); load_css();
// Screenshot scenes must capture settled frames: kill every GTK/libadwaita animation
// (nav-push slides especially — a headless session may starve the frame clock and
// leave a transition frozen mid-flight in the capture).
if crate::cli::shot_scene().is_some() {
if let Some(s) = gtk::Settings::default() {
s.set_gtk_enable_animations(false);
}
}
let nav = adw::NavigationView::new(); let nav = adw::NavigationView::new();
let toasts = adw::ToastOverlay::new(); let toasts = adw::ToastOverlay::new();
@@ -116,6 +167,14 @@ fn build_ui(gtk_app: &adw::Application) {
.content(&toasts) .content(&toasts)
.build(); .build();
let fullscreen = crate::cli::fullscreen_mode();
if fullscreen {
// Chrome-less shell: no CSD rounding/shadow (see CSS — gamescope never ACKs the
// fullscreen state, so GTK would keep them), and ask for fullscreen up front.
window.add_css_class("pf-chromeless");
window.fullscreen();
}
let app = Rc::new(App { let app = Rc::new(App {
window: window.clone(), window: window.clone(),
nav: nav.clone(), nav: nav.clone(),
@@ -124,8 +183,12 @@ fn build_ui(gtk_app: &adw::Application) {
identity, identity,
gamepad: crate::gamepad::GamepadService::start(), gamepad: crate::gamepad::GamepadService::start(),
busy: std::cell::Cell::new(false), busy: std::cell::Cell::new(false),
fullscreen: crate::cli::fullscreen_mode(), fullscreen,
// (`--browse` makes cli_connect_request None — browse mode returns to the
// launcher on session end instead of quitting.)
quit_on_session_end: fullscreen && crate::cli::cli_connect_request().is_some(),
hosts: RefCell::new(None), hosts: RefCell::new(None),
browse: RefCell::new(None),
}); });
// Re-apply the persisted forwarded-controller pin (stable key; the service matches it // Re-apply the persisted forwarded-controller pin (stable key; the service matches it
@@ -138,6 +201,18 @@ fn build_ui(gtk_app: &adw::Application) {
} }
} }
// Browse mode (`--browse host`): the app IS the gamepad library launcher — it becomes
// the ONE root page. No hosts page (whose construction starts the mDNS browse), no
// header-menu actions; `Settings::library_enabled` is deliberately ignored (the flag
// gates the desktop menu item — asking to browse IS the opt-in here).
if let Some((req, paired, mgmt_port)) = crate::cli::cli_browse_request() {
let launcher = crate::ui_gamepad_library::open(app.clone(), req, paired, mgmt_port);
nav.add(&launcher.page);
*app.browse.borrow_mut() = Some(launcher);
window.present();
return;
}
let hosts_ui = Rc::new(crate::ui_hosts::new( let hosts_ui = Rc::new(crate::ui_hosts::new(
app.settings.clone(), app.settings.clone(),
HostsCallbacks { HostsCallbacks {
+85 -20
View File
@@ -84,19 +84,66 @@ pub fn headless_pair(pin: &str) -> glib::ExitCode {
/// already pinned at this address connects silently on its stored pin; an unknown host is /// already pinned at this address connects silently on its stored pin; an unknown host is
/// routed to the PIN ceremony (never a silent TOFU connect — `fp_hex`/`pair_optional` are /// routed to the PIN ceremony (never a silent TOFU connect — `fp_hex`/`pair_optional` are
/// unset, so `initiate_connect`'s manual arm mandates pairing). /// unset, so `initiate_connect`'s manual arm mandates pairing).
///
/// `--launch <id>` asks the host to launch that library title (store-qualified id from
/// `--library`, e.g. `steam:570` — the Decky wrapper's `PF_LAUNCH`); the raw id doubles
/// as the stream title (best-effort — no extra fetch just for a prettier label).
pub fn cli_connect_request() -> Option<ConnectRequest> { pub fn cli_connect_request() -> Option<ConnectRequest> {
if arg_value("--browse").is_some() {
return None; // browse mode owns the session lifecycle (precedence over --connect)
}
let target = std::env::args().skip_while(|a| a != "--connect").nth(1)?; let target = std::env::args().skip_while(|a| a != "--connect").nth(1)?;
let (addr, port) = parse_host_port(&target); let (addr, port) = parse_host_port(&target);
// An unparsable port (`host:notaport`) used to make the whole request `None` → the app
// silently landed on the hosts page with no session and no message. Fall back to the
// native default like the add-host dialog, and say so, instead of doing nothing.
let port = port.unwrap_or_else(|| {
eprintln!("--connect: unparsable port in '{target}', using default 9777");
9777
});
Some(ConnectRequest { Some(ConnectRequest {
name: addr.clone(), name: addr.clone(),
addr, addr,
port: port?, port,
fp_hex: None, fp_hex: None,
pair_optional: false, pair_optional: false,
launch: None, launch: arg_value("--launch").map(|id| (id.clone(), id)),
}) })
} }
/// `--browse host[:port]` — open the gamepad library launcher for that host instead of
/// connecting (the Decky wrapper's `PF_BROWSE`; native port, default 9777). The host must
/// already be paired: the stored pin is what lets the launcher fetch the library and
/// connect silently — no dialog can run under gamescope, so an unpaired target renders
/// the launcher's pair-first scene. Returns the request (name + stored fingerprint from
/// the known-hosts store), whether it's paired, and the mgmt port (`--mgmt <port>`, the
/// wrapper's `PF_MGMT`; default 47990 — browse mode runs no mDNS to learn it).
pub fn cli_browse_request() -> Option<(ConnectRequest, bool, u16)> {
let target = arg_value("--browse")?;
let (addr, port) = parse_host_port(&target);
let port = port.unwrap_or(9777);
let known = crate::trust::KnownHosts::load();
let k = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let mgmt = arg_value("--mgmt")
.and_then(|p| p.parse().ok())
.unwrap_or(crate::library::DEFAULT_MGMT_PORT);
Some((
ConnectRequest {
name: k.map_or_else(|| addr.clone(), |k| k.name.clone()),
addr,
port,
fp_hex: k.map(|k| k.fp_hex.clone()),
pair_optional: false,
launch: None,
},
k.is_some_and(|k| k.paired),
mgmt,
))
}
/// `--library host[:mgmt_port]` — fetch and print the host's game library over the real /// `--library host[:mgmt_port]` — fetch and print the host's game library over the real
/// mTLS + pinned-fingerprint client, no GTK window (scripting, and the live-API proof /// mTLS + pinned-fingerprint client, no GTK window (scripting, and the live-API proof
/// that the library HTTP path works against a real host). The pin comes from `--fp HEX` /// that the library HTTP path works against a real host). The pin comes from `--fp HEX`
@@ -219,26 +266,17 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
// no-art placeholders (monogram tiles), and one solid-color texture standing in // no-art placeholders (monogram tiles), and one solid-color texture standing in
// for a loaded poster (the real poster path, minus the network). // for a loaded poster (the real poster path, minus the network).
"library" | "08-library" => { "library" | "08-library" => {
let game = |id: &str, store: &str, title: &str| crate::library::GameEntry { let (games, art) = mock_library();
id: id.to_string(),
store: store.to_string(),
title: title.to_string(),
art: crate::library::Artwork::default(),
};
let games = vec![
game("steam:570", "steam", "Dota 2"),
game("steam:1091500", "steam", "Cyberpunk 2077"),
game("custom:emu-1", "custom", "RetroArch"),
game("heroic:fortnite", "heroic", "Fortnite"),
game("gog:witcher3", "gog", "The Witcher 3"),
game("lutris:osu", "lutris", "osu!"),
];
let art = vec![(
"steam:570".to_string(),
solid_texture(300, 450, 0x35, 0x84, 0xe4),
)];
crate::ui_library::open_mock(app.clone(), mock_req(), games, art); crate::ui_library::open_mock(app.clone(), mock_req(), games, art);
} }
// The gamepad launcher (`--browse`) with the same injected entries — cursor sits
// at 1 so both recede directions show; aurora + easing render frozen (shot mode).
"gamepad-library" | "09-gamepad-library" => {
let (games, art) = mock_library();
let ui = crate::ui_gamepad_library::open_mock(app.clone(), mock_req(), games, art);
app.nav.push(&ui.page);
*app.browse.borrow_mut() = Some(ui);
}
other => tracing::warn!("unknown PUNKTFUNK_SHOT_SCENE={other:?}; showing hosts only"), other => tracing::warn!("unknown PUNKTFUNK_SHOT_SCENE={other:?}; showing hosts only"),
} }
@@ -268,6 +306,33 @@ pub fn run_shot(app: Rc<App>, scene: &str) {
}); });
} }
/// The mock game set shared by the `library` and `gamepad-library` scenes: mixed stores
/// exercising the badge set, plus one solid-colour poster texture.
fn mock_library() -> (
Vec<crate::library::GameEntry>,
Vec<(String, gtk::gdk::Texture)>,
) {
let game = |id: &str, store: &str, title: &str| crate::library::GameEntry {
id: id.to_string(),
store: store.to_string(),
title: title.to_string(),
art: crate::library::Artwork::default(),
};
let games = vec![
game("steam:570", "steam", "Dota 2"),
game("steam:1091500", "steam", "Cyberpunk 2077"),
game("custom:emu-1", "custom", "RetroArch"),
game("heroic:fortnite", "heroic", "Fortnite"),
game("gog:witcher3", "gog", "The Witcher 3"),
game("lutris:osu", "lutris", "osu!"),
];
let art = vec![(
"steam:570".to_string(),
solid_texture(300, 450, 0x35, 0x84, 0xe4),
)];
(games, art)
}
/// A WxH single-colour RGBA texture — the `library` scene's stand-in for a fetched poster. /// A WxH single-colour RGBA texture — the `library` scene's stand-in for a fetched poster.
fn solid_texture(w: i32, h: i32, r: u8, g: u8, b: u8) -> gtk::gdk::Texture { fn solid_texture(w: i32, h: i32, r: u8, g: u8, b: u8) -> gtk::gdk::Texture {
let px = [r, g, b, 0xff].repeat((w * h) as usize); let px = [r, g, b, 0xff].repeat((w * h) as usize);
+494 -10
View File
@@ -18,6 +18,17 @@
//! idle host-list window would kill the Deck's system input. The pad list for Settings is //! idle host-list window would kill the Deck's system input. The pad list for Settings is
//! built from SDL's ID-based metadata getters, which need no open. //! built from SDL's ID-based metadata getters, which need no open.
//! //!
//! **Menu mode is the one idle exception.** The gamepad library launcher (`--browse`)
//! flips [`GamepadService::set_menu_mode`] on for its lifetime: the worker then holds the
//! active pad open and translates its buttons/stick into [`MenuEvent`]s (polled off the
//! open handle each loop — Apple `GamepadMenuInput` parity: edge-triggered buttons,
//! snapshot-on-entry so a button still held from a previous screen or stream can't ghost-
//! fire, stick/dpad direction with initial-delay auto-repeat). The Valve HIDAPI drivers
//! stay OFF — a plain SDL open of the virtual X360 / evdev pad doesn't touch lizard mode —
//! and an attached session always supersedes menu translation (the stream path is
//! untouched); detach re-snapshots so the escape chord that ended the session fires
//! nothing in the menu.
//!
//! This thread is also the single consumer of the rumble and HID-output pull planes. //! This thread is also the single consumer of the rumble and HID-output pull planes.
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
@@ -50,6 +61,169 @@ const ESCAPE_CHORD: [u32; 4] = [wire::BTN_LB, wire::BTN_RB, wire::BTN_START, wir
/// Hold the [`ESCAPE_CHORD`] at least this long to disconnect (escalates the leave-fullscreen press). /// Hold the [`ESCAPE_CHORD`] at least this long to disconnect (escalates the leave-fullscreen press).
const DISCONNECT_HOLD: Duration = Duration::from_millis(1500); const DISCONNECT_HOLD: Duration = Duration::from_millis(1500);
/// Stick deflection below this is ignored for menu navigation (0.5 of full scale — Apple
/// `GamepadMenuInput` parity; menus want deliberate flicks, not drift).
const MENU_DEADZONE: u16 = 16384;
/// A held direction starts auto-repeating after this initial delay…
const MENU_REPEAT_DELAY: Duration = Duration::from_millis(380);
/// …and then repeats at this cadence until released or changed.
const MENU_REPEAT_INTERVAL: Duration = Duration::from_millis(160);
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MenuDir {
Up,
Down,
Left,
Right,
}
/// One controller action for the launcher UI, translated from the open pad while menu
/// mode is on and no session is attached. Buttons are edge-triggered; `Move` debounces
/// the stick/dpad and auto-repeats ([`MENU_REPEAT_DELAY`]/[`MENU_REPEAT_INTERVAL`]).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MenuEvent {
Move(MenuDir),
/// A — activate the focused item.
Confirm,
/// B — back / quit.
Back,
/// Y (Apple "secondary"; unused by the launcher today, kept for parity).
Secondary,
/// X (Apple "tertiary"; unused).
Tertiary,
/// L1 — jump back 5.
JumpBack,
/// R1 — jump forward 5.
JumpForward,
}
/// Menu haptic pulses — short rumble ticks on the menu pad (never during a stream).
#[derive(Clone, Copy, Debug)]
pub enum MenuPulse {
Move,
Confirm,
Boundary,
}
/// Raw pad state sampled once per worker iteration for menu translation.
#[derive(Clone, Copy, Default)]
struct MenuSample {
/// a, b, x, y, l1, r1 — the order [`MenuNav::poll`] maps to events.
buttons: [bool; 6],
/// Left stick, SDL convention (+y = down).
lx: i16,
ly: i16,
/// up, down, left, right.
dpad: [bool; 4],
}
/// The pure menu-input state machine (no SDL types — unit-tested below). Port of the
/// Swift client's `GamepadMenuInput`: the poll after a [`reset`](Self::reset) adopts the
/// currently-held buttons and direction WITHOUT firing, so a press that crossed a screen
/// handoff (the B that closed a stream, a held A on mode entry) must be released before
/// it can act; buttons fire on the rising edge only.
struct MenuNav {
/// Adopt the next sample silently (set on mode entry / stream detach / pad change).
snapshot_pending: bool,
/// Previous button states, [`MenuSample::buttons`] order.
was: [bool; 6],
dir: Option<MenuDir>,
/// When `dir` engaged — start of the initial-repeat delay.
dir_since: Instant,
last_repeat: Instant,
}
impl MenuNav {
fn new() -> MenuNav {
MenuNav {
snapshot_pending: true,
was: [false; 6],
dir: None,
dir_since: Instant::now(),
last_repeat: Instant::now(),
}
}
/// Arm the snapshot: the next poll adopts held state without firing.
fn reset(&mut self) {
self.snapshot_pending = true;
self.dir = None;
}
/// Direction from the left stick (dominant axis wins past the deadzone), falling back
/// to the discrete dpad. SDL sticks are +y = down.
fn resolve_dir(s: &MenuSample) -> Option<MenuDir> {
let (ax, ay) = (s.lx.unsigned_abs(), s.ly.unsigned_abs());
if ax > MENU_DEADZONE || ay > MENU_DEADZONE {
return Some(if ax >= ay {
if s.lx > 0 {
MenuDir::Right
} else {
MenuDir::Left
}
} else if s.ly > 0 {
MenuDir::Down
} else {
MenuDir::Up
});
}
let [up, down, left, right] = s.dpad;
if left {
Some(MenuDir::Left)
} else if right {
Some(MenuDir::Right)
} else if up {
Some(MenuDir::Up)
} else if down {
Some(MenuDir::Down)
} else {
None
}
}
fn poll(&mut self, s: &MenuSample, now: Instant, out: &mut Vec<MenuEvent>) {
let dir = Self::resolve_dir(s);
if self.snapshot_pending {
self.snapshot_pending = false;
self.was = s.buttons;
self.dir = dir;
self.dir_since = now;
self.last_repeat = now;
return;
}
// buttons order a, b, x, y, l1, r1 → the matching event per index.
const EVENTS: [MenuEvent; 6] = [
MenuEvent::Confirm,
MenuEvent::Back,
MenuEvent::Tertiary,
MenuEvent::Secondary,
MenuEvent::JumpBack,
MenuEvent::JumpForward,
];
for (i, ev) in EVENTS.iter().enumerate() {
if s.buttons[i] && !self.was[i] {
out.push(*ev);
}
self.was[i] = s.buttons[i];
}
if dir != self.dir {
self.dir = dir;
self.dir_since = now;
self.last_repeat = now;
if let Some(d) = dir {
out.push(MenuEvent::Move(d));
}
} else if let Some(d) = dir {
if now.duration_since(self.dir_since) >= MENU_REPEAT_DELAY
&& now.duration_since(self.last_repeat) >= MENU_REPEAT_INTERVAL
{
self.last_repeat = now;
out.push(MenuEvent::Move(d));
}
}
}
}
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct PadInfo { pub struct PadInfo {
pub name: String, pub name: String,
@@ -114,10 +288,27 @@ fn pref_for_type(t: sdl3::gamepad::GamepadType) -> GamepadPref {
} }
} }
/// Best-effort "this machine is a Steam Deck". The Gaming-Mode env short-circuits; desktop
/// mode falls back to DMI (Valve board, Jupiter = LCD / Galileo = OLED — readable inside the
/// flatpak sandbox). Cached: the answer can't change while we run.
pub fn is_steam_deck() -> bool {
static DECK: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*DECK.get_or_init(|| {
if std::env::var_os("SteamDeck").is_some() {
return true;
}
let dmi = |f: &str| std::fs::read_to_string(format!("/sys/class/dmi/id/{f}"));
dmi("board_vendor").is_ok_and(|v| v.trim() == "Valve")
&& dmi("product_name").is_ok_and(|p| matches!(p.trim(), "Jupiter" | "Galileo"))
})
}
enum Ctl { enum Ctl {
Attach(Arc<NativeClient>), Attach(Arc<NativeClient>),
Detach, Detach,
Pin(Option<String>), Pin(Option<String>),
MenuMode(bool),
MenuRumble(MenuPulse),
} }
#[derive(Clone)] #[derive(Clone)]
@@ -131,6 +322,9 @@ pub struct GamepadService {
/// Fires once when the [`ESCAPE_CHORD`] is held past [`DISCONNECT_HOLD`]; the stream page /// Fires once when the [`ESCAPE_CHORD`] is held past [`DISCONNECT_HOLD`]; the stream page
/// consumes it to end the session (the controller equivalent of Ctrl+Alt+Shift+D). /// consumes it to end the session (the controller equivalent of Ctrl+Alt+Shift+D).
disconnect_rx: async_channel::Receiver<()>, disconnect_rx: async_channel::Receiver<()>,
/// Menu-navigation events while menu mode is on and no session is attached; the
/// launcher page consumes them.
menu_rx: async_channel::Receiver<MenuEvent>,
} }
impl GamepadService { impl GamepadService {
@@ -140,11 +334,12 @@ impl GamepadService {
let (ctl, ctl_rx) = std::sync::mpsc::channel(); let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (escape_tx, escape_rx) = async_channel::unbounded(); let (escape_tx, escape_rx) = async_channel::unbounded();
let (disconnect_tx, disconnect_rx) = async_channel::unbounded(); let (disconnect_tx, disconnect_rx) = async_channel::unbounded();
let (menu_tx, menu_rx) = async_channel::unbounded();
let (p, a) = (pads.clone(), active.clone()); let (p, a) = (pads.clone(), active.clone());
if let Err(e) = std::thread::Builder::new() if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into()) .name("punktfunk-gamepad".into())
.spawn(move || { .spawn(move || {
if let Err(e) = run(&p, &a, &ctl_rx, &escape_tx, &disconnect_tx) { if let Err(e) = run(&p, &a, &ctl_rx, &escape_tx, &disconnect_tx, &menu_tx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled"); tracing::warn!(error = %e, "gamepad service ended — pads disabled");
} }
}) })
@@ -157,6 +352,7 @@ impl GamepadService {
ctl, ctl,
escape_rx, escape_rx,
disconnect_rx, disconnect_rx,
menu_rx,
} }
} }
@@ -172,6 +368,25 @@ impl GamepadService {
self.disconnect_rx.clone() self.disconnect_rx.clone()
} }
/// Menu-navigation events ([`MenuEvent`]) — flowing only while menu mode is on and no
/// session is attached. A fresh clone per call; the launcher spawns a future on it.
pub fn menu_events(&self) -> async_channel::Receiver<MenuEvent> {
self.menu_rx.clone()
}
/// Turn menu mode on/off: while on (and no session attached) the worker holds the
/// active pad open and translates it into [`MenuEvent`]s. The launcher flips this on
/// once for its lifetime — an attached session supersedes translation automatically.
pub fn set_menu_mode(&self, on: bool) {
let _ = self.ctl.send(Ctl::MenuMode(on));
}
/// Play a short menu haptic tick on the menu pad (no-op while a session is attached
/// or no pad is open; best-effort on pads without rumble).
pub fn menu_rumble(&self, pulse: MenuPulse) {
let _ = self.ctl.send(Ctl::MenuRumble(pulse));
}
pub fn pads(&self) -> Vec<PadInfo> { pub fn pads(&self) -> Vec<PadInfo> {
self.pads.lock().unwrap().clone() self.pads.lock().unwrap().clone()
} }
@@ -197,8 +412,19 @@ impl GamepadService {
/// What "Automatic" resolves to right now — the virtual pad matching the physical one /// What "Automatic" resolves to right now — the virtual pad matching the physical one
/// (Swift parity); no pad connected leaves the host's own default. /// (Swift parity); no pad connected leaves the host's own default.
///
/// **Steam Deck special case:** this is read at session start, *before* attach — but the
/// Deck's built-in controller is only enumerable with its real 28DE:1205 identity while
/// the Valve HIDAPI drivers run, and those are enabled on attach only (see
/// [`set_valve_hidapi`]); with Steam Input on, SDL sees nothing but Steam's virtual
/// X360 pad anyway. Both cases used to fall through to Xbox 360. On a Deck, a virtual
/// pad (or no pad at all) means the physical controller behind it IS the built-in one —
/// resolve to the Steam Deck virtual pad so the paddles/trackpads/gyro have somewhere
/// to land. A real external controller still wins (it's the one that gets forwarded).
pub fn auto_pref(&self) -> GamepadPref { pub fn auto_pref(&self) -> GamepadPref {
match self.active() { match self.active() {
Some(p) if !p.steam_virtual => p.pref,
_ if is_steam_deck() => GamepadPref::SteamDeck,
Some(p) => p.pref, Some(p) => p.pref,
None => GamepadPref::Auto, None => GamepadPref::Auto,
} }
@@ -337,6 +563,11 @@ struct Worker<'a> {
chord_since: Option<Instant>, chord_since: Option<Instant>,
/// The disconnect signal already fired for the current hold — latched so it fires once. /// The disconnect signal already fired for the current hold — latched so it fires once.
disconnect_fired: bool, disconnect_fired: bool,
/// Menu mode ([`GamepadService::set_menu_mode`]): hold the active pad open while idle
/// and translate it into [`MenuEvent`]s. An attached session pauses translation.
menu_mode: bool,
menu_nav: MenuNav,
menu_tx: async_channel::Sender<MenuEvent>,
} }
impl Worker<'_> { impl Worker<'_> {
@@ -395,12 +626,12 @@ impl Worker<'_> {
}) })
} }
/// Hold exactly the right device: the active pad while a session is attached, nothing /// Hold exactly the right device: the active pad while a session is attached or menu
/// otherwise. The single place that decides to open (= grab) hardware; dropping the /// mode owns navigation, nothing otherwise. The single place that decides to open
/// old handle closes it (`SDL_CloseGamepad`) — on a Deck the firmware watchdog then /// (= grab) hardware; dropping the old handle closes it (`SDL_CloseGamepad`) — on a
/// restores lizard mode. /// Deck the firmware watchdog then restores lizard mode.
fn sync_open(&mut self) { fn sync_open(&mut self) {
let want = if self.attached.is_some() { let want = if self.attached.is_some() || self.menu_mode {
self.active_id() self.active_id()
} else { } else {
None None
@@ -413,7 +644,15 @@ impl Worker<'_> {
match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) { match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) {
Ok(pad) => { Ok(pad) => {
self.open = Some((id, pad)); self.open = Some((id, pad));
// Sensors stream only for an attached session (USB/BT bandwidth); the
// menu needs buttons + stick only.
if self.attached.is_some() {
self.set_sensors(true); self.set_sensors(true);
} else {
// The menu pad changed under us (hot-plug while the launcher is
// open): adopt the new pad's held state instead of firing it.
self.menu_nav.reset();
}
} }
Err(e) => tracing::warn!(id, error = %e, "gamepad open failed"), Err(e) => tracing::warn!(id, error = %e, "gamepad open failed"),
} }
@@ -619,14 +858,42 @@ impl Worker<'_> {
Ok(Ctl::Detach) => { Ok(Ctl::Detach) => {
self.flush_held(); self.flush_held();
self.attached = None; self.attached = None;
self.sync_open(); // closes the held device self.sync_open(); // closes the held device (menu mode keeps it)
set_valve_hidapi(false); set_valve_hidapi(false);
if self.menu_mode {
// Back to the launcher: adopt whatever is still physically held
// (the escape chord that ended the session, a lingering B) so it
// can't ghost-fire menu actions.
self.menu_nav.reset();
}
} }
Ok(Ctl::Pin(key)) => { Ok(Ctl::Pin(key)) => {
let before = self.active_id(); let before = self.active_id();
self.pinned = key; self.pinned = key;
self.refresh_active(before); self.refresh_active(before);
} }
Ok(Ctl::MenuMode(on)) => {
self.menu_mode = on;
if on {
self.menu_nav.reset();
}
self.sync_open();
}
Ok(Ctl::MenuRumble(pulse)) => {
if self.attached.is_none() {
if let Some((_, pad)) = self.open.as_mut() {
let (low, high, ms) = match pulse {
// Light high-freq detent — won't jackhammer at repeat rate.
MenuPulse::Move => (0, 0x3000, 25),
// Fuller both-motor thunk.
MenuPulse::Confirm => (0x5000, 0x5000, 60),
// Dull low-freq wall.
MenuPulse::Boundary => (0x6000, 0, 60),
};
let _ = pad.set_rumble(low, high, ms);
}
}
}
Err(std::sync::mpsc::TryRecvError::Empty) => return true, Err(std::sync::mpsc::TryRecvError::Empty) => return true,
Err(std::sync::mpsc::TryRecvError::Disconnected) => return false, // app gone Err(std::sync::mpsc::TryRecvError::Disconnected) => return false, // app gone
} }
@@ -643,7 +910,16 @@ impl Worker<'_> {
if !self.order.contains(&which) { if !self.order.contains(&which) {
self.order.push(which); self.order.push(which);
if let Some(p) = self.pad_info(which) { if let Some(p) = self.pad_info(which) {
tracing::info!(name = p.name, "gamepad attached"); // Full identity: on a Steam Deck this is the one lever for diagnosing an
// empty controller list — it tells you whether SDL sees the physical pad
// (28DE:1205), Steam Input's virtual pad (28DE:11FF), both, or nothing.
tracing::info!(
name = p.name,
key = p.key,
pref = ?p.pref,
steam_virtual = p.steam_virtual,
"gamepad attached"
);
} }
self.refresh_active(active); self.refresh_active(active);
} }
@@ -758,6 +1034,42 @@ impl Worker<'_> {
} }
} }
/// Sample the open pad and translate it into [`MenuEvent`]s — only while menu mode is
/// on and no session is attached (attach supersedes; SDL events merely wake the loop,
/// so a press is translated the iteration it arrives).
fn menu_poll(&mut self) {
if !self.menu_mode || self.attached.is_some() {
return;
}
let Some((_, pad)) = self.open.as_ref() else {
return;
};
use sdl3::gamepad::{Axis, Button};
let s = MenuSample {
buttons: [
pad.button(Button::South),
pad.button(Button::East),
pad.button(Button::West),
pad.button(Button::North),
pad.button(Button::LeftShoulder),
pad.button(Button::RightShoulder),
],
lx: pad.axis(Axis::LeftX),
ly: pad.axis(Axis::LeftY),
dpad: [
pad.button(Button::DPadUp),
pad.button(Button::DPadDown),
pad.button(Button::DPadLeft),
pad.button(Button::DPadRight),
],
};
let mut out = Vec::new();
self.menu_nav.poll(&s, Instant::now(), &mut out);
for e in out {
let _ = self.menu_tx.try_send(e);
}
}
/// Drain and render the feedback planes — rumble plus HID output (lightbar / /// Drain and render the feedback planes — rumble plus HID output (lightbar /
/// player LEDs / adaptive triggers) — on the active pad; this thread is their single /// player LEDs / adaptive triggers) — on the active pad; this thread is their single
/// consumer. The host re-sends rumble state periodically, so a generous duration with /// consumer. The host re-sends rumble state periodically, so a generous duration with
@@ -821,6 +1133,7 @@ fn run(
ctl: &Receiver<Ctl>, ctl: &Receiver<Ctl>,
escape_tx: &async_channel::Sender<()>, escape_tx: &async_channel::Sender<()>,
disconnect_tx: &async_channel::Sender<()>, disconnect_tx: &async_channel::Sender<()>,
menu_tx: &async_channel::Sender<MenuEvent>,
) -> Result<(), String> { ) -> Result<(), String> {
// Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its // Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its
// own thread. // own thread.
@@ -851,6 +1164,9 @@ fn run(
chord_armed: false, chord_armed: false,
chord_since: None, chord_since: None,
disconnect_fired: false, disconnect_fired: false,
menu_mode: false,
menu_nav: MenuNav::new(),
menu_tx: menu_tx.clone(),
}; };
loop { loop {
@@ -865,8 +1181,13 @@ fn run(
// rumble/HID feedback, and the escape-chord hold check all run once per wakeup, // rumble/HID feedback, and the escape-chord hold check all run once per wakeup,
// so their worst case is one timeout (~10 ms attached, imperceptible for // so their worst case is one timeout (~10 ms attached, imperceptible for
// haptics; DISCONNECT_HOLD is 1500 ms, so 10 ms hold-check granularity is far // haptics; DISCONNECT_HOLD is 1500 ms, so 10 ms hold-check granularity is far
// inside tolerance). Idle (no session) wakes lazily at 30 ms for hotplug + ctl. // inside tolerance; menu mode needs the same cadence for its repeat timing).
let timeout = Duration::from_millis(if w.attached.is_some() { 10 } else { 30 }); // Idle (no session, no menu) wakes lazily at 30 ms for hotplug + ctl.
let timeout = Duration::from_millis(if w.attached.is_some() || w.menu_mode {
10
} else {
30
});
if let Some(event) = pump.wait_event_timeout(timeout) { if let Some(event) = pump.wait_event_timeout(timeout) {
w.handle_event(event); w.handle_event(event);
// Drain whatever else queued while we were waiting or handling. // Drain whatever else queued while we were waiting or handling.
@@ -879,6 +1200,169 @@ fn run(
// new button events; the chord itself is only detected while a session is attached). // new button events; the chord itself is only detected while a session is attached).
w.maybe_fire_disconnect(); w.maybe_fire_disconnect();
w.menu_poll();
w.render_feedback(); w.render_feedback();
} }
} }
#[cfg(test)]
mod menu_nav_tests {
use super::*;
fn sample() -> MenuSample {
MenuSample::default()
}
fn events(nav: &mut MenuNav, s: &MenuSample, at: Instant) -> Vec<MenuEvent> {
let mut out = Vec::new();
nav.poll(s, at, &mut out);
out
}
#[test]
fn snapshot_adopts_held_state_without_firing() {
let mut nav = MenuNav::new();
let t = Instant::now();
let mut held = sample();
held.buttons[0] = true; // A held on entry
held.lx = 30000; // stick already deflected right
assert!(events(&mut nav, &held, t).is_empty(), "snapshot poll fired");
// Still held: nothing (no rising edge, direction unchanged since snapshot).
assert!(events(&mut nav, &held, t + Duration::from_millis(10)).is_empty());
// Release, then press again → now it fires.
assert!(events(&mut nav, &sample(), t + Duration::from_millis(20)).is_empty());
assert_eq!(
events(&mut nav, &held, t + Duration::from_millis(30)),
vec![MenuEvent::Confirm, MenuEvent::Move(MenuDir::Right)]
);
}
#[test]
fn buttons_fire_on_rising_edge_only() {
let mut nav = MenuNav::new();
let t = Instant::now();
events(&mut nav, &sample(), t); // consume the snapshot
let mut s = sample();
s.buttons[1] = true; // B down
assert_eq!(
events(&mut nav, &s, t + Duration::from_millis(10)),
vec![MenuEvent::Back]
);
for i in 2..20 {
assert!(
events(&mut nav, &s, t + Duration::from_millis(10 * i)).is_empty(),
"held button re-fired"
);
}
}
#[test]
fn reset_rearms_the_snapshot() {
let mut nav = MenuNav::new();
let t = Instant::now();
events(&mut nav, &sample(), t);
nav.reset();
let mut s = sample();
s.buttons[1] = true;
assert!(
events(&mut nav, &s, t + Duration::from_millis(10)).is_empty(),
"post-reset poll fired a held button"
);
}
#[test]
fn direction_repeats_after_delay_at_interval() {
let mut nav = MenuNav::new();
let t = Instant::now();
events(&mut nav, &sample(), t);
let mut s = sample();
s.dpad[3] = true; // dpad right
// Engage: fires immediately.
assert_eq!(
events(&mut nav, &s, t + Duration::from_millis(10)),
vec![MenuEvent::Move(MenuDir::Right)]
);
// Inside the initial delay: silent.
assert!(events(&mut nav, &s, t + Duration::from_millis(300)).is_empty());
// Past the delay: repeats…
assert_eq!(
events(&mut nav, &s, t + Duration::from_millis(400)),
vec![MenuEvent::Move(MenuDir::Right)]
);
// …but not faster than the interval…
assert!(events(&mut nav, &s, t + Duration::from_millis(500)).is_empty());
// …and again once it elapses.
assert_eq!(
events(&mut nav, &s, t + Duration::from_millis(570)),
vec![MenuEvent::Move(MenuDir::Right)]
);
// Release cancels; re-engage fires immediately again.
assert!(events(&mut nav, &sample(), t + Duration::from_millis(580)).is_empty());
assert_eq!(
events(&mut nav, &s, t + Duration::from_millis(590)),
vec![MenuEvent::Move(MenuDir::Right)]
);
}
#[test]
fn direction_change_fires_immediately() {
let mut nav = MenuNav::new();
let t = Instant::now();
events(&mut nav, &sample(), t);
let mut right = sample();
right.lx = 30000;
let mut left = sample();
left.lx = -30000;
assert_eq!(
events(&mut nav, &right, t + Duration::from_millis(10)),
vec![MenuEvent::Move(MenuDir::Right)]
);
assert_eq!(
events(&mut nav, &left, t + Duration::from_millis(20)),
vec![MenuEvent::Move(MenuDir::Left)]
);
}
#[test]
fn direction_resolution() {
// Below the deadzone: nothing.
let mut s = sample();
s.lx = MENU_DEADZONE as i16;
assert_eq!(MenuNav::resolve_dir(&s), None);
// Dominant axis wins; SDL +y = down.
s.lx = 20000;
s.ly = 25000;
assert_eq!(MenuNav::resolve_dir(&s), Some(MenuDir::Down));
s.ly = -25000;
assert_eq!(MenuNav::resolve_dir(&s), Some(MenuDir::Up));
s.lx = 26000;
assert_eq!(MenuNav::resolve_dir(&s), Some(MenuDir::Right));
s.lx = -26000;
assert_eq!(MenuNav::resolve_dir(&s), Some(MenuDir::Left));
// Dpad fallback…
let mut d = sample();
d.dpad[1] = true;
assert_eq!(MenuNav::resolve_dir(&d), Some(MenuDir::Down));
// …but the stick overrides it.
d.lx = 30000;
assert_eq!(MenuNav::resolve_dir(&d), Some(MenuDir::Right));
}
#[test]
fn shoulder_and_face_button_mapping() {
let mut nav = MenuNav::new();
let t = Instant::now();
events(&mut nav, &sample(), t);
let mut s = sample();
s.buttons = [false, false, true, true, true, true]; // x, y, l1, r1
assert_eq!(
events(&mut nav, &s, t + Duration::from_millis(10)),
vec![
MenuEvent::Tertiary,
MenuEvent::Secondary,
MenuEvent::JumpBack,
MenuEvent::JumpForward,
]
);
}
}
+35 -4
View File
@@ -266,6 +266,9 @@ impl SessionUi {
inhibit_shortcuts: self.inhibit, inhibit_shortcuts: self.inhibit,
show_stats: self.show_stats, show_stats: self.show_stats,
chromeless: self.app.fullscreen, chromeless: self.app.fullscreen,
// The attach just went out, so a Deck's built-in pad may not have enumerated
// yet — chromeless (controller-first) shows the chord hint regardless.
pad_connected: self.app.gamepad.active().is_some(),
title, title,
}); });
self.app.nav.push(&p.page); self.app.nav.push(&p.page);
@@ -296,21 +299,49 @@ impl SessionUi {
} }
// A pinned connect rejected on trust grounds means the host's cert no // A pinned connect rejected on trust grounds means the host's cert no
// longer matches the stored pin (rotated cert or impostor) — route to // longer matches the stored pin (rotated cert or impostor) — route to
// the PIN ceremony to re-establish trust rather than dead-ending. // the PIN ceremony to re-establish trust rather than dead-ending. Browse
if trust_rejected && !self.tofu { // mode can't: gamescope never maps dialogs, so it renders the advice instead
// (re-pairing is the plugin's job there).
if trust_rejected && !self.tofu && self.app.browse_ui().is_none() {
self.app self.app
.toast("Host fingerprint changed — re-pair with a PIN to continue"); .toast("Host fingerprint changed — re-pair with a PIN to continue");
crate::ui_trust::pin_dialog(self.app.clone(), self.req.clone()); crate::ui_trust::pin_dialog(self.app.clone(), self.req.clone());
} else if trust_rejected && !self.tofu {
self.app
.connect_error("Host identity changed — re-pair from the Punktfunk plugin.");
} else { } else {
// Errors land on the hosts page banner, not a transient toast. // Errors land on the hosts page banner / launcher strip, not a transient toast.
self.app.connect_error(&format!("Couldn't connect — {msg}")); self.app.connect_error(&format!("Couldn't connect — {msg}"));
} }
} }
/// `Ended`: detach gamepads, pop back to the hosts page, and surface the reason. /// `Ended`: detach gamepads, pop back to the launcher (browse mode) or the hosts
/// page, and surface the reason.
fn on_ended(&mut self, err: Option<String>) { fn on_ended(&mut self, err: Option<String>) {
self.close_waiting(); self.close_waiting();
self.app.gamepad.detach(); self.app.gamepad.detach();
// Gaming-Mode `--connect` launch: the app IS the stream. Quit so Steam ends the
// "game" and the Deck returns to Gaming Mode — popping to our own hosts page would
// strand the user in a fullscreen shell with no way back.
if self.app.quit_on_session_end {
if let Some(e) = err {
tracing::warn!(error = %e, "session ended");
}
self.app.window.close();
return;
}
// Browse mode: back to the launcher to pick the next game — B there quits to
// Gaming Mode. (The gamepad worker re-opened the pad and armed the held-state
// snapshot on the detach above, so the chord that ended the session fires nothing.)
if let Some(l) = self.app.browse_ui() {
self.app.nav.pop_to_tag("launcher");
l.on_session_ended();
if let Some(e) = err {
self.app.connect_error(&e);
}
self.app.busy.set(false);
return;
}
self.app.nav.pop_to_tag("hosts"); self.app.nav.pop_to_tag("hosts");
if let Some(h) = self.app.hosts_ui() { if let Some(h) = self.app.hosts_ui() {
h.set_connecting(None); h.set_connecting(None);
+53 -1
View File
@@ -6,8 +6,9 @@
//! verified by its pinned SHA-256 fingerprint (`KnownHost::fp_hex`), not a CA chain. //! verified by its pinned SHA-256 fingerprint (`KnownHost::fp_hex`), not a CA chain.
use serde::Deserialize; use serde::Deserialize;
use std::collections::VecDeque;
use std::io::Read; use std::io::Read;
use std::sync::Arc; use std::sync::{Arc, Mutex};
use std::time::Duration; use std::time::Duration;
/// The management API's default port — matches `mgmt::DEFAULT_PORT` on the host. A /// The management API's default port — matches `mgmt::DEFAULT_PORT` on the host. A
@@ -181,6 +182,57 @@ pub fn fetch_art(pinned: &ureq::Agent, base: &str, url: &str) -> Result<Vec<u8>,
Ok(bytes) Ok(bytes)
} }
/// Concurrent poster fetches — a handful is plenty for a LAN art proxy without turning a
/// big library into a connection burst.
const ART_WORKERS: usize = 3;
/// Fetch poster bytes for `jobs` (entry id → candidate URLs, walked in order until one
/// loads) on a small worker pool; results stream on the returned channel as they land.
/// Dropping the receiver (the consuming page popped) winds the workers down. Shared by
/// the touch grid and the gamepad launcher — the consumer does its own texture decode on
/// the main loop.
pub fn spawn_art_fetch(
base: String,
identity: (String, String),
pin: Option<[u8; 32]>,
jobs: VecDeque<(String, Vec<String>)>,
) -> async_channel::Receiver<(String, Vec<u8>)> {
let queue = Arc::new(Mutex::new(jobs));
let (tx, rx) = async_channel::unbounded::<(String, Vec<u8>)>();
for _ in 0..ART_WORKERS {
let queue = queue.clone();
let tx = tx.clone();
let base = base.clone();
let identity = identity.clone();
std::thread::Builder::new()
.name("punktfunk-lib-art".into())
.spawn(move || {
let Ok(agent) = agent(&identity, pin) else {
return;
};
loop {
let job = queue.lock().unwrap().pop_front();
let Some((id, candidates)) = job else { break };
for url in &candidates {
match fetch_art(&agent, &base, url) {
Ok(bytes) => {
// Receiver gone (page popped) — stop fetching.
if tx.send_blocking((id, bytes)).is_err() {
return;
}
break;
}
// 404 on a guessed CDN path is routine — try the next kind.
Err(e) => tracing::debug!(%id, url, error = %e, "poster miss"),
}
}
}
})
.expect("spawn art thread");
}
rx
}
fn classify(e: ureq::Error) -> LibraryError { fn classify(e: ureq::Error) -> LibraryError {
match e { match e {
ureq::Error::Status(401 | 403, _) => LibraryError::NotPaired, ureq::Error::Status(401 | 403, _) => LibraryError::NotPaired,
+2
View File
@@ -26,6 +26,8 @@ mod session;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
mod trust; mod trust;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
mod ui_gamepad_library;
#[cfg(target_os = "linux")]
mod ui_hosts; mod ui_hosts;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
mod ui_library; mod ui_library;
+125 -21
View File
@@ -45,18 +45,55 @@ pub struct SessionParams {
pub connect_timeout: Duration, pub connect_timeout: Duration,
} }
/// The session pump's share of the unified stats window (design/stats-unification.md):
/// stream facts plus the two stages measured before the presenter. The frame consumer in
/// `ui_stream` contributes the `display` stage and the end-to-end percentiles.
#[derive(Clone, Copy, Default)] #[derive(Clone, Copy, Default)]
pub struct Stats { pub struct Stats {
/// AUs received (reassembled) per second, actual-elapsed-time denominator.
pub fps: f32, pub fps: f32,
/// Received payload bytes × 8 / elapsed (goodput, excludes FEC overhead).
pub mbps: f32, pub mbps: f32,
/// p50 `host+network` stage: capture → received, host-clock corrected (ms).
pub host_net_ms: f32,
/// p50 `host` stage: the host's own capture→fully-sent, from the per-AU 0xCF host
/// timings (design/stats-unification.md Phase 2). Valid only when `split`.
pub host_ms: f32,
/// p50 `network` stage: capture→received minus the host-reported share
/// (`hostnet host`, per-frame, saturating). Valid only when `split`.
pub net_ms: f32,
/// The window had matched host timings — the OSD splits `host+network` into
/// `host + network`. An old host never emits 0xCF, so this stays false and the
/// combined stage renders unchanged.
pub split: bool,
/// p50 `decode` stage: received → decoded, single-clock client-local (ms).
pub decode_ms: f32, pub decode_ms: f32,
/// Median capture→decoded latency over the last window (host-clock corrected). /// Unrecoverable network frame drops this window, and their share of
pub latency_ms: f32, /// received+lost (%). The OSD renders the counter line only when nonzero.
pub lost: u32,
pub lost_pct: f32,
/// The decode path frames actually took this window (`"vaapi"`/`"software"`, empty /// The decode path frames actually took this window (`"vaapi"`/`"software"`, empty
/// until the first frame) — the OSD's trailing tag; tracks a mid-session fallback. /// until the first frame) — the OSD's trailing tag; tracks a mid-session fallback.
pub decoder: &'static str, pub decoder: &'static str,
} }
/// Frames the pump keeps waiting for their 0xCF host timing (pts → capture→received µs).
/// ~2 s at 120 Hz — a timing arrives within a frame or two of its AU, and against an old
/// host (no 0xCF at all) this just caps the dead-weight ring.
const PENDING_SPLIT_CAP: usize = 256;
/// Sort a window of µs samples in place and return `(p50, p95)` per the spec's index
/// rules (`sorted[len/2]`, `sorted[min(len*95/100, len-1)]`); an empty window reads 0.
pub fn window_percentiles(samples: &mut [u64]) -> (u64, u64) {
if samples.is_empty() {
return (0, 0);
}
samples.sort_unstable();
let p50 = samples[samples.len() / 2];
let p95 = samples[(samples.len() * 95 / 100).min(samples.len() - 1)];
(p50, p95)
}
pub enum SessionEvent { pub enum SessionEvent {
Connected { Connected {
connector: Arc<NativeClient>, connector: Arc<NativeClient>,
@@ -219,13 +256,23 @@ fn pump(
let mut window_start = Instant::now(); let mut window_start = Instant::now();
let mut frames_n = 0u32; let mut frames_n = 0u32;
let mut bytes_n = 0u64; let mut bytes_n = 0u64;
let mut decode_us_sum = 0u64; // Stage windows (µs samples): `host+network` = capture→received (host-clock
let mut lat_us: Vec<u64> = Vec::with_capacity(256); // corrected), `decode` = received→decoded (client-local). p50 per 1 s window.
let mut hostnet_us: Vec<u64> = Vec::with_capacity(256);
let mut decode_us: Vec<u64> = Vec::with_capacity(256);
// Host/network split (Phase 2): frames awaiting their per-AU 0xCF host timing,
// correlated by pts_ns. Bounded — an old host never sends any, so entries just age out.
let mut pending_split: std::collections::VecDeque<(u64, u64)> =
std::collections::VecDeque::with_capacity(PENDING_SPLIT_CAP);
let mut host_us_win: Vec<u64> = Vec::with_capacity(256);
let mut net_us_win: Vec<u64> = Vec::with_capacity(256);
// What actually decoded the last frame — a VAAPI failure demotes mid-session, so // What actually decoded the last frame — a VAAPI failure demotes mid-session, so
// this is read off each frame's image variant rather than fixed at startup. // this is read off each frame's image variant rather than fixed at startup.
let mut dec_path: &'static str = ""; let mut dec_path: &'static str = "";
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs. // Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped(); let mut last_dropped = connector.frames_dropped();
// The stats window keeps its own drop cursor — the OSD shows the per-window delta.
let mut window_dropped = last_dropped;
let mut last_kf_req: Option<Instant> = None; let mut last_kf_req: Option<Instant> = None;
let end: Option<String> = loop { let end: Option<String> = loop {
@@ -237,7 +284,11 @@ fn pump(
// every ~816 ms at 60120 Hz anyway, so this rarely times out mid-stream). // every ~816 ms at 60120 Hz anyway, so this rarely times out mid-stream).
match connector.next_frame(Duration::from_millis(20)) { match connector.next_frame(Duration::from_millis(20)) {
Ok(frame) => { Ok(frame) => {
let t0 = Instant::now(); // The `received` point: AU fully reassembled, in hand, before decode.
let received_ns = now_ns();
// fps / goodput count every received AU (spec), decoded or not.
frames_n += 1;
bytes_n += frame.data.len() as u64;
match decoder.decode(&frame.data) { match decoder.decode(&frame.data) {
Ok(Some(image)) => { Ok(Some(image)) => {
total_frames += 1; total_frames += 1;
@@ -252,18 +303,27 @@ fn pump(
}; };
tracing::info!(width = w, height = h, path, "first frame decoded"); tracing::info!(width = w, height = h, path, "first frame decoded");
} }
// Latency: our wall clock expressed in the host's capture clock, // The `decoded` point — travels with the frame so the presenter
// minus the host-stamped capture pts (same math as client-rs). // can measure its `display` stage against it.
let lat = (now_ns() as i128 + clock_offset as i128 - frame.pts_ns as i128) let decoded_ns = now_ns();
// `host+network` stage: received expressed in the host's capture
// clock, minus the host-stamped capture pts (clamped (0, 10 s)).
let hn = (received_ns as i128 + clock_offset as i128 - frame.pts_ns as i128)
.max(0) as u64; .max(0) as u64;
if lat > 0 && lat < 10_000_000_000 { if hn > 0 && hn < 10_000_000_000 {
lat_us.push(lat / 1000); hostnet_us.push(hn / 1000);
// Remember the sample for the host/network split — matched
// against the AU's 0xCF host timing when it arrives.
if pending_split.len() >= PENDING_SPLIT_CAP {
pending_split.pop_front();
} }
decode_us_sum += t0.elapsed().as_micros() as u64; pending_split.push_back((frame.pts_ns, hn / 1000));
frames_n += 1; }
bytes_n += frame.data.len() as u64; // `decode` stage: received→decoded, single clock, no skew.
decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
let _ = frame_tx.force_send(DecodedFrame { let _ = frame_tx.force_send(DecodedFrame {
pts_ns: frame.pts_ns, pts_ns: frame.pts_ns,
decoded_ns,
image, image,
}); });
} }
@@ -271,12 +331,39 @@ fn pump(
// Survivable (loss until the next IDR/RFI recovery) — keep feeding. // Survivable (loss until the next IDR/RFI recovery) — keep feeding.
Err(e) => tracing::debug!(error = %e, "decode error (recovering)"), Err(e) => tracing::debug!(error = %e, "decode error (recovering)"),
} }
// A decode error / VAAPI→software demotion asks for a fresh IDR: the infinite
// GOP has no periodic keyframe, so a rebuilt/erroring decoder would stay
// gray/frozen until an unrelated packet drop happened to request one. Route it
// through the same throttle as loss recovery below.
if decoder.take_keyframe_request() {
let now = Instant::now();
if last_kf_req
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = connector.request_keyframe();
tracing::debug!("requested keyframe (decoder recovery)");
}
}
} }
Err(PunktfunkError::NoFrame) => {} Err(PunktfunkError::NoFrame) => {}
Err(PunktfunkError::Closed) => break Some("Host ended the session".to_string()), Err(PunktfunkError::Closed) => break Some("Host ended the session".to_string()),
Err(e) => break Some(format!("session: {e:?}")), Err(e) => break Some(format!("session: {e:?}")),
} }
// Drain the per-AU host timings (0xCF) non-blockingly and match them to received
// frames by pts: host = the host's own capture→sent, network = our
// capture→received minus it (the two tile per frame by construction). An old
// host never emits any — the deque fills to its cap and the OSD keeps the
// combined `host+network` stage.
while let Ok(t) = connector.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|(p, _)| *p == t.pts_ns) {
let (_, hn_us) = pending_split.remove(i).unwrap();
host_us_win.push(t.host_us as u64);
net_us_win.push(hn_us.saturating_sub(t.host_us as u64));
}
}
// Loss recovery: under infinite GOP the only recovery keyframe is one we request. The // Loss recovery: under infinite GOP the only recovery keyframe is one we request. The
// reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the // reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the
// reference-missing delta frames that follow and returns Ok, so keying off a decode error // reference-missing delta frames that follow and returns Ok, so keying off a decode error
@@ -295,30 +382,47 @@ fn pump(
if window_start.elapsed() >= Duration::from_secs(1) { if window_start.elapsed() >= Duration::from_secs(1) {
let secs = window_start.elapsed().as_secs_f32(); let secs = window_start.elapsed().as_secs_f32();
lat_us.sort_unstable(); let (hn_p50, _) = window_percentiles(&mut hostnet_us);
let p50 = lat_us.get(lat_us.len() / 2).copied().unwrap_or(0); let (dec_p50, _) = window_percentiles(&mut decode_us);
// Host/network split — present only when this window matched 0xCF timings.
let split = !host_us_win.is_empty();
let (host_p50, _) = window_percentiles(&mut host_us_win);
let (net_p50, _) = window_percentiles(&mut net_us_win);
let lost = dropped.saturating_sub(window_dropped) as u32;
window_dropped = dropped;
tracing::debug!( tracing::debug!(
fps = frames_n, fps = frames_n,
lat_p50_us = p50, hostnet_p50_us = hn_p50,
host_p50_us = host_p50,
net_p50_us = net_p50,
decode_p50_us = dec_p50,
lost,
total_frames, total_frames,
"stream window" "stream window"
); );
let _ = ev_tx.try_send(SessionEvent::Stats(Stats { let _ = ev_tx.try_send(SessionEvent::Stats(Stats {
fps: frames_n as f32 / secs, fps: frames_n as f32 / secs,
mbps: bytes_n as f32 * 8.0 / 1e6 / secs, mbps: bytes_n as f32 * 8.0 / 1e6 / secs,
decode_ms: if frames_n > 0 { host_net_ms: hn_p50 as f32 / 1000.0,
decode_us_sum as f32 / frames_n as f32 / 1000.0 host_ms: host_p50 as f32 / 1000.0,
net_ms: net_p50 as f32 / 1000.0,
split,
decode_ms: dec_p50 as f32 / 1000.0,
lost,
lost_pct: if lost > 0 {
lost as f32 * 100.0 / (frames_n + lost) as f32
} else { } else {
0.0 0.0
}, },
latency_ms: p50 as f32 / 1000.0,
decoder: dec_path, decoder: dec_path,
})); }));
window_start = Instant::now(); window_start = Instant::now();
frames_n = 0; frames_n = 0;
bytes_n = 0; bytes_n = 0;
decode_us_sum = 0; hostnet_us.clear();
lat_us.clear(); decode_us.clear();
host_us_win.clear();
net_us_win.clear();
} }
}; };
File diff suppressed because it is too large Load Diff
+9
View File
@@ -153,6 +153,15 @@ pub fn new(settings: Rc<RefCell<Settings>>, cbs: HostsCallbacks) -> HostsUi {
let disc_heading = heading("On this network"); let disc_heading = heading("On this network");
let disc_flow = make_flow(); let disc_flow = make_flow();
// A pointer click (and keyboard activate) emits `child-activated` on the *FlowBox*, never
// the child's own `activate` signal — so bridge it back to the child, where each card wires
// its connect handler (`saved_card`/`discovered_card`). Without this, clicking a card is dead.
for flow in [&saved_flow, &disc_flow] {
flow.connect_child_activated(|_, child| {
child.activate();
});
}
// Shown under the discovered heading while no (unsaved) advert is live yet. // Shown under the discovered heading while no (unsaved) advert is live yet.
let searching = gtk::Box::new(gtk::Orientation::Horizontal, 8); let searching = gtk::Box::new(gtk::Orientation::Horizontal, 8);
let spinner = gtk::Spinner::new(); let spinner = gtk::Spinner::new();
+10 -40
View File
@@ -14,11 +14,6 @@ use gtk::{gdk, glib};
use std::cell::{Cell, RefCell}; use std::cell::{Cell, RefCell};
use std::collections::{HashMap, VecDeque}; use std::collections::{HashMap, VecDeque};
use std::rc::Rc; use std::rc::Rc;
use std::sync::{Arc, Mutex};
/// Concurrent poster fetches — a handful is plenty for a LAN art proxy without turning a
/// big library into a connection burst.
const ART_WORKERS: usize = 3;
/// Everything the page re-renders from. Kept alive by the widget closures (reload/retry/ /// Everything the page re-renders from. Kept alive by the widget closures (reload/retry/
/// card activation); dropped when the page is popped, which also winds down any in-flight /// card activation); dropped when the page is popped, which also winds down any in-flight
@@ -76,6 +71,11 @@ fn build(app: Rc<App>, req: ConnectRequest) -> Rc<State> {
.row_spacing(18) .row_spacing(18)
.valign(gtk::Align::Start) .valign(gtk::Align::Start)
.build(); .build();
// Click/keyboard activation fires `child-activated` on the FlowBox, not the child's own
// `activate` — bridge it so each poster's connect handler (below) runs on click.
flow.connect_child_activated(|_, child| {
child.activate();
});
let content = gtk::Box::new(gtk::Orientation::Vertical, 0); let content = gtk::Box::new(gtk::Orientation::Vertical, 0);
content.set_margin_top(24); content.set_margin_top(24);
content.set_margin_bottom(24); content.set_margin_bottom(24);
@@ -295,39 +295,7 @@ fn load_art(state: &Rc<State>, games: &[GameEntry]) {
} }
let identity = state.app.identity.clone(); let identity = state.app.identity.clone();
let pin = state.req.fp_hex.as_deref().and_then(trust::parse_hex32); let pin = state.req.fp_hex.as_deref().and_then(trust::parse_hex32);
let queue = Arc::new(Mutex::new(jobs)); let rx = library::spawn_art_fetch(base, identity, pin, jobs);
let (tx, rx) = async_channel::unbounded::<(String, Vec<u8>)>();
for _ in 0..ART_WORKERS {
let queue = queue.clone();
let tx = tx.clone();
let base = base.clone();
let identity = identity.clone();
std::thread::Builder::new()
.name("punktfunk-lib-art".into())
.spawn(move || {
let Ok(agent) = library::agent(&identity, pin) else {
return;
};
loop {
let job = queue.lock().unwrap().pop_front();
let Some((id, candidates)) = job else { break };
for url in &candidates {
match library::fetch_art(&agent, &base, url) {
Ok(bytes) => {
// Receiver gone (page popped) — stop fetching.
if tx.send_blocking((id, bytes)).is_err() {
return;
}
break;
}
// 404 on a guessed CDN path is routine — try the next kind.
Err(e) => tracing::debug!(%id, url, error = %e, "poster miss"),
}
}
}
})
.expect("spawn art thread");
}
let weak = Rc::downgrade(state); let weak = Rc::downgrade(state);
glib::spawn_future_local(async move { glib::spawn_future_local(async move {
while let Ok((id, bytes)) = rx.recv().await { while let Ok((id, bytes)) = rx.recv().await {
@@ -349,7 +317,8 @@ fn load_art(state: &Rc<State>, games: &[GameEntry]) {
/// The store badge text — `store` comes from the entry (today `steam`/`custom`; future /// The store badge text — `store` comes from the entry (today `steam`/`custom`; future
/// stores per the host's provider list), with the id prefix as a fallback spelling. /// stores per the host's provider list), with the id prefix as a fallback spelling.
fn store_label(store: &str) -> &'static str { /// Shared with the gamepad launcher's posters.
pub fn store_label(store: &str) -> &'static str {
match store { match store {
"steam" => "Steam", "steam" => "Steam",
"custom" => "Custom", "custom" => "Custom",
@@ -363,7 +332,8 @@ fn store_label(store: &str) -> &'static str {
} }
/// Monogram for the placeholder tile: the first letters of the first two words. /// Monogram for the placeholder tile: the first letters of the first two words.
fn initials(title: &str) -> String { /// Shared with the gamepad launcher's posters.
pub fn initials(title: &str) -> String {
title title
.split_whitespace() .split_whitespace()
.take(2) .take(2)
+9 -1
View File
@@ -16,7 +16,14 @@ const RESOLUTIONS: &[(u32, u32)] = &[
]; ];
/// `0` = the monitor's native refresh, resolved at connect. /// `0` = the monitor's native refresh, resolved at connect.
const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240]; const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240];
const GAMEPADS: &[&str] = &["auto", "xbox360", "dualsense", "xboxone", "dualshock4"]; const GAMEPADS: &[&str] = &[
"auto",
"xbox360",
"dualsense",
"xboxone",
"dualshock4",
"steamdeck",
];
const COMPOSITORS: &[&str] = &["auto", "kwin", "wlroots", "mutter", "gamescope"]; const COMPOSITORS: &[&str] = &["auto", "kwin", "wlroots", "mutter", "gamescope"];
/// Codec setting values (persisted) paired with their display labels below. /// Codec setting values (persisted) paired with their display labels below.
const CODECS: &[&str] = &["auto", "hevc", "h264", "av1"]; const CODECS: &[&str] = &["auto", "hevc", "h264", "av1"];
@@ -403,6 +410,7 @@ pub fn show(
"DualSense", "DualSense",
"Xbox One", "Xbox One",
"DualShock 4", "DualShock 4",
"Steam Deck",
], ],
); );
let inhibit_row = adw::SwitchRow::builder() let inhibit_row = adw::SwitchRow::builder()
+224 -67
View File
@@ -31,33 +31,78 @@ use std::time::{Duration, Instant};
pub struct StreamPage { pub struct StreamPage {
pub page: adw::NavigationPage, pub page: adw::NavigationPage,
stats_label: gtk::Label, stats_label: gtk::Label,
/// Median capture→paintable-set latency (ms) over the frame consumer's last 1 s /// The frame consumer's share of the stats window (end-to-end percentiles + the
/// window — written there, folded into the OSD on each `Stats` event. /// `display` stage) — written there each 1 s window, folded into the OSD on each
present_ms: Rc<Cell<f32>>, /// `Stats` event.
presented: Rc<PresentedStats>,
/// The stream is HDR (PQ) right now — set by the frame consumer from each frame's /// The stream is HDR (PQ) right now — set by the frame consumer from each frame's
/// signaling (the host can flip SDR↔HDR mid-session, in-band). /// signaling (the host can flip SDR↔HDR mid-session, in-band).
hdr: Rc<Cell<bool>>, hdr: Rc<Cell<bool>>,
/// `clock_offset_ns == 0`: the skew handshake didn't run (or same host) — the
/// end-to-end line carries the `(same-host clock)` flag (spec clock rules).
same_host: bool,
/// `W×H@Hz` for the OSD's first line — fixed at connect, per-session.
mode_line: String,
}
/// Presenter-side window results (design/stats-unification.md): end-to-end =
/// capture→displayed measured directly (p50 + p95), `display` stage = decoded→displayed
/// p50. All ms, refreshed once per 1 s window by the frame consumer.
#[derive(Default)]
struct PresentedStats {
e2e_p50_ms: Cell<f32>,
e2e_p95_ms: Cell<f32>,
display_ms: Cell<f32>,
} }
impl StreamPage { impl StreamPage {
/// Render the canonical unified-stats OSD (design/stats-unification.md — Linux
/// endpoint is paintable-set, headline reads `capture→displayed`).
pub fn update_stats(&self, s: Stats) { pub fn update_stats(&self, s: Stats) {
let mut line = format!( let mut line1 = format!("{} · {:.0} fps · {:.1} Mb/s", self.mode_line, s.fps, s.mbps);
"{:.0} fps · {:.1} Mbit/s · dec {:.1} ms · lat {:.1} ms · present {:.1} ms",
s.fps,
s.mbps,
s.decode_ms,
s.latency_ms,
self.present_ms.get()
);
// Which decoder actually ran this window (vaapi/software) — tracks a fallback. // Which decoder actually ran this window (vaapi/software) — tracks a fallback.
if !s.decoder.is_empty() { if !s.decoder.is_empty() {
line.push_str(" · "); line1.push_str(" · ");
line.push_str(s.decoder); line1.push_str(s.decoder);
} }
if self.hdr.get() { if self.hdr.get() {
line.push_str(" · HDR"); line1.push_str(" · HDR");
} }
self.stats_label.set_text(&line); // The equation line: split `host+network` into `host + network` when the host
// reported per-AU timings (0xCF, stats Phase 2); the combined stage otherwise.
let equation = if s.split {
format!(
"= host {:.1} + network {:.1} + decode {:.1} + display {:.1}",
s.host_ms,
s.net_ms,
s.decode_ms,
self.presented.display_ms.get(),
)
} else {
format!(
"= host+network {:.1} + decode {:.1} + display {:.1}",
s.host_net_ms,
s.decode_ms,
self.presented.display_ms.get(),
)
};
let mut text = format!(
"{line1}\n\
end-to-end {:.1} ms p50 · {:.1} p95 · capture→displayed{}\n\
{equation}",
self.presented.e2e_p50_ms.get(),
self.presented.e2e_p95_ms.get(),
if self.same_host {
" (same-host clock)"
} else {
""
},
);
// Counters — only rendered when nonzero this window.
if s.lost > 0 {
text.push_str(&format!("\nlost {} ({:.1}%)", s.lost, s.lost_pct));
}
self.stats_label.set_text(&text);
} }
} }
@@ -84,6 +129,9 @@ pub struct StreamPageArgs {
/// reveal-on-notify chrome hiding) may never fire — the title bar would stay drawn /// reveal-on-notify chrome hiding) may never fire — the title bar would stay drawn
/// over the stream. Chrome-less by construction cannot regress that way. /// over the stream. Chrome-less by construction cannot regress that way.
pub chromeless: bool, pub chromeless: bool,
/// A controller is connected right now — the capture hint mentions the escape chord.
/// (Chromeless implies a controller-first device, so the chord shows there regardless.)
pub pad_connected: bool,
pub title: String, pub title: String,
} }
@@ -119,7 +167,13 @@ fn send_abs(widget: &impl IsA<gtk::Widget>, connector: &NativeClient, x: f64, y:
struct Capture { struct Capture {
connector: Arc<NativeClient>, connector: Arc<NativeClient>,
window: adw::ApplicationWindow, window: adw::ApplicationWindow,
overlay: gtk::Overlay, /// Held WEAKLY. Every input controller + the frame-clock tick are added to this overlay
/// and each captures `Rc<Capture>`; a strong ref back here would close the cycle
/// `overlay → controller → Rc<Capture> → overlay` that GTK can't collect, leaking the
/// whole stream subtree AND the `Arc<NativeClient>` (so `NativeClient::Drop` never runs)
/// on every session end — unbounded growth across the reconnects a Deck does constantly.
/// The live widget tree owns the overlay for the session's lifetime; upgrade at use.
overlay: glib::WeakRef<gtk::Overlay>,
hint: gtk::Label, hint: gtk::Label,
inhibit_shortcuts: bool, inhibit_shortcuts: bool,
captured: Cell<bool>, captured: Cell<bool>,
@@ -133,13 +187,19 @@ struct Capture {
/// VKs / GameStream button ids currently held — flushed up on release. /// VKs / GameStream button ids currently held — flushed up on release.
held_keys: RefCell<HashSet<u8>>, held_keys: RefCell<HashSet<u8>>,
held_buttons: RefCell<HashSet<u32>>, held_buttons: RefCell<HashSet<u32>>,
/// Fractional wheel remainder per axis (x, y), in 120-unit WHEEL_DELTA space. Precision
/// scroll surfaces — the Deck trackpad, hi-res wheels, two-finger touchpad — deliver
/// sub-unit deltas; truncating each event drops the tail. Carry it here instead.
scroll_acc: Cell<(f64, f64)>,
} }
impl Capture { impl Capture {
/// Send the coalesced pointer position, if any — one datagram, one fresh mode read. /// Send the coalesced pointer position, if any — one datagram, one fresh mode read.
fn flush_pending_motion(&self) { fn flush_pending_motion(&self) {
if let Some((x, y)) = self.pending_abs.take() { if let Some((x, y)) = self.pending_abs.take() {
send_abs(&self.overlay, &self.connector, x, y); if let Some(overlay) = self.overlay.upgrade() {
send_abs(&overlay, &self.connector, x, y);
}
} }
} }
@@ -147,8 +207,9 @@ impl Capture {
if self.captured.replace(true) { if self.captured.replace(true) {
return; return;
} }
self.overlay if let Some(overlay) = self.overlay.upgrade() {
.set_cursor(gdk::Cursor::from_name("none", None).as_ref()); overlay.set_cursor(gdk::Cursor::from_name("none", None).as_ref());
}
self.hint.set_visible(false); self.hint.set_visible(false);
if self.inhibit_shortcuts { if self.inhibit_shortcuts {
if let Some(tl) = self if let Some(tl) = self
@@ -165,7 +226,9 @@ impl Capture {
if !self.captured.replace(false) { if !self.captured.replace(false) {
return; return;
} }
self.overlay.set_cursor(None); if let Some(overlay) = self.overlay.upgrade() {
overlay.set_cursor(None);
}
self.hint.set_visible(true); self.hint.set_visible(true);
self.pending_abs.set(None); // never flush motion gathered while captured self.pending_abs.set(None); // never flush motion gathered while captured
if let Some(tl) = self if let Some(tl) = self
@@ -197,46 +260,56 @@ pub fn new(args: StreamPageArgs) -> StreamPage {
inhibit_shortcuts, inhibit_shortcuts,
show_stats, show_stats,
chromeless, chromeless,
pad_connected,
title, title,
} = args; } = args;
let w = build_widgets(&window, &title, chromeless); let w = build_widgets(&window, &title, chromeless, pad_connected);
w.stats_label.set_visible(show_stats); w.stats_label.set_visible(show_stats);
// OSD line-1 facts, fixed for the session (the mode is negotiated per-session).
let mode = connector.mode();
let mode_line = format!("{}×{}@{}", mode.width, mode.height, mode.refresh_hz);
// Offset 0 = the host didn't answer the skew handshake / same host — flagged on the
// end-to-end line so an uncorrected cross-machine number is never shown silently.
let same_host = clock_offset_ns == 0;
let capture = Rc::new(Capture { let capture = Rc::new(Capture {
connector, connector,
window: window.clone(), window: window.clone(),
overlay: w.overlay.clone(), overlay: w.overlay.downgrade(),
hint: w.hint.clone(), hint: w.hint.clone(),
inhibit_shortcuts, inhibit_shortcuts,
captured: Cell::new(false), captured: Cell::new(false),
pending_abs: Cell::new(None), pending_abs: Cell::new(None),
held_keys: RefCell::new(HashSet::new()), held_keys: RefCell::new(HashSet::new()),
held_buttons: RefCell::new(HashSet::new()), held_buttons: RefCell::new(HashSet::new()),
scroll_acc: Cell::new((0.0, 0.0)),
}); });
let present_ms = Rc::new(Cell::new(0.0f32)); let presented = Rc::new(PresentedStats::default());
let hdr = Rc::new(Cell::new(false)); let hdr = Rc::new(Cell::new(false));
spawn_frame_consumer( spawn_frame_consumer(
&w.picture, &w.picture,
frames, frames,
clock_offset_ns, clock_offset_ns,
present_ms.clone(), presented.clone(),
hdr.clone(), hdr.clone(),
); );
attach_keyboard(&w.overlay, &window, &capture, &stop, &w.stats_label); let key_controller = attach_keyboard(&window, &capture, &stop, &w.stats_label);
attach_mouse(&w.overlay, &capture); attach_mouse(&w.overlay, &capture);
attach_scroll(&w.overlay, &capture); attach_scroll(&w.overlay, &capture);
if !chromeless { if !chromeless {
attach_edge_reveal(&w.toolbar, &w.overlay, &window, &capture); attach_edge_reveal(&w.toolbar, &w.overlay, &window, &capture);
} }
let active_handler = attach_capture_lifecycle(&w.overlay, &window, &capture); let active_handler = attach_capture_lifecycle(&w.overlay, &window, &capture);
let escape_future = spawn_escape_watch(&window, &capture, escape_rx); let escape_future = spawn_escape_watch(&window, &capture, escape_rx, &w.fs_hint, chromeless);
let disconnect_future = spawn_disconnect_watch(&window, &capture, &stop, disconnect_rx); let disconnect_future = spawn_disconnect_watch(&window, &capture, &stop, disconnect_rx);
wire_teardown( wire_teardown(
&w.page, &w.page,
&window, &window,
&stop, &stop,
(w.fs_handler, active_handler), (w.fs_handler, active_handler),
key_controller,
escape_future, escape_future,
disconnect_future, disconnect_future,
); );
@@ -244,8 +317,10 @@ pub fn new(args: StreamPageArgs) -> StreamPage {
StreamPage { StreamPage {
page: w.page, page: w.page,
stats_label: w.stats_label, stats_label: w.stats_label,
present_ms, presented,
hdr, hdr,
same_host,
mode_line,
} }
} }
@@ -254,6 +329,9 @@ struct PageWidgets {
picture: gtk::Picture, picture: gtk::Picture,
stats_label: gtk::Label, stats_label: gtk::Label,
hint: gtk::Label, hint: gtk::Label,
/// The transient chord/fullscreen-exit hint — the escape watch re-flashes it in
/// chromeless mode.
fs_hint: gtk::Label,
overlay: gtk::Overlay, overlay: gtk::Overlay,
toolbar: adw::ToolbarView, toolbar: adw::ToolbarView,
page: adw::NavigationPage, page: adw::NavigationPage,
@@ -264,7 +342,12 @@ struct PageWidgets {
/// The offloaded picture under an overlay (stats HUD, capture hint, fullscreen hint), a /// The offloaded picture under an overlay (stats HUD, capture hint, fullscreen hint), a
/// header bar with the fullscreen toggle, and the window's fullscreen behavior. /// header bar with the fullscreen toggle, and the window's fullscreen behavior.
/// `chromeless` (Gaming Mode) builds NO header bar at all — see `StreamPageArgs`. /// `chromeless` (Gaming Mode) builds NO header bar at all — see `StreamPageArgs`.
fn build_widgets(window: &adw::ApplicationWindow, title: &str, chromeless: bool) -> PageWidgets { fn build_widgets(
window: &adw::ApplicationWindow,
title: &str,
chromeless: bool,
pad_connected: bool,
) -> PageWidgets {
let picture = gtk::Picture::new(); let picture = gtk::Picture::new();
picture.set_content_fit(gtk::ContentFit::Contain); picture.set_content_fit(gtk::ContentFit::Contain);
@@ -273,6 +356,22 @@ fn build_widgets(window: &adw::ApplicationWindow, title: &str, chromeless: bool)
// no-op wrapper. Black letterboxing keeps fullscreen scanout-eligible. // no-op wrapper. Black letterboxing keeps fullscreen scanout-eligible.
let offload = gtk::GraphicsOffload::new(Some(&picture)); let offload = gtk::GraphicsOffload::new(Some(&picture));
offload.set_black_background(true); offload.set_black_background(true);
// Whether the raw video dmabuf may be handed to the compositor as a subsurface.
// Under gamescope (chromeless) default OFF: a subsurface makes the COMPOSITOR do the
// NV12→RGB conversion, and gamescope's matrix/range choice for it is outside our
// control (off-colours reported on the Deck) — GTK compositing it itself applies the
// stream's own BT.709-narrow color state. `PUNKTFUNK_OFFLOAD=1|0` overrides either
// way, which also makes the colour question bisectable in one run: offload-off heals →
// compositor conversion; still off → GTK/Mesa import (then try PUNKTFUNK_DECODER=software).
let offload_on = match std::env::var("PUNKTFUNK_OFFLOAD").ok().as_deref() {
Some("0") => false,
Some(_) => true,
None => !chromeless,
};
if !offload_on {
offload.set_enabled(gtk::GraphicsOffloadEnabled::Disabled);
tracing::info!("graphics offload disabled — GTK composites the video itself");
}
let stats_label = gtk::Label::new(None); let stats_label = gtk::Label::new(None);
stats_label.add_css_class("osd"); stats_label.add_css_class("osd");
@@ -282,9 +381,16 @@ fn build_widgets(window: &adw::ApplicationWindow, title: &str, chromeless: bool)
stats_label.set_margin_start(12); stats_label.set_margin_start(12);
stats_label.set_margin_top(12); stats_label.set_margin_top(12);
let hint = gtk::Label::new(Some( // The capture hint speaks the input devices actually present: on a controller-first
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases · Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats", // device (chromeless) or with a pad connected it must surface the chord — keyboard-only
)); // text on a Deck told the user nothing they could press.
let hint = gtk::Label::new(Some(if chromeless {
"Tap the stream to capture input · hold L1 + R1 + Start + Select to leave"
} else if pad_connected {
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases · Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave"
} else {
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases · Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats"
}));
hint.add_css_class("osd"); hint.add_css_class("osd");
hint.set_halign(gtk::Align::Center); hint.set_halign(gtk::Align::Center);
hint.set_valign(gtk::Align::End); hint.set_valign(gtk::Align::End);
@@ -296,7 +402,7 @@ fn build_widgets(window: &adw::ApplicationWindow, title: &str, chromeless: bool)
// devices; the L1+R1+Start+Select chord on a controller). Gaming Mode has no F11, // devices; the L1+R1+Start+Select chord on a controller). Gaming Mode has no F11,
// no header to reveal, and Steam owns window management — only the chord applies. // no header to reveal, and Steam owns window management — only the chord applies.
let fs_hint = gtk::Label::new(Some(if chromeless { let fs_hint = gtk::Label::new(Some(if chromeless {
"L1 + R1 + Start + Select — leave the stream (hold to disconnect)" "Hold L1 + R1 + Start + Select — leave the stream"
} else { } else {
"F11 · mouse to the top edge · L1 + R1 + Start + Select — exit fullscreen (hold to disconnect)" "F11 · mouse to the top edge · L1 + R1 + Start + Select — exit fullscreen (hold to disconnect)"
})); }));
@@ -372,6 +478,7 @@ fn build_widgets(window: &adw::ApplicationWindow, title: &str, chromeless: bool)
picture, picture,
stats_label, stats_label,
hint, hint,
fs_hint,
overlay, overlay,
toolbar, toolbar,
page, page,
@@ -420,12 +527,13 @@ fn attach_edge_reveal(
/// then draws whatever paintable is current on its own frame clock. Ends itself when the /// then draws whatever paintable is current on its own frame clock. Ends itself when the
/// channel closes or the picture is gone. /// channel closes or the picture is gone.
/// ///
/// Also the capture→present-ish measurement point: at each paintable set the frame's /// Also the `displayed` measurement point (design/stats-unification.md): each paintable
/// host capture pts is compared against the local wall clock expressed in the host clock /// set stamps the local wall clock, yielding end-to-end = capture→displayed (host-clock
/// (`clock_offset_ns`, same math as the session's decode latency). This is /// corrected via `clock_offset_ns`, p50+p95, measured directly) and the client-local
/// capture→paintable-SET — GTK's own present adds one compositor cycle after this. The /// `display` stage = decoded→displayed. This is capture→paintable-SET — GTK's own
/// 1 s p50 lands on the stats OSD (via `present_ms`) and in a "present window" debug /// present adds one compositor cycle after this. The 1 s window results land on the
/// line for headless validation. /// stats OSD (via `PresentedStats`) and in a "present window" debug line for headless
/// validation.
/// One-entry cache of `ColorDesc` → `GdkColorState` (signaling changes at most on an /// One-entry cache of `ColorDesc` → `GdkColorState` (signaling changes at most on an
/// SDR↔HDR flip, never per frame). /// SDR↔HDR flip, never per frame).
#[derive(Default)] #[derive(Default)]
@@ -461,11 +569,15 @@ impl ColorStateCache {
}); });
} }
let state = cicp.build_color_state().ok(); let state = cicp.build_color_state().ok();
if state.is_none() { // One line per signaling change — the on-glass colour bisect reads this to tell
tracing::warn!( // "state applied" from "GDK fell back to its YUV default (BT.601)".
match &state {
Some(_) => tracing::info!(?desc, rgb, "colour signaling → GDK color state"),
None => tracing::warn!(
?desc, ?desc,
"GDK can't represent this colour signaling — using default" rgb,
); "GDK can't represent this colour signaling — using default (YUV: BT.601)"
),
} }
self.0 = Some((desc, state.clone())); self.0 = Some((desc, state.clone()));
state state
@@ -476,7 +588,7 @@ fn spawn_frame_consumer(
picture: &gtk::Picture, picture: &gtk::Picture,
frames: async_channel::Receiver<DecodedFrame>, frames: async_channel::Receiver<DecodedFrame>,
clock_offset_ns: i64, clock_offset_ns: i64,
present_ms: Rc<Cell<f32>>, presented_stats: Rc<PresentedStats>,
hdr: Rc<Cell<bool>>, hdr: Rc<Cell<bool>>,
) { ) {
let picture = picture.downgrade(); let picture = picture.downgrade();
@@ -488,7 +600,10 @@ fn spawn_frame_consumer(
let mut yuv_state = ColorStateCache::default(); let mut yuv_state = ColorStateCache::default();
let mut rgb_state = ColorStateCache::default(); let mut rgb_state = ColorStateCache::default();
glib::spawn_future_local(async move { glib::spawn_future_local(async move {
let mut win_lat_us: Vec<u64> = Vec::with_capacity(256); // Window samples (µs): end-to-end capture→displayed (host-clock corrected) and
// the client-local display stage decoded→displayed.
let mut win_e2e_us: Vec<u64> = Vec::with_capacity(256);
let mut win_disp_us: Vec<u64> = Vec::with_capacity(256);
let mut win_start = Instant::now(); let mut win_start = Instant::now();
while let Ok(f) = frames.recv().await { while let Ok(f) = frames.recv().await {
let Some(picture) = picture.upgrade() else { let Some(picture) = picture.upgrade() else {
@@ -561,26 +676,34 @@ fn spawn_frame_consumer(
} }
} }
} }
// Capture→paintable-set latency, host-clock corrected (same math and sanity // The `displayed` stamp: end-to-end = capture→displayed host-clock corrected
// bound as the session's decode-latency window). // (same clamp as the session's stage windows); display = decoded→displayed,
// single clock, no skew.
if presented { if presented {
let lat = (crate::session::now_ns() as i128 + clock_offset_ns as i128 let displayed_ns = crate::session::now_ns();
- f.pts_ns as i128) let e2e = (displayed_ns as i128 + clock_offset_ns as i128 - f.pts_ns as i128).max(0)
.max(0) as u64; as u64;
if lat > 0 && lat < 10_000_000_000 { if e2e > 0 && e2e < 10_000_000_000 {
win_lat_us.push(lat / 1000); win_e2e_us.push(e2e / 1000);
} }
win_disp_us.push(displayed_ns.saturating_sub(f.decoded_ns) / 1000);
} }
if win_start.elapsed() >= Duration::from_secs(1) { if win_start.elapsed() >= Duration::from_secs(1) {
win_lat_us.sort_unstable(); let frames = win_e2e_us.len();
let p50 = win_lat_us.get(win_lat_us.len() / 2).copied().unwrap_or(0); let (e2e_p50, e2e_p95) = crate::session::window_percentiles(&mut win_e2e_us);
let (disp_p50, _) = crate::session::window_percentiles(&mut win_disp_us);
tracing::debug!( tracing::debug!(
frames = win_lat_us.len(), frames,
present_p50_us = p50, e2e_p50_us = e2e_p50,
e2e_p95_us = e2e_p95,
display_p50_us = disp_p50,
"present window" "present window"
); );
present_ms.set(p50 as f32 / 1000.0); presented_stats.e2e_p50_ms.set(e2e_p50 as f32 / 1000.0);
win_lat_us.clear(); presented_stats.e2e_p95_ms.set(e2e_p95 as f32 / 1000.0);
presented_stats.display_ms.set(disp_p50 as f32 / 1000.0);
win_e2e_us.clear();
win_disp_us.clear();
win_start = Instant::now(); win_start = Instant::now();
} }
} }
@@ -590,13 +713,20 @@ fn spawn_frame_consumer(
/// Keyboard, capture-phase: the release (Ctrl+Alt+Shift+Q) / disconnect (Ctrl+Alt+Shift+D) /// Keyboard, capture-phase: the release (Ctrl+Alt+Shift+Q) / disconnect (Ctrl+Alt+Shift+D)
/// / stats (Ctrl+Alt+Shift+S) chords and F11 are handled locally; everything else becomes /// / stats (Ctrl+Alt+Shift+S) chords and F11 are handled locally; everything else becomes
/// a VK on the wire while captured. /// a VK on the wire while captured.
///
/// The controller lives on the **window**, not the stream overlay: a `NavigationView` push
/// followed by `window.fullscreen()` hands keyboard focus to the pushed page's header back
/// button (a sibling of the overlay), so an overlay-scoped key controller never sees a key and
/// every chord — plus all gameplay key forwarding — is silently dropped until the user clicks
/// the stream. The window is always on the key-propagation path regardless of which child holds
/// focus. Returned so `wire_teardown` can remove it when the page goes away (otherwise the
/// chords would keep firing app-wide against a dead session).
fn attach_keyboard( fn attach_keyboard(
overlay: &gtk::Overlay,
window: &adw::ApplicationWindow, window: &adw::ApplicationWindow,
capture: &Rc<Capture>, capture: &Rc<Capture>,
stop: &Arc<AtomicBool>, stop: &Arc<AtomicBool>,
stats: &gtk::Label, stats: &gtk::Label,
) { ) -> gtk::EventControllerKey {
let key = gtk::EventControllerKey::new(); let key = gtk::EventControllerKey::new();
key.set_propagation_phase(gtk::PropagationPhase::Capture); key.set_propagation_phase(gtk::PropagationPhase::Capture);
let cap = capture.clone(); let cap = capture.clone();
@@ -662,7 +792,8 @@ fn attach_keyboard(
} }
} }
}); });
overlay.add_controller(key); window.add_controller(key.clone());
key
} }
/// Mouse: absolute motion + buttons — forwarded only while captured; the click that /// Mouse: absolute motion + buttons — forwarded only while captured; the click that
@@ -681,7 +812,8 @@ fn attach_mouse(overlay: &gtk::Overlay, capture: &Rc<Capture>) {
}); });
overlay.add_controller(motion); overlay.add_controller(motion);
// The per-tick flush. (The tick callback dies with the overlay, so no teardown.) // The per-tick flush. The tick callback dies with the overlay (which `Capture` now holds
// only weakly, so it truly can), taking its `Capture` ref with it — no explicit teardown.
let cap = capture.clone(); let cap = capture.clone();
overlay.add_tick_callback(move |_, _| { overlay.add_tick_callback(move |_, _| {
cap.flush_pending_motion(); cap.flush_pending_motion();
@@ -691,7 +823,9 @@ fn attach_mouse(overlay: &gtk::Overlay, capture: &Rc<Capture>) {
let click = gtk::GestureClick::builder().button(0).build(); let click = gtk::GestureClick::builder().button(0).build();
let cap = capture.clone(); let cap = capture.clone();
click.connect_pressed(move |g, _n, x, y| { click.connect_pressed(move |g, _n, x, y| {
cap.overlay.grab_focus(); if let Some(overlay) = cap.overlay.upgrade() {
overlay.grab_focus();
}
if !cap.captured.get() { if !cap.captured.get() {
cap.engage(); // the engaging click is suppressed toward the host cap.engage(); // the engaging click is suppressed toward the host
return; return;
@@ -727,16 +861,22 @@ fn attach_scroll(overlay: &gtk::Overlay, capture: &Rc<Capture>) {
} }
cap.flush_pending_motion(); // scroll happens at the latest cursor position cap.flush_pending_motion(); // scroll happens at the latest cursor position
// The wire carries WHEEL_DELTA(120) units, positive = up / right; GTK's dy is // The wire carries WHEEL_DELTA(120) units, positive = up / right; GTK's dy is
// positive = down. Smooth fractions survive — libei's discrete scroll is // positive = down. libei's discrete scroll is 120-based too. Accumulate the
// 120-based too. // fractional remainder so precision-scroll sub-unit deltas aren't lost.
let vy = (-dy * 120.0) as i32; let (mut ax, mut ay) = cap.scroll_acc.get();
ay += -dy * 120.0;
ax += dx * 120.0;
let vy = ay.trunc() as i32;
if vy != 0 { if vy != 0 {
ay -= f64::from(vy);
send(&cap.connector, InputKind::MouseScroll, 0, vy, 0, 0); send(&cap.connector, InputKind::MouseScroll, 0, vy, 0, 0);
} }
let vx = (dx * 120.0) as i32; let vx = ax.trunc() as i32;
if vx != 0 { if vx != 0 {
ax -= f64::from(vx);
send(&cap.connector, InputKind::MouseScroll, 1, vx, 0, 0); send(&cap.connector, InputKind::MouseScroll, 1, vx, 0, 0);
} }
cap.scroll_acc.set((ax, ay));
glib::Propagation::Stop glib::Propagation::Stop
}); });
overlay.add_controller(scroll); overlay.add_controller(scroll);
@@ -772,20 +912,30 @@ fn attach_capture_lifecycle(
/// Controller escape chord (gamepad service) → leave fullscreen + release capture. The /// Controller escape chord (gamepad service) → leave fullscreen + release capture. The
/// chord is the only fullscreen exit a controller has (no F11 key; fullscreen hides the /// chord is the only fullscreen exit a controller has (no F11 key; fullscreen hides the
/// chrome). Aborted on page-hidden so a stale future can't act on the shared window. /// chrome). In chromeless mode there is nothing visible to release INTO — a quick press
/// re-flashes the hold-to-leave hint instead, so an experimenting user learns the hold.
/// Aborted on page-hidden so a stale future can't act on the shared window.
fn spawn_escape_watch( fn spawn_escape_watch(
window: &adw::ApplicationWindow, window: &adw::ApplicationWindow,
capture: &Rc<Capture>, capture: &Rc<Capture>,
escape_rx: async_channel::Receiver<()>, escape_rx: async_channel::Receiver<()>,
fs_hint: &gtk::Label,
chromeless: bool,
) -> glib::JoinHandle<()> { ) -> glib::JoinHandle<()> {
let window = window.clone(); let window = window.clone();
let cap = capture.clone(); let cap = capture.clone();
let fs_hint = fs_hint.clone();
glib::spawn_future_local(async move { glib::spawn_future_local(async move {
while escape_rx.recv().await.is_ok() { while escape_rx.recv().await.is_ok() {
if window.is_fullscreen() { if window.is_fullscreen() {
window.unfullscreen(); window.unfullscreen();
} }
cap.release(); cap.release();
if chromeless {
fs_hint.set_visible(true);
let fs_hint = fs_hint.clone();
glib::timeout_add_seconds_local_once(4, move || fs_hint.set_visible(false));
}
} }
}) })
} }
@@ -822,12 +972,14 @@ fn wire_teardown(
window: &adw::ApplicationWindow, window: &adw::ApplicationWindow,
stop: &Arc<AtomicBool>, stop: &Arc<AtomicBool>,
handlers: (glib::SignalHandlerId, glib::SignalHandlerId), handlers: (glib::SignalHandlerId, glib::SignalHandlerId),
key_controller: gtk::EventControllerKey,
escape_future: glib::JoinHandle<()>, escape_future: glib::JoinHandle<()>,
disconnect_future: glib::JoinHandle<()>, disconnect_future: glib::JoinHandle<()>,
) { ) {
let window = window.clone(); let window = window.clone();
let stop_h = stop.clone(); let stop_h = stop.clone();
let handlers = RefCell::new(Some(handlers)); let handlers = RefCell::new(Some(handlers));
let key_controller = RefCell::new(Some(key_controller));
let escape_future = RefCell::new(Some(escape_future)); let escape_future = RefCell::new(Some(escape_future));
let disconnect_future = RefCell::new(Some(disconnect_future)); let disconnect_future = RefCell::new(Some(disconnect_future));
page.connect_hidden(move |_| { page.connect_hidden(move |_| {
@@ -836,6 +988,11 @@ fn wire_teardown(
window.disconnect(fs); window.disconnect(fs);
window.disconnect(active); window.disconnect(active);
} }
// The key controller lives on the window (see `attach_keyboard`) — remove it so its
// chords don't keep firing app-wide against a torn-down session.
if let Some(kc) = key_controller.borrow_mut().take() {
window.remove_controller(&kc);
}
if let Some(f) = escape_future.borrow_mut().take() { if let Some(f) = escape_future.borrow_mut().take() {
f.abort(); f.abort();
} }
+45 -5
View File
@@ -24,11 +24,15 @@ use std::os::fd::RawFd;
use std::ptr; use std::ptr;
/// One decoded frame headed for the presenter, carrying the host capture timestamp so the /// One decoded frame headed for the presenter, carrying the host capture timestamp so the
/// UI can measure capture→paintable-set latency at the moment it presents. /// UI can measure capture→displayed latency at the moment it presents.
pub struct DecodedFrame { pub struct DecodedFrame {
/// Host-clock capture pts (ns) of the AU this image decoded from — compare against /// Host-clock capture pts (ns) of the AU this image decoded from — compare against
/// the local wall clock + `clock_offset_ns` at paintable-set time. /// the local wall clock + `clock_offset_ns` at paintable-set time.
pub pts_ns: u64, pub pts_ns: u64,
/// Local wall clock (ns) when the decoder emitted this image — the `decoded`
/// measurement point (design/stats-unification.md); the presenter subtracts it from
/// its paintable-set stamp for the client-local `display` stage.
pub decoded_ns: u64,
pub image: DecodedImage, pub image: DecodedImage,
} }
@@ -132,8 +136,19 @@ pub struct Decoder {
/// The negotiated codec (from the host's Welcome), so a mid-session VAAPI→software demotion /// The negotiated codec (from the host's Welcome), so a mid-session VAAPI→software demotion
/// rebuilds the software decoder for the SAME codec. /// rebuilds the software decoder for the SAME codec.
codec_id: ffmpeg::codec::Id, codec_id: ffmpeg::codec::Id,
/// Consecutive VAAPI decode errors — a single transient failure (e.g. a reference-missing
/// frame after packet loss) shouldn't cost the whole session its hardware decoder.
vaapi_fails: u32,
/// Set when the decoder needs a fresh IDR to resynchronize (after an error or a demotion).
/// The pump drains it and asks the host — under the infinite GOP there is no periodic
/// keyframe, so a rebuilt/erroring decoder would otherwise stay gray/frozen forever.
want_keyframe: bool,
} }
/// Demote VAAPI→software only after this many consecutive hardware decode errors; a lone
/// transient error just re-requests an IDR and keeps the hardware decoder.
const VAAPI_DEMOTE_AFTER: u32 = 3;
/// Map a negotiated `quic` codec bit to the FFmpeg decoder id the client opens. /// Map a negotiated `quic` codec bit to the FFmpeg decoder id the client opens.
pub fn ffmpeg_codec_id(wire: u8) -> ffmpeg::codec::Id { pub fn ffmpeg_codec_id(wire: u8) -> ffmpeg::codec::Id {
match wire { match wire {
@@ -179,6 +194,8 @@ impl Decoder {
return Ok(Decoder { return Ok(Decoder {
backend: Backend::Vaapi(v), backend: Backend::Vaapi(v),
codec_id, codec_id,
vaapi_fails: 0,
want_keyframe: false,
}); });
} }
Err(e) => { Err(e) => {
@@ -192,20 +209,43 @@ impl Decoder {
Ok(Decoder { Ok(Decoder {
backend: Backend::Software(SoftwareDecoder::new(codec_id)?), backend: Backend::Software(SoftwareDecoder::new(codec_id)?),
codec_id, codec_id,
vaapi_fails: 0,
want_keyframe: false,
}) })
} }
/// Drain the "please ask the host for an IDR" flag — the pump calls this each iteration
/// (throttled) so a demoted/erroring decoder can resynchronize under the infinite GOP.
pub fn take_keyframe_request(&mut self) -> bool {
std::mem::take(&mut self.want_keyframe)
}
/// Feed one access unit; returns the decoded frame (the host's streams are /// Feed one access unit; returns the decoded frame (the host's streams are
/// one-in/one-out). A software decode error after packet loss is survivable — log /// one-in/one-out). A software decode error after packet loss is survivable — log
/// upstream and keep feeding. A VAAPI error demotes to software for the rest of the /// upstream and keep feeding. A VAAPI error re-requests an IDR and retries the hardware
/// session (broken driver, e.g. nvidia-vaapi-driver) — the next IDR resynchronizes. /// decoder; only a persistent streak of failures (a genuinely broken driver, e.g.
/// nvidia-vaapi-driver) demotes to software. Either way `want_keyframe` is set so the
/// pump asks the host for a fresh IDR — under the infinite GOP nothing else resyncs a
/// rebuilt/erroring decoder, so skipping this leaves the picture gray/frozen for good.
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> { pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
match &mut self.backend { match &mut self.backend {
Backend::Vaapi(v) => match v.decode(au) { Backend::Vaapi(v) => match v.decode(au) {
Ok(f) => Ok(f.map(DecodedImage::Dmabuf)), Ok(f) => {
self.vaapi_fails = 0;
Ok(f.map(DecodedImage::Dmabuf))
}
Err(e) => { Err(e) => {
tracing::warn!(error = %e, "VAAPI decode failed — falling back to software"); self.vaapi_fails += 1;
self.want_keyframe = true;
if self.vaapi_fails >= VAAPI_DEMOTE_AFTER {
tracing::warn!(error = %e, fails = self.vaapi_fails,
"VAAPI decode failing repeatedly — demoting to software");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?); self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
} else {
tracing::warn!(error = %e,
"VAAPI decode error — requesting keyframe, keeping hardware decode");
}
Ok(None) Ok(None)
} }
}, },
+1 -1
View File
@@ -27,7 +27,7 @@ GEOMETRY="${GEOMETRY:-1380x860x24}"
SETTLE="${SETTLE:-1.2}" SETTLE="${SETTLE:-1.2}"
SHOT_DISPLAY="${SHOT_DISPLAY:-:99}" SHOT_DISPLAY="${SHOT_DISPLAY:-:99}"
if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair addhost shortcuts library); fi if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair addhost shortcuts library gamepad-library); fi
[ -x "$BIN" ] || { [ -x "$BIN" ] || {
echo "client binary not found: $BIN (build it first: cargo build --release -p punktfunk-client-linux)" >&2 echo "client binary not found: $BIN (build it first: cargo build --release -p punktfunk-client-linux)" >&2
+1 -1
View File
@@ -14,7 +14,7 @@ example of driving the protocol end to end: QUIC control plane, UDP data plane,
- **Receives a real stream**, writes a playable elementary stream (`.h265`/`.h264`/`.av1` — the - **Receives a real stream**, writes a playable elementary stream (`.h265`/`.h264`/`.av1` — the
extension tracks the **negotiated codec**; the probe advertises all three and the host picks), and extension tracks the **negotiated codec**; the probe advertises all three and the host picks), and
reports per-frame **capture→…→reassembled latency** percentiles (the host stamps each frame with reports per-frame **capture→received latency** percentiles (the host stamps each frame with
its capture clock). its capture clock).
- **Verification mode** against a synthetic host — byte-checks deterministic test frames. - **Verification mode** against a synthetic host — byte-checks deterministic test frames.
- **Exercises every plane** with scripted test traffic: - **Exercises every plane** with scripted test traffic:
+89 -15
View File
@@ -4,7 +4,7 @@
//! * **verification** (`frames > 0`, synthetic host): byte-checks deterministic test frames; //! * **verification** (`frames > 0`, synthetic host): byte-checks deterministic test frames;
//! * **stream** (`frames == 0`, virtual host): receives real encoded AUs, writes a playable //! * **stream** (`frames == 0`, virtual host): receives real encoded AUs, writes a playable
//! elementary stream (the dump extension follows the negotiated codec — `.h265`/`.h264`/`.av1`; //! elementary stream (the dump extension follows the negotiated codec — `.h265`/`.h264`/`.av1`;
//! the probe advertises all three), and reports per-frame **capture→…→reassembled latency** //! the probe advertises all three), and reports per-frame **capture→received latency**
//! percentiles (the host stamps each frame with its capture wall clock; same-host runs share //! percentiles (the host stamps each frame with its capture wall clock; same-host runs share
//! that clock). //! that clock).
//! //!
@@ -41,7 +41,7 @@
//! Usage: `punktfunk-probe [--connect HOST:PORT] [--mode WxHxFPS] [--remode WxHxFPS:SECS] //! Usage: `punktfunk-probe [--connect HOST:PORT] [--mode WxHxFPS] [--remode WxHxFPS:SECS]
//! [--out FILE] [--bitrate KBPS] [--codec auto|h264|hevc|av1] [--audio-channels 2|6|8] //! [--out FILE] [--bitrate KBPS] [--codec auto|h264|hevc|av1] [--audio-channels 2|6|8]
//! [--launch APP] [--name NAME] [--speed-test KBPS:MS] //! [--launch APP] [--name NAME] [--speed-test KBPS:MS]
//! [--input-test | --mic-test | --touch-test | --rich-input-test] //! [--input-test | --mic-test [--mic-burst] | --touch-test | --rich-input-test]
//! [--pin HEX | --pair PIN] [--compositor NAME] [--gamepad NAME] | --discover [SECS]` //! [--pin HEX | --pair PIN] [--compositor NAME] [--gamepad NAME] | --discover [SECS]`
//! Env: `PUNKTFUNK_CLIENT_10BIT=1` / `PUNKTFUNK_CLIENT_444=1` advertise the 10-bit / 4:4:4 caps. //! Env: `PUNKTFUNK_CLIENT_10BIT=1` / `PUNKTFUNK_CLIENT_444=1` advertise the 10-bit / 4:4:4 caps.
@@ -65,6 +65,9 @@ struct Args {
input_test: bool, input_test: bool,
/// `--mic-test` — stream a synthetic 440 Hz tone as the mic uplink (proves the mic path). /// `--mic-test` — stream a synthetic 440 Hz tone as the mic uplink (proves the mic path).
mic_test: bool, mic_test: bool,
/// `--mic-burst` — pace the mic-test like a real client's input tap (2× 20 ms per 40 ms),
/// the arrival shape that exercises host-side jitter buffering.
mic_burst: bool,
/// `--touch-test` — drag a synthetic finger in a circle (proves the touch path). /// `--touch-test` — drag a synthetic finger in a circle (proves the touch path).
touch_test: bool, touch_test: bool,
/// `--rich-input-test` — drive the DualSense touchpad + motion over 0xCC (host needs /// `--rich-input-test` — drive the DualSense touchpad + motion over 0xCC (host needs
@@ -205,6 +208,7 @@ fn parse_args() -> Args {
out: get("--out").map(String::from), out: get("--out").map(String::from),
input_test: argv.iter().any(|a| a == "--input-test"), input_test: argv.iter().any(|a| a == "--input-test"),
mic_test: argv.iter().any(|a| a == "--mic-test"), mic_test: argv.iter().any(|a| a == "--mic-test"),
mic_burst: argv.iter().any(|a| a == "--mic-burst"),
touch_test: argv.iter().any(|a| a == "--touch-test"), touch_test: argv.iter().any(|a| a == "--touch-test"),
rich_input_test: argv.iter().any(|a| a == "--rich-input-test"), rich_input_test: argv.iter().any(|a| a == "--rich-input-test"),
pin, pin,
@@ -424,7 +428,9 @@ async fn session(args: Args) -> Result<()> {
// PUNKTFUNK_CLIENT_444=1 advertises VIDEO_CAP_444 (host HEVC 4:4:4 path) — verify the // PUNKTFUNK_CLIENT_444=1 advertises VIDEO_CAP_444 (host HEVC 4:4:4 path) — verify the
// resulting chroma with `ffprobe` on the `--out` .h265. // resulting chroma with `ffprobe` on the `--out` .h265.
video_caps: { video_caps: {
let mut caps = 0u8; // Always ask for per-AU host timings (0xCF) — this is a measurement tool, and the
// host/network split is exactly what it exists to report. Old hosts ignore the bit.
let mut caps = punktfunk_core::quic::VIDEO_CAP_HOST_TIMING;
if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() { if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_10BIT; caps |= punktfunk_core::quic::VIDEO_CAP_10BIT;
} }
@@ -481,7 +487,7 @@ async fn session(args: Args) -> Result<()> {
.await?; .await?;
// Wall-clock skew handshake on the still-private control stream (before --remode/--speed-test // Wall-clock skew handshake on the still-private control stream (before --remode/--speed-test
// take it): align our clock to the host's so the per-frame capture→reassembled latency is valid // take it): align our clock to the host's so the per-frame capture→received latency is valid
// across machines. `None` ⇒ an old host that doesn't answer — fall back to a shared clock (0). // across machines. `None` ⇒ an old host that doesn't answer — fall back to a shared clock (0).
let clock_offset_ns = match punktfunk_core::quic::clock_sync(&mut send, &mut recv).await { let clock_offset_ns = match punktfunk_core::quic::clock_sync(&mut send, &mut recv).await {
Some(skew) => { Some(skew) => {
@@ -738,9 +744,16 @@ async fn session(args: Args) -> Result<()> {
}); });
} }
// Mic plane: stream a synthetic 440 Hz tone as the mic uplink (0xCB), Opus-encoded 5 ms // Mic plane: stream a synthetic 440 Hz tone as the mic uplink (0xCB) — proves client→host
// stereo frames — proves client→host mic passthrough end to end without a real microphone // mic passthrough end to end without a real microphone (the host decodes it into its virtual
// (the host decodes it into its virtual PipeWire source; record that source to hear the tone). // source; record that source to hear the tone). Two pacing modes:
// default — Opus 5 ms frames on a steady 5 ms tick (smooth arrival).
// --mic-burst — two 20 ms Opus frames back-to-back every 40 ms, replicating a real
// client's input-tap cadence (the Mac client's AVAudioEngine tap yields
// ~2048-frame buffers → two packets per ~42 ms). This is the arrival
// pattern that exposed the Windows host's missing jitter buffer (constant
// crackle, 2026-07-03): a steady 5 ms stream never trips it. Record the
// host mic and count silence gaps to regression-test host-side buffering.
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
if args.mic_test { if args.mic_test {
tracing::warn!("--mic-test requires Linux (libopus) — skipped"); tracing::warn!("--mic-test requires Linux (libopus) — skipped");
@@ -748,6 +761,7 @@ async fn session(args: Args) -> Result<()> {
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
if args.mic_test { if args.mic_test {
let conn2 = conn.clone(); let conn2 = conn.clone();
let burst = args.mic_burst;
tokio::spawn(async move { tokio::spawn(async move {
let mut enc = let mut enc =
match opus::Encoder::new(48_000, opus::Channels::Stereo, opus::Application::Voip) { match opus::Encoder::new(48_000, opus::Channels::Stereo, opus::Application::Voip) {
@@ -758,15 +772,23 @@ async fn session(args: Args) -> Result<()> {
} }
}; };
let _ = enc.set_bitrate(opus::Bitrate::Bits(64_000)); let _ = enc.set_bitrate(opus::Bitrate::Bits(64_000));
tracing::info!("mic-test: streaming a 440 Hz tone as the mic uplink"); // Frame size + tick per pacing mode; `per_tick` packets are sent back-to-back.
let (frame, tick_ms, per_tick) = if burst {
(960usize, 40u64, 2u32) // 2× 20 ms every 40 ms — the bursty real-client shape
} else {
(240usize, 5u64, 1u32) // 5 ms frames on a smooth tick
};
tracing::info!(burst, "mic-test: streaming a 440 Hz tone as the mic uplink");
let mut phase = 0.0f32; let mut phase = 0.0f32;
let step = 2.0 * std::f32::consts::PI * 440.0 / 48_000.0; let step = 2.0 * std::f32::consts::PI * 440.0 / 48_000.0;
let mut pcm = [0f32; 240 * 2]; // 5 ms stereo let mut pcm = vec![0f32; frame * 2];
let mut out = [0u8; 4000]; let mut out = [0u8; 4000];
let mut interval = tokio::time::interval(std::time::Duration::from_millis(5)); let mut interval = tokio::time::interval(std::time::Duration::from_millis(tick_ms));
for seq in 0u32.. { let mut seq = 0u32;
'stream: loop {
interval.tick().await; interval.tick().await;
for f in 0..240 { for _ in 0..per_tick {
for f in 0..frame {
let s = (phase.sin()) * 0.25; let s = (phase.sin()) * 0.25;
phase += step; phase += step;
if phase > std::f32::consts::PI * 2.0 { if phase > std::f32::consts::PI * 2.0 {
@@ -778,9 +800,11 @@ async fn session(args: Args) -> Result<()> {
if let Ok(n) = enc.encode_float(&pcm, &mut out) { if let Ok(n) = enc.encode_float(&pcm, &mut out) {
let d = punktfunk_core::quic::encode_mic_datagram(seq, now_ns(), &out[..n]); let d = punktfunk_core::quic::encode_mic_datagram(seq, now_ns(), &out[..n]);
if conn2.send_datagram(d.into()).is_err() { if conn2.send_datagram(d.into()).is_err() {
break; break 'stream;
} }
} }
seq = seq.wrapping_add(1);
}
} }
tracing::info!("mic-test: done"); tracing::info!("mic-test: done");
}); });
@@ -902,6 +926,10 @@ async fn session(args: Args) -> Result<()> {
let audio_bytes = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0)); let audio_bytes = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0));
let rumble_pkts = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0)); let rumble_pkts = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0));
let hidout_pkts = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0)); let hidout_pkts = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0));
// Per-AU host timings (0xCF) → the stream loop, which matches them to received AUs by pts
// and reports the host/network split. try_send: overflow drops samples, never blocks QUIC.
let (host_timing_tx, host_timing_rx) =
std::sync::mpsc::sync_channel::<punktfunk_core::quic::HostTiming>(512);
{ {
let (a, ab, r, h) = ( let (a, ab, r, h) = (
audio_pkts.clone(), audio_pkts.clone(),
@@ -909,6 +937,7 @@ async fn session(args: Args) -> Result<()> {
rumble_pkts.clone(), rumble_pkts.clone(),
hidout_pkts.clone(), hidout_pkts.clone(),
); );
let ht_tx = host_timing_tx;
let conn2 = conn.clone(); let conn2 = conn.clone();
// Build a multistream decoder for the host-RESOLVED layout so the probe actually decodes // Build a multistream decoder for the host-RESOLVED layout so the probe actually decodes
// the surround stream (not just counts bytes) — the headless validator for the encode path. // the surround stream (not just counts bytes) — the headless validator for the encode path.
@@ -956,6 +985,10 @@ async fn session(args: Args) -> Result<()> {
if h.fetch_add(1, Relaxed) < 12 { if h.fetch_add(1, Relaxed) < 12 {
tracing::info!(?hid, "DualSense HID output (0xCD)"); tracing::info!(?hid, "DualSense HID output (0xCD)");
} }
} else if let Some(t) = punktfunk_core::quic::decode_host_timing_datagram(&d) {
// Per-AU host timing (0xCF) — forwarded to the stream loop for the
// host/network latency split.
let _ = ht_tx.try_send(t);
} }
} }
}); });
@@ -1000,6 +1033,12 @@ async fn session(args: Args) -> Result<()> {
let mut mismatched = 0u32; let mut mismatched = 0u32;
let mut bytes = 0u64; let mut bytes = 0u64;
let mut latencies_us: Vec<u64> = Vec::new(); let mut latencies_us: Vec<u64> = Vec::new();
// Host/network split: received AUs awaiting their 0xCF host timing (pts → capture→received
// µs), matched as the datagrams arrive. Bounded — an old host never sends any.
let mut pending_split: std::collections::VecDeque<(u64, u64)> =
std::collections::VecDeque::new();
let mut host_us_v: Vec<u64> = Vec::new();
let mut net_us_v: Vec<u64> = Vec::new();
let mut last_rx = std::time::Instant::now(); let mut last_rx = std::time::Instant::now();
let started = std::time::Instant::now(); let started = std::time::Instant::now();
// Adaptive-FEC loss window: publish a fresh estimate every 750 ms for the LossReport task. // Adaptive-FEC loss window: publish a fresh estimate every 750 ms for the LossReport task.
@@ -1051,12 +1090,25 @@ async fn session(args: Args) -> Result<()> {
continue; continue;
} }
bytes += frame.data.len() as u64; bytes += frame.data.len() as u64;
// capture→reassembled: our receive instant in the host clock (now + offset) // capture→received: our receive instant in the host clock (now + offset)
// minus the host's capture pts. offset is 0 same-host / old host. // minus the host's capture pts. offset is 0 same-host / old host.
let lat = (now_ns() as i128 + clock_offset as i128 - frame.pts_ns as i128) let lat = (now_ns() as i128 + clock_offset as i128 - frame.pts_ns as i128)
.max(0) as u64; .max(0) as u64;
if lat > 0 && lat < 10_000_000_000 { if lat > 0 && lat < 10_000_000_000 {
latencies_us.push(lat / 1000); latencies_us.push(lat / 1000);
pending_split.push_back((frame.pts_ns, lat / 1000));
if pending_split.len() > 1024 {
pending_split.pop_front();
}
}
// Match any host timings (0xCF) that have arrived: host = the reported
// capture→sent, network = our capture→received minus it (per-frame tiling).
while let Ok(t) = host_timing_rx.try_recv() {
if let Some(i) = pending_split.iter().position(|(p, _)| *p == t.pts_ns) {
let (_, hostnet_us) = pending_split.remove(i).unwrap();
host_us_v.push(t.host_us as u64);
net_us_v.push(hostnet_us.saturating_sub(t.host_us as u64));
}
} }
if expected > 0 { if expected > 0 {
// Verification mode: deterministic content. // Verification mode: deterministic content.
@@ -1100,9 +1152,31 @@ async fn session(args: Args) -> Result<()> {
lat_p99_us = pct(0.99), lat_p99_us = pct(0.99),
lat_max_us = latencies_us.last().copied().unwrap_or(0), lat_max_us = latencies_us.last().copied().unwrap_or(0),
skew_corrected, skew_corrected,
"punktfunk/1 stream complete (capture→reassembled latency; skew_corrected=true ⇒ \ "punktfunk/1 stream complete (capture→received latency; skew_corrected=true ⇒ \
cross-machine valid, false ⇒ same-host clock)" cross-machine valid, false ⇒ same-host clock)"
); );
if !host_us_v.is_empty() {
// The host/network split from the per-AU 0xCF timings (design/stats-unification.md
// Phase 2): host = the host's own capture→sent, network = capture→received minus it.
let pcts = |v: &mut Vec<u64>, p: f64| -> u64 {
if v.is_empty() {
return 0;
}
v.sort_unstable();
v[((v.len() as f64 * p) as usize).min(v.len() - 1)]
};
tracing::info!(
timing_samples = host_us_v.len(),
host_p50_us = pcts(&mut host_us_v, 0.50),
host_p95_us = pcts(&mut host_us_v, 0.95),
net_p50_us = pcts(&mut net_us_v, 0.50),
net_p95_us = pcts(&mut net_us_v, 0.95),
"host/network latency split (host = capture→sent on the host; network = wire + \
reassembly)"
);
} else {
tracing::info!("no host timing datagrams (0xCF) — old host; host+network unsplit");
}
if expected > 0 { if expected > 0 {
anyhow::ensure!(mismatched == 0, "{mismatched} corrupted frames"); anyhow::ensure!(mismatched == 0, "{mismatched} corrupted frames");
anyhow::ensure!(ok == expected, "received {ok}/{expected} frames"); anyhow::ensure!(ok == expected, "received {ok}/{expected} frames");
+46 -19
View File
@@ -2,7 +2,7 @@
//! the UI thread, then handed — presenter and all — to the dedicated render thread //! the UI thread, then handed — presenter and all — to the dedicated render thread
//! ([`crate::render`]), which presents decoded frames at stream cadence. The page itself only //! ([`crate::render`]), which presents decoded frames at stream cadence. The page itself only
//! forwards panel size/DPI changes and draws the status-chip HUD overlay (mode · decode path · //! forwards panel size/DPI changes and draws the status-chip HUD overlay (mode · decode path ·
//! HDR · fps/throughput/latency · capture hint). //! HDR · fps/goodput · end-to-end latency + stage equation · capture hint).
use super::style::{edges, uniform}; use super::style::{edges, uniform};
use super::Svc; use super::Svc;
@@ -22,8 +22,9 @@ use windows_reactor::*;
pub(crate) struct HudSample { pub(crate) struct HudSample {
pub(crate) stats: Stats, pub(crate) stats: Stats,
pub(crate) captured: bool, pub(crate) captured: bool,
/// `(presents/s, skipped/s, capture→presented p50 ms)` — see [`crate::render::present_stats`]. /// The render thread's glass-side window (presents/s, skips, end-to-end p50/p95, display
pub(crate) present: (u32, u32, f32), /// stage p50) — see [`crate::render::present_stats`].
pub(crate) present: crate::render::PresentStats,
} }
/// Props for the stream page: the services plus the live HUD sample that drives the overlay /// Props for the stream page: the services plus the live HUD sample that drives the overlay
@@ -171,13 +172,16 @@ fn fmt_uptime(secs: u32) -> String {
} }
} }
/// The streaming HUD overlay (top-right), mirroring the Apple client: a chip row (mode · codec · /// The streaming HUD overlay (top-right), unified stats vocabulary (design/stats-unification.md):
/// decode path · HDR), a stream line (decode fps / bitrate / decode time), a glass line (display /// a chip row (mode · codec · decode path · HDR), a stream line (received fps · goodput ·
/// presents + end-to-end latency decoded vs on-glass), a session line (host · time · loss), and /// presenter fps), the end-to-end headline (capture→on-glass p50/p95, host-clock corrected), the
/// the shortcut hints. Layered over the `SwapChainPanel` in the same grid cell. /// stage equation (= host + network + decode + display when the host reports 0xCF timings, else
/// the combined = host+network + decode + display; stage p50s), a session line
/// (host · time · loss/skips), and the shortcut hints. Layered over the `SwapChainPanel` in the
/// same grid cell.
fn hud_overlay(hud: &HudSample, mode: Option<Mode>, host: &str) -> Element { fn hud_overlay(hud: &HudSample, mode: Option<Mode>, host: &str) -> Element {
let stats = &hud.stats; let stats = &hud.stats;
let (pfps, skipped, glass_ms) = hud.present; let present = &hud.present;
let res = mode let res = mode
.map(|m| format!("{}\u{00D7}{}@{}", m.width, m.height, m.refresh_hz)) .map(|m| format!("{}\u{00D7}{}@{}", m.width, m.height, m.refresh_hz))
.unwrap_or_else(|| "\u{2014}".into()); .unwrap_or_else(|| "\u{2014}".into());
@@ -193,25 +197,47 @@ fn hud_overlay(hud: &HudSample, mode: Option<Mode>, host: &str) -> Element {
if stats.hdr { if stats.hdr {
chips.push(hud_chip("HDR", Color::rgb(255, 205, 90)).into()); chips.push(hud_chip("HDR", Color::rgb(255, 205, 90)).into());
} }
// Received fps + goodput, plus the presenter's own rate (Moonlight's "Rendering frame rate"
// analog — how often the display actually gets a new frame).
let stream_line = format!( let stream_line = format!(
"{:.0} fps \u{00B7} {:.1} Mb/s \u{00B7} decode {:.1} ms", "{:.0} fps \u{00B7} {:.1} Mb/s \u{00B7} display {} fps",
stats.fps, stats.mbps, stats.decode_ms stats.fps, stats.mbps, present.fps
); );
// End-to-end latency (host-clock corrected): capture→decoded from the pump, capture→on-glass // The headline: end-to-end capture→displayed, measured directly post-Present (never the sum
// from the render thread's post-Present stamp. `skipped` = newest-wins drops (expected when // of the stage percentiles). `(same-host clock)` flags an uncorrected clock (offset == 0:
// the stream outpaces the display); `lost` = unrecoverable network drops. // same host, or the host skipped the skew handshake).
let glass_line = format!( let mut e2e_line = format!(
"display {pfps} fps \u{00B7} latency {:.1} ms decoded / {glass_ms:.1} ms on-glass", "end-to-end {:.1} ms p50 \u{00B7} {:.1} p95 \u{00B7} capture\u{2192}on-glass",
stats.latency_ms present.e2e_p50_ms, present.e2e_p95_ms
); );
if stats.same_host {
e2e_line.push_str(" (same-host clock)");
}
// The equation: the stages tile the headline interval per frame; the window p50s only
// approximately sum (percentiles aren't additive). With per-AU 0xCF host timings the opaque
// `host+network` term splits into `host` (host capture→sent) + `network` (the remainder);
// an old host emits none and the combined term stays.
let stage_line = if stats.split {
format!(
"= host {:.1} + network {:.1} + decode {:.1} + display {:.1}",
stats.host_ms, stats.net_ms, stats.decode_ms, present.display_p50_ms
)
} else {
format!(
"= host+network {:.1} + decode {:.1} + display {:.1}",
stats.hostnet_ms, stats.decode_ms, present.display_p50_ms
)
};
let mut session_bits: Vec<String> = Vec::new(); let mut session_bits: Vec<String> = Vec::new();
if !host.is_empty() { if !host.is_empty() {
session_bits.push(host.to_string()); session_bits.push(host.to_string());
} }
// `lost` = unrecoverable network drops (session-cumulative); `skipped` = the render thread's
// newest-wins drops last window (expected when the stream outpaces the display).
session_bits.push(fmt_uptime(stats.uptime_secs)); session_bits.push(fmt_uptime(stats.uptime_secs));
session_bits.push(format!("{} lost", stats.dropped)); session_bits.push(format!("{} lost", stats.dropped));
if skipped > 0 { if present.skipped > 0 {
session_bits.push(format!("{skipped} skipped")); session_bits.push(format!("{} skipped", present.skipped));
} }
let session_line = session_bits.join(" \u{00B7} "); let session_line = session_bits.join(" \u{00B7} ");
let hint = if hud.captured { let hint = if hud.captured {
@@ -228,7 +254,8 @@ fn hud_overlay(hud: &HudSample, mode: Option<Mode>, host: &str) -> Element {
vstack(( vstack((
hstack(chips).spacing(6.0), hstack(chips).spacing(6.0),
dim(&stream_line), dim(&stream_line),
dim(&glass_line), dim(&e2e_line),
dim(&stage_line),
dim(&session_line), dim(&session_line),
text_block(hint) text_block(hint)
.font_size(11.0) .font_size(11.0)
+14 -2
View File
@@ -238,11 +238,23 @@ fn run_headless_cli(args: &[String], identity: (String, String)) {
session::SessionEvent::Connected { session::SessionEvent::Connected {
mode, fingerprint, .. mode, fingerprint, ..
} => tracing::info!(?mode, fp = %trust::hex(&fingerprint), "connected"), } => tracing::info!(?mode, fp = %trust::hex(&fingerprint), "connected"),
// With per-AU 0xCF host timings the combined host+network stage splits into
// host (capture→sent on the host) + net; an old host emits none → combined only.
session::SessionEvent::Stats(s) if s.split => tracing::info!(
fps = format!("{:.0}", s.fps),
mbps = format!("{:.1}", s.mbps),
decode_p50_ms = format!("{:.2}", s.decode_ms),
hostnet_p50_ms = format!("{:.2}", s.hostnet_ms),
host_p50_ms = format!("{:.2}", s.host_ms),
net_p50_ms = format!("{:.2}", s.net_ms),
frames_seen,
"stats"
),
session::SessionEvent::Stats(s) => tracing::info!( session::SessionEvent::Stats(s) => tracing::info!(
fps = format!("{:.0}", s.fps), fps = format!("{:.0}", s.fps),
mbps = format!("{:.1}", s.mbps), mbps = format!("{:.1}", s.mbps),
decode_ms = format!("{:.2}", s.decode_ms), decode_p50_ms = format!("{:.2}", s.decode_ms),
lat_ms = format!("{:.2}", s.latency_ms), hostnet_p50_ms = format!("{:.2}", s.hostnet_ms),
frames_seen, frames_seen,
"stats" "stats"
), ),
+78 -27
View File
@@ -10,27 +10,46 @@
//! draw (and redraws the held frame after a resize — fresh back buffers are blank). //! draw (and redraws the held frame after a resize — fresh back buffers are blank).
use crate::present::Presenter; use crate::present::Presenter;
use crate::session::FrameRx; use crate::session::{FrameRx, FrameTimes};
use crossbeam_channel::RecvTimeoutError; use crossbeam_channel::RecvTimeoutError;
use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, Ordering}; use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, Ordering};
use std::sync::Arc; use std::sync::Arc;
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
/// The last 1-second render window, published for the HUD (one render thread at a time): /// The last 1-second render window, published for the HUD (one render thread at a time):
/// presents/s, frames skipped by the newest-wins drain, and the capture→presented p50 in µs. /// presents/s, frames skipped by the newest-wins drain, the end-to-end (capture→on-glass)
/// p50/p95 and the `display` stage (decoded→displayed) p50, all stamped post-`Present()`, in µs.
/// Zeroed when a render thread starts so a new session never shows the previous one's numbers. /// Zeroed when a render thread starts so a new session never shows the previous one's numbers.
static PRESENT_FPS: AtomicU32 = AtomicU32::new(0); static PRESENT_FPS: AtomicU32 = AtomicU32::new(0);
static PRESENT_SKIPPED: AtomicU32 = AtomicU32::new(0); static PRESENT_SKIPPED: AtomicU32 = AtomicU32::new(0);
static PRESENT_P50_US: AtomicU64 = AtomicU64::new(0); static E2E_P50_US: AtomicU64 = AtomicU64::new(0);
static E2E_P95_US: AtomicU64 = AtomicU64::new(0);
static DISPLAY_P50_US: AtomicU64 = AtomicU64::new(0);
/// `(presents/s, skipped/s, capture→presented p50 ms)` of the last render window — the HUD's /// The last render window's glass-side numbers (see the statics above) — the HUD's headline
/// display-side line. /// (end-to-end) and trailing stage (display) come from here.
pub fn present_stats() -> (u32, u32, f32) { #[derive(Clone, Copy, Default, PartialEq)]
( pub struct PresentStats {
PRESENT_FPS.load(Ordering::Relaxed), /// Presents per second (includes resize redraws of a held frame).
PRESENT_SKIPPED.load(Ordering::Relaxed), pub fps: u32,
PRESENT_P50_US.load(Ordering::Relaxed) as f32 / 1000.0, /// Frames dropped by the newest-wins drain this window (client-side pacing skips).
) pub skipped: u32,
/// End-to-end capture→displayed p50, ms (host-clock corrected, measured directly).
pub e2e_p50_ms: f32,
/// End-to-end capture→displayed p95, ms.
pub e2e_p95_ms: f32,
/// `display` stage p50, ms: decoded → displayed, single-clock client-local.
pub display_p50_ms: f32,
}
pub fn present_stats() -> PresentStats {
PresentStats {
fps: PRESENT_FPS.load(Ordering::Relaxed),
skipped: PRESENT_SKIPPED.load(Ordering::Relaxed),
e2e_p50_ms: E2E_P50_US.load(Ordering::Relaxed) as f32 / 1000.0,
e2e_p95_ms: E2E_P95_US.load(Ordering::Relaxed) as f32 / 1000.0,
display_p50_ms: DISPLAY_P50_US.load(Ordering::Relaxed) as f32 / 1000.0,
}
} }
/// UI-thread → render-thread state. Size is packed into ONE atomic (w<<32|h) so a resize never /// UI-thread → render-thread state. Size is packed into ONE atomic (w<<32|h) so a resize never
@@ -101,8 +120,9 @@ impl Drop for RenderThread {
struct SendPresenter(Presenter); struct SendPresenter(Presenter);
unsafe impl Send for SendPresenter {} unsafe impl Send for SendPresenter {}
/// Spawn the render thread. `frames` carries `(frame, capture pts_ns)`; `clock_offset_ns` maps our /// Spawn the render thread. `frames` carries `(frame, FrameTimes)`; `clock_offset_ns` maps our
/// wall clock onto the host's so the logged present latency is end-to-end (same math as the pump). /// wall clock onto the host's so the end-to-end (capture→on-glass) number is cross-machine valid
/// (same math as the pump's host+network stage).
pub fn spawn( pub fn spawn(
presenter: Presenter, presenter: Presenter,
frames: FrameRx, frames: FrameRx,
@@ -147,12 +167,17 @@ fn run(presenter: SendPresenter, frames: FrameRx, shared: Arc<RenderShared>, clo
let mut applied = (0u32, 0u32, 0u32); // last (w, h, dpi) handed to the presenter let mut applied = (0u32, 0u32, 0u32); // last (w, h, dpi) handed to the presenter
let mut presented = 0u32; let mut presented = 0u32;
let mut dropped = 0u32; let mut dropped = 0u32;
let mut lat_us: Vec<u64> = Vec::with_capacity(256); // 1 s tumbling windows: end-to-end (capture→displayed) and the display stage
// (decoded→displayed), sampled post-Present. Percentiles only (spec: stats-unification.md).
let mut e2e_us: Vec<u64> = Vec::with_capacity(256);
let mut display_us: Vec<u64> = Vec::with_capacity(256);
let mut window_start = Instant::now(); let mut window_start = Instant::now();
let mut last_dpi_poll = Instant::now(); let mut last_dpi_poll = Instant::now();
PRESENT_FPS.store(0, Ordering::Relaxed); PRESENT_FPS.store(0, Ordering::Relaxed);
PRESENT_SKIPPED.store(0, Ordering::Relaxed); PRESENT_SKIPPED.store(0, Ordering::Relaxed);
PRESENT_P50_US.store(0, Ordering::Relaxed); E2E_P50_US.store(0, Ordering::Relaxed);
E2E_P95_US.store(0, Ordering::Relaxed);
DISPLAY_P50_US.store(0, Ordering::Relaxed);
loop { loop {
if shared.stop.load(Ordering::SeqCst) { if shared.stop.load(Ordering::SeqCst) {
@@ -198,29 +223,55 @@ fn run(presenter: SendPresenter, frames: FrameRx, shared: Arc<RenderShared>, clo
p.set_hdr_metadata(meta); p.set_hdr_metadata(meta);
} }
let pts_ns = newest.as_ref().map(|(_, pts)| *pts); let times: Option<FrameTimes> = newest.as_ref().map(|(_, t)| *t);
p.present(newest.map(|(f, _)| f)); p.present(newest.map(|(f, _)| f));
presented += 1; presented += 1;
if let Some(pts) = pts_ns { if let Some(t) = times {
// Capture→presented, host-clock corrected — the glass-side companion to the pump's // The `displayed` point: post-Present() on this thread (the honest best-effort
// capture→decoded p50. // presentation instant on Windows — endpoint label `capture→on-glass`).
let lat = (now_ns() as i128 + clock_offset_ns as i128 - pts as i128).max(0) as u64; let displayed_ns = now_ns();
if lat > 0 && lat < 10_000_000_000 { // End-to-end = capture → displayed, host-clock corrected, measured directly
lat_us.push(lat / 1000); // (never the sum of stage percentiles). Clamped (0, 10 s).
let e2e =
(displayed_ns as i128 + clock_offset_ns as i128 - t.pts_ns as i128).max(0) as u64;
if e2e > 0 && e2e < 10_000_000_000 {
e2e_us.push(e2e / 1000);
}
// `display` stage = decoded → displayed, single-clock client-local.
let disp = displayed_ns.saturating_sub(t.decoded_ns);
if disp < 10_000_000_000 {
display_us.push(disp / 1000);
} }
} }
if window_start.elapsed() >= Duration::from_secs(1) { if window_start.elapsed() >= Duration::from_secs(1) {
lat_us.sort_unstable(); e2e_us.sort_unstable();
let p50 = lat_us.get(lat_us.len() / 2).copied().unwrap_or(0); display_us.sort_unstable();
tracing::debug!(presented, dropped, present_p50_us = p50, "render window"); let p50 = |v: &[u64]| v.get(v.len() / 2).copied().unwrap_or(0);
// p95 = sorted[min(len*95/100, len-1)] — the empty-window case falls to 0 via `get`.
let p95 = |v: &[u64]| {
v.get((v.len() * 95 / 100).min(v.len().saturating_sub(1)))
.copied()
.unwrap_or(0)
};
tracing::debug!(
presented,
dropped,
e2e_p50_us = p50(&e2e_us),
e2e_p95_us = p95(&e2e_us),
display_p50_us = p50(&display_us),
"render window"
);
PRESENT_FPS.store(presented, Ordering::Relaxed); PRESENT_FPS.store(presented, Ordering::Relaxed);
PRESENT_SKIPPED.store(dropped, Ordering::Relaxed); PRESENT_SKIPPED.store(dropped, Ordering::Relaxed);
PRESENT_P50_US.store(p50, Ordering::Relaxed); E2E_P50_US.store(p50(&e2e_us), Ordering::Relaxed);
E2E_P95_US.store(p95(&e2e_us), Ordering::Relaxed);
DISPLAY_P50_US.store(p50(&display_us), Ordering::Relaxed);
window_start = Instant::now(); window_start = Instant::now();
presented = 0; presented = 0;
dropped = 0; dropped = 0;
lat_us.clear(); e2e_us.clear();
display_us.clear();
} }
} }
tracing::info!("render thread exiting"); tracing::info!("render thread exiting");
+103 -30
View File
@@ -46,11 +46,27 @@ pub struct SessionParams {
#[derive(Clone, Copy, Default, PartialEq)] #[derive(Clone, Copy, Default, PartialEq)]
pub struct Stats { pub struct Stats {
/// AUs received (reassembled) per second — actual-elapsed-time denominator.
pub fps: f32, pub fps: f32,
/// Received payload goodput (excludes FEC overhead).
pub mbps: f32, pub mbps: f32,
/// `decode` stage p50 over the last 1 s window: received → decoded, client-local clock.
pub decode_ms: f32, pub decode_ms: f32,
/// Median capture→decoded latency over the last window (host-clock corrected). /// `host+network` stage p50 over the last 1 s window: capture (`pts_ns`) → received,
pub latency_ms: f32, /// host-clock corrected via `clock_offset_ns`.
pub hostnet_ms: f32,
/// `host` stage p50 (host capture→sent, from the per-AU 0xCF host-timing plane). Valid only
/// when `split` — an old host emits no 0xCF and the HUD keeps the combined stage.
pub host_ms: f32,
/// `network` stage p50 (`hostnet host`, tiled per frame before taking the percentile).
/// Valid only when `split`.
pub net_ms: f32,
/// True when any 0xCF host timings matched received AUs this window — the HUD then renders
/// `host + network` instead of the combined `host+network` term.
pub split: bool,
/// True when `clock_offset_ns == 0` (host didn't answer the skew handshake / same host) —
/// the HUD appends `(same-host clock)` to the end-to-end line.
pub same_host: bool,
/// True when decoding on the GPU (D3D11VA) vs. CPU (software). /// True when decoding on the GPU (D3D11VA) vs. CPU (software).
pub hardware: bool, pub hardware: bool,
/// True when the stream is BT.2020 PQ HDR10 (last decoded frame). /// True when the stream is BT.2020 PQ HDR10 (last decoded frame).
@@ -81,9 +97,19 @@ pub enum SessionEvent {
Stats(Stats), Stats(Stats),
} }
/// Decoded frames + their host-capture `pts_ns`, session pump → render thread (crossbeam so that /// Per-frame measurement points carried with a decoded frame to the render thread: the host
/// capture clock (`pts_ns`) and our local `decoded` stamp (wall-clock ns). Post-`Present()` the
/// render thread derives the `display` stage (displayed decoded, single-clock) and the
/// end-to-end headline (displayed + clock_offset pts) from them.
#[derive(Clone, Copy)]
pub struct FrameTimes {
pub pts_ns: u64,
pub decoded_ns: u64,
}
/// Decoded frames + their measurement points, session pump → render thread (crossbeam so that
/// thread can block with a timeout — async-channel has no `recv_timeout`). /// thread can block with a timeout — async-channel has no `recv_timeout`).
pub type FrameRx = crossbeam_channel::Receiver<(DecodedFrame, u64)>; pub type FrameRx = crossbeam_channel::Receiver<(DecodedFrame, FrameTimes)>;
pub struct SessionHandle { pub struct SessionHandle {
pub events: async_channel::Receiver<SessionEvent>, pub events: async_channel::Receiver<SessionEvent>,
@@ -205,7 +231,7 @@ impl AudioDec {
fn pump( fn pump(
params: SessionParams, params: SessionParams,
ev_tx: async_channel::Sender<SessionEvent>, ev_tx: async_channel::Sender<SessionEvent>,
frame_tx: crossbeam_channel::Sender<(DecodedFrame, u64)>, frame_tx: crossbeam_channel::Sender<(DecodedFrame, FrameTimes)>,
frame_rx: FrameRx, frame_rx: FrameRx,
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
) { ) {
@@ -310,8 +336,15 @@ fn pump(
let mut window_start = Instant::now(); let mut window_start = Instant::now();
let mut frames_n = 0u32; let mut frames_n = 0u32;
let mut bytes_n = 0u64; let mut bytes_n = 0u64;
let mut decode_us_sum = 0u64; // 1 s tumbling stage windows (spec: design/stats-unification.md — percentiles, never means).
let mut lat_us: Vec<u64> = Vec::with_capacity(256); let mut hostnet_us: Vec<u64> = Vec::with_capacity(256);
let mut decode_us: Vec<u64> = Vec::with_capacity(256);
// Host/network split (Phase 2): received AUs awaiting their 0xCF host timing, `(pts_ns,
// hostnet_us)`, matched as the datagrams arrive. Bounded — an old host never sends any.
let mut pending_split: std::collections::VecDeque<(u64, u64)> =
std::collections::VecDeque::with_capacity(256);
let mut host_us_w: Vec<u64> = Vec::with_capacity(256);
let mut net_us_w: Vec<u64> = Vec::with_capacity(256);
let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs. // Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped(); let mut last_dropped = connector.frames_dropped();
@@ -323,7 +356,23 @@ fn pump(
} }
match connector.next_frame(Duration::from_millis(4)) { match connector.next_frame(Duration::from_millis(4)) {
Ok(frame) => { Ok(frame) => {
let t0 = Instant::now(); // The `received` point: AU fully reassembled, handed to us, before decode.
let received_ns = now_ns();
// fps = AUs received per second, Mb/s = received goodput (spec: counted at the
// received point, not the decoded one).
frames_n += 1;
bytes_n += frame.data.len() as u64;
// `host+network` stage: capture → received, host-clock corrected. Clamped (0, 10 s).
let hostnet = (received_ns as i128 + clock_offset as i128 - frame.pts_ns as i128)
.max(0) as u64;
if hostnet > 0 && hostnet < 10_000_000_000 {
hostnet_us.push(hostnet / 1000);
// Remember this AU for the 0xCF match below (host/network split).
pending_split.push_back((frame.pts_ns, hostnet / 1000));
if pending_split.len() > 256 {
pending_split.pop_front();
}
}
// A D3D11VA→software demotion (see `Decoder::decode`) starts a FRESH decoder that // A D3D11VA→software demotion (see `Decoder::decode`) starts a FRESH decoder that
// has none of the stream's parameter sets; under infinite GOP it would sit on // has none of the stream's parameter sets; under infinite GOP it would sit on
// "PPS id out of range" forever. Detect the transition and force a new IDR so the // "PPS id out of range" forever. Detect the transition and force a new IDR so the
@@ -336,6 +385,8 @@ fn pump(
} }
match decoded { match decoded {
Ok(Some(decoded)) => { Ok(Some(decoded)) => {
// The `decoded` point: decoder output frame available.
let decoded_ns = now_ns();
total_frames += 1; total_frames += 1;
hdr = decoded.hdr(); hdr = decoded.hdr();
// The backend can demote D3D11VA → software mid-session on a hardware error. // The backend can demote D3D11VA → software mid-session on a hardware error.
@@ -350,19 +401,17 @@ fn pump(
"first frame decoded" "first frame decoded"
); );
} }
// Latency: our wall clock expressed in the host's capture clock, // `decode` stage: received → decoded, single-clock client-local.
// minus the host-stamped capture pts (same math as client-rs). decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
let lat = (now_ns() as i128 + clock_offset as i128 - frame.pts_ns as i128)
.max(0) as u64;
if lat > 0 && lat < 10_000_000_000 {
lat_us.push(lat / 1000);
}
decode_us_sum += t0.elapsed().as_micros() as u64;
frames_n += 1;
bytes_n += frame.data.len() as u64;
// Newest wins: displace the oldest queued frame when the renderer lags. // Newest wins: displace the oldest queued frame when the renderer lags.
if let Err(crossbeam_channel::TrySendError::Full(item)) = if let Err(crossbeam_channel::TrySendError::Full(item)) =
frame_tx.try_send((decoded, frame.pts_ns)) frame_tx.try_send((
decoded,
FrameTimes {
pts_ns: frame.pts_ns,
decoded_ns,
},
))
{ {
let _ = frame_rx.try_recv(); let _ = frame_rx.try_recv();
let _ = frame_tx.try_send(item); let _ = frame_tx.try_send(item);
@@ -411,25 +460,47 @@ fn pump(
*crate::present::LATEST_HDR_META.lock().unwrap() = Some(meta); *crate::present::LATEST_HDR_META.lock().unwrap() = Some(meta);
} }
// Drain the per-AU host-timing plane (0xCF) and match by pts: `host` = the host's own
// capture→sent, `network` = our capture→received minus it — the two tile per frame
// (design/stats-unification.md Phase 2). An old host never emits any; `split` stays false
// and the HUD keeps the combined `host+network` stage.
while let Ok(t) = connector.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|(p, _)| *p == t.pts_ns) {
let (_, hn_us) = pending_split.remove(i).unwrap();
host_us_w.push(t.host_us as u64);
net_us_w.push(hn_us.saturating_sub(t.host_us as u64));
}
}
if window_start.elapsed() >= Duration::from_secs(1) { if window_start.elapsed() >= Duration::from_secs(1) {
let secs = window_start.elapsed().as_secs_f32(); let secs = window_start.elapsed().as_secs_f32();
lat_us.sort_unstable(); hostnet_us.sort_unstable();
let p50 = lat_us.get(lat_us.len() / 2).copied().unwrap_or(0); decode_us.sort_unstable();
host_us_w.sort_unstable();
net_us_w.sort_unstable();
let p50 = |v: &[u64]| v.get(v.len() / 2).copied().unwrap_or(0);
let (hostnet_p50, decode_p50) = (p50(&hostnet_us), p50(&decode_us));
let (host_p50, net_p50) = (p50(&host_us_w), p50(&net_us_w));
let split = !host_us_w.is_empty();
tracing::debug!( tracing::debug!(
fps = frames_n, fps = frames_n,
lat_p50_us = p50, hostnet_p50_us = hostnet_p50,
host_p50_us = host_p50,
net_p50_us = net_p50,
split,
decode_p50_us = decode_p50,
total_frames, total_frames,
"stream window" "stream window"
); );
let _ = ev_tx.try_send(SessionEvent::Stats(Stats { let _ = ev_tx.try_send(SessionEvent::Stats(Stats {
fps: frames_n as f32 / secs, fps: frames_n as f32 / secs,
mbps: bytes_n as f32 * 8.0 / 1e6 / secs, mbps: bytes_n as f32 * 8.0 / 1e6 / secs,
decode_ms: if frames_n > 0 { decode_ms: decode_p50 as f32 / 1000.0,
decode_us_sum as f32 / frames_n as f32 / 1000.0 hostnet_ms: hostnet_p50 as f32 / 1000.0,
} else { host_ms: host_p50 as f32 / 1000.0,
0.0 net_ms: net_p50 as f32 / 1000.0,
}, split,
latency_ms: p50 as f32 / 1000.0, same_host: clock_offset == 0,
hardware, hardware,
hdr, hdr,
codec: connector.codec, codec: connector.codec,
@@ -439,8 +510,10 @@ fn pump(
window_start = Instant::now(); window_start = Instant::now();
frames_n = 0; frames_n = 0;
bytes_n = 0; bytes_n = 0;
decode_us_sum = 0; hostnet_us.clear();
lat_us.clear(); decode_us.clear();
host_us_w.clear();
net_us_w.clear();
} }
}; };
+220 -47
View File
@@ -2,11 +2,17 @@
//! //!
//! Two planes: //! Two planes:
//! * [`control`] — the low-frequency `DeviceIoControl` plane (add/remove a virtual monitor, pin the //! * [`control`] — the low-frequency `DeviceIoControl` plane (add/remove a virtual monitor, pin the
//! render adapter, keepalive, info, clear-all). Owned, clean, versioned — NOT the SudoVDA ABI. //! render adapter, keepalive, info, clear-all, deliver the frame channel). Owned, clean, versioned —
//! * [`frame`] — the IDD-push frame transport: the host creates a ring of shared keyed-mutex textures //! NOT the SudoVDA ABI.
//! (+ a header + a frame-ready event) and the driver opens them and publishes composited frames into //! * [`frame`] — the IDD-push frame transport: the host creates a ring of **unnamed** shared
//! them. This crate owns the [`frame::SharedHeader`] layout, the [`frame::FrameToken`] packing, the //! keyed-mutex textures (+ a header + a frame-ready event), duplicates their handles into the
//! `Global\` object-name scheme, and the driver-status codes. //! driver's WUDFHost process and delivers the handle VALUES over
//! [`control::IOCTL_SET_FRAME_CHANNEL`]; the driver publishes composited frames into them. There is
//! deliberately no object-name scheme: an unnamed object cannot be enumerated, opened by name, or
//! pre-created ("squatted") — only the two endpoint processes ever hold a handle to any frame object
//! (the sealed channel, `design/idd-push-security.md`). This crate owns the [`frame::SharedHeader`]
//! layout, the [`frame::FrameToken`] packing, the channel-delivery struct, and the driver-status
//! codes.
//! //!
//! Both planes were previously hand-duplicated, byte-for-byte, across `idd_push.rs`/`frame_transport.rs` //! Both planes were previously hand-duplicated, byte-for-byte, across `idd_push.rs`/`frame_transport.rs`
//! and `vdisplay/sudovda.rs`/`control.rs` with only "must match" comments guarding them. Defining them //! and `vdisplay/sudovda.rs`/`control.rs` with only "must match" comments guarding them. Defining them
@@ -43,16 +49,22 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// Bumped on any incompatible change to either plane. Exchanged via [`control::IOCTL_GET_INFO`]; host /// Bumped on any incompatible change to either plane. Exchanged via [`control::IOCTL_GET_INFO`]; host
/// and driver assert a match at startup so a mismatched pair fails loudly instead of corrupting. /// and driver assert a match at startup so a mismatched pair fails loudly instead of corrupting.
pub const PROTOCOL_VERSION: u32 = 1; /// v2: the sealed frame channel — the frame objects are unnamed and delivered by handle duplication
/// ([`control::IOCTL_SET_FRAME_CHANNEL`]), and [`control::AddReply`] grew `wudf_pid` (the duplication
/// target). A v1 driver has no channel-delivery IOCTL and expects named objects, so the pairing is
/// incompatible by design.
pub const PROTOCOL_VERSION: u32 = 2;
/// `CTL_CODE(FILE_DEVICE_UNKNOWN = 0x22, func, METHOD_BUFFERED = 0, FILE_ANY_ACCESS = 0)`. /// `CTL_CODE(FILE_DEVICE_UNKNOWN = 0x22, func, METHOD_BUFFERED = 0, FILE_ANY_ACCESS = 0)`.
pub const fn ctl_code(func: u32) -> u32 { pub const fn ctl_code(func: u32) -> u32 {
(0x22u32 << 16) | (func << 2) (0x22u32 << 16) | (func << 2)
} }
/// The control (`DeviceIoControl`) plane: add/remove a virtual monitor + adapter pin + keepalive. /// The control (`DeviceIoControl`) plane: add/remove a virtual monitor + adapter pin + keepalive +
/// frame-channel delivery.
pub mod control { pub mod control {
use super::ctl_code; use super::ctl_code;
use super::frame::RING_LEN;
use bytemuck::{Pod, Zeroable}; use bytemuck::{Pod, Zeroable};
// Contiguous op space at 0x900 — distinct from SudoVDA's gappy 0x800/0x888/0x8FF numbering. // Contiguous op space at 0x900 — distinct from SudoVDA's gappy 0x800/0x888/0x8FF numbering.
@@ -69,6 +81,10 @@ pub mod control {
/// Tear down every virtual monitor (host-startup orphan reap). No payload. First-class op — NOT the /// Tear down every virtual monitor (host-startup orphan reap). No payload. First-class op — NOT the
/// SudoVDA "send-and-hope-it's-ignored" hack. /// SudoVDA "send-and-hope-it's-ignored" hack.
pub const IOCTL_CLEAR_ALL: u32 = ctl_code(0x905); pub const IOCTL_CLEAR_ALL: u32 = ctl_code(0x905);
/// Deliver a monitor's IDD-push frame channel: the handle VALUES of the unnamed shared objects the
/// host duplicated into the driver's WUDFHost process. Input [`SetFrameChannelRequest`]. Sent once
/// after the ring is created and again on every mid-session ring recreate (HDR-mode flip).
pub const IOCTL_SET_FRAME_CHANNEL: u32 = ctl_code(0x906);
/// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns /// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns
/// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this /// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this
@@ -103,6 +119,11 @@ pub mod control {
/// `_reserved` (offset 12): an un-upgraded driver leaves it `0`, so the host can tell its /// `_reserved` (offset 12): an un-upgraded driver leaves it `0`, so the host can tell its
/// preference was ignored (stale driver) and log it instead of silently losing per-client config. /// preference was ignored (stale driver) and log it instead of silently losing per-client config.
pub resolved_monitor_id: u32, pub resolved_monitor_id: u32,
/// The driver's own process id (the WUDFHost hosting `pf_vdisplay`) — the target the host
/// duplicates the unnamed frame-object handles INTO (`OpenProcess(PROCESS_DUP_HANDLE)` +
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32,
} }
/// `IOCTL_REMOVE` input. /// `IOCTL_REMOVE` input.
@@ -129,6 +150,43 @@ pub mod control {
pub watchdog_timeout_s: u32, pub watchdog_timeout_s: u32,
} }
/// `IOCTL_SET_FRAME_CHANNEL` input — the sealed frame channel's bootstrap. Every handle field is a
/// handle VALUE already duplicated into the driver's WUDFHost process by the host. Ownership is
/// **adopt-on-success-only** (`design/idd-push-security.md` invariant 5): the driver owns (and
/// eventually closes) the handles IFF it completes the IOCTL successfully — a replaced or
/// later-unconsumed delivery is then the driver's to close. On ANY error completion (malformed
/// request, unknown `target_id`) the driver must NOT close them: the HOST reaps its remote
/// duplicates (`DUPLICATE_CLOSE_SOURCE`). Exactly one side closes each value; a driver that closed
/// on error would double-close possibly-reused handle values against the host's reap.
///
/// Handle values are only meaningful inside the target process's handle table, so this struct is
/// harmless to any third party: reading it leaks nothing openable, and spoofing it (were the control
/// device reachable — it is ACL'd to SYSTEM + admins) could at worst feed the driver values that
/// don't resolve, a DoS of the attacker's own session. The frame objects themselves are unnamed and
/// therefore unreachable by any process that isn't one of the two endpoints.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetFrameChannelRequest {
/// The OS target id from [`AddReply`] — which monitor this channel belongs to.
pub target_id: u32,
/// The ring generation these textures belong to (must match the shared header's generation at
/// attach time; a stale delivery is dropped by the driver — a fresh one follows every recreate).
pub generation: u32,
/// How many leading entries of `texture_handles` are valid (`1..=`[`RING_LEN`]).
pub ring_len: u32,
pub _pad: u32,
/// The shared-header file-mapping handle (the driver maps it and writes status/publish tokens).
pub header_handle: u64,
/// The frame-ready auto-reset event handle (the driver signals it after each publish).
pub event_handle: u64,
/// The ring textures' shared NT handles (opened via `ID3D11Device1::OpenSharedResource1`).
pub texture_handles: [u64; RING_LEN_USIZE],
}
/// [`RING_LEN`] as a usize for the `texture_handles` array length (the wire struct sizes the array
/// at the compile-time maximum; `ring_len` says how many entries are live).
pub const RING_LEN_USIZE: usize = RING_LEN as usize;
// Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already // Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already
// rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!` // rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!`
// asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss. // asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss.
@@ -142,11 +200,20 @@ pub mod control {
assert!(offset_of!(AddRequest, refresh_hz) == 16); assert!(offset_of!(AddRequest, refresh_hz) == 16);
assert!(offset_of!(AddRequest, preferred_monitor_id) == 20); assert!(offset_of!(AddRequest, preferred_monitor_id) == 20);
assert!(size_of::<AddReply>() == 16); assert!(size_of::<AddReply>() == 20);
assert!(offset_of!(AddReply, adapter_luid_low) == 0); assert!(offset_of!(AddReply, adapter_luid_low) == 0);
assert!(offset_of!(AddReply, adapter_luid_high) == 4); assert!(offset_of!(AddReply, adapter_luid_high) == 4);
assert!(offset_of!(AddReply, target_id) == 8); assert!(offset_of!(AddReply, target_id) == 8);
assert!(offset_of!(AddReply, resolved_monitor_id) == 12); assert!(offset_of!(AddReply, resolved_monitor_id) == 12);
assert!(offset_of!(AddReply, wudf_pid) == 16);
assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE);
assert!(offset_of!(SetFrameChannelRequest, target_id) == 0);
assert!(offset_of!(SetFrameChannelRequest, generation) == 4);
assert!(offset_of!(SetFrameChannelRequest, ring_len) == 8);
assert!(offset_of!(SetFrameChannelRequest, header_handle) == 16);
assert!(offset_of!(SetFrameChannelRequest, event_handle) == 24);
assert!(offset_of!(SetFrameChannelRequest, texture_handles) == 32);
assert!(size_of::<RemoveRequest>() == 8); assert!(size_of::<RemoveRequest>() == 8);
assert!(offset_of!(RemoveRequest, session_id) == 0); assert!(offset_of!(RemoveRequest, session_id) == 0);
@@ -161,11 +228,12 @@ pub mod control {
}; };
} }
/// The IDD-push frame transport: the host-created shared ring header, the publish token, the names, and /// The IDD-push frame transport: the host-created shared ring header, the publish token, and the
/// the driver-status codes. The texture ring itself is host-created D3D11 keyed-mutex textures (opened /// driver-status codes. The texture ring itself is host-created **unnamed** D3D11 keyed-mutex textures;
/// by name on the driver side); only the *layout/contract* lives here. /// the driver reaches them (and the header + event) only through handles the host duplicated into its
/// process and delivered via [`crate::control::IOCTL_SET_FRAME_CHANNEL`] — the sealed channel. Only the
/// *layout/contract* lives here.
pub mod frame { pub mod frame {
use alloc::string::String;
use bytemuck::{Pod, Zeroable}; use bytemuck::{Pod, Zeroable};
/// Header magic (`"PFVD"` LE). The host stamps it LAST (after the ring textures exist) so the driver /// Header magic (`"PFVD"` LE). The host stamps it LAST (after the ring textures exist) so the driver
@@ -195,8 +263,10 @@ pub mod frame {
pub struct SharedHeader { pub struct SharedHeader {
pub magic: u32, pub magic: u32,
pub version: u32, pub version: u32,
/// Bumped by the host on a ring recreate (HDR-mode flip → new texture format/names). The driver /// Bumped by the host on a ring recreate (HDR-mode flip → new texture format + a fresh
/// re-attaches when it changes; a publish carries it so the host rejects a stale-ring publish. /// [`control::IOCTL_SET_FRAME_CHANNEL`](crate::control::IOCTL_SET_FRAME_CHANNEL) delivery). The
/// driver re-attaches when it changes; a publish carries it so the host rejects a stale-ring
/// publish.
pub generation: u32, pub generation: u32,
pub ring_len: u32, pub ring_len: u32,
pub width: u32, pub width: u32,
@@ -245,21 +315,6 @@ pub mod frame {
} }
} }
/// `Global\pfvd-hdr-<target>` — the shared metadata header mapping name.
pub fn header_name(target_id: u32) -> String {
alloc::format!("Global\\pfvd-hdr-{target_id}")
}
/// `Global\pfvd-evt-<target>` — the frame-ready auto-reset event name.
pub fn event_name(target_id: u32) -> String {
alloc::format!("Global\\pfvd-evt-{target_id}")
}
/// `Global\pfvd-tex-<target>-<generation>-<slot>` — a ring texture's shared-handle name. The
/// generation in the name means a recreate's new textures never collide with the old ring's
/// not-yet-released handles.
pub fn texture_name(target_id: u32, generation: u32, slot: u32) -> String {
alloc::format!("Global\\pfvd-tex-{target_id}-{generation}-{slot}")
}
// Size + per-field offsets are load-bearing: both sides access these via raw atomic views over the // Size + per-field offsets are load-bearing: both sides access these via raw atomic views over the
// mapping, so a same-size field reorder would silently corrupt. Pin every offset. The `_pad` after // mapping, so a same-size field reorder would silently corrupt. Pin every offset. The `_pad` after
// `dxgi_format` is what 8-aligns the `u64 latest` at offset 32 — assert that too. // `dxgi_format` is what 8-aligns the `u64 latest` at offset 32 — assert that too.
@@ -292,8 +347,10 @@ pub mod frame {
/// (`design/windows-host-rewrite.md` §2.7). Owning them here with `Pod` derives + `offset_of!` /// (`design/windows-host-rewrite.md` §2.7). Owning them here with `Pod` derives + `offset_of!`
/// asserts makes a one-sided edit a compile error. /// asserts makes a one-sided edit a compile error.
/// ///
/// The host creates the section (privileged, permissive DACL so the restricted WUDFHost token can /// Since v2 the channel is **sealed** (`design/gamepad-channel-sealing.md`, mirroring the frame
/// open it) and the driver maps it. Layout only; the section itself is host-created shared memory. /// channel): the host creates the DATA section ([`XusbShm`]/[`PadShm`]) UNNAMED (SYSTEM-only DACL)
/// and duplicates its handle into the driver's WUDFHost; only the tiny [`PadBootstrap`] mailbox
/// stays named (it carries nothing exploitable). Layout only; the sections are host-created.
pub mod gamepad { pub mod gamepad {
use alloc::string::String; use alloc::string::String;
use bytemuck::{Pod, Zeroable}; use bytemuck::{Pod, Zeroable};
@@ -316,15 +373,68 @@ pub mod gamepad {
/// The section starts zeroed, so `0` always means "no driver has attached (yet)"; a pre-health /// The section starts zeroed, so `0` always means "no driver has attached (yet)"; a pre-health
/// driver never writes the field and reads as not-attached, which the host log line calls out /// driver never writes the field and reads as not-attached, which the host log line calls out
/// (the remedy is the same: reinstall the drivers). Bump on a gamepad-layout change. /// (the remedy is the same: reinstall the drivers). Bump on a gamepad-layout change.
pub const GAMEPAD_PROTO_VERSION: u32 = 1; ///
/// v2: the **sealed pad channel** (`design/gamepad-channel-sealing.md`) — the DATA section
/// ([`XusbShm`]/[`PadShm`]) is UNNAMED and reaches the driver only as a handle the host duplicated
/// into its WUDFHost, bootstrapped through the named [`PadBootstrap`] mailbox; the DATA section
/// gained `pad_index` (carved from reserved space) so the driver rejects a cross-pad delivery.
/// A v1 driver opens `Global\pf…-shm-<i>` (which no longer exists) and a v1 host never creates
/// the mailbox a v2 driver polls, so a mixed pairing fails closed either way.
pub const GAMEPAD_PROTO_VERSION: u32 = 2;
/// `Global\pfxusb-shm-<index>` — the virtual Xbox 360 (XInput) shared section. /// Bootstrap-mailbox magic (`"PFBT"` LE) — the host stamps it LAST (after `host_proto`), so a
pub fn xusb_shm_name(index: u8) -> String { /// driver only trusts a fully-initialized mailbox.
alloc::format!("Global\\pfxusb-shm-{index}") pub const BOOT_MAGIC: u32 = 0x5442_4650;
/// `Global\pfxusb-boot-<index>` — the virtual Xbox 360 pad's bootstrap mailbox ([`PadBootstrap`]).
pub fn xusb_boot_name(index: u8) -> String {
alloc::format!("Global\\pfxusb-boot-{index}")
} }
/// `Global\pfds-shm-<index>` — the virtual DualSense / DualShock 4 shared section. /// `Global\pfds-boot-<index>` — the DualSense / DualShock 4 pad's bootstrap mailbox
pub fn pad_shm_name(index: u8) -> String { /// ([`PadBootstrap`]).
alloc::format!("Global\\pfds-shm-{index}") pub fn pad_boot_name(index: u8) -> String {
alloc::format!("Global\\pfds-boot-{index}")
}
/// The per-pad bootstrap mailbox (32 B, named `Global\pf…-boot-<index>`, SY+LS DACL) — the ONLY
/// named object left on the gamepad channel. It exists because the pad drivers are UMDF HID
/// minidrivers with no control device (hidclass owns the stack), so there is no IOCTL to hand the
/// driver a duplicated handle or learn its WUDFHost pid; this mailbox is the late-bound handshake:
///
/// 1. host creates it (zeroed), stamps `host_proto` then `magic` (in that order);
/// 2. driver opens it by name (pad index from `pszDeviceLocation`), writes `driver_proto`, and —
/// iff `host_proto` matches its own version — publishes `driver_pid`;
/// 3. host polls `driver_pid`, verifies the pid is a genuine WUDFHost, duplicates the unnamed DATA
/// section into it, then writes `data_handle` + `handle_pid` and bumps `handle_seq` LAST;
/// 4. driver sees a fresh `handle_seq` addressed to its own pid, maps `data_handle`, and validates
/// the mapped section's magic + `pad_index` before use.
///
/// Deliberately safe to leave named + LS-openable: it carries only pids (not sensitive) and a
/// handle VALUE (meaningless outside the target WUDFHost's handle table). A sibling LocalService
/// that tampers with it can at worst mis-route a delivery — a gamepad DoS, never a read or an
/// injection (it cannot place a valid section handle in the WUDFHost, and the driver's
/// magic+`pad_index` validation rejects any handle that doesn't resolve to this pad's section).
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct PadBootstrap {
/// [`BOOT_MAGIC`], host-stamped last at creation.
pub magic: u32,
/// The host's [`GAMEPAD_PROTO_VERSION`]. A driver whose own version differs must NOT publish
/// its pid (fail closed) — it still writes `driver_proto` so the host can log the mismatch.
pub host_proto: u32,
/// The driver's WUDFHost process id (driver-written; `0` = no driver yet). The duplication
/// target the host verifies (`verify_is_wudfhost`) before duplicating the DATA section into it.
pub driver_pid: u32,
/// The driver's [`GAMEPAD_PROTO_VERSION`] (driver-written; diagnostics only).
pub driver_proto: u32,
/// The DATA-section handle VALUE the host duplicated into `handle_pid`'s handle table
/// (host-written; valid only inside that process).
pub data_handle: u64,
/// The pid `data_handle` was duplicated for — a driver whose pid differs ignores the delivery.
pub handle_pid: u32,
/// Bumped by the host (host-global monotonic, never 0) AFTER `data_handle`/`handle_pid` are in
/// place — the driver's new-delivery trigger.
pub handle_seq: u32,
} }
/// Virtual Xbox 360 (XInput) shared section (64 B). The host writes the XInput state (a bumped /// Virtual Xbox 360 (XInput) shared section (64 B). The host writes the XInput state (a bumped
@@ -356,7 +466,12 @@ pub mod gamepad {
/// Bumped by the driver on every serviced XInput IOCTL — proves the game-visible path (it /// Bumped by the driver on every serviced XInput IOCTL — proves the game-visible path (it
/// only advances while something polls the slot, so a static value is not an error). /// only advances while something polls the slot, so a static value is not an error).
pub driver_heartbeat: u32, pub driver_heartbeat: u32,
pub _reserved1: [u8; 24], /// The pad index this section serves (host-stamped before the magic). The driver validates it
/// against its own `pszDeviceLocation` index when it maps the delivered handle, so a mis-routed
/// (or bootstrap-tampered) cross-pad delivery is rejected instead of silently cross-wiring two
/// pads. Carved from v1 reserved space (v2).
pub pad_index: u32,
pub _reserved1: [u8; 20],
} }
/// Virtual DualSense / DualShock 4 shared section (256 B). The host writes the `0x01`-style HID /// Virtual DualSense / DualShock 4 shared section (256 B). The host writes the `0x01`-style HID
@@ -384,7 +499,10 @@ pub mod gamepad {
/// Bumped by the driver's ~125 Hz timer each tick — a true liveness heartbeat (unlike the /// Bumped by the driver's ~125 Hz timer each tick — a true liveness heartbeat (unlike the
/// XUSB one, this advances whenever the driver is loaded, game or not). /// XUSB one, this advances whenever the driver is loaded, game or not).
pub driver_heartbeat: u32, pub driver_heartbeat: u32,
pub _reserved1: [u8; 104], /// The pad index this section serves (host-stamped before the magic) — see
/// [`XusbShm::pad_index`]. Carved from v1 reserved space (v2).
pub pad_index: u32,
pub _reserved1: [u8; 100],
} }
// Offsets are the wire contract the shipped drivers already read by hand — pin every one. A failing // Offsets are the wire contract the shipped drivers already read by hand — pin every one. A failing
@@ -408,6 +526,7 @@ pub mod gamepad {
assert!(offset_of!(XusbShm, rumble_small) == 29); assert!(offset_of!(XusbShm, rumble_small) == 29);
assert!(offset_of!(XusbShm, driver_proto) == 32); assert!(offset_of!(XusbShm, driver_proto) == 32);
assert!(offset_of!(XusbShm, driver_heartbeat) == 36); assert!(offset_of!(XusbShm, driver_heartbeat) == 36);
assert!(offset_of!(XusbShm, pad_index) == 40);
assert!(size_of::<PadShm>() == 256); assert!(size_of::<PadShm>() == 256);
assert!(offset_of!(PadShm, magic) == 0); assert!(offset_of!(PadShm, magic) == 0);
@@ -417,6 +536,16 @@ pub mod gamepad {
assert!(offset_of!(PadShm, device_type) == 140); assert!(offset_of!(PadShm, device_type) == 140);
assert!(offset_of!(PadShm, driver_proto) == 144); assert!(offset_of!(PadShm, driver_proto) == 144);
assert!(offset_of!(PadShm, driver_heartbeat) == 148); assert!(offset_of!(PadShm, driver_heartbeat) == 148);
assert!(offset_of!(PadShm, pad_index) == 152);
assert!(size_of::<PadBootstrap>() == 32);
assert!(offset_of!(PadBootstrap, magic) == 0);
assert!(offset_of!(PadBootstrap, host_proto) == 4);
assert!(offset_of!(PadBootstrap, driver_pid) == 8);
assert!(offset_of!(PadBootstrap, driver_proto) == 12);
assert!(offset_of!(PadBootstrap, data_handle) == 16);
assert!(offset_of!(PadBootstrap, handle_pid) == 24);
assert!(offset_of!(PadBootstrap, handle_seq) == 28);
}; };
} }
@@ -487,28 +616,71 @@ mod tests {
adapter_luid_high: -2, adapter_luid_high: -2,
target_id: 262, target_id: 262,
resolved_monitor_id: 7, resolved_monitor_id: 7,
wudf_pid: 4242,
}; };
let rbytes = bytemuck::bytes_of(&reply); let rbytes = bytemuck::bytes_of(&reply);
assert_eq!(rbytes.len(), 16); assert_eq!(rbytes.len(), 20);
assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply); assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply);
// resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible. // resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible.
assert_eq!(rbytes[12..16], 7u32.to_le_bytes()); assert_eq!(rbytes[12..16], 7u32.to_le_bytes());
// The v2 duplication-target pid trails at offset 16.
assert_eq!(rbytes[16..20], 4242u32.to_le_bytes());
} }
#[test] #[test]
fn names_are_stable() { fn frame_channel_request_roundtrips_through_bytes() {
assert_eq!(frame::header_name(10), "Global\\pfvd-hdr-10"); let mut req = control::SetFrameChannelRequest {
assert_eq!(frame::event_name(10), "Global\\pfvd-evt-10"); target_id: 262,
assert_eq!(frame::texture_name(10, 3, 5), "Global\\pfvd-tex-10-3-5"); generation: 3,
ring_len: frame::RING_LEN,
_pad: 0,
header_handle: 0x0000_0000_0000_1a2c,
event_handle: 0x0000_0000_0000_1b30,
texture_handles: [0; control::RING_LEN_USIZE],
};
for (k, t) in req.texture_handles.iter_mut().enumerate() {
*t = 0x2000 + k as u64 * 4;
}
let bytes = bytemuck::bytes_of(&req);
assert_eq!(bytes.len(), 32 + 8 * control::RING_LEN_USIZE);
assert_eq!(
*bytemuck::from_bytes::<control::SetFrameChannelRequest>(bytes),
req
);
// The handle values ride at 8-byte alignment from offset 16 (header, event, then the ring).
assert_eq!(bytes[16..24], 0x1a2cu64.to_le_bytes());
assert_eq!(bytes[24..32], 0x1b30u64.to_le_bytes());
assert_eq!(bytes[32..40], 0x2000u64.to_le_bytes());
} }
#[test] #[test]
fn gamepad_names_and_magics_are_stable() { fn gamepad_names_and_magics_are_stable() {
assert_eq!(gamepad::xusb_shm_name(0), "Global\\pfxusb-shm-0"); assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0");
assert_eq!(gamepad::pad_shm_name(2), "Global\\pfds-shm-2"); assert_eq!(gamepad::pad_boot_name(2), "Global\\pfds-boot-2");
// Lock the exact u32 magics the shipped host/drivers use (inject/{gamepad,dualsense}_windows.rs). // Lock the exact u32 magics the shipped host/drivers use (inject/{gamepad,dualsense}_windows.rs).
assert_eq!(gamepad::XUSB_MAGIC, 0x5558_4650); assert_eq!(gamepad::XUSB_MAGIC, 0x5558_4650);
assert_eq!(gamepad::PAD_MAGIC, 0x5046_4453); assert_eq!(gamepad::PAD_MAGIC, 0x5046_4453);
// "PFBT" little-endian.
assert_eq!(gamepad::BOOT_MAGIC.to_le_bytes(), *b"PFBT");
}
#[test]
fn pad_bootstrap_roundtrips_through_bytes() {
let b = gamepad::PadBootstrap {
magic: gamepad::BOOT_MAGIC,
host_proto: gamepad::GAMEPAD_PROTO_VERSION,
driver_pid: 1234,
driver_proto: gamepad::GAMEPAD_PROTO_VERSION,
data_handle: 0x0000_0000_0000_2a4c,
handle_pid: 1234,
handle_seq: 7,
};
let bytes = bytemuck::bytes_of(&b);
assert_eq!(bytes.len(), 32);
assert_eq!(*bytemuck::from_bytes::<gamepad::PadBootstrap>(bytes), b);
// The handle value rides 8-aligned at offset 16; the seq trails at 28 (written LAST by the host).
assert_eq!(bytes[16..24], 0x2a4cu64.to_le_bytes());
assert_eq!(bytes[28..32], 7u32.to_le_bytes());
} }
#[test] #[test]
@@ -521,6 +693,7 @@ mod tests {
control::IOCTL_PING, control::IOCTL_PING,
control::IOCTL_GET_INFO, control::IOCTL_GET_INFO,
control::IOCTL_CLEAR_ALL, control::IOCTL_CLEAR_ALL,
control::IOCTL_SET_FRAME_CHANNEL,
]; ];
for (i, a) in all.iter().enumerate() { for (i, a) in all.iter().enumerate() {
for b in &all[i + 1..] { for b in &all[i + 1..] {
+59
View File
@@ -635,6 +635,22 @@ impl PunktfunkHdrMeta {
} }
} }
/// One access unit's host-side processing time ([`punktfunk_connection_next_host_timing`]):
/// capture → fully sent, i.e. the whole host pipeline (capture read/convert, encode, FEC+seal,
/// paced send). Correlate to the AU whose `PunktfunkFrame::pts_ns` equals `pts_ns`, then
/// `network = (received_instant + clock_offset pts_ns) host_us` — the unified stats HUD's
/// `host` / `network` split (design/stats-unification.md Phase 2). Best-effort: a lost datagram
/// means that frame simply contributes no sample.
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkHostTiming {
/// The AU's capture stamp (host capture clock — matches `PunktfunkFrame::pts_ns` exactly).
pub pts_ns: u64,
/// Host capture→sent duration, µs.
pub host_us: u32,
}
/// `PunktfunkRichInput::kind` — a touchpad contact (`finger`/`active`/`x`/`y` valid). /// `PunktfunkRichInput::kind` — a touchpad contact (`finger`/`active`/`x`/`y` valid).
pub const PUNKTFUNK_RICH_TOUCHPAD: u8 = 1; pub const PUNKTFUNK_RICH_TOUCHPAD: u8 = 1;
/// `PunktfunkRichInput::kind` — a motion sample (`gyro`/`accel` valid). /// `PunktfunkRichInput::kind` — a motion sample (`gyro`/`accel` valid).
@@ -1759,6 +1775,49 @@ pub unsafe extern "C" fn punktfunk_connection_next_hdr_meta(
}) })
} }
/// Pull the next per-AU host timing (0xCF) into `*out`: the host's capture→sent duration for one
/// access unit, correlated to the AU by `pts_ns` (see [`PunktfunkHostTiming`]).
/// [`PunktfunkStatus::NoFrame`] on timeout, [`PunktfunkStatus::Closed`] once the session ended.
/// A stats consumer drains this non-blockingly (`timeout_ms = 0`) alongside its frame samples;
/// an older host never emits any — keep showing the combined `host+network` stage then. Same
/// threading rules as [`punktfunk_connection_next_rumble`] (one puller, may run alongside the
/// other planes).
///
/// # Safety
/// `c` is a valid connection handle; `out` is writable for one `PunktfunkHostTiming`.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_next_host_timing(
c: *mut PunktfunkConnection,
out: *mut PunktfunkHostTiming,
timeout_ms: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if out.is_null() {
return PunktfunkStatus::NullPointer;
}
match c
.inner
.next_host_timing(std::time::Duration::from_millis(timeout_ms as u64))
{
Ok(t) => {
unsafe {
*out = PunktfunkHostTiming {
pts_ns: t.pts_ns,
host_us: t.host_us,
}
};
PunktfunkStatus::Ok
}
Err(e) => e.status(),
}
})
}
/// Read the session's resolved colour signalling + encode bit depth (from the host's Welcome). /// Read the session's resolved colour signalling + encode bit depth (from the host's Welcome).
/// Each out pointer is filled when non-NULL: `primaries`/`transfer`/`matrix` are CICP code points /// Each out pointer is filled when non-NULL: `primaries`/`transfer`/`matrix` are CICP code points
/// (BT.709 = 1; BT.2020 = 9; PQ transfer = 16, HLG = 18; BT.2020-NCL matrix = 9), `full_range` is /// (BT.709 = 1; BT.2020 = 9; PQ transfer = 16, HLG = 18; BT.2020-NCL matrix = 9), `full_range` is
+96 -2
View File
@@ -140,6 +140,11 @@ const HIDOUT_QUEUE: usize = 32;
/// and low-rate (one on start, re-sent on mastering changes / keyframes); a small ring is ample. /// and low-rate (one on start, re-sent on mastering changes / keyframes); a small ring is ample.
const HDR_META_QUEUE: usize = 8; const HDR_META_QUEUE: usize = 8;
/// Host-timing plane depth (0xCF, one datagram per AU). Sized for a 240 fps stream whose stats
/// consumer drains once per second with headroom; overflow drops the newest sample (try_send) —
/// harmless, it's per-frame observability, not state.
const HOST_TIMING_QUEUE: usize = 512;
/// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames). /// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames).
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct AudioPacket { pub struct AudioPacket {
@@ -161,6 +166,9 @@ pub struct NativeClient {
hidout: Mutex<Receiver<HidOutput>>, hidout: Mutex<Receiver<HidOutput>>,
/// Inbound static HDR metadata (ST.2086 mastering + content light level) — 0xCE datagrams. /// Inbound static HDR metadata (ST.2086 mastering + content light level) — 0xCE datagrams.
hdr_meta: Mutex<Receiver<HdrMeta>>, hdr_meta: Mutex<Receiver<HdrMeta>>,
/// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises
/// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any).
host_timing: Mutex<Receiver<crate::quic::HostTiming>>,
input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>, input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>,
/// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker. /// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker.
mic_tx: tokio::sync::mpsc::UnboundedSender<(u32, u64, Vec<u8>)>, mic_tx: tokio::sync::mpsc::UnboundedSender<(u32, u64, Vec<u8>)>,
@@ -176,6 +184,12 @@ pub struct NativeClient {
/// a recovery keyframe under infinite GOP — the correct loss trigger, since unrecoverable loss /// a recovery keyframe under infinite GOP — the correct loss trigger, since unrecoverable loss
/// yields reference-missing frames the decoder silently conceals (a decode-error trigger misses them). /// yields reference-missing frames the decoder silently conceals (a decode-error trigger misses them).
frames_dropped: Arc<AtomicU64>, frames_dropped: Arc<AtomicU64>,
/// Kernel ids of the client's latency-critical native threads: the internal data-plane pump
/// (UDP receive + FEC reassembly) plus any embedder plane threads registered via
/// [`NativeClient::register_hot_thread`]. The Android client feeds these to an ADPF hint session
/// so the CPU governor keeps the whole video pipeline on fast cores. Empty on platforms without
/// `gettid` (see [`current_hot_tid`]).
hot_tids: Arc<Mutex<Vec<i32>>>,
worker: Option<std::thread::JoinHandle<()>>, worker: Option<std::thread::JoinHandle<()>>,
/// The currently active session mode (the Welcome's, then updated by every accepted /// The currently active session mode (the Welcome's, then updated by every accepted
/// [`NativeClient::request_mode`]). /// [`NativeClient::request_mode`]).
@@ -242,6 +256,32 @@ fn pin_thread_user_interactive() {
#[cfg(not(target_vendor = "apple"))] #[cfg(not(target_vendor = "apple"))]
fn pin_thread_user_interactive() {} fn pin_thread_user_interactive() {}
/// The calling thread's kernel id, for hot-thread performance hints (the Android client's ADPF
/// session today; the consumer is platform-specific). Linux/Android expose `gettid`; elsewhere
/// there's nothing to hint with, so registration is a no-op.
#[cfg(any(target_os = "android", target_os = "linux"))]
fn current_hot_tid() -> Option<i32> {
// SAFETY: `gettid` reads the calling thread's kernel id — an always-safe syscall, no args.
Some(unsafe { libc::gettid() })
}
#[cfg(not(any(target_os = "android", target_os = "linux")))]
fn current_hot_tid() -> Option<i32> {
None
}
/// Record the calling thread's id in the shared hot-thread registry (deduped). Best-effort: a
/// platform without `gettid` or a poisoned lock just skips it — a missed performance hint, not an
/// error on the data path.
fn register_hot_tid(reg: &Mutex<Vec<i32>>) {
if let Some(t) = current_hot_tid() {
if let Ok(mut v) = reg.lock() {
if !v.contains(&t) {
v.push(t);
}
}
}
}
impl NativeClient { impl NativeClient {
/// Connect to a `punktfunk/1` host and start the session at (up to) `mode`. Blocks until the /// Connect to a `punktfunk/1` host and start the session at (up to) `mode`. Blocks until the
/// handshake completes or `timeout` elapses. /// handshake completes or `timeout` elapses.
@@ -283,6 +323,8 @@ impl NativeClient {
let (rumble_tx, rumble_rx) = std::sync::mpsc::sync_channel::<(u16, u16, u16)>(RUMBLE_QUEUE); let (rumble_tx, rumble_rx) = std::sync::mpsc::sync_channel::<(u16, u16, u16)>(RUMBLE_QUEUE);
let (hidout_tx, hidout_rx) = std::sync::mpsc::sync_channel::<HidOutput>(HIDOUT_QUEUE); let (hidout_tx, hidout_rx) = std::sync::mpsc::sync_channel::<HidOutput>(HIDOUT_QUEUE);
let (hdr_meta_tx, hdr_meta_rx) = std::sync::mpsc::sync_channel::<HdrMeta>(HDR_META_QUEUE); let (hdr_meta_tx, hdr_meta_rx) = std::sync::mpsc::sync_channel::<HdrMeta>(HDR_META_QUEUE);
let (host_timing_tx, host_timing_rx) =
std::sync::mpsc::sync_channel::<crate::quic::HostTiming>(HOST_TIMING_QUEUE);
let (input_tx, input_rx) = tokio::sync::mpsc::unbounded_channel::<InputEvent>(); let (input_tx, input_rx) = tokio::sync::mpsc::unbounded_channel::<InputEvent>();
let (mic_tx, mic_rx) = tokio::sync::mpsc::unbounded_channel::<(u32, u64, Vec<u8>)>(); let (mic_tx, mic_rx) = tokio::sync::mpsc::unbounded_channel::<(u32, u64, Vec<u8>)>();
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<RichInput>(); let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<RichInput>();
@@ -292,12 +334,14 @@ impl NativeClient {
let mode_slot = Arc::new(std::sync::Mutex::new(mode)); let mode_slot = Arc::new(std::sync::Mutex::new(mode));
let probe = Arc::new(Mutex::new(ProbeState::default())); let probe = Arc::new(Mutex::new(ProbeState::default()));
let frames_dropped = Arc::new(AtomicU64::new(0)); let frames_dropped = Arc::new(AtomicU64::new(0));
let hot_tids = Arc::new(Mutex::new(Vec::new()));
let host = host.to_string(); let host = host.to_string();
let shutdown_w = shutdown.clone(); let shutdown_w = shutdown.clone();
let mode_slot_w = mode_slot.clone(); let mode_slot_w = mode_slot.clone();
let probe_w = probe.clone(); let probe_w = probe.clone();
let frames_dropped_w = frames_dropped.clone(); let frames_dropped_w = frames_dropped.clone();
let hot_tids_w = hot_tids.clone();
let ctrl_tx_pump = ctrl_tx.clone(); // the data-plane pump sends adaptive-FEC LossReports let ctrl_tx_pump = ctrl_tx.clone(); // the data-plane pump sends adaptive-FEC LossReports
let worker = std::thread::Builder::new() let worker = std::thread::Builder::new()
.name("punktfunk-client".into()) .name("punktfunk-client".into())
@@ -336,6 +380,7 @@ impl NativeClient {
rumble_tx, rumble_tx,
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx,
input_rx, input_rx,
mic_rx, mic_rx,
rich_input_rx, rich_input_rx,
@@ -346,6 +391,7 @@ impl NativeClient {
mode_slot: mode_slot_w, mode_slot: mode_slot_w,
probe: probe_w, probe: probe_w,
frames_dropped: frames_dropped_w, frames_dropped: frames_dropped_w,
hot_tids: hot_tids_w,
})); }));
}) })
.map_err(PunktfunkError::Io)?; .map_err(PunktfunkError::Io)?;
@@ -377,6 +423,7 @@ impl NativeClient {
rumble: Mutex::new(rumble_rx), rumble: Mutex::new(rumble_rx),
hidout: Mutex::new(hidout_rx), hidout: Mutex::new(hidout_rx),
hdr_meta: Mutex::new(hdr_meta_rx), hdr_meta: Mutex::new(hdr_meta_rx),
host_timing: Mutex::new(host_timing_rx),
input_tx, input_tx,
mic_tx, mic_tx,
rich_input_tx, rich_input_tx,
@@ -385,6 +432,7 @@ impl NativeClient {
shutdown, shutdown,
worker: Some(worker), worker: Some(worker),
frames_dropped, frames_dropped,
hot_tids,
mode: mode_slot, mode: mode_slot,
host_fingerprint: fingerprint, host_fingerprint: fingerprint,
resolved_compositor, resolved_compositor,
@@ -526,6 +574,25 @@ impl NativeClient {
self.frames_dropped.load(Ordering::Relaxed) self.frames_dropped.load(Ordering::Relaxed)
} }
/// Register the calling thread as latency-critical so a later
/// [`hot_thread_ids`](Self::hot_thread_ids) includes it. An embedder calls this from its own
/// plane threads (e.g. the Android client's decode + audio threads) to fold them into the same
/// performance-hint session as the internal data-plane pump. Idempotent per thread; a no-op on
/// platforms without `gettid`.
pub fn register_hot_thread(&self) {
register_hot_tid(&self.hot_tids);
}
/// Kernel ids of the client's latency-critical threads: the internal data-plane pump (UDP
/// receive + FEC reassembly) plus any registered via
/// [`register_hot_thread`](Self::register_hot_thread). The Android client feeds these to an ADPF
/// hint session so the CPU governor keeps the whole video pipeline on fast cores. Empty where
/// thread ids aren't available (platforms without `gettid`); call after the first frame so the
/// pump has registered.
pub fn hot_thread_ids(&self) -> Vec<i32> {
self.hot_tids.lock().map(|v| v.clone()).unwrap_or_default()
}
/// Start a bandwidth speed test: ask the host to burst filler over the data plane at /// Start a bandwidth speed test: ask the host to burst filler over the data plane at
/// `target_kbps` of goodput for `duration_ms`, *briefly pausing video*. Non-blocking — the /// `target_kbps` of goodput for `duration_ms`, *briefly pausing video*. Non-blocking — the
/// measurement accumulates in the background; poll [`NativeClient::probe_result`] until its /// measurement accumulates in the background; poll [`NativeClient::probe_result`] until its
@@ -660,6 +727,20 @@ impl NativeClient {
} }
} }
/// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access
/// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network`
/// split (`network = (received + clock_offset pts) host_us`); a stats consumer should
/// drain this non-blockingly alongside its frame samples. An older host never sends any —
/// the HUD then keeps the combined `host+network` stage. Same timeout/closed semantics as
/// [`NativeClient::next_hidout`].
pub fn next_host_timing(&self, timeout: Duration) -> Result<crate::quic::HostTiming> {
match self.host_timing.lock().unwrap().recv_timeout(timeout) {
Ok(t) => Ok(t),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Queue one input event for delivery as a QUIC datagram. /// Queue one input event for delivery as a QUIC datagram.
pub fn send_input(&self, ev: &InputEvent) -> Result<()> { pub fn send_input(&self, ev: &InputEvent) -> Result<()> {
self.input_tx.send(*ev).map_err(|_| PunktfunkError::Closed) self.input_tx.send(*ev).map_err(|_| PunktfunkError::Closed)
@@ -713,6 +794,7 @@ struct WorkerArgs {
rumble_tx: SyncSender<(u16, u16, u16)>, rumble_tx: SyncSender<(u16, u16, u16)>,
hidout_tx: SyncSender<HidOutput>, hidout_tx: SyncSender<HidOutput>,
hdr_meta_tx: SyncSender<HdrMeta>, hdr_meta_tx: SyncSender<HdrMeta>,
host_timing_tx: SyncSender<crate::quic::HostTiming>,
input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>, input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
mic_rx: tokio::sync::mpsc::UnboundedReceiver<(u32, u64, Vec<u8>)>, mic_rx: tokio::sync::mpsc::UnboundedReceiver<(u32, u64, Vec<u8>)>,
rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>, rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
@@ -723,6 +805,7 @@ struct WorkerArgs {
mode_slot: Arc<std::sync::Mutex<Mode>>, mode_slot: Arc<std::sync::Mutex<Mode>>,
probe: Arc<Mutex<ProbeState>>, probe: Arc<Mutex<ProbeState>>,
frames_dropped: Arc<AtomicU64>, frames_dropped: Arc<AtomicU64>,
hot_tids: Arc<Mutex<Vec<i32>>>,
} }
/// The worker: QUIC handshake, then the input/datagram/control tasks + the blocking /// The worker: QUIC handshake, then the input/datagram/control tasks + the blocking
@@ -747,6 +830,7 @@ async fn worker_main(args: WorkerArgs) {
rumble_tx, rumble_tx,
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx,
mut input_rx, mut input_rx,
mut mic_rx, mut mic_rx,
mut rich_input_rx, mut rich_input_rx,
@@ -757,6 +841,7 @@ async fn worker_main(args: WorkerArgs) {
mode_slot, mode_slot,
probe, probe,
frames_dropped, frames_dropped,
hot_tids,
} = args; } = args;
let setup = async { let setup = async {
let remote: std::net::SocketAddr = format!("{host}:{port}") let remote: std::net::SocketAddr = format!("{host}:{port}")
@@ -803,8 +888,10 @@ async fn worker_main(args: WorkerArgs) {
launch: launch.clone(), launch: launch.clone(),
// The embedder's decode/present caps (e.g. the Windows client advertises // The embedder's decode/present caps (e.g. the Windows client advertises
// VIDEO_CAP_10BIT | VIDEO_CAP_HDR). The host only upgrades to a 10-bit / HDR encode // VIDEO_CAP_10BIT | VIDEO_CAP_HDR). The host only upgrades to a 10-bit / HDR encode
// when the matching bit is set, so `0` stays an 8-bit BT.709 stream. // when the matching bit is set, so `0` stays an 8-bit BT.709 stream. HOST_TIMING is
video_caps, // OR'd in unconditionally: every NativeClient build demuxes the 0xCF plane, and the
// bit only asks the host for observability datagrams (never changes the encode).
video_caps: video_caps | crate::quic::VIDEO_CAP_HOST_TIMING,
// Requested surround channel count; the host echoes the resolved value in Welcome. // Requested surround channel count; the host echoes the resolved value in Welcome.
audio_channels, audio_channels,
// The codecs this client can decode + its soft preference (0 = auto). The host // The codecs this client can decode + its soft preference (0 = auto). The host
@@ -1042,6 +1129,11 @@ async fn worker_main(args: WorkerArgs) {
let _ = hdr_meta_tx.try_send(m); let _ = hdr_meta_tx.try_send(m);
} }
} }
Some(&crate::quic::HOST_TIMING_MAGIC) => {
if let Some(t) = crate::quic::decode_host_timing_datagram(&d) {
let _ = host_timing_tx.try_send(t);
}
}
_ => {} // unknown tag — a newer host; ignore _ => {} // unknown tag — a newer host; ignore
} }
} }
@@ -1063,8 +1155,10 @@ async fn worker_main(args: WorkerArgs) {
// decoder queue — it isn't video. // decoder queue — it isn't video.
let pump_shutdown = shutdown.clone(); let pump_shutdown = shutdown.clone();
let pump_probe = probe.clone(); let pump_probe = probe.clone();
let pump_hot_tids = hot_tids.clone();
let _ = tokio::task::spawn_blocking(move || { let _ = tokio::task::spawn_blocking(move || {
pin_thread_user_interactive(); // feeds frame_tx → the client's user-interactive video pump pin_thread_user_interactive(); // feeds frame_tx → the client's user-interactive video pump
register_hot_tid(&pump_hot_tids); // this thread does UDP receive + FEC reassembly — hint it
// Adaptive-FEC loss reporting: every ADAPT_REPORT_INTERVAL, report the loss observed over the // Adaptive-FEC loss reporting: every ADAPT_REPORT_INTERVAL, report the loss observed over the
// window (shards FEC recovered, plus a bump if any frame went unrecoverable) so the host can // window (shards FEC recovered, plus a bump if any frame went unrecoverable) so the host can
// size FEC to the link. Suppressed during a speed test (its FLAG_PROBE filler would skew it). // size FEC to the link. Suppressed during a speed test (its FLAG_PROBE filler would skew it).
+71 -1
View File
@@ -114,6 +114,13 @@ pub const VIDEO_CAP_HDR: u8 = 0x02;
/// [`Welcome::chroma_format`] reflects the real resolved value. Independent of 10-bit/HDR (4:4:4 is a /// [`Welcome::chroma_format`] reflects the real resolved value. Independent of 10-bit/HDR (4:4:4 is a
/// chroma decision, bit depth is a depth decision; the two may combine where the hardware allows). /// chroma decision, bit depth is a depth decision; the two may combine where the hardware allows).
pub const VIDEO_CAP_444: u8 = 0x04; pub const VIDEO_CAP_444: u8 = 0x04;
/// [`Hello::video_caps`] bit: the client consumes per-AU host-timing datagrams
/// ([`HOST_TIMING_MAGIC`], 0xCF) — the host's capture→send duration per frame, letting the client
/// split its `host+network` latency stage into `host` and `network`
/// (design/stats-unification.md Phase 2). The host emits 0xCF ONLY when this bit is set (an older
/// host ignores it and simply never sends any); a client that doesn't set it keeps the combined
/// stage. Purely observability — never changes what the host encodes.
pub const VIDEO_CAP_HOST_TIMING: u8 = 0x08;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** /// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST /// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -390,7 +397,7 @@ pub struct ProbeResult {
/// `client → host`, right after [`Start`]: one round of the wall-clock skew handshake. The client /// `client → host`, right after [`Start`]: one round of the wall-clock skew handshake. The client
/// stamps `t1_ns` (its monotonic-since-epoch clock) and sends; the host echoes it in [`ClockEcho`] /// stamps `t1_ns` (its monotonic-since-epoch clock) and sends; the host echoes it in [`ClockEcho`]
/// with its own receive/send stamps. A few rounds let the client estimate the host↔client clock /// with its own receive/send stamps. A few rounds let the client estimate the host↔client clock
/// offset, so the per-frame `capture→reassembled` latency (the AU `pts_ns` is the host's capture /// offset, so the per-frame `capture→received` latency (the AU `pts_ns` is the host's capture
/// clock) is meaningful across machines, not just same-host. An old host ignores it (the client /// clock) is meaningful across machines, not just same-host. An old host ignores it (the client
/// times out and assumes a shared clock). /// times out and assumes a shared clock).
#[derive(Clone, Copy, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Debug, PartialEq, Eq)]
@@ -1601,6 +1608,50 @@ pub fn decode_hdr_meta_datagram(b: &[u8]) -> Option<HdrMeta> {
}) })
} }
/// Per-AU host-timing datagram tag, host → client (see [`HostTiming`]). Next tag after
/// [`HDR_META_MAGIC`]. Emitted once per access unit, right after its last packet left the host's
/// socket, and only when the client advertised [`VIDEO_CAP_HOST_TIMING`].
pub const HOST_TIMING_MAGIC: u8 = 0xCF;
/// One access unit's host-side processing time: capture → fully sent (the whole host pipeline —
/// capture read/convert, encode, FEC+seal, paced send). The client correlates it to the AU by
/// `pts_ns` (the AU's capture stamp, unique per frame) and derives
/// `network = (received + clock_offset pts_ns) host_us`, so the unified-stats equation's
/// `host+network` stage splits into two per-frame-tiling terms. Best-effort like every side-plane
/// datagram: a lost 0xCF just means that frame contributes no host/network sample.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct HostTiming {
/// The AU's capture stamp (host capture clock — matches the AU's `pts_ns` exactly).
pub pts_ns: u64,
/// Host capture→sent duration, µs (saturated at `u32::MAX` ≈ 71 min — far past the 10 s
/// client-side sanity clamp anyway).
pub host_us: u32,
}
/// Wire length of a [`HOST_TIMING_MAGIC`] datagram: tag + u64 pts + u32 µs = 13 bytes.
const HOST_TIMING_LEN: usize = 1 + 8 + 4;
/// Encode a [`HostTiming`] into a [`HOST_TIMING_MAGIC`] datagram.
pub fn encode_host_timing_datagram(t: &HostTiming) -> Vec<u8> {
let mut b = Vec::with_capacity(HOST_TIMING_LEN);
b.push(HOST_TIMING_MAGIC);
b.extend_from_slice(&t.pts_ns.to_le_bytes());
b.extend_from_slice(&t.host_us.to_le_bytes());
b
}
/// Parse a [`HOST_TIMING_MAGIC`] datagram → [`HostTiming`]. `None` on bad tag or a short buffer
/// (the fixed length bounds every read before it happens).
pub fn decode_host_timing_datagram(b: &[u8]) -> Option<HostTiming> {
if b.len() < HOST_TIMING_LEN || b[0] != HOST_TIMING_MAGIC {
return None;
}
Some(HostTiming {
pts_ns: u64::from_le_bytes(b[1..9].try_into().unwrap()),
host_us: u32::from_le_bytes(b[9..13].try_into().unwrap()),
})
}
/// Async framed-message IO over a quinn stream (`u16 LE length || payload`). /// Async framed-message IO over a quinn stream (`u16 LE length || payload`).
pub mod io { pub mod io {
/// Read one framed message (bounded at 64 KiB — control messages are tiny). /// Read one framed message (bounded at 64 KiB — control messages are tiny).
@@ -2189,6 +2240,25 @@ mod tests {
assert_eq!(decode_hdr_meta_datagram(&bad), None); assert_eq!(decode_hdr_meta_datagram(&bad), None);
} }
#[test]
fn host_timing_datagram_roundtrip_and_truncation() {
let t = HostTiming {
pts_ns: 1_751_500_000_123_456_789, // a realistic 2026 CLOCK_REALTIME capture stamp
host_us: 4_321,
};
let d = encode_host_timing_datagram(&t);
assert_eq!(d[0], HOST_TIMING_MAGIC);
assert_eq!(d.len(), 13);
assert_eq!(decode_host_timing_datagram(&d), Some(t));
// Truncated buffers and a wrong tag are rejected (never partially read).
for n in 0..d.len() {
assert_eq!(decode_host_timing_datagram(&d[..n]), None);
}
let mut bad = d.clone();
bad[0] = HDR_META_MAGIC;
assert_eq!(decode_host_timing_datagram(&bad), None);
}
#[test] #[test]
fn hello_start_roundtrip() { fn hello_start_roundtrip() {
let h = Hello { let h = Hello {
+10 -4
View File
@@ -232,13 +232,19 @@ pf-driver-proto = { path = "../pf-driver-proto" }
bytemuck = { version = "1.19", features = ["derive"] } bytemuck = { version = "1.19", features = ["derive"] }
[features] [features]
# NVENC hardware encode (Windows). OFF by default: it pulls the NVENC SDK, and the host then needs # NVENC hardware encode (Windows). OFF by default (it pulls the NVENC SDK crate); nothing is
# the NVENC entry points (NvEncodeAPICreateInstance / NvEncodeAPIGetMaxSupportedVersion) at link # needed at link time — the entry points are resolved at RUNTIME from the driver's
# time — i.e. `nvencodeapi.lib` from the NVIDIA Video Codec SDK (or an import lib generated from # nvEncodeAPI64.dll (encode/windows/nvenc.rs `load_api`), so the same binary starts fine on
# nvEncodeAPI64.dll) on the linker path. Build the GPU host with `--features nvenc`. # AMD/Intel-only boxes and falls through to AMF/QSV/software. Build the GPU host with
# `--features nvenc`.
nvenc = ["dep:nvidia-video-codec-sdk"] nvenc = ["dep:nvidia-video-codec-sdk"]
# AMD/Intel hardware encode on Windows (AMF/QSV via ffmpeg-next). OFF by default: it needs a # AMD/Intel hardware encode on Windows (AMF/QSV via ffmpeg-next). OFF by default: it needs a
# `FFMPEG_DIR` (BtbN lgpl-shared — includes `*_amf`/`*_qsv`; the GPL-only x264/x265 are never used, # `FFMPEG_DIR` (BtbN lgpl-shared — includes `*_amf`/`*_qsv`; the GPL-only x264/x265 are never used,
# so the LGPL build suffices and keeps the bundled DLLs LGPL, not GPL) at build time and bundles the # so the LGPL build suffices and keeps the bundled DLLs LGPL, not GPL) at build time and bundles the
# FFmpeg DLLs at runtime. Build the all-vendor GPU host with `--features nvenc,amf-qsv`. # FFmpeg DLLs at runtime. Build the all-vendor GPU host with `--features nvenc,amf-qsv`.
amf-qsv = ["dep:ffmpeg-next"] amf-qsv = ["dep:ffmpeg-next"]
# Build-time icon/version-info embedding (build.rs; Windows dev/CI hosts only — Linux packaging
# builds of this crate never execute the winresource block).
[target.'cfg(windows)'.build-dependencies]
winresource = "0.1"
+21 -13
View File
@@ -1,10 +1,9 @@
//! Build script. The only thing it does: with the `nvenc` feature (Windows GPU host), tell the //! Build script: stamps the build version. NVENC deliberately needs NOTHING here — the entry
//! linker to pull the NVENC import library. The NVENC entry points //! points (`NvEncodeAPICreateInstance` / `NvEncodeAPIGetMaxSupportedVersion`) live in
//! (`NvEncodeAPICreateInstance` / `NvEncodeAPIGetMaxSupportedVersion`) live in `nvEncodeAPI64.dll` //! `nvEncodeAPI64.dll`, which only exists where the NVIDIA driver is installed, so
//! (shipped with the NVIDIA driver), so the host links against `nvencodeapi.lib`. Point //! `encode/windows/nvenc.rs` resolves them at RUNTIME (`LoadLibraryExW`). The former link-time
//! `PUNKTFUNK_NVENC_LIB_DIR` at a directory containing `nvencodeapi.lib` — from the NVIDIA Video //! import (`cargo:rustc-link-lib=nvencodeapi`) made the Windows loader kill the all-vendor host
//! Codec SDK, or an import lib generated from the driver's `nvEncodeAPI64.dll` //! binary on every AMD/Intel-only box before `main` ("nvencodeapi64.dll was not found").
//! (`lib /def:nvenc.def /machine:x64 /out:nvencodeapi.lib` with the two exports above).
fn main() { fn main() {
// Build provenance: stamp the exact package/build version into the binary so a running host // Build provenance: stamp the exact package/build version into the binary so a running host
// can report what it is (mgmt /health, the startup log, `--version`) and a stale/shadowed // can report what it is (mgmt /health, the startup log, `--version`) and a stale/shadowed
@@ -19,11 +18,20 @@ fn main() {
println!("cargo:rustc-env=PUNKTFUNK_VERSION={version}"); println!("cargo:rustc-env=PUNKTFUNK_VERSION={version}");
println!("cargo:rerun-if-env-changed=PUNKTFUNK_BUILD_VERSION"); println!("cargo:rerun-if-env-changed=PUNKTFUNK_BUILD_VERSION");
if std::env::var_os("CARGO_FEATURE_NVENC").is_some() { // Windows identity resources: the branded icon + version info. Task Manager / Explorer show a
if let Some(dir) = std::env::var_os("PUNKTFUNK_NVENC_LIB_DIR") { // process by its version-info FileDescription — without one the host appears as a bare
println!("cargo:rustc-link-search=native={}", dir.to_string_lossy()); // "punktfunk-host.exe" with no icon. Same winresource pattern as clients/windows and
} // punktfunk-tray (cfg(windows) = build HOST, so Linux packaging builds skip it; CARGO_CFG_WINDOWS
println!("cargo:rustc-link-lib=dylib=nvencodeapi"); // = TARGET).
println!("cargo:rerun-if-env-changed=PUNKTFUNK_NVENC_LIB_DIR"); #[cfg(windows)]
if std::env::var_os("CARGO_CFG_WINDOWS").is_some() {
let icon = "../../packaging/windows/branding/punktfunk.ico";
println!("cargo:rerun-if-changed={icon}");
winresource::WindowsResource::new()
.set_icon_with_id(icon, "1")
.set("FileDescription", "Punktfunk Host")
.set("ProductName", "Punktfunk")
.compile()
.expect("embed windows icon/version resources");
} }
} }
+450 -5
View File
@@ -42,7 +42,8 @@ pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> {
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> { pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> {
audio_control::ensure_wired_once(); // The capture thread runs the audio wiring plan itself (audio_control::wire_now) before
// resolving its endpoint — a fresh plan per open, because Windows endpoints churn.
wasapi_cap::WasapiLoopbackCapturer::open(channels) wasapi_cap::WasapiLoopbackCapturer::open(channels)
.map(|c| Box::new(c) as Box<dyn AudioCapturer>) .map(|c| Box::new(c) as Box<dyn AudioCapturer>)
} }
@@ -57,10 +58,27 @@ pub fn open_audio_capture(_channels: u32) -> Result<Box<dyn AudioCapturer>> {
/// decoded client-mic PCM (interleaved `f32` at [`SAMPLE_RATE`]) into it, and PipeWire delivers /// decoded client-mic PCM (interleaved `f32` at [`SAMPLE_RATE`]) into it, and PipeWire delivers
/// it to whichever app records the source — silence when no input is flowing. This is how the /// it to whichever app records the source — silence when no input is flowing. This is how the
/// client's microphone reaches host applications (mic passthrough). /// client's microphone reaches host applications (mic passthrough).
///
/// **Liveness contract.** Both backends run a worker thread that CAN die under the host's feet
/// (Linux: the PipeWire daemon restarts with the session; Windows: the audio endpoint is
/// invalidated/removed). A dead backend must be observable — [`push`](Self::push) returns `false`
/// and [`alive`](Self::alive) turns false — so the owning [`MicPump`] drops the instance and
/// reopens. Before this contract existed, a single backend death left `push` feeding a dead
/// queue for the rest of the host's life: the historical "mic passthrough works on no host" bug.
pub trait VirtualMic: Send { pub trait VirtualMic: Send {
/// Push one chunk of interleaved `f32` PCM. Non-blocking — drops if PipeWire is behind /// Push one chunk of interleaved `f32` PCM. Non-blocking — drops if the backend is behind
/// (mic audio is lossy/real-time; a stale chunk is worse than a dropped one). /// (mic audio is lossy/real-time; a stale chunk is worse than a dropped one). Returns
fn push(&self, pcm: &[f32]); /// `false` iff the backend is DEAD (worker thread gone) — the caller must reopen; a merely
/// congested backend drops the chunk and returns `true`.
fn push(&self, pcm: &[f32]) -> bool;
/// Backend liveness without pushing data — lets an idle pump notice a death between
/// sessions, so the mic is already healthy again when the next client connects.
fn alive(&self) -> bool;
/// Drop any buffered-but-unplayed audio. Called after an uplink gap (client muted,
/// session ended) so a recorder never hears a stale burst when audio resumes.
fn discard(&self);
/// The interleaved channel count the source was opened with. /// The interleaved channel count the source was opened with.
fn channels(&self) -> u32 { fn channels(&self) -> u32 {
@@ -78,7 +96,8 @@ pub fn open_virtual_mic(channels: u32) -> Result<Box<dyn VirtualMic>> {
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn open_virtual_mic(channels: u32) -> Result<Box<dyn VirtualMic>> { pub fn open_virtual_mic(channels: u32) -> Result<Box<dyn VirtualMic>> {
audio_control::ensure_wired_once(); // The render thread runs the wiring plan itself (audio_control::wire_now) to resolve — and,
// via the plan's default-device changes, to RESERVE — its target endpoint.
wasapi_mic::WasapiVirtualMic::open(channels).map(|m| Box::new(m) as Box<dyn VirtualMic>) wasapi_mic::WasapiVirtualMic::open(channels).map(|m| Box::new(m) as Box<dyn VirtualMic>)
} }
@@ -87,6 +106,220 @@ pub fn open_virtual_mic(_channels: u32) -> Result<Box<dyn VirtualMic>> {
anyhow::bail!("virtual mic requires Linux + PipeWire or Windows + a virtual audio device") anyhow::bail!("virtual mic requires Linux + PipeWire or Windows + a virtual audio device")
} }
/// Mic is 48 kHz stereo — matches the Opus stereo decoder and the host→client audio layout.
pub const MIC_CHANNELS: u32 = 2;
/// Bound for the shared mic frame queue (drop-newest when full): the host-lifetime queue is
/// shared across all concurrent sessions and must not grow without limit under a near-line-rate
/// flood (security-review 2026-06-28 S6). 64 × 520 ms frames ≈ 0.31.3 s of slack.
const MIC_QUEUE_CAP: usize = 64;
/// Tuning for [`MicPump`]'s open/reopen/flush behaviour — parameterized so the tests can run the
/// real pump loop in milliseconds instead of seconds.
#[derive(Clone, Copy)]
struct PumpTuning {
/// First-retry delay after a failed backend open; doubles per failure up to `backoff_cap`
/// (a persistently-absent PipeWire session / audio endpoint isn't hammered), resets on
/// success.
backoff_start: std::time::Duration,
backoff_cap: std::time::Duration,
/// Idle liveness-probe interval: with no frames flowing, the pump still notices a dead
/// backend this often and reopens — so the mic is healthy BEFORE the next session starts.
heartbeat: std::time::Duration,
/// An uplink gap longer than this discards the backend's buffered audio before pushing the
/// next frame (a recorder must never hear a stale burst from before a mute/session end).
stale_gap: std::time::Duration,
/// A backend that dies before living this long counts as a FAILED open for backoff purposes
/// (an open that succeeds but dies instantly — e.g. a flapping daemon — must not churn at
/// heartbeat rate); one that lived longer resets the backoff.
stable_after: std::time::Duration,
}
const PUMP_TUNING: PumpTuning = PumpTuning {
backoff_start: std::time::Duration::from_secs(2),
backoff_cap: std::time::Duration::from_secs(60),
heartbeat: std::time::Duration::from_secs(1),
stale_gap: std::time::Duration::from_millis(600),
stable_after: std::time::Duration::from_secs(5),
};
/// Host-lifetime virtual-microphone pump: one thread owns the [`VirtualMic`] backend + an Opus
/// decoder; sessions forward the client's Opus mic frames (0xCB) over a clonable `Send` sender,
/// the thread decodes and feeds the backend.
///
/// The rock-solid properties live HERE, not in the backends:
/// - **Eager**: the backend opens at host start (retrying with backoff), NOT on the first mic
/// frame — so the virtual mic device already exists when host apps/games launch and bind
/// their capture device (most games never re-follow a default-device change mid-run).
/// - **Self-healing**: a dead backend (PipeWire restart, Windows endpoint churn) is detected on
/// every push and on an idle heartbeat, and reopened with backoff. Sessions keep their
/// senders; nothing upstream notices.
/// - **Stale-flush**: buffered audio is discarded after an uplink gap (see [`PumpTuning`]).
///
/// Per-frame Opus DECODE errors stay non-fatal (dropped frame): the mic is shared across every
/// concurrent session, so one paired client's junk frames must not deny everyone's mic
/// (security-review 2026-06-28 S2). The thread exits when every sender is dropped (host
/// shutdown), tearing the backend down.
pub struct MicPump {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
}
impl MicPump {
/// Start the host-lifetime pump (Linux/Windows). On platforms without a virtual-mic backend
/// the thread just drains and drops frames (sessions still count the datagrams).
pub fn start() -> MicPump {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let spawned = std::thread::Builder::new()
.name("punktfunk-mic-pump".into())
.spawn(move || {
#[cfg(any(target_os = "linux", target_os = "windows"))]
pump_thread(rx, || open_virtual_mic(MIC_CHANNELS), PUMP_TUNING);
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
{
tracing::warn!("mic passthrough unsupported on this platform — frames dropped");
for _ in rx {}
}
});
if let Err(e) = spawned {
tracing::error!(error = %e, "mic pump thread spawn failed — mic passthrough disabled");
}
MicPump { tx }
}
/// A sender a session forwards the client's Opus mic frames to (`try_send` — never block a
/// datagram loop). Cloned per session; dropping a clone does NOT stop the pump (it holds
/// the original sender for the host life).
pub fn sender(&self) -> std::sync::mpsc::SyncSender<Vec<u8>> {
self.tx.clone()
}
}
/// Sleep for `dur` while draining (and dropping) queued frames, so a closed/reopening backend
/// never accumulates a stale backlog and senders never see a wedged queue. Returns `false` when
/// every sender is gone (host shutdown).
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn drain_sleep(rx: &std::sync::mpsc::Receiver<Vec<u8>>, dur: std::time::Duration) -> bool {
use std::sync::mpsc::RecvTimeoutError;
let deadline = std::time::Instant::now() + dur;
loop {
let left = deadline.saturating_duration_since(std::time::Instant::now());
if left.is_zero() {
return true;
}
match rx.recv_timeout(left.min(std::time::Duration::from_millis(250))) {
Ok(_) => {} // drop frames while closed
Err(RecvTimeoutError::Timeout) => {} // keep waiting
Err(RecvTimeoutError::Disconnected) => return false, // host shutdown
}
}
}
/// The pump loop. `opener` is injected so the tests can run the REAL loop against a mock
/// backend; production passes [`open_virtual_mic`].
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn pump_thread<O>(rx: std::sync::mpsc::Receiver<Vec<u8>>, opener: O, tuning: PumpTuning)
where
O: Fn() -> Result<Box<dyn VirtualMic>>,
{
use std::sync::mpsc::RecvTimeoutError;
use std::time::Instant;
let mut backoff = tuning.backoff_start;
let mut open_fails: u64 = 0;
loop {
// Open phase — eager, from thread start.
let (mic, mut decoder) = loop {
let opened = opener().and_then(|m| {
let d = opus::Decoder::new(SAMPLE_RATE, opus::Channels::Stereo)
.map_err(|e| anyhow::anyhow!("opus decoder: {e}"))?;
Ok((m, d))
});
match opened {
Ok(pair) => break pair,
Err(e) => {
// Throttle (1st, 2nd, 4th, 8th … failure): a box without a PipeWire session
// or virtual audio device would otherwise log every backoff forever.
open_fails += 1;
if open_fails.is_power_of_two() {
tracing::warn!(error = %format!("{e:#}"), attempts = open_fails,
"virtual mic unavailable — retrying with backoff");
}
if !drain_sleep(&rx, backoff) {
return;
}
backoff = (backoff * 2).min(tuning.backoff_cap);
}
}
};
tracing::info!("virtual mic ready (host-lifetime)");
// Drop anything queued while (re)opening — it predates the backend. (The backoff does
// NOT reset here: only an instance that proves stable resets it — see the death triage.)
while rx.try_recv().is_ok() {}
let opened_at = Instant::now();
// Pump phase — runs until the backend dies (break) or the host shuts down (return).
let mut decode_fails: u64 = 0;
let mut pcm = vec![0f32; 5760 * MIC_CHANNELS as usize]; // up to 120 ms scratch
let mut last_push = Instant::now();
loop {
match rx.recv_timeout(tuning.heartbeat) {
Ok(frame) => {
if frame.is_empty() {
continue; // DTX silence — the source underruns to silence on its own
}
if last_push.elapsed() > tuning.stale_gap {
mic.discard();
}
match decoder.decode_float(&frame, &mut pcm, false) {
Ok(samples_per_ch) => {
let total = (samples_per_ch * MIC_CHANNELS as usize).min(pcm.len());
if !mic.push(&pcm[..total]) {
tracing::warn!("virtual mic backend died — reopening");
break;
}
last_push = Instant::now();
decode_fails = 0;
}
Err(e) => {
// Malformed/garbage frame: drop it, keep the shared mic + decoder
// (see the struct docs). Throttled log (1, 2, 4, … fails).
decode_fails += 1;
if decode_fails.is_power_of_two() {
tracing::warn!(error = %e, fails = decode_fails,
"mic opus decode failed — dropping frame");
}
}
}
}
Err(RecvTimeoutError::Timeout) => {
if !mic.alive() {
tracing::warn!("virtual mic backend died while idle — reopening");
break;
}
}
Err(RecvTimeoutError::Disconnected) => {
tracing::debug!("mic pump stopped (host shutting down)");
return;
}
}
}
// Death triage: an instance that lived is a one-off (PipeWire/audio-engine restart) —
// reopen immediately with the backoff reset. One that died right after opening is a
// failed open in disguise (flapping daemon, endpoint racing away): back off like the
// open loop, or the pump would churn open→die→reopen at heartbeat rate.
if opened_at.elapsed() >= tuning.stable_after {
backoff = tuning.backoff_start;
open_fails = 0;
} else {
open_fails += 1;
if !drain_sleep(&rx, backoff) {
return;
}
backoff = (backoff * 2).min(tuning.backoff_cap);
}
}
}
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
#[path = "audio/windows/audio_control.rs"] #[path = "audio/windows/audio_control.rs"]
mod audio_control; mod audio_control;
@@ -98,3 +331,215 @@ mod wasapi_cap;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
#[path = "audio/windows/wasapi_mic.rs"] #[path = "audio/windows/wasapi_mic.rs"]
mod wasapi_mic; mod wasapi_mic;
#[cfg_attr(not(target_os = "windows"), allow(dead_code))]
#[path = "audio/wiring_plan.rs"]
pub(crate) mod wiring_plan;
#[cfg(test)]
mod pump_tests {
use super::*;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
/// Mock backend: records pushes/discards, dies on command.
struct MockMic {
alive: Arc<AtomicBool>,
pushed: Arc<AtomicUsize>,
discards: Arc<AtomicUsize>,
}
impl VirtualMic for MockMic {
fn push(&self, pcm: &[f32]) -> bool {
if !self.alive.load(Ordering::Acquire) {
return false;
}
self.pushed.fetch_add(pcm.len(), Ordering::Relaxed);
true
}
fn alive(&self) -> bool {
self.alive.load(Ordering::Acquire)
}
fn discard(&self) {
self.discards.fetch_add(1, Ordering::Relaxed);
}
}
struct Harness {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
opens: Arc<AtomicUsize>,
alive: Arc<Mutex<Option<Arc<AtomicBool>>>>, // latest instance's kill switch
pushed: Arc<AtomicUsize>,
discards: Arc<AtomicUsize>,
join: std::thread::JoinHandle<()>,
}
/// Run the REAL pump loop against mock backends; `fail_first` opens fail before the first
/// success (exercises the eager retry/backoff path). `dead_on_arrival` opens every instance
/// pre-killed (exercises the rapid-death churn guard). `stable_after` mirrors the tuning
/// field (ZERO = every death counts as stable → immediate reopen, keeping tests fast).
fn start_tuned(fail_first: usize, dead_on_arrival: bool, stable_after: Duration) -> Harness {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let opens = Arc::new(AtomicUsize::new(0));
let alive = Arc::new(Mutex::new(None::<Arc<AtomicBool>>));
let pushed = Arc::new(AtomicUsize::new(0));
let discards = Arc::new(AtomicUsize::new(0));
let (opens2, alive2, pushed2, discards2) = (
opens.clone(),
alive.clone(),
pushed.clone(),
discards.clone(),
);
let tuning = PumpTuning {
backoff_start: Duration::from_millis(10),
backoff_cap: Duration::from_millis(40),
heartbeat: Duration::from_millis(20),
stale_gap: Duration::from_millis(80),
stable_after,
};
let join = std::thread::spawn(move || {
pump_thread(
rx,
move || {
let n = opens2.fetch_add(1, Ordering::SeqCst);
if n < fail_first {
anyhow::bail!("backend not up yet (simulated)");
}
let a = Arc::new(AtomicBool::new(!dead_on_arrival));
*alive2.lock().unwrap() = Some(a.clone());
Ok(Box::new(MockMic {
alive: a,
pushed: pushed2.clone(),
discards: discards2.clone(),
}) as Box<dyn VirtualMic>)
},
tuning,
)
});
Harness {
tx,
opens,
alive,
pushed,
discards,
join,
}
}
fn start(fail_first: usize) -> Harness {
start_tuned(fail_first, false, Duration::ZERO)
}
fn wait_until(what: &str, mut cond: impl FnMut() -> bool) {
for _ in 0..200 {
if cond() {
return;
}
std::thread::sleep(Duration::from_millis(10));
}
panic!("timed out waiting for: {what}");
}
fn opus_frame() -> Vec<u8> {
let mut enc = opus::Encoder::new(48_000, opus::Channels::Stereo, opus::Application::Voip)
.expect("opus encoder");
let pcm = [0.1f32; 960 * 2]; // 20 ms stereo
let mut out = vec![0u8; 4000];
let n = enc.encode_float(&pcm, &mut out).expect("encode");
out.truncate(n);
out
}
/// Eager: the backend opens (after transient failures) with NO frame ever sent.
#[test]
fn opens_eagerly_with_backoff() {
let h = start(3);
wait_until("eager open after 3 failures", || {
h.opens.load(Ordering::SeqCst) >= 4 && h.alive.lock().unwrap().is_some()
});
drop(h.tx);
h.join.join().unwrap();
}
/// Frames flow: opus in → PCM pushed to the backend.
#[test]
fn decodes_and_pushes() {
let h = start(0);
wait_until("open", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
wait_until("pcm pushed", || h.pushed.load(Ordering::SeqCst) > 0);
drop(h.tx);
h.join.join().unwrap();
}
/// A dead backend is noticed WHILE IDLE (heartbeat) and reopened without any traffic.
#[test]
fn reopens_after_idle_death() {
let h = start(0);
wait_until("first open", || h.opens.load(Ordering::SeqCst) >= 1);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.alive
.lock()
.unwrap()
.as_ref()
.unwrap()
.store(false, Ordering::Release); // kill it
wait_until("reopen after idle death", || {
h.opens.load(Ordering::SeqCst) >= 2
});
drop(h.tx);
h.join.join().unwrap();
}
/// A death detected on push (frame flowing) also reopens, and the frame after reopen flows.
#[test]
fn reopens_after_push_death() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.alive
.lock()
.unwrap()
.as_ref()
.unwrap()
.store(false, Ordering::Release);
h.tx.send(opus_frame()).unwrap(); // push sees death → reopen
wait_until("reopen", || h.opens.load(Ordering::SeqCst) >= 2);
h.tx.send(opus_frame()).unwrap();
wait_until("pcm after reopen", || h.pushed.load(Ordering::SeqCst) > 0);
drop(h.tx);
h.join.join().unwrap();
}
/// Instances that die immediately after opening must be retried with BACKOFF, not at
/// heartbeat rate — a flapping backend (daemon up but dropping us instantly) would
/// otherwise churn open→die→reopen every heartbeat forever.
#[test]
fn rapid_death_backs_off() {
// Every instance is dead on arrival; stability threshold high so each death counts
// as a failed open. Without the guard: ~1 reopen per heartbeat (20 ms) ≈ 25 opens in
// 500 ms. With backoff 10→20→40 (cap): ≈ 7.
let h = start_tuned(0, true, Duration::from_secs(10));
std::thread::sleep(Duration::from_millis(500));
let opens = h.opens.load(Ordering::SeqCst);
assert!(opens >= 2, "must keep retrying (got {opens})");
assert!(
opens <= 15,
"must back off, not churn per heartbeat (got {opens})"
);
drop(h.tx);
h.join.join().unwrap();
}
/// An uplink gap discards buffered-stale audio before the next frame plays.
#[test]
fn discards_after_gap() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
wait_until("first push", || h.pushed.load(Ordering::SeqCst) > 0);
std::thread::sleep(Duration::from_millis(150)); // > stale_gap
h.tx.send(opus_frame()).unwrap();
wait_until("discard on gap", || h.discards.load(Ordering::SeqCst) >= 1);
drop(h.tx);
h.join.join().unwrap();
}
}
+134 -12
View File
@@ -16,7 +16,9 @@
use super::{AudioCapturer, VirtualMic, SAMPLE_RATE}; use super::{AudioCapturer, VirtualMic, SAMPLE_RATE};
use anyhow::{anyhow, Context, Result}; use anyhow::{anyhow, Context, Result};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError}; use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError};
use std::sync::Arc;
use std::thread; use std::thread;
use std::time::Duration; use std::time::Duration;
@@ -111,10 +113,28 @@ fn spa_positions(channels: u32) -> [u32; 64] {
/// Virtual microphone: a PipeWire `Audio/Source` node host apps can record from. The host pushes /// Virtual microphone: a PipeWire `Audio/Source` node host apps can record from. The host pushes
/// decoded client-mic PCM in; the loop thread's producer callback drains it (silence on /// decoded client-mic PCM in; the loop thread's producer callback drains it (silence on
/// underrun) into PipeWire buffers. Mirrors [`PwAudioCapturer`] but inverted (Direction::Output). /// underrun) into PipeWire buffers. Mirrors [`PwAudioCapturer`] but inverted (Direction::Output).
///
/// **Why a stream node and not a `support.null-audio-sink` adapter** (the canonical
/// virtual-mic recipe): tested live on this project's headless graph (PipeWire 1.6.2,
/// 2026-07-03), an adapter with `media.class=Audio/Source/Virtual` never gets a clock — the
/// {source, recorder} group runs with QUANT/RATE 0 and delivers pure silence — and WirePlumber
/// rerouted a feeder stream targeting it to the *default sink* instead (which would play the
/// client's voice out of the speakers, straight into the desktop-audio capture: echo). The
/// stream node below, with `RT_PROCESS` + `priority.session` (see the property comments), is
/// validated working on PipeWire 1.4 (Bazzite) and 1.6 (this box) in both attach orderings.
/// Do not "modernize" this to the adapter recipe without re-running that validation.
///
/// **Liveness contract** (see [`VirtualMic`]): the loop thread exits on a core error (PipeWire
/// daemon restart — the node is gone) or a stream error, which flips `alive` — `push` then
/// returns `false` and the owning pump reopens against the new daemon, recreating the node.
pub struct PwMicSource { pub struct PwMicSource {
pcm: std::sync::mpsc::SyncSender<Vec<f32>>, pcm: std::sync::mpsc::SyncSender<(std::time::Instant, Vec<f32>)>,
channels: u32, channels: u32,
quit: pipewire::channel::Sender<Terminate>, quit: pipewire::channel::Sender<Terminate>,
/// False once the loop thread has exited (daemon/stream death or teardown).
alive: Arc<AtomicBool>,
/// One-shot flush request, consumed by the process callback (clears the jitter ring).
flush: Arc<AtomicBool>,
} }
impl PwMicSource { impl PwMicSource {
@@ -123,21 +143,36 @@ impl PwMicSource {
matches!(channels, 1 | 2), matches!(channels, 1 | 2),
"virtual mic supports 1 or 2 channels, got {channels}" "virtual mic supports 1 or 2 channels, got {channels}"
); );
let (pcm_tx, pcm_rx) = sync_channel::<Vec<f32>>(64); let (pcm_tx, pcm_rx) = sync_channel::<(std::time::Instant, Vec<f32>)>(64);
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>(); let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
let alive = Arc::new(AtomicBool::new(true));
let flush = Arc::new(AtomicBool::new(false));
// Bring-up handshake (mirrors the Windows backend): a PipeWire that isn't running (host
// service started before the user session) must surface as an open ERROR — engaging the
// pump's backoff — not as an instantly-dead instance the pump would churn on.
let (ready_tx, ready_rx) = sync_channel::<Result<()>>(1);
let (alive_t, flush_t) = (alive.clone(), flush.clone());
thread::Builder::new() thread::Builder::new()
.name("punktfunk-pw-mic".into()) .name("punktfunk-pw-mic".into())
.spawn(move || { .spawn(move || {
if let Err(e) = mic_pw_thread(pcm_rx, quit_rx, channels) { if let Err(e) = mic_pw_thread(pcm_rx, quit_rx, channels, flush_t, ready_tx) {
tracing::error!(error = %format!("{e:#}"), "pipewire virtual-mic thread failed"); tracing::error!(error = %format!("{e:#}"), "pipewire virtual-mic thread failed");
} }
// Whether a clean quit or a daemon death: this instance is done — the pump reopens.
alive_t.store(false, Ordering::Release);
}) })
.context("spawn pipewire virtual-mic thread")?; .context("spawn pipewire virtual-mic thread")?;
Ok(PwMicSource { match ready_rx.recv_timeout(Duration::from_secs(5)) {
Ok(Ok(())) => Ok(PwMicSource {
pcm: pcm_tx, pcm: pcm_tx,
channels, channels,
quit: quit_tx, quit: quit_tx,
}) alive,
flush,
}),
Ok(Err(e)) => Err(e),
Err(_) => Err(anyhow!("pipewire virtual-mic init timed out")),
}
} }
} }
@@ -148,8 +183,24 @@ impl Drop for PwMicSource {
} }
impl VirtualMic for PwMicSource { impl VirtualMic for PwMicSource {
fn push(&self, pcm: &[f32]) { fn push(&self, pcm: &[f32]) -> bool {
let _ = self.pcm.try_send(pcm.to_vec()); // drop if the PipeWire side is behind if !self.alive.load(Ordering::Acquire) {
return false;
}
// Timestamped so the process callback can age out chunks that sat in the channel while
// no recorder was attached (see the staleness logic there).
match self.pcm.try_send((std::time::Instant::now(), pcm.to_vec())) {
Ok(()) => true,
// Behind is fine (drop the chunk); a gone receiver means the loop thread exited.
Err(std::sync::mpsc::TrySendError::Full(_)) => true,
Err(std::sync::mpsc::TrySendError::Disconnected(_)) => false,
}
}
fn alive(&self) -> bool {
self.alive.load(Ordering::Acquire)
}
fn discard(&self) {
self.flush.store(true, Ordering::Release);
} }
fn channels(&self) -> u32 { fn channels(&self) -> u32 {
self.channels self.channels
@@ -160,22 +211,38 @@ impl VirtualMic for PwMicSource {
/// the process callback drains into PipeWire buffers (capped, so latency stays bounded). /// the process callback drains into PipeWire buffers (capped, so latency stays bounded).
/// `primed` is a jitter buffer gate — see the process callback. /// `primed` is a jitter buffer gate — see the process callback.
struct MicUserData { struct MicUserData {
rx: Receiver<Vec<f32>>, rx: Receiver<(std::time::Instant, Vec<f32>)>,
ring: VecDeque<f32>, ring: VecDeque<f32>,
channels: usize, channels: usize,
primed: bool, primed: bool,
/// One-shot flush request from [`PwMicSource::discard`] (stale-audio drop after a gap).
flush: Arc<AtomicBool>,
/// When the process callback last ran — a long gap means the ring content predates the
/// current consumer (the stream idles with no recorder attached) and must be dropped.
last_run: Option<std::time::Instant>,
} }
/// PCM older than this never reaches a recorder: chunks that aged in the channel while no
/// recorder was attached, and ring content from before a consumer gap, are dropped instead of
/// bursting out as stale audio when recording (re)starts.
const MIC_STALE: Duration = Duration::from_secs(1);
fn mic_pw_thread( fn mic_pw_thread(
pcm_rx: Receiver<Vec<f32>>, pcm_rx: Receiver<(std::time::Instant, Vec<f32>)>,
quit_rx: pipewire::channel::Receiver<Terminate>, quit_rx: pipewire::channel::Receiver<Terminate>,
channels: u32, channels: u32,
flush: Arc<AtomicBool>,
ready: std::sync::mpsc::SyncSender<Result<()>>,
) -> Result<()> { ) -> Result<()> {
use pipewire as pw; use pipewire as pw;
use pw::{properties::properties, spa}; use pw::{properties::properties, spa};
use spa::param::audio::{AudioFormat, AudioInfoRaw}; use spa::param::audio::{AudioFormat, AudioInfoRaw};
use spa::pod::Pod; use spa::pod::Pod;
// The PipeWire objects are lifetime-chained (guards borrow the mainloop/core), so setup and
// the blocking run share one frame; the IIFE lets every setup `?` funnel through the ready
// handshake below (mirrors the Windows render_thread).
let result = (|| -> Result<()> {
crate::pwinit::ensure_init(); crate::pwinit::ensure_init();
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw mic MainLoop")?; let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw mic MainLoop")?;
let context = pw::context::ContextRc::new(&mainloop, None).context("pw mic Context")?; let context = pw::context::ContextRc::new(&mainloop, None).context("pw mic Context")?;
@@ -188,6 +255,26 @@ fn mic_pw_thread(
move |_| mainloop.quit() move |_| mainloop.quit()
}); });
// Death detection: a core error (the daemon restarted/went away — our remote node no longer
// exists) ends this thread, flipping the owner's `alive` flag so the pump reopens against the
// new daemon. Without this, a PipeWire restart left the loop idling on a dead connection and
// the mic silently broken for the rest of the host's life.
let _core_listener = core
.add_listener_local()
.error({
let mainloop = mainloop.clone();
move |id, _seq, res, message| {
tracing::warn!(
id,
res,
message,
"pipewire core error — virtual mic reopening"
);
mainloop.quit();
}
})
.register();
// media.class=Audio/Source advertises us as a microphone (a recordable source), NOT a // media.class=Audio/Source advertises us as a microphone (a recordable source), NOT a
// playback stream — without it, Direction::Output + Playback would route to the speakers. // playback stream — without it, Direction::Output + Playback would route to the speakers.
let stream = pw::stream::StreamBox::new( let stream = pw::stream::StreamBox::new(
@@ -226,12 +313,21 @@ fn mic_pw_thread(
ring: VecDeque::new(), ring: VecDeque::new(),
channels: channels as usize, channels: channels as usize,
primed: false, primed: false,
flush,
last_run: None,
}; };
let _listener = stream let _listener = stream
.add_local_listener_with_user_data(ud) .add_local_listener_with_user_data(ud)
.state_changed(|_s, _ud, old, new| { .state_changed({
let mainloop = mainloop.clone();
move |_s, _ud, old, new| {
tracing::info!(?old, ?new, "pipewire virtual-mic stream state"); tracing::info!(?old, ?new, "pipewire virtual-mic stream state");
// A stream error is unrecoverable for this instance — exit so the pump reopens.
if matches!(new, pw::stream::StreamState::Error(_)) {
mainloop.quit();
}
}
}) })
.param_changed(|_s, _ud, id, param| { .param_changed(|_s, _ud, id, param| {
let Some(param) = param else { return }; let Some(param) = param else { return };
@@ -253,10 +349,26 @@ fn mic_pw_thread(
let Some(mut buffer) = stream.dequeue_buffer() else { let Some(mut buffer) = stream.dequeue_buffer() else {
return; return;
}; };
// Pull all newly-decoded PCM into the ring. // Stale-audio guard, BEFORE pulling new frames: drop the ring when a flush was
while let Ok(frame) = ud.rx.try_recv() { // requested (uplink gap — see the pump) or when this callback itself hasn't run
// for a while (the stream idled with no recorder attached; whatever the ring
// holds predates the consumer). A recorder must never hear a burst of old audio.
let now = std::time::Instant::now();
let idled = ud
.last_run
.is_some_and(|t| now.duration_since(t) > MIC_STALE);
if ud.flush.swap(false, std::sync::atomic::Ordering::AcqRel) || idled {
ud.ring.clear();
ud.primed = false;
}
ud.last_run = Some(now);
// Pull all newly-decoded PCM into the ring, aging out chunks that sat in the
// channel while nothing consumed them (same staleness rule).
while let Ok((t, frame)) = ud.rx.try_recv() {
if now.duration_since(t) <= MIC_STALE {
ud.ring.extend(frame); ud.ring.extend(frame);
} }
}
let stride = 4 * ud.channels; // F32LE interleaved let stride = 4 * ud.channels; // F32LE interleaved
let datas = buffer.datas_mut(); let datas = buffer.datas_mut();
if datas.is_empty() { if datas.is_empty() {
@@ -353,9 +465,19 @@ fn mic_pw_thread(
) )
.context("pw mic stream connect")?; .context("pw mic stream connect")?;
// Setup complete: the daemon connection and stream connect succeeded — report ready,
// then block until quit/death. (A PipeWire that isn't running never reaches this line;
// its connect error surfaces through the handshake as an OPEN failure, so the pump
// backs off instead of churning on instantly-dead instances.)
let _ = ready.send(Ok(()));
mainloop.run(); mainloop.run();
tracing::debug!("pipewire virtual-mic loop exited (source dropped)"); tracing::debug!("pipewire virtual-mic loop exited (source dropped)");
Ok(()) Ok(())
})();
if let Err(e) = &result {
let _ = ready.send(Err(anyhow!("{e:#}")));
}
result
} }
fn pw_thread( fn pw_thread(
@@ -6,64 +6,39 @@
//! ones, or the loopback re-captures the injected mic (an infinite echo). The installer bundles //! ones, or the loopback re-captures the injected mic (an infinite echo). The installer bundles
//! VB-Audio Virtual Cable (the mic target: its "CABLE Input" render endpoint → "CABLE Output" capture) //! VB-Audio Virtual Cable (the mic target: its "CABLE Input" render endpoint → "CABLE Output" capture)
//! and the host auto-installs the Steam Streaming pair (a loopback-capable render). This module wires //! and the host auto-installs the Steam Streaming pair (a loopback-capable render). This module wires
//! them up at startup so no manual Sound-settings fiddling is ever needed: //! them up so no manual Sound-settings fiddling is ever needed:
//! //!
//! * default **PLAYBACK** → a loopback-capable render that is NOT the mic cable (a real output device //! * the **mic inject target** is assigned FIRST (VB-Cable "CABLE Input" preferred) — mic passthrough
//! is what the cable is bundled for, so it wins the cable even when the cable is the only render
//! endpoint on the box (the loopback then reports itself unavailable instead of echoing);
//! * default **PLAYBACK** → a loopback-capable render that is NOT the mic target (a real output device
//! if one exists, else the Steam Streaming Microphone; **never** the Steam Streaming Speakers, whose //! if one exists, else the Steam Streaming Microphone; **never** the Steam Streaming Speakers, whose
//! loopback is silent — validated live). This is the endpoint [`super::wasapi_cap`] loopback-captures //! loopback is silent — validated live). This is the endpoint [`super::wasapi_cap`] captures;
//! for desktop audio. //! * default **RECORDING** → the mic target's capture endpoint (VB-Cable "CABLE Output") so host apps
//! * default **RECORDING** → the virtual mic's capture endpoint (VB-Cable "CABLE Output") so host apps
//! record the client's mic by default. //! record the client's mic by default.
//! //!
//! [`super::wasapi_mic::find_device`] then resolves the mic INJECT target to "CABLE Input" — a render //! The assignment rules are the PURE [`wiring_plan`](super::wiring_plan) module (unit-tested on every
//! candidate that is NOT the default playback — guaranteeing loopback ≠ mic, so there is no echo. //! platform); this module only enumerates endpoints, applies the plan, and logs. [`wire_now`] runs on
//! every mic/capture (re)open — NOT once per process — because endpoints churn (boot-time
//! registration, hotplug, driver installs) and a stale plan was one of the ways mic passthrough died
//! permanently.
//! //!
//! Setting a default endpoint uses the undocumented `IPolicyConfig` COM interface (the only way to set //! Setting a default endpoint uses the undocumented `IPolicyConfig` COM interface (the only way to set
//! a default device programmatically — neither the `windows` nor `wasapi` crate exposes it; it is the //! a default device programmatically — neither the `windows` nor `wasapi` crate exposes it; it is the
//! same call `mmsys.cpl` makes). Opt out with `PUNKTFUNK_KEEP_DEFAULT` to leave the user's chosen //! same call `mmsys.cpl` makes). Opt out with `PUNKTFUNK_KEEP_DEFAULT` to leave the user's chosen
//! defaults untouched. //! defaults untouched (the plan is still computed — the mic must still pick a target).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it. // Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use super::wiring_plan::{plan, Endpoint, Wiring};
use anyhow::{anyhow, bail, Result}; use anyhow::{anyhow, bail, Result};
use std::ffi::c_void; use std::ffi::c_void;
use std::sync::Once; use std::sync::Mutex;
use wasapi::Direction; use wasapi::Direction;
/// Run the audio device auto-wiring exactly once per process, before the first capturer/mic opens.
/// Blocks until done so the default playback is set before the loopback captures it. Best-effort:
/// every failure is logged, never fatal (the host then falls back to whatever the current defaults
/// are — exactly the pre-wiring behaviour).
pub(crate) fn ensure_wired_once() {
static WIRED: Once = Once::new();
WIRED.call_once(|| {
if std::env::var_os("PUNKTFUNK_KEEP_DEFAULT").is_some() {
tracing::info!("PUNKTFUNK_KEEP_DEFAULT set — leaving the audio default devices untouched");
return;
}
// Run on a dedicated COM-MTA thread so we never collide with the caller's apartment mode
// (the capture/mic threads each initialize their own COM separately).
let handle = std::thread::Builder::new()
.name("pf-audio-wiring".into())
.spawn(|| {
if wasapi::initialize_mta().ok().is_err() {
tracing::warn!("audio wiring: COM init (MTA) failed — skipping");
return;
}
if let Err(e) = ensure_audio_wiring() {
tracing::warn!(error = %format!("{e:#}"),
"audio auto-wiring failed — mic/desktop audio may need manual device defaults");
}
});
if let Ok(h) = handle {
let _ = h.join();
}
});
}
/// `(friendly_name, endpoint_id)` for every ACTIVE endpoint in direction `dir`. /// `(friendly_name, endpoint_id)` for every ACTIVE endpoint in direction `dir`.
fn list_endpoints(dir: Direction) -> Vec<(String, String)> { fn list_endpoints(dir: Direction) -> Vec<Endpoint> {
let mut out = Vec::new(); let mut out = Vec::new();
let Ok(en) = wasapi::DeviceEnumerator::new() else { let Ok(en) = wasapi::DeviceEnumerator::new() else {
return out; return out;
@@ -86,79 +61,85 @@ fn list_endpoints(dir: Direction) -> Vec<(String, String)> {
out out
} }
/// Pick the loopback + mic-capture devices and set them as the default playback/recording. /// Enumerate endpoints, compute the assignment, apply the default-device changes (unless
fn ensure_audio_wiring() -> Result<()> { /// `PUNKTFUNK_KEEP_DEFAULT`), and return the plan for the caller to act on (mic target / loopback
/// echo guard). Must run on a COM-initialized thread (the WASAPI worker threads all
/// `initialize_mta` first). Logged only when the assignment changes, so per-open recomputation
/// stays quiet in the steady state.
pub(crate) fn wire_now() -> Wiring {
let renders = list_endpoints(Direction::Render); let renders = list_endpoints(Direction::Render);
let captures = list_endpoints(Direction::Capture); let captures = list_endpoints(Direction::Capture);
if renders.is_empty() { let want = std::env::var("PUNKTFUNK_MIC_DEVICE")
bail!("no active render endpoints to wire"); .ok()
.map(|s| s.to_lowercase());
let wiring = plan(&renders, &captures, want.as_deref());
// Log assignment changes exactly once (first plan included).
static LAST: Mutex<Option<Wiring>> = Mutex::new(None);
let changed = {
let mut last = LAST.lock().unwrap();
let changed = last.as_ref() != Some(&wiring);
*last = Some(wiring.clone());
changed
};
if changed {
tracing::info!(
mic_render = wiring.mic_render.as_ref().map(|(n, _)| n.as_str()),
mic_capture = wiring.mic_capture.as_ref().map(|(n, _)| n.as_str()),
loopback_render = wiring.loopback_render.as_ref().map(|(n, _)| n.as_str()),
renders = ?renders.iter().map(|(n, _)| n.as_str()).collect::<Vec<_>>(),
"audio wiring plan"
);
if wiring.mic_render.is_some() && wiring.loopback_render.is_none() {
tracing::warn!(
"the virtual mic reserved the only usable render endpoint — desktop audio will be \
unavailable until another output device exists (attach one, or let the host \
install the Steam Streaming pair)"
);
}
} }
// A render is unusable as the desktop-audio loopback if it is a VB-Cable endpoint (reserved for if std::env::var_os("PUNKTFUNK_KEEP_DEFAULT").is_some() {
// the mic inject) or the Steam Streaming Speakers (its loopback is silent — validated live). if changed {
let excluded_loopback = tracing::info!(
|ln: &str| ln.contains("cable") || ln.contains("steam streaming speakers"); "PUNKTFUNK_KEEP_DEFAULT set — leaving the audio default devices untouched"
// "virtual-ish" = a known virtual cable; a render WITHOUT these markers is a real output device, );
// the best loopback source (apps render there and the operator can also hear it). }
let virtualish = |ln: &str| { return wiring;
ln.contains("virtual") }
|| ln.contains("cable") if let Some((name, id)) = &wiring.loopback_render {
|| ln.contains("steam streaming") match set_default_endpoint(id) {
|| ln.contains("voicemeeter") Ok(()) => {
}; if changed {
let loopback = renders tracing::info!(device = %name,
.iter() "audio wiring: default playback = desktop-audio loopback source");
.find(|(n, _)| { }
let ln = n.to_lowercase(); }
!excluded_loopback(&ln) && !virtualish(&ln)
})
.or_else(|| {
renders
.iter()
.find(|(n, _)| n.to_lowercase().contains("steam streaming microphone"))
})
.or_else(|| {
renders
.iter()
.find(|(n, _)| !excluded_loopback(&n.to_lowercase()))
});
// The virtual mic's CAPTURE endpoint host apps record from — VB-Cable "CABLE Output" preferred.
let mic_capture = captures
.iter()
.find(|(n, _)| n.to_lowercase().contains("cable output"))
.or_else(|| {
captures
.iter()
.find(|(n, _)| n.to_lowercase().contains("steam streaming microphone"))
})
.or_else(|| {
captures.iter().find(|(n, _)| {
let ln = n.to_lowercase();
ln.contains("voicemeeter") || ln.contains("virtual")
})
});
match loopback {
Some((name, id)) => match set_default_endpoint(id) {
Ok(()) => tracing::info!(device = %name,
"audio wiring: default playback = desktop-audio loopback source"),
Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"), Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"),
"audio wiring: failed to set the default playback device"), "audio wiring: failed to set the default playback device"),
},
None => {
tracing::warn!("audio wiring: no usable desktop-audio loopback render endpoint found")
} }
} }
if let Some((name, id)) = mic_capture { if let Some((name, id)) = &wiring.mic_capture {
match set_default_endpoint(id) { match set_default_endpoint(id) {
Ok(()) => tracing::info!(device = %name, Ok(()) => {
"audio wiring: default recording = virtual mic (apps record the client's mic)"), if changed {
tracing::info!(device = %name,
"audio wiring: default recording = virtual mic (apps record the client's mic)");
}
}
Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"), Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"),
"audio wiring: failed to set the default recording device"), "audio wiring: failed to set the default recording device"),
} }
} }
Ok(()) wiring
}
/// Open a device by endpoint id, with a name for error context.
pub(crate) fn open_endpoint(ep: &Endpoint) -> Result<wasapi::Device> {
wasapi::DeviceEnumerator::new()
.map_err(|e| anyhow!("DeviceEnumerator: {e}"))?
.get_device(&ep.1)
.map_err(|e| anyhow!("open endpoint {:?}: {e}", ep.0))
} }
// --- IPolicyConfig (undocumented): set a default audio endpoint by id, for all three roles. --- // --- IPolicyConfig (undocumented): set a default audio endpoint by id, for all three roles. ---
@@ -6,7 +6,7 @@
//! COM-apartment-bound and not `Send`, so they live on a dedicated thread (mirrors //! COM-apartment-bound and not `Send`, so they live on a dedicated thread (mirrors
//! `linux::PwAudioCapturer`); only the channel + stop flag + join handle are in the struct. //! `linux::PwAudioCapturer`); only the channel + stop flag + join handle are in the struct.
use super::{AudioCapturer, SAMPLE_RATE}; use super::{audio_control, AudioCapturer, SAMPLE_RATE};
use anyhow::{anyhow, Context, Result}; use anyhow::{anyhow, Context, Result};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
@@ -109,14 +109,36 @@ fn capture_thread(
} }
let res = (|| -> Result<()> { let res = (|| -> Result<()> {
// Loopback = capture the RENDER endpoint: get the default render device, but open a CAPTURE // Loopback = capture the RENDER endpoint: get the default render device, but open a CAPTURE
// client with loopback=true over it. NOTE: the virtual mic (`super::wasapi_mic`) is guarded // client with loopback=true over it. ECHO GUARD: the wiring plan reserves one endpoint for
// to NEVER target this same endpoint — otherwise the client's injected mic would be captured // the virtual mic (`super::wasapi_mic` writes the client's voice there) — capturing THAT
// here and streamed back to the client (infinite echo). Keep that guard in sync if this // endpoint would stream the client's own mic straight back to it. Normally the plan has
// device selection ever changes. // already moved the default playback elsewhere; if the default still IS the mic target
let device = DeviceEnumerator::new() // (PUNKTFUNK_KEEP_DEFAULT, or the cable is the only endpoint), capture the plan's loopback
// endpoint explicitly, or refuse — no desktop audio beats an echo loop.
let wiring = audio_control::wire_now();
let default = DeviceEnumerator::new()
.context("DeviceEnumerator")? .context("DeviceEnumerator")?
.get_default_device(&Direction::Render) .get_default_device(&Direction::Render)
.context("default render endpoint (loopback needs a render device)")?; .context("default render endpoint (loopback needs a render device)")?;
let default_is_mic = match (&wiring.mic_render, default.get_id()) {
(Some((_, mic_id)), Ok(id)) => *mic_id == id,
_ => false,
};
let device = if default_is_mic {
let Some(lb) = &wiring.loopback_render else {
anyhow::bail!(
"the only render endpoint is reserved for the virtual mic (capturing it would \
echo the client's voice back) attach another output device or install the \
Steam Streaming pair to get desktop audio"
);
};
tracing::warn!(mic = %wiring.mic_render.as_ref().unwrap().0, loopback = %lb.0,
"default render endpoint is the virtual-mic target — loopback-capturing the plan's \
endpoint instead");
audio_control::open_endpoint(lb)?
} else {
default
};
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?; let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// 48 kHz f32 interleaved in the requested channel layout; autoconvert lets WASAPI's // 48 kHz f32 interleaved in the requested channel layout; autoconvert lets WASAPI's
// shared-mode SRC match the engine mix format to ours (incl. up/downmix to the requested // shared-mode SRC match the engine mix format to ours (incl. up/downmix to the requested

Some files were not shown because too many files have changed in this diff Show More