Compare commits

...
Author SHA1 Message Date
enricobuehler 8abdd74a62 fix(client/desktop): the Deck keeps its trackpad, and a pad stops buzzing at exit
ci / rust-arm64 (pull_request) Successful in 1m55s
ci / web (pull_request) Successful in 1m14s
ci / docs-site (pull_request) Successful in 1m15s
android / android (pull_request) Successful in 8m52s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m2s
ci / rust (pull_request) Successful in 12m50s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m7s
apple / swift (pull_request) Successful in 1m23s
apple / screenshots (pull_request) Skipped
Three faults in the desktop session's gamepad path.

The Steam Deck lost its built-in trackpad-mouse at the start of every session.
SDL's Valve HIDAPI driver clears the pad's digital mappings during
*enumeration*, which is part of bringing the gamepad subsystem up — so holding
the drivers off from inside GamepadService::pumped could never work: receiving
a GamepadSubsystem means the enumeration has already happened. The hint set
there detached a driver that had already done the damage, and lizard mode only
came back seconds later when the firmware watchdog restored it. The presenter
now disables them with its other pre-SDL_Init hints. The threaded worker always
had this right; only the caller-pumped path was wrong, and it could not fix
itself, hence a separate entry point its callers can place correctly.

Player LEDs did nothing at all on any pad that is not a DualSense. The match
arm handled the DualSense raw-effects path and let everything else fall through
a bare `_`, though SDL exposes set_player_index and owns the per-device
pattern. The wire carries a positional bitmask rather than an index, and the
bridge is the popcount: every convention that reaches this wire spells "player
N" as N lit LEDs — the DualSense patterns 0x04/0x0A/0x15/0x1B/0x1F and the
Switch/XInput run 0x01/0x03/0x07/0x0F alike — so counting them works for both,
where reading a bit position would only ever suit one. No lit LED means no
player, not player 0. The remaining unhandled variants are now named rather
than swept up by `_`, so a new one cannot join them silently.

A forwarded pad could be left buzzing when the session ended. detach() only
posts Ctl::Detach; the close that flushes the pad, tells the host to remove it
and explicitly zeroes the motors runs when the pump next drains that message.
Single mode broke out of the loop immediately after detaching and Event::Quit
never detached at all, so both skipped it entirely. The teardown now sits where
every exit converges instead of on the individual breaks. That still leaves the
several paths that leave by `?` on a fatal overlay or present error, so the
pump also silences its slots on Drop — the explicit call stays, because a pad
should go quiet before a long teardown rather than after it. Drop closes the
slots directly rather than draining the queue that would have done it: same
physical outcome, and it touches no lock, where draining reaches an unwrap on a
Mutex that would abort the process if it panicked mid-unwind.
2026-08-04 19:10:45 +02:00
enricobuehler 2c03290a5e Merge pull request 'chore(release): bump workspace version to 0.24.0' (#29) from worktree-release-0240 into main
audit / cargo-audit (push) Successful in 50s
apple / swift (push) Successful in 1m19s
audit / bun-audit (plugin-kit) (push) Failing after 24s
audit / bun-audit (sdk) (push) Failing after 21s
audit / bun-audit (web) (push) Failing after 36s
audit / docs-site-audit (push) Successful in 33s
audit / pnpm-audit (push) Successful in 24s
apple / screenshots (push) Successful in 5m40s
audit / license-gate (push) Successful in 5m48s
android-screenshots / screenshots (push) Successful in 1m33s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m1s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m0s
decky / build-publish (push) Successful in 36s
linux-client-screenshots / screenshots (push) Successful in 1m42s
android / android (push) Successful in 9m17s
sbom / sbom (push) Successful in 28s
arch / build-publish (push) Successful in 13m16s
web-screenshots / screenshots (push) Successful in 4m26s
flatpak / build-publish (push) Successful in 8m8s
release / apple (push) Successful in 10m44s
ci / rust (push) Successful in 6m48s
ci / web (push) Successful in 1m33s
ci / docs-site (push) Successful in 1m36s
ci / rust-arm64 (push) Successful in 1m49s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m56s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m45s
windows-host / package (push) Successful in 11m53s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Successful in 24s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 6s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 13s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
deb / build-publish (push) Successful in 6m25s
deb / build-publish-client-arm64 (push) Successful in 1m19s
docker / deploy-docs (push) Successful in 16s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / builders-arm64cross (push) Successful in 8s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 25s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m22s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m58s
deb / build-publish-host (push) Successful in 4m11s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m10s
Reviewed-on: #29
2026-08-03 19:43:24 +00:00
enricobuehler b6a370a0fd Merge remote-tracking branch 'origin/main' into worktree-release-0240
ci / web (pull_request) Successful in 1m8s
apple / swift (pull_request) Successful in 1m21s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m31s
ci / docs-site (pull_request) Successful in 1m57s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 4m14s
android / android (pull_request) Successful in 4m35s
ci / rust (pull_request) Successful in 6m0s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 2m56s
2026-08-03 21:40:38 +02:00
enricobuehler 7db83445b2 Merge pull request 'fix(host/input): rumble comes back when a controller does' (#25) from worktree-haptics-m1-rumble-seq into main
apple / swift (push) Successful in 1m18s
ci / web (push) Successful in 1m31s
ci / rust-arm64 (push) Successful in 2m9s
ci / docs-site (push) Successful in 1m43s
deb / build-publish-client-arm64 (push) Successful in 1m47s
android / android (push) Successful in 5m55s
deb / build-publish-host (push) Successful in 4m49s
apple / screenshots (push) Successful in 5m54s
arch / build-publish (push) Successful in 8m33s
deb / build-publish (push) Successful in 4m53s
ci / rust (push) Successful in 6m22s
windows-host / package (push) Successful in 12m32s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 28s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 15s
docker / deploy-docs (push) Successful in 30s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 16s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 13s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 33s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 27s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / builders-arm64cross (push) Successful in 14s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 37s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 18m33s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 19m43s
Reviewed-on: #25
2026-08-03 19:17:49 +00:00
enricobuehler 5582a6ea51 Merge branch 'main' into worktree-haptics-m1-rumble-seq
apple / swift (pull_request) Successful in 1m17s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m51s
ci / docs-site (pull_request) Successful in 2m5s
ci / rust-arm64 (pull_request) Successful in 2m22s
android / android (pull_request) Successful in 3m25s
ci / rust (pull_request) Successful in 8m4s
2026-08-03 19:17:01 +00:00
enricobuehler f7b85ec1fd Merge pull request 'fix(host/pads): an unplugged controller actually disappears' (#26) from worktree-haptics-m2-pad-slots into main
android / android (push) Canceled after 1m16s
apple / swift (push) Canceled after 1m18s
apple / screenshots (push) Canceled after 0s
ci / docs-site (push) Successful in 1m21s
arch / build-publish (push) Canceled after 1m33s
ci / rust (push) Canceled after 1m33s
ci / web (push) Canceled after 1m36s
ci / rust-arm64 (push) Canceled after 1m42s
deb / build-publish (push) Canceled after 0s
deb / build-publish-host (push) Canceled after 1m35s
deb / build-publish-client-arm64 (push) Canceled after 48s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 24s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 2s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 18s
windows-host / package (push) Canceled after 1m48s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
Reviewed-on: #26
2026-08-03 19:16:24 +00:00
enricobuehler 327301e012 docs(release): the 0.24.0 notes cover the two controller fixes
ci / web (pull_request) Successful in 1m23s
ci / docs-site (pull_request) Successful in 1m28s
android / android (pull_request) Successful in 5m0s
apple / swift (pull_request) Successful in 1m20s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 6m19s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m20s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m1s
ci / rust (pull_request) Successful in 12m53s
PRs #25 and #26 are going into this release, and neither was in the notes.

Both are user-visible and easy to have lived with without knowing why:
force-feedback stopping for good after a controller reconnect (roughly half of
reconnects, every platform), and an unplugged pad staying visible to the game
for the rest of the session (every time, if it was your only controller).

The whatsnew line for the rumble fix is Play listing copy and that file has a
500-character ceiling, so "A decoder hiccup no longer snowballs into a burst of
broken frames" loses "snowballs into" for "causes" — same meaning, and the new
line is kept short. 498 of 500 used.
2026-08-03 19:52:05 +02:00
enricobuehler ab4cd06e86 Merge remote-tracking branch 'origin/main' into worktree-haptics-m2-pad-slots
ci / docs-site (pull_request) Successful in 3m0s
apple / swift (pull_request) Successful in 1m19s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 4m42s
ci / web (pull_request) Successful in 1m39s
ci / rust-arm64 (pull_request) Failing after 11m17s
ci / rust (pull_request) Successful in 7m36s
2026-08-03 19:47:34 +02:00
enricobuehler 3eab1e41df Merge remote-tracking branch 'origin/main' into worktree-haptics-m1-rumble-seq
ci / web (pull_request) Successful in 1m42s
ci / docs-site (pull_request) Successful in 1m40s
ci / rust-arm64 (pull_request) Successful in 3m22s
apple / swift (pull_request) Successful in 1m23s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 4m8s
ci / rust (pull_request) Successful in 8m11s
2026-08-03 19:47:32 +02:00
enricobuehlerandClaude Opus 5 62573d2781 docs(release): the 0.24.0 notes cover the ABR sweep
ci / web (pull_request) Successful in 1m11s
android / android (pull_request) Canceled after 0s
apple / swift (pull_request) Canceled after 0s
ci / rust (pull_request) Failing after 3m8s
apple / screenshots (pull_request) Canceled after 0s
ci / rust-arm64 (pull_request) Canceled after 1m56s
ci / docs-site (pull_request) Canceled after 32s
windows / build (aarch64-pc-windows-msvc) (pull_request) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (pull_request) Canceled after 0s
PR #28 merged after the bump commit was written, so the notes described a
release that no longer matched the tree. Merged origin/main and added what it
brings: 45 commits since v0.23.0 now, not 39.

Four user-facing entries, because eleven defects in one path is not one bullet
and the pinning is the headline the field reports have been describing for
months ("my bitrate is stuck at 20"):

- the 20 Mbps pin itself, with the measured escape (150 Mbps in ~16 s against
  ~17 minutes) — the number is the point, since the old behaviour was not "slow
  to climb" but "never arrives"
- the five single-window lessons the controller treated as permanent
- throughput counted with FEC parity, which rose with the loss it was meant to
  detect
- the silent host re-target, which made a client's first climb a request to go
  DOWN

The Under the hood section gets the whole sweep in one bullet rather than
scattering it, and PUNKTFUNK_ABR_MAX_MBPS moves from the probe bullet into it
(it now binds at construction, not only on probe-learned ceilings, so it no
longer belongs to the probe).

Play notes gain an ABR line and now run 459/500 chars; the gate's real logic was
re-run against the file, including the byte-identical check. Voice check over
everything above "Under the hood" is clean of internal vocabulary.

Re-verified after the merge: cargo metadata --locked resolves, cargo fmt --all
--check clean, doc lazy-continuation scanner 0 hits. #28 touched no manifest, so
the version bump and the versions-only lock diff are untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 19:47:01 +02:00
enricobuehler d383fa6103 Merge remote-tracking branch 'origin/main' into worktree-release-0240 2026-08-03 19:44:53 +02:00
enricobuehlerandClaude Opus 5 93608980ae chore(release): bump workspace version to 0.24.0
ci / web (pull_request) Successful in 1m5s
android / android (pull_request) Canceled after 1m19s
apple / swift (pull_request) Canceled after 0s
apple / screenshots (pull_request) Canceled after 0s
ci / rust (pull_request) Canceled after 2m50s
ci / rust-arm64 (pull_request) Canceled after 2m0s
ci / docs-site (pull_request) Canceled after 1m29s
windows / build (aarch64-pc-windows-msvc) (pull_request) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (pull_request) Canceled after 0s
A minor bump: 39 commits since v0.23.0 across 121 files. Mostly a fix-up of
0.23.0 — the slice wire's reassembler sized every sentinel-opened AU at
max_frame_bytes and lost 9 of 12 in-flight frames on any link that reorders,
which is the freeze field reports were seeing on Android and the session client
— plus the desktop presenter rebuild (intent model, V-Sync/VRR as real settings,
the driver's queue-free vblank mode where it exists), the Decky settings tab
growing from nine rows to the whole store, a "Forward controllers" off switch
for passthrough couches, and plugin output finally reaching the console's log
page. The canary base is already 0.24 — scripts/ci/pf-version.sh derives it as
one minor ahead of the latest stable tag — so this is the version canary has
been publishing against all along.

No wire, ABI or driver-protocol change: wire protocol 2, C ABI 14, virtual-display
driver protocol 6 and the Windows virtual-gamepad channel 3 are all identical to
0.23.0. No new capability bits either — VIDEO_CAP_MULTI_SLICE took the video-caps
byte's last free bit in 0.23.0 and nothing here needed the next one. The only
generated-header change since the tag is documentation (probe elapsed_ms
semantics), already committed and verified by ci.yml's staleness gate on main.

Lock touched for the 32 workspace members only, via `cargo update --workspace`:
diff against origin/main is versions-only, 32 insertions and 32 deletions (the
33rd 0.23.0 line in the lock is the third-party `wasapi` crate, which sits at
0.23.0 itself — same trap as the last cut). `cargo metadata --locked` resolves;
`cargo fmt --all --check` clean in both the main and the packaging/windows/drivers
workspaces.

api/openapi.json is deliberately left at 0.23.0: it tracks API edits and lags a
release, as in every prior cut.

Notes at docs/releases/v0.24.0.md, per docs/releases/README.md — authored with the
bump so CI's ensure_release seeds the release body at tag creation. Play's "What's
new" at docs/releases/whatsnew/v0.24.0.txt (409/500 chars), which android.yml now
gates as a hard failure at step 1; the gate's own logic was run locally against
this file, including the byte-identical-to-another-release check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 19:39:00 +02:00
enricobuehler 1feeff3ca6 Merge pull request 'fix(abr): eleven defects from a sweep of the Automatic-bitrate path' (#28) from fix/abr-sweep into main
apple / swift (push) Successful in 1m19s
ci / web (push) Successful in 1m23s
ci / docs-site (push) Successful in 1m23s
ci / rust-arm64 (push) Successful in 2m23s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 19s
deb / build-publish-client-arm64 (push) Successful in 1m23s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 16s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 24s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 23s
android / android (push) Successful in 5m58s
deb / build-publish (push) Successful in 5m52s
docker / builders-arm64cross (push) Successful in 13s
deb / build-publish-host (push) Successful in 5m5s
docker / deploy-docs (push) Successful in 36s
apple / screenshots (push) Successful in 6m14s
arch / build-publish (push) Successful in 8m31s
ci / rust (push) Successful in 8m36s
flatpak / build-publish (push) Successful in 7m27s
windows-host / package (push) Successful in 13m53s
windows-host / winget-source (push) Skipped
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m31s
windows-host / canary-manifest (push) Successful in 35s
release / apple (push) Successful in 11m43s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m42s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 4m12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m7s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m2s
Reviewed-on: #28
2026-08-03 17:33:28 +00:00
enricobuehler 1ae8b4d4ca fix(client/abr): let the ceiling follow a host-initiated re-target
apple / swift (pull_request) Successful in 1m15s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 4m8s
ci / web (pull_request) Successful in 2m1s
ci / rust-arm64 (pull_request) Successful in 2m25s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m14s
ci / docs-site (pull_request) Successful in 1m14s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m44s
ci / rust (pull_request) Successful in 9m9s
Interaction between two fixes in this series. The host now tells the client when
a rebuild re-resolves an Automatic rate, and that rate can legitimately sit ABOVE
the client's climb ceiling — the ceiling is the negotiated start rate until the
capacity probe raises it, while the host's re-resolve answers "what do these
pixels actually need" (a 1080p session mirroring a 4K panel resolves ~3× higher).

Left alone, the client would learn the new rate, notice it was above a stale
ceiling, and step the host straight back down off the rate it had just chosen
for itself. So an ack raises the ceiling to meet it. `set_ceiling` only ever
raises and still clamps to PUNKTFUNK_ABR_MAX_MBPS, which is the one limit that
should bind here. No effect on ordinary acks: a climb is never requested above
the effective ceiling to begin with.
2026-08-03 18:53:02 +02:00
enricobuehler 33ecd8e1a5 fix(client/abr): a granted climb disproves the learned cap
Completing the cap-escape fix. Backing the re-probe clock off to 12 s got the
client asking again quickly, but each ask only LIFTED the cap by +12.5 % — so
even a host that had fully recovered still granted the session its real ceiling
one small step at a time, ~4 minutes from the 20 Mbps default to a 300 Mbps
link. The crawl was never the point; re-learning was.

A request granted IN FULL at or above the cap is the host's own word that the
limit is gone. Drop the cap outright at that point instead of nudging it. A
standing limit is unaffected — it answers the same re-probe with another short
ack, which re-latches it and doubles its clock, exactly as before.

Adds the end-to-end regression the sweep was really about: a session pinned at
20 Mbps by a transient cadence refusal, under a probe-measured 300 Mbps ceiling,
now reaches 150 Mbps in 22 windows (~16 s) where it used to need ~17 minutes.
2026-08-03 18:53:02 +02:00
enricobuehler 48565c4e9e fix(host/abr): stop pinning Automatic sessions, and tell the client when the rate moves
Two host-side halves of the same sweep.

**The cadence latch.** `cadence_degraded` — which makes the control task refuse
bitrate CLIMBS — was latched true for as long as the session was escalated
(adaptive capture depth or pipelined retrieve), independently of whether encode
was still missing deadlines. The client cannot tell that refusal apart from an
encoder's real ceiling: both arrive as a short `BitrateChanged`, and two
identical ones latch a cap. Escalation needs ~20 net behind-frames, which a
startup hitch supplies while the ABR is still in slow start at the 20 Mbps
default — so one transient pinned the whole session there, long after the
escalation had bought back the headroom it was for, and escaping cost +12.5 %
per 60 s. An escalated session is still judged strictly (ANY net behind-frame
keeps it flagged, where an unescalated one gets the full bucket), but being
escalated no longer flags it by itself: escalating exists so cadence CAN be
held, and once it is, refusing climbs refuses the thing that worked. The rule
moves into `encode_behind_cadence` so it is stateable and testable.

**The silent re-target.** `adopt_built_bitrate` publishes the rate a rebuilt
pipeline actually opened at — `build_pipeline` re-resolves an Automatic rate
whenever the source delivers a size the session did not negotiate, the
mirrored-panel case — and the encoder's own clamp can land below what the
control task already acked. Neither reached the client, whose controller keeps
its own copy of that number as its climb base. A 1080p client mirroring a 4K
panel therefore believed 20 Mbps while the host encoded 60, and its first climb
computed from the stale base asked for 40: a re-target DOWNWARD, paying an
encoder rebuild to get there. Both paths now push the applied rate to the
control task, which sends `BitrateChanged` — the existing 9-byte message, which
already means precisely this and which clients already handle arriving
unprompted. No wire-format change, no capability negotiation, old clients
unaffected.

2 host tests added.
2026-08-03 18:53:02 +02:00
enricobuehler e9a7373c76 fix(client/abr): measure delivered throughput in media bytes, not wire bytes
The controller's two throughput-driven gates both compare "what the pipeline
carried" against the ENCODER's target: the utilization gate asks whether a clean
window actually tested that target (a calm menu proves nothing), and the
never-decaying proven mark bounds how far every later climb may step.

Both were fed `bytes_received`, which counts every accepted datagram — headers,
FEC parity, probe filler, audio. So the figure rose with the redundancy the host
adds in ANSWER to loss: at 25 % FEC the gate passed with the encoder emitting
~55 % of target, and the proven mark inherited the same inflation permanently.
The signal was weakest exactly on the lossy links it exists for.

Count data-shard payload separately at the reassembler's routing decision — the
same place, and for the same reason, the probe counters are already stamped —
and feed the ABR that. First time both gates are dimensionally honest: a media
rate compared against a media target.
2026-08-03 18:53:02 +02:00
enricobuehler f7a8c2013d fix(client/abr): the controller stops learning the wrong lessons from one window
Six defects found by a sweep of the Automatic-bitrate path, all of them the same
shape: a single window, or a single refusal, taught the controller something it
then treated as permanent.

- Rolling baselines (OWD, client decode, host encode) armed off ONE sample. The
  baseline is a rolling minimum, so one window IS the floor — and `on_ack`
  deliberately clears the encode baseline after every decrease we ourselves
  asked for, re-opening that hole each time. A calm re-seed window followed by
  ordinary motion read as 4 ms of "congestion", backed off, cleared again, and
  ratcheted toward the floor on a link that was never the problem. All three now
  need BASELINE_MIN_WINDOWS of evidence before they may fire, via one shared
  `score_baseline` (the three copies had already drifted apart).

- A mode switch rebased only the encode baseline. Decode and OWD are just as
  mode-scoped: 4K120 decodes slower and puts bigger frames on the wire than
  1080p60, so the old floor was one the new mode cleared on its first window —
  ~30 s of every window scoring bad, i.e. a backoff every other window. A switch
  UP in mode cratered the rate instead of raising it. `proven_kbps` goes with
  them; throughput the old mode's decoder digested is not evidence about this one.

- `proven_kbps` — never decayed, and permanent authority over how far every
  later climb may step — was raised by any window without a decode rise,
  including ones scored SEVERE. The windows that overstate delivered throughput
  are exactly the damaged ones: a stall's backlog draining at once, a flush's
  queue, the FEC surge answering a loss burst. Now only clean windows raise it.

- A learned cap escaped at +12.5 % per ~60 s. The host cannot distinguish a
  durable encoder ceiling from a climb refused while it is transiently behind
  cadence, and the latter routinely latches during slow start at the 20 Mbps
  default — from which crossing the gap to a probe-measured ceiling took upwards
  of twenty minutes. Re-probe after 12 s instead, doubling the interval each time
  the lift is immediately re-learned: a transient is out in one interval, a real
  ceiling settles into a slow poll.

- The decode cap latched AT the rate that choked, authorizing a climb straight
  back into the failure, and a bare jump-to-live flush could teach a "decoder
  knee" from what was a network event. It now latches just under the choke rate
  (inside the ±1/8 band the evidence already required) and only credits a flush
  where the decode signal is absent and cannot speak for itself.

- PUNKTFUNK_ABR_MAX_MBPS bound only probe-learned ceilings, not the negotiated
  start rate — so the one knob an Automatic session gives the operator did
  nothing when the session already started above it. It now binds at
  construction, and a session sitting above its ceiling steps down to it (no
  congestion signal will ever find that: the link is fine, the cap is policy).

Also: a SetBitrate dropped by a full control queue counted toward MAX_UNACKED,
so three of them retired the controller for the session while logging that an
"older host" was at fault. The pump now tells the controller what happened.

Wire format and ABI untouched. 34 abr tests green (3 new).
2026-08-03 18:53:02 +02:00
enricobuehler 926e2ccbdd Merge pull request 'feat(decky): the settings tab covers the whole store, as a SteamOS-style sidebar' (#24) from worktree-decky-stats-overlay-toggle into main
ci / web (push) Successful in 1m7s
ci / rust-arm64 (push) Successful in 2m12s
decky / build-publish (push) Successful in 30s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
ci / docs-site (push) Successful in 1m15s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 10s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 19s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m5s
docker / builders-arm64cross (push) Successful in 10s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m26s
docker / deploy-docs (push) Failing after 6m7s
ci / rust (push) Successful in 8m27s
Reviewed-on: #24
2026-08-03 16:44:55 +00:00
enricobuehler b8b38d082e Merge pull request 'fix(plugins): plugin output reaches the console's log page, and /tmp is no longer hidden from the runner' (#27) from worktree-plugin-logs-and-vh-fixes into main
ci / rust (push) Canceled after 23s
ci / rust-arm64 (push) Canceled after 24s
ci / web (push) Canceled after 24s
ci / docs-site (push) Canceled after 24s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
apple / swift (push) Successful in 1m16s
deb / build-publish-client-arm64 (push) Successful in 1m12s
deb / build-publish (push) Successful in 3m52s
deb / build-publish-host (push) Successful in 4m45s
android / android (push) Successful in 5m24s
apple / screenshots (push) Successful in 5m57s
arch / build-publish (push) Successful in 8m4s
windows-host / package (push) Successful in 17m44s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m34s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m42s
Reviewed-on: #27
2026-08-03 16:44:31 +00:00
enricobuehler 9979489b56 fix(host/pads): an unplugged controller actually disappears
ci / web (pull_request) Successful in 1m11s
apple / swift (pull_request) Successful in 1m18s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m24s
ci / rust-arm64 (pull_request) Successful in 1m35s
android / android (pull_request) Successful in 3m11s
ci / rust (pull_request) Successful in 6m3s
Unplug a controller mid-session and the virtual pad it was driving outlives
it: the game keeps seeing a connected, permanently idle device for the rest of
the session. The single-controller session — the common case — hits this every
time.

`PadSlots::sweep` needs two passes to retire a pad. The first pass to see the
mask bit clear only ARMS the 300 ms devnode-churn grace; the drop lands on a
later pass. But sweep runs only from a state frame, and the producer emits
exactly one frame per detach — `native/input.rs` guards the emit on the bit
still being set — so for a pad with no still-changing sibling in the same
manager, the second pass never comes. Nothing periodic reaches sweep:
`heartbeat` and `pump` walk the slots without it.

Split the two halves. `sweep` still folds a frame's mask into the grace
clocks, and `reap` — new — drops whatever has run out, with no frame needed.
Every manager now reaps on the periodic pump it already runs, so the teardown
completes ~300 ms after the detach instead of never.

`reap` deliberately cannot arm a clock: it only reads `inactive_since` and
clears it, so a pad whose bit never went clear has nothing to run out and no
amount of reaping can drop it. That is what makes it safe on a hot loop, and
it keeps the anti-flap guarantee intact — a mask that blips clear and returns
still never churns a devnode.

The two existing tests hand-fed a SECOND removal frame, which production never
sends; they passed while the real path leaked. Both now drive the unplug
through a pump tick, and PadSlots gains three tests pinning the new
invariants. Verified non-vacuous: with the reap neutered, both manager tests
fail with "the pump tick never completed the unplug".

Behaviour notes: this puts UI_DEV_DESTROY on the GameStream control thread's
budget for the first time, and a mask glitch longer than the grace now really
does flap — which is SWEEP_GRACE working as documented, so the constant stays.

Found by the 2026-08-03 force-feedback sweep (B2 — see the backlog in
punktfunk-planning design/haptics-sweep-2026-08-03.md).
2026-08-03 17:34:47 +02:00
enricobuehler 14502769e0 fix(host/input): rumble comes back when a controller does
android / android (pull_request) Failing after 18s
ci / docs-site (pull_request) Successful in 1m7s
ci / web (pull_request) Successful in 1m9s
apple / swift (pull_request) Successful in 1m15s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m45s
ci / rust (pull_request) Successful in 6m9s
Unplug a pad mid-session and plug it back in, and roughly half the time it
never rumbles again for the rest of the session.

The removal arm restarted the pad's rumble sequence counter. The client's
reorder gate does not restart: `rumble_last_seq` lives for the whole QUIC
connection and has no reset path, so it still holds whatever the pad reached
before the unplug. Restarting the host counter therefore hands the client a
seq it has already seen, and its wrapping half-space compare drops every
envelope until the counter climbs back past the stored value — up to 128
sends. Since the counter only advances on a level change or a ~120 ms renewal
while a level is non-zero, that spans many separate rumble events, so it reads
as a flaky controller rather than a clean outage.

Whether it bites is decided by how much the pad rumbled beforehand, which is
why it looks intermittent: a pad that never rumbled before the re-plug has
`None` on the client side and always heals.

The counter now survives, matching the sibling pad-state gate — whose comment
eleven lines above already explains that a re-plug must arrive with a still-
newer seq to be accepted. The three clears that actually end the stale lease
move into `clear_pad_feedback`, whose signature deliberately has no seq
parameter so the arm cannot regress by editing.

Covered by a regression test that drives the real wire encoder and the real
client gate, and asserts the pre-fix behaviour is genuinely rejected across
the whole forward window, so it cannot pass vacuously.

Found by the 2026-08-03 force-feedback sweep (B1/T5 — see the backlog in
punktfunk-planning design/haptics-sweep-2026-08-03.md).
2026-08-03 16:57:11 +02:00
enricobuehlerandClaude Opus 5 db1faef9fb docs(plugins): don't name a release that doesn't exist yet
ci / web (pull_request) Successful in 1m4s
apple / swift (pull_request) Successful in 1m28s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m45s
ci / rust-arm64 (pull_request) Successful in 2m39s
android / android (pull_request) Successful in 3m16s
ci / rust (pull_request) Successful in 7m58s
The /tmp troubleshooting note said PrivateTmp=yes shipped "until 0.23.1".
0.23.0 is the latest tag and the next number isn't decided, so that could be
wrong on arrival. "In earlier releases" is true whichever number it gets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 16:00:48 +02:00
enricobuehlerandClaude Opus 5 442ea12b96 test(mgmt): pin the two things that would silently break plugin logging
The runner holds the PLUGIN token and nothing else — on Windows its LocalService
principal cannot read the admin one at all. `plugin_may_access` is an exclusion
list, so `/plugins/logs` is reachable today only because it happens not to match
`/ui-credential`. If that ever changed, plugin logs would go quiet in the console
with no other symptom and no failing test. Now asserted on that lane directly.

The second test covers ingest end to end through `GET /logs`: the `plugin:` target
prefix the console's Host/Plugins filter keys on, the level coercion (an unranked
level would sort as 0 and hide under every filter setting), a sourceless line
being attributed to the runner rather than to nothing, the caller's timestamp
surviving the trip, and an oversized batch being refused whole.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 15:56:25 +02:00
enricobuehlerandClaude Opus 5 1e56705b86 fix(sdk/log-ship): a busy plugin's lines survive a POST, and the shutdown tail is actually sent
Two bugs in the log shipper, both found by re-reading it rather than by a
failing test, and both of the kind where the symptom is a missing log line —
which is the one failure a logging path must not have.

The recursion guard was held across the whole `await fetch`, and `enqueue`
checked it. So every line logged while a POST was open was dropped, silently.
That window is milliseconds when the host is healthy and much longer when it is
not, and the lines lost are whatever a busy plugin happened to be saying — so
the shipper was least reliable exactly when it was most needed. The flag now
guards flush re-entry only (the interval can fire while a slow POST is still
open, and two concurrent flushes would splice disjoint batches out of one queue
and deliver them out of order). Nothing on the shipping path logs, so the
recursion it was guarding cannot form; that is now a stated rule at the top of
the file rather than a flag that costs real lines.

An explicit `flush()` hit that same re-entry guard and returned having sent
nothing. That is the shutdown path: the runner flushes once more after its
units' finalizers have run, and those last lines are the ones that say whether
the shutdown was clean. It now waits for an in-flight flush before starting its
own.

Both are covered by tests that fail against the previous code. The first needed
a server that signals when it has the request — logging merely "after calling
flush()" passes against the bug, because flush yields at its own awaits long
before the fetch starts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 15:53:29 +02:00
enricobuehlerandClaude Opus 5 365caa23be fix(plugins): plugin output reaches the console's log page, and /tmp is no longer hidden from the runner
A user could not get the VirtualHere plugin to use their VirtualHere client
and asked, reasonably, where the logs were. There was no good answer, and the
reason they were stuck turned out to be ours.

**The runner could not see /tmp.** `punktfunk-scripting.service` set
PrivateTmp=yes, which hands the unit a private tmpfs. But integrating with
things already running on the box is the entire job of a plugin, and on Linux
those talk over /tmp: VirtualHere's client IPC is the FIFO pair /tmp/vhclient +
/tmp/vhclient_response, X11 is /tmp/.X11-unix. So the plugin launched the vendor
binary happily and could then never reach the daemon behind it — while the same
command worked perfectly in the operator's own shell, because that shell has the
real /tmp. No config change could fix it, which is exactly the loop the report
described. PrivateTmp is now off, with /tmp added to ReadWritePaths (which
ProtectSystem=strict would otherwise make read-only).

**Plugin logs now land in the console.** Plugins are not host child processes —
the runner is a separate bun process that import()s each plugin in-process — so
nothing they print passed through the host's tracing, and the console's Logs
page could not show a single plugin line. The fallback was journalctl on Linux;
on Windows the runner's scheduled task writes no log file at all, so a failing
plugin was diagnosable only by stopping the task and re-running the runner by
hand. Both mean shell access on the host box, which is what the console exists
to avoid — and it left the one question a stuck user asks with no answer.

So the runner now tees its output to POST /api/v1/plugins/logs, and those lines
join the host's own ring under one cursor, targeted plugin:<name>. The console
grows a Host/Plugins switch beside the level filter; an empty Plugins view says
the thing that is actually usually wrong (the runner isn't running) rather than
"adjust the filter".

The shipper keeps stdout authoritative — journald and foreground output are
unchanged whatever the host is doing — and is built so that logging can never
hurt the thing being logged: it never throws into a caller, holds a bounded
queue that drops oldest and then says how many, backs off when the host is away
(a restart is normal), and re-sends a batch the host failed to take. Lines
logged while a POST is in flight are kept, which cost one round to get right:
the first version held its recursion guard across the await and silently dropped
exactly the lines a busy plugin produces.

Runner lines that report a failure (a refused unit file, a crashed plugin, a
give-up) now go out at warn/error instead of all arriving as INFO, so the
console's level filter means something for them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 15:47:51 +02:00
enricobuehlerandClaude Opus 5 f71bee917b Merge origin/main; the presenter rebuild's four settings join the sidebar
ci / web (pull_request) Successful in 1m14s
ci / rust-arm64 (pull_request) Successful in 1m41s
ci / docs-site (pull_request) Successful in 1m36s
ci / rust (pull_request) Successful in 6m1s
`#20` landed while this branch was open and added four settings the
console screen groups under a new "Presentation" header: Prioritize,
Smoothness buffer, V-Sync and Follow variable refresh. A branch whose
whole claim is "everything the store holds is reachable" cannot merge
past those, so they get a Presentation page of their own, in the console
screen's position (after Video, before Audio) and with its wording.
Smoothness buffer is indented under Prioritize and disabled until the
intent is Smoothness — the same relationship the console's `enabled`
gate draws.

The docs conflict resolves to main's side plus this branch's correction:
the 4:4:4 advertisement claim main rewrote is the current one and stays,
while "Android, Decky and the console home don't offer it" was wrong
about two of the three before this branch and about all three after it.
The four new settings' paragraphs pick up the console home and Decky the
same way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:40:05 +02:00
enricobuehlerandClaude Opus 5 6de78213ee feat(decky): the settings tab covers the whole store, as a SteamOS-style sidebar
ci / web (pull_request) Successful in 1m4s
ci / docs-site (pull_request) Successful in 1m25s
ci / rust-arm64 (pull_request) Successful in 1m31s
ci / rust (pull_request) Canceled after 3m0s
The stats overlay was the visible half of a general problem: nine of the
client's settings had a row here and twenty didn't, so a Deck that never
sees a desktop could not reach its own decoder, chroma, HDR, audio
layout, echo cancellation, touch or mouse model, scroll direction,
auto-wake, or either audio endpoint. Everything the store holds is here
now — except the two things a plugin backend genuinely cannot answer,
named in `backend.ts` so the next reader doesn't go looking: which
physical pad is player 1 (SDL's live device list lives in the client
process, and no CLI enumerates it) and the session's remembered window
size, which is not a preference.

Thirty rows is too many to scroll past on a thumbstick, so they are
split across a `SidebarNavigation` — the left-rail-of-categories layout
SteamOS's own Settings uses, and the one Deck users already know. Every
page fits on screen without scrolling, which is the point: the rail is
the index, so nothing is more than one hop away. The categories, their
order and the wording of the rows are the console settings screen's — it
is the other settings editor reachable without leaving Gaming Mode, and
two different orders for one store is how people stop trusting either.
It shows them as one steppable list because it has no pointer and no
room for a rail; here they become the rail's pages. The six pages take
one shared settings object rather than each holding state, so a change
on one is visible on the others the moment you switch.

Three more rules:

- A dependent setting is INDENTED under what it depends on and DISABLED,
  never hidden: mic device and echo cancellation under the microphone,
  controller type under forwarding. The console dims those rows for the
  same reason, and a row that vanishes as you toggle the one above it is
  a moving target for a thumbstick. The device row at the foot of Audio
  is rendered even while it reads, for that reason.
- A picker with nothing to pick doesn't appear: the GPU row shows up
  only where the enumeration found more than one adapter, so it is
  absent on a Deck and present on a Bazzite desktop with a dGPU.
- A setting that behaves differently HERE says so in its own
  description rather than being dropped. Capture system shortcuts holds
  nothing back under gamescope; fullscreen-on-stream can't lose to a
  launch that always passes `--fullscreen`; the client's library toggle
  isn't this plugin's browser. Each says which.

The device pickers are real, not stubs: `list_devices` reads
`--list-adapters` and `--list-audio` off the SESSION binary, the same
two enumerations the GTK shell shells out for because it links no Vulkan
itself. It is cached for the life of the backend (that call inits Vulkan
and PipeWire) with an explicit Refresh for the headset you just plugged
in, and a failure — a client too old to ship the session binary — leaves
the pickers on Automatic and says so instead of claiming you have no
devices. `_parse_audio_endpoints` is split out and unit-tested with the
malformed lines that must never reach a picker.

Two smaller honesty fixes fall out of building it. A Dropdown can only
display a value that is one of its options, and this store has four
other writers — so a stored value the table doesn't list is carried as
its own entry rather than rendering blank or, worse, showing a different
value than the stream will use. And a stored audio endpoint that isn't
currently connected keeps a "(not connected)" entry, the way the Linux
picker keeps "(not detected)", instead of silently re-pointing the next
stream at the default.

The Settings tab's wrapper deliberately stops being a scroll area: a
SidebarNavigation given an indefinite height to fill collapses its rail,
so the pane hands it the full height and keeps its hands off the
overflow, and the footer inset moves inside the pages.

The docs claimed nine things about this plugin that are no longer true,
and three about the console home that stopped being true when its own
row set grew on 2026-07-31 (4:4:4, echo cancellation, auto-wake and the
library toggle are all there in `screens/settings.rs`). Both corrected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:21:15 +02:00
enricobuehler aa3bcfd0d0 Merge pull request 'feat(client/present): desktop presenter rebuild — intent engine, V-Sync/VRR, display-stat split, settings UI' (#20) from worktree-desktop-presenter into main
apple / swift (push) Successful in 1m18s
arch / build-publish (push) Failing after 1m17s
ci / web (push) Successful in 1m46s
ci / docs-site (push) Successful in 1m30s
deb / build-publish-host (push) Failing after 11s
ci / rust-arm64 (push) Successful in 3m51s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 18s
apple / screenshots (push) Successful in 5m54s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 26s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 7s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
deb / build-publish-client-arm64 (push) Successful in 1m4s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m42s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
android / android (push) Successful in 6m54s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m37s
docker / builders-arm64cross (push) Successful in 17s
deb / build-publish (push) Failing after 4m50s
ci / rust (push) Failing after 7m29s
docker / deploy-docs (push) Successful in 36s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m54s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 59s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m3s
flatpak / build-publish (push) Successful in 6m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m21s
Reviewed-on: #20
2026-08-02 22:06:37 +00:00
enricobuehlerandClaude Opus 5 6b3c582eb1 feat(client/present): use the driver's queue-free vblank mode where it exists
apple / swift (pull_request) Successful in 1m14s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m9s
ci / docs-site (pull_request) Successful in 1m30s
ci / rust-arm64 (pull_request) Successful in 2m51s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m3s
android / android (pull_request) Successful in 5m32s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 1m53s
ci / rust (pull_request) Successful in 7m13s
`VK_PRESENT_MODE_FIFO_LATEST_READY_EXT` is FIFO's tear-free vblank pacing that
presents the LATEST READY image at each refresh and retires the older ones,
instead of draining a queue. That is precisely what the software glass gate
emulates — so where the driver offers it, the driver does the job, and it does
it exactly where the gate matters most: a surface with no MAILBOX gets
newest-wins behaviour back without the app holding frames.

Found by asking the surface what it actually offers rather than trusting a
comment: the previous commit's `surface present modes` line read back
`[MAILBOX, 1000361000, FIFO]` on NVIDIA/Wayland, and 1000361000 is this mode.

The extension postdates the Vulkan headers ash 0.38 is generated from (1.3.281),
so there is no binding — hence the bare number in the log. It is hand-declared
here: mode value, extension name, and
`VkPhysicalDevicePresentModeFifoLatestReadyFeaturesEXT` spliced into the device
pNext chain. One trap worth naming: the SURFACE advertises the mode even with
the extension disabled, and using it on that basis is undefined — so the ladder
only offers it when the device feature actually came back true and we enabled it.

The gate/probe predicate had to split in two, and the distinction is the point:

* `needs_glass_gate()` — FIFO and FIFO_RELAXED only. NOT this mode: gating on
  top of a driver that already retires stale images would hold frames back to
  emulate something the presentation engine is doing, paying the serialisation
  twice, which is the ~27 ms the last commit measured.
* `vblank_locked()` — the whole FIFO family INCLUDING this mode, because it
  still presents on the refresh boundary, so the VRR cadence probe's premise
  ("with VRR off, a present waits for vblank") still holds.

Ranking: MAILBOX first (measured good at 1.4 ms), then LATEST_READY, then plain
FIFO — so a MAILBOX-less surface reaches newest-wins in the driver rather than
in our gate.

MEASURED ON GLASS (.21, NVIDIA 610.43.03, GNOME/Wayland): the extension probe,
feature enable and swapchain creation all succeed with a mode ash has no binding
for. Default ladder selects MAILBOX with `fifo_latest_ready=true`; the VRR ladder
selects `present_mode=1000361000` and measures `display 2.6 ms (pace 0.6 + latch
2.0)` — against 13-28 ms for plain FIFO + gate on the same box. The vblank-locked
path is now MAILBOX-class.

That changes the previous commit's reversal. The VRR ladder was reverted to
opt-in because it led with plain FIFO and cost ~27 ms; led with LATEST_READY it
costs 0.6 ms over MAILBOX. So `allow_vrr` is automatic again WHERE THE DEVICE
OFFERS THE MODE, and stays behind `PUNKTFUNK_VRR_FIFO=1` where it does not — on
those drivers the ladder would fall back to plain FIFO and the regression
returns. Both branches are pinned by tests. This also retires a dead switch: the
"Follow variable refresh rate" row did nothing at all after the reversal, and now
does something real on any driver with the extension.

⚠ Still unverified off this box: whether Windows and Intel drivers expose the
mode at all. Nothing measured here carries over — Windows Vulkan WSI goes through
DXGI, so exposing the enum and mapping it usefully onto flip-model semantics are
separate questions, and Intel is a different vendor stack again. Both facts are
logged unconditionally now (`surface present modes` + `fifo_latest_ready=`), so
one run on any box settles it. The code is safe either way: the mode is only
requested where the device feature enabled, and `allow_vrr` only goes automatic
there — everywhere else the shipped MAILBOX-first behaviour is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:01:56 +02:00
enricobuehlerandClaude Opus 5 e08474d96d fix(client/present): log the surface's actual present modes, and document the VRR opt-in
"AMD's Windows driver offers no MAILBOX" is the premise the FIFO glass gate is
built on, and it has been carried in a code comment rather than measured. Present
modes are a property of the (surface, device) pair — they vary by platform
surface, driver version and fullscreen state — so the only way to settle it is to
read them back from real machines. One unconditional log line makes every field
log answer the question.

First reading, .21 (NVIDIA 610.43.03, GNOME/Wayland):
  surface present modes available=[MAILBOX, 1000361000, FIFO]

Two things fall out. No IMMEDIATE and no FIFO_RELAXED on this surface, which is
why a PUNKTFUNK_PRESENT_MODE=immediate run reported mode=fifo — the pin was not
offered and the ladder fell through; previously that looked like a puzzling
result and is now evidence. And 1000361000 is
VK_PRESENT_MODE_FIFO_LATEST_READY_EXT: FIFO's tear-free vblank pacing that
presents the LATEST READY image instead of draining a queue — the driver-native
version of what the glass gate emulates in software, and a candidate to replace
it wherever the driver exposes it (needs VK_EXT_present_mode_fifo_latest_ready
enabled at device creation, so a work package rather than a tweak).

Also documents PUNKTFUNK_VRR_FIFO, which the previous commit introduced without
a docs entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:01:56 +02:00
enricobuehlerandClaude Opus 5 f422ae3e38 fix(client/present): what the first on-glass session found, including a reversed default
WP6 ran against .21 (CachyOS, RTX 5070 Ti, NVIDIA 610.43.03, GNOME/Wayland,
1080p60 HDMI, VRR provably disabled — `org.gnome.mutter experimental-features`
is empty), host and client on the same box, `VK_KHR_present_wait` available.

Five defects that unit tests and both CI gates had passed over:

1. The latch learner and the VRR probe observed NOTHING. Both derived spacings
   with `windows(2)` inside a single batch, but the run loop drains present-wait
   samples every pass, so a batch is normally ONE stamp. `period_us` read back
   exactly the mode fallback — correct by luck on a 60 Hz panel, wrong the moment
   a mode lies, which is the entire reason PanelGrid exists. The tests fed
   40-stamp batches, a shape the live loop never produces. Spacings are now
   measured against the previous stamp across calls.

2. The VRR reference was circular. It compared spacings against the LEARNED
   period, but the grid cannot be learned from our own presents when the stream
   runs below panel rate — we only ever observe multiples ≥ our frame interval,
   so the learner adopts our own cadence and every delta is on-grid by
   construction. It learned 18-22 ms from a 40-50 fps stream and reported VRR on
   a display with VRR off. The reference is now the DISPLAY MODE's period, which
   is the vblank grid presents actually quantize to.

3. The probe is meaningless outside FIFO. MAILBOX deliberately decouples presents
   from scanout, so its stamps are never grid-quantized: same panel, same minute,
   FIFO read `no` (correct, period 16.4 ms) and MAILBOX read `yes` (wrong).
   Outside a FIFO-family mode the honest answer is Unknown, and that is now what
   it reports.

4. Round evaluation was per-CALL rather than per-sample, so the verdict depended
   on how the caller batched its stamps. Closed inside the sample loop now, with
   a test pinning bulk-vs-one-at-a-time equivalence — the same invariant (1)
   violated, in a second place.

5. `force_latency` was dead code without the `pyrowave` feature: a warning in the
   `--no-default-features` build CI actually ships (the Windows ARM64 leg). The
   gate only ever tested default features; it now tests both.

DESIGN REVERSAL — the VRR FIFO-first ladder is opt-in (`PUNKTFUNK_VRR_FIFO=1`),
no longer default. It shipped default-on for `allow_vrr` + fullscreen, which is
the default configuration. Measured A/B, same box, back to back, reproduced
across three runs: FIFO+engine `display 28.4 ms (pace 11.8 + latch 16.6)` versus
MAILBOX `1.4 ms (0.2 + 1.2)`. Under a compositor the FIFO present's on-glass
confirmation arrives a whole refresh later and the presenter serialises behind
it. The VRR upside is real in principle but UNMEASURED — no VRR panel was
available — and a default that is measurably ~27 ms worse on the hardware we
could test, bought against an unproven win on hardware we could not, is the
wrong way round. A test pins the default to MAILBOX; flip it back when a VRR
panel confirms the win.

NOT measured, and not claimed: the FIFO glass gate's own headline. The standing
queue only forms when the stream rate approaches the panel rate, and an idle
GNOME desktop is damage-driven at 40-50 fps on a 60 Hz panel, so `gated`/`forced`
read 0 in every mode and the mechanism never engaged. The 11-13 ms figure is
still the code's inherited documentation, not a fresh measurement. It needs its
actual target: AMD-on-Windows (no MAILBOX, direct scanout) under load.

Rig caveats recorded rather than smoothed over: host and client shared one GPU,
so absolute latencies are contended and run-to-run variance was large, and it
could not be visually confirmed what the physical screen showed. Mode selection,
the fallback ladder, the VRR verdict and the counter plumbing are robust to
that; absolute numbers are not.

Gates: fmt, clippy -D warnings over the five client crates AND the
`--no-default-features` build (added because defect 5 hid there), 160 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:01:56 +02:00
enricobuehlerandClaude Opus 5 e38e3c44c9 feat(client/present): V-Sync and VRR become real settings, and VRR is measured
WP3 of design/desktop-presentation-rebuild.md. The `vsync` and `allow_vrr`
settings have existed since WP1 but nothing consumed them — the swapchain picked
MAILBOX-or-FIFO once, from an env var, and froze. This makes them mean
something, which is also what unblocks their settings rows (deliberately
withheld from WP5 rather than shipped as dead switches).

Present-mode selection is now a preference ladder, not a constant:

* V-Sync off — IMMEDIATE, then FIFO_RELAXED, then the tear-free modes. Asking
  to tear and silently getting vsync is a lie, so the mode that actually took is
  named in the stats line and a refused preference is logged requested-vs-active.
* V-Sync on + VRR allowed + fullscreen — FIFO first. On a variable-refresh panel
  with direct scanout the FIFO present IS the flip, so the panel follows the
  stream's cadence instead of a fixed grid; MAILBOX would decouple presents from
  scanout and re-quantize to the compositor's clock. This is only safe because
  WP2's glass gate bounds the standing queue that historically made FIFO costly.
* Otherwise — MAILBOX then FIFO, the shipped default, unchanged.

`PUNKTFUNK_PRESENT_MODE` still pins a mode outright and now falls back to the
settings (rather than to mailbox) when the name is unknown.

VRR detection is MEASURED, never queried. No portable query exists — SDL exposes
none, Wayland does not report adaptive-sync state, Windows surfaces nothing
through Vulkan — and the platforms that do answer have been caught lying (see
the Android per-uid refresh-rate finding). The discriminator is quantization: on
a fixed-refresh panel every on-glass instant lands on the vblank grid, so the
spacing between presents is ~k×period for whole k even when the stream runs
slower than the panel (it just picks a larger k); under real VRR the panel
refreshes when we present, so the spacing follows our own cadence and sits off
the grid. `CadenceProbe` folds each delta to its distance from the nearest
multiple of the learned period and takes the median. Tri-state: it stays Unknown
below 24 deltas and after a display change, so `vrr` is reported only when it
has been measured — never inferred from what the display claims.

Also fixes the read-once refresh rate: `native.refresh_hz` was sampled at
startup and never revisited, so dragging the window to another monitor left a
60 Hz-seeded clock pacing a 144 Hz panel. `WindowEvent::DisplayChanged` now
relearns the latch grid, resets the cadence verdict, and clears the served-slot
latch.

Settings rows for both, on all three surfaces (GTK, WinUI, console). The
console's V-Sync row is reachable in Gaming Mode, which is the only editor a
Deck user has.

Gates: punktfunk-rust-ci linux/amd64 — fmt, clippy -D warnings over
pf-client-core, pf-presenter, pf-console-ui, the session binary and the GTK
client, 160 tests (the two new ones cover every ladder and both cadence
regimes, including the case that matters most: a stream slower than a FIXED
panel must still read as fixed). WinUI leg on the Windows runner .133:
clippy=0 tests=0, against a tree proven by content to contain the edit.

⚠ On-glass validation is still owed and is NOT claimed here: every box with a
real display was powered off when this landed, so the VRR ladder and the
detector have been exercised only against synthetic stamps in unit tests.

Rebase follow-up: `20de58a7` landed the same "panel grid can be wrong in both
directions" defect fix on Android and extracted the corrected learner into
`punktfunk_core::phase::PanelGrid` for the iOS and desktop presenters to share.
This clock had the identical bug — it capped the learned period at the display
mode's refresh, and the mode is only a CLAIM, so a display really running slower
than it advertises pinned a grid whose instants never arrive, for the session,
with no way back. Adopted the shared learner rather than carrying a second,
buggier copy; still fed the window's MIN spacing, which preserves the k×period
resistance the cap was actually aimed at while the streak requirement lets a
genuinely slower panel be discovered. New test: seed 120 Hz, real panel 60 Hz,
the clock must climb back out.

Took the same commit's third lesson too: the adaptive margin widened on a
latch over 1.5×period (a number picked here), and now widens on the latch
exceeding one period plus the lead already applied — the slot actually aimed at.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:01:56 +02:00
enricobuehlerandClaude Opus 5 b1ac4d02de feat(client/present): the display stat splits, and the intent reaches the settings UI
WP4 + WP5 of design/desktop-presentation-rebuild.md, on top of the WP1/WP2
engine. The engine shipped with no way to choose it and no way to see what it
cost; this closes both.

WP4 — the display stage splits into `pace` (decoded → present-submit, our own
pipeline) + `latch` (submit → on-glass, the presentation queue and the vblank
wait), off the `submitted_ns` stamp WP2 already carried. That split is what
makes a high `display` self-diagnosing: latch dominating is the vsync floor or
a standing queue, pace dominating is us. A `present:` line joins the Detailed
tier naming the live swapchain mode — the answer to most "why is my latch a
whole refresh" questions, since a MAILBOX request silently lands on FIFO
wherever the driver has no mailbox — plus the engine's counters, rendered only
when they are non-zero so a healthy latency session shows just the mode.

Deviation from the plan: the planned `display_adj` twin is NOT here. It was
specified as `display − latch_p50` for parity with the Apple HUD's shaved
figure, but with a real per-sample `pace` percentile that twin is the same
quantity derived worse (subtracting percentiles). `pace` IS the
Apple-comparable number — Apple subtracts its OS present floor, the latch is
ours — and the user docs now say exactly that.

WP5 — Prioritize + Smoothness buffer on all three surfaces: the GTK dialog (a
new Presentation group on the Display page), the WinUI settings page, and the
console settings screen, which is the ONLY editor reachable in Gaming Mode and
so the one that decides whether Deck users can reach this at all. The buffer
control follows the intent the way echo cancellation follows the mic: hidden on
the desktop shells, dimmed and inert on the console, where a row that vanished
mid-list would shift everything under the cursor.

The V-Sync and VRR rows are deliberately NOT here. Their settings exist and are
profile-routed, but the swapchain does not honour them until WP3, and a toggle
that does nothing is exactly how "Full chroma (4:4:4)" shipped inert on desktop
for three releases after being announced.

Buffer labels carry no millisecond hints (Apple/Android derive them from the
session refresh): under a Native mode the shells do not know the refresh at
settings time, so the captions state the cost as one refresh per frame rather
than a confident wrong number.

Docs: the stats page documents the split and the `present:` line, and stops
claiming Linux/Windows measure to the present instant (untrue since
present_wait); client-settings documents both new rows and drops the stale
claim that the desktop 4:4:4 toggle has no effect (it was wired to
VIDEO_CAP_444); configuration documents PUNKTFUNK_PRESENTER and
PUNKTFUNK_PRESENT_DEBUG.

Gates: punktfunk-rust-ci linux/amd64 — fmt, clippy -D warnings over
pf-client-core, pf-presenter, pf-console-ui, the session binary and the GTK
client, 158 tests. The WinUI leg cannot be reached by any Linux or macOS check,
so it was compiled on the Windows runner .133: clippy -D warnings and tests
both exit 0, against a tree proven by content to contain the edit. ⚠ The first
run there reported a false pass — the script printed its done-marker while the
log carried a test failure (a STATUS_DLL_NOT_FOUND launch failure, ffmpeg's
DLLs missing from PATH); the harness now echoes each phase's exit code so the
verdict is a fact in the log rather than an inference from a marker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:01:38 +02:00
enricobuehlerandClaude Opus 5 5f55fa874a feat(client/present): the desktop presenter gains the Apple/Android intent model
WP1+WP2 of design/desktop-presentation-rebuild.md. The shared Linux/Windows
session client presented arrival-paced with no pacing layer at all: two depth-2
newest-wins hops into a drain-to-newest and an immediate present. That IS the
lowest-latency intent, but it was unnamed, unselectable, and had no alternative
— and on a surface without MAILBOX (AMD's Windows driver offers none, and any
compositor holding images does the same) the swapchain's own FIFO becomes a
standing queue worth a measured 11-13 ms at 60 Hz.

WP1 — the settings cluster, under the keys the Apple client already writes into
the shared profile catalog (present_priority / smooth_buffer / vsync /
allow_vrr): mismatched names would ride SettingsOverlay::extra, carried but
never applied. PresentPriority::resolve mirrors the Android reference exactly
(anything but an explicit "smooth" is latency; a buffer outside 1..=3 becomes
2), so a profile authored on any client means the same thing on all of them.
Only the first two are consumed here; vsync/allow_vrr land in WP3.

WP2 — the engine (present_pace.rs, pure state + arithmetic, 6 tests):
- FrameStore: newest-wins slot, or the smoothing FIFO with preroll-to-capacity,
  drop-oldest overflow, and an underflow that re-arms the preroll (repeat by
  omission) — the Apple/Android semantics, with qDrop/qDry counters.
- LatchClock: the panel grid learned from VK_KHR_present_wait glass stamps,
  min positive spacing capped by the mode refresh (measured, never queried —
  VRR and Android's per-uid refresh lie both punish trusting a reported rate).
  It now also publishes the host-facing LatchGrid, so the phase-lock report and
  the local scheduler cannot disagree about the grid.
- PresentGate: one undisplayed present in flight on FIFO surfaces, with the
  100 ms stale force-open. This is the standing-queue killer, and it is inert
  on MAILBOX/IMMEDIATE and without present timing — where behaviour stays
  byte-for-byte the shipped arrival pacing.

Wiring: glass samples drain every pass (a 1 Hz batch would starve clock and
gate) and the waiter pushes an SDL wake, so a gate reopen never waits out the
event timeout; smoothness serves one frame per latch slot and tightens the
loop's wait to that deadline; the adaptive slot margin starts at 0 and widens
+500 us per missed window toward 2.5 ms (a fixed lead was measured to be pure
display tax). PUNKTFUNK_PRESENTER=arrival disables the whole engine for field
A/B without a rebuild.

PyroWave collapses smoothness to latency for the stream: its plane-ring
retirement accounting assumes the depth-2 newest-wins hand-off, and all-intra
frames make buffering moot anyway.

Gates (punktfunk-rust-ci, linux/amd64, sources touched first so a warm target
cannot print a vacuous Finished): clippy -D warnings across pf-client-core,
pf-presenter and punktfunk-client-session; 80 + 32 tests pass; rustfmt clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:00:50 +02:00
enricobuehlerandClaude Opus 5 8af6e2dd02 feat(decky): the stats overlay gets an off switch in Gaming Mode
Field report: "as of version 0.23 of this plugin, there is no setting to
toggle off the stat overlay." Correct, and it never had one — no commit in
`clients/decky` has ever touched a stats key. Every other client does:
the GTK dialog, the Windows page, the Apple app, and the console's own
settings screen all carry the four-tier picker.

The tier defaults to on. `Settings::default` is `show_stats: true` and
`stats_verbosity: None`, which `Settings::stats_verbosity` resolves to
Normal — so a Deck that has only ever been configured through this panel
streams with the overlay up and no way here to put it down. What escapes
exist are not discoverable: Ctrl+Alt+Shift+S wants a keyboard, and the
three-finger touchscreen tap is documented in `docs/stats`, not on the
glass. The console's picker is reachable (X on console home), but that is
a different shortcut than the one-tap stream this panel launches, and a
user editing stream settings here has no reason to look there.

So the row lands here, last in the section, matching the console's
wording. It writes `stats_verbosity` AND the legacy `show_stats` in the
same pairing `Settings::set_stats_verbosity` keeps, so a client too old
for the tiers still honours an Off chosen here; it reads them back the
way `Settings::stats_verbosity` does, so a pre-tier file — including
every file this plugin wrote before today — shows the Normal the stream
actually runs at.

`set_settings` stops replacing the file and merges onto it instead. This
JSON is shared with the desktop client and the console, and holds many
more keys than this panel models (decoder, GPU, profiles, touch/mouse
model). The panel reads it once when it mounts, so a wholesale write
posts a snapshot that predates anything another editor stored while it
sat open — silently reverting it. That was invisible until 0.23.0:
`9c5af8d7` fixed the GTK shell handing the session a spec built from
`Settings::default()`, and only since then does this file reach a stream
at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 23:53:57 +02:00
enricobuehlerandClaude Fable 5 d839f4c2b6 fix(client/windows): settings stop going stale behind your back, and the log has a door
ci / web (push) Successful in 1m1s
ci / rust-arm64 (push) Successful in 2m35s
ci / docs-site (push) Successful in 2m35s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 5s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 33s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4s
deb / build-publish-client-arm64 (push) Successful in 1m16s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
deb / build-publish (push) Successful in 3m52s
apple / swift (push) Successful in 1m18s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
deb / build-publish-host (push) Successful in 4m11s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m28s
ci / rust (push) Successful in 7m1s
android / android (push) Successful in 7m5s
arch / build-publish (push) Successful in 8m17s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m55s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
docker / builders-arm64cross (push) Successful in 8s
apple / screenshots (push) Successful in 5m42s
docker / deploy-docs (push) Successful in 26s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m25s
flatpak / build-publish (push) Canceled after 9m13s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 9m13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 9m11s
A field reporter's codec setting "changed by itself" between sessions. Nothing writes
the negotiated codec back — what they saw was a stale snapshot. `AppCtx.settings` is
loaded ONCE at process start and the page renders from it, but this process is not the
file's only writer (the spawned session persists its match-window size, the console UI
and Decky save too), so the page showed values another process had already replaced —
until a row was touched and `commit`'s rebase pulled the file in, at which point the
value visibly jumped. The 2026-07-31 rebase fix covered the whole-file writers and
missed two spots: nothing re-based on page ENTRY, and the profile-scope commit arm
cloned the snapshot without reloading, so overlay absorption diffed against stale
globals. Both now re-base on the file.

Two more ways a setting could vanish or cost time:

* An older binary's whole-file save DROPPED a newer client's keys — `Settings` had no
  unknown-key passthrough, unlike `SettingsOverlay`, whose `extra` map already gives
  profiles exactly that contract. Extended to the globals: additive, empty on every
  existing store, and an empty map serializes to nothing so no file churns. (`save()`
  was already temp+rename, so the torn-file → silent-Default reset was closed.)
* "Check the client log" never said WHERE. Settings ▸ About grows an Open log folder
  row (%LOCALAPPDATA%\punktfunk\logs, folder not file so the rotated .old generation
  is in reach), and the failed-spawn banner now names the path.

The 4:4:4 caption said "HEVC only, and only where the host can encode it", which sends
people hunting: the host gate is PyroWave or an NVENC backend. It says so now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 23:49:45 +02:00
enricobuehler 0de161e29b Merge pull request 'feat(clients/input): controllers can stop being forwarded, for couches that hand the pad over another way' (#22) from worktree-gamepad-passthrough-toggle into main
ci / web (push) Successful in 1m10s
apple / swift (push) Successful in 1m22s
ci / docs-site (push) Successful in 2m0s
ci / rust-arm64 (push) Successful in 2m37s
decky / build-publish (push) Successful in 48s
deb / build-publish-client-arm64 (push) Successful in 1m27s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m33s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
deb / build-publish (push) Successful in 4m0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m3s
deb / build-publish-host (push) Successful in 3m51s
docker / builders-arm64cross (push) Successful in 14s
arch / build-publish (push) Failing after 5m22s
docker / deploy-docs (push) Successful in 33s
android / android (push) Successful in 7m22s
ci / rust (push) Failing after 9m12s
flatpak / build-publish (push) Failing after 5m29s
release / apple (push) Successful in 9m14s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 8m12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m34s
apple / screenshots (push) Canceled after 4m33s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 10m21s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 11m8s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 1m31s
Reviewed-on: #22
2026-08-02 21:38:01 +00:00
enricobuehlerandClaude Opus 5 b297542c4d feat(clients/input): controllers can stop being forwarded, for couches that hand the pad over another way
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m4s
apple / swift (pull_request) Successful in 1m21s
apple / screenshots (pull_request) Skipped
ci / web (pull_request) Successful in 1m39s
ci / docs-site (pull_request) Successful in 2m6s
ci / rust-arm64 (pull_request) Successful in 2m50s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 1m55s
android / android (pull_request) Successful in 3m27s
ci / rust (pull_request) Successful in 7m59s
A controller that reaches the host by USB passthrough — VirtualHere and friends, or simply a
pad plugged into the host — arrived there twice: once as the real device, once as the virtual
pad this client built from the same hands. Games read both, so a stick drifts against the
centred second pad and menus take every input twice.

New per-client setting, "Forward controllers", default on (today's behaviour). It is tier-P,
so a profile can decline what another profile forwards.

On Linux and Windows it is deliberately stronger than "send nothing". Opening a controller is
what CLAIMS it — SDL's HIDAPI drivers take the device node — and a claimed device is one a
passthrough tool cannot bind, so with this off the session opens no slot at all and never
enables the Valve HIDAPI drivers. Menu navigation is untouched: the launcher still opens the
active pad, and a session supersedes menu mode whether it forwards or not, so the pad is free
for the whole time a stream is up. The consequence, documented at both the setting and the
chord: the controller escape chord is read off forwarded pads, so it is unavailable there.

The Apple and Android input stacks claim nothing, so those clients keep their slots and their
chords and only gate the wire sends — losing tvOS's only controller way out of a stream would
have been the worse bug. Android does stop its DualSense and Steam Controller 2 USB captures,
which do claim the device.

Surfaces: GTK, WinUI, the console settings screen, Apple's touch and gamepad settings, the
Android touch and gamepad settings, and Decky (which also hides the rows that now have nothing
to act on). Everywhere the "which pad" and "pad type" rows grey out while it is off.

Verified: cargo clippy --all-targets -D warnings + 79 tests on pf-client-core, pf-console-ui,
punktfunk-client-session and punktfunk-client-linux (linux/amd64 container, gate proven
non-vacuous with a planted error); swift build for the Apple clients; gradle compile + 49 unit
tests for Android (likewise proven); tsc for Decky. clients/windows is UNCOMPILED — both
Windows boxes were offline; its edits were reviewed against the helper signatures by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:21:26 +02:00
enricobuehlerandClaude Fable 5 98e040fd01 fix(host/stream): the wire holds the session rate when the display outruns it
apple / swift (push) Successful in 1m20s
ci / rust (push) Successful in 5m0s
android / android (push) Successful in 6m30s
ci / rust-arm64 (push) Successful in 1m50s
apple / screenshots (push) Successful in 5m37s
arch / build-publish (push) Successful in 8m0s
ci / web (push) Successful in 1m28s
ci / docs-site (push) Successful in 1m36s
deb / build-publish-client-arm64 (push) Successful in 2m19s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 18s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 16s
windows-host / package (push) Successful in 13m17s
windows-host / winget-source (push) Skipped
deb / build-publish-host (push) Successful in 5m33s
deb / build-publish (push) Successful in 5m39s
docker / builders-arm64cross (push) Successful in 7s
docker / deploy-docs (push) Successful in 24s
windows-host / canary-manifest (push) Successful in 15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m56s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m18s
PUNKTFUNK_VDISPLAY_HZ_MULT promises extra display refreshes without one
extra frame on the wire, but the frame-driven trigger enforced its pace only
as a per-gap floor: sleep to 0.9×interval, then wake on arrival. A source
that always has a frame pending — the overdriven display under uncapped
content — settled at 0.9-interval spacing, 1.11× the negotiated rate. That
is the field report's 132 fps on a 120 fps session: ten percent more
bitrate, encode and decode for frames a 120 Hz panel can only drop.

A credit bucket (PaceBudget) now pins the long-run average at the pacing
rate: credit accrues at one frame per interval of real elapsed time, capped
at 1.25 frames of post-stall burst, and every submitted frame spends one. A
grab may run early only against banked credit, so the 0.9 floor keeps its
per-gap jitter headroom while the average cannot exceed the rate — and a
source at or below it banks faster than it spends and is never delayed.
Anchoring to real elapsed time also keeps the synchronous-encode overlap the
arrival-anchored floor bought (the owed fraction absorbs a constant encode
tail instead of stacking on top of it), and it cannot fight the phase lock's
submit grid: both agree the period is the interval.

The charge lives under the same guard as the gate — the legacy fixed tick
paces by its own grid, and charging it without ever accruing would bank
unbounded debt that stalls the loop if a rebuild later flips the capturer to
arrival-wait.

Verified on .25: native::stream tests 15/15 (three new PaceBudget tests),
punktfunk-host 369/369, clippy -D warnings clean, fmt clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 21:54:09 +02:00
enricobuehlerandClaude Fable 5 5174a59832 fix(capture/kwin): a hidden cursor leaves the stream — KWin's id-0 meta is the hide
Since the 0.22.0 cursor work (the seat-pointer park + the metadata
composite), a KWin capture-model stream always has a cursor — and it never
went away again: not in game, not in Big Picture, not with a controller in
hand (field report, 2026-08-01). The host blended the arrow forever because
pf-capture deliberately ignores SPA_META_Cursor id 0, and once `visible`
latched true nothing on Linux ever cleared it.

Two producer contracts meet on id 0, and one flag now carries which one a
stream follows. KWin rewrites the cursor meta on EVERY enqueued buffer and
writes id 0 whenever Cursor::isOnOutput says the pointer is not in this
stream — which covers both a globally hidden cursor and a client null-cursor
surface (empty geometry intersects nothing). There id 0 IS the hide, and
honoring it is what lets a game hide the pointer mid-stream. Mutter only
rewrites a buffer's meta when the cursor changed, so recycled buffers carry
stale id-0 regions between damage frames — honoring those flickered the
cursor off between hovers (on-glass round 5), and that path keeps its
last-known-state behavior.

The flag rides from the backend that created the output (correct for
registry-pooled reuse too — a kept display only ever matches its own
backend) through capture_virtual_output into the parser's CursorState. The
portal-monitor path stays on the stale-meta contract: the only thing routed
through it today is Mutter's HDR mirror.

Verified on .25: pf-capture 45/45, punktfunk-host 369/369, clippy
-D warnings clean (pf-capture, punktfunk-host, cursor-probe), fmt clean.
On-glass KDE validation still owed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 21:54:09 +02:00
enricobuehlerandClaude Opus 5 c2a6d30d7b fix(android/decode): a codec input slot the feeder can't fill goes back, and so does the AU
deb / build-publish (push) Failing after 2s
deb / build-publish-host (push) Failing after 2s
deb / build-publish-client-arm64 (push) Failing after 3s
apple / swift (push) Successful in 1m19s
ci / rust (push) Successful in 5m52s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 6s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 5s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 9m12s
release / apple (push) Successful in 9m14s
ci / rust-arm64 (push) Successful in 1m57s
android / android (push) Successful in 11m23s
windows-host / package (push) Successful in 11m26s
windows-host / winget-source (push) Skipped
ci / docs-site (push) Failing after 11m59s
ci / web (push) Failing after 12m2s
flatpak / build-publish (push) Successful in 6m25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m33s
docker / builders-arm64cross (push) Successful in 5s
docker / deploy-docs (push) Successful in 28s
apple / screenshots (push) Successful in 5m54s
windows-host / canary-manifest (push) Successful in 18s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m53s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m1s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m57s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m26s
`AMediaCodec_getInputBuffer` returning null for an index the input-available
callback had just handed us dropped both the slot and the access unit on the
floor. Every sibling path in this loop recycles the slot — the orphan-part
discard and the oversize drop both say so in as many words — because nothing was
written and nothing was queued, so it is still ours. Forgetting it leaks one of
the codec's input buffers per occurrence: we never use it again and the codec
never frees what it never received, so the pipeline runs out of input slots,
`pending_aus` overflows into its drop-oldest arm, and the resulting keyframe storm
reads as a decode fault rather than a bookkeeping one.

The AU went with it, silently — no keyframe request, no freeze gate, unlike every
other loss path here — leaving a hole in the reference chain whose concealment
was free to reach the screen.

Both go back now. `break` rather than `continue`, because a codec that cannot
hand out an input buffer it has just advertised is in no state to be fed the rest
of the parked queue on this pass, and retrying the same index against every
parked AU would burn the whole backlog for nothing; the loop comes round again on
the housekeeping wake within 5 ms if it was transient.

Gates: cargo ndk check green on arm64 and armv7, fmt clean, Android clippy at the
same 4 pre-existing warnings as the base commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 20:15:45 +02:00
enricobuehlerandClaude Opus 5 20de58a78a fix(android/present): the panel grid can be wrong in both directions, and the margin listens to the latch
Three defects in the 0.23.0 timeline presenter, all found while root-causing the
field report that turned out to be the slice wire. None of them is that bug; all
three are real, and the first is the one that would still bite once it is fixed.

The panel-period learner could only ever narrow. It is seeded from the display
mode Kotlin asked for — and `preferredDisplayModeId` is a REQUEST the system may
refuse (Smooth Display off, battery saver, thermal, an OEM governor). Ask for
120 Hz on a panel that stays at 60 and the presenter pins an 8.33 ms grid on a
16.67 ms display with no way back, for the rest of the session: it then aims at
instants that never arrive and releases faster than the panel scans. The learner
moves both ways now, and lives in `punktfunk_core::phase::PanelGrid` where it is
host-testable and where the iOS and desktop presenters can share it. The
asymmetry is kept and made explicit — narrowing is immediate (a finer real grid
is always safe to subdivide onto, and it is the per-uid down-rate case the seed
most often gets wrong), widening needs eight consecutive agreeing observations
and then takes the narrowest of them, because one wide sample is a missed
callback and eight in a row is a display that really did slow down.

The glass budget was a prediction with nothing underneath it. `OnFrameRendered`
already reports what actually reached glass, but the budget never consulted it,
so a wrong grid could hand SurfaceFlinger frames indefinitely: BufferQueue fills,
MediaCodec runs out of output buffers, the decoder stalls, and the no-output
backstop starts begging for keyframes. Releases are now counted against their
confirms and the presenter holds back past six outstanding — loose on purpose,
since the callbacks are allowed to arrive batched and a held frame in the
newest-wins slot is a dropped one. It self-clears when the confirms catch up, and
writes the ledger off after the same 100 ms the stale reopen uses, so a platform
that stops confirming can never wedge the stream. `qWait` and `unconfirmed` join
the 1 Hz pf.present line, which is what would have made this visible from a log.

The adaptive latch margin widened on `paced_drops` — the newest-wins store's own
policy evictions, which happen whenever the stream out-runs the panel and say
nothing about SurfaceFlinger's latch lead. On a healthy device that walked the
margin to its 2.5 ms ceiling and re-imposed the display latency the P2e sweep had
just measured away. It now widens on the measured latch exceeding one panel
period plus the live margin, which is what a missed vsync actually looks like.

Also corrects two doc comments that named `display.refreshRate` as the panel_hz
source; it has been the mode table since the A024 down-rate fix.

Gates: 278 punktfunk-core lib tests (7 new PanelGrid cases incl. the refused-mode
regression), clippy -D warnings and fmt clean, cargo ndk check green on arm64 and
armv7. Android clippy reports the same 4 warnings as the base commit and no new
ones. NOT yet confirmed on glass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 20:15:45 +02:00
enricobuehlerandClaude Opus 5 97b2c01ac1 fix(core/packet): a slice-streamed frame costs its own size, not the whole frame ceiling
The 0.23.0 slice wire flushes a block every MIN_STREAM_BLOCK_SHARDS, so every
ordinary access unit is now opened by a SENTINEL — a header with no totals. The
reassembler sized those frames at `max_frame_bytes`, which the QUIC handshake
clamps to 8-64 MiB. That was survivable while sentinels were rare (the streamed
path emitted one only for an AU exceeding a whole FEC block, ~281 KB); it is not
survivable now that every frame is one.

Two consequences, both measured: each access unit allocated and ZEROED a
multi-megabyte buffer, and the in-flight budget (IN_FLIGHT_BUF_FACTOR x
max_frame_bytes) was spent after ~3 concurrent frames — with production geometry,
12 ordinary AUs in flight lost 9 of them outright, every packet dropped before it
could be placed. On a link with normal reorder that is a permanent loss storm:
frames never complete, the re-anchor gate freezes the picture, and the client begs
for keyframes. Only clients advertising VIDEO_CAP_MULTI_SLICE reach this path —
Android and the Linux/Windows session client; Apple and the Windows in-process
client never did, which is why it read as a platform-specific "video pipeline"
fault in the field.

A sentinel carries no total but does pin its own block's extent: a slice sentinel
by its wire base, a legacy one by its full-K position. Size the buffer to that and
grow as later blocks (or the final block's totals) reveal more. The budget is
re-checked on growth for the same reason it is checked at open.

The same flush also drained `pending` to empty whenever the AU's length was an
exact multiple of the shard payload, leaving `finish_streamed` to seal a final
block of one zero-padded FILLER shard. Its derived base overlapped the block
flushed a moment earlier, retro-validation correctly read that as a lying header,
and the whole AU died — one frame in every 1408 on a 1500-MTU link, ~12 s apart at
120 fps, each costing a freeze and a recovery keyframe. A flush now keeps one
whole shard back, restoring the invariant `StreamedAu::pending` already documented.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 20:15:45 +02:00
enricobuehler 29473d6280 Merge pull request 'fix(client/ios): Escape keeps the pointer captured instead of handing it back to iPadOS' (#19) from worktree-ipad-esc-pointer-relock into main
ci / rust-arm64 (push) Failing after 2s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 24s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 27s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 34s
ci / docs-site (push) Successful in 1m9s
apple / swift (push) Successful in 1m21s
docker / builders-arm64cross (push) Successful in 45s
ci / rust (push) Successful in 4m0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 5m6s
docker / deploy-docs (push) Failing after 9s
ci / web (push) Successful in 6m50s
release / apple (push) Successful in 9m10s
apple / screenshots (push) Successful in 5m53s
Reviewed-on: #19
2026-08-02 17:27:54 +00:00
enricobuehlerandClaude Opus 5 b6acbd096e fix(host/vdisplay): waking the PC stops failing the first session
android / android (push) Failing after 2s
apple / swift (push) Successful in 1m21s
ci / rust-arm64 (push) Successful in 1m49s
ci / web (push) Failing after 1s
ci / docs-site (push) Successful in 1m32s
ci / rust (push) Successful in 6m38s
apple / screenshots (push) Successful in 5m55s
arch / build-publish (push) Successful in 8m58s
deb / build-publish (push) Successful in 6m38s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m11s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3m19s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m9s
deb / build-publish-host (push) Successful in 4m57s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 3m43s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 3m59s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
windows-host / package (push) Successful in 12m4s
windows-host / winget-source (push) Skipped
docker / deploy-docs (push) Successful in 55s
windows-host / canary-manifest (push) Successful in 1m30s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Failing after 10m36s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Failing after 10m55s
deb / build-publish-client-arm64 (push) Failing after 11m12s
docker / builders-arm64cross (push) Skipped
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 11m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m27s
A woken Windows host refused every connection with "pf-vdisplay driver
interface not found", on a box where the driver was installed and running.

Resuming re-enters D0 and re-registers the IddCx control interface while
the rest of the resume storm is still going. A client reconnecting a
second after wake lands inside that gap. `ensure_available` probed
exactly ONCE, so it read the gap as a dead driver and answered a device
that was seconds from ready by disabling and re-enabling it — then gave
the interface 4 s to come back, which a contended post-resume PnP does
not meet. The session failed, and the log blamed a missing install.

The recovery also could not tell whether it had recovered anything. It
ran the whole cycle under `SilentlyContinue` and reported
`(Get-PnpDevice).Status` — the DEVICE's status, not the cycle's outcome —
so a disable that was REFUSED left the adapter untouched, started, and
reading `OK`. That is the reporter's `cycled the adapter device …
status=OK` line: a recovery that never happened, announcing success. And
a refusal is the expected case here, not the exotic one:
reset-pf-vdisplay.ps1 stops the host service first precisely because the
host holds the driver's control device open, a step an in-process cycle
structurally cannot take.

- Distinguish a devnode MID-TRANSITION (interface registered, not started
  yet, or the open refused) from one genuinely ABSENT. Wait the first
  out; only the second earns a reload. `Probe` carries the counts.
- Report what the reload DID, not what the device looks like afterwards:
  every failable step is `-ErrorAction Stop` in a `try`, and
  `pnputil /restart-device` is the fallback for the in-use device that
  `Disable-PnpDevice` refuses. Failure paths re-enable, so a half-cycle
  can never strand the adapter DISABLED.
- Give the interface 15 s to arrive after a reload, not 4 — under a 30 s
  hard ceiling so a permanently wedged devnode still fails predictably.
- Serialize recovery: N sessions racing in after a wake perform ONE
  reload, not N interleaved ones. The lock is taken only where no manager
  lock is held, so the order stays one-way.
- Retire the manager's cached control handle when a reload runs, instead
  of letting the next session discover it via a failed IOCTL.
- Surface the real reason. `ensure_available` returns `Result`, so the
  log names how long it waited, whether a reload ran, and how many
  interface instances were seen in what state — the detail that would
  have identified this from the field report's log alone.

`VdisplayDriver::open` now shares the wait (brief, no reload) instead of
carrying a second, drifted copy of it — that path is also reached by
`hw_cursor_capable` mid-handshake, where a reload would be the wrong
trade for one capability bool.

Windows-gated, so verified with scripts/xcheck.sh (check + clippy -D
warnings, --all-targets) and cargo fmt; on-glass wake test still owed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 13:12:54 +02:00
enricobuehlerandClaude Opus 5 d63e913f52 fix(client/ios): Escape keeps the pointer captured instead of handing it back to iPadOS
ci / web (pull_request) Successful in 1m2s
apple / swift (pull_request) Successful in 1m18s
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m26s
ci / docs-site (pull_request) Successful in 3m46s
ci / rust (pull_request) Successful in 4m8s
iPadOS releases the scene's pointer lock by itself when Escape is pressed — the platform's
built-in "let me out", mirroring the web Pointer Lock API's default unlock gesture. Nothing in
our code does it: a bare Esc never touches `captured`, and it keeps forwarding to the host as
the game key it is. But the lock going away flips the mouse onto the absolute UIKit path and
un-hides the iPadOS cursor, so pressing Esc for an in-game menu silently cost the capture until
the user clicked into the video to win it back.

Esc is a GAME key in a stream, not a request to hand the pointer back to iPadOS, so an unwanted
drop is now re-requested. `syncPointerLock` arms a short, bounded burst (3 attempts over ~0.6 s,
no restart inside 2 s) whenever the lock is wanted, was previously HELD, and is now gone; the
first attempt re-asserts `prefersPointerLocked`, later ones present a real false→true transition
and re-anchor the PointerLockChain. Every deliberate release (⌘⎋, ⌃⌥⇧Q, the Stream menu,
resigning active) clears `captured` first, so `wantsPointerLock` is already false when their drop
is observed and none of them are fought.

The "previously held" half of the condition keeps a scene that never qualifies (Stage Manager,
Split View) from paying for a lock that isn't coming — there, a first grant is still driven by
the chain engage in setCaptured/viewDidAppear exactly as before.

While a re-lock is in flight the local cursor stays hidden and absolute pointer MOTION stays
muted, so the couple of frames it takes read as "Esc did nothing to my mouse" rather than a
cursor that blinks in and out and a host cursor that teleports to the pointer's absolute
position. Buttons still forward (they carry no position), so a click mid-relock isn't swallowed.
The burst clears itself on give-up, so the cursor can never stay hidden on a lock the system
won't grant.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 12:08:31 +02:00
enricobuehlerandClaude Opus 5 362595b20f fix(vdisplay/kwin): the streamed output declares it mirrors nothing, so a stored replicationSource can't clone a panel into the stream
apple / swift (pull_request) Successful in 1m25s
ci / rust-arm64 (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 3m13s
ci / rust (pull_request) Successful in 4m22s
ci / docs-site (pull_request) Failing after 18m2s
ci / web (pull_request) Failing after 18m8s
ci / rust (push) Failing after 3s
ci / rust-arm64 (push) Failing after 5s
deb / build-publish-host (push) Failing after 3s
deb / build-publish-client-arm64 (push) Failing after 3s
apple / swift (push) Successful in 1m22s
ci / docs-site (push) Successful in 1m24s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 31s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 23s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 4m14s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 4m13s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 4m15s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 2s
docker / builders-arm64cross (push) Successful in 21s
android / android (push) Successful in 6m2s
docker / deploy-docs (push) Successful in 1m14s
apple / screenshots (push) Successful in 6m3s
arch / build-publish (push) Successful in 8m11s
deb / build-publish (push) Successful in 5m4s
ci / web (push) Failing after 10m16s
windows-host / package (push) Successful in 11m15s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 17s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 10m25s
KWin stores output configuration per *setup* — the exact set of connected
outputs, matched by EDID/connector — in `kwinoutputconfig.json`, and
`replicationSource` is one of the fields it saves and restores
(`OutputConfigurationStore::storeConfig` / `setupToConfig`). Our virtual output
carries a STABLE name on purpose, so once any setup has an entry making
`Virtual-punktfunk` a mirror of a physical head, KWin re-applies it to OUR
output on every session that reproduces that same monitor set — and only that
set, which is why the failure looks environment-dependent: a field report has
the stream cloning the panel whenever exactly one monitor is live, and behaving
normally the moment the others come back (a different setup key, a different
stored entry).

A mirroring output is not a desktop. KWin's `applyMirroring` overrides its scale
and render offset to the source's, so the stream carries the physical screen's
viewport at the physical screen's size instead of the mode the client
negotiated. The protocol says the rest out loud on `priority`: "an output may
not be in the output order if it's disabled or mirroring another screen" — so
the primary assertion this module works so hard to verify silently stops meaning
anything too.

Nothing we sent ever contradicted the stored value. The topology config enabled
our output, took priority 1 and disabled the others, but never stated the one
property that decides whether the thing is its own screen. Now it does:
`set_replication_source(ours, "")` rides along in the config we already build
(free, idempotent — an empty source is exactly what KWin resolves to "mirrors
nothing"), gated on management v13 where the request appeared, since wayland-rs
does not range-check requests and an out-of-range opcode would kill the
connection.

`extend`/`auto` issue no topology calls by design — the streamed output is meant
to join the desk without rearranging it — but a mirror is not an arrangement, it
is a broken source under every topology. So they get `clear_replication_source`,
which enumerates and applies ONLY when our output really is mirroring.

The device's `replication_source` event is now read, so the state is visible: a
mirrored streamed output names its source in a warn instead of leaving "the
stream just shows my monitor" as something only the reporter can see.

Verified on 192.168.1.25 (Ubuntu, cargo 1.96): `cargo test -p pf-vdisplay` 128
pass (7 in `kwin_output_mgmt`), `cargo clippy -p pf-vdisplay --all-targets
--locked -D warnings` clean, `scripts/xcheck.sh linux` clean, fmt clean. NOT yet
on-glass — no KDE box here reproduces a stored mirror; the reporter's setup is
the real test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 11:55:34 +02:00
enricobuehlerandClaude Fable 5 caa47e28e6 fix(client/decode): AV1 hardware decode stops silently opening libdav1d
apple / swift (push) Successful in 1m27s
ci / rust (push) Successful in 5m28s
android / android (push) Successful in 5m59s
ci / rust-arm64 (push) Successful in 2m24s
ci / web (push) Successful in 1m55s
arch / build-publish (push) Successful in 9m23s
ci / docs-site (push) Successful in 1m45s
deb / build-publish-client-arm64 (push) Successful in 2m23s
windows-host / package (push) Successful in 12m12s
windows-host / winget-source (push) Skipped
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 37s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
release / apple (push) Successful in 12m6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
deb / build-publish (push) Successful in 5m54s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
deb / build-publish-host (push) Successful in 5m19s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m0s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m43s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 27s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m44s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
windows-host / canary-manifest (push) Successful in 24s
docker / deploy-docs (push) Successful in 45s
docker / builders-arm64cross (push) Successful in 8s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m7s
apple / screenshots (push) Successful in 5m51s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m17s
flatpak / build-publish (push) Successful in 5m38s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m7s
avcodec_find_decoder(id) returns the registry's FIRST decoder for the id, and
upstream orders the native av1 decoder LAST on purpose ("hwaccel hooks only,
so prefer external decoders" — allcodecs.c). All three hardware backends
selected by id, so every AV1 session opened libdav1d: a software decoder that
silently ignores hw_device_ctx and never calls get_format. Each frame then
failed the backend's hw-format guard and the session burned the demotion
ladder MID-STREAM — field-logged as 68 Vulkan fails → D3D11VA → 102 fails →
software, ~3 s of black — with "hardware decode active" already printed and
the D3D11 profile/pool probes all green. H.264/HEVC never hit this only
because their native decoders happen to be registered first.

Selection is now by capability: find_hw_decoder walks av_codec_iterate and
takes the first decoder whose avcodec_get_hw_config advertises the backend's
surface via HW_DEVICE_CTX, so a build without a usable hw decoder fails at
OPEN in milliseconds and the ladder runs there — the idiom the D3D11 probes
already follow. Registry order still wins among capable decoders, so
H.264/HEVC select exactly what they always did. The software path keeps the
id lookup on purpose: libdav1d is the fastest CPU AV1, and the native av1
decoder has no software path at all.

Every decode log now carries the selected decoder's name — decoder="av1" vs
decoder="libdav1d" is the whole diagnosis, and no log line said it. The
session log names the WIRE codec and drops the FFmpeg id for PyroWave
(ffmpeg_codec_id's fallthrough claimed codec_id=HEVC for wavelet sessions
that never touch FFmpeg).

The CPU lane also stops passing raw PQ off as a tone-map: software-decoded
frames deliberately never take the HDR10 swapchain, but a PQ stream there was
then shown UNtonemapped (washed out) with no warning — the pq-downgrade warn
keys off the swapchain answer — while the Detailed OSD badge claimed the
"HDR→SDR" tone-map that only the hardware lane's CSC runs. The presenter now
warns once when a PQ CpuFrame arrives, and the badge distinguishes
"HDR→SDR (raw)" (no tone-map pass) from the hardware lane's real "HDR→SDR".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:54:43 +02:00
enricobuehlerandClaude Fable 5 652abeb397 fix(host/audio): an unsatisfiable wiring plan waits for an endpoint change instead of hammering
Field case 2026-08: the display isolate invalidated the only real render
endpoint; the mic held the Steam Streaming Microphone, the Speakers were
blacklisted, and the capture loop re-ran the full wiring pass — three
IPolicyConfig SetDefaultEndpoint writes included — every 2 s for 8+
minutes, retrying a verdict that could never change.

- wiring_plan: a plan with no loopback is a typed structural verdict
  (Wiring::loopback_unsatisfiable + an endpoint-set fingerprint); the
  dead leftover() tier (byte-identical to real_hw()) becomes a real last
  resort that accepts ONLY the Steam Streaming Speakers, flagged
  loopback_last_resort — a known-silent-loopback QUALITY risk, never the
  cable/VoiceMeeter echo CORRECTNESS risks. excluded_from_loopback stays
  untouched (judge_default's mid-stream snap-back semantics).
- wasapi_cap: an unsatisfiable plan errors ONCE per topology with the
  render inventory, per-endpoint rejection reasons and only the remedies
  not already taken, then parks on a cheap enumerate-and-hash poll and
  re-plans the instant the set changes; transient failures get a real
  capped exponential backoff (2 s → 60 s, reset on success or set
  change); a last-resort capture re-plans on any set change and promotes
  the 30 s zero-packet breadcrumb to warn.
- audio_control: the recording default is asserted only when the plan
  changed or the default drifted — set_default_endpoint fires all three
  SetDefaultEndpoint roles unconditionally, so the 2 s loop silently
  stomped any operator recording-device change; the "attach one, or let
  the host install the Steam Streaming pair" warn (already satisfied in
  the field case) is replaced by the same diagnosis.

Verified: 19/19 wiring_plan tests (native rustc --test and the Linux CI
image via docker); both Windows files type-check and clippy clean
against wasapi 0.23.0 / windows 0.62.2 for x86_64-pc-windows-msvc via an
xcheck-style stub harness (the in-tree target check dies in
openh264-sys2's build script on macOS, as scripts/xcheck.sh documents).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:36 +02:00
enricobuehlerandClaude Fable 5 48511d1267 fix(abr): probe throughput is measured over the client's receive interval
The capacity probe divided client-side bytes by the HOST's burst
duration — a window wrong on both edges (base snapshotted before the
burst reached the host, frozen only when the ProbeResult landed, while
the host's clock stops the moment ITS send window closes, before the
switch/kernel queue finishes draining toward the client). On a 1 GbE
link a 2 Gbps burst target "measured" 1266 Mbps and set an 886 Mbps
climb ceiling the link could never carry — permanent for the session,
because set_ceiling never lowers.

The reassembler now stamps probe-scoped counters (bytes, packets,
first/last arrival, monotonic ns) at its FLAG_PROBE routing, so video
around the burst contaminates neither numerator nor denominator; the
throughput divisor is the client's first→last arrival interval, with
the host duration kept as the fallback when fewer than two probe
packets arrived. The user-facing speed test shares the corrected
computation (ProbeOutcome/PunktfunkProbeResult layouts unchanged;
elapsed_ms docs updated to the new semantics).

Two guards ride along:
- PUNKTFUNK_ABR_MAX_MBPS clamps inside set_ceiling — the one funnel
  every learned ceiling passes through — so a user cap binds no matter
  what any probe concludes.
- The controller latches decode_cap_kbps when two CONSECUTIVE backoffs
  carry decode-severe evidence (deep decode excursion or jump-to-live
  flush) at a similar pre-backoff rate, mirroring host_cap_kbps for the
  client decoder: a knee below the link ceiling was a permanent 30-60 s
  sawtooth costing a flush + dropped-frame burst per cycle (1440p120
  HEVC field case, knee ~490 Mbps). One spurious flush never latches;
  the cap re-probes on the CAP_REPROBE_WINDOWS clock, so it lifts when
  the decoder recovers.

Also rights the three stale "3 Gbps" probe-clamp comments (the host
constant has been 10 Gbps since MAX_PROBE_KBPS moved).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:36 +02:00
enricobuehlerandClaude Fable 5 3e649d372e fix(capture/stall): the ETW witness testifies in QPC, so compose-silence stops convicting content
The stall classifier's present witness never worked: the consumer was opened
without PROCESS_TRACE_MODE_RAW_TIMESTAMP, so ProcessTrace converted every
event's TimeStamp to FILETIME regardless of the session's ClientContext=1 —
FILETIME ticks (100 ns since 1601) are ~4 orders of magnitude above QPC, so
every ts <= to_q comparison was false. summary() always printed etw=none,
window_counts() always returned presents=0/queue_adds=0 while present_history
was still true (satisfied by the unfiltered ring), and classify() therefore
convicted EVERY compose-silence hole as CONTENT-SILENCE; FRAME-GENERATION —
the class the program exists to catch — was unreachable. Two comments
asserted the wrong contract ("TimeStamp IS a QPC value"); both now state the
real one: ClientContext selects the session clock, RAW_TIMESTAMP is what
stops the FILETIME conversion on delivery.

Three adjacent defects fixed with it:

- summary() and window_counts() each took their own ring lock and their own
  (Instant::now(), qpc_now()) anchor, with OpenProcess syscalls between the
  two calls — the prose and the verdict could disagree about the same hole.
  Merged into window_report(): one snapshot, one anchor, both halves; the
  summary keeps its 300 ms lead-in, the counts keep the gap-only window, and
  the etw=/etw_presents=/etw_queue_adds= log fields are unchanged.

- present_history/queue_history meant "an event EVER sat in the ring" —
  satisfied by events arriving after the hole, or by a previous session's
  leftovers in the never-cleared static RING. Both flags now mean witness
  LIVENESS: at least one event inside a 5 s LOOKBACK ending at the hole's
  start, i.e. the witness demonstrably worked before the hole opened. The
  ring is cleared when a new session starts, so a dead session's events can
  never pose as the next one's history.

- window_counts() accepted only BltQueueAddEntry (1071) as queue history
  while summary() also took BltQueueCompleteIndirectPresent (1068); either
  proves the queue witness works, so the merged reader takes both.

The windowing math is factored into a pure count_window() (plain i64 tick
arithmetic) with unit tests, and the classify() matrix gains the live-witness
zero-presents case. Conviction thresholds are untouched.

Verified: scripts/xcheck.sh windows clippy (-D warnings, --all-targets) green
for pf-frame/pf-win-display/pf-capture/pf-vdisplay; native cargo check green.
The new Windows-gated tests type-check but need a Windows box to run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:36 +02:00
enricobuehlerandClaude Fable 5 0d004c4680 fix(host/handshake): the 4:4:4 gate names the encoder backend, not the capturer
capture_supports_444 was an encoder-backend fact (direct NVENC or PyroWave)
logged under a capture-ish name — a field report burned real time hunting a
capture problem because of it. The 'encode chroma' line now says
ingest_chain_supports_444, a requested-but-declined session logs WHICH gate
lost, and the console UI's Full chroma explainer names the real requirement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:29 +02:00
enricobuehlerandClaude Fable 5 8d7e273a96 feat(client/present): PUNKTFUNK_PRESENT_MODE gains explicit mailbox and fifo_relaxed arms, and the docs stop guessing
The env knob silently folded 'mailbox' and every typo into the default arm,
FIFO_RELAXED was not reachable at all, and clients/session/README.md claimed
the default is FIFO (it is MAILBOX with a FIFO fallback). An AMD-on-Windows
client always lands on FIFO because that driver offers no MAILBOX — now
documented at the picker and in the docs-site client table, next to the ABR
probe/ceiling knobs a field report went looking for and couldn't find.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:28 +02:00
enricobuehlerandClaude Opus 5 e726542f96 docs(winget): the vhost belongs in unom/infra, not on the box
ci / rust-arm64 (push) Successful in 1m23s
ci / web (push) Successful in 1m42s
ci / rust (push) Successful in 4m45s
ci / docs-site (push) Successful in 1m30s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 13s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 9s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
docker / builders-arm64cross (push) Successful in 6s
docker / deploy-docs (push) Successful in 28s
Step 2 told you to add the Caddy vhost by hand on unom-1. That instruction is
what broke the source: ~/caddy/Caddyfile is a copy that deploy-all.sh rsyncs
over from unom/infra, with no .git there to warn you, so the hand-added block
survived until the 2026-07-31 hardening commit rewrote the file from the repo's
own copy and deleted it.

Point step 2 at unom/infra and record how the failure presents, since it does
not look like an ingress problem from the client side: no vhost means no
certificate for that SNI, so Caddy answers with TLS internal_error (alert 80)
before sending one, and winget surfaces that as
WINHTTP_CALLBACK_STATUS_FLAG_SECURITY_CHANNEL_ERROR / 0x8a15003b.

Also note that port 80 is useless for diagnosing it — Caddy 308s every Host to
https including names it has never heard of — and give the SNI probe that does
work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 10:43:39 +02:00
enricobuehlerandClaude Opus 5 e0427a3bb6 feat(ci/android): a release tag without Play notes fails before it builds
ci / rust (push) Canceled after 5m55s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
android / android (push) Successful in 6m28s
Play does not show an empty "What's new" when the file is missing — it carries
the PREVIOUS release's text onto the new version. So the store listing ends up
describing a build nobody is getting, and nothing surfaces it except reading
the listing. That is the shape of the v0.22.3 notes announcing a feature the
tag never contained, and a soft warning in a log nobody reads does not prevent
it.

The gate runs FIRST in the job, before the ten-minute build: a miss costs a
second and leaves nothing half-published — no build, no assets on the Gitea
release, nothing on Play. It rejects three things: a missing file, a file
byte-identical to another release's (the same bug reached by copy-paste rather
than omission), and an empty or over-500-char one.

Length is checked here as well as in play-upload.py on purpose. The uploader
stays the last line of defence and is the only check android-promote.yml gets,
but it runs at step 9; this catches an unedited TEMPLATE copy at step 1. It
counts CHARACTERS, not bytes — Play's cap is 500 chars and `•` is three bytes
in UTF-8, so a `wc -c` check would have called the 356-char v0.23.0 notes 365
and can reject a legal file.

whatsnew/TEMPLATE.txt gives the file a starting point and says what the gate
does and does not enforce: it cannot tell whether the prose was ever edited, so
a copy that still reads "<The headline change>" ships exactly as written.

Canary stays exempt — no curated notes, and Play reusing text for internal
testers costs nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 10:34:32 +02:00
enricobuehlerandClaude Fable 5 02a5bdb965 fix(gamepad): the virtual DualSense stops demanding a firmware update it cannot take
windows-drivers / probe-and-proto (push) Successful in 53s
apple / swift (push) Successful in 1m25s
ci / rust (push) Failing after 2m11s
windows-drivers / driver-build (push) Successful in 1m44s
ci / rust-arm64 (push) Successful in 1m38s
ci / web (push) Successful in 1m4s
android / android (push) Successful in 5m58s
ci / docs-site (push) Successful in 1m11s
apple / screenshots (push) Successful in 6m6s
arch / build-publish (push) Successful in 9m7s
deb / build-publish-client-arm64 (push) Successful in 1m3s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 15s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 4s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
deb / build-publish (push) Successful in 4m57s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
deb / build-publish-host (push) Successful in 4m52s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / builders-arm64cross (push) Successful in 8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 18s
docker / deploy-docs (push) Successful in 26s
windows-host / package (push) Successful in 11m30s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 14s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m35s
The emulated pad's firmware-info feature report (0x20) advertised update
version 0x0154 — a 2021-era number. PlayStation Accessories compares it
against Sony's latest (0x0630 as of 2026-08) and offers an Update that can
only end in "can't complete the update", since the virtual pad speaks no DFU;
libScePad titles (Stellar Blade) surface the same nag in-game. A real pad
plugged in directly reads up to date, which made the prompt look like
punktfunk corrupting the controller.

The old value was chosen to keep the kernel and SDL on the flag0
COMPATIBLE_VIBRATION convention, but parse_ds_output has since learned the
firmware-≥2.24 COMPATIBLE_VIBRATION2 flag as well, so nothing depends on
looking old anymore. Advertise 0x0999 — above anything Sony has shipped and
comfortably ahead of their ~yearly cadence — instead of chasing their exact
latest, which would resurrect the prompt on every Sony release. Writers that
read the version now use the v2 flag; both conventions land in the same
rumble plane. Bumped in both copies of the blob (host uhid + Windows driver);
the DualSense Edge shares them, and its own versioning (0x0217 latest) sits
below the new value too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 10:24:20 +02:00
enricobuehlerandClaude Opus 5 09b9ee8f53 feat(ci/android): a release tag publishes to Play production, not alpha
ci / rust (push) Successful in 5m15s
ci / rust-arm64 (push) Successful in 2m3s
ci / web (push) Successful in 1m15s
ci / docs-site (push) Successful in 1m43s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 8s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
android / android (push) Successful in 5m55s
docker / builders-arm64cross (push) Successful in 13s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 51s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m10s
docker / deploy-docs (push) Successful in 36s
Production access came through on 2026-08-01. Until now a `vX.Y.Z` tag could
only reach `alpha` and someone had to promote it by hand in the Console; it now
goes to `production` at 100% (`completed`). Canary is unchanged on `internal`,
and its run-number versionCodes always outrank production, so testers keep
getting the newer build.

A tag therefore reaches real users with no further click. What keeps that
honest: the tag is only pushed once every platform is green, and Play reviews
each production release before it ships. Ramping instead is `--status
inProgress --user-fraction 0.2` on the upload step.

Play's "What's new" gets its own file, docs/releases/whatsnew/vX.Y.Z.txt — the
vX.Y.Z.md body is ~34 KB against a 500-char cap, so it cannot be reused. Only
tags have one; canary is a moving target and Play carrying the previous text
over is fine for internal testers. Same freeze rule as the notes: once the tag
exists, the file describes what that versionCode shipped.

android-promote.yml is the lever for everything that is not a fresh tag —
promote a tested build, halt a rollout, or roll production back onto an older
versionCode. It is separate from android.yml because promotion must not
rebuild, and an `if:` on all ten build steps is worse than one small workflow.
dry_run defaults to true, so a mis-typed versionCode validates and deletes the
edit instead of publishing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 09:29:20 +02:00
enricobuehlerandClaude Opus 5 43e3c7b69f feat(ci/android): play-upload can attach release notes and promote a build
Two things it could not do, both needed now that a tag ships to production.

Release notes: it never sent `releaseNotes`, so Play's "What's new" was whatever
the previous release said. It now takes --release-notes-file, and refuses text
over Play's 500-char-per-language cap with the actual count — that check has to
happen before the upload, because the API only rejects it at commit, by which
point the AAB is already on Play.

Promotion: --promote assigns a versionCode that is already on Play instead of
uploading, so what reaches production is the byte-identical artifact the testers
ran. Rebuilding would mint a fresh versionCode from possibly-newer sources and
ship something nobody tested. --promote-from asserts the code really is on that
track (a typo'd versionCode now fails before it touches production) and clears
that track in the SAME edit, so the build is never active on both at once.

--user-fraction comes along because --status inProgress is an API error without
it; it is validated as strictly between 0 and 1 rather than left to Google.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 09:29:04 +02:00
enricobuehler ea469162f9 fix(docs): three doc comments start a markdown list they never meant to
apple / swift (push) Successful in 1m14s
ci / rust-arm64 (push) Successful in 3m31s
ci / web (push) Successful in 1m27s
ci / docs-site (push) Successful in 1m53s
apple / screenshots (push) Successful in 5m57s
ci / rust (push) Successful in 8m9s
android-screenshots / screenshots (push) Successful in 1m54s
android / android (push) Successful in 6m53s
deb / build-publish-host (push) Successful in 5m17s
decky / build-publish (push) Successful in 42s
deb / build-publish-client-arm64 (push) Successful in 1m33s
release / apple (push) Successful in 9m50s
arch / build-publish (push) Successful in 11m34s
deb / build-publish (push) Successful in 11m20s
sbom / sbom (push) Successful in 24s
linux-client-screenshots / screenshots (push) Successful in 5m18s
docker / builders-arm64cross (push) Successful in 7s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 10s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m42s
flatpak / build-publish (push) Successful in 6m44s
docker / deploy-docs (push) Successful in 1m25s
web-screenshots / screenshots (push) Successful in 3m57s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 3m6s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 4m43s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m51s
windows-host / package (push) Successful in 11m36s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Successful in 21s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m38s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m1s
Windows clippy on the v0.23.0 tag: `doc_lazy_continuation` in
crates/pf-client-core/src/audio_wasapi.rs:37. The cause is one line break —
`+ wire cost.` begins a line, so the markdown parser reads `+` as a bullet
marker and the following line becomes a lazy continuation of that list item.

Fixed by reflowing so the `+` is mid-line rather than by taking clippy's
suggested indent: indenting would keep the accidental bullet in the rendered
docs, which is the actual defect. Same treatment for the two siblings a sweep
of every `///` line found, both invisible to the Linux gate for their own
reasons:

- gamestream/audio.rs:237 — `+ libopus;` at line start, on the
  cfg(not(linux/windows)) stub, so only a macOS clippy would ever see it.
- mgmt/tests.rs:1653 — `404.` at line start IS an ordered-list marker
  (CommonMark: 1-9 digits + `.`), and it is behind cfg(test), so only an
  --all-targets run sees it.

This is the [[Windows clippy sees what the Linux gate structurally cannot]]
shape again: audio_wasapi.rs is cfg(windows), so no amount of Linux CI would
have caught it.

Verified: a scanner over every .rs doc comment in the tree now reports zero
line-initial list markers with an unindented continuation; rustfmt clean (it
does not reflow doc comments, so these edits are stable).
2026-08-01 01:30:45 +02:00
enricobuehler 213b353dad docs(release): the 0.23.0 notes link to the docs site that exists
ci / rust (push) Canceled after 1m18s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
Both links pointed at https://punktfunk.unom.io/docs/... — the marketing host,
which 404s (verified live; the docs site serves from docs.punktfunk.unom.io, as
README.md has used throughout). The Updating link was the one a reader following
the one-click update section would actually click.

The echo link additionally pointed at the docs ROOT rather than the page it
names; both now resolve to their real slugs, confirmed against
docs-site/content/docs/{echo,updating}.md and by fetching them (200/200, vs 404
for the old host).

Release body re-synced by body-only PATCH.
2026-08-01 01:25:01 +02:00
enricobuehler 23ec0822d8 docs(release): the 0.23.0 notes stop calling gamescope HDR a handheld feature
ci / web (push) Canceled after 0s
ci / rust (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
Two wording fixes to the shipped v0.23.0 notes, which docs/releases/README.md
explicitly allows after the tag — the file stays authoritative and the announce
step re-syncs from it.

"HDR turns itself on for Steam Deck and other gamescope handhelds" framed a
LINUX HOST change as a device one. What 19392918 actually did is default
PUNKTFUNK_GAMESCOPE_HDR on and get the patched gamescope onto the Linux install
routes (Bazzite + Arch sysext, the nix module, the Deck's on-device build
script) — which covers any host running its games through gamescope: a Bazzite
or SteamOS box in Game Mode, an HTPC, a desktop, not only a handheld. The
lead-in carried the same framing and mattered more, since everything above the
first `##` is what the Discord embed shows.

Also rewords the mic-mute lead-in ("stop the room being heard" read oddly).

No claim changes: same features, same scope, same release. The live release body
needs a re-sync — done via a body-only PATCH rather than an announce dispatch,
since announcing also posts to Discord and publishes the stable update manifest,
neither of which should fire before the fleet is green.
2026-08-01 01:21:54 +02:00
enricobuehler 49bbdcf4ef chore(release): bump workspace version to 0.23.0
audit / bun-audit (plugin-kit) (push) Successful in 30s
audit / bun-audit (sdk) (push) Successful in 31s
audit / bun-audit (web) (push) Successful in 33s
audit / pnpm-audit (push) Successful in 28s
audit / cargo-audit (push) Successful in 1m17s
audit / docs-site-audit (push) Successful in 1m2s
ci / web (push) Successful in 1m46s
ci / docs-site (push) Successful in 1m52s
apple / swift (push) Successful in 1m34s
audit / license-gate (push) Successful in 5m54s
ci / rust-arm64 (push) Successful in 6m26s
android-screenshots / screenshots (push) Successful in 3m2s
android / android (push) Successful in 4m59s
ci / rust (push) Canceled after 18m52s
decky / build-publish (push) Successful in 29s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
deb / build-publish (push) Successful in 5m57s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
deb / build-publish-host (push) Successful in 4m26s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
deb / build-publish-client-arm64 (push) Successful in 2m7s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 41s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 13s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Failing after 12s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 11s
release / apple (push) Successful in 10m31s
windows-host / winget-source (push) Skipped
sbom / sbom (push) Successful in 19s
linux-client-screenshots / screenshots (push) Successful in 3m59s
arch / build-publish (push) Failing after 44s
windows-host / canary-manifest (push) Successful in 20s
windows-host / package (push) Successful in 15m12s
docker / builders-arm64cross (push) Successful in 13s
docker / deploy-docs (push) Successful in 34s
apple / screenshots (push) Canceled after 4m13s
web-screenshots / screenshots (push) Successful in 4m56s
flatpak / build-publish (push) Canceled after 11m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 11m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 11m11s
A minor bump: 133 commits since v0.22.3 across 400-odd files. The wire grew two
negotiated abilities (slice-streamed access units, phase-locked capture); the
Android presenter was rebuilt; the microphone path was rebuilt end to end on
every client; the web console moved from polling to the host's event stream;
gamescope HDR is on by default; HDR and 4:4:4 stopped being mutually exclusive
on Windows; and the one-click update apply that missed the 0.22.3 cut ships here.
The canary base is already 0.23 — release.yml derives it as one minor ahead of
the latest stable tag — so this is the version the canary channel has been
publishing against all along.

Lock touched for the 32 workspace members only, via `cargo update --workspace`
rather than a sed: `wasapi` is itself at 0.23.0 and `rustls` at 0.23.41, so the
version space we are moving into is occupied by third-party crates this time.
Diff against origin/main is versions-only, 32 insertions and 32 deletions;
`cargo metadata --locked` resolves; `cargo fmt --all --check` clean in both the
main and the packaging/windows/drivers workspaces.

Notes at docs/releases/v0.23.0.md, per docs/releases/README.md — authored with
the bump so CI's ensure_release seeds the body at tag creation.
2026-08-01 01:01:16 +02:00
enricobuehler 3c509d48c9 feat(core/abi): report_phase earns its version — C ABI 13 -> 14
`punktfunk_connection_report_phase` (fa822744, coherence tail 1d31e4c5) and the
`PUNKTFUNK_CLIENT_CAP_PHASE_LOCK` mirror const (7cf71dd2) grew the embeddable C
surface without moving ABI_VERSION. Every prior additive entry in that doc list
bumped it — v3's wake_on_lan, v5's next_rumble2, v8's clipboard block, v13's
send_pen — precisely so an embedder can ask punktfunk_abi_version() whether the
function it wants to link is there. Left at 13, the one number that answers that
question said "no report_phase" about a core that has one.

The header was already regenerated with both symbols, so it carried the new
surface under the old number; the regen here changes exactly the #define and its
doc block and nothing else, which also confirms the committed header was
otherwise current.

Additive and capability-gated: the host arms on report receipt, the wire grows
only PhaseReport (0x32) — a control message an old host never reads — and a
strict-prefix append on the 0xCF host-timing tail, so WIRE_VERSION stays 2. No
in-tree caller compares ABI_VERSION against a literal; mgmt/tests.rs asserts
against the symbol.

Verified: cargo build -p punktfunk-core regenerates include/punktfunk_core.h to
exactly this diff; punktfunk-core lib suite 134/134; rustfmt clean. The C ABI
harness cannot run on this Mac (`ld: library 'opus' not found`, the documented
pre-existing local linker gap) — CI's Linux leg is the gate for it.
2026-08-01 00:58:11 +02:00
enricobuehler b8d987b145 docs(release): the v0.22.3 notes stop claiming one-click updating it never shipped
The v0.22.3 tag is `1c836afc`, cut 14:36. The one-click apply work landed on
main between 15:01 and 16:28, and this file was then edited at 17:23 (5790a3e3)
to announce it — three "New"/"Under the hood" claims about a build that does not
contain them. The live release body is still the pre-edit text, so nothing wrong
has been published yet; but `announce.yml` re-asserts this file over the release
on every announce, and 0.22.3 has not been announced. Announcing it would have
published the false version and put its lead-in ("can install it for you where
the platform allows") into the Discord embed.

Verified against the tag rather than the commit graph: `update.available` is in
`1c836afc`, `update.applied` is not, and `mgmt/tests.rs` there literally probes
`/api/v1/update/apply-does-not-exist-yet` while `auth.rs` carries the comment
"today it is only a check". The Updates card itself (b275e6d3, cc015626) IS in
the tag, so that bullet stays; only the apply half goes.

The removed material is not lost — it is in docs/releases/v0.23.0.md, which is
the release that actually ships it.
2026-08-01 00:58:11 +02:00
enricobuehler 2d3f9f8690 Merge remote-tracking branch 'origin/main' into audio/mic-latency-echo
ci / docs-site (push) Successful in 1m28s
ci / web (push) Successful in 2m36s
ci / rust (push) Failing after 2m43s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 8s
deb / build-publish (push) Successful in 3m30s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 56s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 15s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
ci / rust-arm64 (push) Successful in 4m10s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m35s
deb / build-publish-client-arm64 (push) Successful in 3m38s
docker / builders-arm64cross (push) Successful in 5s
docker / deploy-docs (push) Successful in 30s
arch / build-publish (push) Successful in 7m0s
android / android (push) Canceled after 7m27s
apple / swift (push) Canceled after 0s
apple / screenshots (push) Canceled after 0s
deb / build-publish-host (push) Canceled after 5m44s
release / apple (push) Canceled after 0s
flatpak / build-publish (push) Canceled after 3m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 3m11s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 1m40s
windows-host / package (push) Canceled after 7m13s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-01 00:54:20 +02:00
enricobuehler 951bcec650 Merge remote-tracking branch 'origin/main' into audio/mic-latency-echo 2026-08-01 00:47:50 +02:00
enricobuehlerandClaude Fable 5 badda070ef docs(android): the stats-array KDoc counts the doubles it actually returns
ci / rust-arm64 (push) Successful in 1m25s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
ci / web (push) Successful in 2m12s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
ci / docs-site (push) Successful in 2m1s
docker / builders-arm64cross (push) Successful in 8s
docker / deploy-docs (push) Successful in 31s
android / android (push) Canceled after 6m2s
ci / rust (push) Canceled after 5m55s
nativeVideoStats grew to 33 with the decode split and the overflow counter, but
its own KDoc still promised 30 and StatsOverlay still said 26 — a count that
was already two extensions stale before this one. Both now list the full index
set, with the JNI KDoc named as the authoritative one so the next extension has
a single place to update.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:47:04 +02:00
enricobuehlerandClaude Fable 5 46bcfc3041 fix(scripts/windows): the installer-run scripts go back to pure ASCII
CI's guard fired on build-web.ps1: an em-dash in the header comment. The
rule exists because PowerShell 5.1 mis-parses non-UTF-8-locale files, and
the check covers every script the installer can run, comments included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:41:07 +02:00
enricobuehlerandClaude Fable 5 f3a39df7b3 test(host/audio): prove a reopen recovered with a live uplink, not one frame
`reopens_after_push_death` failed about one run in nine, and widening its
timeout did not help — the earlier commit blamed the backoff and was wrong.

The pump drops whatever queued while it was down: audio from before the
device came back is stale, so a fresh instance drains the channel right
after opening. The harness counts `opens` from the START of the open, so
the moment the test sees the counter move, the pump has not reached that
drain yet. The single frame it then sent landed inside the drain window and
was discarded exactly as designed, leaving the test waiting for audio that
was never going to arrive.

So the test now keeps feeding, which is what a real uplink does and what
the drain assumes. The sequence advances each time or the de-jitter reads
the repeats as duplicates and drops them for a second, correct reason.

Production behaviour is unchanged: this was the test asserting something
the pump never promised.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:41:07 +02:00
enricobuehlerandClaude Fable 5 f9c56eaf5c feat(android): Automatic prefers AV1 where the silicon says it should
ci / docs-site (push) Successful in 2m12s
ci / web (push) Successful in 2m36s
ci / rust-arm64 (push) Successful in 3m20s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
android / android (push) Successful in 6m13s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / rust (push) Canceled after 6m49s
docker / builders-arm64cross (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Successful in 3m42s
arch / build-publish (push) Successful in 7m46s
deb / build-publish (push) Successful in 5m58s
deb / build-publish-host (push) Successful in 5m58s
apple / swift (push) Successful in 6m9s
apple / screenshots (push) Canceled after 3m57s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 6m53s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 6m21s
windows-host / package (push) Canceled after 0s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
The P3 format A/B (NP3 ↔ RTX 4090, identical conditions) measured AV1 ~1.2 ms
faster end-to-end than HEVC with slightly better codec-pure decode time. Under
"Automatic" the client now sends AV1 as its soft preference when this device
hardware-decodes it (the advertised AV1 bit is already gated on a real,
non-blocked hardware decoder) AND it lacks FEATURE_PartialFrame — a
partial-frame device keeps HEVC, whose slice-progressive overlap AV1 cannot
ride (no slices, the chunked poll never arms). The host honors the preference
only inside its probed shared codec set, so an AV1-less encoder still resolves
HEVC, and an explicit user choice wins unchanged. The codec picker caption
mirrors the same rule so "Automatic" says what it does on this device.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 b69ef02f4d fix(android): a cold-start connect no longer loses HDR or the native mode
A punktfunk:// deep link can reach the connect before the activity is attached
to its display; context.display then throws and the display probes silently
fell to their worst answers — displaySupportsHdr advertised SDR (the whole
session pinned to 8-bit BT.709) and nativeDisplayMode fell back to 1080p60.
Seen live on the NP3: one cold connect advertised hdr=false, the warm retry
true, nothing in the log either way.

Both probes now share probeDisplay: the context display when attached, else
DisplayManager DEFAULT_DISPLAY — which IS the panel on phones and TVs; the
activity-display distinction only matters on multi-display setups, where the
attached path still wins whenever available. Each fallback leg logs itself, so
a downgraded session can never again be silent about why.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 4d45a96ff9 feat(encode/windows): sub-frame readback defaults on where the GPU supports it
Linux parity, validated by the .173 on-glass A/B (no regression; the win goes
to clients that actually consume slice-progressive parts): the caps probe now
reads NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK and seeds resolve_subframe with it
instead of a hard false, so PUNKTFUNK_NVENC_SUBFRAME becomes the tri-state
escape it already is on Linux, and the split×sub-frame arbitration hears the
real forced flag for its log severity.

The A/B also caught the default path opening every session with a WARN: the
submit-time idr_hint missed that NVENC emits the session-opening frame as an
IDR regardless of pic flags, so frame 1's early chunks went out unflagged and
the divergence check fired at every start. The hint now carries the Linux
twin's `opening` term.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 849baea881 feat(android): the stream re-votes its refresh rate and touches keep their curvature
surfaceChanged re-asserts the frame-rate vote (FIXED_SOURCE; ALWAYS only on the
TV low-latency path, mirroring the native hint) — a buffer-geometry change on
some OEM builds silently drops the 120 Hz pin mid-stream. Touch passthrough and
direct-pointer moves forward the MotionEvent historical samples before the
current point, so a fast swipe lands with its real shape; the trackpad path
keeps summed deltas on purpose — its acceleration curve is tuned for per-frame
dt and historicals would change the feel, not the sum.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 c767a904d2 feat(android): the decode stage answers where its time goes, HUD on or off
P3 decode science: every AU is stamped as its last piece enters the codec, so
the decode stage splits into feed (received→queued: hand-off + input-slot wait)
and codec (queued→decoded: the decoder alone — a slice head start would show
here). The split + an always-on capture→decoded e2e ride the 1 Hz pf-present
line, so a wireless HUD-off A/B reads everything from logcat; the HUD equation
gains the split (indices 30/31), the skipped counter tells benign newest-wins
pacing from parked-AU overflow (32), and a −2-refresh Apple-HUD-equivalent twin
makes iPhone comparisons honest (Apple shaves its OS floor; Android shows raw).

Connect now logs the per-mime decoder picks + FEATURE_PartialFrame verdicts
(tag pf.caps) — on the NP3 all three c2.qti low-latency decoders say no, so
parts delivery never arms and P2d is inert there; a debug.punktfunk.force_parts
sysprop overrides the probe for the on-glass question the API cannot answer.
Forced on glass: c2.qti accepts PARTIAL_FRAME pieces without erroring but only
assembles them — codec time unchanged, so the overlap is dead on SM8735 either
way.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Opus 5 0985726415 fix(host,web): an empty update channel stops looking like a broken host
android / android (push) Canceled after 36s
apple / swift (push) Canceled after 0s
apple / screenshots (push) Canceled after 0s
arch / build-publish (push) Canceled after 37s
ci / rust (push) Canceled after 37s
ci / rust-arm64 (push) Canceled after 36s
ci / web (push) Canceled after 34s
ci / docs-site (push) Canceled after 34s
deb / build-publish (push) Canceled after 19s
deb / build-publish-host (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
windows-host / canary-manifest (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 0s
windows-host / package (push) Canceled after 33s
windows-host / winget-source (push) Canceled after 0s
flatpak / build-publish (push) Successful in 5m43s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m9s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 2m19s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
A channel nobody has published to answers `manifest.json` with a 404, and the
check reported that the same way it reports a dead registry or a bad signature:
"Last check failed: feed returned HTTP 404". Every host on the stable channel
shows it today, because the stable manifest only publishes when someone
dispatches `announce` for a release tag — so the first thing an operator sees
from the new Updates card is a red failure caused by nothing being wrong.

The shared checker now distinguishes the two. `feed::fetch_manifest_blocking`
returns a typed `FeedError` instead of a string, and only a 404 on the manifest
ITSELF becomes `NotPublished` — a 404 on the detached signature still fails
loudly, because that is the half-published pair the manifest-then-signature
upload order can produce, and it must stay fail-closed.

The host carries that through as `UpdateStatus.not_published`, mutually
exclusive with `last_error`. It is benign only while no manifest has ever been
seen for the channel: once a check has succeeded, the same 404 means the feed
LOST a document it used to serve, which stays an error. The console then shows a
plain sentence naming the channel instead of the failure banner, and "None
published yet" rather than "Not checked yet".

The Linux client makes the same distinction but deliberately NOT the same
choice: `--check-update` keeps exiting 1 and keeps `error` set, because its
consumer is a shell script and an empty channel is the absence of evidence that
this build is current — not a confirmation that it is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:37:56 +02:00
enricobuehler 23f1debe69 Merge remote-tracking branch 'origin/main' into audio/mic-latency-echo 2026-08-01 00:21:24 +02:00
enricobuehlerandClaude Opus 5 ff190e9825 fix(web): the logs filter row stops touching the card's edge on desktop
windows-host / package (push) Failing after 28s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
ci / web (push) Successful in 2m13s
ci / docs-site (push) Successful in 2m19s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 5s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 19s
ci / rust-arm64 (push) Successful in 3m30s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 34s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
deb / build-publish (push) Successful in 3m43s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 18s
deb / build-publish-host (push) Successful in 4m4s
docker / builders-arm64cross (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 57s
docker / deploy-docs (push) Successful in 27s
deb / build-publish-client-arm64 (push) Successful in 3m1s
ci / rust (push) Successful in 6m29s
arch / build-publish (push) Successful in 7m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 15m42s
Found on glass. The logs card has no CardHeader, so it puts the top padding back
itself — but only at one breakpoint. `CardContent` is `p-4 pt-0 sm:p-6 sm:pt-0`,
and tailwind-merge resolves conflicts only within the same variant: a bare
`pt-6` cancels `pt-0` and leaves `sm:pt-0` standing. Measured on .173: 24px of
top padding at 420px wide, 0px at 1280px, with the level filters and the search
box sitting flush against the card border.

It is the same trap `components/ui/card.tsx` documents for `p-0` — a variant
cancelling its unprefixed counterpart and nothing else — just in the other
direction, so the note now points both ways. This card was the only offender.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 f66de3eba4 fix(web): close the last two high-severity items from the closeout audit
**The streamed-screen pin could still be clobbered by three other write paths.**
Deferring it to the server's value on Save fixed the reported sequence but not
the general case: the draft is only re-seeded while it is CLEAN, so once there is
an unsaved edit its `capture_monitor` is frozen at whatever it was before the
operator used the picker — and `applyAxis` (which spreads the last saved policy),
the built-in preset switch and the custom-preset apply all put that stale value
back. Every write path reads `serverCaptureMonitor()` now; no path spreads the
draft's copy.

**The session⇄game grace input had no accessible name.** That card has its own
`Field` and only DisplayCard's was fixed, so the number input was still announced
as an unnamed spin button. Same treatment: `htmlFor`/`id` for the single control,
`fieldset`/`legend` for the two button groups. Verified in a browser — zero
inputs without an accessible name across the Displays page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 f2e1b9872c fix(web): four "fixes" from this branch that did not actually fix anything
A verification pass re-read every finding from the original sweep against the
code on this branch rather than against the commit messages. It found that four
of them were still broken, two because the edit I made was inert. Commit
messages claim; code decides.

- **The Storybook typecheck was never on.** `tsconfig.json` listed `.storybook`
  as a bare directory name, and tsc silently skips dot-prefixed directories in
  that form — so the entry typechecked nothing at all. Proved it by planting
  `export const __probe: string = 1` in `.storybook/preview.tsx` and watching
  `bun run lint` pass. `.storybook/**/*` is what actually pulls it in; the same
  probe now fails as it should.

- **The Moonlight stale-PIN reset was a no-op.** `submit.reset()` sat at the top
  of `onSubmit`, immediately before `submit.mutate(...)` — which moves the status
  to pending in the same update, so it cleared a flag that was already changing.
  The green "PIN sent" note therefore still greeted the next pairing attempt over
  an empty PIN box. It now resets on the transition that actually matters:
  `pin_pending` going false → true.

- **The session⇄game controls had the enforcement flag inverted**, and I never
  touched it. `enforced.length === 0 || …` reads an EMPTY list as "this build
  enforces everything", when the contract says the opposite in as many words:
  "Empty on a platform with no launch path (macOS), so the console can say so
  instead of offering a switch that does nothing". On exactly the platform the
  flag exists for, every control stayed live and reported success for an axis the
  host would never act on. Absent still means "assume it acts" — that is the
  compatible reading for an older host, and a different case from present-empty.

- **Logout stopped revoking after a restart.** The epoch was a module-level
  counter starting at 1, so it revoked within one process run and then reset —
  and since the seal key derives from the stable mgmt token, a cookie captured
  before a restart unsealed fine and was accepted again for the rest of its
  7-day TTL. One service restart undid the whole fix. It persists next to the
  host's config now. Verified: log out, restart the console, the captured cookie
  still 401s, a fresh login still works.

Two more the pass rated as partial, both worth closing:

- The plugin-UI response filter was a denylist of four header names, so
  `Clear-Site-Data` sailed through — a plugin error page could wipe `pf_session`
  and sign the operator out of the console, on our own origin, because the iframe
  is same-origin by design. It is an allowlist now; a plugin-supplied CSP,
  `X-Frame-Options` or CORS header no longer speaks for us either.

- A half-configured TLS setup now refuses to start instead of logging a warning
  and serving anyway. Neither shape can work — one path missing puts the login
  password on the LAN in the clear, and PUNKTFUNK_UI_SECURE without TLS marks the
  cookie Secure so the browser drops it and login can never stick. Exiting with a
  reason beats a console that looks fine and is not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 dc57aa653c feat(web): the console can say what just happened, and hand a phone the way in
**Recent activity.** The console could describe the present — a status snapshot —
but never the recent past. A client that connected and left while you were on
another page left no trace anywhere you could look, and the host's own log is a
developer artifact rather than a narrative. The event stream was already open for
cache invalidation, so a feed costs one ring buffer next to it: every frame is
recorded, labelled per kind, and rendered newest-first on the dashboard.

Deliberately in-memory and bounded to 200. It starts empty on a page load and
fills as things happen, which is the honest shape for a live tail — an audit
trail would need the host to keep one, and pretending otherwise would be worse
than not having it.

**Connect a device.** The console knew the host's address and identity all along
and never offered either in a form you could hand to a phone: pairing meant
reading an IP off the Host page and retyping it on a couch. There is a card now
with the address and a `punktfunk://connect/<uniqueid>` deep link, both
copyable — the link is the shipped client grammar
(clients/shared/deeplink-vectors.json), so an installed client opens straight
onto this host. No QR: rendering one needs an encoder we do not bundle, and a
wrong QR is worse than none.

**Installable.** A web manifest and the theme/apple meta tags, so the console can
live on a phone's home screen — which is where it is used from as often as from a
desk. No service worker on purpose: an offline shell for a console whose every
screen is live host state would only ever show stale numbers convincingly. The
manifest is reachable without a session (install needs it, and it says nothing
the login page doesn't); /api stays gated, verified.

Verified in a browser: three host-emitted events appear in the feed with the
right labels, the deep link renders and copies as
`punktfunk://connect/abc123`, and the manifest serves 200 as
application/manifest+json while /api/v1/host still answers 401.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 b8f603c8a1 feat(web): the numbers behind "it's slow to start" and a way to clean up after a plugin
Three things the host already reports and the console never showed.

**Stream diagnostics.** `RuntimeStatus.stream` has carried the session bring-up
time, the last mid-stream resize cost, the client's FEC parity floor and the
packet size for as long as the endpoint has existed, and the dashboard showed
none of them. So "it takes ages to start" and "it hitches when I change
resolution" had no number attached anywhere in the console — you had to take a
stats capture to see a value the status endpoint was already returning. The two
timings are native-plane only and null until the first frame lands, so each
appears once it means something.

**Loss and FEC recovery while the capture runs.** The health chart existed but
only in the saved-recording view, which is backwards: dropped frames and FEC
recovery are what you watch a live capture for. It now sits under the latency and
throughput charts on the live card, keeping the GameStream caveat (only `frames`
is instrumented on that plane).

**Provider-owned library entries.** A plugin can sync entries into the library,
and the host then refuses to edit or delete them one at a time — correct, and
completely opaque once the plugin is gone: its games sit in the library with no
console-side way to remove them. `DELETE /library/provider/{provider}` is the
documented clean-uninstall path and nothing called it. There is a card now that
names each provider, counts what it owns, filters the grid to it, and removes its
entries in one go.

Also: the dashboard's PIN tile really does say "Waiting"/"None" now. The earlier
commit added the strings but the edit that was supposed to use them silently did
not apply, so the tile still rendered a bare "●". Caught by auditing every
message key for a call site — the other 543 are wired.

Verified in a browser: the providers card shows, counts, and filtering hides
non-provider entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 0751265105 fix(web): editing a library entry warns before it wipes what the console cannot see
`PUT /library/custom/{id}` replaces the whole entry — the host assigns
`slot.prep = input.prep` and `slot.detect = input.detect` outright
(library/custom.rs). But `GET /library` returns a `GameEntry`, which carries
neither field, so the console builds its payload from a read model that has
already lost them. Editing a title to fix a typo silently cleared any prep/undo
commands and detection hints the entry had.

The console cannot round-trip what the read API will not tell it, so this is a
warning, not a fix: the edit form now says plainly that saving replaces the entry
and that anything configured outside the console will be cleared. The actual fix
is host-side — expose `detect` and `prep` on the library read model — and is
noted in the code where it belongs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 b9b0df349d fix(web): the charts stop lying about time, and logging out logs you out
The stats charts drew every sample as an evenly-spaced slot, because recharts
defaults to a category axis. A capture that idled for two minutes rendered that
gap as a single step — so the one view you open specifically to find where the
time went was the view least able to show it. All three charts use a numeric time
axis now, so the spacing is the elapsed time.

They also joined samples across a session boundary into one continuous line,
implying a continuity that never existed: the stream stopped and somebody else
started a new one. A capture is split at each `session_id` change now. And the
live card plotted the whole capture-so-far every 2 s, re-serialising and
re-plotting an unbounded series for a capture left running all evening; it plots
a bounded tail and says so, with the full series still in the saved recording.

Logging out only deleted the browser's copy of the cookie. The session is
stateless, so a value captured beforehand — a shared machine, a shell history, a
TLS-inspecting proxy — stayed valid for its full 7-day TTL and there was nothing
the operator could do about it. Sessions carry an epoch now and logging out bumps
it, which invalidates every cookie issued so far. Verified end to end: a captured
cookie works, survives nothing across a logout, and a fresh login still works.

The rest:

- The update card could not show its own timeout warning. It was suppressed by a
  `job` field read from the last snapshot — which, when the host has gone away
  mid-job, is exactly the case the warning exists for. Nothing ever cleared the
  applying state either, so the card waited forever with no way out; there is a
  button now. "Check now" also surfaces the host's 429 instead of looking dead.
- A running install survives a reload: the job id lived only in component state,
  so refreshing lost sight of an install that was still running while the Install
  buttons stayed armed against a host that answers 409. The host keeps the list —
  ask it.
- An all-sources-failed catalog said "no plugins available". That is a successful
  request carrying nothing, not an empty store; it names the sources that failed.
- The Installed tab rendered "vundefined" for a plugin with no recorded version
  (nullable in the contract, typed required here).
- The Displays "In effect" badges were computed from the local draft, so they
  restated the operator's unsaved edits back to them as though the host had
  adopted them. They read the API's `effective` now. A failed background poll no
  longer replaces a form someone is editing, and leaving the page with unsaved
  edits prompts — the old `beforeunload` guard never fired for in-app navigation,
  which is how you actually leave.
- Enter or Space on a preset's rename/update/delete icon applied the preset
  instead of running the action: keydown bubbled to the card.
- Dates follow the console's locale, not the browser's. The dashboard's
  PIN-pending tile says "Waiting"/"None" instead of "●"/"—".
- The Bun entry warns when TLS is half-configured, or when PUNKTFUNK_UI_SECURE
  is set without it — both of which silently break login.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 de17ceb8f8 fix(web): a newly installed plugin shows up on its own, and the display form can be used without a mouse
Installing a plugin left the sidebar unchanged until a reload. The reason is
timing, not caching: the host restarts the scripting runner AFTER the job reports
done, and the plugin only registers its UI once that comes back — several seconds
later, by which point the one-shot invalidation had already run and found the old
list. The nav then waited out the 30 s idle poll, which in practice meant "until
I reloaded". Anything that changes the installed set now switches the directory
to a 2 s poll for a minute, so the entry lands about a second after the plugin
actually comes up. Measured end to end in a browser: 29 s → 7 s, with the plugin
registering at 6 s.

The plugin entries also never animated. They are rendered outside the `motion.nav`
that carries the variants and the stagger, so they inherited neither and simply
appeared — most visibly in exactly the case above, where one shows up in a nav
that is already on screen. They get their own animation container now, matching
the main nav. (A motion-wrapped div around the link, not `motion(Link)`, which
erases TanStack's typed `params`.)

The accessibility pass on the display form, where the console's densest controls
live:

- The Custom block's numeric inputs had a `<label>` with no `htmlFor` next to an
  `<input>` with no `id`, which labels nothing at all — a screen reader announced
  them as unnamed spin buttons. Single controls are paired properly now; the
  button groups became real `<fieldset>`/`<legend>`, which is what they are.
- Every option group signalled its active choice with fill colour alone. They
  carry `aria-pressed` now, so the state is available to assistive tech and not
  only to people who can compare two button variants.
- `QueryState`'s error branch is a live region, so a query that fails announces
  the failure instead of silently swapping one region for another.
- Motion honours `prefers-reduced-motion` instead of overriding it.
- `<html lang>` follows the locale instead of claiming "en" while the app renders
  German. Verified: switching to de flips the attribute.
- "Close menu", "Language" and "Loading" went through the message catalogue.

Also: ten dead message keys removed (a whole removed Clients page and the old
Settings token field), and the README no longer tells operators to set the
management token under "Settings → API token" — that field is gone and the token
has been server-side only for some time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 9e505aba41 fix(web): the console stops swallowing the host's answer when it says no
The host writes genuinely useful refusals — "entry is owned by provider `x`,
update it through its reconcile" — and a dozen call sites threw them away. The
pattern was always one of two: a mutation whose `error` nothing rendered, or an
`await mutateAsync(...)` with no catch, which additionally produced an unhandled
rejection. Either way the operator clicked, nothing visible happened, and the
thing they asked for silently hadn't.

Fixed at each site, with the host's own message shown where there is one:

- Adding or editing a library entry kept the form open and said why, instead of
  closing it as if it had saved and taking the typing with it. Deleting one
  reports the refusal rather than leaving the card sitting there.
- The GPU preference, capture start/stop, recording delete and download, and the
  dashboard's stop-session / request-keyframe / end-game all report failure. The
  failed capture STOP is the one that mattered most: it is "stop & save", so a
  swallowed error meant minutes of recording vanished with nothing on screen.
- The recordings Download had a comment claiming the detail view surfaces its
  errors. It only does that for the selected row, and Download is on every row.

Two related fixes in the same area:

- `apiFetch` no longer navigates to /login synchronously from inside whichever
  call noticed a 401 — very often a background poll the user never started.
  Tearing the page down mid-render took unsaved editing state with it, which the
  Displays page explicitly models. It defers a beat and coalesces, so a burst of
  parallel 401s schedules one navigation.
- The plugin liveness probe treated the auth gate's 302 → /login → 200 HTML as a
  healthy plugin, and rendered the console's own login page inside the plugin's
  iframe. It also gave up permanently on the first failed probe, so the runner
  restart at the end of every install threw away whatever was open in another
  plugin. It rejects the redirect and keeps probing on a slower beat while down.

`apiErrorMessage` moves out of the display card into src/lib/errors.ts, since
half the console needs it now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 4a5d4b0a71 feat(web): the console follows the host's events instead of asking ten times a minute
The host has published every lifecycle transition on GET /api/v1/events since the
API existed — client connect/disconnect, session and stream start/end, pairing
decisions, display create/release, library, store and plugin changes — and
nothing consumed a byte of it. The console instead polled ten endpoints on 1-5 s
timers, so a change was up to 5 s stale and two pages could disagree while you
looked at them. The Library page polled not at all: install a game in Steam and
it never appeared until a full reload.

The console now subscribes once and invalidates exactly the queries an event
affects. Events never carry data into the cache — they only say "this is stale" —
so an unknown future kind costs nothing and a missed event degrades to the
polling that is still there underneath, now at a slow safety-net interval. The
fast ticks that remain are the ones events cannot express: the live stream
numbers while streaming, and a lingering display's teardown countdown.

Four things had to be true for this to work, and none of them were. Each was
found by measuring, not by reading:

- Nitro's `localFetch` accumulates the response and only builds it when the
  handler returns, so nothing streams through the deployed Bun server. Three
  frames sent a second apart arrived together, three seconds late, when the
  upstream closed — and an SSE stream never closes, so nothing would ever have
  arrived. /api/v1/events gets its own route that hands back a web Response
  wrapping the upstream stream, which passes straight through.
- Hydration mounts the app shell and discards it ~15 ms later. A subscription
  owned by that effect opened, closed, and never came back. It is a refcounted
  module singleton now, with a grace period so a remount re-attaches instead of
  reconnecting.
- `getRouter()` runs more than once in the browser, and each call built its own
  QueryClient. The subscription held the first, the live pages read the second,
  and every invalidation went to a cache nobody was reading. One client per
  browser session; the server still gets a fresh one per request, which it must.
- `invalidateQueries` only refetches queries that currently have an observer.
  An event means the HOST changed, so every cached copy is wrong whether or not
  something is watching it.

Two features fall out of the same work:

- **Automation** — a page for GET/PUT /api/v1/hooks. The host has run these
  hooks all along and the console never showed them, so the only way to see what
  your machine does when a stream starts was to open the config file. Writing one
  means writing a shell command the host will execute, so saving re-asks for the
  console password, like an update or an unreviewed install.
- The Host page warns when another Moonlight-compatible server (Sunshine,
  Apollo) is running on the same machine. The host has detected this at startup
  for ages and reported it in /local/summary; nothing surfaced it. It is the most
  common reason a host looks installed and working but no client can reach it.

Verified in a real browser against a mock host: three events drive three
refetches of a query with no polling timer, the conflicts card names the
intruder, the hook list and its dialog render, and the console reports no errors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 55e01c1460 fix(web): stop shipping 7 MB of sound the console cannot play
`@unom/ui/button` reaches `sound/defaults.js`, which resolves two game-UI sprite
sheets with `new URL(…, import.meta.url)` at module scope — a 4.8 MB .wav and a
2.2 MB .mp3. Vite emitted both into the build, so they rode into the Windows
installer and the .deb. The console never mounts UnomProviders, so no player is
bundled and not one byte of it could ever be played. A build-time rewrite of
those two expressions takes the asset payload from 8.2 MB to 1.5 MB; the login
page and the button chunk are unchanged. Deleting the plugin is the whole revert
if the console ever wants click sounds.

Also:

- `bun run dev` forwards the management bearer, so developing against a real
  host stops 401ing into a /login bounce that dev has no gate to satisfy.
- `check-i18n` runs after `build`, not only inside `codegen`. It exists to stop a
  zero-message console shipping, and the CI job and the installer build both
  install with `--ignore-scripts`, so it had never once run where it mattered.
- Bun's idle timeout goes from its 10 s default to 120 s. The host sends SSE
  keep-alives every 15 s, so anything long-lived proxied through the console was
  cut by us first — which the event stream is about to depend on.
- The typecheck covers the Storybook config and preview.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 e30d94573a fix(web): the logs come back after a host restart, and eight more that quietly lied
The Logs page died permanently every time the host restarted — which the
console's own update flow does. The host's log ring restarts at seq 1 while the
page's cursor stays where it got to, and `GET /logs?after=8000` against a fresh
ring is not an error, it is an empty page forever: no error, no dropped badge,
stale lines on screen, and nothing short of a full reload to get out. A restart
always breaks the poll first, so a failed poll now triggers a re-read from the
start of the ring, and a page whose newest entry is older than what we hold is
recognised as the sequence having restarted.

Follow mode also stopped following at exactly the wrong moment. The autoscroll
effect was keyed on the rendered row count, which pins at the 1000-row DOM cap —
so once the log got busy enough to matter, the effect never re-ran again. It is
keyed on the newest rendered seq now. And pausing now actually pauses: stopping
the interval left React Query's focus/reconnect refetches landing, which evicted
the very lines the operator had paused on.

The rest:

- A plugin could white-screen the whole console by registering `icon:
  "constructor"`. The icon map is a plain object, so the inherited key resolved
  to `Object`, which is truthy — the fallback never fired and React was handed
  `Object` as a component, from inside the app shell.
- Saving a display arrangement deleted the saved position of every device that
  was not connected at that moment: the host replaces the whole map, and we only
  ever sent the displays we could see.
- Flipping DDC, PnP or dedicated-game-sessions committed whatever unsaved edits
  the Custom block was holding, then cleared the "unsaved" badge so there was no
  trace of it. Those three apply on top of the SAVED policy now.
- Saving the Custom block put the streamed-screen pin back to whatever it was
  when the form was seeded, undoing a change made in the picker below it.
- "End now" on a running game calls the host's only stop, which ends EVERY live
  session; on a grace row with no app id it ended every waiting game. Both say so
  first now, when there is more than one to lose.
- Edit and Delete were offered on library entries owned by a provider plugin,
  which the host refuses with 409 — silently. They are attributed instead.
- An install whose first poll failed never polled again, and one whose host
  restarted spun forever with no way to dismiss it.
- Submitting a second pairing PIN showed the previous attempt's "PIN sent"
  before a digit was typed, and the paired list it points you at never refreshed.
- The streamed-screen picker claimed an env pin during every slow load, and rows
  that cannot be picked now look that way instead of silently eating the click.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Opus 5 4575134c21 fix(web): one bad password from anywhere stops locking out the whole console
The console's login throttle was documented as per-IP and was not. Nitro's
`localFetch` hands the app a synthetic request whose socket has no
`remoteAddress`, so `getRequestIP()` returned undefined for every request and
every attempt was charged to one shared "unknown" bucket. Five wrong guesses
from any LAN peer locked out everyone — including the operator, and including
the update-apply route, which shares that budget. The Bun entry is the only
place the real peer is knowable, so it now stamps it into a header (deleting
any client-supplied copy first) and `peerAddress()` reads it back.

Verified on a real build bound to 0.0.0.0: seven wrong logins from 127.0.0.1
lock 127.0.0.1 out, a different peer still logs in on the first try, and a
request forging the header is charged to its real address.

Also on the way through:

- Installing an unreviewed package and adding a catalog source now re-ask for
  the console password, like applying an update already did. A 7-day session
  cookie should not be able to run new code on the host, and `store/install`
  with `accept_unverified` did exactly that through the generic passthrough.
  The gate sits at the trust boundary — adding a source, or a raw spec — not
  on every install from a source the operator already chose to trust.
- The ui-credential denylist is matched against the normalised path too, so
  `/api//v1/...` and friends can no longer walk around it.
- The console serves nosniff, a no-referrer policy, and a CSP that pins
  frame-ancestors, object-src and base-uri.
- A plugin UI's response no longer re-emits the content-encoding that `fetch`
  already decoded (which made compressed plugin pages fail to load), no longer
  sets cookies on the console's origin, and OPTIONS reaches the plugin instead
  of being refused 405 by us.
- An unreachable host reads as 502 on these routes, matching the passthrough,
  instead of a bare 500.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:20:10 +02:00
enricobuehlerandClaude Fable 5 10a1863cc7 test(host/audio): give the reopen tests longer than the backoff they wait for
`wait_until` allowed 200 × 10 ms — exactly the 2 s `backoff_start` the reopen
path spends before it can succeed. On a warm, idle machine it wins the race;
on a cold binary or a loaded box it does not, and `reopens_after_push_death`
failed 3 of ~5 cold runs while gating this branch. CI is always cold.

Six seconds costs nothing when the test passes and only delays a genuine
failure, so the budget now clears the backoff with room to spare.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:15:54 +02:00
enricobuehlerandClaude Opus 5 6a4ffcb15c fix(client): HDR stops leaking out of the stream and blowing out the console UI
ci / web (push) Successful in 1m3s
ci / rust-arm64 (push) Successful in 1m23s
ci / docs-site (push) Successful in 1m12s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 6s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 8s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 7s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m29s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 49s
deb / build-publish-client-arm64 (push) Successful in 1m57s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
deb / build-publish-host (push) Successful in 3m40s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m55s
deb / build-publish (push) Successful in 5m59s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m7s
docker / builders-arm64cross (push) Successful in 4s
docker / deploy-docs (push) Successful in 27s
android / android (push) Successful in 8m10s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m12s
arch / build-publish (push) Successful in 9m35s
flatpak / build-publish (push) Successful in 6m57s
ci / rust (push) Successful in 10m44s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 7m54s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 6m17s
apple / swift (push) Successful in 5m10s
apple / screenshots (push) Canceled after 19m1s
The gamepad/console UI looked right on launch, and wrong forever after the first HDR
session: connect to an HDR host, disconnect, and the UI came back overblown with wrong
colours.

The UI is not its own renderer. It is a `pf_presenter::overlay::Overlay` composited into
the SAME swapchain the stream used, and it draws plain sRGB with no HDR awareness at all —
so the swapchain's colorspace decides how its pixels are read. `present` switches SDR↔HDR10
from the FRAME's colour signalling, and a UI-only present is `FrameInput::Redraw`, which
carries none: the mode block is skipped entirely and nothing ever hands HDR10 back. The
UI's sRGB mid-tones were then emitted as PQ code points, i.e. near-peak nits.

`leave_hdr` drops back to SDR, called where the UI-only present already happens and gated
on the existing `browse_idle` — Browse mode with no live connector, i.e. the UI owns the
screen. That covers every route back to the UI rather than just the Ended/Failed arms, and
it is guarded internally so idle iterations stay free.

It also bails when minimized, which is load-bearing rather than an optimization:
`recreate_swapchain` keeps the old swapchain at a zero extent, but `set_hdr_mode` would by
then have rebuilt the CSC and overlay pipes against the SDR format — mismatched against
live HDR10 images. `present` early-returns on a zero extent above its HDR block, so this
was unreachable until a caller outside `present` existed.

Deliberately not applied to the `resize_scrim` arm of the same present: that scrim is a
mid-stream gap in a session that is still HDR, and flipping there would rebuild the
swapchain twice per resize.

Does not address the adjacent case: an overlay drawn DURING a live HDR session (the stats
HUD, the resize scrim) is blown out the same way. That needs the overlay to PQ-encode when
`hdr_active`; dropping to SDR is only correct here because the console UI shows exactly
when no stream is live.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:08:38 +02:00
enricobuehlerandClaude Fable 5 a7143a6510 Merge origin/main into audio/mic-latency-echo
Main moved through the same surfaces while the audio work was in flight, so
three files needed hand-resolution:

- clients/windows/src/app/settings.rs — main gave Windows its speaker and
  microphone endpoint pickers, the gap this branch could only report. Both
  keep their rows: the pickers, then Echo cancellation, and the microphone
  description keeps the sentence naming the mute chord.
- crates/pf-console-ui/src/screens/settings.rs — both sides grew the couch
  row list. Main's seven new rows and Echo cancellation are all reachable
  in Gaming Mode; the count is 22 and the rationale comment names echo
  cancellation among the fields that would otherwise be unreachable there.
- crates/pf-presenter/src/run.rs — both sides added a stats test at the same
  line. Both are kept.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:04:34 +02:00
enricobuehlerandClaude Opus 5 43a631ea9c fix(clients): a host that re-keys stops locking the client out for good
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m21s
ci / rust-arm64 (push) Successful in 2m48s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 7s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
deb / build-publish-client-arm64 (push) Successful in 2m25s
docker / deploy-docs (push) Canceled after 25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m31s
deb / build-publish (push) Successful in 3m52s
apple / swift (push) Successful in 5m20s
arch / build-publish (push) Successful in 6m52s
docker / builders-arm64cross (push) Successful in 8s
android / android (push) Canceled after 7m17s
deb / build-publish-host (push) Successful in 5m45s
apple / screenshots (push) Canceled after 1m5s
ci / rust (push) Canceled after 7m24s
flatpak / build-publish (push) Canceled after 3m32s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m58s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 3m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 2m47s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 11s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
Reinstall a host, wipe its ProgramData, or otherwise regenerate its identity,
and the desktop clients refused it forever: "Host identity rejected — wrong
fingerprint, or the host requires pairing", including immediately after a
successful re-pair. There was no way out of it from the UI — the host list
showed two cards for one address and forgetting the wrong one was a guess.

`KnownHosts::upsert` matches on the FINGERPRINT, which is what lets a host that
moved address keep its record and everything the user set on it. A host that
changed identity matched nothing, so pairing appended a SECOND record for an
address that already had one, and `find_by_addr` returned whichever came first
in the file — the dead one, every time.

Trust decisions (PIN ceremony, delegated approval, TOFU accept, headless pair —
all funnelled through `persist_host`, plus the Windows shell's two direct
upserts) now go through `upsert_trusted`, which retires any OTHER record for
that address. Retired means DELETED, not demoted: a record whose certificate
the host no longer holds cannot connect, so keeping it only reproduces the two-
cards-one-address confusion this fixes. What described the box rather than the
identity — its MAC, its OS chain, the bound profile, the pinned cards, when it
was last used — moves onto the record that survives, so a reinstall doesn't
quietly cost the user their setup. What described the dead identity does not:
`paired` and `clipboard_sync` are decisions about one specific certificate and
have to be made again for a new one, and the retired record's stable id stays
retired (a deep link written from it falls through to the `host=` recovery the
link grammar already specifies).

Only trust decisions may retire a record. The wake path's address re-key and
every learn-from-advert path stay on plain `upsert`: those are driven by
unauthenticated mDNS, and letting an advert delete a saved host by claiming its
address would trade this bug for a much worse one. A plain reconnect still
fails closed on a pin mismatch — nothing here changes what the pin is checked
against.

Stores that already hold the duplicate recover on the next connect, not at
load: which of two records is live isn't knowable at load time and guessing
wrong would throw away the good one. Instead `find_by_addr` stops being
positional — a real fingerprint beats a placeholder, and among real ones the
newest trust decision wins, since records are only ever appended by one. The
next successful pair then cleans the store up for good. Every lookup that picks
a pin or a per-host decision for a connect now goes through it (the session's
pin and clipboard read, the deep-link resolver, orchestrate's plan, both speed
tests, the CLI's --wake and --library, which had also been ignoring the port),
and an advert's learned MAC/OS lands on the record it identified rather than on
a stale namesake that merely came first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:00:49 +02:00
enricobuehlerandClaude Opus 5 6af067da2d fix(client): the bottom rows stop smearing — the decode pool is taller than the picture
ci / web (push) Successful in 1m49s
ci / rust-arm64 (push) Successful in 2m19s
ci / docs-site (push) Successful in 2m29s
ci / rust (push) Successful in 7m3s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 19s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 8s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
deb / build-publish-client-arm64 (push) Successful in 2m28s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 21s
deb / build-publish (push) Successful in 4m17s
deb / build-publish-host (push) Successful in 4m51s
docker / builders-arm64cross (push) Successful in 6s
docker / deploy-docs (push) Successful in 36s
android / android (push) Successful in 8m24s
arch / build-publish (push) Successful in 8m28s
flatpak / build-publish (push) Successful in 6m8s
apple / swift (push) Successful in 5m54s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 13m5s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 3m10s
apple / screenshots (push) Canceled after 7m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 16m25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 15m57s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 5m3s
A user's 1080p stream repeated its last row of pixels over the final few rows, so the
image looked stretched at the bottom.

The Vulkan-Video CSC pass sampled the decoded planes with the fullscreen triangle's
normalized 0..1 UVs, but its render target is built at the CROPPED frame size. Those are
not the same rectangle: FFmpeg sizes the decode pool from `avctx->coded_*`, and H.264
codes `16 * mb_height` — so a 1080-row picture decodes into a 1088-row pool. Destination
row 1079 sampled source row ~1087.5, dragging the 8 alignment rows into view and squashing
the picture 0.7%. Encoders fill that padding by replicating the last picture line, which
is why it reads as a smeared bottom row rather than garbage.

Confirmed on glass (.173, RTX, H.264 1080p, vulkan-video):
  Vulkan Video first frame width=1920 height=1080 pool_w=1920 pool_h=1088

`VkVideoFrame` now carries the pool extent and `record_csc` takes a `uv_scale`, written to
the shader's `params.zw` — which the CSC shader already reserved for a use like this. The
chroma cositing offset is unchanged and stays correct: `textureSize` reports the pool
width, which is the space the scaled UV is already in.

Only the Vulkan-Video path passes a scale below 1.0. D3D11VA already clamps this in its
VideoProcessor blit (the same bug, seen as a green bar there because DXVA padding is
uninitialized rather than replicated); dmabuf imports its planes at the crop over the real
stride; PyroWave allocates its ring at exact stream dims. Apple and Android crop at the OS
layer. Every other call site passes [1.0, 1.0], so the change is inert there.

Also adds a one-time first-frame layout log mirroring the D3D11VA one, so the frame-vs-pool
gap is visible in the field instead of having to be re-derived from FFmpeg internals.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 23:40:33 +02:00
enricobuehlerandClaude Fable 5 8e877ad25f fix(android): a mute is a pause, not a hole the host tries to conceal
The Android uplink kept advancing `seq` while muted, so the first frame
after an unmute looked to the host like loss the width of the mute. The
de-jitter reads that as a gap: up to five concealment frames of stale
voice, and a seq gap counted in the uplink-health line. Past 600 ms the
pump's stale flush resets the chain first and hides it, which is why the
usual long mute looks fine — a quick toggle does not.

Freeze `seq` while muted, as the desktop uplink already does, so the
frame after an unmute continues the chain. `reset_stream` says it
plainly: a pause is not loss, and must not conceal or count a gap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 23:15:03 +02:00
enricobuehler 0c4a543831 Merge branch 'audio/desktop-mute-and-aec-toggle' into audio/mic-latency-echo 2026-07-31 23:14:09 +02:00
enricobuehler 4c98e2f788 Merge branch 'audio/android-mic-mute' into audio/mic-latency-echo 2026-07-31 23:14:09 +02:00
enricobuehler 6be8e5d6fa Merge branch 'audio/apple-mic-mute' into audio/mic-latency-echo 2026-07-31 23:14:09 +02:00
enricobuehlerandClaude Fable 5 7ec3107b4a feat(client): the mic has an off switch, and echo cancellation has a switch too
Ctrl+Alt+Shift+V mutes and unmutes the microphone mid-stream — V for
voice, since M and S were taken. The uplink keeps running while muted:
`MicStreamer::spawn` takes a shared AtomicBool the capture callback reads
every quantum, and a muted callback drains whole frames and sends
nothing. Stopping the stream instead would have re-primed the device
buffers and, on Linux, re-run source selection on every unmute — a
second of glitch for a key people press mid-sentence. The sequence
counter deliberately does NOT advance while muted, so the host sees one
continuous sequence with a pause rather than a gap the size of the mute,
which its de-jitter would try to conceal frame by frame (its 600 ms
stale-flush covers the rest).

The mute lives on SessionHandle as a MicControl with two flags, not one:
`live` is raised by the pump only once the uplink is actually running, so
a session with the mic off in Settings — or whose capture device wouldn't
open — reports "nothing to mute", the chord says so in the log, and no
indicator appears. Per session, never persisted.

Muted state draws as a persistent "Microphone muted" badge in the stream's
top-right corner, off `FrameCtx::mic_muted` rather than the stats text: it
has to be there with the stats overlay Off, which is where most people
leave it. The Detailed mic line still reads throughput, so it simply falls
to zero — the badge is what answers "am I muted".

Echo cancellation stops being an env-only lever. `Settings::echo_cancel`
(default on, `#[serde(default)]` so every stored file loads with it on)
now gates the same hooks PUNKTFUNK_NO_AEC gated: the echo-cancelled
PipeWire source preference and WASAPI's Communications stream category.
The env var still wins, one-way — it can only turn AEC off, never back on
— and both `aec_enabled` helpers say so. The row ships in the GTK, WinUI
and console settings, under the microphone toggle and greyed out while it
is off, matching what Apple and Android shipped in wave 1.

SettingsOverlay grows `echo_cancel` as a first-class field — apply,
absorb, clear, is_empty — instead of riding the `extra` passthrough, where
`clear_override("echo_cancel")` answered false. The JSON key is the one
Apple and Android already write, so one catalog round-trips through all
three.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 23:07:30 +02:00
enricobuehlerandClaude Fable 5 9f72a3b6ad feat(host): HDR and 4:4:4 stop being mutually exclusive on Windows
windows-host / package (push) Failing after 22s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
ci / web (push) Successful in 1m7s
ci / rust-arm64 (push) Successful in 1m30s
ci / docs-site (push) Successful in 1m31s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 19s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
deb / build-publish-client-arm64 (push) Successful in 2m21s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m6s
deb / build-publish (push) Successful in 4m43s
docker / builders-arm64cross (push) Successful in 11s
docker / deploy-docs (push) Successful in 33s
deb / build-publish-host (push) Successful in 5m4s
android / android (push) Successful in 6m40s
ci / rust (push) Successful in 7m10s
arch / build-publish (push) Successful in 8m11s
apple / swift (push) Successful in 5m8s
apple / screenshots (push) Canceled after 10m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 16m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 15m50s
An HDR display cost you full chroma: the IDD-push capturer's only 10-bit
output was P010, so a session that negotiated 4:4:4 on an HDR desktop was
converted to 4:2:0 at capture time — *after* the Welcome had already told
the client 4:4:4. The client believed it (nothing on the wire contradicts
a Welcome), and the new chroma tag in the stats overlay is what finally
made the discrepancy visible.

Everything except the source was already in place, which is why this is
small: the NVENC config layer has stamped `FREXT` + `chromaFormatIDC=3` +
`pixelBitDepthMinus8=2` — HEVC Main 4:4:4 10 — with a unit test since the
4:4:4 work landed, `PixelFormat::Rgb10a2` already maps to `ABGR10` and
already counts as a full-chroma input, and the desktop client learned the
10-bit 4:4:4 Vulkan pool format in 74863c96. The one missing piece was a
capture format that keeps 10 bits AND full chroma.

`HdrRgb10Converter` is that piece: one full-res pass from the FP16 scRGB
desktop to packed `R10G10B10A2` in BT.2020 PQ, reusing the P010 shader's
`scrgb_to_pq2020` verbatim so both HDR outputs share bit-identical colour
math — it simply stops before the RGB→YUV matrix, the studio-range
squeeze and the chroma decimation. NVENC then does the CSC to YUV 4:4:4
itself under FREXT, exactly as the SDR BGRA passthrough has always done
at 8 bits.

No swizzle is involved and that is worth stating, because it looks like
it should be: NVENC names packed formats from the MSB down, so its
`ABGR10` (A2B10G10R10) puts R in the low 10 bits — bit-identical to DXGI
`R10G10B10A2_UNORM`. It is the same relationship the proven SDR pair
relies on between DXGI `B8G8R8A8` and NVENC's `ARGB`.

The honesty gap closes as a consequence: `capturer_supports_444` no
longer has a depth it cannot serve, so the chroma resolved before the
Welcome is the chroma the wire carries. AV1 is deliberately untouched —
Range Extensions are HEVC-only and no consumer encoder does AV1 4:4:4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 23:06:44 +02:00
enricobuehlerandClaude Fable 5 253e0bbe7c feat(android): the mic has an off switch you can reach mid-stream
Wave 1 gave the Android client a mic worth using. It gave it no way to stop
talking: leaving the stream, or digging through Settings to turn the whole
feature off, were the only ways to stop the room being heard. Now a tap (or
Select + Y on a pad) mutes it, and the screen says so while it lasts.

How muting gates the capture, and why that way. The AAudio input stream is
never stopped: a stop/start would re-run the input-preset fallback ladder and
re-prime the buffers on every toggle — hundreds of milliseconds, and possibly
a landing on a different rung, silently losing the HAL echo canceller wave 1
went to some trouble to get. Instead the encode loop reads an AtomicBool per
10 ms frame and, while it is set, drains the frame out of its ring and drops
it there — the last point before it would have become an Opus packet. Nothing
is encoded, nothing is sent, and the realtime capture callback is untouched,
so its allocation-free discipline and the queue policy stay exactly as wave 1
verified them. A toggle costs one atomic store and takes effect on the next
10 ms boundary.

The frame counter keeps advancing across a mute, because it numbers the
captured 10 ms TIMELINE rather than the datagrams. The gap the host then sees
is exactly the audio that never came: its de-jitter conceals at most a few
frames of it before the pump's 600 ms stale-gap flush resets the chain
outright, which is the right reading of a mute. Encoding silence instead
would have kept a pointless uplink and a host-side ring alive for its whole
duration.

Mute is per session and nothing is persisted — a new stream always starts
unmuted, and no new setting exists. The flag lives on the session handle
rather than on the capture, so the mic stop/start a surface recreate performs
brings the user's choice back with it, with no window in which the fresh
capture could send an unmuted frame.

The control is offered on the evidence that a capture is actually running
(nativeMicActive), not on the setting: with the mic disabled, RECORD_AUDIO
denied, or every AAudio input rung refused, there is nothing on screen to
lie about. On touch it is a pill in the corner the stats HUD doesn't use —
the one in-stream control, so it sits above the gesture layer to take its own
taps — dim while live, a red "Muted" badge while it isn't. On TV that badge
is the indicator alone: Select + Y is the control there, and a focusable
button would fight the game for the D-pad. Y is deliberately not one of the
exit chord's buttons, so neither chord can be reached through the other.

One honest consequence of keeping the stream open: the platform's recording
indicator stays lit while muted, because the mic really is still open. What
stops is the encode and the send.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:41:09 +02:00
enricobuehlerandClaude Fable 5 47eb8c9f6f fix(windows): a web-console deploy that half-succeeded stops reporting success
windows-host / package (push) Failing after 26s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
ci / web (push) Successful in 1m7s
apple / swift (push) Successful in 1m28s
ci / rust-arm64 (push) Successful in 1m57s
ci / docs-site (push) Successful in 2m31s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 6s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 7s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
deb / build-publish-client-arm64 (push) Successful in 2m22s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 48s
deb / build-publish-host (push) Successful in 4m38s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m26s
deb / build-publish (push) Successful in 5m45s
docker / builders-arm64cross (push) Successful in 13s
docker / deploy-docs (push) Successful in 31s
android / android (push) Successful in 7m33s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m55s
arch / build-publish (push) Successful in 8m58s
ci / rust (push) Successful in 7m17s
flatpak / build-publish (push) Successful in 7m11s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m2s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m39s
release / apple (push) Successful in 16m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m39s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m59s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m53s
apple / screenshots (push) Canceled after 16m31s
Found by investigating a console that had been serving errors on .173 for
hours while every health check said HTTP 200.

Nitro's `entry.mjs` imports its sibling chunks by CONTENT HASH, so a
`.output` that mixes two builds is not degraded — it is dead: bun answers
every page with a `ResolveMessage` JSON body ("Cannot find module
../_/router-<hash>.mjs") under a 200 status. Two defects here let exactly
that ship and then hid it:

* The pre-copy `Remove-Item` used `-ErrorAction SilentlyContinue`, so a
  removal blocked by a still-running bun was swallowed and `Copy-Item`
  merged the new build into the old tree. (Reproduced live: an older
  task-based copy of this script, run against the now supervised-child
  host, tried `schtasks /end` for a task that no longer exists, never
  stopped the service, and so could never unlock the files.) The removal
  is now verified and refuses to copy over a tree it could not clear.
* The success probe read only the status code, so it reported a healthy
  console for a server that serves nothing but an error. It now checks
  that `/login` actually returns HTML, and says so loudly when the body
  is a module-resolution error instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:40:26 +02:00
enricobuehlerandClaude Fable 5 766991cf6a feat(apple): the mic has an off switch you can reach mid-stream
Until now the only way to stop sending the room was to end the session and turn
"Send microphone to the host" off in Settings — a per-app setting for something
that is really a per-conversation act. The mic now mutes from inside the stream:
a button on the HUD card, the Stream menu's ⌃⌥⇧A (macOS menu bar and iPad
hardware keyboards), the same chord while input is captured (both platforms
detect it in InputCapture, where the reserved ⌃⌥⇧Q/D/S already live — ⌃⌥⇧M is
the mouse-model flip, cross-client, so the mic gets A), and on iPhone/iPad a mic
disc beside the touch exit for the stats tiers whose HUD carries no buttons.

Muting is local and instant: it gates capture on this device, the host is never
asked and never told. The muted state gets its own badge over the stream —
independent of the stats overlay, because "am I muted?" is not a statistic and
the overlay is exactly what a player turns off. The badge is also the way back:
tapping it unmutes, which is the guaranteed path for a touch user who muted with
the overlay off.

Mute is session state and is deliberately not persisted. Every stream starts live
if the mic is enabled at all, rather than carrying a mute nobody remembers making
into a call three days later.

The mechanism is the one wave 1 built. `SessionAudio.setMicMuted` — which mutes
the voice processor's input on the combined engine and pauses the capture engine
on the split one — stays the single muting path; what changes is that it now
takes an EFFECTIVE mute the session composes from its two reasons: the user's
mute and the background keep-alive's privacy mute. Neither can clear the other,
so a user who muted before pocketing their phone comes back still muted, and
backgrounding no longer un-mutes anyone on return. It also latches the state, so
a mute made while the microphone permission prompt is still open lands on the
engine that grant creates instead of being lost.

The control is offered only where there is something to mute: the session's
resolved `micEnabled` (a profile can turn the mic on or off), a platform with an
app-accessible input (never tvOS), and a TCC grant the OS hasn't refused. Absent
rather than greyed on the HUD and the touch discs, greyed on the menu, and the
macOS start-of-stream banner only teaches ⌃⌥⇧A when the session actually sends a
microphone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:35:48 +02:00
enricobuehler 41e7035441 Merge branch 'audio/apple-mic' into audio/mic-latency-echo 2026-07-31 22:12:52 +02:00
enricobuehler 27861b52b8 Merge branch 'audio/android-mic' into audio/mic-latency-echo 2026-07-31 22:12:52 +02:00
enricobuehler c846b165ae Merge branch 'audio/core-desktop-uplink' into audio/mic-latency-echo 2026-07-31 22:12:51 +02:00
enricobuehler d3870294d0 Merge branch 'audio/host-mic-jitter' into audio/mic-latency-echo 2026-07-31 22:12:51 +02:00
enricobuehlerandClaude Fable 5 e861565e27 docs: "Why do I hear myself" — the four echo loops, and which knob stops each
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:11:19 +02:00
enricobuehlerandClaude Fable 5 79856d2c50 fix(host/audio): a VoiceMeeter box can no longer loop the mic into the stream
wiring_plan could hand the mic Voicemeeter Input and the loopback
Voicemeeter Aux Input — two strips of the same internal mixer, i.e. a
digital feedback loop with no acoustic path to break it, because
leftover() never asked virtualish() and the exclusion list only knew
"cable" and the Steam Speakers. Now every VoiceMeeter or generically
"virtual" render is excluded from loopback, and the last-resort tier
refuses anything virtual: a box with only mixer endpoints gets
loopback=None, honest like the cable-only case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:11:17 +02:00
enricobuehlerandClaude Fable 5 efac33fd33 feat(host/audio): the virtual mic buffers for the client it has, not the worst one ever
The mic pump grows a real de-jitter (audio/mic_jitter.rs): a two-frame
reorder window in front of the decoder, libopus concealment on sequence
gaps (up to 5 frames — a lost datagram no longer drains the ring into a
silence + re-prime crackle), and an adaptive target depth measured from
inter-arrival jitter, clamped to 10–60 ms. Both backend rings now prime
at one consumer quantum + that target: the old bursty Mac client still
measures ~42 ms and lands where the fixed 48 ms prime protected it, a
modern 10 ms-cadence client settles at ~25–35 ms, and a 2048-frame
recorder on Linux stops buying 128 ms of latency from the 3-quanta
clamp. Depth stuck above target sheds near-silent frames a few ms per
100 ms — never speech, never a hard clear.

PUNKTFUNK_MIC_LEGACY_BUFFER=1 (documented) is the one-release escape
hatch back to the fixed constants, and a "mic uplink health" line every
30 s (depth/target, cadence, gaps, conceals, reorders, drops, re-primes)
finally says which side of the link a bad mic lives on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:11:08 +02:00
enricobuehlerandClaude Fable 5 2ce0bea830 feat(client): desktop phase-locked capture, a real display volume on Windows, and the console's missing rows
The cross-platform half of the gap sweep:

* Phase-locked capture reaches the DESKTOP clients (it shipped on
  Apple/Android only, though the desktop presenter has the best latch
  signal of all — true on-glass stamps via VK_KHR_present_wait). The
  presenter's 1 Hz fold publishes a latch grid (anchor = last on-glass
  instant; period = min positive present spacing, capped by the display
  mode's refresh so an arrival-paced sub-panel-rate stream can't claim a
  slower grid); the session pump folds every AU's arrival stamp against
  it with the SHARED `phase::circular_latch` statistic and sends the
  ~1 Hz PhaseReport (1 ms uncertainty — reference-client parity). The
  cap is advertised only when present timing is real
  (`VulkanDecodeDevice::present_timing` gates `SessionParams::phase_lock`),
  and the host's applied grid offset from the 0xCF tail is logged so an
  on-glass run can watch the controller engage.
* `Hello::display_hdr` stops being hardcoded `None`: Windows reads the
  panel's colour volume from DXGI (`IDXGIOutput6::GetDesc1`, the
  `--window-pos` output else the primary, advanced-color outputs only,
  gated on the HDR setting) so the host's virtual-display EDID matches
  the real glass. Linux keeps the EDID defaults — no portable
  Wayland/X11 query exists — and the comment now says exactly that.
* The console settings screen (the ONLY editor in Gaming Mode) learns
  the rows it was missing: render scale, full chroma 4:4:4, invert
  scroll, capture system shortcuts, fullscreen-on-stream, auto-wake and
  the game-library toggle.
* A spec-run session's device picks (GPU adapter, speaker, microphone)
  now come from the `--resolved-spec` instead of a raw Settings load —
  the last store read the spec path still owed (§5), and what would
  make those fields profileable.
* `session_args()` documents why the GTK/CLI spawners pass no
  `--window-pos` (Wayland exposes no global coordinates to read and SDL
  can't apply them).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:05:24 +02:00
enricobuehlerandClaude Fable 5 fd75e66041 feat(windows): the shell spawns from the shared brain, and links, tests and pairing stop losing things
Windows-shell parity gaps from the 2026-07-31 sweep, closed:

* The spawner goes through the shared brain: `ConnectPlan::for_target` →
  `session_args()` + a written `--resolved-spec`, replacing the
  hand-assembled argv that meant Windows sessions ALWAYS took the compat
  path (re-resolving every setting from the stores — the drift
  orchestrate.rs documents as a trap) and that any field added to the
  spec was silently Windows-dead. Fullscreen now comes from the plan's
  effective settings (profile-aware) instead of a caller argument, the
  spec temp file is cleaned up at exit, and the reader finally parses
  the `{"window":…}` line so the SPAWNER persists the match-window size
  (§5) instead of the renderer's load-modify-save fallback.
* A deep link to an unpaired host runs the trust ceremony instead of
  refusing: the Pair screen opens seeded with what the link CLAIMED —
  name shown as claimed, fingerprint carried, and the launch/profile
  surviving the detour (`Target` grew a `launch` field for exactly
  this). GTK parity; a shared link was a dead end here before.
* The speed test learns the Ask tier: a bound host whose profile
  INHERITS bitrate now offers both "Set as default" and "Set in
  profile" instead of silently creating a profile override. The global
  write (and the forwarded-controller handler) also rebase on the file
  before their whole-struct saves — two more stale-snapshot writers.
* The Controllers card shows the detected-pad inventory (read-only,
  with the Steam-Input-virtual note) — the fastest answer to "is my
  controller even detected?", present on GTK all along.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:05:05 +02:00
enricobuehlerandClaude Fable 5 f3e122b0d0 feat(apple): the mic hears you, not the game — voice processing joins the engines
The Apple client on a loudspeaker was the primary reported echo source:
two AVAudioEngines by design (arbitrary mic/speaker pairs, two clocks)
meant no unit ever saw render and capture together, which is exactly
what setVoiceProcessingEnabled needs. With the mic enabled and the new
"Echo cancellation" toggle on (both defaults), playback and capture now
share ONE engine with the system voice processor engaged — AEC, noise
suppression and AGC — and the input node's other-audio ducking pinned
to .min with advanced ducking off, because this is a game stream, not a
call: the host's audio must never dip under the outgoing voice.

The two-engine path survives where it is the right answer: echo cancel
off, macOS sessions with a hand-picked speaker/mic UID or input channel
(the voice processor only follows the system default devices, and its
capture side is its own mono mix — a per-channel pick can't survive
it), and any machine whose voice processor refuses to engage. Every
failure falls back to a working configuration rather than silence. The
capture chain reads the input format after the processor is enabled,
so its mono/lower-rate output flows through the existing converter
unchanged; a first-run permission grant swaps the playback-only engine
for the combined one in place, reusing the same jitter ring and drain
thread; background mic muting mutes the processor's input instead of
pausing the shared engine (playback keeps running).

echo_cancel rides the full settings plumbing micEnabled has — defaults
key, effective settings, profile overlay (serialized as `echo_cancel`;
the Rust catalog carries it via unknown-key passthrough until it grows
the field), a captioned toggle beside the Microphone row, and a gamepad
settings row. tvOS behavior is untouched (playback only, as before).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:58:31 +02:00
enricobuehlerandClaude Fable 5 e54258b8ac feat(apple): the mic uplink drops to mono 10 ms packets and stops shoveling its FIFO
The capture path carried three latencies that had nothing to do with the
network: a 2048-frame tap request (42.7 ms bursts before the first
sample could even be sliced), 20 ms Opus framing, and a mono signal
duplicated into both stereo channels because the encoder was configured
like the downlink. CoreAudio's Opus encoder takes mFramesPerPacket=480
and mChannelsPerFrame=1 just fine — probed empirically: the converter
truly emits 10 ms mono CELT packets (TOC config 30), one per 480-frame
chunk, and the stereo-shaped decoder upmixes them with the tone intact —
so the uplink now asks for 10 ms tap buffers and encodes 48 kbps mono
10 ms packets, with 960 kept only as an init-time fallback. The host
decodes any Opus frame ≤120 ms, so nothing changes on the wire's far
end.

The tap thread also stops burning cycles per callback: the mono fold and
resampler scratch buffers are allocated once (regrown only if a larger
device quantum ever arrives), and the chunk slicer walks a head index
instead of removeFirst — which memmoved the entire backlog for every
packet on a render-adjacent thread. iOS additionally asks the session
for 5 ms IO quanta at 48 kHz when the mic is on (best-effort; the
hardware decides).

Verified: swift build (macOS arm64), swift test OpusCodecTests +
AudioChannelFoldTests, and an iOS arm64 cross-build; the 480/mono
behavior confirmed by TOC inspection on macOS 15.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:57:48 +02:00
enricobuehlerandClaude Fable 5 a681de7e5c feat(android): the mic goes through the echo canceller, and the host stops hearing itself
The capture stream opened under AAudio's default VoiceRecognition input
preset, which deliberately bypasses the HAL's acoustic echo canceller —
so a phone playing the game audio out of its own speaker fed that audio
straight back to the host. Two layers fix it, both behind a new "Echo
cancellation" setting (default ON, next to the Microphone toggle in the
touch and console settings, per-profile like every tier-P setting):

- Native: the mic opens under the VoiceCommunication preset (HAL AEC/NS
  on the capture path) and allocates an audio session id. The open
  ladder is Exclusive+voice → Shared+voice → Exclusive → Shared — some
  HALs refuse the preset or a session id outright, and a mic without
  echo cancellation still beats no mic; the last rungs are exactly the
  preset-less open this always did.

- Kotlin backstop: nativeStartMic now returns the allocated session id
  (0 = none), and StreamScreen hangs the Java AcousticEchoCanceler +
  NoiseSuppressor off it (guarded by isAvailable), releasing them on
  every mic-stop path — the surface teardown and the final dispose — so
  a surface recreate re-attaches instead of leaking effect engines.

The playback stream is untouched: retagging it voice/communication
would route it through the phone-call chain and regress quality.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:53:01 +02:00
enricobuehlerandClaude Fable 5 6286f91f89 feat(host/audio): the mic uplink keeps its sequence, and a backlog heals itself
The 0xCB datagrams always carried a seq + pts, and the ingest threw both
away one line after decoding them — the pump saw an anonymous byte pile.
Frames now travel as MicFrame {seq, pts_ns, opus} so the de-jitter that
follows can reorder, conceal and measure.

The shared queue also stops being a latency reservoir: cap 64 → 12, and
a pump that wakes to a backlog deeper than 6 frames jumps to the newest
4 instead of replaying the pile — before this, a scheduling stall could
park up to 1.28 s of standing mic delay that only a >600 ms silence gap
ever flushed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:50:29 +02:00
enricobuehlerandClaude Fable 5 55e01b4dcb feat(client): the stats overlay learns whether your mic frames arrive
NativeClient::mic_stats reaches the per-second stream window: sent and
dropped (queue-full + stale-shed) frame deltas join the tracing line and
the Stats event, and the Detailed OSD tier renders a mic line while the
uplink is live — a healthy 10 ms-frame mic reads ~100 f/s, and a drop
term means the client is shedding backlog, not the network eating audio.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:49:12 +02:00
enricobuehlerandClaude Fable 5 d17e942db0 feat(client/audio): the mic speaks 10 ms mono and asks the OS to cancel the echo
Uplink format on both desktops: 10 ms mono frames (was 20 ms stereo),
Opus Voip at 48 kbps with in-band FEC against 10 % assumed loss — half
the frame-fill latency, half the samples, and a lost datagram's audio
now rides in its successor. One datagram per frame, unchanged wire.

Linux: the capture stream finally asks for its own quantum
(NODE_LATENCY 480/48000) instead of inheriting the graph's 1024-2048
sample bursts, and when the user picked no mic it prefers an existing
echo-cancel source over the default (PUNKTFUNK_NO_AEC=1 opts out;
loading module-echo-cancel ourselves needs a load_module the pipewire
crate doesn't expose yet). Windows: the capture client declares
AudioCategory_Communications before Initialize so an endpoint's
communications APO (the system AEC) can engage; capture stays stereo
via autoconvert — the proven path — and downmixes to mono in code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:49:05 +02:00
enricobuehlerandClaude Fable 5 a7d4213778 feat(android): the mic uplink drops to mono 10 ms frames, and stops hoarding stale audio
Latency, three ways, all inside mic.rs:

- 48 kHz stereo 20 ms becomes mono 10 ms: speech gains nothing from a
  second channel, the shorter frame shaves a buffering interval off the
  uplink, and the host already decodes any Opus frame <= 120 ms with its
  stereo decoder (mono packets upmix) — no protocol change. The encoder
  follows: 48 kbps, complexity 5, in-band FEC at an assumed 10% loss so
  a dropped datagram reconstructs from its successor instead of a hole.

- The latency ratchet is gone: the capture callback drops the NEWEST
  chunk when the hand-off channel fills, so an encode-side stall used to
  convert into standing mic delay that never drained. The encode loop
  now drains the whole backlog in one lump and, past ~60 ms, jumps to
  the newest ~20 ms (one audible blip, live again), counting what it
  shed in the periodic log line. The realtime callback stays exactly as
  allocation-free as it was.

- The encode thread registers with the client's hot-thread set, so the
  ADPF session keeps mic encode on a fast core alongside audio decode.

No .frames_per_data_callback() pin: AAudio's own docs say leaving it
unset is the lowest-latency path (the callback runs at the device's
optimal burst), and the encode side re-chunks to 10 ms frames anyway.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:46:44 +02:00
enricobuehlerandClaude Fable 5 15392bd707 fix(core/client): the mic queue stops hoarding a second of your voice
MIC_QUEUE claimed 64 was ~320 ms of 5 ms frames; the frames were 20 ms, so
it really allowed 1.28 s — and since a full tokio mpsc can only refuse the
FRESH frame, one worker stall turned the whole backlog into permanent
standing mic latency. The queue shrinks to 12 and the pump's mic task now
sheds oldest-first past a ~60 ms backlog, so a stall costs a short dropout
and heals itself. New per-stage counters (sent / dropped-full /
dropped-stale) surface through NativeClient::mic_stats for the stats HUDs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:46:02 +02:00
enricobuehlerandClaude Fable 5 698925a036 fix(clients): settings saves stop reverting each other, Windows gets audio pickers, and a game link launches its game
The rest of the client-gaps handoff, plus what a parity sweep surfaced:

* Every whole-file settings save rebases on the file first — the GTK
  dialog close and its two speed-test apply arms, the Windows
  per-control commit, the console screen. The file has five writers and
  no merge (profiles.rs documents the debt); saving a shell-lifetime
  snapshot silently reverted whatever another writer stored meanwhile,
  most visibly the spawner-persisted match-window size (item 4).
* Windows honors the Speaker/Microphone picks: `audio_wasapi` grows
  endpoint enumeration (on its own MTA thread — UI threads are STA) and
  resolves PUNKTFUNK_AUDIO_SINK/SOURCE as endpoint ids, falling back to
  the default when the picked device is gone; the settings page gets
  the two rows (defaults scope, "(not detected)" like the GPU row).
  `speaker_device`/`mic_device` stop being Linux-only fields (item 5).
* A punktfunk:// link with `launch=` opened a plain desktop session on
  Windows — the id was parsed, validated, planned, and dropped at the
  last hop. It now rides `initiate_launch` (plus a waking variant for
  the dial-first path). Linux always forwarded it.
* The console UI offered "VAAPI" on Windows — a dead option there that
  also hid d3d11va, the actual Windows hardware path. The decoder list
  is per-OS now.
* The Windows gamepad picker learns "Steam Deck" (the GTK picker had
  it; the host-side pad has always existed).
* Doc fixes: gpu.rs named a nonexistent env var (PUNKTFUNK_ADAPTER →
  PUNKTFUNK_VK_ADAPTER), and session/Cargo.toml claimed video decode is
  Linux-only while explaining what the ARM64 leg drops.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:21:26 +02:00
enricobuehlerandClaude Fable 5 74863c96b3 feat(client): 4:4:4 can stay on hardware, and the overlay says what you actually got
Two halves of the same honesty problem (client-gaps handoff items 1+2,
done together as the handoff asked).

The presenter learns full chroma: `vkframe_plane_views` accepts the
2-plane 4:4:4 pool formats (8-bit, and the 10-bit 3PACK16 sibling) —
what NVIDIA's Vulkan Video reports for HEVC RExt decode — with the
accepted set extracted into `vkframe_plane_formats` and pinned by a
decision-table test. The CSC shader needed nothing: its 4:2:0 siting
correction already self-disables when the plane widths match. The VAAPI
leg gets the same treatment (NV24 in `drm_fourcc_for` and the dmabuf
import, full-size chroma plane), and the Vulkan decoder's sw-format
gate admits NV24/P410. 3-plane 4:4:4 stays rejected — it needs a third
CSC binding — and demotes cleanly like every other unsupported format.

Design call (the handoff's fork, argued here as requested): (B)+(C),
not (A). No capability probe gates VIDEO_CAP_444 — software decode is
the guaranteed display floor on both OSes (swscale → RGBA), the decoder
ladder demotes on its own, and a probe-gated bit would turn the switch
inert on exactly the boxes that rely on the fallback. What (A) wanted
from a prediction, the overlay now delivers as ground truth:

The Detailed tier prints the encoder's target next to the measured
rate — `19.4 Mb/s · target 20 Mb/s (auto)` — and the resolved chroma,
`4:4:4→4:2:0` when the host declined the ask (mirroring `HDR→SDR`).
The target is live: `NativeClient::current_bitrate_kbps()` mirrors
every BitrateChanged ack, so an Automatic session's ABR re-targets are
visible as they move. This is the figure whose absence let the
settings-drop bug (9c5af8d7) ship four releases — measured goodput
alone cannot distinguish "the encoder is capped at 20" from "my
200 Mb/s grant met a cheap scene".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:21:10 +02:00
260 changed files with 19375 additions and 2224 deletions
+95
View File
@@ -0,0 +1,95 @@
# Move a versionCode that is ALREADY on Google Play between tracks — no rebuild.
#
# Why this is separate from android.yml: promotion must not rebuild. A rebuild produces a fresh
# versionCode (github.run_number) from possibly-newer sources, so it ships something nobody tested;
# promoting assigns the byte-identical artifact the testers already ran. Bolting this onto
# android.yml would mean an `if:` on all ten of its build steps.
#
# What it is for:
# * promote a tested build up a track (alpha -> production)
# * roll production back by re-pointing it at an older versionCode (to_track=production,
# version_code=<the good one>, from_track blank)
# * halt a rollout (status=halted)
#
# Defaults are deliberately the safe ones: dry_run starts TRUE, so a mis-typed versionCode
# validates and deletes the edit instead of publishing. Flip it to false only when the dry run
# printed what you meant.
name: android-promote
# Two concurrent promotions would race on the same Play edit; the loser fails with a stale-edit
# error. One at a time, and never cancel one mid-flight — a half-applied track change is worse
# than a queued one.
concurrency:
group: android-promote
cancel-in-progress: false
on:
workflow_dispatch:
inputs:
version_code:
description: 'versionCode already on Play (e.g. 10816)'
required: true
to_track:
description: 'destination track'
required: true
default: 'production'
from_track:
description: 'track to verify it is on, then clear (blank = touch nothing else)'
required: false
default: 'alpha'
notes_tag:
description: "tag whose docs/releases/whatsnew/<tag>.txt to attach, e.g. v0.23.0 (blank = none)"
required: false
default: ''
status:
description: 'completed (100%) | inProgress (needs user_fraction) | halted | draft'
required: true
default: 'completed'
user_fraction:
description: 'staged rollout fraction for inProgress, e.g. 0.2 (blank otherwise)'
required: false
default: ''
dry_run:
description: 'validate only, publish nothing'
required: true
default: 'true'
jobs:
promote:
runs-on: ubuntu-24.04
# Same image as android.yml purely for python3 + openssl (play-upload.py's only deps); it is
# already warm on the runner. Nothing here builds.
container:
image: 192.168.1.58:5010/punktfunk-android-ci:latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- name: Promote
env:
SERVICE_ACCOUNT_JSON: ${{ secrets.SERVICE_ACCOUNT_JSON }}
VERSION_CODE: ${{ inputs.version_code }}
TO_TRACK: ${{ inputs.to_track }}
FROM_TRACK: ${{ inputs.from_track }}
NOTES_TAG: ${{ inputs.notes_tag }}
STATUS: ${{ inputs.status }}
USER_FRACTION: ${{ inputs.user_fraction }}
DRY_RUN: ${{ inputs.dry_run }}
run: |
set -- --package io.unom.punktfunk \
--promote "$VERSION_CODE" \
--track "$TO_TRACK" --status "$STATUS"
# Explicit `if`, not `[ ] && …`: under `sh -e` a false AND-OR list that ends up LAST in
# the script aborts the step, and these get reordered.
if [ -n "$FROM_TRACK" ]; then set -- "$@" --promote-from "$FROM_TRACK"; fi
if [ -n "$USER_FRACTION" ]; then set -- "$@" --user-fraction "$USER_FRACTION"; fi
if [ -n "$NOTES_TAG" ]; then
NOTES="docs/releases/whatsnew/${NOTES_TAG}.txt"
# Fail loudly rather than silently publishing with the PREVIOUS release's text still
# showing on the store listing.
[ -f "$NOTES" ] || { echo "ERROR: no such notes file: $NOTES"; exit 1; }
set -- "$@" --release-notes-file "$NOTES"
fi
if [ "$DRY_RUN" = "true" ]; then set -- "$@" --no-commit; fi
echo "promoting versionCode=$VERSION_CODE -> $TO_TRACK (dry_run=$DRY_RUN)"
python3 clients/android/ci/play-upload.py "$@"
+76 -10
View File
@@ -34,9 +34,10 @@ on:
- 'rust-toolchain.toml'
- 'scripts/ci/**'
- '.gitea/workflows/android.yml'
# Single project version: a `vX.Y.Z` tag is THE release (uploads to Play's `alpha` closed
# track for manual promotion + attaches the .aab/.apk to the unified Gitea Release). A main
# push is canary (Play `internal`).
# Single project version: a `vX.Y.Z` tag is THE release (publishes to Play `production` at
# 100% + attaches the .aab/.apk to the unified Gitea Release). A main push is canary
# (Play `internal`). Production access was granted 2026-08-01; before that a tag could only
# reach `alpha` and someone had to promote it by hand in the Console.
tags: ['v*']
pull_request:
paths:
@@ -75,6 +76,56 @@ jobs:
steps:
- uses: actions/checkout@v4
# FIRST, because it costs a second and everything after it costs ten minutes.
#
# A release tag MUST carry its own Play "What's new". If the file is absent Play does not
# show nothing — it carries the PREVIOUS release's text onto this version, so production
# users read notes for a build they are not getting. That is the same defect the v0.22.3
# notes shipped (see docs/releases/README.md), and it is invisible until someone reads the
# store listing. Failing here also means a missing file cannot leave a half-published
# release: nothing is built, nothing is attached to the Gitea release, nothing reaches Play.
#
# Canary is exempt on purpose: it has no curated notes, and Play reusing text for internal
# testers costs nothing.
- name: Play release notes gate (tags only)
if: startsWith(github.ref, 'refs/tags/v')
run: |
NOTES="docs/releases/whatsnew/${GITHUB_REF_NAME}.txt"
if [ ! -f "$NOTES" ]; then
echo "ERROR: $NOTES does not exist."
echo "A production release needs its own Play 'What's new' (<=500 chars, written for"
echo "phone/TV users). Without it Play reuses the previous release's text."
echo "See docs/releases/README.md; copy docs/releases/whatsnew/TEMPLATE.txt."
exit 1
fi
# A verbatim copy of another release's file is the same bug wearing a hat: the store
# listing still describes the wrong build. Cheap to check, and only ever trips on an
# actual copy-paste that was never edited.
for other in docs/releases/whatsnew/*.txt; do
if [ "$other" != "$NOTES" ] && [ "$other" != "docs/releases/whatsnew/TEMPLATE.txt" ]; then
if cmp -s "$NOTES" "$other"; then
echo "ERROR: $NOTES is byte-identical to $other."
echo "Write notes describing THIS release, not the one before it."
exit 1
fi
fi
done
# Length is checked here as well as in play-upload.py. Not redundant: the uploader is
# the last line of defence (and the only one android-promote.yml gets), but it runs at
# step 9 — this catches an unedited TEMPLATE copy at step 1 instead of after the build.
# Must count CHARACTERS, not bytes: Play's cap is 500 chars and `•` is 3 bytes in UTF-8,
# so `wc -c` would reject a file that is comfortably legal.
python3 - "$NOTES" <<'PY'
import sys
path = sys.argv[1]
text = open(path, encoding="utf-8").read().strip()
if not text:
sys.exit(f"ERROR: {path} is empty.")
if len(text) > 500:
sys.exit(f"ERROR: {path} is {len(text)} chars; Play allows 500. Trim it.")
print(f"Play release notes OK: {path} ({len(text)}/500 chars)")
PY
# Everything below the checkout used to be four download steps (JDK, SDK,
# NDK+CMake, cargo-ndk — the flakiest, heaviest part of the job); it is all baked
# into the image now. This guard only re-asserts the Android targets so a
@@ -122,11 +173,20 @@ jobs:
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
case "$GITHUB_REF" in
refs/tags/v*) VN="${GITHUB_REF_NAME#v}"; TRACK="alpha" ;; # alpha = built-in closed testing
refs/tags/v*) VN="${GITHUB_REF_NAME#v}"; TRACK="production" ;;
*) VN="${PF_BASE}-ci${GITHUB_RUN_NUMBER}"; TRACK="internal" ;;
esac
echo "VERSION_NAME=$VN" >> "$GITHUB_ENV"
echo "PLAY_TRACK=$TRACK" >> "$GITHUB_ENV"
# Play's own "What's new" (500-char cap, its own file — the vX.Y.Z.md body is ~34 KB).
# On a tag the gate step above already proved this exists, so the else branch is only
# ever the canary path. See docs/releases/README.md.
NOTES="docs/releases/whatsnew/${GITHUB_REF_NAME}.txt"
if [ -f "$NOTES" ]; then
echo "PLAY_NOTES=$NOTES" >> "$GITHUB_ENV"
else
echo "no Play release notes at $NOTES (canary — Play keeps the previous text)"
fi
echo "android version $VN -> Play track '$TRACK'"
- name: Build Release (signed AAB + universal APK)
@@ -199,15 +259,21 @@ jobs:
# Direct Publishing-API upload instead of r0adkll/upload-google-play — that action hides the
# real API error behind "Unknown error occurred."; this prints it. stdlib + openssl only (no
# pip), reuses SERVICE_ACCOUNT_JSON (raw JSON or base64), auto-handles changesNotSentForReview.
# Track: canary main -> `internal`; a vX.Y.Z release -> `alpha` (closed testing) for manual
# promotion to production in the Play console.
# Track: canary main -> `internal`; a vX.Y.Z release -> `production` at 100% (`completed`).
#
# A tag therefore ships to real users with no further click. Two things keep that honest:
# the tag is only pushed once every platform is green, and Play reviews each production
# release before it reaches anyone. To ramp instead of going straight to 100%, this is
# `--status inProgress --user-fraction 0.2`; to undo a bad one, halt or roll back from the
# Console (or `android-promote.yml`, which can re-point production at an older versionCode).
- name: Upload to Google Play
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
env:
SERVICE_ACCOUNT_JSON: ${{ secrets.SERVICE_ACCOUNT_JSON }}
run: |
echo "uploading to Play track '$PLAY_TRACK'"
python3 clients/android/ci/play-upload.py \
--package io.unom.punktfunk \
--aab clients/android/app/build/outputs/bundle/release/app-release.aab \
--track "$PLAY_TRACK" --status completed
set -- --package io.unom.punktfunk \
--aab clients/android/app/build/outputs/bundle/release/app-release.aab \
--track "$PLAY_TRACK" --status completed
if [ -n "${PLAY_NOTES:-}" ]; then set -- "$@" --release-notes-file "$PLAY_NOTES"; fi
python3 clients/android/ci/play-upload.py "$@"
Generated
+32 -32
View File
@@ -947,7 +947,7 @@ dependencies = [
[[package]]
name = "cursor-probe"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"pf-capture",
@@ -1036,7 +1036,7 @@ dependencies = [
[[package]]
name = "display-disturb"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
@@ -2221,7 +2221,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.22.3"
version = "0.24.0"
[[package]]
name = "lazy_static"
@@ -2326,7 +2326,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"bindgen",
"cmake",
@@ -2361,7 +2361,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"punktfunk-core",
]
@@ -2850,7 +2850,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2871,7 +2871,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ash",
@@ -2897,7 +2897,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2915,7 +2915,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ash",
@@ -2936,7 +2936,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ash",
@@ -2960,7 +2960,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"ash",
"bindgen",
@@ -2969,7 +2969,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"libc",
@@ -2981,7 +2981,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2995,11 +2995,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.22.3"
version = "0.24.0"
[[package]]
name = "pf-inject"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3028,14 +3028,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ash",
@@ -3050,7 +3050,7 @@ dependencies = [
[[package]]
name = "pf-update"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"serde",
"serde_json",
@@ -3058,7 +3058,7 @@ dependencies = [
[[package]]
name = "pf-update-check"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"base64",
@@ -3070,7 +3070,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3103,7 +3103,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"pf-paths",
@@ -3115,7 +3115,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ash",
@@ -3323,7 +3323,7 @@ dependencies = [
[[package]]
name = "punktfunk-cli"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"pf-client-core",
"punktfunk-core",
@@ -3334,7 +3334,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"android_logger",
"jni",
@@ -3350,7 +3350,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3367,7 +3367,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3382,7 +3382,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3402,7 +3402,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -3434,7 +3434,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"aes",
"aes-gcm",
@@ -3519,7 +3519,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3533,7 +3533,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"anyhow",
"ksni",
@@ -3556,7 +3556,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.22.3"
version = "0.24.0"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -53,7 +53,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.22.3"
version = "0.24.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+97 -3
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.22.3"
"version": "0.23.0"
},
"paths": {
"/api/v1/clients": {
@@ -2170,6 +2170,51 @@
}
}
},
"/api/v1/plugins/logs": {
"post": {
"tags": [
"plugins"
],
"summary": "Ingest runner log lines",
"description": "The plugin/script runner ships its output here so the console's **Logs** page can show it.\n\nPlugins are not host child processes — the runner is a separate `bun` process that `import()`s\neach plugin in-process — so nothing a plugin logs passes through the host's own `tracing`, and\nbefore this endpoint the console's log page could not show a single plugin line. On Linux the\nfallback was `journalctl --user -u punktfunk-scripting`; on Windows the runner task writes no\nlog file at all, so a failing plugin was diagnosable only by stopping the scheduled task and\nre-running the runner by hand. Both are shell access on the host box, which is exactly what the\nconsole exists to avoid.\n\nLines land in the same ring as the host's own, sharing one `seq` cursor, targeted\n`plugin:<source>` — so `GET /logs` needs no second cursor and the console needs no second poll.",
"operationId": "ingestPluginLogs",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PluginLogBatch"
}
}
},
"required": true
},
"responses": {
"204": {
"description": "Lines ingested"
},
"400": {
"description": "Batch too large",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/plugins/{id}": {
"put": {
"tags": [
@@ -3495,7 +3540,7 @@
"operationId": "forceUpdateCheck",
"responses": {
"200": {
"description": "Refreshed update-check state (`last_error` carries a failed check)",
"description": "Refreshed update-check state (`last_error` carries a failed check; `not_published` an empty channel, which is not one)",
"content": {
"application/json": {
"schema": {
@@ -6238,6 +6283,50 @@
"gamestream"
]
},
"PluginLogBatch": {
"type": "object",
"description": "A batch of runner log lines.",
"required": [
"entries"
],
"properties": {
"entries": {
"type": "array",
"items": {
"$ref": "#/components/schemas/PluginLogLine"
}
}
}
},
"PluginLogLine": {
"type": "object",
"description": "One log line produced by the runner or a plugin inside it (`POST /plugins/logs`).",
"required": [
"ts_ms",
"level",
"source",
"msg"
],
"properties": {
"level": {
"type": "string",
"description": "`ERROR` | `WARN` | `INFO` | `DEBUG` | `TRACE`. Anything else is coerced to `INFO`."
},
"msg": {
"type": "string"
},
"source": {
"type": "string",
"description": "Which unit emitted it — a plugin's `definePlugin` name, a package name, or `runner`.\nSurfaced in the console's target column as `plugin:<source>`."
},
"ts_ms": {
"type": "integer",
"format": "int64",
"description": "When the line was produced, unix milliseconds. Kept verbatim — see\n[`crate::log_capture::LogRing::push_remote`].",
"minimum": 0
}
}
},
"PluginRegistration": {
"type": "object",
"description": "Register/renew body for `PUT /plugins/{id}`.",
@@ -7372,7 +7461,8 @@
"apply",
"channel_hint",
"check_disabled",
"available"
"available",
"not_published"
],
"properties": {
"apply": {
@@ -7452,6 +7542,10 @@
}
]
},
"not_published": {
"type": "boolean",
"description": "The check reached the feed and found this channel has **no release published yet** —\nan expected state (a channel nobody has announced to answers with a 404), not a\nfailure. Mutually exclusive with `last_error`, so a UI can say \"nothing published yet\"\ninstead of painting an empty feed as a broken host. Never set once a manifest has been\nseen for this channel: a feed that loses a document it used to serve stays an error."
},
"opt_in_hint": {
"type": [
"string",
@@ -395,9 +395,21 @@ private fun buildSettingsRows(
"mic", null, "Microphone", "Send this device's microphone to the host's virtual mic.",
s.micEnabled,
) { update(s.copy(micEnabled = it)) },
toggle(
"echoCancel", null, "Echo cancellation",
"Filter the stream's own audio out of the mic pickup. Applies while the microphone is on.",
s.echoCancel,
) { update(s.copy(echoCancel = it)) },
toggle(
"padForward", "Controllers", "Forward controllers",
"Send this device's controllers to the host. Turn it off when your controller " +
"already reaches the host another way — USB passthrough such as VirtualHere — " +
"so games don't see two of them.",
s.gamepadForwarding,
) { update(s.copy(gamepadForwarding = it)) },
choice(
"padType", "Controllers", "Controller type",
"padType", null, "Controller type",
"The virtual pad the host creates — Automatic matches this controller.",
GAMEPAD_OPTIONS, s.gamepad,
) { update(s.copy(gamepad = it)) },
@@ -2,6 +2,7 @@ package io.unom.punktfunk
import android.content.Context
import android.os.Build
import android.util.Log
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.VideoDecoders
@@ -45,18 +46,40 @@ suspend fun connectToHost(
// Transport-level half of "Low-latency mode (experimental)" (DSCP marking on the media
// sockets) — must be applied before connect, since sockets are tagged at creation.
NativeBridge.nativeSetLowLatencyMode(settings.lowLatencyMode)
val multiSlice = VideoDecoders.multiSliceTolerant()
val partialFrame = VideoDecoders.partialFrameCapable()
// Slice-progressive delivery: decoder truth AND the async decode loop — the legacy
// sync loop feeds whole AUs only, so parts must never arrive when it is selected.
val frameParts = settings.lowLatencyMode && partialFrame
val codecBits = VideoDecoders.decodableCodecBits()
// Automatic codec (P5, measured NP3 ↔ RTX 4090): AV1 beat HEVC by ~1.2 ms end-to-end at
// identical conditions, so under "Automatic" this device prefers AV1 when it hardware-
// decodes it (the AV1 bit is only ever set for a real, non-blocked hardware decoder) AND
// it lacks FEATURE_PartialFrame — a partial-frame device keeps HEVC, whose slice overlap
// AV1 cannot ride (AV1 has no slices; the host's chunked poll never arms). The host
// honors the preference only inside the probed shared codec set, so an AV1-less encoder
// still resolves HEVC. An explicit user choice always wins unchanged.
val preferredCodec = settings.preferredCodec().takeIf { it != 0 }
?: if (codecBits and 4 != 0 && !partialFrame) 4 else 0
// The connect-time capability readout (`adb logcat -s pf.caps`): the P2 slice pipeline
// is client-inert unless BOTH probes pass — this line says which decoder failed one.
Log.i(
"pf.caps",
VideoDecoders.capsReport() +
" → multiSlice=$multiSlice parts=$frameParts prefer=$preferredCodec" +
" (lowLatency=${settings.lowLatencyMode})",
)
NativeBridge.nativeConnect(
host, port, w, h, hz,
identity.certPem, identity.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, VideoDecoders.multiSliceTolerant(),
// Slice-progressive delivery: decoder truth AND the async decode loop — the legacy
// sync loop feeds whole AUs only, so parts must never arrive when it is selected.
settings.lowLatencyMode && VideoDecoders.partialFrameCapable(),
hdrEnabled, multiSlice,
frameParts,
settings.audioChannels,
// What this device can decode (H.264|HEVC always, AV1 when a real decoder exists) +
// the user's soft codec preference — the host resolves the emitted codec from both.
VideoDecoders.decodableCodecBits(), settings.preferredCodec(), timeoutMs,
// the soft codec preference (user choice, or the Automatic AV1 rule above) — the
// host resolves the emitted codec from both.
codecBits, preferredCodec, timeoutMs,
launch,
// The host's approval-list / trust-store label for this device — the same
// Build.MODEL convention the pairing dialogs use for nativePair.
@@ -38,10 +38,12 @@ data class SettingsOverlay(
val compositor: Int? = null,
val audioChannels: Int? = null,
val micEnabled: Boolean? = null,
val echoCancel: Boolean? = null,
val touchMode: TouchMode? = null,
val mouseMode: MouseMode? = null,
val invertScroll: Boolean? = null,
val gamepad: Int? = null,
val gamepadForwarding: Boolean? = null,
val statsVerbosity: StatsVerbosity? = null,
/**
* Android-only tier-P addition (design §3): the decode pipeline is a device fact everywhere
@@ -70,10 +72,12 @@ data class SettingsOverlay(
compositor = compositor ?: base.compositor,
audioChannels = audioChannels ?: base.audioChannels,
micEnabled = micEnabled ?: base.micEnabled,
echoCancel = echoCancel ?: base.echoCancel,
touchMode = touchMode ?: base.touchMode,
mouseMode = mouseMode ?: base.mouseMode,
invertScroll = invertScroll ?: base.invertScroll,
gamepad = gamepad ?: base.gamepad,
gamepadForwarding = gamepadForwarding ?: base.gamepadForwarding,
statsVerbosity = statsVerbosity ?: base.statsVerbosity,
lowLatencyMode = lowLatencyMode ?: base.lowLatencyMode,
presentPriority = presentPriority ?: base.presentPriority,
@@ -103,10 +107,14 @@ data class SettingsOverlay(
compositor = if (after.compositor != before.compositor) after.compositor else compositor,
audioChannels = if (after.audioChannels != before.audioChannels) after.audioChannels else audioChannels,
micEnabled = if (after.micEnabled != before.micEnabled) after.micEnabled else micEnabled,
echoCancel = if (after.echoCancel != before.echoCancel) after.echoCancel else echoCancel,
touchMode = if (after.touchMode != before.touchMode) after.touchMode else touchMode,
mouseMode = if (after.mouseMode != before.mouseMode) after.mouseMode else mouseMode,
invertScroll = if (after.invertScroll != before.invertScroll) after.invertScroll else invertScroll,
gamepad = if (after.gamepad != before.gamepad) after.gamepad else gamepad,
gamepadForwarding =
if (after.gamepadForwarding != before.gamepadForwarding) after.gamepadForwarding
else gamepadForwarding,
statsVerbosity = if (after.statsVerbosity != before.statsVerbosity) after.statsVerbosity else statsVerbosity,
lowLatencyMode = if (after.lowLatencyMode != before.lowLatencyMode) after.lowLatencyMode else lowLatencyMode,
presentPriority = if (after.presentPriority != before.presentPriority) after.presentPriority else presentPriority,
@@ -128,10 +136,12 @@ data class SettingsOverlay(
"compositor" -> copy(compositor = null)
"audio_channels" -> copy(audioChannels = null)
"mic_enabled" -> copy(micEnabled = null)
"echo_cancel" -> copy(echoCancel = null)
"touch_mode" -> copy(touchMode = null)
"mouse_mode" -> copy(mouseMode = null)
"invert_scroll" -> copy(invertScroll = null)
"gamepad" -> copy(gamepad = null)
"gamepad_forwarding" -> copy(gamepadForwarding = null)
"stats_verbosity" -> copy(statsVerbosity = null)
"low_latency_mode" -> copy(lowLatencyMode = null)
"present_priority" -> copy(presentPriority = null)
@@ -150,10 +160,12 @@ data class SettingsOverlay(
if (compositor != null) add("compositor")
if (audioChannels != null) add("audio_channels")
if (micEnabled != null) add("mic_enabled")
if (echoCancel != null) add("echo_cancel")
if (touchMode != null) add("touch_mode")
if (mouseMode != null) add("mouse_mode")
if (invertScroll != null) add("invert_scroll")
if (gamepad != null) add("gamepad")
if (gamepadForwarding != null) add("gamepad_forwarding")
if (statsVerbosity != null) add("stats_verbosity")
if (lowLatencyMode != null) add("low_latency_mode")
if (presentPriority != null) add("present_priority")
@@ -180,10 +192,12 @@ data class SettingsOverlay(
compositor?.let { j.put("compositor", it) }
audioChannels?.let { j.put("audio_channels", it) }
micEnabled?.let { j.put("mic_enabled", it) }
echoCancel?.let { j.put("echo_cancel", it) }
touchMode?.let { j.put("touch_mode", it.name) }
mouseMode?.let { j.put("mouse_mode", it.storedName) }
invertScroll?.let { j.put("invert_scroll", it) }
gamepad?.let { j.put("gamepad", it) }
gamepadForwarding?.let { j.put("gamepad_forwarding", it) }
statsVerbosity?.let { j.put("stats_verbosity", it.name) }
lowLatencyMode?.let { j.put("low_latency_mode", it) }
presentPriority?.let { j.put("present_priority", it) }
@@ -198,9 +212,10 @@ data class SettingsOverlay(
/** Keys this build models; everything else in a stored overlay is carried through. */
private val KNOWN = setOf(
"width", "height", "refresh_hz", "bitrate_kbps", "render_scale", "codec",
"hdr_enabled", "compositor", "audio_channels", "mic_enabled", "touch_mode",
"mouse_mode", "invert_scroll", "gamepad", "stats_verbosity", "low_latency_mode",
"present_priority", "smooth_buffer",
"hdr_enabled", "compositor", "audio_channels", "mic_enabled", "echo_cancel",
"touch_mode", "mouse_mode", "invert_scroll", "gamepad", "gamepad_forwarding",
"stats_verbosity",
"low_latency_mode", "present_priority", "smooth_buffer",
)
internal fun fromJson(j: JSONObject): SettingsOverlay = SettingsOverlay(
@@ -214,12 +229,14 @@ data class SettingsOverlay(
compositor = j.optIntOrNull("compositor"),
audioChannels = j.optIntOrNull("audio_channels"),
micEnabled = j.optBooleanOrNull("mic_enabled"),
echoCancel = j.optBooleanOrNull("echo_cancel"),
touchMode = j.optStringOrNull("touch_mode")
?.let { n -> TouchMode.entries.firstOrNull { it.name == n } },
mouseMode = j.optStringOrNull("mouse_mode")
?.let { n -> MouseMode.entries.firstOrNull { it.storedName == n } },
invertScroll = j.optBooleanOrNull("invert_scroll"),
gamepad = j.optIntOrNull("gamepad"),
gamepadForwarding = j.optBooleanOrNull("gamepad_forwarding"),
statsVerbosity = j.optStringOrNull("stats_verbosity")
?.let { n -> StatsVerbosity.entries.firstOrNull { it.name == n } },
lowLatencyMode = j.optBooleanOrNull("low_latency_mode"),
@@ -1,6 +1,7 @@
package io.unom.punktfunk
import android.content.Context
import android.hardware.display.DisplayManager
import android.os.Build
import android.util.Log
import android.view.Display
@@ -33,6 +34,17 @@ data class Settings(
val hdrEnabled: Boolean = true,
val compositor: Int = 0,
val gamepad: Int = 0,
/**
* Forward this device's controllers to the host at all. Default on — that was the
* unconditional behaviour before this became a setting.
*
* Off is for a couch whose controller reaches the host another way: a USB passthrough tool
* (VirtualHere and friends), or a pad simply plugged into the host itself. Leaving it on
* there gives the host two controllers for one pair of hands, and games read both. It also
* stops this device CLAIMING the pad — a device held open is one a passthrough tool can't
* bind — which is why it gates the USB capture paths, not just the wire sends.
*/
val gamepadForwarding: Boolean = true,
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
@@ -41,6 +53,15 @@ data class Settings(
* the host resolves (AV1 is only advertised/offered when the device has a real AV1 decoder). */
val codec: String = "auto",
val micEnabled: Boolean = false,
/**
* Cancel acoustic echo on the mic uplink (plus noise suppression): the capture opens under
* the VoiceCommunication preset so the HAL's own AEC/NS process it, with the Java effects
* attached as a backstop where available. On by default — a phone/tablet plays the game audio
* out of the same device its mic hears, so without this the host hears its own stream back.
* Turn off for a headset-only setup where the untouched full-band capture sounds better.
* Only meaningful while [micEnabled] is on.
*/
val echoCancel: Boolean = true,
/**
* How much the in-stream stats overlay shows — see [StatsVerbosity]. Defaults to
* [StatsVerbosity.NORMAL] (the res/fps line + latency headline + reliability counters); the full
@@ -206,9 +227,11 @@ class SettingsStore(context: Context) {
hdrEnabled = prefs.getBoolean(K_HDR, true),
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
gamepadForwarding = prefs.getBoolean(K_GAMEPAD_FORWARDING, true),
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
codec = prefs.getString(K_CODEC, "auto") ?: "auto",
micEnabled = prefs.getBoolean(K_MIC, false),
echoCancel = prefs.getBoolean(K_ECHO_CANCEL, true),
statsVerbosity = prefs.getString(K_STATS_VERBOSITY, null)
?.let { name -> StatsVerbosity.entries.firstOrNull { it.name == name } }
// Migration from the pre-tier Boolean "stats_hud_enabled": an explicit OFF stays off;
@@ -251,9 +274,11 @@ class SettingsStore(context: Context) {
.putBoolean(K_HDR, s.hdrEnabled)
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
.putBoolean(K_GAMEPAD_FORWARDING, s.gamepadForwarding)
.putInt(K_AUDIO_CH, s.audioChannels)
.putString(K_CODEC, s.codec)
.putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_ECHO_CANCEL, s.echoCancel)
.putString(K_STATS_VERBOSITY, s.statsVerbosity.name)
.putString(K_TOUCH_MODE, s.touchMode.name)
.putBoolean(K_GAMEPAD_UI, s.gamepadUiEnabled)
@@ -279,9 +304,11 @@ class SettingsStore(context: Context) {
const val K_HDR = "hdr_enabled"
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
const val K_GAMEPAD_FORWARDING = "gamepad_forwarding"
const val K_AUDIO_CH = "audio_channels"
const val K_CODEC = "codec"
const val K_MIC = "mic_enabled"
const val K_ECHO_CANCEL = "echo_cancel"
const val K_STATS_VERBOSITY = "stats_verbosity"
/** Pre-tier Boolean the [K_STATS_VERBOSITY] enum replaced — read once for migration, never
@@ -319,14 +346,31 @@ class SettingsStore(context: Context) {
}
}
/**
* The display to probe for capability/mode queries: the context's own display when it is already
* associated with one, else the DEFAULT display via [DisplayManager]. A `punktfunk://` deep-link
* COLD start can reach the connect before the activity is attached to its display —
* `context.display` then throws, and the old `false`/1080p60 fallbacks silently downgraded the
* whole session (no HDR advertised / non-native mode) with nothing in the log. The default
* display IS the panel on phones and TVs; the activity-display distinction only matters on
* multi-display setups, where the attached path still wins whenever it is available.
*/
private fun probeDisplay(context: Context): Display? =
runCatching { context.display }.getOrNull()
?: runCatching {
context.getSystemService(DisplayManager::class.java)
?.getDisplay(Display.DEFAULT_DISPLAY)
}.getOrNull().also {
if (it != null) Log.i("punktfunk", "display probe: context unattached — using DEFAULT_DISPLAY")
}
/**
* The device's native display mode as a landscape `(width, height, hz)` — the long edge is the
* width, since we stream a desktop. Falls back to 1920×1080@60 if the display can't be read.
* [context] must be a visual (Activity) context.
* width, since we stream a desktop. Falls back to 1920×1080@60 if no display can be read at all
* (see [probeDisplay] for the cold-start fallback that makes that a last resort).
*/
fun nativeDisplayMode(context: Context): Triple<Int, Int, Int> {
// getDisplay() throws on a non-visual context rather than returning null — guard it.
val display = runCatching { context.display }.getOrNull() ?: return Triple(1920, 1080, 60)
val display = probeDisplay(context) ?: return Triple(1920, 1080, 60)
val mode = display.mode
val w = mode.physicalWidth
val h = mode.physicalHeight
@@ -341,7 +385,12 @@ fun nativeDisplayMode(context: Context): Triple<Int, Int, Int> {
* capability gate the Apple/Windows clients apply.
*/
fun displaySupportsHdr(context: Context): Boolean {
val display = runCatching { context.display }.getOrNull() ?: return false
val display = probeDisplay(context)
if (display == null) {
// Distinguishable from a real SDR verdict — a silent `false` here cost an HDR session.
Log.w("punktfunk", "display HDR probe: no display reachable — advertising SDR")
return false
}
val types = buildSet {
// API 34+: the sanctioned per-mode query (Display.Mode.getSupportedHdrTypes). The
// deprecated Display-level hdrCapabilities can return EMPTY on Android 14+ devices
@@ -673,12 +673,22 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
// Only codecs this device can actually decode are offered — a preference the client never
// advertises would be a dead setting (see [codecOptionsFor]).
val av1Capable = remember { VideoDecoders.pickDecoder("video/av01") != null }
// Mirror the Automatic AV1 rule in HostConnect (hardware AV1 AND no partial-frame
// support) so the picker says what "Automatic" actually does on THIS device.
val autoPrefersAv1 = remember {
VideoDecoders.decodableCodecBits() and 4 != 0 && !VideoDecoders.partialFrameCapable()
}
SettingDropdown(
label = "Video codec",
options = codecOptionsFor(s.codec, av1Capable),
selected = s.codec,
field = "codec",
caption = "A preference — the host falls back if it can't encode this one.",
caption = if (autoPrefersAv1) {
"A preference — the host falls back if it can't encode this one. " +
"Automatic prefers AV1 on this device."
} else {
"A preference — the host falls back if it can't encode this one."
},
) { c -> update(s.copy(codec = c)) }
// HDR is only meaningful on a panel that can present HDR10; on an SDR display the toggle is
@@ -794,17 +804,37 @@ private fun AudioSettings(s: Settings, update: (Settings) -> Unit, onMicChange:
field = "mic_enabled",
onCheckedChange = onMicChange,
)
ToggleRow(
title = "Echo cancellation",
subtitle = "Filters the stream's own audio out of the mic pickup",
checked = s.echoCancel,
enabled = s.micEnabled,
field = "echo_cancel",
onCheckedChange = { on -> update(s.copy(echoCancel = on)) },
)
}
}
@Composable
private fun ControllerSettings(s: Settings, update: (Settings) -> Unit, onOpenControllers: () -> Unit) {
SettingsGroup(footer = "Applies from the next session.") {
// The master switch, above everything it governs. Profileable, so it shows in both
// scopes: a "Work" profile can decline to forward what "Game" forwards.
ToggleRow(
title = "Forward controllers",
subtitle = "Send this device's controllers to the host. Turn it off when your " +
"controller already reaches the host another way — USB passthrough such as " +
"VirtualHere, or a pad plugged into the host — so games don't see two of them",
checked = s.gamepadForwarding,
field = "gamepad_forwarding",
onCheckedChange = { on -> update(s.copy(gamepadForwarding = on)) },
)
SettingDropdown(
label = "Controller type",
options = GAMEPAD_OPTIONS,
selected = s.gamepad,
field = "gamepad",
enabled = s.gamepadForwarding,
caption = "The virtual pad the host creates. Automatic matches your controller; " +
"every connected one is forwarded as its own player.",
) { g -> update(s.copy(gamepad = g)) }
@@ -834,6 +864,7 @@ private fun ControllerSettings(s: Settings, update: (Settings) -> Unit, onOpenCo
subtitle = "Stream a Steam Controller 2 as-is — Steam on the host drives its " +
"trackpads, gyro and haptics directly",
checked = s.sc2Capture,
enabled = s.gamepadForwarding,
onCheckedChange = { on -> update(s.copy(sc2Capture = on)) },
)
// Same no-vibrator-gate reasoning as the SC2 row: this capture renders feedback on
@@ -843,6 +874,7 @@ private fun ControllerSettings(s: Settings, update: (Settings) -> Unit, onOpenCo
subtitle = "Drive a USB-connected Sony pad directly — rumble on any phone, " +
"plus adaptive triggers, lightbar and gyro",
checked = s.dsCapture,
enabled = s.gamepadForwarding,
onCheckedChange = { on -> update(s.copy(dsCapture = on)) },
)
}
@@ -995,6 +1027,7 @@ private fun <T> SettingDropdown(
selected: T,
field: String? = null,
caption: String? = null,
enabled: Boolean = true,
onSelect: (T) -> Unit,
) {
var expanded by remember { mutableStateOf(false) }
@@ -1002,18 +1035,25 @@ private fun <T> SettingDropdown(
?: options.firstOrNull()?.second.orEmpty()
Column {
OverrideBadge(field)
ExposedDropdownMenuBox(expanded = expanded, onExpandedChange = { expanded = it }) {
ExposedDropdownMenuBox(
expanded = expanded && enabled,
onExpandedChange = { if (enabled) expanded = it },
) {
OutlinedTextField(
value = selectedLabel,
onValueChange = {},
readOnly = true,
enabled = enabled,
label = { Text(label) },
trailingIcon = { ExposedDropdownMenuDefaults.TrailingIcon(expanded = expanded) },
modifier = Modifier
.menuAnchor(ExposedDropdownMenuAnchorType.PrimaryNotEditable)
.fillMaxWidth(),
)
ExposedDropdownMenu(expanded = expanded, onDismissRequest = { expanded = false }) {
ExposedDropdownMenu(
expanded = expanded && enabled,
onDismissRequest = { expanded = false },
) {
options.forEach { (value, lbl) ->
DropdownMenuItem(
text = { Text(lbl) },
@@ -18,11 +18,13 @@ import kotlin.math.roundToInt
* The live stats overlay — the unified HUD (`design/stats-unification.md`): headline is
* `capture→displayed` tiled by `host+network` + `decode` + `display` when the platform delivered
* OnFrameRendered render callbacks this window (`dispValid`), falling back to the v1
* `capture→decoded` headline without the `display` term when it didn't. Reads the 26-double
* layout from [NativeBridge.nativeVideoStats]:
* `capture→decoded` headline without the `display` term when it didn't. Reads the 33-double
* layout from [NativeBridge.nativeVideoStats] (that KDoc is the authoritative index list):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lostTotal, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms, lost, skipped,
* fec, frames, dispValid, displayP50Ms, e2eDispP50Ms, e2eDispP95Ms]`.
* fec, frames, dispValid, displayP50Ms, e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms,
* presentsWindow, presenterActive, feedP50Ms, codecP50Ms, skippedOverflowWindow]`. Every read
* is length-guarded, so an older native lib simply omits the lines it can't feed.
*
* [verbosity] selects how many lines render (each tier a superset of the last — see
* [StatsVerbosity]):
@@ -129,10 +131,30 @@ internal fun StatsOverlay(
} else {
""
}
// P3 decode split (s[30]/s[31]): `feed` = received→queued (hand-off + input-slot
// wait) + `codec` = queued→decoded (codec-pure) — rendered when a sample landed.
val decodeTerm = if (s.size >= 33 && (s[30] > 0 || s[31] > 0)) {
"decode ${"%.1f".format(s[15])} " +
"(feed ${"%.1f".format(s[30])} + codec ${"%.1f".format(s[31])})"
} else {
"decode ${"%.1f".format(s[15])}"
}
statLine(
"= $hostTerms + decode ${"%.1f".format(s[15])}$displayTerm$presents",
"= $hostTerms + $decodeTerm$displayTerm$presents",
Color.White,
)
// Metric fairness: the Apple client's HUD shaves ~2 refresh periods of OS
// pipeline floor off its shown display/end-to-end; Android shows raw. This twin
// applies the same shave so iPhone↔Android HUD numbers compare directly.
if (dispValid && hz > 0) {
val shave = 2000.0 / hz
statLine(
"≈ Apple-HUD equiv: end-to-end " +
"${"%.1f".format((s[24] - shave).coerceAtLeast(0.0))} · display " +
"${"%.1f".format((s[23] - shave).coerceAtLeast(0.0))} (2 refresh)",
Color(0xFFA8D8B8),
)
}
}
}
counterLine(s, lost)?.let { statLine(it, Color(0xFFFFB0B0)) }
@@ -178,12 +200,17 @@ private fun counterLine(s: DoubleArray, lostTotal: Long): String? {
val fec = s[20].toLong()
val frames = s[21].toLong()
if (lost == 0L && skipped == 0L && fec == 0L) return null
// The overflow subset of `skipped` (s[32]): whole AUs dropped before feeding — the decoder
// fell behind. Absent (0 / old layout) the plain count keeps meaning benign pacing drops.
val overflow = if (s.size >= 33) s[32].toLong() else 0L
return buildList {
if (lost > 0) {
val pct = 100.0 * lost / (frames + lost).coerceAtLeast(1)
add("lost $lost (${"%.1f".format(pct)}%)")
}
if (skipped > 0) add("skipped $skipped")
if (skipped > 0) {
add(if (overflow > 0) "skipped $skipped (⚠ $overflow overflow)" else "skipped $skipped")
}
if (fec > 0) add("FEC $fec")
}.joinToString(" · ")
}
@@ -9,11 +9,15 @@ import android.content.IntentFilter
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.hardware.usb.UsbManager
import android.media.audiofx.AcousticEchoCanceler
import android.media.audiofx.AudioEffect
import android.media.audiofx.NoiseSuppressor
import android.net.wifi.WifiManager
import android.os.Build
import android.text.InputType
import android.util.Log
import android.view.KeyEvent
import android.view.Surface
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.View
@@ -25,12 +29,20 @@ import android.view.inputmethod.InputMethodManager
import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.aspectRatio
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.MicOff
import androidx.compose.material3.Icon
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
@@ -41,6 +53,7 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.platform.LocalContext
@@ -97,6 +110,38 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
Manifest.permission.RECORD_AUDIO,
) == PackageManager.PERMISSION_GRANTED
// The Java AEC/NS pair backstopping the native VoiceCommunication capture preset, hung off the
// audio session id `nativeStartMic` returns. Attached in surfaceCreated (where the mic starts)
// and released on every path that stops the mic — the surface teardown AND the final dispose —
// so a surface recreate re-attaches to the fresh stream instead of leaking effect engines.
// All three touch points run on the main thread; a plain list is race-free.
val micEffects = remember { mutableListOf<AudioEffect>() }
// In-stream mic mute. Per SESSION and never persisted (no setting backs it): a new stream
// always starts unmuted. The authoritative flag lives on the native handle, which is why a mute
// survives the mic stop/start a surface recreate performs — this state is the UI's mirror of
// it, and survives the same recreate because the composition outlives the surface.
var micMuted by remember(handle) { mutableStateOf(false) }
// Whether a capture is actually RUNNING, not merely wanted — set from surfaceCreated on what
// nativeMicActive reports. A device that refused every AAudio input rung gets no mute control
// rather than one that lies about a mic being heard.
var micRunning by remember(handle) { mutableStateOf(false) }
// Transient confirmation of a mic-chord toggle (null = nothing showing). Only the gamepad path
// needs it: the touch button confirms itself by changing under the finger, but a chord has no
// on-screen state of its own, and "did that register?" is exactly the doubt to answer.
var micHint by remember { mutableStateOf<String?>(null) }
LaunchedEffect(micHint) {
if (micHint != null) {
delay(1600)
micHint = null
}
}
// The one place mute is toggled — Compose state + the native flag, always together.
val setMicMuted = { muted: Boolean ->
micMuted = muted
NativeBridge.nativeSetMicMuted(handle, muted)
}
// Live decode stats for the HUD. `statsOn` (verbosity != OFF) gates the whole native pipeline:
// the per-frame sampling (nativeSetVideoStatsEnabled — a hidden HUD costs one atomic load per
// frame) AND the 1 s poll loop, which only runs while the overlay is visible. Enabling resets
@@ -276,7 +321,9 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
// Multi-controller router: a stable wire pad index per connected controller, per-device axis
// state, Arrival/Remove on hot-plug, and feedback routed back by pad index. Forwards every
// controller (Automatic). Built here, released on dispose.
val router = GamepadRouter(context, handle, initialSettings.gamepad)
val router = GamepadRouter(
context, handle, initialSettings.gamepad, initialSettings.gamepadForwarding,
)
activity?.gamepadRouter = router
// Select+Start+L1+R1 chord leaves the stream — a deliberate quit (signal it so the host skips
// the keep-alive linger), unlike a host-ended / backgrounded drop. The router debounces it
@@ -287,6 +334,16 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed }
// Select + Y toggles the mic — the couch reach for the on-screen mute button, which a
// gamepad/TV user has no pointer for. Ignored when no capture is running (there is nothing
// to mute, and claiming otherwise would be the lie the control exists to avoid).
router.onMicChord = {
if (micRunning) {
val next = !micMuted
setMicMuted(next)
micHint = if (next) "Microphone muted" else "Microphone live"
}
}
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
@@ -387,7 +444,11 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
// The menu-time capture (UI navigation) must let go before the stream-mode capture can
// claim the interfaces; it resumes in onDispose once the stream releases them.
activity?.stopSc2MenuNav()
val sc2 = if (initialSettings.sc2Capture) Sc2Capture(context, router) else null
val sc2 = if (initialSettings.sc2Capture && initialSettings.gamepadForwarding) {
Sc2Capture(context, router)
} else {
null
}
var sc2UsbReceiver: BroadcastReceiver? = null
if (sc2 != null) {
feedback.onHidRaw = sc2::onHidRaw
@@ -437,7 +498,11 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
// the automatic fallback. Host feedback routes back through feedback.sink; the claim
// frees the pad's InputDevice slot itself (see DsCapture.startUsb), so the wire index
// hands over deterministically.
val ds = if (initialSettings.dsCapture) DsCapture(context, router) else null
val ds = if (initialSettings.dsCapture && initialSettings.gamepadForwarding) {
DsCapture(context, router)
} else {
null
}
var dsUsbReceiver: BroadcastReceiver? = null
if (ds != null) {
feedback.sink = ds
@@ -483,6 +548,7 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
dsUsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
ds?.stop() // rumble-stop on the physical pad + release the USB link + free the wire slot
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.onMicChord = null // same: no mute toggle on buttons released during teardown
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
@@ -519,6 +585,7 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
activity?.requestedOrientation =
priorOrientation ?: ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
// Leaving the stream: stop the mic + audio + decode threads and tear down the session.
releaseMicEffects(micEffects)
NativeBridge.nativeStopMic(handle)
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
@@ -609,10 +676,38 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
.roundToInt(),
)
NativeBridge.nativeStartAudio(handle, lowLatencyMode)
if (micWanted) NativeBridge.nativeStartMic(handle)
if (micWanted) {
val sessionId =
NativeBridge.nativeStartMic(handle, initialSettings.echoCancel)
if (initialSettings.echoCancel) {
attachMicEffects(sessionId, micEffects)
}
// Did a capture actually open? That — not the setting — is what
// puts the mute control on screen. A restart after a surface
// recreate comes back already muted if the user muted: the flag
// lives on the session handle, so nothing has to be re-applied.
micRunning = NativeBridge.nativeMicActive(handle)
}
}
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {}
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
// Re-assert the frame-rate vote: a buffer-geometry change can reset
// the surface's frame-rate setting on some OEM builds, silently
// dropping the 120 Hz pin mid-stream. Mirrors the native hint's
// policy (FIXED_SOURCE; ALWAYS only on the TV low-latency path —
// phones stay seamless so a re-hint can never force a mode flicker).
if (streamHz > 0) runCatching {
holder.surface.setFrameRate(
streamHz.toFloat(),
Surface.FRAME_RATE_COMPATIBILITY_FIXED_SOURCE,
if (isTv && lowLatencyMode) {
Surface.CHANGE_FRAME_RATE_ALWAYS
} else {
Surface.CHANGE_FRAME_RATE_ONLY_IF_SEAMLESS
},
)
}
}
override fun surfaceDestroyed(holder: SurfaceHolder) {
// Surface gone (backgrounding, or on the way out). Stop the threads that
@@ -620,7 +715,12 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
// DisposableEffect has closed it, the handle is freed; dereferencing it
// here is the use-after-free that crashed on back-navigation.
if (!closed.get()) {
releaseMicEffects(micEffects)
NativeBridge.nativeStopMic(handle)
// No capture, no control — but the MUTE state is deliberately left
// standing (native keeps it on the handle), so the restart in
// surfaceCreated brings the user's choice back with it.
micRunning = false
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
}
@@ -695,9 +795,106 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
}
},
)
// Mic mute, LAST in the stack — the one in-stream control, so unlike the purely visual
// overlays above it has to sit on top of the gesture layer to receive its own taps (it
// costs the stream that small corner of touch area, which is why it exists only while a
// capture actually runs). On TV it is the indicator alone: the Select + Y chord is the
// control there, and a focusable button would fight the game for the D-pad.
if (micRunning && (micMuted || !isTv)) {
MicMuteControl(
muted = micMuted,
onToggle = if (isTv) null else ({ setMicMuted(!micMuted) }),
modifier = Modifier.align(Alignment.TopEnd).padding(12.dp),
)
}
// Chord confirmation (gamepad/TV) — the counterpart to the button changing under a finger.
micHint?.let { MicChordHint(it, Modifier.align(Alignment.TopCenter).padding(top = 16.dp)) }
}
}
/**
* Attach the Java echo-canceller + noise-suppressor pair to the mic stream's audio session — the
* backstop for HALs whose VoiceCommunication capture path doesn't cancel on its own (the native
* side already opened the stream under that preset). [sessionId] `<= 0` means native allocated no
* session (echo cancellation off, or the preset fell back to the plain open), so there is nothing
* to hang an effect on. Created effects land in [into] for [releaseMicEffects]; `create()`
* returning null (unsupported / claimed) is quietly nothing — the HAL preset still does its part.
* Needs no extra permission: the effect APIs attach to our own recording session.
*/
private fun attachMicEffects(sessionId: Int, into: MutableList<AudioEffect>) {
if (sessionId <= 0) return
if (AcousticEchoCanceler.isAvailable()) {
AcousticEchoCanceler.create(sessionId)?.let { it.setEnabled(true); into.add(it) }
}
if (NoiseSuppressor.isAvailable()) {
NoiseSuppressor.create(sessionId)?.let { it.setEnabled(true); into.add(it) }
}
}
/** Release every attached mic effect engine. Idempotent — the list is cleared, and both stop
* paths (surface teardown, final dispose) may call it in either order. */
private fun releaseMicEffects(effects: MutableList<AudioEffect>) {
effects.forEach { runCatching { it.release() } }
effects.clear()
}
/**
* The in-stream mic control and its muted indicator, in one element: a dim mic glyph while the
* uplink is live, a red **Muted** badge while it isn't — so the state that matters is the loud one,
* readable at couch distance and impossible to mistake for the stream's own picture.
*
* [onToggle] `null` makes it a pure indicator (the TV/gamepad surface, where the Select + Y chord
* is the control); non-null makes the badge itself the touch target. Rendering it at all is the
* caller's decision — it means a capture is genuinely running.
*/
@Composable
private fun MicMuteControl(muted: Boolean, onToggle: (() -> Unit)?, modifier: Modifier = Modifier) {
val shape = RoundedCornerShape(10.dp)
Row(
modifier = modifier
.clip(shape)
.background(if (muted) Color(0xE0B3261E) else Color.Black.copy(alpha = 0.45f))
.then(if (onToggle != null) Modifier.clickable(onClick = onToggle) else Modifier)
.padding(horizontal = 12.dp, vertical = 10.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Icon(
imageVector = if (muted) Icons.Filled.MicOff else Icons.Filled.Mic,
// Spoken state first, then the action — a talkback user needs to know they are muted
// before they need to know how to stop being muted.
contentDescription = if (muted) {
"Microphone muted. Activate to unmute."
} else {
"Microphone live. Activate to mute."
},
tint = Color.White,
modifier = Modifier.size(20.dp),
)
if (muted) {
Spacer(Modifier.width(6.dp))
Text("Muted", color = Color.White, fontSize = 14.sp)
}
}
}
/**
* Transient confirmation that the mic chord (Select + Y) registered. The badge above already says
* *muted*, but nothing on screen says *un*muted — and "did that press do anything?" is the whole
* doubt a chord with no button under the finger creates. Same pill vocabulary as the other
* in-stream cues; the caller clears it after a beat.
*/
@Composable
private fun MicChordHint(text: String, modifier: Modifier = Modifier) {
Text(
text,
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/**
* The "hold to quit" cue shown while the gamepad exit chord (Select + Start + L1 + R1) is held. The
* chord no longer quits on a quick press — the router debounces it on a ~1 s hold — so this confirms
@@ -105,8 +105,20 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, styl
NativeBridge.nativeSendTouch(handle, it, 2, 0, 0, sw, sh)
}
c.positionChanged() ->
ids[c.id]?.let {
NativeBridge.nativeSendTouch(handle, it, 1, x, y, sw, sh)
ids[c.id]?.let { id ->
// Batched MotionEvents coalesce intermediate points into the
// historical list — forward them in order so a fast swipe keeps
// its real curvature on the host (usually empty during a stream:
// unbuffered dispatch is requested, so this costs nothing).
for (hs in c.historical) {
NativeBridge.nativeSendTouch(
handle, id, 1,
hs.position.x.roundToInt().coerceIn(0, sw - 1),
hs.position.y.roundToInt().coerceIn(0, sh - 1),
sw, sh,
)
}
NativeBridge.nativeSendTouch(handle, id, 1, x, y, sw, sh)
}
}
c.consume()
@@ -289,7 +301,10 @@ internal suspend fun PointerInputScope.streamTouchInput(
accY -= outY
}
} else {
moveAbs(p.position.x, p.position.y) // direct: cursor follows the finger
// Direct: cursor follows the finger — historical points first (batched
// MotionEvent samples), so the host cursor traces the finger's real path.
for (hs in p.historical) moveAbs(hs.position.x, hs.position.y)
moveAbs(p.position.x, p.position.y)
}
}
ev.changes.forEach { it.consume() }
+95 -16
View File
@@ -6,13 +6,23 @@ Why hand-rolled: stdlib + `openssl` only (no pip on the runner), and it prints G
error at the stage it fails instead of a catch-all. Reuses the SERVICE_ACCOUNT_JSON secret and
tolerates it being raw JSON *or* base64-encoded JSON.
Usage:
Usage (upload a new build):
SERVICE_ACCOUNT_JSON='<raw-or-base64 SA key>' \
python3 play-upload.py --package io.unom.punktfunk \
--aab path/to/app-release.aab --track internal --status completed [--no-commit]
--no-commit: do insert -> upload -> track-update -> validate, then delete the edit (publishes
nothing). Use it to dry-run the credentials/AAB without touching the live track.
Usage (promote a build that is already on Play, no rebuild):
python3 play-upload.py --package io.unom.punktfunk \
--promote 10816 --promote-from alpha --track production
Promotion assigns an EXISTING versionCode to another track, so what ships to production is the
byte-identical artifact the testers ran — rebuilding would burn a fresh versionCode and ship
something nobody has tested. --promote-from additionally asserts the code really is on that track
(catches a typo'd versionCode before it reaches production) and empties it in the SAME edit, so
the move is atomic: testers are never left pinned to a code that production also serves.
--no-commit: do insert -> upload/assign -> track-update -> validate, then delete the edit
(publishes nothing). Use it to dry-run the credentials/AAB/notes without touching the live track.
"""
import argparse, base64, json, os, subprocess, sys, tempfile, time
import urllib.request, urllib.parse, urllib.error
@@ -104,18 +114,79 @@ def access_token(sa) -> str:
return tok["access_token"]
# Play's "What's new" is capped at 500 characters per language. The cap lives in the Console
# (the REST reference does not state it) and the API rejects longer text at commit — i.e. AFTER
# the AAB has uploaded — so check it up front and print the actual count.
NOTES_MAX = 500
def load_release_notes(path, language):
with open(path, encoding="utf-8") as f:
text = f.read().strip()
if not text:
sys.exit(f"ERROR: release-notes file is empty: {path}")
if len(text) > NOTES_MAX:
sys.exit(f"ERROR: release notes are {len(text)} chars, Play allows {NOTES_MAX}: {path}")
print(f"release notes: {len(text)}/{NOTES_MAX} chars ({language})")
return [{"language": language, "text": text}]
def put_track(app, edit, tok, track, version_codes, status, user_fraction=None, notes=None):
"""PUT one track. An empty version_codes list clears the track (what promotion does to the
track it promoted OUT of)."""
release = {"status": status, "versionCodes": [str(v) for v in version_codes]}
if user_fraction is not None:
release["userFraction"] = user_fraction
if notes:
release["releaseNotes"] = notes
# Clearing a track means "no active releases", not "an empty release".
body = {"track": track, "releases": [release] if version_codes else []}
call("PUT", f"{app}/edits/{edit}/tracks/{track}", token=tok,
data=json.dumps(body).encode(), content_type="application/json")
def assert_on_track(app, edit, tok, track, vc):
"""Fail before anything is written if --promote names a versionCode that is not actually on
the track we claim to be promoting out of."""
got = call("GET", f"{app}/edits/{edit}/tracks/{track}", token=tok)
live = [c for r in got.get("releases", []) for c in r.get("versionCodes", [])]
if str(vc) not in live:
sys.exit(f"ERROR: versionCode {vc} is not on track '{track}' (it has: {live or 'nothing'})")
print(f"verified versionCode={vc} is live on '{track}'")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--package", required=True)
ap.add_argument("--aab", required=True)
ap.add_argument("--aab", help="upload this bundle (mutually exclusive with --promote)")
ap.add_argument("--promote", type=int, metavar="VERSIONCODE",
help="assign an already-uploaded versionCode instead of uploading")
ap.add_argument("--promote-from", metavar="TRACK",
help="with --promote: assert the code is on TRACK, then clear TRACK")
ap.add_argument("--track", default="internal")
ap.add_argument("--status", default="completed")
ap.add_argument("--user-fraction", type=float,
help="staged rollout fraction, 0<f<1; required by --status inProgress")
ap.add_argument("--release-notes-file", help="Play 'What's new' text (<=500 chars)")
ap.add_argument("--release-notes-language", default="en-US")
ap.add_argument("--no-commit", action="store_true")
a = ap.parse_args()
if not os.path.isfile(a.aab):
if bool(a.aab) == bool(a.promote):
sys.exit("ERROR: pass exactly one of --aab (upload) or --promote (assign an existing code)")
if a.promote_from and not a.promote:
sys.exit("ERROR: --promote-from only applies to --promote")
# inProgress without a fraction is an API error; halted/completed with one is also rejected.
if a.status == "inProgress" and a.user_fraction is None:
sys.exit("ERROR: --status inProgress requires --user-fraction")
if a.user_fraction is not None and not (0 < a.user_fraction < 1):
sys.exit(f"ERROR: --user-fraction must be strictly between 0 and 1 (got {a.user_fraction})")
if a.aab and not os.path.isfile(a.aab):
sys.exit(f"ERROR: AAB not found: {a.aab}")
notes = load_release_notes(a.release_notes_file, a.release_notes_language) \
if a.release_notes_file else None
sa = load_sa()
tok = access_token(sa)
print(f"authenticated as {sa['client_email']} (project {sa.get('project_id')})")
@@ -123,17 +194,25 @@ def main():
try:
edit = call("POST", f"{app}/edits", token=tok)["id"]
with open(a.aab, "rb") as f:
blob = f.read()
print(f"uploading {a.aab} ({len(blob)} bytes) ...")
vc = call("POST", f"{UPLOAD}/{a.package}/edits/{edit}/bundles?uploadType=media",
token=tok, data=blob, content_type="application/octet-stream")["versionCode"]
print(f"uploaded versionCode={vc}")
call("PUT", f"{app}/edits/{edit}/tracks/{a.track}", token=tok,
data=json.dumps({"track": a.track,
"releases": [{"status": a.status, "versionCodes": [str(vc)]}]}).encode(),
content_type="application/json")
print(f"assigned versionCode={vc} -> track={a.track} status={a.status}")
if a.promote:
vc = a.promote
if a.promote_from:
assert_on_track(app, edit, tok, a.promote_from, vc)
else:
with open(a.aab, "rb") as f:
blob = f.read()
print(f"uploading {a.aab} ({len(blob)} bytes) ...")
vc = call("POST", f"{UPLOAD}/{a.package}/edits/{edit}/bundles?uploadType=media",
token=tok, data=blob, content_type="application/octet-stream")["versionCode"]
print(f"uploaded versionCode={vc}")
put_track(app, edit, tok, a.track, [vc], a.status, a.user_fraction, notes)
print(f"assigned versionCode={vc} -> track={a.track} status={a.status}"
+ (f" userFraction={a.user_fraction}" if a.user_fraction is not None else ""))
# Same edit as the assignment above, so the code is never active on both tracks at once.
if a.promote_from:
put_track(app, edit, tok, a.promote_from, [], a.status)
print(f"cleared track '{a.promote_from}'")
if a.no_commit:
call("POST", f"{app}/edits/{edit}:validate", token=tok)
@@ -33,7 +33,24 @@ import java.util.concurrent.ConcurrentHashMap
* InputManager hot-plug callbacks both land there). [deviceForPad] is read from the feedback poll
* threads, so the slot table is a [ConcurrentHashMap].
*/
class GamepadRouter(context: Context, private val handle: Long, private val setting: Int) {
class GamepadRouter(
context: Context,
private val handle: Long,
private val setting: Int,
/**
* Forward this device's controllers to the host at all (`Settings.gamepadForwarding`,
* default true). Off is for a couch whose controller reaches the host another way — USB
* passthrough such as VirtualHere, or a pad plugged into the host itself — where forwarding
* as well would give the host two pads for one pair of hands.
*
* Off still opens slots and tracks held state; it only stops the wire sends. That is
* deliberate: the exit and mic chords are read off the same slots, and a couch that lost its
* quit shortcut because a forwarding preference was off would be the worse bug. Nothing is
* claimed by keeping a slot — the Android input stack shares controllers — unlike the USB
* capture links, which `StreamScreen` does not start at all while this is off.
*/
private val forwarding: Boolean = true,
) {
/** One forwarded controller: its stable wire pad index, per-device axis state, and held buttons. */
private class Slot(val index: Int, val mapper: Gamepad.AxisMapper) {
@@ -65,6 +82,16 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
*/
var onExitArmed: ((armed: Boolean) -> Unit)? = null
/**
* Invoked (main thread) each time the mic-mute chord ([MIC_CHORD], Select + Y) is COMPLETED on
* a pad — the couch equivalent of the stream's on-screen mute button, which a gamepad user
* cannot reach. `StreamScreen` wires it to the mute toggle. Unlike the exit chord this fires
* immediately: muting is the kind of thing you want to have already happened, and the on-screen
* indicator makes an accidental toggle self-evident. The buttons still go to the host — the
* chord adds a meaning to them rather than swallowing them, exactly as the exit chord does.
*/
var onMicChord: (() -> Unit)? = null
private val mainHandler = Handler(Looper.getMainLooper())
/** The pending exit-chord hold timer, or null when the chord isn't currently armed. */
private var pendingExit: Runnable? = null
@@ -108,16 +135,29 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
/**
* One button transition on [slot] — the shared body behind [onButton] and an [ExternalPad]'s
* transitions: forward the wire event, track held state, and arm/disarm the exit chord.
* transitions: forward the wire event, track held state, arm/disarm the exit chord, and fire
* the mic-mute chord ([MIC_CHORD]).
*/
private fun slotButton(slot: Slot, bit: Int, down: Boolean, send: Boolean) {
if (down) {
if (send) NativeBridge.nativeSendGamepadButton(handle, bit, true, slot.index)
if (send && forwarding) {
NativeBridge.nativeSendGamepadButton(handle, bit, true, slot.index)
}
val wasHeld = slot.held
slot.held = slot.held or bit
// Full chord now held on this pad → start the hold countdown (idempotent while held).
if (slot.held and EXIT_CHORD == EXIT_CHORD) armExit()
// Mic mute, edge-triggered on the button that COMPLETES the chord: a genuine press
// (`wasHeld` lacks the bit, so an auto-repeat DOWN can't re-fire it) of a chord member
// that leaves the whole chord held. Any other button pressed while Select + Y are down
// fails the middle test, so the toggle happens once per chord, not once per press.
if (wasHeld and bit == 0 && bit and MIC_CHORD != 0 && slot.held and MIC_CHORD == MIC_CHORD) {
onMicChord?.invoke()
}
} else {
if (send) NativeBridge.nativeSendGamepadButton(handle, bit, false, slot.index)
if (send && forwarding) {
NativeBridge.nativeSendGamepadButton(handle, bit, false, slot.index)
}
slot.held = slot.held and bit.inv()
// A chord button lifted before the hold elapsed → cancel, unless another pad still
// holds the full chord.
@@ -167,7 +207,7 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
val dev = event.device ?: return false
if (!isForwardable(dev)) return false
val slot = slotFor(dev) ?: return false
slot.mapper.onMotion(event)
if (forwarding) slot.mapper.onMotion(event)
return true
}
@@ -202,24 +242,26 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
/** One axis update ([Gamepad].AXIS_*: stick i16 +y=up / trigger 0..255). On-change only. */
fun axis(id: Int, value: Int) {
if (slot != null) NativeBridge.nativeSendGamepadAxis(handle, id, value, index)
if (slot != null && forwarding) NativeBridge.nativeSendGamepadAxis(handle, id, value, index)
}
/** One raw HID report, forwarded verbatim for the host's as-is virtual pad. */
fun hidReport(buf: java.nio.ByteBuffer, len: Int) {
if (slot != null) NativeBridge.nativeSendPadHidReport(handle, index, buf, len)
if (slot != null && forwarding) NativeBridge.nativeSendPadHidReport(handle, index, buf, len)
}
/** One touchpad contact on the rich plane: [finger] 0/1, x/y normalized 0..65535 in
* SCREEN convention (+y down); `active = false` lifts the finger. On-change only. */
fun touch(finger: Int, active: Boolean, x: Int, y: Int) {
if (slot != null) NativeBridge.nativeSendPadTouch(handle, index, finger, active, x, y)
if (slot != null && forwarding) {
NativeBridge.nativeSendPadTouch(handle, index, finger, active, x, y)
}
}
/** One motion sample on the rich plane (gyro pitch/yaw/roll + accel, raw device i16
* units — the host passes them straight into the virtual pad's report). Per report. */
fun motion(gyro: IntArray, accel: IntArray) {
if (slot != null) {
if (slot != null && forwarding) {
NativeBridge.nativeSendPadMotion(
handle, index,
gyro[0], gyro[1], gyro[2],
@@ -241,7 +283,7 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
// Synthetic ids live below any real InputDevice id (those are positive), so they can't
// collide and InputDevice.getDevice(id) resolves them to null for the feedback path.
val syntheticId = EXTERNAL_ID_BASE - index
NativeBridge.nativeSendGamepadArrival(handle, pref, index)
if (forwarding) NativeBridge.nativeSendGamepadArrival(handle, pref, index)
slots[syntheticId] = Slot(index, Gamepad.AxisMapper(handle, index))
return ExternalPad(syntheticId, index)
}
@@ -298,7 +340,7 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
// Automatic resolves the pad's type from its VID/PID; an explicit setting forces every pad
// to that type (a single global choice — matches the handshake's session-default pref).
val pref = if (setting == Gamepad.PREF_AUTO) Gamepad.prefFor(dev) else setting
NativeBridge.nativeSendGamepadArrival(handle, pref, index)
if (forwarding) NativeBridge.nativeSendGamepadArrival(handle, pref, index)
val slot = Slot(index, Gamepad.AxisMapper(handle, index))
slots[dev.id] = slot
return slot
@@ -311,7 +353,7 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
private fun closeSlot(deviceId: Int) {
val slot = slots.remove(deviceId) ?: return
releaseHeld(slot)
NativeBridge.nativeSendGamepadRemove(handle, slot.index)
if (forwarding) NativeBridge.nativeSendGamepadRemove(handle, slot.index)
// If this pad was mid-exit-chord, its removal may have left no pad holding it — drop the timer.
if (slots.values.none { it.held and EXIT_CHORD == EXIT_CHORD }) disarmExit()
// Release this controller's feedback bindings (close its lights session / cancel rumble).
@@ -323,11 +365,11 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
var bits = slot.held
while (bits != 0) {
val bit = bits and -bits // lowest set bit
NativeBridge.nativeSendGamepadButton(handle, bit, false, slot.index)
if (forwarding) NativeBridge.nativeSendGamepadButton(handle, bit, false, slot.index)
bits = bits and bit.inv()
}
slot.held = 0
slot.mapper.reset() // zero sticks/triggers + release the HAT dpad
if (forwarding) slot.mapper.reset() // zero sticks/triggers + release the HAT dpad
}
/** Lowest wire index 0..[MAX_PADS) not held by a slot, or null when full — stable lowest-free keeps indices from shuffling on hot-plug. */
@@ -351,6 +393,14 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
*/
const val EXIT_HOLD_MS = 1000L
/**
* Mic-mute chord: Select + Y. Y is deliberately NOT one of [EXIT_CHORD]'s buttons, so no
* way of reaching the exit chord can pass through this one on the way (and vice versa) —
* and Select is a menu button rather than a twitch action, which makes the pair unlikely
* to occur inside real play.
*/
const val MIC_CHORD = Gamepad.BTN_BACK or Gamepad.BTN_Y
/** Synthetic slot-key base for [ExternalPad]s — below every real (positive) InputDevice id. */
const val EXTERNAL_ID_BASE = -1000
}
@@ -248,11 +248,12 @@ object NativeBridge {
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 30 doubles (unified stats spec, `design/stats-unification.md`):
* Returns 33 doubles (unified stats spec, `design/stats-unification.md`):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
* netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
* e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive]`
* e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
* feedP50Ms, codecP50Ms, skippedOverflowWindow]`
* (the flags are 1.0/0.0; indexes 2/3 are the end-to-end capture→decoded headline; 1013
* describe the negotiated video feed — bit depth 8/10, CICP primaries/transfer, and the HEVC
* chroma_format_idc 1=4:2:0 / 3=4:4:4; 14/15 are the stage p50s tiling the headline —
@@ -263,7 +264,12 @@ object NativeBridge {
* `display` stage from the OnFrameRendered render timestamps — when `dispValid` is 1.0 the
* headline becomes the directly-measured capture→displayed pair at 24/25, tiled by
* `host+network` + `decode` + `display` (23), and when 0.0 the HUD falls back to the
* capture→decoded headline at 2/3 without the `display` term).
* capture→decoded headline at 2/3 without the `display` term; 2629 split the `display`
* term the timeline presenter owns — `pace` = decoded→release, `latch` = release→displayed,
* the window's on-glass confirm count, and whether the presenter is active at all; 30/31
* split `decode` (15) the same way — `feed` = received→queued (hand-off + input-slot wait),
* `codec` = queued→decoded, the decoder's own time; 32 is the parked-AU overflow subset of
* `skipped` (19), i.e. the decoder falling behind rather than benign newest-wins pacing).
* Poll ~1 Hz; each call resets the measurement window.
*/
external fun nativeVideoStats(handle: Long): DoubleArray?
@@ -288,15 +294,53 @@ object NativeBridge {
external fun nativeStopAudio(handle: Long)
/**
* Start mic uplink: AAudio input → Opus (48 kHz stereo, 20 ms) → host (`send_mic` / 0xCB), all in
* Rust. No-op if already running. The caller MUST hold RECORD_AUDIO; otherwise the AAudio input
* stream fails to open and the rest of the session keeps streaming.
* Start mic uplink: AAudio input → Opus (48 kHz mono, 10 ms) → host (`send_mic` / 0xCB), all in
* Rust. [echoCancel] opens the capture under the VoiceCommunication preset (the HAL's own echo
* canceller / noise suppressor) and allocates an audio session id; the return value is that id
* (`> 0`) so the caller can attach the Java [android.media.audiofx.AcousticEchoCanceler] /
* [android.media.audiofx.NoiseSuppressor] as a backstop — `0` when none was allocated
* (echoCancel off, the device refused the preset and the open fell back to the plain path, or
* the mic failed entirely). No-op if already running (returns the running capture's id). The
* caller MUST hold RECORD_AUDIO; otherwise the AAudio input stream fails to open and the rest
* of the session keeps streaming.
*/
external fun nativeStartMic(handle: Long)
external fun nativeStartMic(handle: Long, echoCancel: Boolean): Int
/** Stop + join the mic thread and close the AAudio input stream. No-op on `0`. */
/**
* Stop + join the mic thread and close the AAudio input stream. No-op on `0`. Leaves the
* session's mute state ([nativeSetMicMuted]) alone — a surface recreate stops and restarts the
* mic, and a user who muted must stay muted through it.
*/
external fun nativeStopMic(handle: Long)
/**
* Mute/unmute the mic uplink mid-stream. Muting does NOT stop the capture: the AAudio input
* stream, the input preset it settled on and its primed buffers stay as they are, and the
* encode loop drops each 10 ms frame instead of encoding + sending it — so room audio is never
* encoded and nothing goes on the wire, while a toggle costs an atomic store and takes effect
* on the next 10 ms boundary (a stop/start would re-run the preset fallback ladder and re-prime
* buffers every time).
*
* Sticky for the SESSION — the flag lives on the handle, not on the capture — so the mic
* restart a surface recreate performs comes back muted, with no window for an unmuted frame to
* escape; a fresh session always starts unmuted. Nothing here is persisted. No-op on `0`.
* Cheap (one atomic store); UI-safe.
*
* One honest consequence of keeping the stream open: the platform's own recording indicator
* stays lit while muted, because the mic really is still open. What stops is the encode and the
* send — no captured audio leaves the process.
*/
external fun nativeSetMicMuted(handle: Long, muted: Boolean)
/**
* Is a mic capture actually RUNNING — i.e. did [nativeStartMic] open a stream, and has
* [nativeStopMic] not been called since? Offer the in-stream mute control on THIS rather than
* on the user's setting: a device that refused every AAudio input rung (or a missing
* RECORD_AUDIO grant) then shows no control instead of a lie about a mic being heard. `false`
* on a `0` handle. Cheap; UI-safe.
*/
external fun nativeMicActive(handle: Long): Boolean
// ---- Input: Kotlin captures, Rust forwards to the host (send_input) ----
/** Relative mouse move; dx/dy are device-pixel deltas (screen +y down). */
@@ -100,6 +100,34 @@ object VideoDecoders {
}
}
/**
* One-line per-mime probe readout for the connect log (`adb logcat -s pf.caps`): which
* decoder each advertised mime resolves to and whether it declares `FEATURE_PartialFrame` —
* the P2 slice-pipeline gate that is otherwise invisible until a stream behaves differently.
*/
fun capsReport(): String {
val mimes = buildList {
add("video/avc")
add("video/hevc")
if (decodableCodecBits() and 4 != 0) add("video/av01")
}
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
.getOrNull() ?: return "codec list unavailable"
return mimes.joinToString(" ") { mime ->
val pick = pickDecoder(mime)?.name
val partial = pick?.let { p ->
infos.firstOrNull { it.name == p }?.let { info ->
runCatching {
info.getCapabilitiesForType(mime)
.isFeatureSupported(CodecCapabilities.FEATURE_PartialFrame)
}.getOrNull()
}
}
"${mime.removePrefix("video/")}=${pick ?: "platform-default"}" +
" partialFrame=${partial ?: "?"}"
}
}
fun pickDecoder(mime: String): DecoderChoice? {
if (mime.isEmpty()) return null
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
+70 -19
View File
@@ -16,7 +16,7 @@ use std::time::{Duration, Instant};
use super::display::{
apply_hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags, take_stamp};
use super::presenter::{presenter_disabled_by_sysprop, PresentMeter, PresentPriority, Presenter};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
@@ -280,6 +280,10 @@ pub(super) fn run_async(
let mut oversized_dropped: u64 = 0;
// Slice-progressive continuity ledger (see `PartFeed`).
let mut part_open: Option<PartFeed> = None;
// Queued-instant stamps (pts → realtime ns at the AU's LAST piece entering the codec) — the
// P3 decode-split ledger: `feed` = received→queued, `codec` = queued→decoded. Always on
// (one vDSO clock read per AU); consumed by `present_ready`.
let mut queued_stamps: VecDeque<(u64, i128)> = VecDeque::new();
// Freeze-until-reanchor gate (see the sync loop for the rationale). Armed on a frame-index gap
// (the feeder's Au verdict), a parked-AU overflow drop, a dropped-count climb, or a recoverable
// codec error; `recovery_flags` carries each AU's user_flags from `dispatch_event` (feed) to
@@ -349,7 +353,7 @@ pub(super) fn run_async(
p.on_vsync();
}
}
stats.note_skipped(aus_dropped); // parked-AU overflow drops are client-side skips too
stats.note_skipped_overflow(aus_dropped); // parked-AU overflow: skips, flagged as such
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
@@ -361,6 +365,7 @@ pub(super) fn run_async(
&mut fed,
&mut oversized_dropped,
&mut part_open,
&mut queued_stamps,
&mut gate,
);
let had_output = !ready.is_empty();
@@ -372,6 +377,8 @@ pub(super) fn run_async(
&mut ready,
&stats,
&in_flight,
&mut queued_stamps,
&meter,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut presenter,
@@ -385,7 +392,7 @@ pub(super) fn run_async(
// even when the choreographer clock is absent.
if let Some(p) = presenter.as_mut() {
let clock = vsync.as_ref().map(|v| v.shared().as_ref());
if p.pump(&codec, clock, &tracker, &stats, now_monotonic_ns()) {
if p.pump(&codec, clock, &tracker, &meter, &stats, now_monotonic_ns()) {
rendered += 1;
}
// The 1 Hz window flush doubles as the phase-lock report tick. v3 sensor: the
@@ -762,6 +769,7 @@ pub(super) struct PartFeed {
/// [`BUFFER_FLAG_PARTIAL_FRAME`] except the AU's last, all at the AU's pts. `part_open` is the
/// continuity ledger — any break (gap, orphan, oversize) abandons the AU per [`PartFeed::pts_us`]'s
/// close contract and re-syncs at the next `first`.
#[allow(clippy::too_many_arguments)] // one call site; the split ledger threads through like the gate
fn feed_ready(
codec: &MediaCodec,
client: &NativeClient,
@@ -770,6 +778,7 @@ fn feed_ready(
fed: &mut u64,
oversized_dropped: &mut u64,
part_open: &mut Option<PartFeed>,
queued_stamps: &mut VecDeque<(u64, i128)>,
gate: &mut ReanchorGate,
) {
while !pending_aus.is_empty() && !free_inputs.is_empty() {
@@ -813,8 +822,21 @@ fn feed_ready(
}
}
let Some(dst) = codec.input_buffer(idx) else {
log::warn!("decode: input_buffer({idx}) returned None — dropping AU");
continue;
// Nothing was written and nothing was queued, so BOTH stay ours. Dropping the slot
// here leaked one of the codec's input buffers per occurrence — we forget it and the
// codec never frees what it never received, so the pipeline quietly runs out of input
// slots, `pending_aus` overflows, and the resulting drop storm reads as a decode
// fault. Dropping the AU on top of that punched a hole in the reference chain with no
// keyframe request behind it, unlike every sibling path here.
//
// `break`, not `continue`: a codec that cannot hand out an input buffer it just
// advertised is in no state to be fed the rest of the parked queue this pass, and
// retrying the same index against every parked AU would burn the whole backlog. The
// loop re-runs within the housekeeping wake (≤ 5 ms) if it was transient.
log::warn!("decode: input_buffer({idx}) returned None — retrying next pass");
free_inputs.push_front(idx);
pending_aus.push_front(frame);
break;
};
let au = &frame.data;
if au.len() > dst.len() {
@@ -859,9 +881,15 @@ fn feed_ready(
}
} else {
// `fed` counts ACCESS UNITS toward the HUD's fed/decoded balance — the closing
// piece (or a whole AU) bumps it.
// piece (or a whole AU) bumps it. The queued stamp marks the same instant (the AU
// is fully in the codec's hands): the P3 decode split measures `codec` from here,
// so a slice-progressive head start shows up as codec-pure shrink.
if last {
*fed += 1;
queued_stamps.push_back((pts_us, now_realtime_ns()));
if queued_stamps.len() > IN_FLIGHT_CAP {
queued_stamps.pop_front(); // stale — codec never echoed it back
}
}
*part_open = if last {
None
@@ -876,7 +904,8 @@ fn feed_ready(
}
}
/// Route the ready outputs toward glass. With the timeline presenter (default): fold each output
/// Route the ready outputs toward glass, recording each one's decode-split + e2e first. With the
/// timeline presenter (default): fold each output
/// through the re-anchor gate in pts order, hand the approved ones to the presenter's store
/// (newest-wins / smoothing FIFO — the actual release happens in `Presenter::pump`, budgeted and
/// timeline-timed), and release withheld concealment unrendered. Legacy (`arrival` sysprop):
@@ -892,6 +921,8 @@ fn present_ready(
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
queued_stamps: &mut VecDeque<(u64, i128)>,
meter: &PresentMeter,
clock_offset: i64,
tracker: &DisplayTracker,
presenter: &mut Option<Presenter>,
@@ -903,22 +934,42 @@ fn present_ready(
if ready.is_empty() {
return;
}
// Pair each output's decode stage (feeds the ABR decode signal always; the HUD histogram only
// while visible) — both consume the receipt map, so enter for either.
if stats.enabled() || measure_decode {
// Pair each output's decode stage (the ABR decode signal + the HUD histogram consume the
// receipt map; the P3 split's codec-pure half needs only the queued stamp, so it records
// even with both off — that keeps the 1 Hz pf.present mirror HUD-off readable).
{
let want_stage = stats.enabled() || measure_decode;
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
);
let received_ns = if want_stage {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
)
} else {
None
};
let queued = take_stamp(queued_stamps, o.pts_us);
let codec_us = queued.map(|q| ((o.decoded_ns - q).max(0) / 1000) as u64);
let feed_us = match (queued, received_ns) {
(Some(q), Some(r)) => Some(((q - r).max(0) / 1000) as u64),
_ => None,
};
// Always-on e2e for the 1 Hz pf.present mirror (same formula + clamp as the HUD's
// capture→decoded headline in `note_decoded_pts`).
let e2e_ns = o.decoded_ns + clock_offset as i128 - o.pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
meter.note_decode(feed_us, codec_us, e2e_us);
if let Some(c) = codec_us {
stats.note_decode_split(feed_us, c);
}
}
}
// Fold EVERY output through the gate in pts (== decode) order — even the ones newest-wins discards —
+21 -2
View File
@@ -20,7 +20,8 @@ pub(super) fn now_realtime_ns() -> i128 {
/// entries older than it are evicted (decode order == input order here — low-latency, no
/// B-frames — so anything before it was dropped inside the codec or stamped before a flush).
/// `decoded_ns` is the availability instant: the dequeue (sync loop) or the output callback's
/// stamp (async loop).
/// stamp (async loop). Returns the receipt stamp it paired (if any) so the caller can split the
/// `decode` stage further (feed wait vs codec-pure) without re-walking the map.
pub(super) fn note_decoded_pts(
client: &NativeClient,
measure_decode: bool,
@@ -29,7 +30,7 @@ pub(super) fn note_decoded_pts(
clock_offset: i64,
pts_us: u64,
decoded_ns: i128,
) {
) -> Option<i128> {
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
@@ -61,6 +62,24 @@ pub(super) fn note_decoded_pts(
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
stats.note_decoded(e2e_us, decode_us);
}
received_ns
}
/// The queued-instant stamp for a decoded output, keyed by the echoed `presentationTimeUs` — the
/// same monotonic evict-as-you-go pairing as [`take_flags`], over an `(pts_us, realtime_ns)` map
/// (the feed side stamps each AU as its last piece enters the codec). A miss returns `None` —
/// the split is simply not recorded for that frame.
pub(super) fn take_stamp(map: &mut VecDeque<(u64, i128)>, pts_us: u64) -> Option<i128> {
while let Some(&(p, t)) = map.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
map.pop_front();
if p == pts_us {
return Some(t);
}
}
None
}
/// The AU `user_flags` for a decoded output, keyed by the echoed `presentationTimeUs`. Recovery
+8 -3
View File
@@ -115,9 +115,14 @@ pub(crate) struct DecodeOptions {
/// The smoothness buffer depth (`smooth_buffer` setting): 0 = automatic (2), else 1..=3.
/// Only meaningful with `present_priority` = smooth.
pub smooth_buffer: i32,
/// The display mode's own refresh rate (Kotlin's `display.refreshRate` at stream start;
/// 0 = unknown) — the latch grid the presenter subdivides onto when the app's choreographer
/// stream is down-rated below the panel (see `vsync.rs`).
/// SEED for the panel's refresh period — the latch grid the presenter subdivides onto when
/// the app's choreographer stream is down-rated below the panel (see `vsync.rs`). Kotlin
/// resolves it from the display mode TABLE (`MainActivity.streamPanelFps`), not
/// `display.refreshRate`, which reports a per-uid override rather than the panel. 0 = unknown.
///
/// ⚠ Only a seed: `preferredDisplayModeId` is a REQUEST the system may refuse, so the mode
/// named here is not necessarily the one the panel ends up in. The measured timeline spacing
/// corrects it in both directions ([`punktfunk_core::phase::PanelGrid`]).
pub panel_hz: i32,
}
+223 -25
View File
@@ -4,10 +4,12 @@
//! * a **newest-wins slot** (or a small smoothing FIFO, by user intent) between decode and
//! release, so a burst coalesces in the app — as an explicit, counted drop — instead of
//! queueing behind the display;
//! * a **glass budget of exactly one**: at most one undisplayed release in flight to
//! SurfaceFlinger, reopened on the clock-predicted latch (with a 100 ms stale force-open as
//! the liveness backstop, mirroring Apple's `PresentGate.staleAfter`). The BufferQueue can
//! hold at most the frame being scanned out plus one — a standing queue is unconstructible;
//! * a **glass budget of one**: at most one undisplayed release in flight to SurfaceFlinger,
//! reopened on the clock-predicted latch (with a 100 ms stale force-open as the liveness
//! backstop, mirroring Apple's `PresentGate.staleAfter`), and bounded underneath by what
//! `OnFrameRendered` actually confirmed reached glass ([`UNDISPLAYED_CAP`]) — because the
//! prediction is only as good as the panel grid behind it, and 0.23.0 shipped a grid that
//! could be wrong in one direction forever;
//! * a **timed release**: `AMediaCodec_releaseOutputBufferAtTime` targeting the platform's own
//! frame timeline (API 33+, via [`super::vsync`]), so the latch phase is deterministic instead
//! of inheriting network + decode jitter. On the 31/32 fallback the release is ASAP —
@@ -20,6 +22,7 @@
use ndk::media::media_codec::MediaCodec;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, AtomicI32, Ordering};
use std::sync::Mutex;
use std::time::Instant;
@@ -36,9 +39,9 @@ use super::vsync::VsyncShared;
///
/// 2.5 ms: SF's latch runs ~1-2 ms before present on modern devices (its `sfOffset`), and the
/// release itself is a binder call well under a ms. 4 ms measured latch p50 8-10; each ms cut
/// here is a ms off every frame's display stage. If a device misses at this margin the `paced`
/// counter shows it (a miss presents one vsync later, coalescing the next frame) — that is the
/// signal to widen, not stutter.
/// here is a ms off every frame's display stage. A device that misses at the live margin shows it
/// as a measured latch beyond one panel period (see the adaptation in
/// [`Presenter::flush_log`]) — that, not a drop counter, is the signal to widen.
const LATCH_MARGIN_NS: i64 = 2_500_000;
/// `debug.punktfunk.latch_margin_us` (0..=8000 µs): PIN the submit margin for a sweep —
@@ -71,6 +74,26 @@ fn latch_margin_ns() -> Option<i64> {
/// `forced` — reads 0 on healthy systems (Apple's `PresentGate.staleAfter`, same value).
const STALE_REOPEN_NS: i64 = 100_000_000;
/// Releases still unconfirmed by `OnFrameRendered` at which the presenter stops handing
/// SurfaceFlinger more work.
///
/// The reopen above is a PREDICTION off the learned panel grid. A grid finer than the panel
/// (0.23.0 could pin one permanently — see [`punktfunk_core::phase::PanelGrid`]) reopens the
/// budget before the display has consumed anything, and the presenter then releases faster than
/// the panel scans: the BufferQueue fills, MediaCodec runs out of output buffers, the decoder
/// stalls, and the no-output backstop starts begging for keyframes. The render callback is the
/// ground truth about what actually reached glass, so it bounds the prediction.
///
/// Six, not one: the platform is explicitly allowed to deliver these callbacks BATCHED, and this
/// module's own `RENDERED_CAP` note records them trailing a release by a vsync or two — so a
/// healthy device sits at 1-3 outstanding and a tight cap would throttle it for nothing (a held
/// frame in the newest-wins slot is a DROPPED frame the moment a fresher one decodes). This is
/// not a pacing knob; it is the "something is structurally wrong" rail, and a presenter genuinely
/// out-running its display climbs past any fixed cap within a second. If a device's BufferQueue
/// is shallower than this the rail simply never engages and the no-output backstop handles it,
/// exactly as before — best-effort, never worse than not having it.
const UNDISPLAYED_CAP: i32 = 6;
/// Fallback latch-prediction period while the vsync clock is unmeasured/absent: one 120 Hz frame.
const FALLBACK_PERIOD_NS: i64 = 8_333_333;
@@ -121,11 +144,28 @@ struct InFlight {
/// a HUD-off wireless A/B readable from logcat.
pub(super) struct PresentMeter {
inner: Mutex<PresentMeterInner>,
/// Frames released to SurfaceFlinger that `OnFrameRendered` has not yet confirmed reached
/// glass. The presenter's structural rail (see [`UNDISPLAYED_CAP`]) and the pf-present line's
/// queue-depth readout. Lock-free because the release side runs on the decode loop and the
/// confirm side on the codec's callback thread, once per frame each.
undisplayed: AtomicI32,
/// This device delivers render callbacks at all (API ≥ 33 and the platform accepted the
/// registration). Until one arrives, `undisplayed` is meaningless and the rail stays down.
confirms: AtomicBool,
}
struct PresentMeterInner {
latch_us: Vec<u64>,
displays: u64,
/// The `decode` stage's feed split, received→queued µs (P3 science: hand-off + input-slot
/// wait). Empty when no receipt stamp matched (HUD off and ABR not measuring decode).
feed_us: Vec<u64>,
/// The codec-pure half, queued→decoded µs, measured from the AU's LAST piece — always on,
/// so a HUD-off logcat A/B still reads the decoder's own time.
codec_us: Vec<u64>,
/// Capture→decoded end-to-end µs (skew-corrected, clamped) — always on for the same reason:
/// the wireless A/B's headline without having to reach the on-screen HUD.
e2e_us: Vec<u64>,
}
impl PresentMeter {
@@ -134,12 +174,27 @@ impl PresentMeter {
inner: Mutex::new(PresentMeterInner {
latch_us: Vec::with_capacity(256),
displays: 0,
feed_us: Vec::with_capacity(256),
codec_us: Vec::with_capacity(256),
e2e_us: Vec::with_capacity(256),
}),
undisplayed: AtomicI32::new(0),
confirms: AtomicBool::new(false),
}
}
/// One displayed frame's release→displayed latch, µs. Callback thread; poison-proof.
///
/// Also the glass budget's CONFIRM: this frame left the BufferQueue, so one outstanding
/// release is settled. Clamped at zero — the legacy `arrival` path renders without going
/// through [`Presenter::pump`], so confirms can outnumber counted releases.
pub(super) fn note_latch(&self, latch_us: Option<u64>) {
self.confirms.store(true, Ordering::Relaxed);
let _ = self
.undisplayed
.fetch_update(Ordering::Relaxed, Ordering::Relaxed, |v| {
Some((v - 1).max(0))
});
let mut g = self
.inner
.lock()
@@ -152,14 +207,71 @@ impl PresentMeter {
}
}
fn drain(&self) -> (Vec<u64>, u64) {
/// One frame handed to SurfaceFlinger, awaiting its confirm. Decode thread.
fn note_released(&self) {
self.undisplayed.fetch_add(1, Ordering::Relaxed);
}
/// Releases still unconfirmed, and whether confirms happen on this device at all.
fn outstanding(&self) -> (i32, bool) {
(
self.undisplayed.load(Ordering::Relaxed),
self.confirms.load(Ordering::Relaxed),
)
}
/// Write off the outstanding releases: the platform stopped confirming (it is allowed to
/// drop callbacks under load) or SurfaceFlinger discarded the buffers without presenting
/// them. Never stall the stream on a ledger we cannot audit.
fn forgive_outstanding(&self) {
self.undisplayed.store(0, Ordering::Relaxed);
}
/// One decoded frame's always-on measurements: the `decode`-stage split (feed =
/// received→queued when a receipt stamp matched; codec = queued→decoded when the queued
/// stamp did) and the capture→decoded end-to-end, µs. Decode thread; poison-proof.
pub(super) fn note_decode(
&self,
feed_us: Option<u64>,
codec_us: Option<u64>,
e2e_us: Option<u64>,
) {
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(f) = feed_us {
if g.feed_us.len() < 4096 {
g.feed_us.push(f);
}
}
if let Some(c) = codec_us {
if g.codec_us.len() < 4096 {
g.codec_us.push(c);
}
}
if let Some(e) = e2e_us {
if g.e2e_us.len() < 4096 {
g.e2e_us.push(e);
}
}
}
#[allow(clippy::type_complexity)] // one caller unpacks it in place; a struct would be noise
fn drain(&self) -> (Vec<u64>, u64, Vec<u64>, Vec<u64>, Vec<u64>) {
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let displays = g.displays;
g.displays = 0;
(std::mem::take(&mut g.latch_us), displays)
(
std::mem::take(&mut g.latch_us),
displays,
std::mem::take(&mut g.feed_us),
std::mem::take(&mut g.codec_us),
std::mem::take(&mut g.e2e_us),
)
}
}
@@ -190,6 +302,13 @@ pub(super) struct Presenter {
no_budget: u64,
forced: u64,
dry: u64,
/// Pump passes that held a frame back because too many earlier releases were still
/// unconfirmed ([`UNDISPLAYED_CAP`]) — reads 0 on a healthy device, and a climbing value is
/// the signature of a presenter out-running its display.
queue_waits: u64,
/// When the unconfirmed-release rail first engaged, so it can be forgiven if the confirms
/// simply stopped coming. `None` while the rail is down.
backed_up_since: Option<i64>,
pace_us: Vec<u64>,
last_flush: Instant,
/// The live submit margin. Starts at 0 (P2e on-glass: SurfaceFlinger latched every
@@ -231,6 +350,8 @@ impl Presenter {
no_budget: 0,
forced: 0,
dry: 0,
queue_waits: 0,
backed_up_since: None,
pace_us: Vec::with_capacity(256),
last_flush: Instant::now(),
margin_ns,
@@ -285,6 +406,7 @@ impl Presenter {
codec: &MediaCodec,
clock: Option<&VsyncShared>,
tracker: &DisplayTracker,
meter: &PresentMeter,
stats: &crate::stats::VideoStats,
now_mono_ns: i64,
) -> bool {
@@ -297,6 +419,10 @@ impl Presenter {
self.inflight = None;
}
}
// The measured rail beneath that prediction (see `UNDISPLAYED_CAP`). Evaluated on every
// pass — frame waiting or not — so its forgiveness timer measures real elapsed time
// rather than how often a frame happened to be ready.
let backlogged = self.unconfirmed_backlog(meter, now_mono_ns);
// Pick the frame this pump may release.
let frame = if self.fifo_capacity == 0 {
self.frames.pop_back() // submit() kept it a single slot; back == the newest
@@ -324,9 +450,12 @@ impl Presenter {
self.frames.pop_front()
};
let Some(frame) = frame else { return false };
if self.inflight.is_some() {
if self.inflight.is_some() || backlogged {
// Budget closed — park it back; a fresher submit replaces it (newest-wins), the next
// vsync tick / loop pass retries the pairing.
if backlogged {
self.queue_waits += 1;
}
self.no_budget += 1;
match self.fifo_capacity {
0 => self.frames.push_back(frame),
@@ -363,6 +492,7 @@ impl Presenter {
released_at_ns: now_mono_ns,
});
self.released += 1;
meter.note_released();
let release_real_ns = now_realtime_ns();
let pace_us = ((release_real_ns - frame.decoded_ns).max(0) / 1000) as u64;
if self.pace_us.len() < 4096 {
@@ -373,6 +503,33 @@ impl Presenter {
true
}
/// Whether SurfaceFlinger is sitting on too many unconfirmed releases to be handed another.
///
/// The predicted reopen is only as good as the panel grid behind it; this is the measured
/// rail underneath it (see [`UNDISPLAYED_CAP`]). It self-clears two ways — the confirms catch
/// up, or [`STALE_REOPEN_NS`] passes with the backlog stuck, which means the ledger itself is
/// unreliable (callbacks dropped under load, or SF discarded the buffers) and is written off
/// rather than allowed to wedge the stream.
fn unconfirmed_backlog(&mut self, meter: &PresentMeter, now_ns: i64) -> bool {
let (outstanding, confirms_live) = meter.outstanding();
if !confirms_live || outstanding < UNDISPLAYED_CAP {
self.backed_up_since = None;
return false;
}
match self.backed_up_since {
Some(t) if now_ns - t > STALE_REOPEN_NS => {
meter.forgive_outstanding();
self.backed_up_since = None;
self.forced += 1;
false
}
_ => {
self.backed_up_since.get_or_insert(now_ns);
true
}
}
}
/// Release every held buffer unrendered — the teardown path, BEFORE `codec.stop()`.
pub(super) fn release_all(&mut self, codec: &MediaCodec) {
while let Some(f) = self.frames.pop_front() {
@@ -385,8 +542,12 @@ impl Presenter {
/// `pf-present` line, so a HUD-off on-device A/B is readable wirelessly:
/// `released` (to glass) / `displays` (OnFrameRendered confirms) / `paced` (policy drops) /
/// `noBudget` (waits on the closed budget) / `forced` (stale force-opens — 0 when healthy) /
/// `qDry` (FIFO underflows) / `pace` (decodedrelease) / `latch` (release→displayed) /
/// `vsync` (the measured panel period).
/// `qDry` (FIFO underflows) / `qWait` (pumps held back by unconfirmed releases — 0 when
/// healthy) / `unconfirmed` (releases OnFrameRendered hasn't settled) /
/// `pace` (decoded→release) / `latch` (release→displayed) /
/// `feed`+`codec` (the decode stage split: received→queued hand-off/slot wait + the
/// codec-pure queued→decoded time) / `e2e` (capture→decoded, skew-corrected — the wireless
/// A/B headline) / `vsync` (the measured panel period).
///
/// Returns this window's CIRCULAR latch statistics `(vector-mean latch ns mod panel period,
/// coherence ‰)` when a window actually flushed — the phase-lock reporter's v2 error signal
@@ -400,23 +561,29 @@ impl Presenter {
return None;
}
self.last_flush = Instant::now();
let (latch, displays) = meter.drain();
let (latch, displays, feed, codec, e2e) = meter.drain();
if self.released == 0 && displays == 0 {
return None; // idle stream — nothing worth a line
}
let (pace_p50, pace_max) = p50_max_ms(std::mem::take(&mut self.pace_us));
let (feed_p50, feed_max) = p50_max_ms(feed);
let (codec_p50, codec_max) = p50_max_ms(codec);
let (e2e_p50, e2e_max) = p50_max_ms(e2e);
let circ = clock.and_then(|c| {
punktfunk_core::phase::circular_latch(&latch, c.panel_period_ns().max(c.period_ns()))
});
let latch_samples = latch.len();
let (latch_p50, latch_max) = p50_max_ms(latch);
let period_ms = clock.map(|c| c.period_ns() as f64 / 1e6).unwrap_or(0.0);
let panel_ms = clock
.map(|c| c.panel_period_ns() as f64 / 1e6)
.unwrap_or(0.0);
let panel_ns = clock.map(|c| c.panel_period_ns()).unwrap_or(0);
let (outstanding, _) = meter.outstanding();
log::info!(
target: "pf.present",
"released={} displays={} paced={} noBudget={} forced={} qDry={} \
paceMs p50={:.2} max={:.2} latchMs p50={:.2} max={:.2} circ={:.2}ms coh={} \
qWait={} unconfirmed={} \
paceMs p50={:.2} max={:.2} latchMs p50={:.2} max={:.2} \
feedMs p50={:.2} max={:.2} codecMs p50={:.2} max={:.2} \
e2eMs p50={:.2} max={:.2} circ={:.2}ms coh={} \
vsyncMs={:.2} panelMs={:.2}",
self.released,
displays,
@@ -424,32 +591,63 @@ impl Presenter {
self.no_budget,
self.forced,
self.dry,
self.queue_waits,
outstanding,
pace_p50,
pace_max,
latch_p50,
latch_max,
feed_p50,
feed_max,
codec_p50,
codec_max,
e2e_p50,
e2e_max,
circ.map(|(m, _)| m as f64 / 1e6).unwrap_or(0.0),
circ.map(|(_, c)| c).unwrap_or(0),
period_ms,
panel_ms,
panel_ns as f64 / 1e6,
);
self.released = 0;
// Margin adaptation: repeated latch misses in one window (a miss presents a vsync
// late and coalesces the next frame into `paced`) mean this device's SF does need
// lead — widen toward the pre-sweep ceiling. One-way by design: a margin that once
// proved necessary is never re-gambled mid-stream (the next stream restarts at 0).
if !self.margin_pinned && self.paced_drops > 2 && self.margin_ns < LATCH_MARGIN_NS {
// Margin adaptation, off the MEASURED latch. A release targets the first grid point past
// `now + margin`, so a frame that makes its vsync is on glass within one panel period of
// that margin; beyond it, SurfaceFlinger wanted more lead and the frame waited out an
// extra refresh. Widen toward the pre-sweep ceiling. One-way by design: a margin that
// once proved necessary is never re-gambled mid-stream (the next stream restarts at 0).
//
// ⚠ NOT `paced_drops`, which 0.23.0 used: those are the newest-wins store's own policy
// evictions — a second frame decoding while one is held — which happen whenever the
// stream out-runs the panel and say nothing at all about SF's latch lead. Driving the
// margin from them widened it to the ceiling on healthy devices, re-imposing the 2.5 ms
// of pure display latency the P2e sweep had just measured away.
let latch_p50_ns = (latch_p50 * 1e6) as i64;
if !self.margin_pinned
&& self.margin_ns < LATCH_MARGIN_NS
&& panel_ns > 0
&& latch_samples >= 8
&& latch_p50_ns > panel_ns + self.margin_ns
{
self.margin_ns = (self.margin_ns + 500_000).min(LATCH_MARGIN_NS);
log::warn!(
"presenter: {} latch misses in 1s — margin widened to {}us",
self.paced_drops,
"presenter: latch p50 {:.2}ms over the {:.2}ms panel period — margin widened to {}us",
latch_p50,
panel_ns as f64 / 1e6,
self.margin_ns / 1_000
);
}
if self.queue_waits > 0 {
log::warn!(
"presenter: {} pump(s) held back — {} release(s) still unconfirmed by \
OnFrameRendered (the display is not keeping up with the release rate)",
self.queue_waits,
outstanding
);
}
self.paced_drops = 0;
self.no_budget = 0;
self.forced = 0;
self.dry = 0;
self.queue_waits = 0;
circ
}
}
+32 -22
View File
@@ -58,8 +58,10 @@ pub(super) struct VsyncShared {
/// video to THIS rate would cap the stream — hence `panel_period_ns` + the subdivision in
/// [`Self::next_target`].
period_ns: AtomicI64,
/// The panel's own refresh period (from the display mode Kotlin resolved at stream start;
/// 0 = unknown). The grid SurfaceFlinger actually latches on.
/// The panel's own refresh period — the grid SurfaceFlinger actually latches on (0 = unknown).
/// Seeded from the display mode Kotlin resolved at stream start and then corrected by
/// measurement; the learner itself is [`punktfunk_core::phase::PanelGrid`], owned by the
/// choreographer thread (see [`CallbackCtx::panel`]) and published here for the decode loop.
panel_period_ns: AtomicI64,
/// Callback count, for the one-shot cadence diagnostic log.
ticks: std::sync::atomic::AtomicU32,
@@ -231,6 +233,11 @@ struct CallbackCtx {
choreographer: *mut c_void,
shared: Arc<VsyncShared>,
on_tick: Box<dyn Fn() + Send>,
/// The panel-period learner. `Cell` rather than an atomic because it is touched from exactly
/// one thread — callbacks only ever fire inside this thread's looper poll (see the struct
/// doc) — and its streak state is nobody else's business; only the settled period is
/// published, to `shared.panel_period_ns`.
panel: std::cell::Cell<punktfunk_core::phase::PanelGrid>,
}
impl CallbackCtx {
@@ -240,22 +247,25 @@ impl CallbackCtx {
.shared
.last_vsync_ns
.swap(frame_time_ns, Ordering::Relaxed);
// Panel-grid learner: timeline spacing is SurfaceFlinger's own grid, and the finest
// spacing ever observed is the panel's true period — trustworthy where the configured
// value is not (under a per-uid frame-rate override, `Display.getRefreshRate` REPORTS
// THE OVERRIDE, observed on-glass: a 120 Hz panel read back as 60 while early timelines
// ran at 8.28 ms). Corrects DOWNWARD only: subdividing onto a finer real grid is always
// valid, widening on a later down-rated window never is.
// Panel-grid learner: timeline spacing is SurfaceFlinger's own grid, and therefore the
// only honest witness to what the panel is doing — the configured mode is not (under a
// per-uid frame-rate override `Display.getRefreshRate` REPORTS THE OVERRIDE, observed
// on-glass: a 120 Hz panel read back as 60 while its timelines ran at 8.28 ms), and
// neither is the mode Kotlin *requested* (`preferredDisplayModeId` is a hint the system
// may refuse). Both directions matter and the asymmetry lives in `PanelGrid`.
if timelines.len() >= 2 {
let spacing = timelines[1].expected_present_ns - timelines[0].expected_present_ns;
if (2_000_000..=42_000_000).contains(&spacing) {
let cur = self.shared.panel_period_ns.load(Ordering::Relaxed);
if cur == 0 || spacing < cur - 200_000 {
self.shared
.panel_period_ns
.store(spacing, Ordering::Relaxed);
}
let mut grid = self.panel.get();
if grid.observe(spacing) {
self.shared
.panel_period_ns
.store(grid.period_ns(), Ordering::Relaxed);
log::info!(
"vsync: panel grid now {:.2}ms",
grid.period_ns() as f64 / 1e6
);
}
self.panel.set(grid);
}
// One-shot cadence diagnostic (3rd tick, once deltas exist): the callback cadence vs the
// panel period is exactly the down-rating question, and this line answers it on-glass.
@@ -372,8 +382,9 @@ pub(super) struct VsyncClock {
impl VsyncClock {
/// Spawn the choreographer thread. `on_tick` fires once per vsync ON THAT THREAD — it must
/// only do something cheap and `Send` (the decode loop passes an event-channel send).
/// `panel_hz` is the display mode's own refresh rate (0 = unknown), the latch grid that
/// [`VsyncShared::next_target`] subdivides onto. `None` when the platform surface is missing
/// `panel_hz` SEEDS the panel-grid learner (0 = unknown) the latch grid that
/// [`VsyncShared::next_target`] subdivides onto. A seed, not a fact: it names the display
/// mode Kotlin *requested*, and the observed timeline spacing is what settles it. `None` when the platform surface is missing
/// (very old device) — the presenter then runs clock-less (ASAP targets, predicted-latch
/// budget).
pub(super) fn start(panel_hz: i32, on_tick: Box<dyn Fn() + Send>) -> Option<VsyncClock> {
@@ -383,11 +394,9 @@ impl VsyncClock {
stop: AtomicBool::new(false),
last_vsync_ns: AtomicI64::new(0),
period_ns: AtomicI64::new(0),
panel_period_ns: AtomicI64::new(if panel_hz > 0 {
1_000_000_000 / panel_hz as i64
} else {
0
}),
panel_period_ns: AtomicI64::new(
punktfunk_core::phase::PanelGrid::seeded(panel_hz).period_ns(),
),
ticks: std::sync::atomic::AtomicU32::new(0),
timelines: Mutex::new(Vec::new()),
});
@@ -408,6 +417,7 @@ impl VsyncClock {
choreographer,
shared: thread_shared,
on_tick,
panel: std::cell::Cell::new(punktfunk_core::phase::PanelGrid::seeded(panel_hz)),
};
ctx.repost();
// The bounded poll doubles as the stop check: no cross-thread wake needed, worst
+178 -46
View File
@@ -1,16 +1,22 @@
//! Android microphone uplink (android-only): capture mic PCM via AAudio (LowLatency **input**),
//! Opus-encode 20 ms stereo frames, and push them to the host over the connector's mic plane
//! Opus-encode 10 ms mono frames, and push them to the host over the connector's mic plane
//! (`send_mic` → 0xCB datagram). The mirror of [`crate::audio`] in reverse: AAudio's realtime input
//! callback hands captured interleaved f32 to a channel; a worker thread we own does the Opus
//! encode + send (encoding is too heavy for the realtime callback, exactly as decode is on the
//! playback side). Like the playback path, the realtime callback is allocation-free: captured
//! bursts are copied into pre-allocated buffers from a recycle free-list (pool empty = drop the
//! chunk, never allocate on the capture thread). Format matches the host decoder + the Linux
//! client: 48 kHz **stereo**, 20 ms, Opus VOIP.
//! callback hands captured f32 to a channel; a worker thread we own does the Opus encode + send
//! (encoding is too heavy for the realtime callback, exactly as decode is on the playback side).
//! Like the playback path, the realtime callback is allocation-free: captured bursts are copied
//! into pre-allocated buffers from a recycle free-list (pool empty = drop the chunk, never
//! allocate on the capture thread). Format: 48 kHz **mono**, 10 ms, Opus VOIP with in-band FEC —
//! the host decodes any Opus frame ≤ 120 ms with its stereo decoder (mono packets upmix), so this
//! needs no protocol change; speech gains nothing from stereo, and the shorter frame shaves a
//! buffering interval off the uplink.
//!
//! **Mute** is a flag the encode loop reads per 10 ms frame, never a stream teardown: the AAudio
//! input stream, the input-preset ladder it settled on and its primed buffers all survive a
//! mute/unmute untouched, so toggling costs an atomic load and nothing else.
use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
AudioStream, AudioStreamBuilder,
AudioCallbackResult, AudioDirection, AudioFormat, AudioInputPreset, AudioPerformanceMode,
AudioSharingMode, AudioStream, AudioStreamBuilder, SessionId,
};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
@@ -20,31 +26,57 @@ use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, SyncSender, TryS
use std::sync::Arc;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
const CHANNELS: usize = 2;
const CHANNELS: usize = 1;
const SAMPLE_RATE: i32 = 48_000;
/// 20 ms per channel @ 48 kHz — the Linux client's frame; the host accepts ≤ 120 ms.
const FRAME_SAMPLES: usize = 960;
/// 10 ms per channel @ 48 kHz — half the desktop clients' 20 ms frame, trading a little Opus
/// header overhead for one less buffered interval; the host accepts ≤ 120 ms.
const FRAME_SAMPLES: usize = 480;
/// Captured-chunk hand-off depth (each ~ one burst); drops on overflow (best-effort uplink).
/// Bursts are sized in frames, so the wall-time depth is unchanged by the stereo→mono move.
const RING_CHUNKS: usize = 64;
/// Free-list buffer capacity, in interleaved f32 samples: comfortably above a LowLatency input
/// burst (typically ≤ ~480 frames). A device with larger bursts costs each buffer a one-time grow
/// on the capture thread, after which the steady state is allocation-free again.
const CHUNK_CAP_SAMPLES: usize = 1920; // 20 ms stereo
/// Opus VOIP target bitrate (speech; tunable).
const MIC_BITRATE: i32 = 64_000;
/// burst (typically ≤ ~480 frames — mono, so samples = frames). A device with larger bursts costs
/// each buffer a one-time grow on the capture thread, after which the steady state is
/// allocation-free again.
const CHUNK_CAP_SAMPLES: usize = 960; // 20 ms mono — the same wall-time as the old stereo value
/// Opus VOIP target bitrate (mono speech; tunable).
const MIC_BITRATE: i32 = 48_000;
/// Encode-side self-heal threshold, in queued 10 ms frames (~60 ms): waking to more than this
/// means the uplink stalled — and because the capture callback drops the NEWEST chunk when the
/// channel is full, a stall otherwise converts to standing mic delay that never drains (real-time
/// playback host-side never makes time back up). Skip to the newest few frames instead.
const BACKLOG_MAX_FRAMES: usize = 6;
/// What a self-heal keeps: ~20 ms of the freshest audio (one audible blip, live again).
const BACKLOG_KEEP_FRAMES: usize = 2;
/// Owned by [`crate::session::SessionHandle`]: the live AAudio input stream + the encode thread.
pub struct MicCapture {
_stream: AudioStream, // dropping it stops + closes the AAudio input stream
/// The audio-session id AAudio allocated (`> 0`) when echo cancellation asked for one — the
/// hook Kotlin hangs the Java `AcousticEchoCanceler`/`NoiseSuppressor` on. `0` = none.
session_id: i32,
shutdown: Arc<AtomicBool>,
join: Option<std::thread::JoinHandle<()>>,
}
impl MicCapture {
/// Open AAudio (LowLatency, 48 kHz/stereo/f32) for **input** with a realtime callback that
/// forwards captured PCM to a channel, then spawn the Opus encode + uplink thread. `None` on
/// failure (the caller leaves the rest of the session streaming).
pub fn start(client: Arc<NativeClient>) -> Option<MicCapture> {
/// Open AAudio (LowLatency, 48 kHz/mono/f32) for **input** with a realtime callback that
/// forwards captured PCM to a channel, then spawn the Opus encode + uplink thread. With
/// `echo_cancel` the stream opens under the `VoiceCommunication` input preset the HAL's own
/// echo canceller / noise suppressor on the capture path (the default `VoiceRecognition`
/// preset deliberately bypasses them, which is why the host used to hear its own stream back
/// from a speaker-playing phone) — and allocates an audio session id for Kotlin's Java-effect
/// backstop. `None` on failure (the caller leaves the rest of the session streaming).
///
/// `muted` is the SESSION's live mic-mute flag (owned by `SessionHandle`, not by this capture),
/// honoured per frame by [`encode_loop`]. Sharing it rather than owning it is what makes mute
/// survive the mic stop/start a surface recreate performs — and means a capture started while
/// muted never encodes its first frame, so there is no window for one to escape.
pub fn start(
client: Arc<NativeClient>,
echo_cancel: bool,
muted: Arc<AtomicBool>,
) -> Option<MicCapture> {
let captured = Arc::new(AtomicU64::new(0));
// Chunks discarded on the capture thread (free-list empty / encoder lagging); logged
// throttled from the encode worker.
@@ -52,7 +84,9 @@ impl MicCapture {
// One open attempt at a given sharing mode (same pattern as [`crate::audio`]: `open_stream`
// consumes the builder AND the callback, so each try rebuilds the channels it captures).
let try_open = |sharing: AudioSharingMode| -> ndk::audio::Result<(
let try_open = |sharing: AudioSharingMode,
voice: bool|
-> ndk::audio::Result<(
AudioStream,
Receiver<Vec<f32>>,
SyncSender<Vec<f32>>,
@@ -99,13 +133,25 @@ impl MicCapture {
AudioCallbackResult::Continue
};
let stream = AudioStreamBuilder::new()?
// NOTE: no `.frames_per_data_callback(...)`: AAudio's own docs call leaving it unset
// the lowest-latency path (the callback then runs at the device's optimal burst,
// while pinning a size inserts an adaptation buffer), and the encode side re-chunks
// to 10 ms frames regardless of how the bursts arrive.
let mut builder = AudioStreamBuilder::new()?
.direction(AudioDirection::Input)
.sample_rate(SAMPLE_RATE)
.channel_count(CHANNELS as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(sharing)
.sharing_mode(sharing);
if voice {
// VoiceCommunication routes the capture through the HAL's AEC/NS; the allocated
// session id (`None` = allocate) is what Kotlin attaches the Java effects to.
builder = builder
.input_preset(AudioInputPreset::VoiceCommunication)
.session_id(None);
}
let stream = builder
.data_callback(Box::new(callback))
.error_callback(Box::new(|_s, e| {
log::warn!("mic: AAudio error (device reroute/disconnect?): {e:?}");
@@ -114,21 +160,52 @@ impl MicCapture {
Ok((stream, rx, free_tx))
};
// Exclusive first — MMAP-exclusive is AAudio's lowest-latency path — falling back to Shared
// when the device refuses (no MMAP, mic claimed, …). The started-log below prints the mode
// the device actually GRANTED (`share=`).
let (stream, rx, free_tx) = match try_open(AudioSharingMode::Exclusive) {
Ok(opened) => opened,
Err(e) => {
log::info!("mic: Exclusive open failed ({e}) — retrying Shared");
match try_open(AudioSharingMode::Shared) {
Ok(opened) => opened,
Err(e) => {
log::error!("mic: open_stream (RECORD_AUDIO granted?): {e}");
return None;
}
// Exclusive first — MMAP-exclusive is AAudio's lowest-latency path — falling back to
// Shared when the device refuses (no MMAP, mic claimed, …); and each sharing mode with
// the voice preset before without it, because some HALs reject VoiceCommunication (or a
// session id) outright and a mic without echo cancellation still beats no mic. The
// ladder's last rungs are exactly the preset-less open this always did. The started-log
// below prints what the device actually GRANTED (`share=`/`session=`).
let attempts: &[(AudioSharingMode, bool)] = if echo_cancel {
&[
(AudioSharingMode::Exclusive, true),
(AudioSharingMode::Shared, true),
(AudioSharingMode::Exclusive, false),
(AudioSharingMode::Shared, false),
]
} else {
&[
(AudioSharingMode::Exclusive, false),
(AudioSharingMode::Shared, false),
]
};
let mut opened = None;
for &(sharing, voice) in attempts {
match try_open(sharing, voice) {
Ok(o) => {
opened = Some(o);
break;
}
Err(e) => log::info!(
"mic: open {sharing:?}{} failed ({e}) — trying the next fallback",
if voice { "+VoiceCommunication" } else { "" },
),
}
}
let (stream, rx, free_tx) = match opened {
Some(o) => o,
None => {
log::error!("mic: open_stream (RECORD_AUDIO granted?): every mode refused");
return None;
}
};
// The session id AAudio actually allocated (only a voice rung asks for one): `> 0` is the
// handle Kotlin hangs the Java AcousticEchoCanceler/NoiseSuppressor off as the HAL
// preset's backstop; `0` = none, nothing to attach.
let session_id = match stream.session_id() {
SessionId::Allocated(id) => id.get(),
SessionId::None => 0,
};
if let Err(e) = stream.request_start() {
@@ -136,7 +213,7 @@ impl MicCapture {
return None;
}
log::info!(
"mic: AAudio input started rate={} ch={} fmt={:?} share={:?}",
"mic: AAudio input started rate={} ch={} fmt={:?} share={:?} session={session_id}",
stream.sample_rate(),
stream.channel_count(),
stream.format(),
@@ -147,15 +224,21 @@ impl MicCapture {
let sd = shutdown.clone();
let join = std::thread::Builder::new()
.name("pf-mic".into())
.spawn(move || encode_loop(client, rx, free_tx, sd, captured, dropped))
.spawn(move || encode_loop(client, rx, free_tx, sd, muted, captured, dropped))
.ok();
Some(MicCapture {
_stream: stream,
session_id,
shutdown,
join,
})
}
/// The audio-session id AAudio allocated (`> 0`; see [`MicCapture::start`]), `0` = none.
pub fn session_id(&self) -> i32 {
self.session_id
}
}
impl Drop for MicCapture {
@@ -168,20 +251,29 @@ impl Drop for MicCapture {
}
}
/// Consumer: drain captured f32 → accumulate → Opus `encode_float` 20 ms stereo frames → `send_mic`.
/// Consumer: drain captured f32 → accumulate → Opus `encode_float` 10 ms mono frames → `send_mic`.
/// Drained chunk buffers go back to the callback's free-list; the encode scratch is reused across
/// frames (only the packet Vec handed to `send_mic` is allocated per frame — it's sent away owned).
///
/// While `muted` is set a formed frame is dropped instead of encoded (see the frame loop) — the
/// capture side keeps running exactly as it does unmuted, so nothing about the stream, its ring or
/// its backlog behaviour changes across a toggle.
fn encode_loop(
client: Arc<NativeClient>,
rx: Receiver<Vec<f32>>,
free_tx: SyncSender<Vec<f32>>,
shutdown: Arc<AtomicBool>,
muted: Arc<AtomicBool>,
captured: Arc<AtomicU64>,
dropped: Arc<AtomicU64>,
) {
// Fold this Opus-encode/uplink thread into the client's hot-thread set so the ADPF session the
// decode thread opens keeps mic encode on a fast core too (the playback side's decode_loop
// does the same). No-op below API 33.
client.register_hot_thread();
let mut enc = match opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Channels::Mono,
opus::Application::Voip,
) {
Ok(e) => e,
@@ -191,13 +283,21 @@ fn encode_loop(
}
};
let _ = enc.set_bitrate(opus::Bitrate::Bits(MIC_BITRATE));
// Speech tuning: complexity 5 roughly halves encode cost for no audible loss at this rate,
// and in-band FEC at an assumed 10% loss lets the host's decoder reconstruct a dropped
// datagram from its successor instead of playing a hole (the uplink is fire-and-forget).
let _ = enc.set_complexity(5);
let _ = enc.set_inband_fec(true);
let _ = enc.set_packet_loss_perc(10);
let frame = FRAME_SAMPLES * CHANNELS;
let mut ring: VecDeque<f32> = VecDeque::with_capacity(frame * 4);
let mut pcm = vec![0f32; frame]; // reusable encode scratch (one 20 ms frame)
let mut out = vec![0u8; 4000]; // max Opus packet for a 20 ms frame fits easily
let mut pcm = vec![0f32; frame]; // reusable encode scratch (one 10 ms frame)
let mut out = vec![0u8; 4000]; // max Opus packet for a 10 ms frame fits easily
let mut seq: u32 = 0;
let mut sent: u64 = 0;
let mut stale: u64 = 0; // frames shed by the backlog self-heal (see BACKLOG_MAX_FRAMES)
let mut muted_frames: u64 = 0; // frames dropped unencoded because the user muted
let mut peak = 0f32; // loudest |sample| since the last log — tells speech from silence
while !shutdown.load(Ordering::Relaxed) {
@@ -207,11 +307,41 @@ fn encode_loop(
// callback's free-list (dropped only if the pool is momentarily full).
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
// Drain whatever else queued while we were away, so a post-stall backlog lands as
// ONE lump the self-heal below can size up — chunk-at-a-time it would be encoded
// (and inflicted on the host as standing delay) before it ever looked deep.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
}
Err(RecvTimeoutError::Timeout) => continue, // wake to re-check shutdown
Err(RecvTimeoutError::Disconnected) => break,
}
// Self-heal the latency ratchet: a stall (scheduler hiccup, a slow send) queues stale
// audio, and every ms of it would ride the stream as mic delay for the rest of the
// session. Jump to the newest ~20 ms (one audible blip), counting the shed.
if ring.len() > BACKLOG_MAX_FRAMES * frame {
let excess = ring.len() - BACKLOG_KEEP_FRAMES * frame;
ring.drain(..excess);
stale += (excess / frame) as u64;
}
while ring.len() >= frame {
// Muted: drop the frame at the last point before it would become an Opus packet —
// room audio is never encoded and nothing goes on the wire. `seq` does NOT advance:
// it numbers the datagrams the host de-jitters, and that side reads a seq jump as
// loss (conceal + a counted gap) where a mute is a pause. Freezing it means the
// frame after an unmute continues the chain, which is what the host's own
// `reset_stream` doc calls for and what the desktop uplink does. (Encoding silence
// instead would keep a pointless uplink and a host-side ring alive for the whole
// mute.) `peak` is the loudest sample the UPLINK carried since the last log, so a
// dropped frame resets rather than raises it.
if muted.load(Ordering::Relaxed) {
ring.drain(..frame);
muted_frames += 1;
peak = 0.0;
continue;
}
for (dst, src) in pcm.iter_mut().zip(ring.drain(..frame)) {
*dst = src;
}
@@ -227,9 +357,10 @@ fn encode_loop(
let _ = client.send_mic(seq, pts, out[..len].to_vec());
seq = seq.wrapping_add(1);
sent += 1;
if sent % 250 == 0 {
if sent % 500 == 0 {
log::info!(
"mic: sent={sent} captured_frames={} dropped_chunks={} peak={peak:.3}",
"mic: sent={sent} captured_frames={} dropped_chunks={} \
stale_frames={stale} muted_frames={muted_frames} peak={peak:.3}",
captured.load(Ordering::Relaxed),
dropped.load(Ordering::Relaxed),
);
@@ -241,7 +372,8 @@ fn encode_loop(
}
}
log::info!(
"mic: stopped (sent={sent} captured_frames={} dropped_chunks={})",
"mic: stopped (sent={sent} captured_frames={} dropped_chunks={} stale_frames={stale} \
muted_frames={muted_frames})",
captured.load(Ordering::Relaxed),
dropped.load(Ordering::Relaxed),
);
+46 -3
View File
@@ -83,6 +83,28 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetLowLaten
punktfunk_core::transport::set_dscp_default(enabled != 0);
}
/// `debug.punktfunk.force_parts` = 1: arm slice-progressive parts delivery even when the
/// Kotlin `FEATURE_PartialFrame` probe said no — the rebuild-free on-glass experiment for a
/// decoder that may accept `BUFFER_FLAG_PARTIAL_FRAME` without declaring the feature (the NP3's
/// c2.qti decoders declare nothing). Android-only; everywhere else the probe verdict stands.
#[cfg(target_os = "android")]
fn force_parts_sysprop() -> bool {
let mut buf = [0u8; 92]; // PROP_VALUE_MAX
// SAFETY: __system_property_get with a valid name + PROP_VALUE_MAX buffer is always safe.
let n = unsafe {
libc::__system_property_get(
c"debug.punktfunk.force_parts".as_ptr(),
buf.as_mut_ptr().cast(),
)
};
n > 0 && std::str::from_utf8(&buf[..n as usize]).unwrap_or("").trim() == "1"
}
#[cfg(not(target_os = "android"))]
fn force_parts_sysprop() -> bool {
false
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs, launch,
/// deviceName): Long`.
@@ -155,6 +177,24 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
} else {
Some((cert, key))
};
// Slice-progressive parts, by decoder truth (Kotlin's FEATURE_PartialFrame probe) — with a
// sysprop escape hatch for the on-glass science question the probe can't answer: does the
// decoder ACTUALLY choke on BUFFER_FLAG_PARTIAL_FRAME input, or does it merely not declare
// the feature? (`adb shell setprop debug.punktfunk.force_parts 1` + stream restart; a codec
// that can't take parts errors recoverably and the reanchor gate + keyframe path recovers.)
let force_parts = force_parts_sysprop();
let frame_parts = frame_parts_ok != 0 || force_parts;
// The connect-time capability readout (`adb logcat -s pf.caps`): the P2 slice pipeline is
// inert client-side unless BOTH probes pass — this line is the one place that says which.
log::info!(
target: "pf.caps",
"decoder caps: multi_slice={} partial_frame={}{} hdr={} codec_bits={:#x}",
multi_slice_ok != 0,
frame_parts_ok != 0,
if force_parts { " (FORCED by sysprop)" } else { "" },
hdr_enabled != 0,
video_codecs,
);
let pin: Option<[u8; 32]> = if pin_hex.is_empty() {
None
} else {
@@ -230,9 +270,10 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// should say what the client does).
punktfunk_core::quic::CLIENT_CAP_PHASE_LOCK,
// Slice-progressive delivery, by decoder truth (Kotlin probes FEATURE_PartialFrame on
// every decoder this device would use): AU prefixes then arrive as `Frame::part`
// pieces and the decode loop feeds them with BUFFER_FLAG_PARTIAL_FRAME.
frame_parts_ok != 0,
// every decoder this device would use; `debug.punktfunk.force_parts` overrides for the
// on-glass experiment): AU prefixes then arrive as `Frame::part` pieces and the decode
// loop feeds them with BUFFER_FLAG_PARTIAL_FRAME.
frame_parts,
launch, // a store-qualified library id to boot into a game, or None for the desktop
device_name, // Kotlin's Build.MODEL — the host's approval-list / trust-store label
pin, // Some → Crypto on host-fp mismatch
@@ -250,6 +291,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
audio: Mutex::new(None),
#[cfg(target_os = "android")]
mic: Mutex::new(None),
// A fresh session is never muted (mute is per-session UI state, not a setting).
mic_muted: Arc::new(std::sync::atomic::AtomicBool::new(false)),
};
Box::into_raw(Box::new(handle)) as jlong
}
@@ -61,6 +61,13 @@ pub(crate) struct SessionHandle {
audio: Mutex<Option<crate::audio::AudioPlayback>>,
#[cfg(target_os = "android")]
mic: Mutex<Option<crate::mic::MicCapture>>,
/// In-stream mic mute, set via `nativeSetMicMuted` and read per 10 ms frame by the mic's
/// encode loop ([`crate::mic`]). Session-lifetime rather than per-[`crate::mic::MicCapture`]
/// for the same reason the stats gate is: the mic stops and restarts across a surface
/// recreate, and a mute the user set must come back with it — with no window in which the
/// fresh capture could send an unmuted frame. Per session and never persisted: a new session
/// starts unmuted.
pub mic_muted: Arc<AtomicBool>,
}
struct VideoThread {
+98 -15
View File
@@ -177,11 +177,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD
/// (unified stats spec, `design/stats-unification.md`). Returns 26 doubles
/// (unified stats spec, `design/stats-unification.md`). Returns 33 doubles
/// `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
/// netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
/// e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive]`
/// e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
/// feedP50Ms, codecP50Ms, skippedOverflowWindow]`
/// (the flags are 1.0/0.0; indexes 021 match the previous 22-double layout — 013 the original
/// 14-double one with the latency pair re-based to the end-to-end capture→decoded headline, 14/15
/// the stage p50s tiling it: `host+network` = capture→received, `decode` = received→decoded; 16/17
@@ -198,7 +199,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
/// term — `pace` = decoded→release (store + glass budget) p50 at 26, `latch` =
/// release→displayed (SurfaceFlinger) p50 at 27, the window's on-glass confirm count at 28
/// (`presents` vs `fps` is the presenter-health pair), and 29 = 1.0 while the timeline presenter
/// is active this session), or `null` when no decode thread is running.
/// is active this session; 30/31 are the `decode` stage's split p50s — `feed` =
/// received→queued (hand-off + input-slot wait) at 30 and `codec` = queued→decoded (codec-pure,
/// from the AU's last piece) at 31, both 0.0 when no sample landed (sync loop); 32 is the
/// parked-AU overflow subset of the window's `skipped` at 19 (decoder fell behind, vs benign
/// newest-wins pacing)), or `null` when no decode thread is running.
/// Poll ~1 Hz from the UI; each call
/// resets the measurement window. Not android-gated — pure `jni` + connector reads, so it links on
/// the host build too (Kotlin only ever calls it on device).
@@ -222,7 +227,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
.drain(h.client.frames_dropped(), h.client.fec_recovered_shards());
let mode = h.client.mode();
let color = h.client.color;
let buf: [f64; 30] = [
let buf: [f64; 33] = [
snap.fps,
snap.mbps,
snap.e2e_p50_ms,
@@ -270,6 +275,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
snap.latch_p50_ms,
snap.presents as f64,
if h.stats.presenter_active() { 1.0 } else { 0.0 },
// The `decode` stage's split (P3 science): feed = received→queued (hand-off +
// input-slot wait), codec = queued→decoded (codec-pure) — and the parked-AU
// overflow subset of `skipped` (decoder-health vs benign pacing drops).
snap.feed_p50_ms,
snap.codec_p50_ms,
snap.skipped_overflow as f64,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
@@ -390,33 +401,49 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopAudio(
})
}
/// `NativeBridge.nativeStartMic(handle)` — start mic capture (AAudio input → Opus → host `send_mic`).
/// No-op if already running or on a `0` handle. Caller MUST hold RECORD_AUDIO; a failure (e.g. no
/// permission) leaves the rest of the session streaming.
/// `NativeBridge.nativeStartMic(handle, echoCancel): Int` — start mic capture (AAudio input →
/// Opus → host `send_mic`). `echoCancel` opens the capture under the `VoiceCommunication` preset
/// (the HAL's echo canceller / noise suppressor) and allocates an audio session id; the return
/// value is that id (`> 0`), so Kotlin can attach the Java `AcousticEchoCanceler`/`NoiseSuppressor`
/// as a backstop — `0` when none was allocated (echoCancel off, the preset fell back to the plain
/// open, a `0` handle, or the mic failed entirely). Already running (a surface recreate) returns
/// the running capture's id. Caller MUST hold RECORD_AUDIO; a failure (e.g. no permission) leaves
/// the rest of the session streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartMic(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
echo_cancel: jboolean,
) -> jni::sys::jint {
if handle == 0 {
return;
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.mic.lock().unwrap();
if guard.is_some() {
return; // already capturing
if let Some(m) = guard.as_ref() {
return m.session_id(); // already capturing — same stream, same session
}
match crate::mic::MicCapture::start(h.client.clone()) {
Some(m) => *guard = Some(m),
None => log::error!("nativeStartMic: mic init failed (RECORD_AUDIO? — session unaffected)"),
// The capture SHARES the session's mute flag, so one started while muted stays muted (and
// sends nothing) from its very first frame — see `SessionHandle::mic_muted`.
match crate::mic::MicCapture::start(h.client.clone(), echo_cancel != 0, h.mic_muted.clone()) {
Some(m) => {
let session_id = m.session_id();
*guard = Some(m);
session_id
}
None => {
log::error!("nativeStartMic: mic init failed (RECORD_AUDIO? — session unaffected)");
0
}
}
}
/// `NativeBridge.nativeStopMic(handle)` — stop + join the mic thread and close the AAudio input
/// stream (without closing the session). No-op on `0`.
/// stream (without closing the session). No-op on `0`. Leaves the session's mute state alone: a
/// surface recreate stops and restarts the mic, and a user who muted must stay muted through it.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopMic(
@@ -432,3 +459,59 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopMic(
}
})
}
/// `NativeBridge.nativeSetMicMuted(handle, muted)` — mute/unmute the mic uplink mid-stream.
///
/// Muting deliberately does NOT stop the capture: the AAudio input stream, the input-preset rung
/// it settled on and its primed buffers all stay exactly as they are, and the encode loop simply
/// drops each 10 ms frame instead of encoding + sending it. A stop/start would re-run the preset
/// fallback ladder and re-prime buffers on every toggle — hundreds of ms, and possibly a different
/// rung (echo cancellation silently lost). This way a toggle costs one atomic store here and one
/// relaxed load per frame there, and takes effect on the very next 10 ms boundary.
///
/// Sticky for the SESSION (the flag lives on the handle, not on the capture), so the mic restart a
/// surface recreate performs comes back muted with no window for an unmuted frame to escape; a
/// fresh session always starts unmuted. No-op on `0`. Not android-gated — pure `jni` + an atomic
/// store, so it links on the host build too.
///
/// One honest consequence of keeping the stream open: the platform's own recording indicator stays
/// lit while muted, because the mic really is still open. What stops is the encode and the send —
/// no captured audio leaves the process.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetMicMuted(
_env: JNIEnv,
_this: JObject,
handle: jlong,
muted: jboolean,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.mic_muted
.store(muted != 0, std::sync::atomic::Ordering::Relaxed);
}
})
}
/// `NativeBridge.nativeMicActive(handle): Boolean` — is a mic capture actually RUNNING? `true` only
/// between a `nativeStartMic` that opened a stream and the matching `nativeStopMic`. The in-stream
/// mute control is offered on this evidence rather than on the user's setting, so a device that
/// refused every AAudio input rung (or a missing RECORD_AUDIO grant) shows no control instead of a
/// lie about a mic that is being heard. `false` on a `0` handle. Cheap (one uncontended lock).
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeMicActive(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
jni_guard(0, || {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
jboolean::from(h.mic.lock().unwrap().is_some())
})
}
+70
View File
@@ -76,6 +76,12 @@ struct Inner {
/// The other half of the split, release→displayed (SurfaceFlinger's latch + scanout), µs —
/// from the `OnFrameRendered` render timestamps. `pace + latch ≈ display` per frame.
latch_us: Vec<u64>,
/// The `decode` stage's feed split, received→queued (hand-off + input-slot wait), µs. Empty
/// when no receipt stamp matched (HUD off and ABR not measuring decode).
feed_us: Vec<u64>,
/// The other half, queued→decoded (codec-pure: the decoder's own time on the AU, measured
/// from its LAST piece so a slice-progressive head start shows up as a shrink here), µs.
codec_us: Vec<u64>,
/// Frames confirmed on glass this window (`OnFrameRendered` callbacks) — the `presents`-vs-
/// `fps` health pair: presents ≪ fps means the presenter is dropping/serializing; an fps
/// deficit is upstream.
@@ -83,6 +89,10 @@ struct Inner {
/// Client-side newest-wins/pacing drops this window (decoded frames released without
/// rendering, or parked AUs dropped on overflow) — the spec's `skipped` counter.
skipped: u64,
/// The subset of `skipped` that was parked-AU OVERFLOW (the decoder fell behind and whole
/// AUs were dropped before feeding) — a decoder-health signal, vs the benign newest-wins
/// pacing majority. Always ≤ `skipped`.
skipped_overflow: u64,
/// Baselines for windowing the session-cumulative connector counters: the unrecoverable-drop
/// and FEC-recovered totals as of the last drain (or the enable that opened the window), so
/// each snapshot reports only THIS window's `lost` / `FEC` (spec line 4).
@@ -119,6 +129,11 @@ pub struct Snapshot {
/// path / no render callbacks).
pub pace_p50_ms: f64,
pub latch_p50_ms: f64,
/// The `decode` stage's split p50s (ms): `feed` = received→queued (hand-off + input-slot
/// wait), `codec` = queued→decoded (codec-pure, from the AU's last piece). 0.0 when no
/// sample landed (sync loop / no receipt stamps).
pub feed_p50_ms: f64,
pub codec_p50_ms: f64,
/// Frames confirmed on glass this window (`OnFrameRendered` callbacks).
pub presents: u64,
/// Phase-2 `host` / `network` split p50s (ms) — 0.0 when no 0xCF timing matched this window
@@ -135,6 +150,8 @@ pub struct Snapshot {
pub lost: u64,
/// Client-side newest-wins/pacing drops this window (spec `skipped`).
pub skipped: u64,
/// The parked-AU overflow subset of `skipped` (decoder fell behind; ≤ `skipped`).
pub skipped_overflow: u64,
/// FEC shards recovered this window (spec `FEC`, windowed from the cumulative counter).
pub fec: u64,
}
@@ -167,8 +184,11 @@ impl VideoStats {
e2e_disp_us: Vec::with_capacity(256),
pace_us: Vec::with_capacity(256),
latch_us: Vec::with_capacity(256),
feed_us: Vec::with_capacity(256),
codec_us: Vec::with_capacity(256),
presents: 0,
skipped: 0,
skipped_overflow: 0,
last_dropped_total: 0,
last_fec_total: 0,
skew_corrected: false,
@@ -219,8 +239,11 @@ impl VideoStats {
g.e2e_disp_us.clear();
g.pace_us.clear();
g.latch_us.clear();
g.feed_us.clear();
g.codec_us.clear();
g.presents = 0;
g.skipped = 0;
g.skipped_overflow = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
}
@@ -314,6 +337,45 @@ impl VideoStats {
g.skipped += n;
}
/// Record parked-AU OVERFLOW drops (whole AUs dropped before feeding — the decoder fell
/// behind). Counts into `skipped` too, plus the overflow-only counter, so the HUD can tell
/// benign newest-wins pacing from a decoder that can't keep up.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_skipped_overflow(&self, n: u64) {
if n == 0 || !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.skipped += n;
g.skipped_overflow += n;
}
/// Record one decoded frame's `decode`-stage split: `feed` = received→queued (hand-off +
/// input-slot wait; absent when no receipt stamp matched) and `codec` = queued→decoded
/// (codec-pure, measured from the AU's LAST piece — a slice-progressive head start shows
/// as a shrink here), both µs.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_decode_split(&self, feed_us: Option<u64>, codec_us: u64) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(f) = feed_us {
g.feed_us.push(f);
}
g.codec_us.push(codec_us);
}
/// Record one decoded output frame: its capture→decoded `end-to-end` sample and its
/// received→decoded `decode` stage sample (either may be absent — e.g. the receipt stamp for
/// this pts predates the HUD being shown).
@@ -408,6 +470,8 @@ impl VideoStats {
g.e2e_disp_us.sort_unstable();
g.pace_us.sort_unstable();
g.latch_us.sort_unstable();
g.feed_us.sort_unstable();
g.codec_us.sort_unstable();
let snap = Snapshot {
fps,
mbps,
@@ -421,6 +485,8 @@ impl VideoStats {
disp_valid: !g.e2e_disp_us.is_empty(),
pace_p50_ms: pctl_ms(&g.pace_us, 0.50),
latch_p50_ms: pctl_ms(&g.latch_us, 0.50),
feed_p50_ms: pctl_ms(&g.feed_us, 0.50),
codec_p50_ms: pctl_ms(&g.codec_us, 0.50),
presents: g.presents,
host_p50_ms: pctl_ms(&g.host_us, 0.50),
net_p50_ms: pctl_ms(&g.net_us, 0.50),
@@ -429,6 +495,7 @@ impl VideoStats {
frames: g.frames,
lost: dropped_total.saturating_sub(g.last_dropped_total),
skipped: g.skipped,
skipped_overflow: g.skipped_overflow,
fec: fec_total.saturating_sub(g.last_fec_total),
};
g.window_start = Instant::now();
@@ -443,8 +510,11 @@ impl VideoStats {
g.e2e_disp_us.clear();
g.pace_us.clear();
g.latch_us.clear();
g.feed_us.clear();
g.codec_us.clear();
g.presents = 0;
g.skipped = 0;
g.skipped_overflow = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
snap
@@ -315,7 +315,15 @@ struct ContentView: View {
clipboardAvailable: model.connection?.hostSupportsClipboard == true,
clipboardOn: model.clipboardEnabled,
toggleClipboard: { model.toggleClipboardSync() },
micAvailable: model.micAvailable,
micMuted: model.micMuted,
toggleMicMute: { model.toggleMicMute() },
disconnect: { model.disconnect() }))
// A fired while input was CAPTURED (InputCapture's chord path posts it the menu's
// identical equivalent can't reach a captured stream). Same toggle either way.
.onReceive(NotificationCenter.default.publisher(for: .punktfunkToggleMicMute)) { _ in
model.toggleMicMute()
}
#endif
#if os(macOS)
// Fullscreen only while a session is up (incl. the trust prompt over the blurred stream),
@@ -724,31 +732,48 @@ struct ContentView: View {
}
.animation(.smooth(duration: 0.28), value: statsVerbosity)
}
#if os(macOS) || os(tvOS)
// The start-of-stream shortcut banner (Windows-client parity): the platform's
// reserved controls on a glass pill, bottom-centre, for the first 6 seconds of
// every session independent of the stats HUD, so the keys are discoverable
// even with statistics off. The banner's own task drops it (cancelled cleanly
// if the session view goes away first). On tvOS it carries the ONLY exits
// Menu/B is swallowed during a session (the `.onExitCommand {}` in the tvOS
// session branch), so the hold gestures must be told to the user.
// The bottom-centre stack: the muted-microphone badge over the start-of-stream
// shortcut banner. ONE overlay for both, so the two can never land on top of each
// other in the seconds where they overlap.
.overlay(alignment: .bottom) {
if captureEnabled && showShortcutHint {
Text(Self.shortcutHintText)
.font(.geist(Self.shortcutHintFont, relativeTo: .caption))
.foregroundStyle(.secondary)
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.padding(.bottom, 24)
.transition(.opacity)
.task {
try? await Task.sleep(for: .seconds(6))
withAnimation(.easeOut(duration: 0.6)) { showShortcutHint = false }
}
VStack(spacing: 8) {
#if !os(tvOS)
// Shown for as long as the mic is muted, at every stats tier and with the
// overlay off see MicMutedBadge. tvOS has no microphone to mute.
if captureEnabled && model.micMuted {
MicMutedBadge { model.setMicMuted(false) }
.transition(.opacity.combined(with: .scale(scale: 0.9)))
}
#endif
#if os(macOS) || os(tvOS)
// The start-of-stream shortcut banner (Windows-client parity): the
// platform's reserved controls on a glass pill for the first 6 seconds of
// every session independent of the stats HUD, so the keys are
// discoverable even with statistics off. The banner's own task drops it
// (cancelled cleanly if the session view goes away first). On tvOS it
// carries the ONLY exits Menu/B is swallowed during a session (the
// `.onExitCommand {}` in the tvOS session branch), so the hold gestures
// must be told to the user.
if captureEnabled && showShortcutHint {
Text(shortcutHintText)
.font(.geist(Self.shortcutHintFont, relativeTo: .caption))
.foregroundStyle(.secondary)
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.transition(.opacity)
.task {
try? await Task.sleep(for: .seconds(6))
withAnimation(.easeOut(duration: 0.6)) {
showShortcutHint = false
}
}
}
#endif
}
.padding(.bottom, 24)
.animation(.easeOut(duration: 0.2), value: model.micMuted)
}
#endif
#if os(iOS)
// Touch users have no menu / D, so when the HUD's Disconnect button isn't on
// screen the overlay off, or the compact pill (which carries no button)
@@ -766,21 +791,24 @@ struct ContentView: View {
.overlay(alignment: .topLeading) {
if captureEnabled,
statsVerbosity == .compact || (statsVerbosity == .off && showTouchExit) {
Button { model.disconnect() } label: {
Image(systemName: "xmark")
.font(.headline.weight(.semibold))
.frame(width: 36, height: 36)
// Floating glass disc over the frame (26+, material fallback).
// interactive: the disc IS the tap target, so the glass reacts
// to press.
.glassBackground(Circle(), interactive: true)
// Match the hit region to the visible disc so every tap also
// triggers the interactive-glass press highlight.
.contentShape(Circle())
HStack(spacing: 10) {
Button { model.disconnect() } label: { touchDisc("xmark") }
.buttonStyle(.plain)
.accessibilityLabel("Disconnect")
// The mic toggle rides the same discs, for the same reason: in these
// tiers the HUD carries no buttons (compact is a stat pill, off is
// nothing), so this is a touch-only user's ONLY way to mute. Absent
// not greyed when the session sends no microphone at all.
if model.micAvailable {
Button { model.toggleMicMute() } label: {
touchDisc(model.micMuted ? "mic.slash.fill" : "mic.fill")
}
.buttonStyle(.plain)
.accessibilityLabel(
model.micMuted ? "Unmute microphone" : "Mute microphone")
}
}
.buttonStyle(.plain)
.padding(12)
.accessibilityLabel("Disconnect")
.transition(.opacity)
.task {
guard statsVerbosity == .off else { return }
@@ -794,14 +822,34 @@ struct ContentView: View {
}
}
#if os(iOS)
/// One touch-control disc: an SF Symbol on a floating glass disc over the frame (26+,
/// material fallback), sized as a comfortable tap target. `interactive`: the disc IS the tap
/// target, so the glass reacts to press, and the hit region is matched to the visible disc so
/// every tap triggers that press highlight.
private func touchDisc(_ symbol: String) -> some View {
Image(systemName: symbol)
.font(.headline.weight(.semibold))
.frame(width: 36, height: 36)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
#endif
#if os(macOS)
private static let shortcutHintText =
"Click the stream to capture · ⌃⌥⇧Q releases the mouse · ⌃⌥⇧D disconnects · ⌃⌥⇧S stats"
/// The reserved combos, told once per session. The mute segment appears only when the session
/// actually sends a microphone teaching a shortcut for a mic that isn't on would be a lie.
private var shortcutHintText: String {
let base =
"Click the stream to capture · ⌃⌥⇧Q releases the mouse · ⌃⌥⇧D disconnects · ⌃⌥⇧S stats"
return model.micAvailable ? base + " · ⌃⌥⇧A mutes the mic" : base
}
private static let shortcutHintFont: CGFloat = 12
#elseif os(tvOS)
private static let shortcutHintText =
private var shortcutHintText: String {
"Hold the remote's Back button — or L1+R1+Start+Select on a controller — to disconnect"
+ " · Touch surface moves the pointer · press clicks · Play/Pause right-clicks"
}
private static let shortcutHintFont: CGFloat = 22 // read from the couch
#endif
@@ -1,6 +1,9 @@
// Session state for the app shell: owns the connection, the input capture, the trust
// handshake phase, and the pump-thread main-actor stats relay.
// AVFoundation: AVCaptureDevice.authorizationStatus (the mic TCC grant behind `micAvailable`)
// and, on tvOS, AVPlayer.eligibleForHDRPlayback (the TV-capability HDR gate).
import AVFoundation
import Foundation
import os
import PunktfunkKit
@@ -11,9 +14,6 @@ import SwiftUI
#elseif canImport(UIKit)
import UIKit
#endif
#if os(tvOS)
import AVFoundation // AVPlayer.eligibleForHDRPlayback the TV-capability HDR gate
#endif
/// 1 Hz latency-stage line mirrored to the unified log so the stages can be read WITHOUT the
/// on-screen HUD (Console.app, wirelessly on an iPad/Apple TV). The HUD is not a neutral
@@ -137,6 +137,14 @@ final class SessionModel: ObservableObject {
/// Mirrors StreamView's capture state (it owns the input capture; this drives the
/// HUD's "click to capture" / " releases" hint).
@Published var mouseCaptured = false
/// The USER's in-stream mic mute (the HUD button, the Stream menu's A, the captured-state
/// chord, the iOS mic disc) session state, deliberately NOT persisted: a mute is for the
/// people in the room right now, so every new session starts live if the mic is on at all.
/// One of the two inputs to the effective mute; `isBackgrounded` is the other, and
/// `applyMicMute` composes them a user mute survives a trip through the background, and the
/// background's privacy mute never clears the user's choice. Local and instant: it gates
/// capture on this device, nothing is sent to the host.
@Published private(set) var micMuted = false
/// Resize overlay (design/midstream-resolution-resize.md client resize UX): true from the
/// instant a Match-window resize starts steering toward a new size until a frame at that size
/// decodes (or a safety timeout). Drives the blur+spinner so the unavoidable host-rebuild delay
@@ -440,7 +448,7 @@ final class SessionModel: ObservableObject {
guard phase == .streaming, let conn = connection, !isBackgrounded else { return }
isBackgrounded = true
conn.setVideoDropped(true)
audio?.setMicMuted(true)
applyMicMute() // now muted for privacy on top of the user's own mute, not instead of it
// Non-deliberate on fire (keep the host linger) so a user who returns late reconnects fast,
// exactly like today's network-drop path. min 1 minute guards a nonsense setting.
let minutes = max(1, timeoutMinutes)
@@ -465,13 +473,57 @@ final class SessionModel: ObservableObject {
backgroundDeadline = nil
backgroundTimer?.cancel()
backgroundTimer = nil
audio?.setMicMuted(false)
applyMicMute() // back to the user's own choice which may well still be "muted"
if let conn = connection {
conn.setVideoDropped(false)
conn.requestKeyframe()
}
}
// MARK: - Microphone mute (in-stream, per session)
/// Whether this session has a mic uplink there is any point in muting: the mic must be on in
/// the session's RESOLVED settings (a profile can turn it on or off), the platform must have
/// an app-accessible input at all, and the OS must not have refused us one. Drives whether the
/// mute control is offered a live-looking mute button over a session that sends no
/// microphone would be a lie. Same three conditions `SessionAudio` starts an uplink on
/// (`.notDetermined` counts: the prompt is pending and a grant starts the uplink mid-session).
var micAvailable: Bool {
#if os(tvOS)
return false // no app-accessible microphone SessionAudio never opens an uplink either
#else
guard settings.micEnabled else { return false }
switch AVCaptureDevice.authorizationStatus(for: .audio) {
case .authorized, .notDetermined: return true
default: return false // denied / restricted there is no uplink to mute
}
#endif
}
/// Flip the user's mute. The in-stream surfaces (HUD button, Stream menu, A while
/// captured, the iOS mic disc) all land here.
func toggleMicMute() {
setMicMuted(!micMuted)
}
/// Set the user's mute directly (the badge's tap-to-unmute). Ignored when the session has no
/// microphone, so a stale surface can't leave a phantom "muted" badge over a session that was
/// never sending anything.
func setMicMuted(_ muted: Bool) {
guard micAvailable, micMuted != muted else { return }
micMuted = muted
applyMicMute()
}
/// Push the EFFECTIVE mute the user's choice OR the background keep-alive's privacy mute
/// onto the audio engine. The two reasons are composed here and nowhere else: whichever one
/// changed, the other still holds, so returning from the background can't un-mute a user who
/// muted mid-stream, and a user unmuting while backgrounded (Live Activity, another window)
/// doesn't open the mic behind their back.
private func applyMicMute() {
audio?.setMicMuted(micMuted || isBackgrounded)
}
/// Follow a live stats-overlay cycle (S, the three-finger tap, the Stream menu). Those
/// surfaces write the GLOBAL setting as they always have; this moves the session's own tier
/// with it, so cycling still works in a session a profile put on a different tier.
@@ -509,6 +561,9 @@ final class SessionModel: ObservableObject {
backgroundTimer = nil
isBackgrounded = false
backgroundDeadline = nil
// The mic mute is per-session and never persisted: the next stream starts live (if the
// mic is enabled), rather than silently carrying a mute nobody remembers making.
micMuted = false
let audio = self.audio
self.audio = nil
// Gamepad capture is main-actor (releases held buttons on the wire while the
@@ -609,14 +664,19 @@ final class SessionModel: ObservableObject {
speakerUID: settings.speakerUID,
micUID: settings.micUID,
micChannel: settings.micChannel,
micEnabled: settings.micEnabled)
micEnabled: settings.micEnabled,
echoCancel: settings.echoCancel)
self.audio = audio
// Gamepads: forward every controller GamepadManager selected each on its own wire pad
// index (a pin forwards only one, Automatic forwards all) and render the host's feedback
// back to the pad it's addressed to (rumble always; lightbar/player-LEDs/adaptive-triggers
// when a pad's virtual device is a DualSense). Same trust gate as audio nothing is
// forwarded during the trust prompt.
let capture = GamepadCapture(connection: conn, manager: .shared)
// `gamepadForwarding` off means the host gets this device's pads from somewhere else
// (USB passthrough, or a pad plugged into the host) capture still runs, and still
// watches for the escape chord, but puts nothing on the wire.
let capture = GamepadCapture(
connection: conn, manager: .shared, forwarding: settings.gamepadForwarding)
// The cross-client escape chord (hold L1+R1+Start+Select 1.5 s) on tvOS the only
// controller way out of a stream (B/Menu is swallowed during sessions; see ContentView).
capture.onDisconnectRequest = { [weak self] in self?.disconnect() }
@@ -1,7 +1,8 @@
// The app's "Stream" menu (macOS menu bar + iPad hardware-keyboard shortcuts). These live at
// the Scene level so they keep working when the HUD overlay is hidden. The shortcuts are the
// CROSS-CLIENT set every punktfunk client reserves Ctrl+Alt+Shift+Q (release the captured
// mouse) / +D (disconnect) / +S (stats) and the menu is their discoverable surface on macOS
// mouse) / +D (disconnect) / +S (stats), plus +A (mute the microphone), the Apple clients'
// addition to it and the menu is their discoverable surface on macOS
// (the Linux client has its GTK Shortcuts window, Windows its start-of-stream banner). While
// input is CAPTURED these key equivalents never reach the menu (the stream view swallows
// keys); InputCapture's monitor detects the same combos there and performs the same actions
@@ -27,6 +28,12 @@ struct SessionFocus {
/// Clipboard sync is live (host-acked) drives the item's Stop/Share title.
var clipboardOn: Bool
var toggleClipboard: () -> Void
/// The session has a mic uplink at all (its resolved `micEnabled`) gates the mute item, so
/// it is never an enabled control over a session that sends no microphone.
var micAvailable: Bool
/// The user's mic mute is engaged drives the item's Mute/Unmute title.
var micMuted: Bool
var toggleMicMute: () -> Void
var disconnect: () -> Void
}
@@ -60,6 +67,17 @@ struct StreamCommands: Commands {
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
// Mic mute, local and instant (it gates capture on this device the host is never
// asked). Per SESSION: it starts off every time, so this item is a live toggle, not a
// setting. Greyed when the session sends no microphone at all (Settings mic off, or
// a profile that turns it off) rather than pretending there is something to mute.
// Captured, the combo is handled by InputCapture's chord path before menus see it;
// this item is the released-state path and the shortcut's documentation.
Button(session?.micMuted == true ? "Unmute Microphone" : "Mute Microphone") {
session?.toggleMicMute()
}
.keyboardShortcut("a", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true || session?.micAvailable != true)
#if os(macOS)
// Mid-session clipboard flip (design/clipboard-and-file-transfer.md §5.3). Greyed
// when the host doesn't advertise the cap (older host / operator policy off).
@@ -179,6 +179,18 @@ struct StreamHUDView: View {
.foregroundStyle(.secondary)
}
#endif
// Mic mute the in-stream toggle, on the same card as the other in-overlay action.
// Absent (not greyed) when the session sends no microphone: the HUD is a status card,
// and a dead control on it would read as "there is a mic, and it is on". The muted
// STATE is not this button's job the badge over the stream says that at every tier
// and with the overlay off entirely. tvOS gets no control: no microphone, and a
// focusable one would steal the controller's A press from the host.
#if !os(tvOS)
if model.micAvailable {
Button(micButtonTitle) { model.toggleMicMute() }
.font(.geist(12, relativeTo: .caption))
}
#endif
// D lives on the app's Stream menu (so it still works when the HUD is hidden)
// and in InputCapture's monitor while captured; this button is the in-overlay,
// click-to-disconnect affordance. tvOS deliberately gets NEITHER a button (a
@@ -195,6 +207,19 @@ struct StreamHUDView: View {
}
}
#if !os(tvOS)
/// The mute button's wording. macOS names the chord, exactly as its Disconnect button does;
/// iOS/iPadOS spells the action out (the HUD's buttons there carry no shortcuts, even where a
/// hardware keyboard could fire one the Stream menu is that keyboard's surface).
private var micButtonTitle: String {
#if os(macOS)
return model.micMuted ? "Unmute Mic (⌃⌥⇧A)" : "Mute Mic (⌃⌥⇧A)"
#else
return model.micMuted ? "Unmute Microphone" : "Mute Microphone"
#endif
}
#endif
// MARK: - Card metrics
/// The card's inner content padding. Roomier on tvOS the stat text auto-scales for the
@@ -242,6 +267,42 @@ struct StreamHUDView: View {
}
}
#if !os(tvOS)
/// The muted-microphone badge the mute STATE, as opposed to the buttons that flip it. It rides
/// over the stream whenever the mic is muted, INDEPENDENT of the stats overlay (which the user
/// may have cycled off, and which the compact tier reduces to a stat line): "am I muted?" is not a
/// statistic, and a mute you can't see is how people talk to nobody for a minute. Same glass
/// language as the HUD, sized like the start-of-stream banner it shares the bottom edge with.
///
/// It is also a control: tapping it unmutes. That is the guaranteed way back for a touch user who
/// muted with the overlay off, and it costs the badge nothing (it is on screen either way).
struct MicMutedBadge: View {
let onUnmute: () -> Void
var body: some View {
Button(action: onUnmute) {
HStack(spacing: 7) {
Image(systemName: "mic.slash.fill")
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(.red)
Text("Microphone muted")
.font(.geist(12, .medium, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.9))
}
.padding(.horizontal, 14)
.padding(.vertical, 8)
// interactive: the badge IS the tap target, so the glass reacts to press.
.glassBackground(Capsule(), interactive: true)
.contentShape(Capsule())
}
.buttonStyle(.plain)
.environment(\.colorScheme, .dark) // reads over any frame, like the resize overlay
.accessibilityLabel("Microphone muted")
.accessibilityHint("Unmutes the microphone")
}
}
#endif
#if os(iOS)
/// Device display geometry the overlay needs but UIKit doesn't expose publicly.
enum DeviceMetrics {
@@ -26,12 +26,14 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.streamHz) private var hz = 60
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.gamepadForwarding) private var gamepadForwarding = true
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) private var enable444 = false
@AppStorage(DefaultsKey.codec) private var codec = "auto"
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@AppStorage(DefaultsKey.echoCancel) private var echoCancel = true
// The overlay tier's raw string (rows tag by rawValue); the absent-key default runs the
// legacy-hudEnabled migration (same pattern as ContentView/SettingsView).
@AppStorage(DefaultsKey.statsVerbosity) private var statsVerbosityRaw
@@ -316,9 +318,21 @@ struct GamepadSettingsView: View {
id: "mic", icon: "mic", label: "Microphone",
detail: "Send this device's microphone to the host's virtual mic.",
value: $micEnabled),
toggleRow(
id: "echoCancel", icon: "waveform", label: "Echo cancellation",
detail: "Cancel the audio this device plays out of the mic signal — stops "
+ "speaker setups feeding the game back to the host.",
value: $echoCancel),
toggleRow(
id: "padForward", header: "Controller", icon: "gamecontroller",
label: "Forward controllers",
detail: "Send this device's controllers to the host. Turn it off when your "
+ "controller already reaches the host another way — USB passthrough such "
+ "as VirtualHere — so games don't see two of them.",
value: $gamepadForwarding),
choiceRow(
id: "pad", header: "Controller", icon: "gamecontroller", label: "Use controller",
id: "pad", icon: "gamecontroller", label: "Use controller",
detail: "Which pad is forwarded to the host, as player 1.",
options: controllers, current: gamepads.preferredID
) { gamepads.preferredID = $0 },
@@ -98,6 +98,10 @@ enum SettingsFields {
.init(name: "mic_enabled", key: DefaultsKey.micEnabled,
overlay: \.micEnabled, effective: \.micEnabled)
}
static var echoCancel: SettingsField<Bool> {
.init(name: "echo_cancel", key: DefaultsKey.echoCancel,
overlay: \.echoCancel, effective: \.echoCancel)
}
static var touchMode: SettingsField<String> {
.init(name: "touch_mode", key: DefaultsKey.touchMode,
overlay: \.touchMode, effective: \.touchMode)
@@ -118,6 +122,10 @@ enum SettingsFields {
.init(name: "gamepad", key: DefaultsKey.gamepadType,
overlay: \.gamepadType, effective: \.gamepadType)
}
static var gamepadForwarding: SettingsField<Bool> {
.init(name: "gamepad_forwarding", key: DefaultsKey.gamepadForwarding,
overlay: \.gamepadForwarding, effective: \.gamepadForwarding)
}
static var statsVerbosity: SettingsField<String> {
.init(name: "stats_verbosity", key: DefaultsKey.statsVerbosity,
overlay: \.statsVerbosity, effective: \.statsVerbosity)
@@ -175,7 +183,9 @@ extension SettingsView {
base.compositor = compositor
base.audioChannels = audioChannels
base.micEnabled = micEnabled
base.echoCancel = echoCancel
base.gamepadType = gamepadType
base.gamepadForwarding = gamepadForwarding
base.statsVerbosity = statsVerbosityRaw
base.fullscreenWhileStreaming = fullscreenWhileStreaming
base.presentPriority = presentPriority
@@ -581,6 +581,10 @@ extension SettingsView {
field: "mic_enabled") {
Toggle("Send microphone to the host", isOn: scoped(SettingsFields.micEnabled))
}
described(echoCancelCaption, field: "echo_cancel") {
Toggle("Echo cancellation", isOn: scoped(SettingsFields.echoCancel))
.disabled(!effective.micEnabled)
}
#if os(macOS)
if !inProfileScope {
Picker("Microphone", selection: $micUID) {
@@ -619,10 +623,33 @@ extension SettingsView {
}
}
/// Honest about the macOS escape hatch: the voice processor only follows the system
/// default devices, so hand-picked endpoints silently keep the raw path (see
/// SessionAudio's topology note) better said here than discovered mid-call.
private var echoCancelCaption: String {
let base = "Voice processing cancels the audio this device plays out of the mic "
+ "signal, so a speaker setup doesn't feed the game back to the host."
#if os(macOS)
return base + " Follows the system default devices — a hand-picked speaker, "
+ "microphone or input channel streams the raw mic instead."
#else
return base
#endif
}
// MARK: - Controllers
@ViewBuilder var controllersSection: some View {
Section {
// The master switch, above everything it governs. Profileable, so it renders in
// both scopes: a "Work" profile can decline to forward what "Game" forwards.
described("Sends controllers connected to this device to the host. Turn it off when "
+ "your controller already reaches the host another way — USB passthrough such "
+ "as VirtualHere, or a pad plugged into the host itself — so games don't see "
+ "two of them.",
field: "gamepad_forwarding") {
Toggle("Forward controllers", isOn: scoped(SettingsFields.gamepadForwarding))
}
// Which physical pad this device forwards, and what its own haptics do, are facts
// about THIS device (tier G) only the virtual pad the host creates is profileable.
if !inProfileScope {
@@ -641,6 +668,7 @@ extension SettingsView {
Text(option.label).tag(option.tag)
}
}
.disabled(!effective.gamepadForwarding)
}
}
described("The virtual pad created on the host. Automatic matches your controller "
@@ -651,6 +679,7 @@ extension SettingsView {
Text(option.label).tag(option.tag)
}
}
.disabled(!effective.gamepadForwarding)
}
#if os(iOS)
// iPhone only in practice: hidden where the device itself can't play haptics (iPad).
@@ -49,6 +49,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.renderScale) var renderScale = 1.0
@AppStorage(DefaultsKey.compositor) var compositor = 0
@AppStorage(DefaultsKey.gamepadType) var gamepadType = 0
@AppStorage(DefaultsKey.gamepadForwarding) var gamepadForwarding = true
@AppStorage(DefaultsKey.bitrateKbps) var bitrateKbps = 0
@AppStorage(DefaultsKey.presentPriority) var presentPriority =
SettingsOptions.presentPriorityDefault
@@ -65,6 +66,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.libraryEnabled) var libraryEnabled = true
@AppStorage(DefaultsKey.fullscreenWhileStreaming) var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) var micEnabled = true
@AppStorage(DefaultsKey.echoCancel) var echoCancel = true
@AppStorage(DefaultsKey.audioChannels) var audioChannels = 2
@AppStorage(DefaultsKey.codec) var codec = "auto"
// The overlay tier's raw string (the pickers tag by rawValue); the absent-key default runs
@@ -1,7 +1,9 @@
// Opus PCM through CoreAudio's built-in codec (kAudioFormatOpus, macOS 10.13+ / iOS
// 11+) no bundled libopus. The host's audio plane is raw Opus packets (48 kHz stereo,
// one frame per packet); AVAudioConverter handles them as single-packet
// AVAudioCompressedBuffers with explicit packet descriptions.
// one frame per packet); the mic uplink is 48 kHz MONO packets (one microphone bus
// the host's decoder upmixes, so duplicating it into a second channel only cost bits).
// AVAudioConverter handles both as single-packet AVAudioCompressedBuffers with explicit
// packet descriptions.
//
// Both classes are single-threaded by contract (one per direction, owned by their
// drain/capture pipelines).
@@ -14,16 +16,16 @@ enum OpusCodecError: Error {
case convertFailed(String)
}
/// 48 kHz stereo float32 interleaved the PCM side of both converters and the layout
/// of the playback ring buffer.
/// 48 kHz stereo float32 interleaved the decoder's PCM side (the host plane's shape).
func opusPCMFormat() -> AVAudioFormat? {
AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: 48_000, channels: 2, interleaved: true)
}
/// The compressed side: raw Opus, `framesPerPacket` nominal samples per packet at 48 kHz
/// (240 = the host's 5 ms audio plane; 960 = the 20 ms packets the encoder emits).
private func opusFormat(framesPerPacket: UInt32) -> AVAudioFormat? {
/// (240 = the host's 5 ms audio plane; 480 = the 10 ms packets the encoder emits) and
/// `channels` (2 = the host plane, 1 = the mic uplink).
private func opusFormat(framesPerPacket: UInt32, channels: UInt32) -> AVAudioFormat? {
var desc = AudioStreamBasicDescription(
mSampleRate: 48_000,
mFormatID: kAudioFormatOpus,
@@ -31,7 +33,7 @@ private func opusFormat(framesPerPacket: UInt32) -> AVAudioFormat? {
mBytesPerPacket: 0,
mFramesPerPacket: framesPerPacket,
mBytesPerFrame: 0,
mChannelsPerFrame: 2,
mChannelsPerFrame: channels,
mBitsPerChannel: 0,
mReserved: 0)
return AVAudioFormat(streamDescription: &desc)
@@ -45,7 +47,8 @@ final class OpusDecoder {
/// `framesPerPacket`: the sender's packet duration in samples (host audio = 240).
init(framesPerPacket: UInt32) throws {
guard let pcm = opusPCMFormat(), let opus = opusFormat(framesPerPacket: framesPerPacket),
guard let pcm = opusPCMFormat(),
let opus = opusFormat(framesPerPacket: framesPerPacket, channels: 2),
let converter = AVAudioConverter(from: opus, to: pcm)
else { throw OpusCodecError.unavailable }
self.converter = converter
@@ -90,24 +93,43 @@ final class OpusDecoder {
}
final class OpusEncoder {
/// The encoder's packet duration: 960 samples = 20 ms, CoreAudio's default Opus
/// framing. The host's mic service decodes any Opus frame size up to 120 ms.
static let framesPerPacket: AVAudioFrameCount = 960
/// The encoder's packet duration in samples: 480 = 10 ms, halving the packetization
/// latency of the old 20 ms framing. CoreAudio honors it mFramesPerPacket 480/mono
/// creates a converter that truly emits 10 ms CELT packets (TOC config 30), one per
/// 480-frame chunk, verified by inspection of the emitted TOC bytes and per-packet
/// frame accounting. 960 stays as the fallback should an older codec refuse 480.
/// The host's mic service decodes any Opus frame size up to 120 ms, so either is
/// wire-compatible.
let framesPerPacket: AVAudioFrameCount
/// 48 kHz MONO float32 interleaved the uplink carries one microphone bus.
let pcmFormat: AVAudioFormat
private let converter: AVAudioConverter
private let outBuf: AVAudioCompressedBuffer
let pcmFormat: AVAudioFormat
init() throws {
guard let pcm = opusPCMFormat(),
let opus = opusFormat(framesPerPacket: UInt32(Self.framesPerPacket)),
let converter = AVAudioConverter(from: pcm, to: opus)
guard let pcm = AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: 48_000, channels: 1,
interleaved: true)
else { throw OpusCodecError.unavailable }
converter.bitRate = 96_000
self.converter = converter
self.pcmFormat = pcm
var made: (converter: AVAudioConverter, fpp: AVAudioFrameCount)?
for fpp: AVAudioFrameCount in [480, 960] {
if let opus = opusFormat(framesPerPacket: UInt32(fpp), channels: 1),
let converter = AVAudioConverter(from: pcm, to: opus) {
made = (converter, fpp)
break
}
}
guard let made else { throw OpusCodecError.unavailable }
// 48 kbps: transparent for mono voice the old 96 kbps budget was sized for
// the duplicated-stereo framing this encoder no longer emits.
made.converter.bitRate = 48_000
converter = made.converter
framesPerPacket = made.fpp
pcmFormat = pcm
outBuf = AVAudioCompressedBuffer(
format: opus, packetCapacity: 4, maximumPacketSize: 1500)
format: made.converter.outputFormat, packetCapacity: 4, maximumPacketSize: 1500)
}
/// Encode exactly `framesPerPacket` frames of `pcmFormat` audio; returns the encoded
@@ -5,15 +5,22 @@
// AVAudioSourceNode pulls from the ring (silence on underrun with re-priming, so a
// network gap costs one dip, not permanent crackle).
//
// mic host: a second AVAudioEngine taps the input device, folds it to one mono bus (the
// chosen channel of a multi-channel interface, or a sum of all channels), resamples to 48 kHz
// stereo, slices 20 ms chunks, Opus-encodes, and sendMic()s each packet the host feeds them
// into a virtual PipeWire source.
// mic host: a tap on the input node folds the capture to one mono bus (the chosen channel
// of a multi-channel interface, or a sum of all channels), resamples to 48 kHz mono, slices
// 10 ms chunks, Opus-encodes, and sendMic()s each packet the host feeds them into a
// virtual PipeWire source.
//
// Engine topology. With the mic enabled and echo cancellation on (both defaults), BOTH
// directions run on ONE AVAudioEngine with the system voice processor engaged
// (`setVoiceProcessingEnabled`) AEC needs render and capture on the same unit so it can
// subtract what the speaker is playing from what the mic hears; without it, a loudspeaker
// client feeds the host's own game audio straight back to it (the primary reported echo
// source). The voice processor can only follow the system DEFAULT devices, so explicit
// endpoint choices fall back to the old two-engine topology see `wantsCombined` for the
// exact decision, and `startCapture` for why two engines handle arbitrary device pairs.
//
// Devices are chosen by UID ("" = system default: the engine is then never pinned to a
// concrete device and follows default-device changes). Two engines, not one a single
// AVAudioEngine ties input+output to one aggregate clock, separate engines keep
// arbitrary mic/speaker combinations trivial.
// concrete device and follows default-device changes).
import AVFoundation
import os
@@ -40,7 +47,21 @@ public final class SessionAudio {
private let stateLock = NSLock()
private var playbackEngine: AVAudioEngine?
private var captureEngine: AVAudioEngine?
/// The one engine running BOTH directions when the voice processor is engaged;
/// `playbackEngine`/`captureEngine` stay nil while this is set.
private var combinedEngine: AVAudioEngine?
private var drainStarted = false
/// The mute LATCH: the effective mute the owner last asked for (see `setMicMuted`). Held
/// because the uplink engine can appear LATER than the request the mic permission prompt
/// is answered at the user's leisure, and the engine is built on the grant so a mute set
/// in the meantime must be waiting for it. Applied by whichever start path wins the race.
/// Guarded by `stateLock`, like the engines it applies to.
private var micMuted = false
/// The playback jitter ring created by whichever engine starts playback first and KEPT
/// across an engine rebuild (the permission-grant upgrade in `startEngines` swaps engines,
/// not the ring, so the drain thread never has to be re-pointed). Main-thread confined,
/// like every start path.
private var ring: AudioRing?
#if !os(macOS)
/// AVAudioSession `setCategory`/`setActive` are synchronous and block on the audio server, so
/// they must not run on the main thread (UI stall AVFoundation warns about it). PROCESS-WIDE
@@ -69,11 +90,15 @@ public final class SessionAudio {
/// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending.
public func start(speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool) {
/// `echoCancel` picks the engine topology see the header note and `wantsCombined`.
public func start(
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool, echoCancel: Bool
) {
#if os(macOS)
// No AVAudioSession on macOS start the engines directly (caller's thread, as before).
startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel, micEnabled: micEnabled)
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
micEnabled: micEnabled, echoCancel: echoCancel)
#else
// Configure + activate the session OFF the main thread (it blocks on the audio server),
// then start the engines back on the main thread once it's active engine routing/format
@@ -85,7 +110,7 @@ public final class SessionAudio {
guard let self, !self.flag.isStopped else { return }
self.startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
micEnabled: micEnabled)
micEnabled: micEnabled, echoCancel: echoCancel)
}
}
#endif
@@ -104,6 +129,12 @@ public final class SessionAudio {
try session.setCategory(
.playAndRecord, mode: .default,
options: [.allowBluetoothA2DP, .defaultToSpeaker])
// Uplink latency: ask for 5 ms IO quanta at the wire rate (the default ~10-23 ms
// quantum is most of the mic path's burst latency). Best-effort the hardware
// has the final word (a Bluetooth route will ignore both), and whatever quantum
// is actually granted, the capture tap handles the buffers it gets.
try? session.setPreferredIOBufferDuration(0.005)
try? session.setPreferredSampleRate(48_000)
} else {
try session.setCategory(.playback, mode: .default)
}
@@ -117,32 +148,80 @@ public final class SessionAudio {
}
#endif
/// Build + start the playback engine (and the mic uplink when enabled + authorized). Main
/// thread (engine setup); on iOS/tvOS the session is already active by the time this runs.
/// Build + start the engines combined (voice-processed) or split, per `wantsCombined`
/// with the mic uplink only when enabled + authorized. Main thread (engine setup); on
/// iOS/tvOS the session is already active by the time this runs.
private func startEngines(
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool, echoCancel: Bool
) {
startPlayback(speakerUID: speakerUID)
#if os(tvOS)
// No app-accessible microphone input on tvOS playback only.
startPlayback(speakerUID: speakerUID)
#else
guard micEnabled else { return }
guard micEnabled else {
startPlayback(speakerUID: speakerUID)
return
}
let combined = wantsCombined(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
echoCancel: echoCancel)
switch AVCaptureDevice.authorizationStatus(for: .audio) {
case .authorized:
startCapture(micUID: micUID, micChannel: micChannel)
if combined {
startCombined(speakerUID: speakerUID, micUID: micUID, micChannel: micChannel)
} else {
startPlayback(speakerUID: speakerUID)
startCapture(micUID: micUID, micChannel: micChannel)
}
case .notDetermined:
// Playback must not wait out the permission prompt (the user answers at their
// leisure) start it now, and on a grant either bolt the capture engine on
// (split) or swap the playback engine for the combined one (the ring and its
// drain thread carry over see `makePlaybackChain`).
startPlayback(speakerUID: speakerUID)
AVCaptureDevice.requestAccess(for: .audio) { [weak self] granted in
DispatchQueue.main.async {
guard let self, granted, !self.flag.isStopped else { return }
self.startCapture(micUID: micUID, micChannel: micChannel)
if combined {
self.stateLock.lock()
let playback = self.playbackEngine
self.playbackEngine = nil
self.stateLock.unlock()
playback?.stop()
self.startCombined(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel)
} else {
self.startCapture(micUID: micUID, micChannel: micChannel)
}
}
}
default:
startPlayback(speakerUID: speakerUID)
log.warning("microphone access denied — mic uplink disabled (System Settings → Privacy)")
}
#endif
}
#if !os(tvOS)
/// One engine or two: the voice processor requires render + capture on one unit, and that
/// unit can only follow the system DEFAULT devices so echo cancellation gets the combined
/// engine only while nothing is explicitly pinned. On macOS a chosen speaker/mic UID or a
/// picked input channel (the voice processor's capture side is its own mono mix a
/// per-channel pick can't survive it) keeps today's two-engine path, AEC-less but honoring
/// the exact endpoints the user named. On iOS routes are session-managed and the UIDs are
/// ignored, so the toggle alone decides.
private func wantsCombined(
speakerUID: String, micUID: String, micChannel: Int, echoCancel: Bool
) -> Bool {
guard echoCancel else { return false }
#if os(macOS)
return speakerUID.isEmpty && micUID.isEmpty && micChannel == 0
#else
return true
#endif
}
#endif
/// Stop both directions. Safe from any thread; waits the drain thread out ( its
/// poll timeout) so the caller can close the connection right after.
public func stop() {
@@ -152,6 +231,8 @@ public final class SessionAudio {
captureEngine = nil
let playback = playbackEngine
playbackEngine = nil
let combined = combinedEngine
combinedEngine = nil
let wasDraining = drainStarted
drainStarted = false
stateLock.unlock()
@@ -160,6 +241,10 @@ public final class SessionAudio {
capture.stop()
}
playback?.stop()
if let combined {
combined.inputNode.removeTap(onBus: 0)
combined.stop()
}
#if !os(macOS)
// Release the session so audio we interrupted (Music, podcasts) gets its resume cue. Like
// activation, setActive is synchronous/blocking run it on the shared serial session queue
@@ -180,15 +265,38 @@ public final class SessionAudio {
}
}
/// Background keep-alive: silence the mic uplink while backgrounded (privacy no room audio
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
/// this stops the actual capture, which is the privacy-relevant part. Main thread.
/// Silence the mic uplink (no room audio leaves the device) or restore it. THE one muting
/// mechanism: the owner composes its reasons the user's in-stream mute and the background
/// keep-alive's privacy mute into one effective state and passes that here, so neither can
/// clear the other (see `SessionModel.applyMicMute`).
///
/// Two-engine sessions pause/resume the capture engine; a combined session instead mutes the
/// voice processor's input (playback shares that engine and must keep running, so the engine
/// itself never pauses the mute zeroes the mic at the IO unit, and the tap encodes silence).
/// Local and instant either way: nothing is negotiated with the host, and the packets that do
/// leave carry silence. A no-op when there's no uplink (playback-only / tvOS / mic disabled),
/// except that the state is LATCHED for an uplink that starts later. The audio SESSION stays
/// active for background playback, so iOS may keep showing the recording indicator until a
/// full reconfigure either path stops room audio leaving the device, which is the
/// privacy-relevant part. Main thread.
public func setMicMuted(_ muted: Bool) {
stateLock.lock()
micMuted = muted
let capture = captureEngine
let combined = combinedEngine
stateLock.unlock()
apply(micMuted: muted, capture: capture, combined: combined)
}
/// Push the latched mute onto whichever engine carries the uplink. Split out from
/// `setMicMuted` because the start paths call it too, with the engine they just started
/// that's how a mute requested before the permission grant lands on the engine the grant
/// creates. Never resumes a stopped session's engine.
private func apply(micMuted muted: Bool, capture: AVAudioEngine?, combined: AVAudioEngine?) {
if let combined {
combined.inputNode.isVoiceProcessingInputMuted = muted
return
}
guard let capture else { return }
if muted {
capture.pause()
@@ -199,28 +307,21 @@ public final class SessionAudio {
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
/// The playback jitter ring + the source node draining it shared by the plain playback
/// engine and the combined voice-processing engine, and REUSED across an engine rebuild
/// (same session, same ring: the drain thread keeps writing right through the swap). nil
/// when the host's channel layout can't be expressed (already logged). Main thread.
private func makePlaybackChain()
-> (ring: AudioRing, source: AVAudioSourceNode, format: AVAudioFormat)?
{
// Build the playback layout from the host-RESOLVED channel count (never the request):
// 2 = stereo / 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = Int(connection.resolvedAudioChannels)
// 1 s interleaved capacity, ~20 ms prefill (four 5 ms host packets of jitter absorption
// before the first sample plays), both scaled by the channel count.
let ring = AudioRing(
let ring = self.ring ?? AudioRing(
capacity: 48_000 * channels, prefill: 960 * channels, channels: channels)
let engine = AVAudioEngine()
#if os(macOS)
if !speakerUID.isEmpty {
if let dev = AudioDevices.deviceID(forUID: speakerUID),
let unit = engine.outputNode.audioUnit {
if !Self.setDevice(dev, on: unit) {
log.error("could not select speaker \(speakerUID) — using default")
}
} else {
log.warning("speaker \(speakerUID) not present — using default")
}
}
#endif
self.ring = ring
// Engine-native deinterleaved float; the render block deinterleaves from the ring. Surround
// uses an explicit wire-order channel layout; the mixer downmixes to the output device when
@@ -234,7 +335,7 @@ public final class SessionAudio {
}
guard let format else {
log.error("could not build \(channels)-channel audio format — audio disabled")
return
return nil
}
let scratch = ScratchBuffer() // block-owned; freed with the closure
let source = AVAudioSourceNode(format: format) { _, _, frameCount, abl -> OSStatus in
@@ -252,6 +353,24 @@ public final class SessionAudio {
}
return noErr
}
return (ring, source, format)
}
private func startPlayback(speakerUID: String) {
guard let (ring, source, format) = makePlaybackChain() else { return }
let engine = AVAudioEngine()
#if os(macOS)
if !speakerUID.isEmpty {
if let dev = AudioDevices.deviceID(forUID: speakerUID),
let unit = engine.outputNode.audioUnit {
if !Self.setDevice(dev, on: unit) {
log.error("could not select speaker \(speakerUID) — using default")
}
} else {
log.warning("speaker \(speakerUID) not present — using default")
}
}
#endif
engine.attach(source)
engine.connect(source, to: engine.mainMixerNode, format: format)
engine.prepare()
@@ -272,8 +391,14 @@ public final class SessionAudio {
startDrain(into: ring)
}
/// Idempotent the permission-grant engine swap reaches here a second time with the
/// drain thread already feeding the (carried-over) ring.
private func startDrain(into ring: AudioRing) {
stateLock.lock()
if drainStarted {
stateLock.unlock()
return
}
drainStarted = true
stateLock.unlock()
let thread = Thread { [connection, flag, drainDone] in
@@ -308,6 +433,80 @@ public final class SessionAudio {
// MARK: - Mic (mic host)
#if !os(tvOS)
/// One engine, both directions: engage the system voice processor on the shared IO unit
/// (AEC + noise suppression + AGC), hang the playback source off its render side and the
/// mic tap off its capture side. Every failure falls back to a WORKING configuration
/// the split path (no AEC) when the voice processor won't engage, plain playback when the
/// mic chain can't be built a session never loses audio to the echo-cancel feature.
private func startCombined(speakerUID: String, micUID: String, micChannel: Int) {
let engine = AVAudioEngine()
let input = engine.inputNode
do {
// Before anything reads the input's format: the voice processor changes it (often
// to its own mono mix, sometimes at a lower rate) installMicTap reads the format
// AFTER this, so the converter chain adapts to whatever the processor emits.
try input.setVoiceProcessingEnabled(true)
} catch {
log.warning("""
voice processing unavailable (\(error.localizedDescription)) separate \
engines, no echo cancellation
""")
startPlayback(speakerUID: speakerUID)
startCapture(micUID: micUID, micChannel: micChannel)
return
}
// Symmetric enable for the render side; with both directions on one engine the
// input-node enable already covers it, so a refusal here is not a failure.
try? engine.outputNode.setVoiceProcessingEnabled(true)
// This is a game stream, not a call: never duck the host's audio under the outgoing
// voice. .min is the closest to "off" the API offers, and advanced (selective)
// ducking stays off with it.
input.voiceProcessingOtherAudioDuckingConfiguration = .init(
enableAdvancedDucking: false, duckingLevel: .min)
guard let (ring, source, format) = makePlaybackChain() else {
// Playback impossible (logged) keep the uplink alive, as the split path would.
startCapture(micUID: micUID, micChannel: micChannel)
return
}
engine.attach(source)
engine.connect(source, to: engine.mainMixerNode, format: format)
guard installMicTap(on: input, micUID: micUID, micChannel: micChannel) else {
// Mic chain unavailable (logged) keep the session audible on the plain playback
// engine rather than playing through an idle voice processor.
startPlayback(speakerUID: speakerUID)
return
}
engine.prepare()
do {
try engine.start()
} catch {
log.error("combined engine failed to start: \(error.localizedDescription)")
input.removeTap(onBus: 0)
startPlayback(speakerUID: speakerUID) // no echo cancellation beats no audio
return
}
stateLock.lock()
if flag.isStopped {
stateLock.unlock()
input.removeTap(onBus: 0)
engine.stop() // stop() already ran don't strand a started engine (or a hot mic)
return
}
combinedEngine = engine
let muted = micMuted // latched before this engine existed (a mute during the prompt)
stateLock.unlock()
apply(micMuted: muted, capture: nil, combined: engine)
startDrain(into: ring)
log.info("audio engines joined — voice processing (echo cancellation) active")
}
/// The split path: capture on its OWN engine, playback on another the pre-echo-cancel
/// topology, kept verbatim. Two engines, not one a single AVAudioEngine ties
/// input+output to one aggregate clock, separate engines keep arbitrary mic/speaker
/// combinations trivial. That freedom is exactly why the voice processor can't ride this
/// path (AEC needs both directions on one unit) and why explicitly pinned endpoints land
/// here see `wantsCombined`.
private func startCapture(micUID: String, micChannel: Int) {
let engine = AVAudioEngine()
let input = engine.inputNode
@@ -322,11 +521,44 @@ public final class SessionAudio {
}
}
#endif
guard installMicTap(on: input, micUID: micUID, micChannel: micChannel) else { return }
engine.prepare()
do {
try engine.start()
} catch {
log.error("capture engine failed to start: \(error.localizedDescription)")
input.removeTap(onBus: 0)
return
}
stateLock.lock()
if flag.isStopped {
// stop() ran while we were starting (the permission prompt resolves at the
// user's leisure) tear the engine down ourselves, nobody else owns it now.
stateLock.unlock()
input.removeTap(onBus: 0)
engine.stop()
return
}
captureEngine = engine
let muted = micMuted // latched before this engine existed (a mute during the prompt)
stateLock.unlock()
apply(micMuted: muted, capture: engine, combined: nil)
log.info("mic uplink started (\(micUID.isEmpty ? "default input" : micUID))")
}
/// Resolve the input's live format + fold plan, build the monoOpus chain, and install the
/// capture tap on `input` everything mic except engine ownership, shared verbatim by the
/// combined and split topologies. Reads `input.outputFormat(forBus:)` at call time, so the
/// chain follows whatever the node emits: the raw device format, or the voice processor's
/// own mix when that's enabled. False (logged) when no input is usable or the encoder
/// can't be built; the tap is installed on true.
private func installMicTap(
on input: AVAudioInputNode, micUID: String, micChannel: Int
) -> Bool {
let inFormat = input.outputFormat(forBus: 0)
guard inFormat.sampleRate > 0, inFormat.channelCount > 0 else {
log.error("no usable input device — mic uplink disabled")
return
return false
}
// Multi-channel-interface handling. A pro interface exposes N discrete inputs with the mic
@@ -378,22 +610,38 @@ public final class SessionAudio {
#endif
// Encode a single mono bus (folded from `inFormat` in the tap): the resampler goes
// mono@inputSR the encoder's 48 kHz stereo, so it handles both the rate change and the
// monostereo duplication, and the wrong-channel downmix never happens.
// mono@inputSR the encoder's 48 kHz mono, so it handles the rate change and the
// wrong-channel downmix never happens. Mono end to end the host's decoder upmixes,
// so the old duplicate-into-stereo step only cost bits and cycles.
//
// `mono`/`staging` are the per-callback scratch buffers, preallocated HERE (grown only
// if a larger-than-expected device quantum ever arrives) the steady-state tap path
// allocates nothing.
let scratchFrames: AVAudioFrameCount = 8192
let stagingCapacity = { (frames: AVAudioFrameCount) -> AVAudioFrameCount in
AVAudioFrameCount(
(Double(frames) * 48_000 / inFormat.sampleRate).rounded(.up)) + 64
}
guard let monoFormat = AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: inFormat.sampleRate,
channels: 1, interleaved: false),
let encoder = try? OpusEncoder(),
let resampler = AVAudioConverter(from: monoFormat, to: encoder.pcmFormat),
let chunk = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: OpusEncoder.framesPerPacket)
pcmFormat: encoder.pcmFormat, frameCapacity: encoder.framesPerPacket),
let monoScratch = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: scratchFrames),
let stagingScratch = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: stagingCapacity(scratchFrames))
else {
log.error("Opus encoder unavailable — mic uplink disabled")
return
return false
}
// Tap-thread-confined state: resample into `staging`, accumulate in `fifo`,
// slice 960-frame chunks for the encoder.
// Tap-thread-confined state: fold into `mono`, resample into `staging`, accumulate in
// `fifo`, slice `framesPerPacket` (10 ms) chunks for the encoder.
var mono = monoScratch
var staging = stagingScratch
var fifo: [Float] = []
fifo.reserveCapacity(48_000)
var seq: UInt32 = 0
@@ -412,14 +660,26 @@ public final class SessionAudio {
var inputPeak: Float = 0
var levelReported = false
input.installTap(onBus: 0, bufferSize: 2048, format: inFormat) { buffer, _ in
// 480 frames = 10 ms, matching the packet duration. Advisory CoreAudio delivers the
// device quantum whatever we ask (the old 2048 request came back as 42.7 ms bursts, most
// of the uplink's latency) but where the system honors it, the tap fires per-packet.
input.installTap(onBus: 0, bufferSize: 480, format: inFormat) { buffer, _ in
if flag.isStopped { return }
let frames = Int(buffer.frameLength)
guard frames > 0, let src = buffer.floatChannelData,
let mono = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: buffer.frameLength),
let dst = mono.floatChannelData?[0]
else { return }
guard frames > 0, let src = buffer.floatChannelData else { return }
if frames > Int(mono.frameCapacity) {
// A quantum larger than the scratch (bufferSize is advisory both ways) regrow
// once to the new high-water mark; the steady state stays allocation-free.
guard let biggerMono = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: buffer.frameLength),
let biggerStaging = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat,
frameCapacity: stagingCapacity(buffer.frameLength))
else { return }
mono = biggerMono
staging = biggerStaging
}
guard let dst = mono.floatChannelData?[0] else { return }
mono.frameLength = buffer.frameLength
// Fold the multi-channel input down to the one mono bus we encode.
@@ -451,11 +711,6 @@ public final class SessionAudio {
}
}
let ratio = 48_000 / inFormat.sampleRate
let outCapacity = AVAudioFrameCount((Double(frames) * ratio).rounded(.up) + 64)
guard let staging = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: outCapacity)
else { return }
var fed = false
var convError: NSError?
let status = resampler.convert(to: staging, error: &convError) { _, outStatus in
@@ -469,16 +724,20 @@ public final class SessionAudio {
}
guard status != .error, let p = staging.floatChannelData?[0] else { return }
fifo.append(contentsOf: UnsafeBufferPointer(
start: p, count: Int(staging.frameLength) * 2))
start: p, count: Int(staging.frameLength)))
let samplesPerChunk = Int(OpusEncoder.framesPerPacket) * 2
while fifo.count >= samplesPerChunk {
chunk.frameLength = OpusEncoder.framesPerPacket
// Consume whole chunks through a head index, then drop the eaten prefix in ONE
// move of the sub-chunk remainder. The old per-chunk removeFirst memmoved the
// entire backlog for every packet O(n) on the render-adjacent tap thread.
let samplesPerChunk = Int(encoder.framesPerPacket)
var head = 0
while fifo.count - head >= samplesPerChunk {
chunk.frameLength = encoder.framesPerPacket
fifo.withUnsafeBufferPointer { src in
chunk.floatChannelData![0].update(
from: src.baseAddress!, count: samplesPerChunk)
from: src.baseAddress! + head, count: samplesPerChunk)
}
fifo.removeFirst(samplesPerChunk)
head += samplesPerChunk
guard let packets = try? encoder.encode(chunk) else { continue }
for packet in packets {
connection.sendMic(
@@ -486,28 +745,9 @@ public final class SessionAudio {
seq &+= 1
}
}
if head > 0 { fifo.removeFirst(head) } // keeps capacity no realloc
}
engine.prepare()
do {
try engine.start()
} catch {
log.error("capture engine failed to start: \(error.localizedDescription)")
input.removeTap(onBus: 0)
return
}
stateLock.lock()
if flag.isStopped {
// stop() ran while we were starting (the permission prompt resolves at the
// user's leisure) tear the engine down ourselves, nobody else owns it now.
stateLock.unlock()
input.removeTap(onBus: 0)
engine.stop()
return
}
captureEngine = engine
stateLock.unlock()
log.info("mic uplink started (\(micUID.isEmpty ? "default input" : micUID))")
return true
}
/// Fold `channels` of input (`floatChannelData` layout: `interleaved` one buffer strided by
@@ -98,9 +98,27 @@ public final class GamepadCapture {
/// gameplay can't end it (see ContentView's tvOS session branch).
public var onDisconnectRequest: (() -> Void)?
public init(connection: PunktfunkConnection, manager: GamepadManager) {
/// Forward this device's controllers to the host at all (`Settings.gamepadForwarding`,
/// default true). Off is for a couch whose controller reaches the host another way USB
/// passthrough such as VirtualHere, or a pad plugged into the host itself where
/// forwarding as well would give the host two pads for one pair of hands.
///
/// Off still opens slots and tracks button state; it just sends nothing (see `wire`). That
/// is deliberate, not laziness: the escape chord is read off the same slots, and on tvOS it
/// is the ONLY controller way out of a stream a session that silently lost its exit
/// because a forwarding preference was off would be a worse bug than the one this fixes.
/// Unlike pf-client-core's slots, GameController claims nothing exclusive, so holding one
/// open costs the host nothing and blocks no passthrough tool.
public let forwarding: Bool
/// The connection, or nil while forwarding is off every wire send goes through this, so
/// "don't forward" is one fact in one place rather than a condition at twelve call sites.
private var wire: PunktfunkConnection? { forwarding ? connection : nil }
public init(connection: PunktfunkConnection, manager: GamepadManager, forwarding: Bool = true) {
self.connection = connection
self.manager = manager
self.forwarding = forwarding
}
public func start() {
@@ -205,8 +223,8 @@ public final class GamepadCapture {
// core re-sends it a few times against datagram loss; an older host ignores it and uses
// the session-default kind. Then wake the host pad (pads are created lazily from the first
// event; a DualSense's UHID handshake + initial lightbar write only start then).
connection.send(.gamepadArrival(pref: slot.pref.rawValue, pad: slot.pad))
connection.send(.gamepadAxis(GamepadWire.axisLSX, value: 0, pad: slot.pad))
wire?.send(.gamepadArrival(pref: slot.pref.rawValue, pad: slot.pad))
wire?.send(.gamepadAxis(GamepadWire.axisLSX, value: 0, pad: slot.pad))
sync(slot, ext)
if let tp = Self.touchpad(ext) {
@@ -233,7 +251,7 @@ public final class GamepadCapture {
flush(slot)
// Sent after the flush so the core stamps it with a seq past the zeroing snapshots; the host
// seq-gates it, so a reordered snapshot can't resurrect the removed pad.
connection.send(.gamepadRemove(pad: slot.pad))
wire?.send(.gamepadRemove(pad: slot.pad))
let c = slot.controller
if let ext = c.extendedGamepad {
ext.valueChangedHandler = nil
@@ -275,7 +293,7 @@ public final class GamepadCapture {
let changed = newButtons ^ slot.buttons
if changed != 0 {
for bit in GamepadWire.allButtons where changed & bit != 0 {
connection.send(.gamepadButton(bit, down: newButtons & bit != 0, pad: slot.pad))
wire?.send(.gamepadButton(bit, down: newButtons & bit != 0, pad: slot.pad))
}
slot.buttons = newButtons
}
@@ -288,7 +306,7 @@ public final class GamepadCapture {
Int32(g.rightTrigger.value * 255),
]
for (i, v) in newAxes.enumerated() where v != slot.axes[i] {
connection.send(.gamepadAxis(UInt32(i), value: v, pad: slot.pad))
wire?.send(.gamepadAxis(UInt32(i), value: v, pad: slot.pad))
slot.axes[i] = v
}
updateEscapeChord()
@@ -302,7 +320,7 @@ public final class GamepadCapture {
let bit = GamepadWire.guide
let now = down ? (slot.buttons | bit) : (slot.buttons & ~bit)
guard now != slot.buttons else { return }
connection.send(.gamepadButton(bit, down: down, pad: slot.pad))
wire?.send(.gamepadButton(bit, down: down, pad: slot.pad))
slot.buttons = now
}
@@ -365,13 +383,13 @@ public final class GamepadCapture {
if lifted {
if slot.fingerActive[finger] {
slot.fingerActive[finger] = false
connection.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(finger), active: false, x: 0, y: 0)
wire?.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(finger), active: false, x: 0, y: 0)
}
return
}
slot.fingerActive[finger] = true
let w = GamepadWire.touchpad(x: x, y: y)
connection.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(finger), active: true, x: w.x, y: w.y)
wire?.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(finger), active: true, x: w.x, y: w.y)
}
private func forwardMotion(_ slot: Slot, _ m: GCMotion) {
@@ -394,7 +412,7 @@ public final class GamepadCapture {
}
let gs = GamepadWire.gyroLSBPerRadS
let as_ = GamepadWire.accelLSBPerG
connection.sendMotion(
wire?.sendMotion(
pad: UInt8(slot.pad),
gyro: (
GamepadWire.motionRaw(Float(m.rotationRate.x), scale: gs),
@@ -432,15 +450,15 @@ public final class GamepadCapture {
/// GamepadRemove (that's `closeSlot`).
private func flush(_ slot: Slot) {
for bit in GamepadWire.allButtons where slot.buttons & bit != 0 {
connection.send(.gamepadButton(bit, down: false, pad: slot.pad))
wire?.send(.gamepadButton(bit, down: false, pad: slot.pad))
}
slot.buttons = 0
for (i, v) in slot.axes.enumerated() where v != 0 {
connection.send(.gamepadAxis(UInt32(i), value: 0, pad: slot.pad))
wire?.send(.gamepadAxis(UInt32(i), value: 0, pad: slot.pad))
slot.axes[i] = 0
}
for (f, active) in slot.fingerActive.enumerated() where active {
connection.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(f), active: false, x: 0, y: 0)
wire?.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(f), active: false, x: 0, y: 0)
slot.fingerActive[f] = false
}
}
@@ -128,11 +128,19 @@ public final class InputCapture {
/// carries the same key equivalents for discoverability) can't see them, so the monitor is the
/// captured-state delivery path; released, the events pass through and the menu handles them.
/// Q releases the captured mouse/keyboard; D disconnects; S cycles the stats
/// overlay tier (off compact normal detailed). Main queue.
/// overlay tier (off compact normal detailed). A (`onToggleMicMute`, below) rides
/// the same path. Main queue.
public var onReleaseCapture: (() -> Void)?
public var onDisconnect: (() -> Void)?
public var onCycleStats: (() -> Void)?
/// Fired on A mute/unmute the microphone uplink, the one in-stream control a captured
/// session can't otherwise reach (the HUD's button is behind a grabbed cursor). Same delivery
/// rule as the combos above: only WHILE FORWARDING, because that's when the menu's identical
/// key equivalent can't fire. M the obvious letter is long since the mouse-model flip
/// (cross-client), so A ("audio in") is the mic's. Main queue.
public var onToggleMicMute: (() -> Void)?
/// Fired on F (macOS) toggle the streaming window in/out of fullscreen. Detected in the
/// monitor only WHILE FORWARDING, for the same reason as the combos: a captured stream view
/// swallows keys, so the Stream menu's identical F equivalent never reaches it; released, the
@@ -140,13 +148,13 @@ public final class InputCapture {
public var onToggleFullscreen: (() -> Void)?
#if os(iOS)
/// Windows VKs of the three modifier classes in the Q release chord, both L/R sides:
/// Windows VKs of the three modifier classes in the chords, both L/R sides:
/// control (0xA2/0xA3), option (0xA4/0xA5), shift (0xA0/0xA1). Used to sift the HID key stream.
private static let chordModifierVKs: Set<UInt32> = [0xA2, 0xA3, 0xA4, 0xA5, 0xA0, 0xA1]
/// Whether Control AND Option AND Shift are all currently held (either side of each counts)
/// the modifier precondition for the iPad Q release chord.
private var hasReleaseChordModifiers: Bool {
/// the modifier precondition for the iPad chords (Q releases capture, A mutes the mic).
private var hasChordModifiers: Bool {
let m = chordModifiersDown
return (m.contains(0xA2) || m.contains(0xA3)) // control
&& (m.contains(0xA4) || m.contains(0xA5)) // option
@@ -284,6 +292,10 @@ public final class InputCapture {
self.suppressedVK = 0x53
self.onCycleStats?()
return nil
case 0 /* A */:
self.suppressedVK = 0x41
self.onToggleMicMute?()
return nil
default:
break
}
@@ -704,7 +716,7 @@ public final class InputCapture {
}
}
#if os(iOS)
// Track Control/Option/Shift for the Q release chord below in both forwarding
// Track Control/Option/Shift for the chords below in both forwarding
// states (like `cmdKeysDown`) so a modifier held before capture engaged still counts.
if Self.chordModifierVKs.contains(vk) {
if pressed { self.chordModifiersDown.insert(vk) } else { self.chordModifiersDown.remove(vk) }
@@ -732,11 +744,19 @@ public final class InputCapture {
// otherwise). The Q is latched (`suppressedVK`) so its keyUp can't type into the host;
// the modifiers were forwarded as they went down and are flushed by the release
// path (setCaptured(false) releaseAll). VK 0x51 is layout-independent (physical Q).
if pressed, vk == 0x51, self.hasReleaseChordModifiers {
if pressed, vk == 0x51, self.hasChordModifiers {
self.suppressedVK = 0x51
self.onReleaseCapture?()
return
}
// A mutes/unmutes the mic uplink same detection, same latching, and needed here
// for the same reason as on macOS: a captured iPad swallows the Stream menu's
// identical key equivalent. VK 0x41 is layout-independent (physical A).
if pressed, vk == 0x41, self.hasChordModifiers {
self.suppressedVK = 0x41
self.onToggleMicMute?()
return
}
#endif
// Release direction of the toggle: GC's Esc-down can beat the NSEvent
// monitor never type Esc into the host while is held ( is reserved).
@@ -881,6 +881,12 @@ public final class StreamLayerView: NSView {
guard self?.window?.isKeyWindow == true else { return }
NotificationCenter.default.post(name: .punktfunkToggleFullscreen, object: nil)
}
capture.onToggleMicMute = { [weak self] in
// Session-level state the view doesn't own post to the app (same routing as the
// fullscreen chord), so the captured and released paths end at one toggle.
guard self?.window?.isKeyWindow == true else { return }
NotificationCenter.default.post(name: .punktfunkToggleMicMute, object: nil)
}
capture.onCycleStats = { [weak self] in
guard self?.window?.isKeyWindow == true else { return }
// Advance the shared tier setting directly every @AppStorage reader (the HUD's
@@ -175,6 +175,46 @@ public final class StreamViewController: StreamViewControllerBase {
/// renegotiates the host mode (1:1, no presenter resample). iOS only (iPhone naturally no-ops
/// its fixed full-screen scene; tvOS drives display modes via AVDisplayManager instead).
private var matchFollower: MatchWindowFollower?
// MARK: Escape-drop re-lock
//
// iPadOS releases the pointer lock BY ITSELF when the user presses Escape the platform's
// built-in "let me out", mirroring the web Pointer Lock API's default unlock gesture. Nothing
// in our code does it: a bare Esc never touches `captured`, so it keeps forwarding to the host
// as the game key it is. But the lock going away flips the mouse onto the absolute UIKit path
// and un-hides the iPadOS cursor, so hitting Esc for an in-game menu silently costs the capture
// until the user clicks to win it back. Esc is a GAME key here, not a request to hand the
// pointer back to iPadOS, so an unwanted drop is re-requested below. The DELIBERATE releases
// (, Q, the Stream menu, backgrounding) all clear `captured` first, so `wantsPointerLock`
// is already false when their drop is observed and none of them are fought here.
/// Whether this capture ever actually held the lock. Only a lock we HELD is worth winning back
/// never having been granted one means the scene doesn't qualify, not that Esc took it.
/// Cleared when capture ends, so each capture starts from a clean slate.
private var pointerLockWasEngaged = false
/// Attempts spent in the current re-lock burst, and when the burst began.
private var pointerRelockAttempt = 0
private var pointerRelockBurstStart: CFTimeInterval = 0
/// True from an unwanted drop until the lock is back (or the burst gives up). While pending,
/// the local cursor stays hidden and absolute pointer MOTION stays muted, so a re-lock that
/// lands a frame or two later is invisible instead of flashing the iPadOS cursor and
/// teleporting the host's to the pointer's absolute position.
private var pointerRelockPending = false
/// Forces `prefersPointerLocked` to report false for one resolve pass, so the escalated attempt
/// presents the system with a genuine falsetrue transition instead of re-asserting a value it
/// already holds. See `requestPointerRelock()`.
private var pointerLockForcedOff = false
/// A burst is 3 attempts, and a burst can't restart inside 2 s. A scene the system will never
/// lock (Stage Manager, Split View) therefore costs three cheap re-resolves and then falls back
/// to today's click-to-recapture, rather than retrying forever.
private static let pointerRelockAttemptLimit = 3
private static let pointerRelockBurstWindow: CFTimeInterval = 2
/// Gap between attempts in a burst long enough for the system to answer the previous
/// re-resolve, short enough that the whole burst fits in ~0.6 s. Must exceed
/// `pointerLockForcedOffHold` so an escalated attempt is back to preferring the lock before the
/// next attempt evaluates.
private static let pointerRelockRetryDelay: TimeInterval = 0.2
/// How long an escalated attempt reports `prefersPointerLocked == false` before flipping back,
/// so the system observes a real transition instead of coalescing the flip away.
private static let pointerLockForcedOffHold: TimeInterval = 0.05
#endif
/// Reads whether the scene's pointer is actually locked right now; nil = state
@@ -260,7 +300,7 @@ public final class StreamViewController: StreamViewControllerBase {
captured && pointerCaptureEnabled && UIDevice.current.userInterfaceIdiom == .pad
}
public override var prefersPointerLocked: Bool { wantsPointerLock }
public override var prefersPointerLocked: Bool { wantsPointerLock && !pointerLockForcedOff }
public override var prefersHomeIndicatorAutoHidden: Bool { true }
// NOTE: we deliberately do NOT override `childViewControllerForPointerLock`. The default
@@ -383,6 +423,11 @@ public final class StreamViewController: StreamViewControllerBase {
// is the exact mirror of the GCMouse handlers, which fire only while locked.
streamView.onPointerMoveAbs = { [weak self] p in
guard let self, self.inputCapture?.gcMouseForwarding == false else { return }
// A re-lock is in flight after an Esc-drop: the absolute path would teleport the host
// cursor to wherever the local pointer sits, undoing the relative aiming we're about to
// resume. Motion only BUTTONS still forward (they carry no position, so a click during
// the couple of frames a re-lock takes must not be swallowed mid-firefight).
guard !self.pointerRelockPending else { return }
self.inputCapture?.sendMouseAbs(
x: p.x, y: p.y, surfaceWidth: p.w, surfaceHeight: p.h)
}
@@ -424,6 +469,12 @@ public final class StreamViewController: StreamViewControllerBase {
capture.onReleaseCapture = { [weak self] in
self?.setCaptured(false)
}
// A mutes/unmutes the mic uplink. Session state this controller doesn't own, so it
// posts to the app exactly as the macOS chord does the Stream menu's identical
// equivalent (which a captured scene swallows) ends at the same toggle.
capture.onToggleMicMute = {
NotificationCenter.default.post(name: .punktfunkToggleMicMute, object: nil)
}
capture.onPreempted = { [weak self] in
self?.setCaptured(false)
}
@@ -687,6 +738,24 @@ public final class StreamViewController: StreamViewControllerBase {
/// change and capture toggle. Main queue.
private func syncPointerLock() {
let locked = pointerLockEngaged() == true
// Wanted, previously HELD, and now gone is the Esc-drop signature. The "previously held"
// half matters: a lock that was never granted is a scene that doesn't qualify (Stage
// Manager, Split View), and burst-requesting there would hide the cursor for the burst's
// duration to win a lock that isn't coming. A first grant is already driven by the chain
// engage in setCaptured/viewDidAppear.
if locked {
pointerLockWasEngaged = true
pointerRelockPending = false
pointerRelockAttempt = 0
} else if wantsPointerLock, pointerLockWasEngaged {
requestPointerRelock()
} else {
// Capture is gone (or the lock was never ours) settle, and let the next capture
// start from a clean "never held" slate.
if !wantsPointerLock { pointerLockWasEngaged = false }
pointerRelockPending = false
pointerRelockAttempt = 0
}
let useGCMouse = captured && locked
// Lock dropped (or capture ended) while the GCMouse path held a button down: once
// gcMouseForwarding flips false its release handler is gated off, so flush any held
@@ -698,7 +767,83 @@ public final class StreamViewController: StreamViewControllerBase {
pointerInteraction?.invalidate() // re-resolve the hidden/visible cursor for the state
if iosInputDebug {
iosInputLog.debug(
"pointer lock isLocked=\(locked, privacy: .public) captured=\(self.captured, privacy: .public)")
"""
pointer lock isLocked=\(locked, privacy: .public) \
captured=\(self.captured, privacy: .public) \
relockPending=\(self.pointerRelockPending, privacy: .public) \
relockAttempt=\(self.pointerRelockAttempt, privacy: .public)
""")
}
}
/// Ask the system for the lock back after it dropped one we still want (see the Escape-drop
/// note on the state above). Bounded to a short burst; idempotent within it. Main queue.
private func requestPointerRelock() {
// Only a frontmost scene can hold the lock at all. Anywhere else the drop is the system
// saying we don't qualify, not the Esc key re-asking would be noise, and the qualifying
// states (foreground, appearance, reparent) each re-resolve on their own already.
guard view.window?.windowScene?.activationState == .foregroundActive else {
pointerRelockPending = false
return
}
let now = CACurrentMediaTime()
// attempt == 0 is a fresh burst (first drop, or one the settle branch cleared); the window
// is the backstop for the pathological case where a grant is immediately revoked again and
// re-arms us. Even then this stays timer-driven at a few Hz never a spin.
if pointerRelockAttempt == 0 || now - pointerRelockBurstStart > Self.pointerRelockBurstWindow {
pointerRelockBurstStart = now
pointerRelockAttempt = 0
}
guard pointerRelockAttempt < Self.pointerRelockAttemptLimit else {
// Out of budget: fall back to exactly today's behavior the iPadOS cursor comes back
// and a click into the video re-captures. The caller invalidates the interaction, so
// the cursor can never stay hidden on a lock the system won't grant.
pointerRelockPending = false
return
}
pointerRelockAttempt += 1
pointerRelockPending = true
let escalate = pointerRelockAttempt > 1
// Deferred a turn so a whose GC keystroke lands after the system's unlock notification
// has already cleared `captured` then the guard below drops this attempt instead of
// fighting the user's own release.
DispatchQueue.main.async { [weak self] in
guard let self, self.pointerRelockPending else { return }
guard self.wantsPointerLock, self.pointerLockEngaged() != true else {
// The grant landed, or the capture went away under us ( / Q / resign).
// Settle through the one decision point rather than returning with `pending` still
// set that flag hides the cursor, so it must never outlive the burst.
self.syncPointerLock()
return
}
if escalate {
// Re-asserting a value the system already holds didn't take. Present a real
// falsetrue transition instead the documented way to change your mind about the
// lock and re-anchor the chain in case a reparent broke the downward walk to us.
// Held for a beat rather than cleared on the next turn: the system resolves the
// property asynchronously, and a same-turn flip back to true can be coalesced into
// no transition at all. We are already unlocked, so the false pass costs nothing.
self.pointerLockForcedOff = true
self.setNeedsUpdateOfPrefersPointerLocked()
self.updatePointerLockChain()
DispatchQueue.main.asyncAfter(deadline: .now() + Self.pointerLockForcedOffHold) {
[weak self] in
guard let self else { return }
self.pointerLockForcedOff = false
self.setNeedsUpdateOfPrefersPointerLocked()
}
} else {
self.setNeedsUpdateOfPrefersPointerLocked()
}
// A GRANT arrives as a didChange syncPointerLock, which settles the burst and makes
// this retry a no-op. Routed back through syncPointerLock (not straight into another
// requestPointerRelock) so the give-up path re-resolves the cursor through the one
// place that does it.
DispatchQueue.main.asyncAfter(deadline: .now() + Self.pointerRelockRetryDelay) {
[weak self] in
guard let self, self.pointerRelockPending else { return }
self.syncPointerLock()
}
}
}
#endif
@@ -718,7 +863,11 @@ extension StreamViewController: UIPointerInteractionDelegate {
// host renders its own cursor from GCMouse deltas and a visible local one would just
// diverge. When the lock isn't held the cursor stays VISIBLE so the user can aim; the
// pointer is forwarded as an absolute position, both cursors tracking together.
captured && pointerLockEngaged() == true ? .hidden() : nil
// except across an Esc-drop we're actively re-locking (`pointerRelockPending`): staying
// hidden for those couple of frames is what turns the fix into "Esc did nothing to my
// mouse" rather than a cursor that blinks in and out. The burst is bounded and clears
// itself on give-up, so the cursor can never stay hidden on a lock that isn't coming.
captured && (pointerLockEngaged() == true || pointerRelockPending) ? .hidden() : nil
}
}
#endif
@@ -32,6 +32,12 @@ public enum DefaultsKey {
public static let compositor = "punktfunk.compositor"
public static let gamepadType = "punktfunk.gamepadType"
public static let gamepadID = "punktfunk.gamepadID"
/// Forward this device's controllers to the host at all (default true). Off is for a
/// couch whose controller reaches the host another way USB passthrough such as
/// VirtualHere, or a pad plugged into the host where forwarding as well would give the
/// host two pads for one pair of hands. Read at connect: `SessionModel` then never starts
/// `GamepadCapture`, so no slot opens, no arrival is sent and no virtual pad is built.
public static let gamepadForwarding = "punktfunk.gamepadForwarding"
public static let bitrateKbps = "punktfunk.bitrateKbps"
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the in-core decode + AVAudioEngine layout.
@@ -42,6 +48,13 @@ public enum DefaultsKey {
/// falls back. Drives the decoder via `Welcome.codec`.
public static let codec = "punktfunk.codec"
public static let micEnabled = "punktfunk.micEnabled"
/// Echo cancellation for the mic uplink (on by default): playback + capture share ONE
/// audio engine so the system voice processor can subtract what this device is playing
/// from what its mic hears without it a loudspeaker client feeds the game audio straight
/// back to the host. Off = the raw two-engine capture path. macOS: an explicitly pinned
/// speaker/mic or mic channel also bypasses it (the voice processor only follows the
/// system default devices) see SessionAudio's topology note.
public static let echoCancel = "punktfunk.echoCancel"
public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID"
/// macOS: which input channel of the chosen mic device feeds the host. 0 = "Auto" (sum every
@@ -193,6 +206,12 @@ extension Notification.Name {
/// state. macOS only.
public static let punktfunkToggleFullscreen = Notification.Name("io.unom.punktfunk.toggle-fullscreen")
/// Posted by InputCapture's chord path (A) when the combo fires while input is CAPTURED
/// the state in which the Stream menu's identical key equivalent never reaches the app. The
/// live session's owner (ContentView) flips the session's mic mute. Released, the menu item
/// handles the same combo directly; both end at `SessionModel.toggleMicMute`.
public static let punktfunkToggleMicMute = Notification.Name("io.unom.punktfunk.toggle-mic-mute")
/// Posted by the Live Activity's / Shortcuts' End-stream intent (`EndStreamIntent.perform`,
/// which runs in the app's process): the app tears the active session down deliberately
/// (quit-close the host). Same cross-process-signal pattern as `punktfunkReleaseCapture`
@@ -29,10 +29,12 @@ public struct EffectiveSettings: Equatable, Sendable {
public var compositor = 0
public var audioChannels = 2
public var micEnabled = true
public var echoCancel = true
public var touchMode = "trackpad"
public var mouseMode = "capture"
public var invertScroll = false
public var gamepadType = 0
public var gamepadForwarding = true
/// A `StatsVerbosity` raw value; the enum lives in PunktfunkKit, which this module can't see.
public var statsVerbosity = "normal"
public var fullscreenWhileStreaming = true
@@ -87,10 +89,12 @@ public struct EffectiveSettings: Equatable, Sendable {
compositor = int(DefaultsKey.compositor, compositor)
audioChannels = int(DefaultsKey.audioChannels, audioChannels)
micEnabled = bool(DefaultsKey.micEnabled, micEnabled)
echoCancel = bool(DefaultsKey.echoCancel, echoCancel)
touchMode = str(DefaultsKey.touchMode, touchMode)
mouseMode = str(DefaultsKey.mouseMode, mouseMode)
invertScroll = bool(DefaultsKey.invertScroll, invertScroll)
gamepadType = int(DefaultsKey.gamepadType, gamepadType)
gamepadForwarding = bool(DefaultsKey.gamepadForwarding, gamepadForwarding)
statsVerbosity = Self.storedStatsVerbosity(defaults)
fullscreenWhileStreaming = bool(
DefaultsKey.fullscreenWhileStreaming, fullscreenWhileStreaming)
@@ -133,10 +137,12 @@ public struct EffectiveSettings: Equatable, Sendable {
if let v = overlay.compositor { s.compositor = v }
if let v = overlay.audioChannels { s.audioChannels = v }
if let v = overlay.micEnabled { s.micEnabled = v }
if let v = overlay.echoCancel { s.echoCancel = v }
if let v = overlay.touchMode { s.touchMode = v }
if let v = overlay.mouseMode { s.mouseMode = v }
if let v = overlay.invertScroll { s.invertScroll = v }
if let v = overlay.gamepadType { s.gamepadType = v }
if let v = overlay.gamepadForwarding { s.gamepadForwarding = v }
if let v = overlay.statsVerbosity { s.statsVerbosity = v }
if let v = overlay.fullscreenWhileStreaming { s.fullscreenWhileStreaming = v }
if let v = overlay.enable444 { s.enable444 = v }
@@ -105,10 +105,12 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
public var compositor: Int?
public var audioChannels: Int?
public var micEnabled: Bool?
public var echoCancel: Bool?
public var touchMode: String?
public var mouseMode: String?
public var invertScroll: Bool?
public var gamepadType: Int?
public var gamepadForwarding: Bool?
/// A `StatsVerbosity` raw value ("off"/"compact"/"normal"/"detailed") the enum lives in
/// PunktfunkKit, which this module must not depend on.
public var statsVerbosity: String?
@@ -145,10 +147,12 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
case compositor
case audioChannels = "audio_channels"
case micEnabled = "mic_enabled"
case echoCancel = "echo_cancel"
case touchMode = "touch_mode"
case mouseMode = "mouse_mode"
case invertScroll = "invert_scroll"
case gamepadType = "gamepad"
case gamepadForwarding = "gamepad_forwarding"
case statsVerbosity = "stats_verbosity"
case fullscreenWhileStreaming = "fullscreen_on_stream"
case enable444 = "enable_444"
@@ -177,10 +181,12 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
compositor = int(.compositor)
audioChannels = int(.audioChannels)
micEnabled = bool(.micEnabled)
echoCancel = bool(.echoCancel)
touchMode = str(.touchMode)
mouseMode = str(.mouseMode)
invertScroll = bool(.invertScroll)
gamepadType = int(.gamepadType)
gamepadForwarding = bool(.gamepadForwarding)
statsVerbosity = str(.statsVerbosity)
fullscreenWhileStreaming = bool(.fullscreenWhileStreaming)
enable444 = bool(.enable444)
@@ -211,10 +217,13 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
try c.encodeIfPresent(compositor, forKey: AnyKey(Key.compositor.rawValue))
try c.encodeIfPresent(audioChannels, forKey: AnyKey(Key.audioChannels.rawValue))
try c.encodeIfPresent(micEnabled, forKey: AnyKey(Key.micEnabled.rawValue))
try c.encodeIfPresent(echoCancel, forKey: AnyKey(Key.echoCancel.rawValue))
try c.encodeIfPresent(touchMode, forKey: AnyKey(Key.touchMode.rawValue))
try c.encodeIfPresent(mouseMode, forKey: AnyKey(Key.mouseMode.rawValue))
try c.encodeIfPresent(invertScroll, forKey: AnyKey(Key.invertScroll.rawValue))
try c.encodeIfPresent(gamepadType, forKey: AnyKey(Key.gamepadType.rawValue))
try c.encodeIfPresent(
gamepadForwarding, forKey: AnyKey(Key.gamepadForwarding.rawValue))
try c.encodeIfPresent(statsVerbosity, forKey: AnyKey(Key.statsVerbosity.rawValue))
try c.encodeIfPresent(
fullscreenWhileStreaming, forKey: AnyKey(Key.fullscreenWhileStreaming.rawValue))
@@ -262,10 +271,12 @@ public enum OverlayField {
case "compositor": overlay.compositor = nil
case "audio_channels": overlay.audioChannels = nil
case "mic_enabled": overlay.micEnabled = nil
case "echo_cancel": overlay.echoCancel = nil
case "touch_mode": overlay.touchMode = nil
case "mouse_mode": overlay.mouseMode = nil
case "invert_scroll": overlay.invertScroll = nil
case "gamepad": overlay.gamepadType = nil
case "gamepad_forwarding": overlay.gamepadForwarding = nil
case "stats_verbosity": overlay.statsVerbosity = nil
case "fullscreen_on_stream": overlay.fullscreenWhileStreaming = nil
case "enable_444": overlay.enable444 = nil
@@ -296,10 +307,12 @@ public enum OverlayField {
case "compositor": return o.compositor != nil
case "audio_channels": return o.audioChannels != nil
case "mic_enabled": return o.micEnabled != nil
case "echo_cancel": return o.echoCancel != nil
case "touch_mode": return o.touchMode != nil
case "mouse_mode": return o.mouseMode != nil
case "invert_scroll": return o.invertScroll != nil
case "gamepad": return o.gamepadType != nil
case "gamepad_forwarding": return o.gamepadForwarding != nil
case "stats_verbosity": return o.statsVerbosity != nil
case "fullscreen_on_stream": return o.fullscreenWhileStreaming != nil
case "enable_444": return o.enable444 != nil
@@ -9,32 +9,34 @@ import XCTest
@testable import PunktfunkKit
final class OpusCodecTests: XCTestCase {
/// Encode a 440 Hz stereo tone, decode it back, and require the result to be
/// recognizably the same signal (Opus is lossy check correlation, not bytes).
/// Encode a 440 Hz mono tone (the uplink's shape), decode it back through the
/// STEREO-configured decoder (the host-plane shape Opus upmixes mono packets), and
/// require the result to be recognizably the same signal (Opus is lossy check
/// correlation, not bytes).
func testEncodeDecodeRoundTripPreservesTone() throws {
let encoder = try OpusEncoder()
let decoder = try OpusDecoder(framesPerPacket: UInt32(OpusEncoder.framesPerPacket))
let decoder = try OpusDecoder(framesPerPacket: UInt32(encoder.framesPerPacket))
let pcmFormat = encoder.pcmFormat
let frames = OpusEncoder.framesPerPacket
let frames = encoder.framesPerPacket
var packets: [Data] = []
var phase: Float = 0
let step = 2 * Float.pi * 440 / 48_000
// 50 packets = 1 s of tone.
for _ in 0..<50 {
// 1 s of tone, whatever packet duration the encoder chose (10 ms 100 chunks).
let chunks = Int(48_000 / frames)
for _ in 0..<chunks {
let buf = AVAudioPCMBuffer(pcmFormat: pcmFormat, frameCapacity: frames)!
buf.frameLength = frames
let p = buf.floatChannelData![0] // interleaved: one plane, L R L R
let p = buf.floatChannelData![0] // mono: one plane
for f in 0..<Int(frames) {
let s = sin(phase) * 0.5
p[f] = sin(phase) * 0.5
phase += step
p[f * 2] = s
p[f * 2 + 1] = s
}
packets.append(contentsOf: try encoder.encode(buf))
}
XCTAssertGreaterThanOrEqual(packets.count, 45, "encoder must emit ~one packet per buffer")
XCTAssertGreaterThanOrEqual(
packets.count, chunks - 5, "encoder must emit ~one packet per buffer")
XCTAssertTrue(packets.allSatisfy { !$0.isEmpty })
var decoded: [Float] = []
@@ -62,29 +62,30 @@ final class RemoteFirstLightTests: XCTestCase {
host: host, port: port, width: 1280, height: 720, refreshHz: 60)
defer { conn.close() }
// Mic uplink: 2 s of 440 Hz tone (the host's mic service opens its virtual
// Mic uplink: 2 s of 440 Hz mono tone (the host's mic service opens its virtual
// source on the first frame check its log).
let encoder = try OpusEncoder()
let chunk = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: OpusEncoder.framesPerPacket)!
pcmFormat: encoder.pcmFormat, frameCapacity: encoder.framesPerPacket)!
var phase: Float = 0
let step = 2 * Float.pi * 440 / 48_000
var seq: UInt32 = 0
for _ in 0..<100 {
chunk.frameLength = OpusEncoder.framesPerPacket
let p = chunk.floatChannelData![0]
for f in 0..<Int(OpusEncoder.framesPerPacket) {
let s = sin(phase) * 0.25
let chunks = 2 * 48_000 / Int(encoder.framesPerPacket)
let packetNs = UInt64(encoder.framesPerPacket) * 1_000_000_000 / 48_000
for _ in 0..<chunks {
chunk.frameLength = encoder.framesPerPacket
let p = chunk.floatChannelData![0] // mono: one plane
for f in 0..<Int(encoder.framesPerPacket) {
p[f] = sin(phase) * 0.25
phase += step
p[f * 2] = s
p[f * 2 + 1] = s
}
for packet in try encoder.encode(chunk) {
conn.sendMic(packet, seq: seq, ptsNs: UInt64(seq) * 20_000_000)
conn.sendMic(packet, seq: seq, ptsNs: UInt64(seq) * packetNs)
seq &+= 1
}
}
XCTAssertGreaterThanOrEqual(seq, 95, "mic encoder must emit ~one packet per chunk")
XCTAssertGreaterThanOrEqual(
seq, UInt32(chunks - 5), "mic encoder must emit ~one packet per chunk")
// Downlink: pull host audio packets and decode them (the host streams its sink
// monitor silence still produces packets).
+10 -3
View File
@@ -24,8 +24,15 @@ the panel looks and feels native to Gaming Mode.
browser (aurora backdrop + poster coverflow; A plays, B returns to Gaming Mode). Pins survive
plugin reinstalls (stored next to the client's config) and follow a host across IP changes
(matched by certificate fingerprint).
5. **Settings**resolution / refresh / bitrate / gamepad type / host compositor / mic, written
to the client's config.
5. **Settings**the client's whole settings store, written to its config. Laid out like SteamOS's
own Settings: a left rail of categories (`SidebarNavigation`), one page each, so no page needs
scrolling. The categories and their order are the console settings screen's — Stream (resolution
/ refresh / render scale / bitrate / compositor), Video (codec / decoder / GPU / HDR / 4:4:4),
Presentation (prioritize / smoothness buffer / V-Sync / VRR), Audio (channels / output + mic
device / echo cancellation), Controllers, Touch & mouse, Interface (stats overlay / auto-wake /
library / fullscreen). The device pickers are populated
from the session binary (`--list-adapters` / `--list-audio`); the GPU row appears only where
there is more than one adapter.
6. **About** — plugin version, an explicit "Check for updates" button, the setup-guide link, and
a force-stop for a wedged stream client.
@@ -93,7 +100,7 @@ restart is required for an out-of-band install to appear.
| --- | --- |
| `src/index.tsx` | Plugin entry: the QAM panel + route registration. |
| `src/page.tsx` | The `/punktfunk` fullscreen page — Hosts (with per-host details) / Settings / About tabs. |
| `src/settings.tsx` · `src/pair.tsx` | Stream-settings section; the gamepad-navigable PIN-pairing modal. |
| `src/settings.tsx` · `src/pair.tsx` | The settings screen (a `SidebarNavigation` of seven category pages over one shared settings object); the gamepad-navigable PIN-pairing modal. |
| `src/library.tsx` | The per-host game picker (pin/unpin, "Open library on screen") + the pinned-game launch helper. |
| `src/hostmgmt.tsx` | Add / edit host dialogs — mutate the shared known-hosts store (`client-known-hosts.json`) via the flatpak client's headless modes, so a host saved here shows up in the desktop client too. |
| `src/ui.tsx` | Shared UI primitives for the fullscreen page + modals (right-aligned row actions, consistent Field layout). |
+152 -4
View File
@@ -21,6 +21,10 @@ The backend's jobs are the things Steam can't do:
the frontend so it can create/point the Steam shortcut.
* **get_settings() / set_settings()** read/write the flatpak client's stream settings JSON
(resolution / bitrate / gamepad), so the Deck UI configures the stream the client reads.
``set_settings`` MERGES onto the file: it is shared with the desktop client and the console.
* **list_devices() / refresh_devices()** the GPUs and audio endpoints the settings tab's
device pickers offer, read from the session binary (``--list-adapters`` / ``--list-audio``)
and cached, since enumerating them costs a Vulkan + PipeWire init.
* **kill_stream()** force-stop a wedged stream (``flatpak kill``).
* **check_update()** report pending updates for BOTH the plugin and the client. The plugin's
comes from the registry's per-channel ``manifest.json`` (the frontend then drives Decky's own
@@ -343,6 +347,9 @@ def _flatpak() -> str | None:
# settings in the same ~/.config/punktfunk (the flatpak's sandbox HOME resolves to the real
# home), so nothing else in this file has to care which one answered.
NATIVE_BIN = "punktfunk-client"
# The Vulkan session binary the shell execs to stream — and the only thing that can enumerate
# this device's GPUs and audio endpoints for the settings pickers.
SESSION_BIN = "punktfunk-session"
# Prefixes to try when PATH doesn't have it. The Decky backend runs with a minimal PATH, and
# SteamOS's read-only /usr pushes native installs into a sysext or the user's own prefix.
@@ -398,6 +405,25 @@ def _client_argv() -> list[str] | None:
return [native] if native else None
def _session_argv() -> list[str] | None:
"""The argv PREFIX that runs the SESSION binary headlessly, or None when it isn't there.
The device enumerations the settings pickers need (`--list-adapters`, `--list-audio`) live on
`punktfunk-session`, not on the client: the GTK shell deliberately links no Vulkan itself and
shells out to the session for exactly the same two lists (clients/linux/src/app.rs). The
flatpak installs both binaries into /app/bin, so `--command=` picks the other one; a native
install puts them in the same bindir, so the session is the client's sibling.
"""
prefix = _client_argv()
if not prefix:
return None
if prefix[0] == _flatpak():
# `flatpak run --command=<bin> <app>` — the app id must stay LAST.
return [*prefix[:-1], f"--command={SESSION_BIN}", prefix[-1]]
sibling = Path(prefix[0]).with_name(SESSION_BIN)
return [str(sibling)] if sibling.exists() else None
def _client_is_flatpak() -> bool:
"""Is the client this plugin actually drives the FLATPAK one?
@@ -511,6 +537,63 @@ async def _run_client(client_args: list[str], timeout: float = 20.0) -> tuple[in
return -1, "", ""
def _parse_audio_endpoints(out: str) -> tuple[list[dict], list[dict]]:
"""Split `punktfunk-session --list-audio` into ``(sinks, sources)``.
Its format is one endpoint per line, ``sink|source<TAB>node.name<TAB>description``. The
node.name is what gets STORED (it is the stable id the client resolves against), so a line
without one is unusable and dropped; a missing description falls back to the name rather than
rendering a picker entry with no label. Anything else on the line is ignored, so an extra
trailing column in a future client can't break this.
"""
sinks: list[dict] = []
sources: list[dict] = []
for line in out.splitlines():
parts = line.split("\t")
if len(parts) < 3 or not parts[1].strip():
continue
kind, name, description = parts[0].strip(), parts[1].strip(), parts[2].strip()
entry = {"name": name, "description": description or name}
if kind == "sink":
sinks.append(entry)
elif kind == "source":
sources.append(entry)
return sinks, sources
async def _run_session(session_args: list[str], timeout: float = 25.0) -> tuple[int, str]:
"""Run the SESSION binary headlessly, returning ``(returncode, stdout)``; ``(-1, "")`` when
it isn't installed or the call errors/times out.
Only ever used for the two read-only device enumerations the launch path goes through the
Steam shortcut and the wrapper script, never through here. The timeout is generous because
`--list-adapters` initialises Vulkan on a cold flatpak."""
prefix = _session_argv()
if not prefix:
return -1, ""
proc = None
try:
proc = await asyncio.create_subprocess_exec(
*prefix, *session_args,
stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.DEVNULL,
env=_flatpak_env(),
)
out, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout)
rc = proc.returncode if proc.returncode is not None else -1
return rc, (out or b"").decode("utf-8", "replace")
except asyncio.TimeoutError:
decky.logger.warning("session %s timed out", " ".join(session_args))
if proc:
try:
proc.kill()
except ProcessLookupError:
pass
return -1, ""
except Exception: # noqa: BLE001
decky.logger.exception("session %s failed", " ".join(session_args))
return -1, ""
# The QAM panel and the full page each mount their own hosts view, and Gaming Mode remounts the
# QAM often — every mount calls list_hosts, which spawns a flatpak cold-start plus a reachability
# probe. Cache the last result briefly so back-to-back opens reuse it instead of re-probing; any
@@ -518,6 +601,11 @@ async def _run_client(client_args: list[str], timeout: float = 20.0) -> tuple[in
_HOSTS_TTL_S = 12.0
_hosts_cache: dict = {"at": 0.0, "probed": None, "data": None}
# The settings tab's device lists (GPUs / audio endpoints). No TTL: this is hardware, and reading
# it costs a Vulkan + PipeWire init. Held for the life of the plugin backend; `refresh_devices`
# clears it for the user who just plugged a headset in.
_devices_cache: dict = {"data": None}
def _invalidate_hosts_cache() -> None:
_hosts_cache["data"] = None
@@ -1044,24 +1132,84 @@ class Plugin:
try:
return json.loads(_settings_path().read_text())
except (OSError, json.JSONDecodeError):
# The client's own defaults (native display, host-default bitrate, auto pad).
# The client's own defaults (native display, host-default bitrate, auto pad,
# stats overlay at Normal — `Settings::default` is `show_stats: true`).
return {
"width": 0, "height": 0, "refresh_hz": 0, "render_scale": 1.0,
"bitrate_kbps": 0, "codec": "auto", "gamepad": "auto", "compositor": "auto",
"bitrate_kbps": 0, "codec": "auto", "gamepad": "auto",
"gamepad_forwarding": True, "compositor": "auto",
"inhibit_shortcuts": True, "mic_enabled": False,
"stats_verbosity": "normal", "show_stats": True,
}
async def set_settings(self, settings: dict) -> dict:
"""Write the stream settings JSON the (sandboxed) client reads on launch."""
"""Write the stream settings JSON the (sandboxed) client reads on launch.
MERGED onto whatever is on disk, never a wholesale replace: this file is shared with
the desktop client and the console's settings screen, and it holds far more keys than
this panel models (decoder, GPU, profiles, touch/mouse model). The panel reads it once
when it mounts, so a straight write would post a snapshot that predates anything those
other editors stored in the meantime silently reverting it.
"""
try:
d = _client_config_dir()
d.mkdir(parents=True, exist_ok=True)
_settings_path().write_text(json.dumps(settings, indent=2))
try:
on_disk = json.loads(_settings_path().read_text())
if not isinstance(on_disk, dict):
on_disk = {}
except (OSError, json.JSONDecodeError):
on_disk = {} # no file yet (or an unreadable one): this write creates it
on_disk.update(settings)
_settings_path().write_text(json.dumps(on_disk, indent=2))
return {"ok": True}
except OSError as exc:
decky.logger.exception("could not write settings")
return {"ok": False, "error": str(exc)}
async def list_devices(self) -> dict:
"""GPUs + audio endpoints for the settings tab's device pickers.
Two subprocesses that initialise Vulkan and PipeWire, so the result is cached for the
Decky session: hardware doesn't come and go often enough to justify paying that on every
remount of the page, and a stale entry is harmless a picked device that has since
vanished falls back to the OS default in the client anyway. `refresh_devices` clears it.
Best-effort in the same way every other client call here is: no session binary (an old
flatpak that predates the two-binary split, or a native install missing its sibling) just
means empty lists and `ok: false`, which the UI shows as "couldn't read" rather than as
"you have no devices".
"""
if _devices_cache["data"] is not None:
return _devices_cache["data"]
adapters: list[str] = []
sinks: list[dict] = []
sources: list[dict] = []
rc_a, out_a = await _run_session(["--list-adapters"])
if rc_a == 0:
adapters = [ln.strip() for ln in out_a.splitlines() if ln.strip()]
rc_d, out_d = await _run_session(["--list-audio"])
if rc_d == 0:
sinks, sources = _parse_audio_endpoints(out_d)
result = {
"ok": rc_a == 0 or rc_d == 0,
"adapters": adapters,
"sinks": sinks,
"sources": sources,
}
# Only a run that actually answered is worth remembering — caching a failure would make
# a client installed after the page was first opened stay invisible until a Decky restart.
if result["ok"]:
_devices_cache["data"] = result
return result
async def refresh_devices(self) -> dict:
"""Drop the cached enumeration and read it again (a headset was just plugged in)."""
_devices_cache["data"] = None
return await self.list_devices()
# ---- Shared known-hosts store (the SAME file the desktop client reads/writes) ----
async def list_hosts(self, probe: bool = True) -> dict:
+27
View File
@@ -144,6 +144,33 @@ got = asyncio.run(plugin.get_pins())["pins"]
check("pins: paired via known-hosts fp (case-insensitive)", got[0]["paired"] is True)
shutil.rmtree(decky.DECKY_USER_HOME, ignore_errors=True)
# ---- `--list-audio` parsing (the settings tab's device pickers) --------------------------
sinks, sources = main._parse_audio_endpoints(
"sink\talsa_output.pci-0000_04_00.6.analog-stereo\tSteam Deck Speakers\n"
"sink\tbluez_output.AC_12_2F.1\tWH-1000XM4\n"
"source\talsa_input.pci-0000_04_00.6.analog-stereo\tSteam Deck Microphone\n"
)
check("audio: sinks parsed", [d["name"] for d in sinks] == [
"alsa_output.pci-0000_04_00.6.analog-stereo", "bluez_output.AC_12_2F.1"
])
check("audio: sources parsed", len(sources) == 1)
check("audio: description kept", sinks[1]["description"] == "WH-1000XM4")
# Junk the picker must not offer: no node.name is unusable (it is the id that gets stored), a
# short line is malformed, and an unknown kind belongs to neither list. A blank description
# falls back to the name so no entry renders unlabelled.
sinks, sources = main._parse_audio_endpoints(
"sink\t\tNo node name\n"
"sink\tonly-two-columns\n"
"monitor\tsome.monitor\tNot a sink or source\n"
"source\tbare.node\t\n"
"\n"
)
check("audio: junk lines dropped", sinks == [])
check("audio: blank description falls back to the node name", sources == [
{"name": "bare.node", "description": "bare.node"}
])
print()
if failures:
print(f"{failures} check(s) FAILED")
+88 -10
View File
@@ -101,24 +101,97 @@ export interface RunnerInfo {
client_bin?: string;
}
// The slice of the flatpak client's settings JSON this UI surfaces. The file can hold more
// keys (decoder, … set from the desktop client's own UI) — they round-trip untouched
// because get_settings returns the whole parsed file and patches are object spreads.
// The flatpak client's settings JSON — the SAME `client-gtk-settings.json` the desktop client
// and the console's settings screen own, so a value changed in any of them shows in the others.
//
// Every field the client's `Settings` struct persists is modelled here EXCEPT the ones that
// cannot be answered from a plugin backend or aren't settings at all:
// • `forward_pad` — which physical pad is player 1. Needs SDL's live device list, which only
// the client process has; there is no CLI that enumerates pads.
// • `last_window_w/h` — the session's remembered window size, written BY the client, not a
// preference anyone sets.
// Both round-trip untouched: get_settings returns the whole parsed file, patches are object
// spreads, and set_settings merges onto what's on disk.
//
// Optional (`?`) marks a key the client writes with a serde `default`, so a store written before
// that key existed simply lacks it. Read those through the same fallback the client uses —
// `?? true` for the default-on ones, never `!!` — or a pre-existing file reads as "off" here
// while the stream runs with it on.
export interface StreamSettings {
// ---- Stream mode ----
width: number; // 0 = native
height: number; // 0 = native
refresh_hz: number; // 0 = native
render_scale?: number; // render-resolution multiplier; 1.0 = native (absent in pre-scale files)
bitrate_kbps: number; // 0 = host default
codec?: string; // "auto" | "hevc" | "h264" | "av1" — soft preference (absent in pre-codec files)
gamepad: string; // "auto" | "xbox360" | "xboxone" | "dualsense" | "dualshock4" | "steamdeck"
compositor: string; // "auto" | "kwin" | "wlroots" | "mutter" | "gamescope"
// Round-trips only — deliberately NOT offered as a row here. It decides whether the session
// grabs the keyboard so Alt+Tab/Super reach the host, and Game Mode is gamescope: it has no
// compositor shortcuts to inhibit and hands the focused window every key already. A toggle
// here would be a dead one. The desktop client's row still edits this same file.
inhibit_shortcuts: boolean;
// Stream mode follows the session window instead of width/height, renegotiating on resize.
// Overrides width/height while on; degenerates to the display's native mode on fullscreen.
match_window?: boolean;
// ---- Video ----
codec?: string; // "auto" | "hevc" | "h264" | "av1" | "pyrowave" (absent in pre-codec files)
decoder?: string; // "auto" | "vulkan" | "vaapi" | "software"
hdr_enabled?: boolean; // default ON — advertise 10-bit/HDR10
enable_444?: boolean; // default off — ask for full chroma
adapter?: string; // decode/present GPU by marketing name; "" = automatic
// ---- Presentation ----
// What the client optimises for when a decoded frame is ready: "latency" | "smooth". Shared
// with the Apple and Android clients under this name, so one profile reads the same everywhere.
present_priority?: string;
smooth_buffer?: number; // frames held back under "smooth"; 0 = Automatic (resolves to 2), else 13
vsync?: boolean; // default ON — tear-free; off asks for a tearing present mode (best-effort)
allow_vrr?: boolean; // default ON — let a VRR panel refresh in step with the stream
// ---- Audio ----
audio_channels?: number; // 2 (stereo) | 6 (5.1) | 8 (7.1)
speaker_device?: string; // PipeWire node.name for playback; "" = system default
mic_enabled: boolean;
mic_device?: string; // PipeWire node.name for capture; "" = system default
echo_cancel?: boolean; // default ON; only meaningful while mic_enabled
// ---- Controllers ----
gamepad: string; // "auto" | "xbox360" | "xboxone" | "dualsense" | "dualshock4" | "steamdeck"
// Forward this device's controllers at all. Absent in pre-forwarding files, where the
// client's own serde default (true) applies — so `?? true` at every read, never `!!`.
gamepad_forwarding?: boolean;
// ---- Touchscreen, mouse & keyboard ----
touch_mode?: string; // "trackpad" | "pointer" | "touch"
mouse_mode?: string; // "capture" | "desktop"
invert_scroll?: boolean;
// Whether the session grabs the keyboard so Alt+Tab/Super reach the host.
inhibit_shortcuts: boolean;
// ---- Interface & behaviour ----
// Stats-overlay tier: "off" | "compact" | "normal" | "detailed". Absent in a pre-tier file,
// which resolves through `show_stats` — read both the way the client's
// `Settings::stats_verbosity` does, and write both the way `set_stats_verbosity` does.
stats_verbosity?: string;
// The legacy on/off the tier supersedes; kept written in sync so a client that predates the
// tiers still honours an Off chosen here.
show_stats?: boolean;
fullscreen_on_stream?: boolean;
auto_wake?: boolean; // default ON — Wake-on-LAN a sleeping host before connecting
library_enabled?: boolean; // the CLIENT's own library browser (this plugin has its own)
}
// One audio endpoint from the client's enumeration: the stable id that gets stored, plus the
// human name to show.
export interface AudioDevice {
name: string; // PipeWire node.name — what `speaker_device` / `mic_device` store
description: string; // human label ("Steam Deck Speakers")
}
// What the device pickers need, read from the session binary (`--list-adapters` / `--list-audio`).
// `ok: false` = the session binary couldn't be run or failed; every list is then empty and the
// pickers stay on their stored value rather than pretending the device is gone.
export interface DeviceLists {
ok: boolean;
adapters: string[]; // Vulkan physical devices, discrete first
sinks: AudioDevice[]; // playback endpoints
sources: AudioDevice[]; // capture endpoints
}
export interface UpdateInfo {
@@ -185,6 +258,11 @@ export const getSettings = callable<[], StreamSettings>("get_settings");
export const setSettings = callable<[settings: StreamSettings], { ok: boolean }>(
"set_settings",
);
// GPUs + audio endpoints for the device pickers. Costs a subprocess that initialises Vulkan and
// PipeWire, so it is called ONCE when the settings tab mounts and never on the launch path.
export const listDevices = callable<[], DeviceLists>("list_devices");
// The same, bypassing the backend's cache — for the user who just plugged in a headset.
export const refreshDevices = callable<[], DeviceLists>("refresh_devices");
export const killStream = callable<[], { ok: boolean }>("kill_stream");
// Send a Wake-on-LAN magic packet to a saved host (headless flatpak --wake) so a sleeping host is
// up by the time the stream connects. The MAC is looked up from the flatpak client's own
+7 -1
View File
@@ -334,8 +334,14 @@ const HostsTab: FC<{
</div>
);
// NOT `tabScroll`: the settings screen is a SidebarNavigation, which lays out its own rail +
// content pane and scrolls the pane itself. Wrapping it in an outer scroll area would give it an
// indefinite height to fill, collapsing the rail — so this pane only hands it the full height and
// keeps its hands off the overflow. The footer inset lives inside the pages instead.
const settingsPane: CSSProperties = { height: "100%", overflow: "hidden" };
const SettingsTab: FC = () => (
<div style={tabScroll}>
<div style={settingsPane}>
<SettingsSection />
</div>
);
+608 -152
View File
@@ -1,10 +1,59 @@
// Stream settings — resolution / refresh / bitrate / gamepad / compositor / mic, written to
// the flatpak client's JSON (main.py set_settings), which the client reads on launch. The
// accepted gamepad/compositor names mirror punktfunk-core's `*Pref::from_name`.
import { Dropdown, Field, SliderField, Spinner, ToggleField } from "@decky/ui";
import { CSSProperties, FC, useEffect, useState } from "react";
import { getSettings, setSettings, StreamSettings } from "./backend";
import { RowActions } from "./ui";
// Stream settings — the client's WHOLE settings store, written to the JSON the client reads on
// launch (main.py set_settings, merged onto what's on disk). This is the same
// `client-gtk-settings.json` the desktop client and the console's settings screen own, so a value
// changed in any of the three shows in the other two.
//
// SHAPE OF THIS SCREEN. Thirty rows is too many to scroll past on a thumbstick, so they are split
// across a `SidebarNavigation` — the same left-rail-of-categories layout SteamOS's own Settings
// uses, and the one Deck users already know. Every page fits on screen without scrolling, which is
// the whole point of the split: the rail is the index, so nothing is more than one hop away.
//
// The categories, their order, and the wording of the rows are the console's settings screen
// (pf-console-ui/src/screens/settings.rs) — that screen is the other settings editor a user
// reaches without leaving Gaming Mode, and two different orders for one store is how people stop
// trusting either. It shows them as one steppable list because it has no pointer and no room for
// a rail; here they become the rail's pages, same groups, same sequence. Three more rules:
//
// • A setting that depends on another is INDENTED under it and DISABLED, never hidden — the
// console dims those rows rather than dropping them, and a row that vanishes as you toggle
// the one above it is a moving target for a thumbstick.
// • A picker whose options this device doesn't have doesn't appear at all (the GPU row on a
// one-GPU Deck). A dead control is worse than an absent one.
// • Anything that behaves differently *here* than it does on a desktop says so in its own
// description, rather than being silently dropped from the screen.
//
// The accepted gamepad/compositor/codec/decoder names mirror punktfunk-core's `*Pref::from_name`
// and the console's tables; the tier/mode names mirror the `StatsVerbosity` / `TouchMode` /
// `MouseMode` enums, which serialize lowercase.
import {
DialogButton,
Dropdown,
Field,
SidebarNavigation,
SliderField,
Spinner,
ToggleField,
} from "@decky/ui";
import { CSSProperties, FC, ReactElement, ReactNode, useEffect, useState } from "react";
import {
FaDesktop,
FaGamepad,
FaHandPointer,
FaSlidersH,
FaTv,
FaVideo,
FaVolumeUp,
} from "react-icons/fa";
import {
AudioDevice,
DeviceLists,
getSettings,
listDevices,
refreshDevices,
setSettings,
StreamSettings,
} from "./backend";
import { actionButton, RowActions } from "./ui";
// Decky's Dropdown has no width prop — it fills whatever container it's in, and a
// `childrenContainerWidth="max"` Field is the whole row. Wrapping it in this fit-content shell
@@ -17,50 +66,543 @@ const selectShell: CSSProperties = {
maxWidth: "24em",
};
// ----------------------------------------------------------------------------------------
// Option tables — the console's, so the two Gaming-Mode editors offer the same choices.
// ----------------------------------------------------------------------------------------
// "native" and "match" are virtual: they store `width`/`height` of 0 with `match_window` off/on.
// Match window is offered even though this plugin's launches are always fullscreen (where it
// degenerates to the display's native mode) — leaving it out would make the row lie about a
// store the desktop client can set it in.
const MATCH_WINDOW = "match";
const RESOLUTIONS: [number, number, string][] = [
[0, 0, "Native display"],
[1280, 720, "1280 × 720"],
[1280, 800, "1280 × 800 (Deck)"],
[1920, 1080, "1920 × 1080"],
[2560, 1440, "2560 × 1440"],
[3840, 2160, "3840 × 2160"],
];
const resolutionKey = (w: number, h: number): string => (w === 0 && h === 0 ? "native" : `${w}x${h}`);
const REFRESH = [0, 30, 60, 90, 120];
// Render-resolution multipliers (mirrors punktfunk_core::render_scale::PRESETS). 1.0 = native.
const RENDER_SCALES = [0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0];
const renderScaleLabel = (x: number): string =>
x === 1 ? "Native (1×)" : x > 1 ? `${x}× · supersample` : `${x}×`;
const GAMEPADS = ["auto", "xbox360", "xboxone", "dualsense", "dualshock4", "steamdeck"];
const GAMEPAD_LABELS: Record<string, string> = {
auto: "Automatic",
xbox360: "Xbox 360",
xboxone: "Xbox One",
dualsense: "DualSense",
dualshock4: "DualShock 4",
steamdeck: "Steam Deck",
const COMPOSITORS: [string, string][] = [
["auto", "Automatic"],
["kwin", "KDE Plasma (KWin)"],
["wlroots", "Sway (wlroots)"],
["mutter", "GNOME (Mutter)"],
["gamescope", "gamescope"],
];
const CODECS: [string, string][] = [
["auto", "Automatic"],
["hevc", "HEVC (H.265)"],
["h264", "H.264 (AVC)"],
["av1", "AV1"],
// Opt-in wired-LAN low-latency codec (100400 Mbit/s class, 8-bit SDR). Only ever selected
// when the host advertises it too; anything else falls back to HEVC.
["pyrowave", "PyroWave (wired LAN)"],
];
const DECODERS: [string, string][] = [
["auto", "Automatic"],
["vulkan", "Vulkan Video"],
["vaapi", "VAAPI"],
["software", "Software"],
];
// Presentation intent — the `present_priority` key shared with the Apple and Android clients, so
// one profile reads the same on every device.
const PRESENT_PRIORITIES: [string, string][] = [
["latency", "Lowest latency"],
["smooth", "Smoothness"],
];
// Smoothness buffer depth in frames; 0 = Automatic (resolves to 2).
const SMOOTH_BUFFERS: [number, string][] = [
[0, "Automatic"],
[1, "1 frame"],
[2, "2 frames"],
[3, "3 frames"],
];
const AUDIO_CHANNELS: [number, string][] = [
[2, "Stereo"],
[6, "5.1 surround"],
[8, "7.1 surround"],
];
const GAMEPADS: [string, string][] = [
["auto", "Automatic"],
["xbox360", "Xbox 360"],
["xboxone", "Xbox One"],
["dualsense", "DualSense"],
["dualshock4", "DualShock 4"],
["steamdeck", "Steam Deck"],
];
const TOUCH_MODES: [string, string][] = [
["trackpad", "Trackpad"],
["pointer", "Direct pointer"],
["touch", "Touch passthrough"],
];
const MOUSE_MODES: [string, string][] = [
["capture", "Capture (games)"],
["desktop", "Desktop (absolute)"],
];
const STATS_TIERS: [string, string][] = [
["off", "Off"],
["compact", "Compact"],
["normal", "Normal"],
["detailed", "Detailed"],
];
// ----------------------------------------------------------------------------------------
// Row primitives — every picker row is Field + right-aligned, content-sized Dropdown, so the
// twelve of them below stay one line each and can't drift apart.
// ----------------------------------------------------------------------------------------
const SelectRow = <T extends string | number>({
label,
description,
options,
value,
onChange,
formatUnknown,
disabled,
indent,
}: {
label: string;
description?: ReactNode;
options: [T, string][];
value: T;
onChange: (v: T) => void;
// How to name a stored value this table doesn't list (see below); defaults to the raw value.
formatUnknown?: (v: T) => string;
disabled?: boolean;
indent?: boolean;
}): ReactElement => {
// A Dropdown can only display a value that is one of its options, and this store has four other
// writers — the desktop client, the console, a settings profile, a newer client with presets
// this build doesn't know. Rather than render a blank control (or, worse, silently show a
// different value than the stream will actually use), carry the stored one as its own entry.
const shown: [T, string][] = options.some(([v]) => v === value)
? options
: [...options, [value, formatUnknown ? formatUnknown(value) : String(value)]];
return (
<Field
label={label}
description={description}
disabled={disabled}
indentLevel={indent ? 1 : undefined}
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
disabled={disabled}
rgOptions={shown.map(([data, l]) => ({ data, label: l }))}
selectedOption={value}
onChange={(o) => onChange(o.data as T)}
/>
</div>
</RowActions>
</Field>
);
};
// Mirrors the desktop client's picker (ui_settings.rs CODECS) — a soft preference the host
// falls back from when its GPU can't encode it.
const CODECS = ["auto", "hevc", "h264", "av1"];
const CODEC_LABELS: Record<string, string> = {
auto: "Automatic",
hevc: "HEVC (H.265)",
h264: "H.264 (AVC)",
av1: "AV1",
// An audio-endpoint picker. The stored value is a PipeWire `node.name`; "" means "whatever the OS
// is using". A stored endpoint that isn't in the current enumeration still gets an entry — it is
// a real preference that simply isn't plugged in right now, and dropping it would silently
// re-point the next stream at the default without ever showing the user why.
const DeviceRow: FC<{
label: string;
description: string;
devices: AudioDevice[] | null;
value: string;
onChange: (v: string) => void;
disabled?: boolean;
indent?: boolean;
}> = ({ label, description, devices, value, onChange, disabled, indent }) => {
const options: [string, string][] = [["", "System default"]];
for (const d of devices ?? []) options.push([d.name, d.description]);
if (value && !options.some(([name]) => name === value)) {
options.push([value, `${value} (not connected)`]);
}
return (
<SelectRow
label={label}
description={devices === null ? "Reading this device's audio endpoints…" : description}
options={options}
value={value}
onChange={onChange}
disabled={disabled || devices === null}
indent={indent}
/>
);
};
const COMPOSITORS = ["auto", "kwin", "wlroots", "mutter", "gamescope"];
const COMPOSITOR_LABELS: Record<string, string> = {
auto: "Automatic",
kwin: "KDE Plasma (KWin)",
wlroots: "Sway (wlroots)",
mutter: "GNOME (Mutter)",
gamescope: "gamescope",
// ----------------------------------------------------------------------------------------
// The pages. One settings object, seven views on it — every page takes the same context rather
// than fetching or holding state of its own, so a change on one page is visible on the others
// the moment you switch.
// ----------------------------------------------------------------------------------------
interface PageCtx {
s: StreamSettings;
patch: (p: Partial<StreamSettings>) => void;
devices: DeviceLists | null;
reading: boolean;
readDevices: (again: boolean) => void;
}
// SidebarNavigation gives each page Steam's own padding, but the routed page still renders
// UNDER Gaming Mode's footer hint bar, so the last row of a page needs to clear it (the same
// inset the tabs use).
const pageBody: CSSProperties = { paddingBottom: "80px" };
const StreamPage: FC<PageCtx> = ({ s, patch }) => {
const renderScale = s.render_scale ?? 1;
const resolution = s.match_window ? MATCH_WINDOW : resolutionKey(s.width, s.height);
return (
<div style={pageBody}>
<SelectRow
label="Resolution"
description="The host creates a virtual display at exactly this size — no scaling. Match window follows the stream window instead, which in Gaming Mode means the Deck's native size."
options={[
...RESOLUTIONS.map(([w, h, label]) => [resolutionKey(w, h), label] as [string, string]),
[MATCH_WINDOW, "Match window"] as [string, string],
]}
value={resolution}
// A size set from a desktop profile that isn't one of these presets, spelled the way the
// presets are rather than left as the raw "1600x900" key.
formatUnknown={(v) => v.replace("x", " × ")}
onChange={(v) => {
if (v === MATCH_WINDOW) {
// The tri-state the console stores: the flag on, the explicit size cleared.
patch({ match_window: true, width: 0, height: 0 });
return;
}
const found = RESOLUTIONS.find(([w, h]) => resolutionKey(w, h) === v);
patch({ match_window: false, width: found?.[0] ?? 0, height: found?.[1] ?? 0 });
}}
/>
<SelectRow
label="Refresh rate"
description="Native follows the display the stream is on."
options={REFRESH.map((r) => [r, r === 0 ? "Native" : `${r} Hz`] as [number, string])}
value={s.refresh_hz}
formatUnknown={(v) => `${v} Hz`}
onChange={(v) => patch({ refresh_hz: v })}
/>
<SelectRow
label="Render scale"
description="The host renders larger or smaller than the stream mode and the Deck resamples — above 1× supersamples for sharpness, below 1× saves bandwidth."
options={RENDER_SCALES.map((x) => [x, renderScaleLabel(x)] as [number, string])}
// Snap the stored value to the nearest preset so the dropdown always shows a match.
value={RENDER_SCALES.reduce((best, x) =>
Math.abs(x - renderScale) < Math.abs(best - renderScale) ? x : best,
)}
onChange={(v) => patch({ render_scale: v })}
/>
<SliderField
label="Bitrate"
description="0 = the host's own default (20 Mbit/s)."
value={Math.round(s.bitrate_kbps / 1000)}
min={0}
max={150}
step={5}
showValue
valueSuffix=" Mbit/s"
onChange={(v) => patch({ bitrate_kbps: v * 1000 })}
/>
<SelectRow
label="Host compositor"
description="Which compositor drives the virtual display — honoured only if it's available on the host. Automatic suits almost every host."
options={COMPOSITORS}
value={s.compositor}
onChange={(v) => patch({ compositor: v })}
/>
</div>
);
};
const VideoPage: FC<PageCtx> = ({ s, patch, devices }) => {
// Only worth a row on a box that actually has a choice to make. A Deck has one adapter, and a
// picker with a single option is a control that can't do anything.
const showGpuRow = (devices?.adapters.length ?? 0) > 1;
return (
<div style={pageBody}>
<SelectRow
label="Video codec"
description="A preference — the host falls back when its GPU can't encode this one."
options={CODECS}
value={s.codec ?? "auto"}
onChange={(v) => patch({ codec: v })}
/>
<SelectRow
label="Video decoder"
description="How the Deck decodes the stream. Automatic prefers Vulkan Video, then VAAPI, then software."
options={DECODERS}
value={s.decoder ?? "auto"}
onChange={(v) => patch({ decoder: v })}
/>
{showGpuRow && (
<SelectRow
label="Decode GPU"
description="Which adapter decodes and presents the stream. Automatic picks the discrete GPU where there is one."
options={[
["", "Automatic"],
...(devices?.adapters ?? []).map((a) => [a, a] as [string, string]),
]}
value={s.adapter ?? ""}
onChange={(v) => patch({ adapter: v })}
/>
)}
<ToggleField
label="10-bit HDR"
description="Advertise HDR10 so the host sends 10-bit when the content is HDR. Off means never ask for 10-bit."
checked={s.hdr_enabled ?? true}
onChange={(v) => patch({ hdr_enabled: v })}
/>
<ToggleField
label="Full chroma (4:4:4)"
description="Full-colour video: crisp small text and thin lines, at more bandwidth. Needs an NVIDIA host (NVENC) or the PyroWave codec — other encoders stream 4:2:0 and the session falls back silently."
checked={s.enable_444 ?? false}
onChange={(v) => patch({ enable_444: v })}
/>
</div>
);
};
const PresentationPage: FC<PageCtx> = ({ s, patch }) => {
const smooth = (s.present_priority ?? "latency") === "smooth";
return (
<div style={pageBody}>
<SelectRow
label="Prioritize"
description="What to optimise for when a decoded frame is ready. Lowest latency shows each frame the moment the display can take it — a network hiccup becomes an occasional repeated or skipped frame. Smoothness buffers a little to even those out."
options={PRESENT_PRIORITIES}
value={s.present_priority ?? "latency"}
onChange={(v) => patch({ present_priority: v })}
/>
<SelectRow
label="Smoothness buffer"
description="Frames held back before showing. Each one absorbs about a refresh of network hiccup and adds a refresh of delay. Automatic holds two."
options={SMOOTH_BUFFERS}
value={s.smooth_buffer ?? 0}
formatUnknown={(v) => `${v} frames`}
onChange={(v) => patch({ smooth_buffer: v })}
disabled={!smooth}
indent
/>
<ToggleField
label="V-Sync"
description="Tear-free. Off removes the wait for the screen's refresh — the lowest possible delay, at the cost of visible tearing. Best-effort: not every driver offers it, and the Detailed stats overlay names the mode actually in use."
checked={s.vsync ?? true}
onChange={(v) => patch({ vsync: v })}
/>
<ToggleField
label="Follow variable refresh"
description="On a VRR screen, let the panel refresh in step with the stream instead of on a fixed cadence. Applies to fullscreen sessions — which a Gaming-Mode stream always is — and is harmless on a fixed-refresh screen."
checked={s.allow_vrr ?? true}
onChange={(v) => patch({ allow_vrr: v })}
/>
</div>
);
};
const AudioPage: FC<PageCtx> = ({ s, patch, devices, reading, readDevices }) => {
const micOn = s.mic_enabled;
// What the pickers get: null while the enumeration is in flight (they show a loading state),
// [] when it answered but couldn't read the endpoints (System default plus whatever is
// stored), and the real list otherwise.
const endpoints = (list: AudioDevice[] | undefined): AudioDevice[] | null =>
reading || !devices ? null : devices.ok ? (list ?? []) : [];
return (
<div style={pageBody}>
<SelectRow
label="Audio channels"
description="The speaker layout requested from the host, which clamps it to what it can capture."
options={AUDIO_CHANNELS}
value={s.audio_channels ?? 2}
formatUnknown={(v) => `${v} channels`}
onChange={(v) => patch({ audio_channels: v })}
/>
<DeviceRow
label="Output device"
description="Where stream audio plays. System default follows whatever the Deck is using, including a headset you plug in mid-stream."
devices={endpoints(devices?.sinks)}
value={s.speaker_device ?? ""}
onChange={(v) => patch({ speaker_device: v })}
/>
<ToggleField
label="Stream microphone"
description="Send the Deck's microphone to the host's virtual mic. Ctrl+Alt+Shift+V mutes and unmutes it mid-stream."
checked={micOn}
onChange={(v) => patch({ mic_enabled: v })}
/>
<DeviceRow
label="Microphone device"
description="Which input the mic uplink captures from."
devices={endpoints(devices?.sources)}
value={s.mic_device ?? ""}
onChange={(v) => patch({ mic_device: v })}
disabled={!micOn}
indent
/>
<ToggleField
label="Echo cancellation"
description="Stops the host's audio, playing from the Deck's speakers, being picked up and sent back. Turn it off if your microphone already runs its own processing."
checked={s.echo_cancel ?? true}
onChange={(v) => patch({ echo_cancel: v })}
disabled={!micOn}
indentLevel={1}
/>
{/* The escape hatch for a headset plugged in after this page was opened, and the honest
answer when the enumeration failed outright (a client too old to ship the session
binary). Rendered unconditionally, including while it is reading: a row that comes and
goes under a thumbstick is a moving target, so only its wording changes. */}
<Field
label={
!reading && devices && !devices.ok ? "Couldn't read this device's hardware" : "Devices"
}
description={
reading
? "Reading this device's audio endpoints and GPUs…"
: devices && !devices.ok
? "The output, microphone and GPU pickers fall back to Automatic. Reading them needs the client's session binary, which a client older than the two-binary split doesn't ship — update it from the About tab."
: "Plugged something in just now? Read the audio endpoints and GPUs again."
}
childrenContainerWidth="max"
>
<RowActions>
<DialogButton style={actionButton} disabled={reading} onClick={() => readDevices(true)}>
{reading ? <Spinner style={{ height: "1em" }} /> : "Refresh"}
</DialogButton>
</RowActions>
</Field>
</div>
);
};
const ControllersPage: FC<PageCtx> = ({ s, patch }) => {
const forwarding = s.gamepad_forwarding ?? true;
return (
<div style={pageBody}>
<ToggleField
label="Forward controllers"
description="Send controllers connected to the Deck to the host. Turn it off when your controller already reaches the host another way — USB passthrough such as VirtualHere, or a pad plugged into the host — so games don't see two of them."
checked={forwarding}
onChange={(v) => patch({ gamepad_forwarding: v })}
/>
<SelectRow
label="Controller type"
description="The virtual pad the host creates. Automatic matches the controller you're holding."
options={GAMEPADS}
value={s.gamepad}
onChange={(v) => patch({ gamepad: v })}
disabled={!forwarding}
indent
/>
{forwarding && (s.gamepad === "steamdeck" || s.gamepad === "auto") && (
<Field
label="⚠ Disable Steam Input"
description="On a Deck, Automatic forwards the built-in controller as a Steam Deck pad — paddles, both trackpads, and gyro included. For that, Steam Input must be OFF for Punktfunk: on the game page tap ⚙ → Controller Settings → set Steam Input to Off. Otherwise Steam keeps the Deck's controls and only the sticks + buttons reach the host."
indentLevel={1}
/>
)}
</div>
);
};
const PointerPage: FC<PageCtx> = ({ s, patch }) => (
<div style={pageBody}>
<SelectRow
label="Touch mode"
description="How the touchscreen drives the host: Trackpad (relative cursor, tap to click), Direct pointer (the cursor jumps to your finger), or Touch passthrough (every finger is a host contact — only helps apps that understand touch)."
options={TOUCH_MODES}
value={s.touch_mode ?? "trackpad"}
onChange={(v) => patch({ touch_mode: v })}
/>
<SelectRow
label="Mouse mode"
description="How a physical mouse drives the host: Capture locks the pointer for games, Desktop leaves it free and sends absolute positions. Ctrl+Alt+Shift+M switches it live mid-stream."
options={MOUSE_MODES}
value={s.mouse_mode ?? "capture"}
onChange={(v) => patch({ mouse_mode: v })}
/>
<ToggleField
label="Invert scroll direction"
description="Reverses the wheel and trackpad scroll direction sent to the host."
checked={s.invert_scroll ?? false}
onChange={(v) => patch({ invert_scroll: v })}
/>
<ToggleField
label="Capture system shortcuts"
description="Sends Alt+Tab, Super and friends to the host while input is captured, instead of leaving them to the local desktop. Gaming Mode is gamescope, which has no shortcuts to hold back — this is for a keyboard attached to the Deck in Desktop Mode, and for the desktop client sharing these settings."
checked={s.inhibit_shortcuts}
onChange={(v) => patch({ inhibit_shortcuts: v })}
/>
</div>
);
const InterfacePage: FC<PageCtx> = ({ s, patch }) => {
// `Settings::stats_verbosity`: no tier = a pre-tier store, resolved through the legacy bool,
// which itself defaults to true.
const statsTier = s.stats_verbosity ?? ((s.show_stats ?? true) ? "normal" : "off");
return (
<div style={pageBody}>
<SelectRow
label="Statistics overlay"
description="How much the in-stream overlay shows: Compact (fps · latency · bitrate on one line) → Normal → Detailed. A three-finger tap on the touchscreen cycles it mid-stream."
options={STATS_TIERS}
value={statsTier}
// Both keys, in sync — the same pairing `Settings::set_stats_verbosity` keeps, so a
// client too old for the tiers still honours an Off chosen here.
onChange={(v) => patch({ stats_verbosity: v, show_stats: v !== "off" })}
/>
<ToggleField
label="Wake hosts automatically"
description="Send Wake-on-LAN to a sleeping host before connecting and wait for it to boot. Turn it off for hosts reached over a VPN, where an offline-looking host is really just unreachable by broadcast and the wait only adds delay."
checked={s.auto_wake ?? true}
onChange={(v) => patch({ auto_wake: v })}
/>
<ToggleField
label="Show game library in the client"
description="Lets the client's own host cards browse a paired host's games. This plugin's library browser works either way — this is for the client's screens."
checked={s.library_enabled ?? false}
onChange={(v) => patch({ library_enabled: v })}
/>
<ToggleField
label="Start streams fullscreen"
description="Streams open fullscreen instead of windowed. Launches from this plugin are always fullscreen whatever this says — it's here because the desktop client reads the same settings."
checked={s.fullscreen_on_stream ?? true}
onChange={(v) => patch({ fullscreen_on_stream: v })}
/>
</div>
);
};
// ----------------------------------------------------------------------------------------
export const SettingsSection: FC = () => {
const [s, setS] = useState<StreamSettings | null>(null);
// null until the enumeration answers — the pickers show a loading state rather than briefly
// claiming this device has no endpoints.
const [devices, setDevices] = useState<DeviceLists | null>(null);
const [reading, setReading] = useState(true);
const readDevices = (again: boolean) => {
setReading(true);
void (again ? refreshDevices() : listDevices())
.then(setDevices)
.finally(() => setReading(false));
};
useEffect(() => {
void getSettings().then(setS);
// Deliberately not awaited together with the settings: a cold flatpak initialising Vulkan
// takes seconds, and the rest of the screen must not wait for it.
readDevices(false);
}, []);
const patch = (p: Partial<StreamSettings>) => {
@@ -74,128 +616,42 @@ export const SettingsSection: FC = () => {
if (!s) return <Spinner style={{ height: "1.5em" }} />;
const resIdx = Math.max(
0,
RESOLUTIONS.findIndex(([w, h]) => w === s.width && h === s.height),
);
const ctx: PageCtx = { s, patch, devices, reading, readDevices };
return (
<>
<Field
label="Resolution"
description="The host creates a virtual output at exactly this size"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={RESOLUTIONS.map(([, , label], i) => ({ data: i, label }))}
selectedOption={resIdx}
onChange={(o) => {
const [w, h] = RESOLUTIONS[o.data as number];
patch({ width: w, height: h });
}}
/>
</div>
</RowActions>
</Field>
<Field label="Refresh rate" childrenContainerWidth="max">
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={REFRESH.map((r) => ({ data: r, label: r === 0 ? "Native" : `${r} Hz` }))}
selectedOption={s.refresh_hz}
onChange={(o) => patch({ refresh_hz: o.data as number })}
/>
</div>
</RowActions>
</Field>
<Field
label="Render scale"
description="Supersample for sharpness (> 1×, more bandwidth) or render below native (< 1×) — the Deck resamples to its screen"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={RENDER_SCALES.map((x) => ({ data: x, label: renderScaleLabel(x) }))}
// Snap the stored value to the nearest preset so the dropdown always shows a match.
selectedOption={RENDER_SCALES.reduce((best, x) =>
Math.abs(x - (s.render_scale ?? 1)) < Math.abs(best - (s.render_scale ?? 1)) ? x : best,
)}
onChange={(o) => patch({ render_scale: o.data as number })}
/>
</div>
</RowActions>
</Field>
<SliderField
label="Bitrate"
description="Mbit/s · 0 = host default"
value={Math.round(s.bitrate_kbps / 1000)}
min={0}
max={150}
step={5}
showValue
valueSuffix=" Mbit/s"
onChange={(v) => patch({ bitrate_kbps: v * 1000 })}
/>
<Field
label="Video codec"
description="Preferred stream codec — the host falls back when its GPU can't encode it"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={CODECS.map((c) => ({ data: c, label: CODEC_LABELS[c] ?? c }))}
selectedOption={s.codec ?? "auto"}
onChange={(o) => patch({ codec: o.data as string })}
/>
</div>
</RowActions>
</Field>
<Field
label="Gamepad type"
description="Which virtual controller the host creates for your inputs"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={GAMEPADS.map((g) => ({ data: g, label: GAMEPAD_LABELS[g] ?? g }))}
selectedOption={s.gamepad}
onChange={(o) => patch({ gamepad: o.data as string })}
/>
</div>
</RowActions>
</Field>
{(s.gamepad === "steamdeck" || s.gamepad === "auto") && (
<Field
label="⚠ Disable Steam Input"
description="On a Deck, Automatic forwards the built-in controller as a Steam Deck pad — paddles, both trackpads, and gyro included. For that, Steam Input must be OFF for Punktfunk: on the game page tap ⚙ → Controller Settings → set Steam Input to Off. Otherwise Steam keeps the Deck's controls and only the sticks + buttons reach the host."
/>
)}
<Field
label="Host compositor"
description="Which compositor backend the host uses for the virtual display — Automatic suits almost every host"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={COMPOSITORS.map((c) => ({ data: c, label: COMPOSITOR_LABELS[c] ?? c }))}
selectedOption={s.compositor}
onChange={(o) => patch({ compositor: o.data as string })}
/>
</div>
</RowActions>
</Field>
<ToggleField
label="Stream microphone"
description="Send the Deck's microphone to the host's virtual mic"
checked={s.mic_enabled}
onChange={(v) => patch({ mic_enabled: v })}
/>
</>
<SidebarNavigation
// We are already inside the plugin's own `/punktfunk` route, rendered in a tab. Route
// reporting would have this nav push entries of its own onto the router and fight the
// page for the back gesture; the pages are addressed by `identifier` instead.
disableRouteReporting
pages={[
{ title: "Stream", identifier: "stream", icon: <FaDesktop />, content: <StreamPage {...ctx} /> },
{ title: "Video", identifier: "video", icon: <FaVideo />, content: <VideoPage {...ctx} /> },
{
title: "Presentation",
identifier: "presentation",
icon: <FaTv />,
content: <PresentationPage {...ctx} />,
},
{ title: "Audio", identifier: "audio", icon: <FaVolumeUp />, content: <AudioPage {...ctx} /> },
{
title: "Controllers",
identifier: "controllers",
icon: <FaGamepad />,
content: <ControllersPage {...ctx} />,
},
{
title: "Touch & mouse",
identifier: "pointer",
icon: <FaHandPointer />,
content: <PointerPage {...ctx} />,
},
{
title: "Interface",
identifier: "interface",
icon: <FaSlidersH />,
content: <InterfacePage {...ctx} />,
},
]}
/>
);
};
+14 -3
View File
@@ -749,7 +749,12 @@ impl AppModel {
dialog.connect_response(Some("apply"), move |_, _| {
let where_to = match &target {
SpeedTestTarget::Global => {
// Rebase on the file before the whole-file save (same
// discipline as the settings dialog): another writer — the
// spawner's window-size persist, a second window's dialog —
// may have moved it under this shell's snapshot.
let mut s = settings.borrow_mut();
*s = Settings::load();
s.bitrate_kbps = recommended_kbps;
s.save();
"the default bitrate".to_string()
@@ -765,7 +770,9 @@ impl AppModel {
});
}
dialog.connect_response(Some("apply-global"), move |_, _| {
// Rebase on the file first — see the Global arm above.
let mut s = settings.borrow_mut();
*s = Settings::load();
s.bitrate_kbps = recommended_kbps;
s.save();
toasts.add_toast(adw::Toast::new(&format!(
@@ -796,9 +803,7 @@ impl SpeedTestTarget {
// Resolved exactly the way a connect resolves it: the one-off pick this test was
// started with (a pinned card carries one), else the host's binding.
let bound = trust::KnownHosts::load()
.hosts
.iter()
.find(|h| h.addr == req.addr && h.port == req.port)
.find_by_addr(&req.addr, req.port)
.and_then(|h| h.profile_id.clone());
let reference = match req.profile.as_deref() {
Some("") => return SpeedTestTarget::Global,
@@ -1014,6 +1019,12 @@ pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow
<property name="accelerator">&lt;Control&gt;&lt;Alt&gt;&lt;Shift&gt;s</property>
</object>
</child>
<child>
<object class="GtkShortcutsShortcut">
<property name="title">Mute or unmute your microphone (only while the stream sends one)</property>
<property name="accelerator">&lt;Control&gt;&lt;Alt&gt;&lt;Shift&gt;v</property>
</object>
</child>
</object>
</child>
</object>
+21 -11
View File
@@ -164,9 +164,7 @@ pub fn cli_wake() -> glib::ExitCode {
let (addr, port) = parse_host_port(&target);
let port = port.unwrap_or(9777);
let mac = crate::trust::KnownHosts::load()
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port)
.find_by_addr(&addr, port)
.map(|h| h.mac.clone())
.unwrap_or_default();
if mac.is_empty() {
@@ -201,9 +199,7 @@ pub fn headless_library(target: &str) -> glib::ExitCode {
.and_then(crate::trust::parse_hex32)
.or_else(|| {
crate::trust::KnownHosts::load()
.hosts
.iter()
.find(|h| h.addr == addr)
.find_by_addr(&addr, port)
.and_then(|h| crate::trust::parse_hex32(&h.fp_hex))
});
match crate::library::fetch_games(&addr, port, &identity, pin) {
@@ -343,9 +339,8 @@ pub fn headless_add_host(target: &str) -> glib::ExitCode {
// No fingerprint yet — an address-keyed placeholder. Refresh the name if it already exists.
let mut known = KnownHosts::load();
if let Some(h) = known
.hosts
.iter_mut()
.find(|h| h.addr == addr && h.port == port)
.index_by_addr(&addr, port)
.and_then(|i| known.hosts.get_mut(i))
{
h.name = name;
} else {
@@ -485,8 +480,23 @@ fn headless_check_update() -> glib::ExitCode {
"installed {} ({}, {})",
status.current, status.kind, status.channel
);
println!("available {}", status.latest);
if let Some(err) = &status.error {
// `latest` falls back to `current` when the check couldn't run — printing that as
// "available" would read as a confirmed answer we don't have.
if status.error.is_some() {
println!("available unknown");
} else {
println!("available {}", status.latest);
}
if status.not_published {
// Says what it is, in words, instead of a raw HTTP status. The exit code still
// reports "could not tell" (see the doc comment above): an empty channel is the
// absence of evidence that this build is current, and a mistyped
// PUNKTFUNK_UPDATE_FEED is indistinguishable from one out here.
println!(
"update nothing published on the {} channel yet",
status.channel
);
} else if let Some(err) = &status.error {
eprintln!("check-update: {err}");
} else if status.update_available {
println!("update yes");
+199 -2
View File
@@ -156,6 +156,20 @@ mod index {
pub fn gamepad(s: &Settings) -> u32 {
GAMEPADS.iter().position(|&g| g == s.gamepad).unwrap_or(0) as u32
}
pub fn present_priority(s: &Settings) -> u32 {
// Unknown values (a newer client's intent) read as the default, exactly as
// `PresentPriority::resolve` treats them.
PRESENT_PRIORITIES
.iter()
.position(|&p| p == s.present_priority)
.unwrap_or(0) as u32
}
pub fn smooth_buffer(s: &Settings) -> u32 {
// The index IS the stored value: 0 = Automatic, 1..3 = frames.
u32::from(s.smooth_buffer).min(SMOOTH_BUFFER_LABELS.len() as u32 - 1)
}
}
/// The chip palette a profile can carry (`StreamProfile.accent`). Eight entries rather than a
@@ -607,6 +621,9 @@ fn commit_profile(active: &StreamProfile, touched: &Touched, values: &Settings)
if touched.has("mic_enabled") {
o.mic_enabled = Some(values.mic_enabled);
}
if touched.has("echo_cancel") {
o.echo_cancel = Some(values.echo_cancel);
}
if touched.has("touch_mode") {
o.touch_mode = Some(values.touch_mode.clone());
}
@@ -622,12 +639,27 @@ fn commit_profile(active: &StreamProfile, touched: &Touched, values: &Settings)
if touched.has("gamepad") {
o.gamepad = Some(values.gamepad.clone());
}
if touched.has("gamepad_forwarding") {
o.gamepad_forwarding = Some(values.gamepad_forwarding);
}
if touched.has("stats_verbosity") {
o.stats_verbosity = Some(values.stats_verbosity());
}
if touched.has("fullscreen_on_stream") {
o.fullscreen_on_stream = Some(values.fullscreen_on_stream);
}
if touched.has("present_priority") {
o.present_priority = Some(values.present_priority.clone());
}
if touched.has("smooth_buffer") {
o.smooth_buffer = Some(values.smooth_buffer);
}
if touched.has("vsync") {
o.vsync = Some(values.vsync);
}
if touched.has("allow_vrr") {
o.allow_vrr = Some(values.allow_vrr);
}
// Resets are not handled here: they clear the field and re-seed their row the moment the
// user asks, so by the time this runs the catalog already reflects them and the row is no
// longer marked touched.
@@ -681,6 +713,20 @@ const TOUCH_MODE_CAPTIONS: &[&str] = &[
"The cursor jumps to your finger — a tap clicks there",
"Real multi-touch reaches the host — for touch-native apps",
];
/// Presentation-intent values (persisted under the `present_priority` key the Apple and
/// Android clients share) + labels + dynamic captions. Captions stay ONE line, like the
/// touch/mouse rows.
const PRESENT_PRIORITIES: &[&str] = &["latency", "smooth"];
const PRESENT_PRIORITY_LABELS: &[&str] = &["Lowest latency", "Smoothness"];
const PRESENT_PRIORITY_CAPTIONS: &[&str] = &[
"Each frame shows the moment the display can take it",
"Buffers a little to even out network hiccups",
];
/// Smoothness buffer depth, in frames — the index IS the stored `smooth_buffer` value
/// (0 = Automatic, which resolves to 2). No millisecond hints: the cost is one refresh
/// per frame, and the session's refresh isn't known here when the mode is Native.
const SMOOTH_BUFFER_LABELS: &[&str] = &["Automatic", "1 frame", "2 frames", "3 frames"];
/// Physical-mouse model values (persisted) + labels + dynamic captions — same idiom as
/// the touch rows. Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[&str] = &["capture", "desktop"];
@@ -1210,6 +1256,50 @@ pub fn show_scoped(
row
});
// ---- Display: Presentation ----
// The intent pair the Apple and Android clients already carry. The buffer row only
// means anything under Smoothness, so it hides itself the rest of the time rather
// than sitting there inert.
let present_row = ChoiceRow::new(
&dialog,
inline,
"Prioritize",
PRESENT_PRIORITY_CAPTIONS[0],
PRESENT_PRIORITY_LABELS,
);
let buffer_row = ChoiceRow::new(
&dialog,
inline,
"Smoothness buffer",
"Each frame held absorbs one refresh of hiccup and adds one of delay",
SMOOTH_BUFFER_LABELS,
);
{
let w = present_row.widget().clone();
let buffer = buffer_row.widget().clone();
present_row.connect_changed(move |i| {
let i = (i as usize).min(PRESENT_PRIORITY_CAPTIONS.len() - 1);
set_row_subtitle(&w, PRESENT_PRIORITY_CAPTIONS[i]);
buffer.set_visible(PRESENT_PRIORITIES[i] == "smooth");
});
}
let vsync_row = adw::SwitchRow::builder()
.title("V-Sync")
.subtitle(
"Tear-free. Turning it off removes the wait for the screen's refresh — the \
lowest possible delay, at the cost of visible tearing. Not every driver \
offers it; the stats overlay names the mode actually in use",
)
.build();
let vrr_row = adw::SwitchRow::builder()
.title("Follow variable refresh rate")
.subtitle(
"On a VRR/FreeSync/G-Sync screen, let the panel refresh in step with the \
stream instead of on a fixed cadence. Applies to fullscreen sessions; \
harmless on a fixed-refresh screen",
)
.build();
// ---- Display: Host output ----
let compositor_row = ChoiceRow::new(
&dialog,
@@ -1301,7 +1391,11 @@ pub fn show_scoped(
);
let mic_row = adw::SwitchRow::builder()
.title("Stream microphone")
.subtitle("Sends your microphone to the host's virtual mic")
.subtitle("Sends your microphone to the host's virtual mic — Ctrl+Alt+Shift+V mutes it mid-stream")
.build();
let echo_row = adw::SwitchRow::builder()
.title("Echo cancellation")
.subtitle("Keeps the host's audio, playing from this machine's speakers, out of the uplink")
.build();
// Endpoint pickers (from the PipeWire probe): visible labels are descriptions, the
// stored value is the node name. Hidden when the probe found nothing; a saved
@@ -1345,12 +1439,23 @@ pub fn show_scoped(
"Microphone",
"The input that feeds the host's virtual mic",
);
// The device pick only matters while the mic streams at all.
// The device pick and the echo canceller only matter while the mic streams at all — both
// follow it. One handler each (the pickers are optional, the echo row never is), and the
// initial state is set here because the seed block further down fires these too.
//
// Insensitivity covers the whole row, including the per-row Reset a profile scope adds:
// an echo_cancel override can only be reset while the mic row is on. Turn it on, reset,
// turn it back off — the alternative is a control that looks live and isn't.
if let Some(r) = &micdev_row {
let w = r.widget().clone();
w.set_sensitive(mic_row.is_active());
mic_row.connect_active_notify(move |m| w.set_sensitive(m.is_active()));
}
{
let w = echo_row.clone();
w.set_sensitive(mic_row.is_active());
mic_row.connect_active_notify(move |m| w.set_sensitive(m.is_active()));
}
// ---- Controllers ----
// Controller forwarding: Automatic forwards EVERY real controller, each as its own pad
@@ -1358,6 +1463,17 @@ pub fn show_scoped(
// controller (single-player). The pin is persisted by stable key (`Settings::forward_pad`),
// so it survives restarts — and disconnects: an offline pinned pad keeps its entry here
// instead of silently snapping back to Automatic.
// Off = this device's controllers are not sent at all, because they reach the host
// another way (USB passthrough such as VirtualHere, or a pad plugged into the host).
// It also stops the session OPENING the pad, which is what frees the device for a
// passthrough tool to bind — so the two rows below have nothing to act on while it is
// off, and are desensitised to say so.
let pad_forward_row = adw::SwitchRow::builder()
.title("Forward controllers")
.subtitle(
"Send this device's controllers to the host — off if it already has them another way",
)
.build();
let pads = gamepads.pads();
let saved_pin = settings.borrow().forward_pad.clone();
let mut pad_names = vec!["Automatic (all controllers)".to_string()];
@@ -1426,6 +1542,18 @@ pub fn show_scoped(
"Steam Deck",
],
);
// Both pad rows only mean something while something is being forwarded (the same
// relationship mic → echo cancellation draws just above, initial state included: the
// seed's `set_active` fires this only when it CHANGES the switch).
{
let (f, t) = (forward_row.widget().clone(), pad_row.widget().clone());
f.set_sensitive(seed.gamepad_forwarding);
t.set_sensitive(seed.gamepad_forwarding);
pad_forward_row.connect_active_notify(move |r| {
f.set_sensitive(r.is_active());
t.set_sensitive(r.is_active());
});
}
// ---- Seed from the effective settings for this scope ----
{
@@ -1436,6 +1564,7 @@ pub fn show_scoped(
hz_row.set_selected(index::refresh(s));
scale_row.set_selected(index::render_scale(s));
bitrate_row.set_value(f64::from(s.bitrate_kbps) / 1000.0);
pad_forward_row.set_active(s.gamepad_forwarding);
pad_row.set_selected(index::gamepad(s));
let touch_i = index::touch(s);
touch_row.set_selected(touch_i);
@@ -1453,6 +1582,7 @@ pub fn show_scoped(
inhibit_row.set_active(s.inhibit_shortcuts);
invert_row.set_active(s.invert_scroll);
mic_row.set_active(s.mic_enabled);
echo_row.set_active(s.echo_cancel);
hdr_row.set_active(s.hdr_enabled);
chroma_row.set_active(s.enable_444);
library_row.set_active(s.library_enabled);
@@ -1460,6 +1590,19 @@ pub fn show_scoped(
let codec_i = index::codec(s);
codec_row.set_selected(codec_i);
set_row_subtitle(codec_row.widget(), codec_caption(codec_i));
let present_i = index::present_priority(s);
present_row.set_selected(present_i);
set_row_subtitle(
present_row.widget(),
PRESENT_PRIORITY_CAPTIONS[present_i as usize],
);
buffer_row.set_selected(index::smooth_buffer(s));
// `set_selected` never fires the changed hook, so mirror its visibility rule here.
buffer_row
.widget()
.set_visible(PRESENT_PRIORITIES[present_i as usize] == "smooth");
vsync_row.set_active(s.vsync);
vrr_row.set_active(s.allow_vrr);
}
// ---- Override markers, per-row reset, and the touch that creates an override ----
@@ -1652,6 +1795,26 @@ pub fn show_scoped(
index::surround
);
choice!(pad_row, "gamepad", o.gamepad.is_some(), index::gamepad);
toggle!(
pad_forward_row,
"gamepad_forwarding",
o.gamepad_forwarding.is_some(),
gamepad_forwarding
);
choice!(
present_row,
"present_priority",
o.present_priority.is_some(),
index::present_priority
);
choice!(
buffer_row,
"smooth_buffer",
o.smooth_buffer.is_some(),
index::smooth_buffer
);
toggle!(vsync_row, "vsync", o.vsync.is_some(), vsync);
toggle!(vrr_row, "allow_vrr", o.allow_vrr.is_some(), allow_vrr);
toggle!(hdr_row, "hdr_enabled", o.hdr_enabled.is_some(), hdr_enabled);
toggle!(chroma_row, "enable_444", o.enable_444.is_some(), enable_444);
toggle!(
@@ -1673,6 +1836,12 @@ pub fn show_scoped(
invert_scroll
);
toggle!(mic_row, "mic_enabled", o.mic_enabled.is_some(), mic_enabled);
toggle!(
echo_row,
"echo_cancel",
o.echo_cancel.is_some(),
echo_cancel
);
{
let revert = {
let (row, globals, touched) =
@@ -1750,6 +1919,11 @@ pub fn show_scoped(
if let (Some(r), false) = (&gpu_row, profile_mode) {
quality_group.add(r.widget());
}
let presentation_group = group("Presentation", "");
presentation_group.add(present_row.widget());
presentation_group.add(buffer_row.widget());
presentation_group.add(&vsync_row);
presentation_group.add(&vrr_row);
// The one form-level note (deliberately not repeated on every row).
let output_group = group(
"Host output",
@@ -1758,6 +1932,7 @@ pub fn show_scoped(
output_group.add(compositor_row.widget());
display.add(&resolution_group);
display.add(&quality_group);
display.add(&presentation_group);
display.add(&output_group);
let input = page("Input", "input-keyboard-symbolic");
@@ -1781,6 +1956,7 @@ pub fn show_scoped(
audio_group.add(r.widget());
}
audio_group.add(&mic_row);
audio_group.add(&echo_row);
if let (Some(r), false) = (&micdev_row, profile_mode) {
audio_group.add(r.widget());
}
@@ -1817,6 +1993,10 @@ pub fn show_scoped(
controllers_group.add(&row);
}
}
// Profileable, so it shows in both scopes — unlike the pin below it, which is about
// which of THIS device's pads goes first: a "Work" profile can decline to forward
// controllers to a host that a "Game" profile forwards them to.
controllers_group.add(&pad_forward_row);
if !profile_mode {
controllers_group.add(forward_row.widget());
}
@@ -1889,7 +2069,9 @@ pub fn show_scoped(
s.auto_wake = wake_row.is_active();
s.inhibit_shortcuts = inhibit_row.is_active();
s.invert_scroll = invert_row.is_active();
s.gamepad_forwarding = pad_forward_row.is_active();
s.mic_enabled = mic_row.is_active();
s.echo_cancel = echo_row.is_active();
s.hdr_enabled = hdr_row.is_active();
s.enable_444 = chroma_row.is_active();
s.audio_channels = match surround_row.selected() {
@@ -1898,6 +2080,14 @@ pub fn show_scoped(
_ => 2,
};
s.codec = CODECS[(codec_row.selected() as usize).min(CODECS.len() - 1)].to_string();
s.present_priority = PRESENT_PRIORITIES
[(present_row.selected() as usize).min(PRESENT_PRIORITIES.len() - 1)]
.to_string();
// The index IS the value (0 = Automatic).
s.smooth_buffer =
(buffer_row.selected() as u8).min(SMOOTH_BUFFER_LABELS.len() as u8 - 1);
s.vsync = vsync_row.is_active();
s.allow_vrr = vrr_row.is_active();
s.library_enabled = library_row.is_active();
};
@@ -1910,7 +2100,14 @@ pub fn show_scoped(
commit_profile(active, &touched, &values);
}
None => {
// Rebase on the file, not the shell's start-of-app snapshot: the settings file
// has other whole-file writers (the spawner persists `last_window_w/h` after a
// match-window resize — see `profiles.rs` on why there's no merge), and saving
// the stale snapshot here would silently revert whatever they stored while this
// app was open. The rows carry every value this dialog owns, so applying them
// onto a fresh load loses nothing.
let mut s = settings.borrow_mut();
*s = Settings::load();
apply_rows(&mut s);
s.save();
}
+2 -1
View File
@@ -17,7 +17,8 @@ default = ["ui", "pyrowave"]
# PyroWave client decode (the wired-LAN wavelet codec) — enables the decode backend + the
# planar present path. ON by default; each session still opts in explicitly (the Settings
# codec pick, or PUNKTFUNK_PREFER_PYROWAVE=1). The Windows ARM64 leg builds
# --no-default-features and so skips it (video decode is Linux-only anyway).
# --no-default-features and so skips it — note that also drops `ui` (the Skia console/OSD),
# not just pyrowave; hardware video decode itself (D3D11VA / Vulkan Video) is unaffected.
pyrowave = ["pf-client-core/pyrowave", "pf-presenter/pyrowave"]
# The Skia console UI (stats OSD, capture HUD, later the gamepad library). Dropping it
# (`--no-default-features`) is the ~15 MB-smaller power-user build: same streaming,
+2 -1
View File
@@ -61,6 +61,7 @@ default ≈1000 nits). The host still gates the upgrade behind its `PUNKTFUNK_10
policy.
Debug/bisect knobs: `PUNKTFUNK_DECODER=vulkan|vaapi|d3d11va|software`, `PUNKTFUNK_PRESENT_MODE=
mailbox|immediate` (default FIFO), `PUNKTFUNK_VK_DEVICE=<index>` (multi-GPU), and
mailbox|fifo|immediate|fifo_relaxed` (default MAILBOX, FIFO where the surface offers no
MAILBOX — AMD on Windows), `PUNKTFUNK_VK_DEVICE=<index>` (multi-GPU), and
`PUNKTFUNK_HW_FAULT=import` (fault every VAAPI dmabuf import — proves the three-strike
demotion to software on healthy hardware).
+7 -3
View File
@@ -169,6 +169,11 @@ pub fn run(target: Option<&str>) -> u8 {
mouse_mode: settings_at_start.mouse_mode(),
invert_scroll: settings_at_start.invert_scroll,
inhibit_shortcuts: settings_at_start.inhibit_shortcuts,
// Presentation-tier like the rows above: latched at console start, a per-host
// profile cannot move it in this mode (the documented P4 gap).
present_priority: settings_at_start.present_priority(),
vsync: settings_at_start.vsync,
allow_vrr: settings_at_start.allow_vrr,
json_status,
on_connected: Some(Box::new(move |fingerprint: [u8; 32]| {
let fp_hex = trust::hex(&fingerprint);
@@ -448,9 +453,8 @@ impl ServiceState {
// Manual entries have no fingerprint yet, so `upsert` (fp-keyed) would
// collide two of them — key manual saves by address instead.
if let Some(h) = known
.hosts
.iter_mut()
.find(|h| h.addr == addr && h.port == port)
.index_by_addr(&addr, port)
.and_then(|i| known.hosts.get_mut(i))
{
if !name.is_empty() {
h.name = name;
+59 -17
View File
@@ -175,10 +175,11 @@ mod session_main {
// the last store read the compat path still owes. `addr` is moved into the struct
// below, so read it first.
let clipboard = clipboard_override.unwrap_or_else(|| {
// The record this address RESOLVES to, not "any record mentioning it": a retired
// duplicate must never be the one that hands a host the clipboard.
trust::KnownHosts::load()
.hosts
.iter()
.any(|h| h.addr == addr && h.port == port && h.clipboard_sync)
.find_by_addr(&addr, port)
.is_some_and(|h| h.clipboard_sync)
});
// Re-apply the shell-persisted forwarded-controller pin (stable `vid:pid:name`
// key) to OUR gamepad service — the shells' in-process services can't reach this
@@ -187,6 +188,12 @@ mod session_main {
if !settings.forward_pad.is_empty() {
gamepad.set_pinned(Some(settings.forward_pad.clone()));
}
// Whether to forward controllers AT ALL (off = the pad reaches the host by some other
// route — VirtualHere and friends). Set unconditionally, not only when off: browse mode
// reuses one service across launches, so a stream that follows one with it off must put
// it back. It goes on before the attach below, so a non-forwarding session never opens
// — never grabs — the device.
gamepad.set_forwarding(settings.gamepad_forwarding);
let mode = Mode {
width: if settings.width == 0 {
native.width
@@ -218,6 +225,8 @@ mod session_main {
height: sh,
..mode
};
// Before the struct literal — `vulkan` moves into it below.
let phase_lock = vulkan.as_ref().is_some_and(|v| v.present_timing);
SessionParams {
host: addr,
port,
@@ -248,9 +257,14 @@ mod session_main {
// 4:4:4 is opt-in and off by default (Settings "Full chroma"): the bit only says
// "upgrade me if you can" — the host still gates on its own policy, its capturer,
// HEVC, and a real GPU 4:4:4 encode probe, and answers the resolved chroma in the
// Welcome BEFORE we build a decoder. Advertised unconditionally when the user asks
// for it because every decode path here can produce it: the hardware ones where the
// driver decodes RExt, and swscale for the rest (the decoder demotes on its own).
// Welcome BEFORE we build a decoder. Advertised whenever the user asks because
// every path can DISPLAY it: the Vulkan presenter samples the 2-plane 4:4:4 pool
// formats (hardware RExt decode where the driver offers it — NVIDIA today) and
// swscale converts anything else for the software rung, with the decoder ladder
// demoting on its own. No capability probe gates the bit — software decode is the
// guaranteed floor — but the cost is VISIBLE, not silent: the Detailed stats
// overlay prints the resolved chroma ("4:4:4→4:2:0" when the host declined) and
// the decode path frames actually took.
video_caps: punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE
| if settings.hdr_enabled {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
@@ -262,9 +276,19 @@ mod session_main {
} else {
0
},
// No portable Wayland/X11 display-volume query yet, so the host keeps its EDID
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually.
// This panel's HDR colour volume → the host's virtual-display EDID, so host
// apps tone-map to the real glass. Windows reads it from DXGI (the
// `--window-pos` monitor; advanced-color outputs only) — gated on the HDR
// setting, since with 10-bit/HDR unadvertised above the volume is noise. No
// portable Wayland/X11 query exists yet, so Linux keeps the host's EDID
// defaults; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session pump) pins one
// manually on either OS and wins over both.
#[cfg(windows)]
display_hdr: settings
.hdr_enabled
.then(|| pf_client_core::video_d3d11::display_hdr_volume(window_pos()))
.flatten(),
#[cfg(not(windows))]
display_hdr: None,
// The presenter renders the host cursor locally in desktop mouse mode (M2 cursor
// channel); capture-mode sessions keep the composited cursor, so only advertise
@@ -272,6 +296,7 @@ mod session_main {
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled,
echo_cancel: settings.echo_cancel,
clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter
// demotes to software on boxes whose Vulkan can't import the dmabufs).
@@ -284,6 +309,13 @@ mod session_main {
connect_timeout: connect_timeout(),
force_software,
profile,
// Phase-locked capture (design/phase-locked-capture.md, Apple/Android parity):
// advertised only when the presenter has real on-glass latch stamps
// (VK_KHR_present_wait) — without them there is no latch grid to report. The
// grid itself is written by the presenter (run_session clones the Arc out of
// these params) and folded into ~1 Hz PhaseReports by the session pump.
phase_lock,
latch_grid: std::sync::Arc::new(pf_client_core::session::LatchGrid::default()),
}
}
@@ -438,11 +470,21 @@ mod session_main {
// The Settings device picks → env, unless the user already forced one by hand:
// the GPU (the shells' pickers store the adapter's marketing name) for the
// presenter's device selection, and the audio endpoints (PipeWire node names)
// for the playback/mic streams' `target.object`. Before any Vulkan call, like
// the RADV knob (covers --connect and --browse).
// presenter's device selection, and the audio endpoints (PipeWire node names /
// WASAPI endpoint ids) for the playback/mic streams. Before any Vulkan call,
// like the RADV knob (covers --connect and --browse).
//
// Spec mode takes them from the SPEC's settings — the spawner's resolve — which
// keeps the §5 zero-store-reads invariant and lets a profile overlay reach these
// fields if they ever become profileable. Parsed leniently here (the `--connect`
// flow re-reads the spec authoritatively and errors there); the compat path and
// `--browse` (which never carries a spec) still load the store.
{
let s = trust::Settings::load();
let s = arg_value("--resolved-spec")
.and_then(|p| {
pf_client_core::orchestrate::ResolvedSpec::read(std::path::Path::new(&p)).ok()
})
.map_or_else(trust::Settings::load, |spec| spec.settings);
for (var, value) in [
("PUNKTFUNK_VK_ADAPTER", &s.adapter),
("PUNKTFUNK_AUDIO_SINK", &s.speaker_device),
@@ -540,10 +582,7 @@ mod session_main {
// connects silently; an unknown host is REFUSED — there is no dialog here, and a
// silent TOFU would defeat the pinning model. Pair via the desktop client.
let known = trust::KnownHosts::load();
let known_host = known
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port);
let known_host = known.find_by_addr(&addr, port);
let pin = arg_value("--fp")
.as_deref()
.and_then(trust::parse_hex32)
@@ -584,6 +623,9 @@ mod session_main {
mouse_mode: settings.mouse_mode(),
invert_scroll: settings.invert_scroll,
inhibit_shortcuts: settings.inhibit_shortcuts,
present_priority: settings.present_priority(),
vsync: settings.vsync,
allow_vrr: settings.allow_vrr,
json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
// This host's card carries the accent bar in the desktop client now.
+40 -15
View File
@@ -80,14 +80,42 @@ fn initiate_opts(
}
/// Start a stream that launches a library title on connect (`--launch id`) — the library
/// page's tap-to-play. The library only opens for paired hosts, so the pin resolves like
/// a normal initiate; a host forgotten mid-visit routes to the PIN ceremony instead.
/// page's tap-to-play, and a deep link's `launch=` (which this used to drop: the link
/// opened a plain desktop session). The library only opens for paired hosts, so the pin
/// resolves like a normal initiate; a host forgotten mid-visit routes to the PIN ceremony
/// instead.
pub(crate) fn initiate_launch(
ctx: &Arc<AppCtx>,
target: Target,
launch: String,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
initiate_launch_opts(ctx, target, launch, set_screen, set_status, false)
}
/// [`initiate_launch`] with the dial-first wake of [`initiate_waking`] — a deep link's
/// `launch=` toward a saved host that isn't advertising but has a known MAC.
pub(crate) fn initiate_launch_waking(
ctx: &Arc<AppCtx>,
target: Target,
launch: String,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
if ctx.settings.lock().unwrap().auto_wake {
crate::wol::wake(&target.mac, target.addr.parse().ok());
}
initiate_launch_opts(ctx, target, launch, set_screen, set_status, true)
}
fn initiate_launch_opts(
ctx: &Arc<AppCtx>,
target: Target,
launch: String,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
wake_on_fail: bool,
) {
*ctx.shared.target.lock().unwrap() = target.clone();
let known = KnownHosts::load();
@@ -112,6 +140,7 @@ pub(crate) fn initiate_launch(
set_status,
ConnectOpts {
launch: Some(launch),
wake_on_fail,
..ConnectOpts::default()
},
);
@@ -222,16 +251,6 @@ fn connect_spawn(
*ctx.shared.session.lock().unwrap() = child.clone();
ctx.shared.stats_line.lock().unwrap().clear();
ctx.shared.browse.store(false, Ordering::SeqCst);
// Through the same resolver the session uses, not the raw globals: "Start streams
// fullscreen" is a profileable (tier-P) field, so a host bound to a windowed profile has to
// win here too — the child takes this decision from the argv, not from its own settings.
let fullscreen = pf_client_core::trust::effective_settings(
&target.addr,
target.port,
target.profile.as_deref(),
)
.0
.fullscreen_on_stream;
set_status.call(String::new());
set_screen.call(if opts.awaiting_approval {
Screen::RequestAccess
@@ -249,13 +268,15 @@ fn connect_spawn(
// The closure owns `target`/`fp_hex`; the call itself borrows copies.
let (addr, port, fp_arg) = (target.addr.clone(), target.port, fp_hex.clone());
let profile_arg = target.profile.clone();
// The launch id: an explicit opts pick (the library's tap-to-play), else one riding
// the target — a deep link's `launch=` that detoured through the PIN ceremony.
let launch_arg = opts.launch.clone().or_else(|| target.launch.clone());
let spawned = crate::spawn::spawn_session(
&addr,
port,
&fp_arg,
opts.connect_timeout.as_secs(),
fullscreen,
opts.launch.as_deref(),
launch_arg.as_deref(),
profile_arg.as_deref(),
child,
move |event| {
@@ -277,8 +298,10 @@ fn connect_spawn(
// host PAIRED so future connects are silent. Plain TOFU persists
// it *unpaired* (pinned): the child connected pinned to the
// advertised fingerprint, so ready proves the host holds it.
// Either way an authorised decision, so `upsert_trusted`: a dead
// record for this address is retired instead of shadowing this one.
let mut k = KnownHosts::load();
k.upsert(KnownHost {
k.upsert_trusted(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
@@ -502,6 +525,8 @@ fn wake_and_connect(
// Came back on a new IP (DHCP): dial the fresh address and re-key the saved
// host so the pin stays reachable next time (keyed by fingerprint;
// addr/port overwritten, `paired`/`mac` preserved by `upsert`).
// Plain `upsert` on purpose — this is an mDNS advert talking, not a trust
// decision, so it may never retire another saved host at that address.
if let Some((addr, port)) =
resolved.filter(|(a, p)| *a != target.addr || *p != target.port)
{
+4
View File
@@ -21,6 +21,10 @@ const STREAM_SHORTCUTS: &[(&str, &str)] = &[
"Ctrl+Alt+Shift+S",
"Cycle the statistics overlay (off \u{00B7} compact \u{00B7} normal \u{00B7} detailed)",
),
(
"Ctrl+Alt+Shift+V",
"Mute or unmute your microphone (only while the stream sends one)",
),
(
"LB+RB+Start+Back",
"Controller: release input / leave fullscreen \u{2014} hold to disconnect",
+11 -2
View File
@@ -623,8 +623,14 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
actions.push(
icon_btn("Settings", Symbol::Setting)
.on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Settings)
let (c, ss) = (ctx.clone(), set_screen.clone());
move || {
// Re-base the settings snapshot on the file before the page
// renders — this process is not its only writer (see
// settings::refresh_snapshot).
super::settings::refresh_snapshot(&c);
ss.call(Screen::Settings)
}
})
.into(),
);
@@ -670,6 +676,7 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
pair_optional: false,
mac: k.mac.clone(),
profile: None,
launch: None,
};
// Online = advertising on mDNS OR proven reachable by the last probe sweep (the latter
// covers a routed/Tailscale host that never advertises — the display companion to
@@ -959,6 +966,7 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
pair_optional: h.pair == "optional",
mac: h.mac.clone(),
profile: None,
launch: None,
};
let (ctx2, ss, st) = (ctx.clone(), set_screen.clone(), set_status.clone());
let (badge, kind) = if h.pair == "required" {
@@ -1052,6 +1060,7 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
pair_optional: false,
mac: Vec::new(),
profile: None,
launch: None,
},
&ss,
&st,
+10 -4
View File
@@ -2,7 +2,8 @@
//! Settings).
use super::style::*;
use super::Screen;
use super::{AppCtx, Screen};
use std::sync::Arc;
use windows_reactor::*;
/// punktfunk's own license (MIT OR Apache-2.0).
@@ -15,10 +16,15 @@ const APP_LICENSE: &str = concat!(
/// scripts/gen-third-party-notices.sh; the MSIX also ships this under licenses/).
const THIRD_PARTY_NOTICES: &str = include_str!("../../../../THIRD-PARTY-NOTICES.txt");
pub(crate) fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
pub(crate) fn licenses_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Element {
let back_btn = button("Back").accent().icon(Symbol::Back).on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Settings)
let (c, ss) = (ctx.clone(), set_screen.clone());
move || {
// Back RE-ENTERS the settings page — re-base its snapshot on the file, same
// as the hosts page's Settings button (see settings::refresh_snapshot).
super::settings::refresh_snapshot(&c);
ss.call(Screen::Settings)
}
});
let app_card = card(
+53 -13
View File
@@ -107,6 +107,10 @@ pub(crate) struct Target {
/// `None` honors the binding. It never rebinds anything — the default changes only through
/// the picker in the host editor (design/client-settings-profiles.md §5.2).
pub(crate) profile: Option<String>,
/// A library title id (`steam:570`, …) to launch on connect — carried on the target so it
/// survives a detour through the PIN ceremony (a deep link's `launch=` toward an unpaired
/// host must still launch the game once pairing succeeds).
pub(crate) launch: Option<String>,
}
/// Stable app services handed to the page components as props. Each routed screen that uses
@@ -168,6 +172,10 @@ pub(crate) struct Shared {
pub struct AppCtx {
pub(crate) identity: (String, String),
/// The settings snapshot the UI renders from. Loaded once at startup, and RE-BASED on
/// the file when the settings page is (re)entered (`settings::refresh_snapshot`) and
/// inside every `commit` — this process is not the file's only writer (session resize,
/// console UI, Decky), so a plain process-lifetime snapshot goes stale on screen.
pub(crate) settings: Mutex<Settings>,
pub(crate) gamepad: GamepadService,
pub(crate) shared: Arc<Shared>,
@@ -392,22 +400,54 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
pair_optional: false,
mac: p.host.mac.clone(),
profile: p.profile_override.clone(),
launch: None, // routed explicitly below (initiate_launch*)
};
// With a MAC it takes the dial first wake path, so a sleeping host wakes
// instead of erroring — exactly what clicking its tile would do.
if p.wake && !target.mac.is_empty() {
connect::initiate_waking(&ctx, target, &set_screen, &set_status);
} else {
connect::initiate(&ctx, target, &set_screen, &set_status);
// instead of erroring — exactly what clicking its tile would do. The
// link's `launch=` id must reach the session (`--launch`) — this arm used
// to drop it, so a game link opened a plain desktop session.
match (p.launch.clone(), p.wake && !target.mac.is_empty()) {
(Some(id), true) => {
connect::initiate_launch_waking(
&ctx,
target,
id,
&set_screen,
&set_status,
);
}
(Some(id), false) => {
connect::initiate_launch(&ctx, target, id, &set_screen, &set_status);
}
(None, true) => {
connect::initiate_waking(&ctx, target, &set_screen, &set_status)
}
(None, false) => connect::initiate(&ctx, target, &set_screen, &set_status),
}
}
// Known but never pinned, or not known at all: a link may not pair or trust on
// its own, so it lands on the host list with the reason shown. The user pairs
// there, under their own eyes.
Ok(PlanOutcome::ConfirmUnknown(u)) => refuse(format!(
"{} isn't paired with this device yet \u{2014} pair it, then use the link again.",
u.name.clone().unwrap_or_else(|| u.addr.clone())
)),
// Known but never pinned, or not known at all: a link may not pair and may not
// trust on its own, so it opens the ordinary PIN ceremony seeded with what the
// link CLAIMED — name shown as claimed, the fingerprint pre-filling the pin so
// the first connect is verified against it rather than blind TOFU, and the
// launch/profile surviving the detour (§3.1; GTK-shell parity — this used to
// refuse outright and make shared links a dead end on Windows).
Ok(PlanOutcome::ConfirmUnknown(u)) => {
let name = u.name.clone().unwrap_or_else(|| u.addr.clone());
*ctx.shared.target.lock().unwrap() = Target {
name: name.clone(),
addr: u.addr.clone(),
port: u.port,
fp_hex: u.fp.clone(),
pair_optional: false,
mac: Vec::new(),
profile: u.profile.clone(),
launch: u.launch.clone(),
};
set_status.call(format!(
"{name} isn't paired with this device yet \u{2014} pair it to continue."
));
set_screen.call(Screen::Pair);
}
Ok(PlanOutcome::Unsupported(route)) => refuse(format!(
"Punktfunk can't open \u{201c}{}\u{201d} links yet.",
route.as_str()
@@ -652,7 +692,7 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
&set_settings_rev,
nav_progress,
),
Screen::Licenses => licenses::licenses_page(&set_screen),
Screen::Licenses => licenses::licenses_page(ctx, &set_screen),
Screen::Help => help::help_page(&set_screen),
Screen::Pair => component(pair::pair_page, svc),
Screen::SpeedTest => component(speed::speed_page, SpeedProps { svc, state: speed }),
+3 -1
View File
@@ -50,8 +50,10 @@ pub(crate) fn pair_page(props: &Svc, cx: &mut RenderCx) -> Element {
std::time::Duration::from_secs(90),
) {
Ok(fp) => {
// The PIN ceremony is an authorised trust decision, so this also
// retires a dead record for the same address (a re-keyed host).
let mut k = KnownHosts::load();
k.upsert(KnownHost {
k.upsert_trusted(KnownHost {
name: target3.name.clone(),
addr: target3.addr.clone(),
port: target3.port,
+339 -13
View File
@@ -75,6 +75,9 @@ const GAMEPADS: &[(&str, &str)] = &[
("dualsense", "DualSense"),
("xboxone", "Xbox One"),
("dualshock4", "DualShock 4"),
// Kept in lockstep with the GTK picker: this row was missing here, so a Windows
// user could not ask the host for the Deck-shaped pad (trackpads, back grips).
("steamdeck", "Steam Deck"),
];
/// Stats-overlay tiers: `(stored value, display label)` — the cross-client verbosity ladder
/// (Compact ⊂ Normal ⊂ Detailed); Ctrl+Alt+Shift+S cycles it live in the session window.
@@ -98,6 +101,19 @@ const MOUSE_MODES: &[(&str, &str)] = &[
("capture", "Capture (games)"),
("desktop", "Desktop (absolute)"),
];
/// Presentation intent: `(stored value, display label)` — the `present_priority` key the
/// Apple and Android clients share, so one profile means the same thing everywhere.
const PRESENT_PRIORITIES: &[(&str, &str)] =
&[("latency", "Lowest latency"), ("smooth", "Smoothness")];
/// Smoothness buffer depth in frames: `(stored value, display label)`. `0` = Automatic,
/// which resolves to 2 (`PresentPriority::resolve`). No millisecond hints — the cost is
/// one refresh per frame, and the refresh isn't known here when the mode is Native.
const SMOOTH_BUFFERS: &[(u8, &str)] = &[
(0, "Automatic"),
(1, "1 frame"),
(2, "2 frames"),
(3, "3 frames"),
];
/// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to
/// auto-detect when the choice is unavailable. Only meaningful against a Linux host.
const COMPOSITORS: &[(&str, &str)] = &[
@@ -393,14 +409,31 @@ fn commit(
edit: impl FnOnce(&mut Settings),
) {
if scope.is_empty() {
// Rebase on the file before the whole-struct save: the process-lifetime snapshot
// in `ctx.settings` is not the only writer — a spawned session persists its
// match-window size, the console's own settings screen saves too — and saving the
// stale snapshot would silently revert whatever they stored (the same
// load-modify-save family as the GTK dialog's 2026-07-31 fix; profiles.rs
// documents why there's no merge). The edit lands on the fresh load, and the
// snapshot follows so every row keeps rendering what's on disk.
let mut s = ctx.settings.lock().unwrap();
*s = Settings::load();
edit(&mut s);
s.save();
rev.1.call(rev.0 + 1);
return;
}
let mut catalog = ProfilesFile::load();
let base = ctx.settings.lock().unwrap().clone();
// The same rebase as the global arm above: `base` is what `absorb`'s before/after
// effective settings derive from, and the snapshot is not the file — another process
// (session resize, console UI, Decky) may have moved a global under us. The historical
// rebase fix ("settings saves stop reverting each other") covered the whole-file
// writers but missed this arm.
let base = {
let mut s = ctx.settings.lock().unwrap();
*s = Settings::load();
s.clone()
};
let Some(p) = catalog.profiles.iter_mut().find(|p| p.id == scope) else {
return; // deleted from under us; the next render falls back to the defaults scope
};
@@ -414,6 +447,17 @@ fn commit(
rev.1.call(rev.0 + 1);
}
/// Re-base the process-lifetime settings snapshot on the file — called from the navigation
/// handlers that (re)enter this page, NOT per render pass. `ctx.settings` is loaded once at
/// process start and this process is not the file's only writer (a spawned session persists
/// its match-window size, the console UI and Decky save too — profiles.rs documents the
/// family), so without this the page opens showing values another process already replaced,
/// which then visibly "jump" the moment a row is touched and `commit`'s rebase pulls the
/// file in. The field report this fixes: a codec setting that "changed by itself".
pub(crate) fn refresh_snapshot(ctx: &Arc<AppCtx>) {
*ctx.settings.lock().unwrap() = Settings::load();
}
/// Which tier-P rows the profile in scope overrides. Plain bools rather than a lookup so the
/// call sites read as `over.codec` — the row and its flag stay visibly paired.
#[derive(Default)]
@@ -428,13 +472,19 @@ struct OverrideFlags {
compositor: bool,
audio_channels: bool,
mic_enabled: bool,
echo_cancel: bool,
touch_mode: bool,
mouse_mode: bool,
invert_scroll: bool,
inhibit_shortcuts: bool,
gamepad: bool,
gamepad_forwarding: bool,
stats_verbosity: bool,
fullscreen_on_stream: bool,
present_priority: bool,
smooth_buffer: bool,
vsync: bool,
allow_vrr: bool,
}
impl OverrideFlags {
@@ -455,13 +505,19 @@ impl OverrideFlags {
compositor: o.compositor.is_some(),
audio_channels: o.audio_channels.is_some(),
mic_enabled: o.mic_enabled.is_some(),
echo_cancel: o.echo_cancel.is_some(),
touch_mode: o.touch_mode.is_some(),
mouse_mode: o.mouse_mode.is_some(),
invert_scroll: o.invert_scroll.is_some(),
inhibit_shortcuts: o.inhibit_shortcuts.is_some(),
gamepad: o.gamepad.is_some(),
gamepad_forwarding: o.gamepad_forwarding.is_some(),
stats_verbosity: o.stats_verbosity.is_some(),
fullscreen_on_stream: o.fullscreen_on_stream.is_some(),
present_priority: o.present_priority.is_some(),
smooth_buffer: o.smooth_buffer.is_some(),
vsync: o.vsync.is_some(),
allow_vrr: o.allow_vrr.is_some(),
}
}
}
@@ -838,6 +894,32 @@ pub(crate) fn settings_page(
let chroma_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.enable_444, |s, on| {
s.enable_444 = on
});
// Presentation intent (design/desktop-presentation-rebuild.md). The buffer row is
// rendered only under Smoothness — `commit` bumps the revision, so flipping the
// intent re-renders the section and the row appears/disappears with it.
let (present_names, present_i) = presets(PRESENT_PRIORITIES, |v| *v == s.present_priority);
let present_combo = setting_combo(
ctx,
scope,
(rev, set_rev),
present_names,
present_i,
|s, i| s.present_priority = PRESENT_PRIORITIES[i].0.to_string(),
);
let smoothing = s.present_priority == "smooth";
let (buffer_names, buffer_i) = presets(SMOOTH_BUFFERS, |v| *v == s.smooth_buffer);
let buffer_combo = setting_combo(
ctx,
scope,
(rev, set_rev),
buffer_names,
buffer_i,
|s, i| s.smooth_buffer = SMOOTH_BUFFERS[i].0,
);
let vsync_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.vsync, |s, on| s.vsync = on);
let vrr_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.allow_vrr, |s, on| {
s.allow_vrr = on
});
// --- Input -----------------------------------------------------------------------------
// Controller forwarding: Automatic forwards EVERY real controller, each as its own pad;
@@ -875,13 +957,20 @@ pub(crate) fn settings_page(
keys.get(sel - 1).cloned()
};
// Apply live to the gamepad service and persist — the spawned session
// reads `forward_pad` at connect.
// reads `forward_pad` at connect. Rebase on the file first (the same
// discipline as `commit()`): this handler bypasses commit and a stale
// whole-struct save would revert other writers.
svc.set_pinned(key.clone());
let mut s = ctx2.settings.lock().unwrap();
*s = Settings::load();
s.forward_pad = key.unwrap_or_default();
s.save();
})
};
let pad_forward_toggle =
setting_toggle(ctx, scope, (rev, set_rev), s.gamepad_forwarding, |s, on| {
s.gamepad_forwarding = on
});
let (pad_names, pad_i) = presets(GAMEPADS, |v| {
GamepadPref::from_name(v) == GamepadPref::from_name(&s.gamepad)
});
@@ -913,6 +1002,39 @@ pub(crate) fn settings_page(
let mic_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.mic_enabled, |s, on| {
s.mic_enabled = on
});
// Endpoint pickers (the WASAPI probe — the GTK client's PipeWire twins): visible
// labels are friendly names, the stored value is the endpoint id. Hidden when the
// probe found at most the default; a saved device that's gone keeps a revertable
// "(not detected)" entry, like the GPU row. Device facts — defaults scope only.
let (speakers, mics) = pf_client_core::audio::devices().unwrap_or_default();
let dev_combo = |saved: &str,
devs: &[pf_client_core::audio::AudioDevice],
apply: fn(&mut Settings, String)| {
let mut names = vec!["System default".to_string()];
let mut keys = vec![String::new()];
for d in devs {
names.push(d.description.clone());
keys.push(d.name.clone());
}
if !saved.is_empty() && !keys.iter().any(|k| k == saved) {
names.push(format!("{saved} (not detected)"));
keys.push(saved.to_string());
}
(keys.len() > 1).then(|| {
let current = keys.iter().position(|k| k == saved).unwrap_or(0);
setting_combo(ctx, scope, (rev, set_rev), names, current, move |s, i| {
apply(s, keys[i.min(keys.len() - 1)].clone());
})
})
};
let speaker_combo = dev_combo(&s.speaker_device, &speakers, |s, v| s.speaker_device = v);
let mic_dev_combo = dev_combo(&s.mic_device, &mics, |s, v| s.mic_device = v);
// Echo cancellation is meaningless without an uplink, so it greys out with the mic above
// it. Every commit bumps `rev` and re-renders this screen, so the two stay in step live.
let echo_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.echo_cancel, |s, on| {
s.echo_cancel = on
})
.enabled(s.mic_enabled);
let (hud_names, hud_i) = presets(STATS_TIERS, |v| *v == s.stats_verbosity());
let hud_combo = setting_combo(ctx, scope, (rev, set_rev), hud_names, hud_i, |s, i| {
@@ -923,6 +1045,16 @@ pub(crate) fn settings_page(
let ss = set_screen.clone();
button("Third-party licenses").on_click(move || ss.call(Screen::Licenses))
};
// The client log's home (%LOCALAPPDATA%\punktfunk\logs) — the file every "check the
// client log" message means, which until this row had no way in from the UI at all.
// The folder rather than the file so the rotated `.old` generation is in reach too.
// Best-effort, like the log itself: a missing dir or a failed spawn stays silent.
let logs_button = button("Open log folder").on_click(|| {
if let Some(dir) = crate::logfile::log_dir() {
let _ = std::fs::create_dir_all(&dir);
let _ = std::process::Command::new("explorer.exe").arg(&dir).spawn();
}
});
let library_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.library_enabled, |s, on| {
s.library_enabled = on
});
@@ -1016,8 +1148,9 @@ pub(crate) fn settings_page(
"HDR10, when the host has HDR content and this display supports it. \
HEVC only; otherwise the stream stays SDR.",
),
// Wording shared with the GTK client (its chroma_row) — same setting,
// same constraints.
// First sentence shared with the GTK client (its chroma_row); the
// constraint sentence names the real gate (host: PyroWave || NVENC) —
// "where the host can encode it" cost field users the discovery time.
described_overridable(
(rev, set_rev),
scope,
@@ -1026,7 +1159,8 @@ pub(crate) fn settings_page(
over.enable_444,
chroma_toggle,
"Full-colour video: crisp small text and thin lines, at more \
bandwidth. HEVC only, and only where the host can encode it.",
bandwidth. Requires an NVIDIA host (NVENC) or the PyroWave \
codec \u{2014} other encoders stream 4:2:0.",
),
],
None,
@@ -1056,6 +1190,60 @@ pub(crate) fn settings_page(
},
None,
));
out.extend(group(
Some("Presentation"),
{
let mut fields = vec![described_overridable(
(rev, set_rev),
scope,
"present_priority",
"Prioritize",
over.present_priority,
present_combo,
"Lowest latency shows each frame the moment the display can take \
it \u{2014} a network hiccup becomes an occasional repeated or \
skipped frame. Smoothness buffers a little to even those out.",
)];
if smoothing {
fields.push(described_overridable(
(rev, set_rev),
scope,
"smooth_buffer",
"Smoothness buffer",
over.smooth_buffer,
buffer_combo,
"Frames held back before showing. Each one absorbs about a \
refresh of network hiccup and adds a refresh of delay. \
Automatic holds two.",
));
}
fields.push(described_overridable(
(rev, set_rev),
scope,
"vsync",
"V-Sync",
over.vsync,
vsync_toggle,
"Tear-free. Turning it off removes the wait for the screen\u{2019}s \
refresh \u{2014} the lowest possible delay, at the cost of visible \
tearing. Not every driver offers it; the stats overlay names the \
mode actually in use.",
));
fields.push(described_overridable(
(rev, set_rev),
scope,
"allow_vrr",
"Follow variable refresh rate",
over.allow_vrr,
vrr_toggle,
"On a VRR/FreeSync/G-Sync screen, let the panel refresh in step with \
the stream instead of on a fixed cadence. Applies to fullscreen \
sessions; harmless on a fixed-refresh screen.",
));
fields
},
None,
));
out.extend(group(
Some("Host output"),
vec![described_overridable(
@@ -1134,6 +1322,63 @@ pub(crate) fn settings_page(
group(
None,
[
// The read-only pad inventory (GTK parity): what THIS device sees right
// now — the fastest answer to "is my controller even detected?". A
// device fact, so defaults scope only, like the forward picker below.
(!profile_mode).then(|| {
let inventory: Element = if pads.is_empty() {
text_block("No controllers detected")
.font_size(12.0)
.foreground(ThemeRef::SecondaryText)
.into()
} else {
vstack(
pads.iter()
.map(|p| {
let sub = if p.steam_virtual {
"Steam Input's virtual pad \u{2014} Automatic skips \
it while a real pad is connected"
.to_string()
} else {
p.kind_label().to_string()
};
vstack((
text_block(p.name.clone()).semibold(),
text_block(sub)
.font_size(11.0)
.foreground(ThemeRef::SecondaryText),
))
.spacing(1.0)
.into()
})
.collect::<Vec<Element>>(),
)
.spacing(8.0)
.into()
};
described_labeled(
"Detected controllers",
inventory,
"Plug in or pair a controller and it appears here.",
)
}),
// Whether ANY controller is forwarded — profileable, so it renders in
// both scopes (a "Work" profile can decline what "Game" forwards),
// unlike the device-fact picker below it.
Some(described_overridable(
(rev, set_rev),
scope,
"gamepad_forwarding",
"Forward controllers",
over.gamepad_forwarding,
pad_forward_toggle,
"Sends controllers connected to this PC to the host. Turn it off when \
your controller already reaches the host another way \u{2014} USB \
passthrough such as VirtualHere, or a pad plugged into the host \
itself \u{2014} so games don't see two of them. Off, this PC never \
opens the controller at all, which is what leaves it free for a \
passthrough tool to claim.",
)),
// NOT Apple's wording: Apple forwards ONE pad as player 1, this client
// forwards every controller as its own player. Same picker, different rule.
// Which physical pad this device forwards is a device fact (tier G), so it
@@ -1168,8 +1413,8 @@ pub(crate) fn settings_page(
"Audio",
group(
None,
vec![
described_overridable(
[
Some(described_overridable(
(rev, set_rev),
scope,
"audio_channels",
@@ -1178,17 +1423,57 @@ pub(crate) fn settings_page(
channels_combo,
"The speaker layout requested from the host. It downmixes if its own \
output has fewer channels.",
),
described_overridable(
)),
// The endpoint picks are facts about THIS device's hardware — never
// per profile, like Decoder/GPU.
(!profile_mode)
.then(|| {
speaker_combo.map(|c| {
described_labeled(
"Speaker",
c,
"Host audio plays here \u{2014} System default follows \
the Windows output device.",
)
})
})
.flatten(),
Some(described_overridable(
(rev, set_rev),
scope,
"mic_enabled",
"Stream microphone to the host",
over.mic_enabled,
mic_toggle,
"This device\u{2019}s microphone feeds the host\u{2019}s virtual mic.",
),
],
"This device\u{2019}s microphone feeds the host\u{2019}s virtual mic. \
Ctrl+Alt+Shift+V mutes and unmutes it during a stream.",
)),
(!profile_mode)
.then(|| {
mic_dev_combo.map(|c| {
described_labeled(
"Microphone",
c,
"The input that feeds the host\u{2019}s virtual mic.",
)
})
})
.flatten(),
Some(described_overridable(
(rev, set_rev),
scope,
"echo_cancel",
"Echo cancellation",
over.echo_cancel,
echo_toggle,
"Keeps the host\u{2019}s audio, playing from this machine\u{2019}s \
speakers, from being picked up and sent straight back. Turn it off if \
your microphone already does its own processing.",
)),
]
.into_iter()
.flatten()
.collect(),
Some("Applies from the next session."),
),
),
@@ -1196,7 +1481,16 @@ pub(crate) fn settings_page(
"About",
group(
None,
vec![about_identity.into(), licenses_button.into()],
vec![
about_identity.into(),
described_labeled(
"Diagnostics",
logs_button,
"The client log (client.log, plus the session\u{2019}s whole \
receive/decode/present trail) \u{2014} attach it to a bug report.",
),
licenses_button.into(),
],
None,
),
),
@@ -1587,5 +1881,37 @@ mod tests {
..Default::default()
};
assert!(OverrideFlags::of(Some(&p2)).resolution);
// The audio pair: the mic and its echo canceller are separate overrides, so a profile
// can pin one without claiming the other.
let mut p3 = StreamProfile::new("t3".to_string());
p3.overrides = SettingsOverlay {
echo_cancel: Some(false),
..Default::default()
};
let f3 = OverrideFlags::of(Some(&p3));
assert!(f3.echo_cancel);
assert!(!f3.mic_enabled);
// The presentation pair, likewise independent: pinning the intent doesn't claim
// the buffer (a "Smoothness, whatever the global buffer is" profile is valid).
let mut p4 = StreamProfile::new("t4".to_string());
p4.overrides = SettingsOverlay {
present_priority: Some("smooth".into()),
..Default::default()
};
let f4 = OverrideFlags::of(Some(&p4));
assert!(f4.present_priority);
assert!(!f4.smooth_buffer);
// V-Sync and VRR are independent of each other and of the intent pair.
let mut p5 = StreamProfile::new("t5".to_string());
p5.overrides = SettingsOverlay {
vsync: Some(false),
..Default::default()
};
let f5 = OverrideFlags::of(Some(&p5));
assert!(f5.vsync);
assert!(!f5.allow_vrr && !f5.present_priority);
}
}
+74 -40
View File
@@ -130,9 +130,7 @@ pub(crate) fn speed_page(props: &SpeedProps, cx: &mut RenderCx) -> Element {
// it: the one-off this test was started with, else the host's binding.
let target = ctx.shared.target.lock().unwrap().clone();
let bound = KnownHosts::load()
.hosts
.iter()
.find(|h| h.addr == target.addr && h.port == target.port)
.find_by_addr(&target.addr, target.port)
.and_then(|h| h.profile_id.clone());
let profile = match target.profile.as_deref() {
Some("") => None,
@@ -140,39 +138,80 @@ pub(crate) fn speed_page(props: &SpeedProps, cx: &mut RenderCx) -> Element {
None => bound,
}
.and_then(|reference| ProfilesFile::load().resolve(&reference).0.cloned());
let apply_btn = {
let (ctx, ss, kbps) = (ctx.clone(), set_screen.clone(), *recommended_kbps);
let profile = profile.clone();
button(match &profile {
Some(p) => format!(
"Set {recommended_mbps:.0} Mb/s in \u{201c}{}\u{201d}",
p.name
),
None => format!("Use {recommended_mbps:.0} Mb/s"),
})
.accent()
.icon(Symbol::Accept)
.on_click(move || {
match &profile {
Some(p) => {
let mut catalog = ProfilesFile::load();
if let Some(slot) = catalog.profiles.iter_mut().find(|x| x.id == p.id) {
slot.overrides.bitrate_kbps = Some(kbps);
if let Err(e) = catalog.save() {
tracing::warn!(error = %format!("{e:#}"),
"saving the measured bitrate");
}
}
}
None => {
let mut s = ctx.settings.lock().unwrap();
s.bitrate_kbps = kbps;
s.save();
let kbps = *recommended_kbps;
let write_global = {
let (ctx, ss) = (ctx.clone(), set_screen.clone());
move || {
// Rebase on the file before the whole-struct save — same discipline as
// `commit()`; another writer may have moved it under this snapshot.
let mut s = ctx.settings.lock().unwrap();
*s = crate::trust::Settings::load();
s.bitrate_kbps = kbps;
s.save();
ss.call(Screen::Hosts);
}
};
let write_profile = |id: String| {
let ss = set_screen.clone();
move || {
let mut catalog = ProfilesFile::load();
if let Some(slot) = catalog.profiles.iter_mut().find(|x| x.id == id) {
slot.overrides.bitrate_kbps = Some(kbps);
if let Err(e) = catalog.save() {
tracing::warn!(error = %format!("{e:#}"),
"saving the measured bitrate");
}
}
ss.call(Screen::Hosts);
})
}
};
// Which button(s): no binding → the global; a binding that already overrides
// bitrate → that override (it's what this host reads). Bound but INHERITING
// bitrate could legitimately mean either layer — offer both rather than
// guessing (the GTK client's Ask tier; this shell used to silently CREATE an
// override on the profile).
let mut buttons: Vec<Element> = Vec::new();
match &profile {
None => buttons.push(
button(format!("Use {recommended_mbps:.0} Mb/s"))
.accent()
.icon(Symbol::Accept)
.on_click(write_global.clone())
.into(),
),
Some(p) if p.overrides.bitrate_kbps.is_some() => buttons.push(
button(format!(
"Set {recommended_mbps:.0} Mb/s in \u{201c}{}\u{201d}",
p.name
))
.accent()
.icon(Symbol::Accept)
.on_click(write_profile(p.id.clone()))
.into(),
),
Some(p) => {
buttons.push(
button("Set as default")
.icon(Symbol::Accept)
.on_click(write_global.clone())
.into(),
);
buttons.push(
button(format!("Set in \u{201c}{}\u{201d}", p.name))
.accent()
.icon(Symbol::Accept)
.on_click(write_profile(p.id.clone()))
.into(),
);
}
}
buttons.push({
let ss = set_screen.clone();
button("Close")
.icon(Symbol::Cancel)
.on_click(move || ss.call(Screen::Hosts))
.into()
});
let results = card(
vstack((
text_block(format!("{mbps:.0} Mbit/s"))
@@ -190,14 +229,9 @@ pub(crate) fn speed_page(props: &SpeedProps, cx: &mut RenderCx) -> Element {
.font_size(12.0)
.foreground(ThemeRef::SecondaryText)
.horizontal_alignment(HorizontalAlignment::Center),
hstack((apply_btn, {
let ss = set_screen.clone();
button("Close")
.icon(Symbol::Cancel)
.on_click(move || ss.call(Screen::Hosts))
}))
.spacing(8.0)
.horizontal_alignment(HorizontalAlignment::Center),
hstack(buttons)
.spacing(8.0)
.horizontal_alignment(HorizontalAlignment::Center),
))
.spacing(12.0),
);
+2 -1
View File
@@ -3,7 +3,8 @@
//! Streaming (decode + present) runs in the spawned `punktfunk-session` binary; the shell only
//! needs the list of real (hardware) adapters to offer on a multi-GPU box (a hybrid laptop or an
//! eGPU). The picked adapter description is persisted (`crate::trust::Settings::adapter`) and read
//! by the session child at connect (`PUNKTFUNK_ADAPTER` remains the session binary's env override).
//! by the session child at connect (`PUNKTFUNK_VK_ADAPTER` remains the session binary's env
//! override).
use windows::core::Interface;
use windows::Win32::dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
+3 -2
View File
@@ -21,11 +21,12 @@ const ROTATE_BYTES: u64 = 10 * 1024 * 1024;
static SINK: OnceLock<Option<Arc<Mutex<File>>>> = OnceLock::new();
fn log_dir() -> Option<PathBuf> {
/// The log directory — Settings ▸ About's "Open log folder" opens it in Explorer.
pub(crate) fn log_dir() -> Option<PathBuf> {
Some(PathBuf::from(std::env::var_os("LOCALAPPDATA")?).join(r"punktfunk\logs"))
}
/// The log file's path, for the "logs land here" startup line (and any future UI affordance).
/// The log file's path, for the "logs land here" startup line and the failed-spawn banner.
pub(crate) fn path() -> Option<PathBuf> {
Some(log_dir()?.join("client.log"))
}
+85 -26
View File
@@ -55,9 +55,19 @@ impl SessionChild {
/// One parsed stdout line of the session contract; `None` for anything unrecognized.
enum ChildLine {
Ready,
Error { msg: String, trust_rejected: bool },
Error {
msg: String,
trust_rejected: bool,
},
Ended(String),
Stats(String),
/// The session window's logical size settled here under match-window — the SPAWNER
/// persists it (design/client-architecture-split.md §5). This shell ignored the line
/// until 2026-07-31, so its sessions fell back to persisting from the renderer.
Window {
w: u32,
h: u32,
},
}
fn parse_line(line: &str) -> Option<ChildLine> {
@@ -77,6 +87,12 @@ fn parse_line(line: &str) -> Option<ChildLine> {
if let Some(msg) = v.get("ended").and_then(|m| m.as_str()) {
return Some(ChildLine::Ended(msg.to_string()));
}
if let Some(win) = v.get("window") {
let dim = |k: &str| win.get(k).and_then(|n| n.as_u64()).map(|n| n as u32);
if let (Some(w), Some(h)) = (dim("w"), dim("h")) {
return Some(ChildLine::Window { w, h });
}
}
None
}
@@ -89,7 +105,14 @@ fn parse_line(line: &str) -> Option<ChildLine> {
/// connect that silently drops back to the host list.
pub(crate) fn silent_exit_banner(code: i32) -> Option<String> {
(code != 0 && code != -1).then(|| {
format!("The session didn't start (punktfunk-session exited with code {code}). Check the client log.")
// Name the log's actual location — "check the client log" without a path is a
// scavenger hunt (Settings ▸ About's "Open log folder" reaches it too).
let log = crate::logfile::path()
.map(|p| p.display().to_string())
.unwrap_or_else(|| "the client log".into());
format!(
"The session didn't start (punktfunk-session exited with code {code}). Check {log}."
)
})
}
@@ -107,42 +130,60 @@ pub(crate) fn session_binary() -> std::path::PathBuf {
/// Spawn the session binary for a connect with `fp_hex` pinned and feed its lifecycle to
/// `on_event` from a reader thread. The child is parked in `slot` so Disconnect/Cancel
/// can kill it. `fullscreen` starts the stream window fullscreen (the Settings "Start
/// streams fullscreen" toggle); `launch` carries a library title id for the host to
/// launch during the handshake. `Err` = the spawn itself failed (binary missing?) —
/// surfaced as a connect error by the caller.
/// can kill it. `launch` carries a library title id for the host to launch during the
/// handshake; `profile` is a ONE-OFF settings-profile pick. `Err` = the spawn itself
/// failed (binary missing?) — surfaced as a connect error by the caller.
///
/// The argv and the `--resolved-spec` both come from the shared brain
/// ([`ConnectPlan::for_target`] → `session_args()` + `spec()`): this shell hand-assembled
/// its argv until 2026-07-31, which meant no spec — its sessions took the compat path and
/// re-resolved every setting from the stores (the drift `orchestrate.rs` documents as a
/// trap), and any field added to the spec was silently Windows-dead. Fullscreen now also
/// comes from the plan's EFFECTIVE settings (profile-aware) instead of a caller argument.
#[allow(clippy::too_many_arguments)] // one cohesive spawn spec (session_params precedent)
pub(crate) fn spawn_session(
addr: &str,
port: u16,
fp_hex: &str,
connect_timeout_secs: u64,
fullscreen: bool,
launch: Option<&str>,
profile: Option<&str>,
slot: SessionChild,
on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
use pf_client_core::orchestrate::{ConnectPlan, HostTarget};
let mut plan = ConnectPlan::for_target(
HostTarget {
name: String::new(), // display-only; this shell's screens carry their own copy
addr: addr.to_string(),
port,
fp_hex: Some(fp_hex.to_string()),
mac: Vec::new(), // wake ran before this spawn (initiate_waking) — not the plan's job
id: None,
},
launch.map(str::to_string),
profile.map(str::to_string),
);
plan.connect_timeout_secs = Some(connect_timeout_secs);
let mut cmd = Command::new(session_binary());
cmd.arg("--connect")
.arg(format!("{addr}:{port}"))
.arg("--fp")
.arg(fp_hex)
.arg("--connect-timeout")
.arg(connect_timeout_secs.to_string());
if fullscreen {
cmd.arg("--fullscreen");
}
if let Some(id) = launch {
cmd.arg("--launch").arg(id);
}
// Only a ONE-OFF pick rides the flag: without it the session resolves the host's own
// binding through the same helper this shell would have used, so the two can't disagree.
if let Some(reference) = profile {
cmd.arg("--profile").arg(reference);
}
let mut args = plan.session_args();
// Spec mode (design/client-architecture-split.md §5): the child reads no stores and
// cannot disagree with us about a file either of us might write. A spec we fail to
// write is not fatal — the compat path resolves the same values via the same helper.
let spec_path = match plan.spec(plan.clipboard).write_temp() {
Ok(path) => {
args.push("--resolved-spec".into());
args.push(path.to_string_lossy().into_owned());
Some(path)
}
Err(e) => {
tracing::warn!(error = %e, "couldn't write the resolved spec; the session will resolve for itself");
None
}
};
cmd.args(args);
add_window_pos(&mut cmd);
spawn_with(cmd, &format!("{addr}:{port}"), slot, on_event)
spawn_with(cmd, &format!("{addr}:{port}"), spec_path, slot, on_event)
}
/// Spawn the session binary in `--browse` mode: the console (gamepad) library for a
@@ -169,7 +210,7 @@ pub(crate) fn spawn_browse(
}
add_window_pos(&mut cmd);
let label = target.map_or_else(|| "console".to_string(), |(a, p)| format!("{a}:{p}"));
spawn_with(cmd, &label, slot, on_event)
spawn_with(cmd, &label, None, slot, on_event)
}
/// Hand the shell window's position to the child (`--window-pos`) so the session window
@@ -182,9 +223,11 @@ fn add_window_pos(cmd: &mut Command) {
}
/// The shared spawn + stdout-contract reader behind [`spawn_session`]/[`spawn_browse`].
/// `spec_path` is the child's `--resolved-spec` temp file, deleted once the child exits.
fn spawn_with(
mut cmd: Command,
host_label: &str,
spec_path: Option<std::path::PathBuf>,
slot: SessionChild,
mut on_event: impl FnMut(SpawnEvent) + Send + 'static,
) -> Result<(), String> {
@@ -225,9 +268,19 @@ fn spawn_with(
trust_rejected,
}) => error = Some((msg, trust_rejected)),
Some(ChildLine::Ended(msg)) => ended = Some(msg),
// The window size is the spawner's to persist — the renderer only
// reports it (same handling as orchestrate's own reader).
Some(ChildLine::Window { w, h }) => {
pf_client_core::orchestrate::persist_window_size(w, h);
}
None => {}
}
}
// The spec has done its job the moment the child has read it; a leftover temp
// file in %TEMP% is litter, and one per launch adds up.
if let Some(path) = &spec_path {
let _ = std::fs::remove_file(path);
}
// EOF — reap the child (killed-by-Disconnect lands here too; -1 = no code).
let code = slot
.0
@@ -277,6 +330,12 @@ mod tests {
Some(ChildLine::Stats(s)) => assert!(s.starts_with("1280")),
_ => panic!("stats line"),
}
// The match-window report: the SPAWNER persists it (§5) — dropping this line was
// why Windows sessions fell back to renderer-local persistence.
match parse_line("{\"window\":{\"w\":1600,\"h\":900}}") {
Some(ChildLine::Window { w, h }) => assert_eq!((w, h), (1600, 900)),
_ => panic!("window line"),
}
assert!(parse_line("").is_none());
assert!(parse_line("{\"other\":1}").is_none());
}
+15 -7
View File
@@ -488,12 +488,15 @@ pub(crate) fn note_hdr_capture_failed(source: HdrSource) {
}
#[cfg(target_os = "windows")]
pub fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
// but only a backend that ingests RGB and CSCs it to 4:4:4 itself can use it — today just
// direct-NVENC (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR
// display can't be known here (the virtual display's mode settles after the Welcome); that
// combination downgrades at capture time — the capturer emits P010 and the encoder's caps
// cross-check reports the 4:2:0 truth (the in-band SPS keeps the client correct either way).
// IDD-push delivers full-chroma RGB for a 4:4:4 session — BGRA on an SDR display, packed 10-bit
// BT.2020 PQ (`Rgb10a2`) on an HDR one — skipping the subsampling converters entirely. Only a
// backend that ingests RGB and CSCs it to 4:4:4 itself can use that: today just direct-NVENC
// (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring).
//
// The display's HDR state is deliberately NOT part of this answer, and no longer needs to be:
// both depths have a full-chroma source now, so the chroma resolved here — before the Welcome —
// is the chroma the stream really carries. (It used to be a lie whenever the display was HDR:
// this returned true, the Welcome promised 4:4:4, and the capturer then quietly emitted P010.)
encoder_ingests_rgb_444
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
@@ -609,7 +612,10 @@ pub fn open_portal_monitor(
/// 10-bit PQ/BT.2020 formats instead of the SDR set — pass it only when the output was actually
/// brought up HDR (a gamescope spawned with `--hdr-enabled` off our `pipewire-hdr` build); the
/// host resolves that in `capture::capturer_supports_hdr_for` **before** the Welcome, because a
/// session that negotiated PQ cannot fall back to SDR afterwards.
/// session that negotiated PQ cannot fall back to SDR afterwards. `cursor_id0_hides` declares the
/// producer's cursor-meta contract — pass it for outputs whose compositor rewrites
/// `SPA_META_Cursor` on every buffer (KWin), where an `id == 0` meta is an authoritative
/// "pointer hidden" the composited/forwarded cursor must honor.
#[cfg(target_os = "linux")]
#[allow(clippy::too_many_arguments)]
pub fn open_virtual_output(
@@ -622,6 +628,7 @@ pub fn open_virtual_output(
want_hdr: bool,
policy: ZeroCopyPolicy,
expect_exact_dims: bool,
cursor_id0_hides: bool,
) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::from_virtual_output(
remote_fd,
@@ -633,6 +640,7 @@ pub fn open_virtual_output(
want_hdr && !hdr_capture_failed(HdrSource::VirtualOutput),
policy,
expect_exact_dims,
cursor_id0_hides,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+14 -1
View File
@@ -72,6 +72,11 @@ struct CaptureOpts {
/// the doomed birth mode. `false` everywhere else (Mutter SIZES the monitor from negotiation and
/// gamescope fixates its own — gating those would starve legitimate first frames).
expect_exact_dims: bool,
/// The producer rewrites `SPA_META_Cursor` on EVERY buffer, so an `id == 0` meta is an
/// authoritative "pointer hidden / off this output" the blend must honor (KWin). `false` for
/// the stale-meta producers (Mutter recycles buffers without rewriting the region) — see
/// [`pw_cursor::CursorState::id0_hides`](pw_cursor) for the full contract.
cursor_id0_hides: bool,
}
/// The shared state the PipeWire thread PUBLISHES and the capturer READS — one struct instead of
@@ -301,6 +306,10 @@ impl PortalCapturer {
want_444: false,
want_hdr,
expect_exact_dims: false,
// The portal-monitor path today is Mutter (the GNOME HDR mirror) — the stale-meta
// id-0 contract. A KDE portal capture would rewrite per buffer, but nothing routes
// one through here yet; the virtual-output path below carries the real flag.
cursor_id0_hides: false,
},
policy,
)?
@@ -316,7 +325,8 @@ impl PortalCapturer {
/// the GPU zero-copy path subject to `PUNKTFUNK_ZEROCOPY`. `want_444` (a 4:4:4 session) makes the
/// zero-copy worker convert tiled dmabufs to planar YUV444 on the GPU instead of NV12/RGB.
/// `want_hdr` runs the 10-bit PQ/BT.2020 offer instead of the SDR set — see
/// [`crate::open_virtual_output`] for who is allowed to pass it.
/// [`crate::open_virtual_output`] for who is allowed to pass it. `cursor_id0_hides` declares
/// the producer's cursor-meta contract ([`CaptureOpts::cursor_id0_hides`]).
#[allow(clippy::too_many_arguments)]
pub fn from_virtual_output(
remote_fd: Option<OwnedFd>,
@@ -328,6 +338,7 @@ impl PortalCapturer {
want_hdr: bool,
policy: ZeroCopyPolicy,
expect_exact_dims: bool,
cursor_id0_hides: bool,
) -> Result<PortalCapturer> {
tracing::info!(
node_id,
@@ -335,6 +346,7 @@ impl PortalCapturer {
want_444,
want_hdr,
expect_exact_dims,
cursor_id0_hides,
"connecting PipeWire to virtual output"
);
// Most virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise
@@ -350,6 +362,7 @@ impl PortalCapturer {
want_444,
want_hdr,
expect_exact_dims,
cursor_id0_hides,
},
policy,
)?
+2 -1
View File
@@ -811,6 +811,7 @@ pub fn pipewire_thread(
want_444,
want_hdr,
expect_exact_dims,
cursor_id0_hides,
..
} = opts;
crate::pwinit::ensure_init();
@@ -985,7 +986,7 @@ pub fn pipewire_thread(
yuv444: want_444,
linear_nv12_failed: false,
dbg_log_n: 0,
cursor: CursorState::default(),
cursor: CursorState::new(cursor_id0_hides),
expect_dims: if expect_exact_dims {
preferred.map(|(w, h, _)| (w, h))
} else {
+75 -8
View File
@@ -39,9 +39,23 @@ pub(super) struct CursorState {
/// negotiated). Per-stream deliberately — a host serves many sessions per process, and a
/// process-wide latch made the second session's triage read as "no meta".
seen_meta: bool,
/// This stream's producer rewrites the cursor meta on EVERY buffer, so an `id == 0` meta is
/// an authoritative "pointer hidden / off this output" rather than a stale recycled region.
/// True for KWin virtual outputs; false for the stale-meta producers (Mutter) — see
/// [`note_cursor_id`].
id0_hides: bool,
}
impl CursorState {
/// The per-stream state, declaring which `id == 0` contract the producer follows
/// ([`Self::id0_hides`]).
pub(super) fn new(id0_hides: bool) -> CursorState {
CursorState {
id0_hides,
..CursorState::default()
}
}
/// A shareable overlay for the encode/forward paths, or `None` before the first bitmap
/// arrived. A HIDDEN pointer still yields `Some` (with `visible: false`): the
/// cursor-forward channel needs "known but hidden" — an app grabbed the pointer, the
@@ -79,6 +93,31 @@ pub(super) fn decode_bitmap_pixel(vfmt: u32, s: &[u8]) -> (u8, u8, u8, u8) {
}
}
/// Apply one parsed `spa_meta_cursor.id` to the visibility state; returns whether the rest of the
/// meta region (position, bitmap) is worth parsing.
///
/// Two producer contracts meet on `id == 0`. **KWin** rewrites the cursor meta on EVERY enqueued
/// buffer, and writes id 0 whenever `Cursor::isOnOutput` says the pointer is not in this stream —
/// which covers a globally hidden cursor AND a client null-cursor surface (empty cursor geometry
/// intersects nothing). There id 0 is the authoritative hide, and honoring it is what lets a game
/// or Big Picture hide the pointer mid-stream ([`CursorState::id0_hides`], set for KWin virtual
/// outputs; without it the composited arrow outlived every hide — the 0.22.0 field report).
/// **Mutter** only rewrites a buffer's meta region when the cursor changed, so recycled buffers
/// between damage frames carry a stale id-0 meta — treating that as hidden flickered the cursor
/// off between hovers (on-glass round 5). There the last-known state holds, and a pointer that
/// really left/hid simply stops producing updates (the M3 hidden hint has no Mutter signal —
/// Windows has its own CURSOR_SUPPRESSED source).
fn note_cursor_id(cursor: &mut CursorState, id: u32) -> bool {
if id == 0 {
if cursor.id0_hides {
cursor.visible = false;
}
return false;
}
cursor.visible = true;
true
}
/// Update `cursor` from the newest buffer's `SPA_META_Cursor` (no-op when the buffer carries no
/// cursor meta — producer doesn't support it, or the portal isn't in Metadata cursor mode).
/// Called for EVERY dequeued buffer, before the stale-frame skip, so pointer-only movements
@@ -121,16 +160,9 @@ pub(super) fn update_cursor_meta(cursor: &mut CursorState, spa_buf: *mut spa::sy
(*cur).bitmap_offset,
)
};
if id == 0 {
// SPA contract: id 0 = "no cursor information", NOT "cursor hidden". Mutter only
// REWRITES a buffer's meta region when the cursor changed, so recycled buffers
// between damage frames carry a stale id-0 meta — treating that as hidden flickered
// the cursor off between hovers (on-glass round 5). Keep the last-known state; a
// pointer that really left/hid simply stops producing updates. (The M3 hidden hint
// loses its Mutter signal — Windows has its own CURSOR_SUPPRESSED source.)
if !note_cursor_id(cursor, id) {
return;
}
cursor.visible = true;
cursor.x = pos_x - hot_x;
cursor.y = pos_y - hot_y;
cursor.hot_x = hot_x;
@@ -367,9 +399,44 @@ mod tests {
hot_x: 0,
hot_y: 0,
seen_meta: true,
id0_hides: false,
}
}
// ---- note_cursor_id: the two producer id-0 contracts --------------------------------------
#[test]
fn id_zero_hides_only_on_a_rewriting_producer() {
// KWin contract (`id0_hides`): id 0 is written fresh on every buffer, so it IS the hide —
// a game or Big Picture hiding the pointer must reach the stream.
let mut kwin = cursor(10, 10, 8, 8, (255, 255, 255), 255);
kwin.id0_hides = true;
assert!(!note_cursor_id(&mut kwin, 0), "id 0 parses no further");
let o = kwin.overlay().expect("bitmap stays cached across a hide");
assert!(!o.visible, "KWin id 0 must hide the overlay");
// The pointer coming back re-shows the SAME cached bitmap.
assert!(note_cursor_id(&mut kwin, 1));
assert!(kwin.overlay().expect("still cached").visible);
// Mutter contract: recycled buffers carry stale id-0 metas — the last-known state holds
// (honoring them flickered the cursor off between hovers, on-glass round 5).
let mut mutter = cursor(10, 10, 8, 8, (255, 255, 255), 255);
assert!(!note_cursor_id(&mut mutter, 0));
assert!(
mutter.overlay().expect("cached").visible,
"a stale-meta producer's id 0 must NOT hide"
);
}
#[test]
fn id_zero_before_any_bitmap_yields_no_overlay() {
// A KWin stream whose pointer was never on the output: hides arrive before any bitmap —
// `overlay()` must stay `None` (nothing to blend), not a phantom empty cursor.
let mut c = CursorState::new(true);
assert!(!note_cursor_id(&mut c, 0));
assert!(c.overlay().is_none());
}
// ---- bitmap_extent: the guard whose absence SIGSEGVs uncatchably -------------------------
#[test]
+137 -3
View File
@@ -44,8 +44,8 @@ use windows::Win32::Graphics::Direct3D11::{
D3D11_USAGE_IMMUTABLE, D3D11_USAGE_STAGING, D3D11_VIEWPORT,
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_P010, DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM,
DXGI_FORMAT_R16_UNORM, DXGI_SAMPLE_DESC,
DXGI_FORMAT, DXGI_FORMAT_P010, DXGI_FORMAT_R10G10B10A2_UNORM, DXGI_FORMAT_R16G16B16A16_FLOAT,
DXGI_FORMAT_R16G16_UNORM, DXGI_FORMAT_R16_UNORM, DXGI_SAMPLE_DESC,
};
/// How many times DXGI has actually called our hooked `NtGdiDdDDIGetCachedHybridQueryValue`.
@@ -362,11 +362,145 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
}
";
/// scRGB FP16 → **R10G10B10A2** (BT.2020 PQ, FULL-range RGB) — one full-res pass, the HDR twin of
/// the SDR 4:4:4 BGRA passthrough. Keeps full chroma all the way to the encoder: NVENC ingests the
/// packed 10-bit RGB (`NV_ENC_BUFFER_FORMAT_ABGR10`) and CSCs it to YUV **4:4:4** itself under
/// FREXT, per the BT.2020/PQ VUI the encoder writes — HEVC Main 4:4:4 10. Without this the HDR
/// path had only [`HdrP010Converter`], whose chroma pass subsamples, so a session that negotiated
/// 4:4:4 on an HDR display silently fell back to 4:2:0.
///
/// The colour math is [`HDR_P010_COMMON`]'s `scrgb_to_pq2020` verbatim — the SAME pixels the P010
/// luma pass starts from — so the two HDR outputs agree bit-for-bit before quantization. Only the
/// destination differs: no RGB→YUV, no studio-range squeeze and no chroma decimation here, just the
/// hardware's UNORM quantization of the PQ values into 10 bits per channel.
///
/// Channel order: DXGI `R10G10B10A2_UNORM` stores R in the low 10 bits, which is exactly what
/// NVENC calls `ABGR10` (it names A2B10G10R10 from the MSB down) — the same relationship the SDR
/// path relies on between DXGI `B8G8R8A8` and NVENC's `ARGB`. So the shader writes natural RGB
/// order and no swizzle is needed.
pub(crate) struct HdrRgb10Converter {
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
}
/// R10G10B10A2 pass PS — full-res, writes PQ-encoded BT.2020 RGB straight to the packed 10-bit
/// target. `saturate` is implicit in the UNORM render target; `scrgb_to_pq2020` already clamps.
const HDR_RGB10_PS: &str = r"
#include_common
float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
return float4(scrgb_to_pq2020(uv), 1.0);
}
";
impl HdrRgb10Converter {
pub(crate) fn new(device: &ID3D11Device) -> Result<Self> {
// SAFETY: every call is a `?`-checked D3D11 method on the live `device` borrow, over
// fully-initialized stack descriptors and live `Option` out-params; `compile_shader`
// receives `s!()` literals (its contract). Each created COM interface owns its own
// reference, and no raw pointer outlives the call that produced it.
unsafe {
let src = HDR_RGB10_PS.replace("#include_common", HDR_P010_COMMON);
let vsb = compile_shader(HDR_VS, s!("main"), s!("vs_5_0"))?;
let psb = compile_shader(&src, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps = None;
device.CreatePixelShader(&psb, None, Some(&mut ps))?;
// POINT, like the P010 luma pass: this is a 1:1 full-res resample, so every RT pixel
// maps to exactly one source texel centre and filtering would only blur it.
let sd = D3D11_SAMPLER_DESC {
Filter: D3D11_FILTER_MIN_MAG_MIP_POINT,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
ComparisonFunc: D3D11_COMPARISON_NEVER,
MaxLOD: f32::MAX,
..Default::default()
};
let mut sampler = None;
device.CreateSamplerState(&sd, Some(&mut sampler))?;
Ok(Self {
vs: vs.context("rgb10 vs")?,
ps: ps.context("rgb10 ps")?,
sampler: sampler.context("rgb10 sampler")?,
})
}
}
/// A plain (non-planar) RTV of the packed 10-bit output texture. Built once per out-ring slot,
/// like the P010 plane views — never per frame.
pub(crate) fn rtv(
device: &ID3D11Device,
dst: &ID3D11Texture2D,
) -> Result<ID3D11RenderTargetView> {
// SAFETY: one `?`-checked `CreateRenderTargetView` on the live `device` borrow, with a
// fully-initialized descriptor local whose address is taken only for the synchronous call,
// plus a live `Option` out-param.
unsafe {
let desc = D3D11_RENDER_TARGET_VIEW_DESC {
Format: DXGI_FORMAT_R10G10B10A2_UNORM,
ViewDimension: D3D11_RTV_DIMENSION_TEXTURE2D,
Anonymous: D3D11_RENDER_TARGET_VIEW_DESC_0 {
Texture2D: D3D11_TEX2D_RTV { MipSlice: 0 },
},
};
let mut rtv: Option<ID3D11RenderTargetView> = None;
device
.CreateRenderTargetView(
dst,
Some(&desc as *const D3D11_RENDER_TARGET_VIEW_DESC),
Some(&mut rtv),
)
.context("CreateRenderTargetView(R10G10B10A2 out slot)")?;
rtv.context("rgb10 rtv null")
}
}
/// Convert `src_srv` (FP16 scRGB, WxH) into the `R10G10B10A2` texture behind `rtv`.
pub(crate) fn convert(
&self,
ctx: &ID3D11DeviceContext,
src_srv: &ID3D11ShaderResourceView,
rtv: &ID3D11RenderTargetView,
w: u32,
h: u32,
) -> Result<()> {
// SAFETY: all D3D11 work runs on the caller's live `ctx` borrow (the owning capture
// thread's immediate context) over borrowed slices of fully-initialized locals and clones
// of the caller's live SRV/RTV. No raw pointers and no mapping on this path.
unsafe {
ctx.OMSetBlendState(None, None, 0xffff_ffff); // opaque overwrite
ctx.VSSetShader(&self.vs, None);
ctx.PSSetShaderResources(0, Some(&[Some(src_srv.clone())]));
ctx.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
ctx.IASetInputLayout(None);
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
let vp = D3D11_VIEWPORT {
TopLeftX: 0.0,
TopLeftY: 0.0,
Width: w as f32,
Height: h as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
};
ctx.RSSetViewports(Some(&[vp]));
ctx.OMSetRenderTargets(Some(&[Some(rtv.clone())]), None);
ctx.PSSetShader(&self.ps, None);
ctx.Draw(3, 0);
// Unbind for the next frame's re-RTV / NVENC read.
ctx.OMSetRenderTargets(Some(&[None]), None);
ctx.PSSetShaderResources(0, Some(&[None]));
Ok(())
}
}
}
/// scRGB FP16 → **P010** (BT.2020 PQ, 10-bit limited/studio range) conversion, in OUR OWN shader (two
/// passes: full-res luma + half-res chroma). NVIDIA's D3D11 VideoProcessor cannot do RGB→P010 (renders
/// green), so we quantize to studio-range 10-bit YUV directly and feed NVENC native P010 — skipping
/// NVENC's internal RGB→YUV CSC (which runs on the contended SM). One per capture device (rebuilt on
/// device recreate).
/// device recreate). The 4:4:4 twin is [`HdrRgb10Converter`].
///
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
+81 -43
View File
@@ -20,8 +20,8 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::dxgi::{
make_device, BgraToYuvPlanes, D3d11Frame, HdrP010Converter, PyroFrameShare, VideoConverter,
WinCaptureTarget,
make_device, BgraToYuvPlanes, D3d11Frame, HdrP010Converter, HdrRgb10Converter, PyroFrameShare,
VideoConverter, WinCaptureTarget,
};
use super::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{bail, Context, Result};
@@ -44,8 +44,8 @@ use windows::Win32::Graphics::Direct3D11::{
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_NV12, DXGI_FORMAT_P010,
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM, DXGI_FORMAT_R16_UNORM,
DXGI_FORMAT_R8G8_UNORM, DXGI_FORMAT_R8_UNORM, DXGI_SAMPLE_DESC,
DXGI_FORMAT_R10G10B10A2_UNORM, DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM,
DXGI_FORMAT_R16_UNORM, DXGI_FORMAT_R8G8_UNORM, DXGI_FORMAT_R8_UNORM, DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{
CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4, IDXGIKeyedMutex, IDXGIResource1,
@@ -178,6 +178,9 @@ struct OutSlot {
/// `(luma R16_UNORM, chroma R16G16_UNORM)` plane views. `None` for NV12/BGRA outputs, which the
/// video processor or a plain `CopyResource` writes without an RTV of ours.
p010: Option<(ID3D11RenderTargetView, ID3D11RenderTargetView)>,
/// Plain RTV of the packed 10-bit slot, for the HDR + 4:4:4 output ([`HdrRgb10Converter`]).
/// `None` for every other format.
rgb10: Option<ID3D11RenderTargetView>,
}
/// One PyroWave output-ring slot: the two SEPARATE shareable plane textures the wavelet encoder
@@ -438,8 +441,9 @@ pub struct IddPushCapturer {
/// THROUGH (a plain copy into the out ring, no NV12 VideoConverter) so NVENC gets full-chroma
/// RGB and CSCs to 4:4:4 itself — measured on-glass: `chromaFormatIDC=3` + ARGB input yields
/// TRUE 4:4:4 and the conversion follows the VUI matrix (BT.709 limited, always written).
/// While the display is HDR this is overridden to the P010 path (no 10-bit 4:4:4 source):
/// the stream honestly downgrades to 4:2:0 — the encoder's caps cross-check reports it.
/// While the display is HDR the same idea runs at 10 bits: [`HdrRgb10Converter`] writes packed
/// BT.2020 PQ RGB and NVENC CSCs that to YUV 4:4:4 (Main 4:4:4 10). Either way the chroma the
/// Welcome promised is the chroma the wire carries.
want_444: bool,
/// A PyroWave (wavelet) session (design/pyrowave-windows-host-zerocopy.md +
/// design/pyrowave-444-hdr.md). When set, frames come from the separate-plane `pyro_ring`
@@ -521,10 +525,15 @@ pub struct IddPushCapturer {
/// SDR — keeps the colour-convert OFF the contended 3D/compute engine. Built lazily; rebuilt on a
/// size/HDR flip.
video_conv: Option<VideoConverter>,
/// FP16 scRGB slot → P010 (BT.2020 PQ limited) via two shader passes, used while the display is HDR
/// FP16 scRGB slot → P010 (BT.2020 PQ limited) via two shader passes, used while the display is
/// HDR and the session did NOT negotiate 4:4:4 (that case takes [`Self::hdr_rgb10_conv`])
/// (NVIDIA's VideoProcessor can't do RGB→P010). The passes run on the 3D engine, but it still skips
/// NVENC's internal SM-side CSC. Built lazily.
hdr_p010_conv: Option<HdrP010Converter>,
/// FP16 scRGB slot → packed 10-bit BT.2020 PQ RGB, used while the display is HDR **and** the
/// session negotiated 4:4:4 — the full-chroma twin of [`Self::hdr_p010_conv`]. Rebuilt with the
/// ring on a mode/HDR flip.
hdr_rgb10_conv: Option<HdrRgb10Converter>,
last_seq: u64,
last_present: Option<(ID3D11Texture2D, PixelFormat)>,
status_logged: bool,
@@ -649,8 +658,10 @@ impl IddPushCapturer {
/// SM-side CSC, because the video processor can only produce subsampled output). We do NOT
/// gate HDR on the client's advertised `VIDEO_CAP_10BIT` — clients under-report it (e.g. the
/// Mac advertises 10-bit only when its OWN display is HDR), yet all decode Main10 +
/// auto-switch, exactly as on the WGC path. HDR wins over 4:4:4 (there is no 10-bit
/// full-chroma source): the stream downgrades to 4:2:0 with a warning.
/// auto-switch, exactly as on the WGC path. HDR and 4:4:4 now COMPOSE: an HDR display that
/// negotiated full chroma emits packed 10-bit BT.2020 PQ RGB (`Rgb10a2`) for NVENC to CSC to
/// YUV 4:4:4 — HEVC Main 4:4:4 10. (Before, HDR won and the stream silently downgraded to
/// 4:2:0 *after* the Welcome had already promised 4:4:4.)
fn out_format(&self) -> (DXGI_FORMAT, PixelFormat) {
// PyroWave never uses this out-ring (it has its own separate-plane `pyro_ring`); the
// format here only labels the frame. SDR sessions label NV12 (BT.709 limited), HDR
@@ -664,7 +675,10 @@ impl IddPushCapturer {
}
if self.display_hdr {
if self.want_444 {
warn_444_hdr_downgrade_once();
// HDR + full chroma: packed 10-bit RGB (BT.2020 PQ), which NVENC CSCs to YUV
// 4:4:4 itself — the HDR twin of the SDR BGRA passthrough below. No subsampling
// anywhere on this path (see `HdrRgb10Converter`).
return (DXGI_FORMAT_R10G10B10A2_UNORM, PixelFormat::Rgb10a2);
}
(DXGI_FORMAT_P010, PixelFormat::P010)
} else if self.want_444 {
@@ -798,6 +812,7 @@ impl IddPushCapturer {
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
self.hdr_p010_conv = None;
self.hdr_rgb10_conv = None;
// The PyroWave CSC is mode-baked too (BgraToYuvPlanes picks different SDR vs HDR shaders
// and R8/R8G8 vs R16/R16G16 outputs). Without this, a display_hdr flip (Downgrade point D:
// client_10bit=true but HDR couldn't enable at open) reused the stale SDR converter against
@@ -981,7 +996,12 @@ impl IddPushCapturer {
} else {
None
};
self.out_ring.push(OutSlot { tex, p010 });
let rgb10 = if format == DXGI_FORMAT_R10G10B10A2_UNORM {
Some(HdrRgb10Converter::rtv(&self.device, &tex)?)
} else {
None
};
self.out_ring.push(OutSlot { tex, p010, rgb10 });
}
}
Ok(())
@@ -1076,7 +1096,13 @@ impl IddPushCapturer {
/// SDR display, or the FP16→P010 shader on an HDR display. Both keep NVENC's RGB→YUV CSC off the SM.
/// An SDR 4:4:4 session needs NO converter — the BGRA slot passes through (see `out_format`).
fn ensure_converter(&mut self) -> Result<()> {
if self.display_hdr {
if self.display_hdr && self.want_444 {
// HDR + full chroma: one full-res pass to packed 10-bit BT.2020 PQ RGB; NVENC does
// the RGB→YUV444 CSC (there is nothing to subsample, so no second pass).
if self.hdr_rgb10_conv.is_none() {
self.hdr_rgb10_conv = Some(HdrRgb10Converter::new(&self.device)?);
}
} else if self.display_hdr {
if self.hdr_p010_conv.is_none() {
self.hdr_p010_conv = Some(HdrP010Converter::new(
&self.device,
@@ -1537,7 +1563,7 @@ impl IddPushCapturer {
self.ensure_out_ring()?;
self.ensure_converter()?;
let s = &self.out_ring[i];
(Some((s.tex.clone(), s.p010.clone())), None)
(Some((s.tex.clone(), s.p010.clone(), s.rgb10.clone())), None)
};
let (_, pf) = self.out_format();
let ring_len = if self.pyrowave {
@@ -1584,11 +1610,20 @@ impl IddPushCapturer {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
conv.convert(&self.context, src, y_rtv, cbcr_rtv, self.width, self.height)?;
}
} else if self.display_hdr && self.want_444 {
// HDR 4:4:4: FP16 slot SRV → packed 10-bit BT.2020 PQ RGB; NVENC ingests it as
// ABGR10 and CSCs to YUV 4:4:4 under FREXT (HEVC Main 4:4:4 10).
if let Some(conv) = self.hdr_rgb10_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
let (_, _, rtv) = out.as_ref().expect("out ring");
let rtv = rtv.as_ref().expect("Rgb10a2 out slot has an RTV");
conv.convert(&self.context, src, rtv, self.width, self.height)?;
}
} else if self.display_hdr {
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
if let Some(conv) = self.hdr_p010_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
let (_, rtvs) = out.as_ref().expect("out ring");
let (_, rtvs, _) = out.as_ref().expect("out ring");
// The slot's P010 plane views, built once in `ensure_out_ring`.
let (y_rtv, uv_rtv) = rtvs.as_ref().expect("P010 out slot has plane RTVs");
conv.convert(&self.context, src, y_rtv, uv_rtv, self.width, self.height)?;
@@ -1635,6 +1670,22 @@ impl IddPushCapturer {
// the running correlated/total tally — lives on `StallWatch` (sweep Phase 5.4). It was
// ~65 lines of log prose inside `try_consume`, which is the hot loop, and its two
// counters were capturer fields that nothing else touched.
// One ETW read serves both evidence fields: the prose summary spans the gap plus
// the same 300 ms lead-in the report's OS-event correlation uses (the disturbance
// that CAUSED the hole lands just before it), while the discriminator counts span
// the GAP ONLY — no lead-in: presents from the healthy flow right before the hole
// would falsely acquit the content (the stall-ending frame's own present lands at
// the window edge and stays well under the acquit bar). Both halves must come from
// the same ring snapshot under the same clock anchor, or the prose and the verdict
// can disagree about the same hole.
let (etw, etw_counts) = self
.etw
.as_ref()
.and_then(|w| {
now.checked_sub(stall.gap)
.map(|from| w.window_report(from, now, Duration::from_millis(300)))
})
.unzip();
let evidence = StallEvidence {
// A publisher re-attach restarts `offered_total` near zero; a ring recreate resets
// the stall watch before that can matter, but guard the delta anyway (a restarted
@@ -1647,24 +1698,14 @@ impl IddPushCapturer {
}
}),
max_heartbeat_age_ms: self.max_hb_age_us / 1_000,
// The probe + ETW reads span the same window the report's OS-event correlation
// uses (the gap plus a lead-in for the disturbance that CAUSED it).
// The probe read spans the same window the report's OS-event correlation uses
// (the gap plus a lead-in for the disturbance that CAUSED it).
probes: now
.checked_sub(stall.gap + Duration::from_millis(300))
.zip(self.probes.as_deref())
.map(|(from, p)| p.window(from, now)),
etw: self.etw.as_ref().and_then(|w| {
now.checked_sub(stall.gap + Duration::from_millis(300))
.map(|from| w.summary(from, now))
}),
// The discriminator counts span the GAP ONLY — no lead-in: presents from the
// healthy flow right before the hole would falsely acquit the content. The
// stall-ending frame's own present lands at the window edge and stays well
// under the acquit bar.
etw_counts: self.etw.as_ref().and_then(|w| {
now.checked_sub(stall.gap)
.map(|from| w.window_counts(from, now))
}),
etw,
etw_counts,
};
self.stall_watch.report(&stall, now, &evidence);
}
@@ -2008,21 +2049,6 @@ impl Capturer for IddPushCapturer {
}
}
/// A 4:4:4 session while the display is HDR: there is no 10-bit full-chroma source (the FP16
/// desktop needs the PQ tone curve, which the P010 shader provides at 4:2:0), so the stream
/// honestly downgrades — the encoder's `chroma_444` caps cross-check reports it and the in-band
/// SPS keeps the client decoding correctly. Once per process: the state can flap mid-session.
fn warn_444_hdr_downgrade_once() {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
tracing::warn!(
"4:4:4 negotiated but the display is HDR — no 10-bit full-chroma source exists; \
encoding HDR 4:2:0 (P010) instead (disable HDR on the virtual display for 4:4:4)"
);
}
}
impl Drop for IddPushCapturer {
fn drop(&mut self) {
// A channel session ending while the secure-desktop guard is engaged must not leave the
@@ -2433,6 +2459,18 @@ mod tests {
),
StallClass::ContentSilence
);
// A LIVE witness (history true = it demonstrably worked just before the hole) reading
// an exact zero is the strongest content conviction — the zero is a measurement, not
// an absence.
assert_eq!(
classify(
gap,
&StallVerdict::ComposeSilence,
Some(&probes(Some(16_000), Some(20_000), Some(30_000))),
Some(&counts(0, 0))
),
StallClass::ContentSilence
);
// The present witness does NOT overrule the driver's own verdicts or the harder
// classes — it only refines compose-silence.
assert_eq!(
@@ -18,7 +18,7 @@
//! A second provider rides the same session: `Microsoft-Windows-DXGI` (user-mode), filtered to
//! `Present`/`PresentMultiplaneOverlay` starts (ids 42/55) — one event per swapchain present,
//! stamped with the PRESENTING process id. Together they are the compose-silence discriminator
//! ([`EtwWatch::window_counts`]): DXGI presents flowing while `BltQueueAddEntry` gaps = the OS
//! ([`EtwWatch::window_report`]): DXGI presents flowing while `BltQueueAddEntry` gaps = the OS
//! display path dropped composed frames (the real display-path bug); BOTH silent = the content
//! stopped presenting (benign pause — menus/loading/game hitch). The predecessor witnesses are
//! retired for cause: DxgKrnl id 184 `Present` never fires on the modern redirected path, and
@@ -48,7 +48,8 @@ use windows::Win32::System::Diagnostics::Etw::{
EVENT_CONTROL_CODE_ENABLE_PROVIDER, EVENT_FILTER_DESCRIPTOR, EVENT_FILTER_TYPE_EVENT_ID,
EVENT_RECORD, EVENT_TRACE_CONTROL_STOP, EVENT_TRACE_LOGFILEW, EVENT_TRACE_PROPERTIES,
EVENT_TRACE_REAL_TIME_MODE, PROCESSTRACE_HANDLE, PROCESS_TRACE_MODE_EVENT_RECORD,
PROCESS_TRACE_MODE_REAL_TIME, TRACE_LEVEL_INFORMATION, WNODE_FLAG_TRACED_GUID,
PROCESS_TRACE_MODE_RAW_TIMESTAMP, PROCESS_TRACE_MODE_REAL_TIME, TRACE_LEVEL_INFORMATION,
WNODE_FLAG_TRACED_GUID,
};
use windows::Win32::System::Performance::{QueryPerformanceCounter, QueryPerformanceFrequency};
use windows::Win32::System::Threading::{
@@ -122,8 +123,13 @@ fn qpc_freq() -> i64 {
})
}
/// The consumer's per-event callback — record id + QPC timestamp (the session's `ClientContext`
/// is 1, so `TimeStamp` IS a QPC value) and return; runs on the consumer thread.
/// The consumer's per-event callback — record id + timestamp + pid into the ring and return;
/// runs on the consumer thread. `TimeStamp` is a raw QPC value only because BOTH halves of the
/// clock contract hold: `ClientContext = 1` makes QPC the session clock, and the consumer is
/// opened with `PROCESS_TRACE_MODE_RAW_TIMESTAMP`, which is what stops ProcessTrace converting
/// every event's timestamp to FILETIME (100 ns units since 1601) on delivery. Without the flag
/// the conversion happens REGARDLESS of the session clock, and every `ts <= to_q` comparison
/// downstream is against the wrong clock — never true, a witness that silently reads empty.
unsafe extern "system" fn on_event(record: *mut EVENT_RECORD) {
if record.is_null() {
return;
@@ -150,10 +156,10 @@ pub(super) struct EtwWatch {
}
// SAFETY: both fields are plain kernel handle VALUES (u64 wrappers) owned by this watch; every
// operation on them (summary reads the static ring; Drop stops/closes) is thread-safe by the ETW
// API contract, and the singleton hands out only `Arc<EtwWatch>`.
// operation on them (window_report reads the static ring; Drop stops/closes) is thread-safe by
// the ETW API contract, and the singleton hands out only `Arc<EtwWatch>`.
unsafe impl Send for EtwWatch {}
// SAFETY: as above — `&EtwWatch` exposes only `summary` (static-ring reads).
// SAFETY: as above — `&EtwWatch` exposes only `window_report` (static-ring reads).
unsafe impl Sync for EtwWatch {}
static WATCH: Mutex<Weak<EtwWatch>> = Mutex::new(Weak::new());
@@ -201,8 +207,10 @@ impl EtwWatch {
let mut session = CONTROLTRACE_HANDLE::default();
// SAFETY: `buf` is a live, zeroed allocation of base + name bytes; every write below is a
// field of the properties struct at its head; `LoggerNameOffset = base` points at the
// appended name space (ETW copies the name there itself). ClientContext 1 = QPC clock —
// what makes event timestamps comparable to our probe windows.
// appended name space (ETW copies the name there itself). ClientContext 1 selects QPC as
// the SESSION clock — necessary but not sufficient for QPC comparisons: ProcessTrace
// still converts every event's timestamp to FILETIME on delivery unless the consumer is
// opened with PROCESS_TRACE_MODE_RAW_TIMESTAMP (set below).
let rc = unsafe {
let props = buf.as_mut_ptr().cast::<EVENT_TRACE_PROPERTIES>();
(*props).Wnode.BufferSize = buf.len() as u32;
@@ -224,6 +232,11 @@ impl EtwWatch {
);
return None;
}
// A fresh session gets a fresh ring: the static [`RING`] outlives any `EtwWatch`, so
// whatever is in it belongs to a DEAD session — leaking it forward would let a previous
// session's presents pose as this session's witness history. Race-free here: the
// consumer thread that repopulates it is spawned below.
RING.lock().unwrap().clear();
// Enable DxgKrnl with a kernel-side event-id filter — the whole point: the provider's
// vblank/DPC keywords never reach us. Fatal on failure (the DDI families + queue
@@ -240,7 +253,7 @@ impl EtwWatch {
return None;
}
// The DXGI (user-mode) present witness rides the same session. Degraded-not-fatal: a
// refusal only costs the per-process present counts — `window_counts` then reports
// refusal only costs the per-process present counts — `window_report` then reports
// no present history and classification stays honest (Unattributed, never a guess).
if !enable_provider(session, &DXGI, &DXGI_FILTER_IDS) {
tracing::debug!(
@@ -252,8 +265,12 @@ impl EtwWatch {
LoggerName: PWSTR(name.as_ptr() as *mut _),
..Default::default()
};
log.Anonymous1.ProcessTraceMode =
PROCESS_TRACE_MODE_REAL_TIME | PROCESS_TRACE_MODE_EVENT_RECORD;
// RAW_TIMESTAMP is load-bearing: it stops ProcessTrace converting `EVENT_HEADER.TimeStamp`
// to FILETIME on delivery, so events arrive stamped in the session clock (QPC, per the
// ClientContext above) — the only clock the window edges are computed in.
log.Anonymous1.ProcessTraceMode = PROCESS_TRACE_MODE_REAL_TIME
| PROCESS_TRACE_MODE_EVENT_RECORD
| PROCESS_TRACE_MODE_RAW_TIMESTAMP;
log.Anonymous2.EventRecordCallback = Some(on_event);
// SAFETY: `log` is a fully-initialized local; `name` outlives the call (OpenTrace copies
// what it needs before returning).
@@ -303,14 +320,34 @@ impl EtwWatch {
Some(Self { session, consumer })
}
/// Summarize the DDI activity inside `[from, to]` — the correlation line a stall report
/// carries. Brackets that merely SPAN the window count too (a freeze-long `SetPowerState`
/// has both edges outside the hole it caused). `"none"` when the window is clean.
pub(super) fn summary(&self, from: Instant, to: Instant) -> String {
// Instant → QPC: anchor both clocks now and offset backwards.
/// One stall window's ETW evidence, both halves from a SINGLE ring snapshot under a SINGLE
/// `(Instant::now(), qpc_now())` anchor: the DDI/present prose summary a stall report
/// carries, and the structured discriminator counts the classifier folds in. The summary
/// covers `[hole_from - lead_in, hole_to]` — the disturbance that CAUSED a hole lands just
/// before DWM stops delivering, so the prose needs the lead-in. The counts cover
/// `[hole_from, hole_to]` ONLY — presents from the healthy flow inside the lead-in would
/// falsely acquit the content. Two separate reads (two locks, two anchors, syscalls in
/// between) would let events arriving between them make the prose and the verdict disagree
/// about the same hole — hence one method returning both.
///
/// Brackets that merely SPAN the summary window count too (a freeze-long `SetPowerState`
/// has both edges outside the hole it caused). The summary reads `"none"` when the window
/// is clean.
pub(super) fn window_report(
&self,
hole_from: Instant,
hole_to: Instant,
lead_in: Duration,
) -> (String, EtwWindowCounts) {
// Instant → QPC: anchor both clocks once and offset backwards; every window edge below
// derives from this one anchor.
let (now_i, now_q, freq) = (Instant::now(), qpc_now(), qpc_freq());
let to_q = now_q - duration_qpc(now_i.saturating_duration_since(to), freq);
let from_q = now_q - duration_qpc(now_i.saturating_duration_since(from), freq);
let to_q = now_q - duration_qpc(now_i.saturating_duration_since(hole_to), freq);
let from_q = now_q - duration_qpc(now_i.saturating_duration_since(hole_from), freq);
let summary_from_q = from_q - duration_qpc(lead_in, freq);
// One snapshot, then the lock drops: everything below — including the OpenProcess
// syscalls behind `process_name` — runs off the copy, so the consumer callback never
// queues behind a stall report.
let events: Vec<(i64, u16, u32)> = {
let ring = RING.lock().unwrap();
ring.iter()
@@ -318,6 +355,7 @@ impl EtwWatch {
.copied()
.collect()
};
let counts = count_window(&events, from_q, to_q, duration_qpc(LOOKBACK, freq));
let ms = |dq: i64| dq.max(0) * 1_000 / freq;
let mut parts = Vec::new();
for (start_id, stop_id, label) in [
@@ -335,7 +373,7 @@ impl EtwWatch {
} else if id == stop_id {
if let Some(s) = open.take() {
// The bracket [s, ts] counts when it intersects the window.
if s <= to_q && ts >= from_q {
if s <= to_q && ts >= summary_from_q {
count += 1;
max_ms = max_ms.max(ms(ts - s));
}
@@ -362,43 +400,34 @@ impl EtwWatch {
] {
let count = events
.iter()
.filter(|(ts, i, _)| *i == id && *ts >= from_q && *ts <= to_q)
.filter(|(ts, i, _)| *i == id && *ts >= summary_from_q && *ts <= to_q)
.count();
if count > 0 {
parts.push(format!("{label}×{count}"));
}
}
// Present + queue accounting (DXGI 42/55 + BltQueueAddEntry/Complete): total presents
// inside the window plus the top presenters, NAMED — the line that splits a
// inside the summary window plus the top presenters, NAMED — the line that splits a
// compose-silence hole into "the content stopped presenting" (no presents anywhere)
// versus "presents flowed and the display path dropped them" (presents at rate while
// the queue starves). "Present×0" is printed explicitly when the stream has history
// but the window is empty — silence is a finding, not an absence.
// the queue starves). "Present×0" is printed explicitly when the witness was LIVE
// before the hole ([`LOOKBACK`]) but the window is empty — silence is a finding, not
// an absence; a dead witness's window prints nothing rather than a fake zero.
let mut per_pid: Vec<(u32, u32)> = Vec::new();
let mut have_present_history = false;
let (mut adds, mut completes) = (0u32, 0u32);
let mut have_queue_history = false;
for &(ts, id, pid) in &events {
if ts < summary_from_q || ts > to_q {
continue;
}
match id {
DXGI_PRESENT_ID | DXGI_PRESENT_MPO_ID => {
have_present_history = true;
if ts >= from_q && ts <= to_q {
match per_pid.iter_mut().find(|(p, _)| *p == pid) {
Some((_, c)) => *c += 1,
None => per_pid.push((pid, 1)),
}
}
}
BLT_ADD_ID | BLT_COMPLETE_ID => {
have_queue_history = true;
if ts >= from_q && ts <= to_q {
if id == BLT_ADD_ID {
adds += 1;
} else {
completes += 1;
}
match per_pid.iter_mut().find(|(p, _)| *p == pid) {
Some((_, c)) => *c += 1,
None => per_pid.push((pid, 1)),
}
}
BLT_ADD_ID => adds += 1,
BLT_COMPLETE_ID => completes += 1,
_ => {}
}
}
@@ -415,61 +444,80 @@ impl EtwWatch {
.collect::<Vec<_>>()
.join(",");
parts.push(format!("Present×{total}({top})"));
} else if have_present_history {
} else if counts.present_history {
parts.push("Present×0".to_string());
}
if have_queue_history {
if counts.queue_history || adds > 0 || completes > 0 {
parts.push(format!("blt-queue add×{adds} complete×{completes}"));
}
if parts.is_empty() {
let summary = if parts.is_empty() {
"none".to_string()
} else {
parts.join(" ")
}
}
/// The structured discriminator read for `[from, to]` (the stall classifier's evidence):
/// how many swapchain presents (DXGI 42/55, any process) and how many virtual-display
/// queue entries (`BltQueueAddEntry`) landed in the window, plus whether each stream has
/// EVER produced an event (distinguishing a true zero from a witness that is not working —
/// e.g. the DXGI enable was refused, or an OS build renumbered the BltQueue events).
pub(super) fn window_counts(&self, from: Instant, to: Instant) -> EtwWindowCounts {
let (now_i, now_q, freq) = (Instant::now(), qpc_now(), qpc_freq());
let to_q = now_q - duration_qpc(now_i.saturating_duration_since(to), freq);
let from_q = now_q - duration_qpc(now_i.saturating_duration_since(from), freq);
let ring = RING.lock().unwrap();
let mut out = EtwWindowCounts::default();
for &(ts, id, _) in ring.iter() {
match id {
DXGI_PRESENT_ID | DXGI_PRESENT_MPO_ID => {
out.present_history = true;
if ts >= from_q && ts <= to_q {
out.presents += 1;
}
}
BLT_ADD_ID => {
out.queue_history = true;
if ts >= from_q && ts <= to_q {
out.queue_adds += 1;
}
}
_ => {}
}
}
out
};
(summary, counts)
}
}
/// [`EtwWatch::window_counts`]'s read: the compose-silence discriminator's structured evidence.
/// Witness-liveness lookback: the [`EtwWindowCounts`] history flags are true only when the
/// stream produced at least one event inside the `LOOKBACK` window ENDING at the hole's start.
/// "Ever produced an event" would be wrong in both directions: an event that arrived only AFTER
/// the hole (the resume burst, the stall-ending frame) proves nothing about whether the witness
/// was working DURING it, and a provider that died mid-session (or whose events aged out of the
/// ring) would keep flying a stale known-working flag forever. Demonstrated life immediately
/// BEFORE the hole is the claim the classifier actually needs; 5 s is far longer than any
/// pre-stall active-flow gate, so a genuinely working witness cannot blink false across a
/// frame-time lull.
const LOOKBACK: Duration = Duration::from_secs(5);
/// The discriminator's windowing math, factored pure (plain i64 QPC-tick arithmetic, no ETW,
/// no clock reads) so the ring→counts contract is unit-testable without a session: presents
/// (DXGI 42/55, any process) and queue entries (`BltQueueAddEntry`) inside `[from_q, to_q]`,
/// witness liveness from `[from_q - lookback_q, from_q]` (see [`LOOKBACK`]). A
/// `BltQueueCompleteIndirectPresent` proves the queue witness works exactly as an add does —
/// both ride the same provider enable — so either satisfies `queue_history`.
fn count_window(
events: &[(i64, u16, u32)],
from_q: i64,
to_q: i64,
lookback_q: i64,
) -> EtwWindowCounts {
let mut out = EtwWindowCounts::default();
for &(ts, id, _) in events {
let in_window = ts >= from_q && ts <= to_q;
let in_lookback = ts >= from_q.saturating_sub(lookback_q) && ts <= from_q;
match id {
DXGI_PRESENT_ID | DXGI_PRESENT_MPO_ID => {
out.present_history |= in_lookback;
if in_window {
out.presents += 1;
}
}
BLT_ADD_ID => {
out.queue_history |= in_lookback;
if in_window {
out.queue_adds += 1;
}
}
BLT_COMPLETE_ID => out.queue_history |= in_lookback,
_ => {}
}
}
out
}
/// [`EtwWatch::window_report`]'s structured half: the compose-silence discriminator's evidence.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
pub(super) struct EtwWindowCounts {
/// Swapchain presents (any process — the game AND dwm both count) inside the window.
pub(super) presents: u32,
/// `BltQueueAddEntry` events (frames entering the virtual display's kernel queue) inside it.
pub(super) queue_adds: u32,
/// The present stream has produced at least one event EVER (witness known-working).
/// The present stream demonstrated liveness inside [`LOOKBACK`] BEFORE the hole opened — a
/// working witness whose in-window zero is a reading, not a dead one whose zero is noise.
pub(super) present_history: bool,
/// The queue stream has produced at least one event EVER (witness known-working).
/// Queue-stream liveness inside [`LOOKBACK`] before the hole (`BltQueueAddEntry` or
/// `BltQueueCompleteIndirectPresent` — either proves the witness works).
pub(super) queue_history: bool,
}
@@ -561,3 +609,72 @@ impl Drop for EtwWatch {
}
}
}
// The module only compiles on Windows (lib.rs gates `mod windows`), so plain `cfg(test)` here
// already means "Windows tests" — and [`count_window`] itself is pure tick math, no session.
#[cfg(test)]
mod tests {
use super::*;
/// [`count_window`]'s contract: counts come from the hole window `[from, to]`; liveness
/// comes ONLY from the lookback window ending at the hole's start. An event after the hole
/// (the resume burst) or older than the lookback (a dead provider's leftovers) must not fly
/// the known-working flag — those are exactly the shapes that used to convict every
/// compose-silence hole as content.
#[test]
fn count_window_liveness_and_windowing() {
// Hole [1000, 2000], lookback 500 → liveness window [500, 1000]. Plain ticks.
let (from, to, lb) = (1_000i64, 2_000i64, 500i64);
let ev = |ts: i64, id: u16| (ts, id, 42u32);
// The healthy shape: liveness demonstrated before the hole, activity inside it.
let events = [
ev(600, DXGI_PRESENT_ID), // lookback → present witness live
ev(700, BLT_COMPLETE_ID), // lookback → queue witness live (completes count)
ev(1_100, DXGI_PRESENT_ID), // in-window present
ev(1_200, DXGI_PRESENT_MPO_ID), // in-window present (MPO path)
ev(1_300, BLT_ADD_ID), // in-window queue add
ev(1_400, 430), // non-witness id: never counted here
];
assert_eq!(
count_window(&events, from, to, lb),
EtwWindowCounts {
presents: 2,
queue_adds: 1,
present_history: true,
queue_history: true,
}
);
// In-window events count but do NOT confer liveness — the witness must have worked
// BEFORE the hole for its zeros elsewhere to mean anything.
let window_only = [ev(1_500, DXGI_PRESENT_ID), ev(1_600, BLT_ADD_ID)];
let c = count_window(&window_only, from, to, lb);
assert_eq!((c.presents, c.queue_adds), (1, 1));
assert!(!c.present_history && !c.queue_history);
// An event only AFTER the hole proves nothing about the witness during it.
let after_only = [ev(2_100, DXGI_PRESENT_ID), ev(2_200, BLT_ADD_ID)];
assert_eq!(
count_window(&after_only, from, to, lb),
EtwWindowCounts::default()
);
// Events that aged past the lookback (a previous session's leftovers) don't either.
let stale = [ev(499, DXGI_PRESENT_ID), ev(1, BLT_ADD_ID)];
assert_eq!(
count_window(&stale, from, to, lb),
EtwWindowCounts::default()
);
// Both lookback edges are inclusive; the hole-start event is both liveness and count.
let edges = [ev(500, DXGI_PRESENT_ID), ev(1_000, BLT_ADD_ID)];
let c = count_window(&edges, from, to, lb);
assert!(c.present_history && c.queue_history);
assert_eq!((c.presents, c.queue_adds), (0, 1));
// A lookback reaching below tick 0 saturates instead of wrapping.
let c = count_window(&[ev(0, DXGI_PRESENT_ID)], 3, to, i64::MAX);
assert!(c.present_history);
}
}
@@ -644,6 +644,7 @@ impl IddPushCapturer {
out_idx: 0,
video_conv: None,
hdr_p010_conv: None,
hdr_rgb10_conv: None,
last_seq: 0,
last_present: None,
status_logged: false,
@@ -57,7 +57,7 @@ pub(super) struct StallEvidence {
/// The DxgKrnl DDI activity inside the window (Phase A.3 ETW summary); `None` when the
/// session is unavailable (non-admin dev run).
pub(super) etw: Option<String>,
/// The structured present-vs-queue counts for the window ([`EtwWatch::window_counts`]) —
/// The structured present-vs-queue counts for the window ([`EtwWatch::window_report`]) —
/// the compose-silence discriminator: presents flowing while the queue starves = the OS
/// display path dropped composed frames; both silent = the content stopped presenting.
/// `None` when the ETW session is unavailable.
+2
View File
@@ -68,6 +68,8 @@ windows = { git = "https://github.com/microsoft/windows-rs", rev = "acb5a1a74410
"d3dcommon",
"dxgi",
"handleapi",
# RECT/HMONITOR for DXGI_OUTPUT_DESC1 (the display-HDR volume query).
"windef",
# IDXGIResource1::CreateSharedHandle takes an optional SECURITY_ATTRIBUTES.
"minwinbase",
# The GlobalAlloc block the clipboard takes ownership of (clipboard.rs).
+112 -22
View File
@@ -14,9 +14,12 @@ use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
const SAMPLE_RATE: u32 = 48_000;
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
/// Mic capture is MONO: voice is mono at the source, the host accepts any Opus channel
/// layout (its stereo decoder upmixes), and half the samples halve the encode + wire cost.
const MIC_CHANNELS: usize = 1;
/// Mic frames are 10 ms (480 mono samples) — any size ≤ 120 ms is fine host-side; 10 ms
/// halves the frame-fill share of mouth-to-ear latency vs the old 20 ms.
const MIC_FRAME: usize = 480;
struct Terminate;
@@ -327,20 +330,31 @@ fn pw_thread(
Ok(())
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks,
/// ship them as 0xCB datagrams into the host's virtual PipeWire source.
/// The microphone uplink: capture the default input device (or the picked / echo-cancelled
/// source), Opus-encode 10 ms mono chunks, ship them as 0xCB datagrams into the host's
/// virtual PipeWire source.
pub struct MicStreamer {
quit_tx: pipewire::channel::Sender<Terminate>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
/// `muted` is the in-stream mute (B4), shared live with the capture callback: set, the
/// callback keeps pulling and discarding whole frames but sends nothing. Muting by
/// STOPPING the stream was rejected — it re-primes the device buffers and re-runs the
/// source selection below on every unmute, so the first second back is glitchy.
///
/// `echo_cancel` is the Settings toggle; `PUNKTFUNK_NO_AEC=1` overrides it off.
pub fn spawn(
connector: Arc<NativeClient>,
muted: Arc<std::sync::atomic::AtomicBool>,
echo_cancel: bool,
) -> Result<MicStreamer> {
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, quit_rx) {
if let Err(e) = mic_thread(&connector, quit_rx, muted, echo_cancel) {
tracing::warn!(error = %e, "mic uplink thread ended");
}
})
@@ -361,19 +375,76 @@ impl Drop for MicStreamer {
}
}
/// Capture-side state: accumulated PCM and the Opus encoder (encoding a 20 ms frame is
/// ~100 µs — fine inside the process callback).
/// Capture-side state: accumulated PCM and the Opus encoder (encoding a 10 ms frame is
/// well under 100 µs — fine inside the process callback).
struct MicData {
connector: Arc<NativeClient>,
ring: VecDeque<f32>,
encoder: opus::Encoder,
seq: u32,
out: Vec<u8>,
/// The in-stream mute (B4), flipped by the session's chord. Read per callback.
muted: Arc<std::sync::atomic::AtomicBool>,
}
/// Whether the mic echo-cancellation hooks run this session: the `echo_cancel` setting, with
/// `PUNKTFUNK_NO_AEC=1` as a one-way override OFF. The env var wins — it is the escape hatch
/// for a box whose canceller misbehaves, and it predates the setting; nothing turns AEC back
/// on once it is set. Here the hook is the echo-cancelled-source preference below; the WASAPI
/// twin gates its Communications stream category the same way.
fn aec_enabled(echo_cancel: bool) -> bool {
echo_cancel && !std::env::var("PUNKTFUNK_NO_AEC").is_ok_and(|v| !v.is_empty() && v != "0")
}
/// The capture stream's `target.object`, in preference order: the Settings microphone pick
/// (`Settings::mic_device` via session main's `PUNKTFUNK_AUDIO_SOURCE`) verbatim, else — so a
/// desktop that already runs `module-echo-cancel` stops feeding its own downlink audio back
/// into the host's virtual mic — the first echo-cancelled source in the graph. `None` = the
/// user picked nothing and no such source exists: PipeWire's default routing, as before.
///
/// Preference-only by design: loading `libpipewire-module-echo-cancel` ourselves needs
/// `pw_context_load_module`, which the pipewire crate (0.9) doesn't expose safely — until it
/// does, we only ever target processing the user (or their session) already set up.
fn mic_capture_target(echo_cancel: bool) -> Option<String> {
if let Ok(target) = std::env::var("PUNKTFUNK_AUDIO_SOURCE") {
if !target.is_empty() {
return Some(target);
}
}
if !aec_enabled(echo_cancel) {
return None;
}
let name = echo_cancel_source()?;
tracing::info!(
source = %name,
"mic capture targets the echo-cancelled source (Echo cancellation off, or \
PUNKTFUNK_NO_AEC=1, disables this)"
);
Some(name)
}
/// Find an existing echo-cancelled capture node: the first `Audio/Source` whose `node.name`
/// or description says echo-cancel (`module-echo-cancel`'s convention — `echo-cancel-*`
/// nodes, "Echo-Cancel …" descriptions; PulseAudio-compat setups match too). One registry
/// roundtrip via [`devices`]; any failure reads as "none".
fn echo_cancel_source() -> Option<String> {
let (_, sources) = devices().ok()?;
sources.into_iter().find_map(|d| {
let name = d.name.to_ascii_lowercase();
let desc = d.description.to_ascii_lowercase();
(name.contains("echo-cancel")
|| name.contains("echo_cancel")
|| desc.contains("echo-cancel")
|| desc.contains("echo cancel"))
.then_some(d.name)
})
}
fn mic_thread(
connector: &Arc<NativeClient>,
quit_rx: pipewire::channel::Receiver<Terminate>,
muted: Arc<std::sync::atomic::AtomicBool>,
echo_cancel: bool,
) -> Result<()> {
use pipewire as pw;
use pw::{properties::properties, spa};
@@ -384,9 +455,14 @@ fn mic_thread(
PW_INIT.call_once(pw::init);
let mut encoder =
opus::Encoder::new(SAMPLE_RATE, opus::Channels::Stereo, opus::Application::Voip)
opus::Encoder::new(SAMPLE_RATE, opus::Channels::Mono, opus::Application::Voip)
.map_err(|e| anyhow::anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
// Voice tuning: 48 kbps mono is transparent for speech; in-band FEC + an assumed 10 %
// loss let the host's decoder rebuild a lost 0xCB datagram from its successor instead
// of concealing (datagrams are fire-and-forget — this FEC is the only redundancy).
let _ = encoder.set_bitrate(opus::Bitrate::Bits(48_000));
let _ = encoder.set_inband_fec(true);
let _ = encoder.set_packet_loss_perc(10);
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw mic MainLoop")?;
let context = pw::context::ContextRc::new(&mainloop, None).context("pw mic Context")?;
@@ -405,14 +481,15 @@ fn mic_thread(
*pw::keys::MEDIA_ROLE => "Communication",
*pw::keys::NODE_NAME => "punktfunk-mic-capture",
*pw::keys::NODE_DESCRIPTION => "Punktfunk Microphone",
// ~10 ms quantum (one mic frame). Without it the capture stream inherits the graph
// quantum — commonly 10242048 samples, so the mic arrived in 2143 ms bursts that
// sat ahead of the encoder as latency (the playback stream always asked for 5 ms).
*pw::keys::NODE_LATENCY => "480/48000",
};
// The Settings microphone pick (`Settings::mic_device` via session main).
if let Ok(target) = std::env::var("PUNKTFUNK_AUDIO_SOURCE") {
if !target.is_empty() {
// Raw key: the `keys::TARGET_OBJECT` constant is feature-gated on a newer
// libpipewire than we require; the wire name is stable.
props.insert("target.object", target);
}
if let Some(target) = mic_capture_target(echo_cancel) {
// Raw key: the `keys::TARGET_OBJECT` constant is feature-gated on a newer
// libpipewire than we require; the wire name is stable.
props.insert("target.object", target);
}
let stream = pw::stream::StreamBox::new(&core, "punktfunk-mic-capture", props)
.context("pw mic Stream")?;
@@ -423,6 +500,7 @@ fn mic_thread(
encoder,
seq: 0,
out: vec![0u8; 4000],
muted,
};
let _listener = stream
@@ -447,9 +525,20 @@ fn mic_thread(
.push_back(f32::from_le_bytes([s[0], s[1], s[2], s[3]]));
}
}
// Ship every complete 20 ms stereo frame.
while ud.ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ud.ring.drain(..MIC_FRAME * CHANNELS).collect();
// Muted (B4): the stream stays open and the device keeps its primed buffers —
// only the sending stops. Whole frames are discarded so the ring can't grow,
// and `seq` deliberately does NOT advance: the host sees one continuous
// sequence with a silent pause in the middle rather than a gap the size of the
// mute, which its de-jitter would try to conceal frame by frame.
if ud.muted.load(std::sync::atomic::Ordering::Relaxed) {
let whole =
(ud.ring.len() / (MIC_FRAME * MIC_CHANNELS)) * (MIC_FRAME * MIC_CHANNELS);
ud.ring.drain(..whole);
return;
}
// Ship every complete 10 ms mono frame.
while ud.ring.len() >= MIC_FRAME * MIC_CHANNELS {
let pcm: Vec<f32> = ud.ring.drain(..MIC_FRAME * MIC_CHANNELS).collect();
match ud.encoder.encode_float(&pcm, &mut ud.out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
@@ -473,7 +562,8 @@ fn mic_thread(
let mut info = AudioInfoRaw::new();
info.set_format(AudioFormat::F32LE);
info.set_rate(SAMPLE_RATE);
info.set_channels(CHANNELS as u32);
// Mono: the stream's adapter downmixes whatever layout the source really has.
info.set_channels(MIC_CHANNELS as u32);
let obj = pw::spa::pod::Object {
type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(),
id: pw::spa::param::ParamType::EnumFormat.as_raw(),
+185 -29
View File
@@ -23,14 +23,107 @@ use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
use wasapi::{
AudioClientProperties, DeviceEnumerator, Direction, SampleType, StreamCategory, StreamMode,
WaveFormat,
};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
/// Mic capture requests STEREO from WASAPI (autoconvert matrixes any endpoint layout down to
/// it — the proven path; `read_from_device_to_deque` then delivers our requested format) and
/// downmixes to MONO in code before the encoder: voice is mono at the source, the host accepts
/// any Opus channel layout (its stereo decoder upmixes), and half the samples halve the
/// encode + wire cost. The render path is multichannel — its channel count + block align are
/// runtime, driven by the host-resolved layout.
const CAPT_CHANNELS: usize = 2;
/// Mic frames are 10 ms (480 mono samples) — any size ≤ 120 ms is fine host-side; 10 ms
/// halves the frame-fill share of mouth-to-ear latency vs the old 20 ms.
const MIC_FRAME: usize = 480;
/// A selectable WASAPI endpoint for the settings pickers.
#[derive(Clone, Debug)]
pub struct AudioDevice {
/// The `IMMDevice` endpoint id (`{0.0.0.00000000}.{…}`) — the stable key the render and
/// capture threads resolve via [`DeviceEnumerator::get_device`]. (The PipeWire twin
/// stores `node.name` here; both are "the stable key", so the Settings fields and env
/// contract stay OS-agnostic.)
pub name: String,
/// The endpoint's friendly name ("Speakers (Realtek …)") — what the picker shows.
pub description: String,
}
/// Enumerate active audio endpoints: `(sinks, sources)` — the WASAPI twin of the PipeWire
/// probe (same tuple shape; no devices → the caller simply shows no pickers). Runs on its
/// own short-lived MTA thread: the caller is typically a UI thread whose COM apartment is
/// STA, where a direct `CoInitializeEx(MTA)` would fail with `RPC_E_CHANGED_MODE`.
pub fn devices() -> Result<(Vec<AudioDevice>, Vec<AudioDevice>)> {
std::thread::Builder::new()
.name("pf-audio-enum".into())
.spawn(|| -> Result<(Vec<AudioDevice>, Vec<AudioDevice>)> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let enumerator = DeviceEnumerator::new().context("DeviceEnumerator")?;
let mut out = (Vec::new(), Vec::new());
for (direction, list) in [
(Direction::Render, &mut out.0),
(Direction::Capture, &mut out.1),
] {
let coll = enumerator
.get_device_collection(&direction)
.context("device collection")?;
for i in 0..coll.get_nbr_devices().context("device count")? {
// One broken endpoint (driver limbo) must not hide the rest.
let Ok(dev) = coll.get_device_at_index(i) else {
continue;
};
let (Ok(id), Ok(name)) = (dev.get_id(), dev.get_friendlyname()) else {
continue;
};
list.push(AudioDevice {
name: id,
description: name,
});
}
}
Ok(out)
})
.context("spawn audio enumeration thread")?
.join()
.map_err(|_| anyhow!("audio enumeration thread panicked"))?
}
/// The endpoint an env pick names (`PUNKTFUNK_AUDIO_SINK`/`SOURCE` — endpoint ids, the
/// Settings device pickers via session main), or the OS default. A picked device that's
/// gone (unplugged USB DAC, remote session) falls back to the default with a warning —
/// audio keeps working, like the PipeWire twin's `target.object` behavior.
fn pick_device(
enumerator: &DeviceEnumerator,
direction: &Direction,
var: &str,
) -> Result<wasapi::Device> {
if let Some(id) = std::env::var(var).ok().filter(|v| !v.is_empty()) {
match enumerator.get_device(&id) {
Ok(d) => {
tracing::info!(
var,
endpoint = %d.get_friendlyname().unwrap_or_else(|_| id.clone()),
"using the picked audio endpoint"
);
return Ok(d);
}
Err(e) => tracing::warn!(
var,
endpoint_id = %id,
error = %e,
"picked audio endpoint not found — using the default"
),
}
}
enumerator
.get_default_device(direction)
.context("default endpoint")
}
pub struct AudioPlayer {
pcm_tx: SyncSender<Vec<f32>>,
@@ -66,7 +159,8 @@ impl AudioPlayer {
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!(channels, "WASAPI render: 48 kHz f32 (default endpoint)");
// Default endpoint unless PUNKTFUNK_AUDIO_SINK picked one (logged there).
tracing::info!(channels, "WASAPI render: 48 kHz f32");
Ok(AudioPlayer {
pcm_tx,
recycle_rx,
@@ -124,10 +218,9 @@ fn render_thread(
// F32LE interleaved: channels × 4 bytes/sample. Stereo (channels == 2) is byte-identical
// to the old fixed path (mask 0x3, block align 8).
let block_align = channels as usize * 4;
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let enumerator = DeviceEnumerator::new().context("DeviceEnumerator")?;
let device = pick_device(&enumerator, &Direction::Render, "PUNKTFUNK_AUDIO_SINK")
.context("render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// The explicit dwChannelMask is the wire order (FL FR FC LFE RL RR SL SR); 5.1 = 0x3F,
// 7.1 = 0x63F. WASAPI delivers channels in ascending mask-bit order, which equals the wire
@@ -217,21 +310,31 @@ fn render_thread(
res
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks, ship
/// them as 0xCB datagrams into the host's virtual mic source.
/// The microphone uplink: capture the default input device, Opus-encode 10 ms mono chunks,
/// ship them as 0xCB datagrams into the host's virtual mic source.
pub struct MicStreamer {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
/// `muted` is the in-stream mute (B4), shared live with the capture loop: set, the loop
/// keeps reading the endpoint and discarding whole frames but sends nothing. Muting by
/// STOPPING the client was rejected — an `IAudioClient` stop/start re-primes the endpoint
/// buffers and re-runs the category negotiation below on every unmute.
///
/// `echo_cancel` is the Settings toggle; `PUNKTFUNK_NO_AEC=1` overrides it off.
pub fn spawn(
connector: Arc<NativeClient>,
muted: Arc<AtomicBool>,
echo_cancel: bool,
) -> Result<MicStreamer> {
let stop = Arc::new(AtomicBool::new(false));
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
if let Err(e) = mic_thread(&connector, stop_t, muted, echo_cancel) {
tracing::warn!(error = %format!("{e:#}"), "mic uplink thread ended");
}
})
@@ -252,25 +355,58 @@ impl Drop for MicStreamer {
}
}
fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()> {
/// Whether the mic echo-cancellation hooks run this session: the `echo_cancel` setting, with
/// `PUNKTFUNK_NO_AEC=1` as a one-way override OFF. The env var wins — it is the escape hatch
/// for a box whose canceller misbehaves, and it predates the setting; nothing turns AEC back
/// on once it is set. Here the hook is the Communications stream category below; the PipeWire
/// twin gates its echo-cancelled-source preference the same way.
fn aec_enabled(echo_cancel: bool) -> bool {
echo_cancel && !std::env::var("PUNKTFUNK_NO_AEC").is_ok_and(|v| !v.is_empty() && v != "0")
}
fn mic_thread(
connector: &Arc<NativeClient>,
stop: Arc<AtomicBool>,
muted: Arc<AtomicBool>,
echo_cancel: bool,
) -> Result<()> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let mut encoder = opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Channels::Mono,
opus::Application::Voip,
)
.map_err(|e| anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
// Voice tuning: 48 kbps mono is transparent for speech; in-band FEC + an assumed 10 %
// loss let the host's decoder rebuild a lost 0xCB datagram from its successor instead
// of concealing (datagrams are fire-and-forget — this FEC is the only redundancy).
let _ = encoder.set_bitrate(opus::Bitrate::Bits(48_000));
let _ = encoder.set_inband_fec(true);
let _ = encoder.set_packet_loss_perc(10);
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Capture)
.context("default capture endpoint (no microphone?)")?;
let enumerator = DeviceEnumerator::new().context("DeviceEnumerator")?;
let device = pick_device(&enumerator, &Direction::Capture, "PUNKTFUNK_AUDIO_SOURCE")
.context("capture endpoint (no microphone?)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
// Communications category → the endpoint's communications signal-processing chain. A
// driver/APO stack with an echo canceller only engages it for communications-category
// streams; the default (Other) category never did, so the downlink audio playing on
// this box fed straight back into the host's virtual mic. Must precede Initialize
// (SetClientProperties is a pre-init call; the wasapi crate QIs IAudioClient2 inside).
// Best-effort: an endpoint without IAudioClient2 just keeps the default category.
// The "Echo cancellation" setting opts out, and PUNKTFUNK_NO_AEC=1 overrides that off
// (same lever as the Linux echo-cancel-source preference) — see `aec_enabled`.
if aec_enabled(echo_cancel) {
if let Err(e) = audio_client.set_properties(
AudioClientProperties::new().set_category(StreamCategory::Communications),
) {
tracing::debug!(error = %e, "mic capture: Communications category not set");
}
}
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CAPT_CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
@@ -308,13 +444,33 @@ fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / 4) * 4;
for c in bytes.drain(..whole).collect::<Vec<u8>>().chunks_exact(4) {
ring.push_back(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
// One stereo capture frame (8 bytes) → one mono sample: average L/R. Autoconvert
// already matrixed the endpoint's real layout (mono/stereo/array mic) into the
// stereo stream we initialized, so this is the only downmix left to do.
let stereo_frame = 4 * CAPT_CHANNELS;
let whole = (bytes.len() / stereo_frame) * stereo_frame;
for c in bytes
.drain(..whole)
.collect::<Vec<u8>>()
.chunks_exact(stereo_frame)
{
let l = f32::from_le_bytes([c[0], c[1], c[2], c[3]]);
let r = f32::from_le_bytes([c[4], c[5], c[6], c[7]]);
ring.push_back((l + r) * 0.5);
}
// Ship every complete 20 ms stereo frame.
while ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME * CHANNELS).collect();
// Muted (B4): the capture client stays started and keeps its primed buffers — only
// the sending stops. Whole frames are discarded so the ring can't grow, and `seq`
// deliberately does NOT advance: the host sees one continuous sequence with a silent
// pause in the middle rather than a gap the size of the mute, which its de-jitter
// would try to conceal frame by frame.
if muted.load(Ordering::Relaxed) {
let drop_n = (ring.len() / MIC_FRAME) * MIC_FRAME;
ring.drain(..drop_n);
continue;
}
// Ship every complete 10 ms mono frame.
while ring.len() >= MIC_FRAME {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME).collect();
match encoder.encode_float(&pcm, &mut out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
+1 -5
View File
@@ -366,11 +366,7 @@ pub fn resolve_host(link: &DeepLink, known: &KnownHosts) -> HostResolution {
.then(|| parse_addr_port(&link.host_ref))
.flatten();
for candidate in [literal.clone(), link.host.clone()].into_iter().flatten() {
if let Some(i) = known
.hosts
.iter()
.position(|h| h.addr == candidate.0 && h.port == candidate.1)
{
if let Some(i) = known.index_by_addr(&candidate.0, candidate.1) {
return HostResolution::Known(i);
}
}
+196 -8
View File
@@ -285,6 +285,21 @@ fn set_valve_hidapi(enabled: bool) {
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAM", v);
}
/// Disable the Valve HIDAPI drivers **before SDL exists** — call this alongside the other
/// pre-`SDL_Init` hints, not after a subsystem is up.
///
/// The damage these drivers do happens at *enumeration*, which is part of initialising the
/// joystick/gamepad subsystem. Setting the hint afterwards does detach the driver, but only after
/// it has already sent the Deck its `ID_CLEAR_DIGITAL_MAPPINGS` + `TRACKPAD_NONE` — so the
/// built-in trackpad-mouse dies system-wide and stays dead until the firmware watchdog restores
/// lizard mode seconds later. The threaded worker ([`run`]) has always done this in the right
/// order; the caller-pumped path could not, because by the time it receives a
/// [`sdl3::GamepadSubsystem`] the enumeration has already happened. Hence a separate entry point
/// its callers can put in the right place.
pub fn preinit_disable_valve_hidapi() {
set_valve_hidapi(false);
}
/// Map the SDL-reported controller type to the virtual pad we'd ask the host to create.
fn pref_for_type(t: sdl3::gamepad::GamepadType) -> GamepadPref {
use sdl3::gamepad::GamepadType as T;
@@ -336,6 +351,7 @@ enum Ctl {
Detach,
Pin(Option<String>),
KindOverride(GamepadPref),
Forwarding(bool),
MenuMode(bool),
MenuRumble(MenuPulse),
}
@@ -392,9 +408,12 @@ impl GamepadService {
/// and calls [`GamepadPump::tick`] once per loop iteration (the threaded worker's
/// per-wakeup work: ctl drain, chord-hold check, menu repeat, feedback).
///
/// Like the threaded worker, this disables the Valve HIDAPI drivers up front (their
/// mere enumeration kills the Deck's trackpad-mouse system-wide); they are enabled
/// for the duration of an attached session only.
/// The Valve HIDAPI drivers are held off here too, but this is **too late to be the only
/// place it happens**: the `subsystem` argument means enumeration is already done, and that
/// is when the Deck driver kills the trackpad-mouse. The caller must also call
/// [`preinit_disable_valve_hidapi`] with its other pre-`SDL_Init` hints. This call still
/// earns its place — it re-asserts "off" for a process that ran a session earlier — but on
/// its own it only detaches a driver that has already done the damage.
pub fn pumped(subsystem: sdl3::GamepadSubsystem) -> (GamepadService, GamepadPump) {
set_valve_hidapi(false);
let pads = Arc::new(Mutex::new(Vec::new()));
@@ -482,6 +501,26 @@ impl GamepadService {
let _ = self.ctl.send(Ctl::KindOverride(pref));
}
/// Forward this device's controllers to the host at all ([`Settings::gamepad_forwarding`],
/// default on). Off is for a couch whose pad reaches the host another way — a USB
/// passthrough tool like VirtualHere, or a controller plugged into the host itself —
/// where forwarding as well would give the host two pads for one pair of hands.
///
/// Off holds no slot open, so nothing is sent AND nothing is *grabbed*: no arrival, no
/// virtual pad host-side, and the hidraw node stays free for the passthrough tool to
/// bind (SDL's HIDAPI drivers take it at open — a held device cannot be bound away).
/// It follows that the escape chord, which only listens on forwarded pads, is not
/// available while off; the keyboard chord and the client's own UI still end a session.
///
/// Menu navigation is untouched: the launcher still opens the active pad to drive its
/// UI, and a session — which supersedes menu mode whether it forwards or not — releases
/// it again, so the pad is free for the whole time a stream is up.
///
/// [`Settings::gamepad_forwarding`]: crate::trust::Settings::gamepad_forwarding
pub fn set_forwarding(&self, on: bool) {
let _ = self.ctl.send(Ctl::Forwarding(on));
}
pub fn attach(&self, connector: Arc<NativeClient>) {
let _ = self.ctl.send(Ctl::Attach(connector));
}
@@ -535,6 +574,38 @@ impl GamepadPump {
self.worker.menu_poll();
self.worker.render_feedback();
}
/// Close every forwarded slot — flush its held wire state, tell the host to remove the pad,
/// and physically silence it. Call once on the way out of the caller's event loop.
///
/// [`GamepadService::detach`] only *posts* `Ctl::Detach`; the close — the flush, the host-side
/// `GamepadRemove`, and the explicit `set_rumble(0, 0)` backstop in `close_slot_at` — happens
/// when the pump next drains it. An exit path that detached and then left the loop without
/// another [`tick`](Self::tick) therefore skipped all of it, and nothing else would: the slots
/// hold no `Drop` that silences them. A pad left mid-buzz stayed buzzing.
///
/// This closes the slots directly rather than draining the queued `Ctl::Detach` that would
/// have done it. Same physical outcome by a shorter path, and deliberately so: this also runs
/// from `Drop`, and `drain_ctl` reaches `Mutex::lock().unwrap()`, which on a poisoned lock
/// would panic — during an unwind that aborts the process. Closing a slot touches no lock.
///
/// Idempotent, and safe with nothing attached.
pub fn shutdown(&mut self) {
self.worker.close_all_slots();
}
}
/// The silence backstop of last resort. A caller's loop can also leave by `?` on a fatal overlay
/// or present error — several paths do — and those would skip an explicit
/// [`shutdown`](GamepadPump::shutdown) entirely, leaving a forwarded pad buzzing on the way out.
///
/// Callers should still call `shutdown` at their normal exit rather than lean on this: the pad
/// wants to go quiet *before* a long teardown (session join, `vkDeviceWaitIdle`), not after it.
/// Doing both is free — `shutdown` is idempotent.
impl Drop for GamepadPump {
fn drop(&mut self) {
self.shutdown();
}
}
/// The lowest wire pad index (0..[`MAX_PADS`](punktfunk_core::input::MAX_PADS)) not already held
@@ -721,6 +792,10 @@ struct Worker {
/// connected pads, so it survives restarts and disconnects. A pin forwards ONLY that pad
/// (an explicit single-player choice); Automatic forwards every real controller.
pinned: Option<String>,
/// Forward controllers to an attached session at all ([`GamepadService::set_forwarding`]).
/// Off makes [`Self::forwarded_ids`] empty, so a session opens no slot — the whole point
/// being that the hardware stays ungrabbed for a USB passthrough tool.
forwarding: bool,
/// The user's explicit "controller type" setting ([`GamepadService::set_kind_override`]);
/// `Auto` = per-pad detection. Applied at slot open to the kind DECLARED to the host, never
/// to [`Slot::pref`] — the local feedback paths must keep reading the physical pad.
@@ -815,6 +890,11 @@ impl Worker {
/// back to the single most-recent pad when only a Steam-virtual pad is present (the Deck
/// game-mode case — otherwise its gyro/paddles/input would have nowhere to land).
fn forwarded_ids(&self) -> Vec<u32> {
// Forwarding off: nothing is forwarded, so nothing is opened either — the device stays
// free for whatever route the user's controller actually takes to the host.
if !self.forwarding {
return Vec::new();
}
if let Some(key) = &self.pinned {
if let Some(id) = self
.order
@@ -1243,10 +1323,16 @@ impl Worker {
Ok(Ctl::Attach(c)) => {
self.attached = Some(c);
self.reset_chord(); // every session starts un-latched (Attach doesn't flush)
// The Valve HIDAPI drivers run only in-session (see set_valve_hidapi);
// enabling them re-enumerates a Deck's built-in pad with paddles/
// trackpads/gyro first-class — sync_open opens a slot per forwarded pad.
set_valve_hidapi(true);
// The Valve HIDAPI drivers run only in-session (see set_valve_hidapi);
// enabling them re-enumerates a Deck's built-in pad with paddles/
// trackpads/gyro first-class — sync_open opens a slot per forwarded pad.
// Not with forwarding off: this session opens no slot, and the drivers'
// mere enumeration both kills the Deck's trackpad-mouse and is the
// opposite of leaving the hardware alone for a passthrough tool.
if self.forwarding {
set_valve_hidapi(true);
}
self.sync_open();
}
Ok(Ctl::Detach) => {
@@ -1269,6 +1355,31 @@ impl Worker {
self.refresh_active();
}
Ok(Ctl::KindOverride(pref)) => self.kind_override = pref,
Ok(Ctl::Forwarding(on)) => {
if self.forwarding == on {
continue;
}
self.forwarding = on;
self.reset_chord(); // no forwarded pad can be mid-chord across the flip
// Applied live rather than at attach only, so a mid-session flip (an
// in-stream settings screen) takes effect on the pad in your hands.
//
// The Valve HIDAPI drivers are an in-session-only thing (see
// set_valve_hidapi), and forwarding off is — for their purpose — not in
// session. Order matters and differs by direction: ON must enable them
// BEFORE `sync_open`, or a Deck's built-in pad opens under its old
// identity; OFF must disable them AFTER, so no slot outlives the driver
// that opened it.
let attached = self.attached.is_some();
if on && attached {
set_valve_hidapi(true);
}
self.sync_open();
if !on && attached {
set_valve_hidapi(false);
}
}
Ok(Ctl::MenuMode(on)) => {
self.menu_mode = on;
if on {
@@ -1565,6 +1676,11 @@ impl Worker {
HidOutput::PlayerLeds { bits, .. } if is_ds => {
let _ = slot.pad.send_effect(&Ds5Feedback::player_packet(bits));
}
// Every other pad with player LEDs gets them through SDL, which owns the
// per-device pattern. This used to fall through and do nothing at all.
HidOutput::PlayerLeds { bits, .. } => {
let _ = set_player_leds(&slot.pad, bits);
}
HidOutput::Trigger {
which, ref effect, ..
} if is_ds => {
@@ -1572,12 +1688,43 @@ impl Worker {
.pad
.send_effect(&Ds5Feedback::trigger_packet(which, effect));
}
_ => {}
// Deliberately unhandled, listed rather than left to a bare `_` so a new
// variant cannot join them silently: adaptive triggers exist only on a
// DualSense, and the trackpad-haptic / raw-passthrough planes are DS-specific
// and carried by `send_effect` above when the pad is one.
HidOutput::Trigger { .. }
| HidOutput::TrackpadHaptic { .. }
| HidOutput::HidRaw { .. } => {}
}
}
}
}
/// The SDL player index for the wire's positional player-LED `bits`, or `None` for "no player".
///
/// The wire carries a bitmask — one bit per LED, low 5 — while SDL wants a player *index* and owns
/// the per-device pattern. The count bridges them: every convention that reaches this wire spells
/// "player N" as N lit LEDs, both the DualSense patterns (`0x04`, `0x0A`, `0x15`, `0x1B`, `0x1F`)
/// and the Switch/XInput run of low bits (`0x01`, `0x03`, `0x07`, `0x0F`). SDL's index is 0-based,
/// so player 1 is index 0; no lit LED means *no* player rather than player 0.
///
/// Split out from [`set_player_leds`] so the mapping is testable — an `sdl3::Gamepad` needs a real
/// device, so nothing that takes one can be.
fn player_index_from_bits(bits: u8) -> Option<u16> {
match (bits & 0x1F).count_ones() {
0 => None,
n => Some((n - 1) as u16),
}
}
/// Drive a non-DualSense pad's player LEDs from the wire's positional `bits`.
fn set_player_leds(pad: &sdl3::gamepad::Gamepad, bits: u8) -> Result<(), sdl3::Error> {
match player_index_from_bits(bits) {
None => pad.unset_player_index(),
Some(i) => pad.set_player_index(i),
}
}
/// The wire pad index a [`HidOutput`] is addressed to (every variant carries `pad`).
fn hidout_pad(h: &HidOutput) -> u8 {
match h {
@@ -1608,6 +1755,7 @@ impl Worker {
menu_open: None,
order: Vec::new(),
pinned: None,
forwarding: true,
kind_override: GamepadPref::Auto,
attached: None,
escape_tx,
@@ -1946,3 +2094,43 @@ mod slot_tests {
);
}
}
#[cfg(test)]
mod player_led_tests {
use super::*;
/// Both conventions that reach this wire spell "player N" as N lit LEDs, so the count is the
/// player number regardless of WHICH bits a given pad lights. Pinned because the mapping is
/// otherwise only obvious once you have seen both patterns side by side.
#[test]
fn player_index_counts_lit_leds_for_both_conventions() {
// DualSense / hid-playstation patterns — non-contiguous, symmetric about the centre LED.
assert_eq!(player_index_from_bits(0x04), Some(0)); // player 1
assert_eq!(player_index_from_bits(0x0A), Some(1)); // player 2
assert_eq!(player_index_from_bits(0x15), Some(2)); // player 3
assert_eq!(player_index_from_bits(0x1B), Some(3)); // player 4
assert_eq!(player_index_from_bits(0x1F), Some(4)); // player 5
// Switch/XInput style — a contiguous run of low bits, the same count each time.
assert_eq!(player_index_from_bits(0x01), Some(0));
assert_eq!(player_index_from_bits(0x03), Some(1));
assert_eq!(player_index_from_bits(0x07), Some(2));
assert_eq!(player_index_from_bits(0x0F), Some(3));
}
/// No lit LED is "no player", NOT player 0 — the difference between LEDs off and player 1 lit.
#[test]
fn no_lit_led_is_no_player() {
assert_eq!(player_index_from_bits(0x00), None);
// Only the low 5 bits are player LEDs; junk above them must not invent a player.
assert_eq!(player_index_from_bits(0xE0), None);
}
/// The mask is applied before counting, so out-of-range bits cannot inflate the index past
/// the 5 real LEDs.
#[test]
fn high_bits_are_masked_off_before_counting() {
assert_eq!(player_index_from_bits(0xFF), Some(4)); // 0x1F worth of LEDs, not 8
assert_eq!(player_index_from_bits(0xE4), Some(0)); // 0x04 with junk on top
}
}
+11 -3
View File
@@ -132,9 +132,7 @@ impl ConnectPlan {
// exactly the right thing: no profile binding, no clipboard opt-in.
let fallback = KnownHost::default();
let stored = known
.hosts
.iter()
.find(|h| h.addr == host.addr && h.port == host.port)
.find_by_addr(&host.addr, host.port)
.unwrap_or(&fallback);
let mut plan = ConnectPlan::resolve(
stored,
@@ -230,6 +228,12 @@ impl ConnectPlan {
if self.settings.fullscreen_on_stream {
args.push("--fullscreen".into());
}
// Deliberately NO `--window-pos` here. The Windows shell appends its own (its
// window's desktop coordinates place the session on the same monitor), but on
// Wayland neither GTK can read global window coordinates nor can SDL apply
// them — the compositor owns placement — so from the GTK/CLI spawners the flag
// would be a silent no-op everywhere it matters. X11 could carry it, but a
// Linux-only special case that most Linux sessions ignore isn't worth the drift.
args
}
}
@@ -978,6 +982,10 @@ mod tests {
height: 1440,
bitrate_kbps: 55000,
codec: "av1".into(),
present_priority: "smooth".into(),
smooth_buffer: 2,
vsync: false,
allow_vrr: false,
..Default::default()
},
clipboard: true,
+183
View File
@@ -62,6 +62,8 @@ pub struct SettingsOverlay {
#[serde(skip_serializing_if = "Option::is_none")]
pub mic_enabled: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub echo_cancel: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub touch_mode: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub mouse_mode: Option<String>,
@@ -72,9 +74,23 @@ pub struct SettingsOverlay {
#[serde(skip_serializing_if = "Option::is_none")]
pub gamepad: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub gamepad_forwarding: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub stats_verbosity: Option<StatsVerbosity>,
#[serde(skip_serializing_if = "Option::is_none")]
pub fullscreen_on_stream: Option<bool>,
/// The presentation cluster — the keys the Apple client already writes into this
/// same catalog shape (`present_priority`/`smooth_buffer`/`vsync`/`allow_vrr`;
/// Android carries the first two). First-class here so a profile authored on any
/// client applies on all of them instead of riding `extra` unapplied.
#[serde(skip_serializing_if = "Option::is_none")]
pub present_priority: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub smooth_buffer: Option<u8>,
#[serde(skip_serializing_if = "Option::is_none")]
pub vsync: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub allow_vrr: Option<bool>,
/// Overlay keys a newer client wrote and this one doesn't model — carried through a
/// load→save round-trip untouched.
#[serde(flatten)]
@@ -122,6 +138,9 @@ impl SettingsOverlay {
if let Some(v) = self.mic_enabled {
s.mic_enabled = v;
}
if let Some(v) = self.echo_cancel {
s.echo_cancel = v;
}
if let Some(v) = &self.touch_mode {
s.touch_mode = v.clone();
}
@@ -137,6 +156,9 @@ impl SettingsOverlay {
if let Some(v) = &self.gamepad {
s.gamepad = v.clone();
}
if let Some(v) = self.gamepad_forwarding {
s.gamepad_forwarding = v;
}
if let Some(v) = self.stats_verbosity {
// Through the setter so the legacy `show_stats` bool stays coherent for
// pre-tier binaries reading the same settings file.
@@ -145,6 +167,18 @@ impl SettingsOverlay {
if let Some(v) = self.fullscreen_on_stream {
s.fullscreen_on_stream = v;
}
if let Some(v) = &self.present_priority {
s.present_priority = v.clone();
}
if let Some(v) = self.smooth_buffer {
s.smooth_buffer = v;
}
if let Some(v) = self.vsync {
s.vsync = v;
}
if let Some(v) = self.allow_vrr {
s.allow_vrr = v;
}
s
}
@@ -197,6 +231,9 @@ impl SettingsOverlay {
if after.mic_enabled != before.mic_enabled {
self.mic_enabled = Some(after.mic_enabled);
}
if after.echo_cancel != before.echo_cancel {
self.echo_cancel = Some(after.echo_cancel);
}
if after.touch_mode != before.touch_mode {
self.touch_mode = Some(after.touch_mode.clone());
}
@@ -212,12 +249,27 @@ impl SettingsOverlay {
if after.gamepad != before.gamepad {
self.gamepad = Some(after.gamepad.clone());
}
if after.gamepad_forwarding != before.gamepad_forwarding {
self.gamepad_forwarding = Some(after.gamepad_forwarding);
}
if after.stats_verbosity() != before.stats_verbosity() {
self.stats_verbosity = Some(after.stats_verbosity());
}
if after.fullscreen_on_stream != before.fullscreen_on_stream {
self.fullscreen_on_stream = Some(after.fullscreen_on_stream);
}
if after.present_priority != before.present_priority {
self.present_priority = Some(after.present_priority.clone());
}
if after.smooth_buffer != before.smooth_buffer {
self.smooth_buffer = Some(after.smooth_buffer);
}
if after.vsync != before.vsync {
self.vsync = Some(after.vsync);
}
if after.allow_vrr != before.allow_vrr {
self.allow_vrr = Some(after.allow_vrr);
}
}
/// Drop one override by its overlay field name, putting the row back to inheriting. The
@@ -243,13 +295,19 @@ impl SettingsOverlay {
"compositor" => self.compositor = None,
"audio_channels" => self.audio_channels = None,
"mic_enabled" => self.mic_enabled = None,
"echo_cancel" => self.echo_cancel = None,
"touch_mode" => self.touch_mode = None,
"mouse_mode" => self.mouse_mode = None,
"invert_scroll" => self.invert_scroll = None,
"inhibit_shortcuts" => self.inhibit_shortcuts = None,
"gamepad" => self.gamepad = None,
"gamepad_forwarding" => self.gamepad_forwarding = None,
"stats_verbosity" => self.stats_verbosity = None,
"fullscreen_on_stream" => self.fullscreen_on_stream = None,
"present_priority" => self.present_priority = None,
"smooth_buffer" => self.smooth_buffer = None,
"vsync" => self.vsync = None,
"allow_vrr" => self.allow_vrr = None,
_ => return false,
}
true
@@ -424,6 +482,10 @@ mod tests {
assert_eq!((out.width, out.height), (1920, 1080));
assert_eq!(out.bitrate_kbps, 20000);
assert_eq!(out.codec, "hevc");
assert!(
out.gamepad_forwarding,
"default on, and an empty overlay leaves it alone"
);
assert!(empty.is_empty());
let overlay = SettingsOverlay {
@@ -437,14 +499,20 @@ mod tests {
compositor: Some("gamescope".into()),
audio_channels: Some(6),
mic_enabled: Some(true),
echo_cancel: Some(false),
touch_mode: Some("pointer".into()),
mouse_mode: Some("desktop".into()),
invert_scroll: Some(true),
inhibit_shortcuts: Some(false),
gamepad: Some("dualsense".into()),
gamepad_forwarding: Some(false),
match_window: Some(true),
fullscreen_on_stream: Some(false),
stats_verbosity: Some(StatsVerbosity::Detailed),
present_priority: Some("smooth".into()),
smooth_buffer: Some(3),
vsync: Some(false),
allow_vrr: Some(false),
..Default::default()
};
assert!(!overlay.is_empty());
@@ -457,14 +525,20 @@ mod tests {
assert_eq!(out.compositor, "gamescope");
assert_eq!(out.audio_channels, 6);
assert!(out.mic_enabled);
assert!(!out.echo_cancel);
assert_eq!(out.touch_mode, "pointer");
assert_eq!(out.mouse_mode, "desktop");
assert!(out.invert_scroll);
assert!(!out.inhibit_shortcuts);
assert_eq!(out.gamepad, "dualsense");
assert!(!out.gamepad_forwarding);
assert!(out.match_window);
assert!(!out.fullscreen_on_stream);
assert_eq!(out.stats_verbosity(), StatsVerbosity::Detailed);
assert_eq!(out.present_priority, "smooth");
assert_eq!(out.smooth_buffer, 3);
assert!(!out.vsync);
assert!(!out.allow_vrr);
// The tier goes through the setter, so the legacy bool a pre-tier binary reads
// stays coherent with it.
assert!(out.show_stats);
@@ -529,6 +603,92 @@ mod tests {
assert_eq!(o2, o);
}
/// `echo_cancel` is a first-class overlay field, not an `extra` passenger: it applies,
/// absorbs, clears, and serialises under the `echo_cancel` key the Apple and Android
/// clients write — one catalog has to round-trip through all three.
#[test]
fn echo_cancel_is_a_first_class_override() {
let base = Settings::default();
assert!(base.echo_cancel, "the setting ships on");
let mut o = SettingsOverlay::default();
let before = o.apply(&base);
let mut after = before.clone();
after.echo_cancel = false;
o.absorb(&before, &after);
assert_eq!(o.echo_cancel, Some(false));
assert!(!o.apply(&base).echo_cancel);
assert!(
o.extra.is_empty(),
"modelled fields must never land in the passthrough"
);
// Serialised under the shared key, and read back from a foreign client's file.
let text = serde_json::to_string(&o).unwrap();
assert!(text.contains("\"echo_cancel\":false"), "{text}");
let from_apple: SettingsOverlay =
serde_json::from_str(r#"{"mic_enabled":true,"echo_cancel":false}"#).unwrap();
assert_eq!(from_apple.echo_cancel, Some(false));
assert!(from_apple.extra.is_empty());
assert!(o.clear("echo_cancel"));
assert_eq!(o.echo_cancel, None);
assert!(o.is_empty());
}
/// The presentation cluster is first-class, not `extra` passengers: it applies,
/// absorbs, clears, and serialises under the exact keys the Apple client already
/// writes (`present_priority`/`smooth_buffer`/`vsync`/`allow_vrr`) — one catalog
/// has to round-trip through every platform, and a mismatched key would be carried
/// but never applied.
#[test]
fn presentation_cluster_is_first_class() {
let base = Settings::default();
let mut o = SettingsOverlay::default();
let before = o.apply(&base);
let mut after = before.clone();
after.present_priority = "smooth".into();
o.absorb(&before, &after);
let before = o.apply(&base);
let mut after = before.clone();
after.smooth_buffer = 1;
o.absorb(&before, &after);
assert_eq!(o.present_priority.as_deref(), Some("smooth"));
assert_eq!(o.smooth_buffer, Some(1));
assert!(
o.extra.is_empty(),
"modelled fields must never land in the passthrough"
);
let out = o.apply(&base);
assert_eq!(
out.present_priority(),
crate::trust::PresentPriority::Smooth { buffer: 1 }
);
// Serialised under the shared keys, and read back from a foreign client's file.
let text = serde_json::to_string(&o).unwrap();
assert!(text.contains("\"present_priority\":\"smooth\""), "{text}");
assert!(text.contains("\"smooth_buffer\":1"), "{text}");
let from_apple: SettingsOverlay = serde_json::from_str(
r#"{"present_priority":"latency","smooth_buffer":2,"vsync":true,"allow_vrr":false}"#,
)
.unwrap();
assert_eq!(from_apple.present_priority.as_deref(), Some("latency"));
assert_eq!(from_apple.smooth_buffer, Some(2));
assert_eq!(from_apple.vsync, Some(true));
assert_eq!(from_apple.allow_vrr, Some(false));
assert!(from_apple.extra.is_empty());
assert!(o.clear("present_priority"));
assert!(o.clear("smooth_buffer"));
assert_eq!(o.present_priority, None);
assert!(o.is_empty());
let mut vrr = from_apple;
assert!(vrr.clear("vsync"));
assert!(vrr.clear("allow_vrr"));
assert_eq!((vrr.vsync, vrr.allow_vrr), (None, None));
}
/// `clear` is the explicit way back to inheriting, including the resolution tri-state.
#[test]
fn clear_drops_one_override() {
@@ -547,6 +707,29 @@ mod tests {
assert!(!o.clear("no_such_field"));
}
/// Controller forwarding defaults ON, so its interesting override is the FALSE one — and a
/// `false` that `apply` dropped would silently forward a pad the profile said not to.
/// `absorb` must record it, `clear` must undo it, and the serialized name both carry is the
/// one every client's reset button sends.
#[test]
fn gamepad_forwarding_overrides_off_and_resets_back() {
let base = Settings::default();
assert!(base.gamepad_forwarding, "the shipped default");
let mut o = SettingsOverlay::default();
let mut after = base.clone();
after.gamepad_forwarding = false;
o.absorb(&base, &after);
assert_eq!(o.gamepad_forwarding, Some(false));
assert!(!o.apply(&base).gamepad_forwarding);
assert!(o.clear("gamepad_forwarding"));
assert_eq!(o.gamepad_forwarding, None);
assert!(o.is_empty());
// Back to inheriting: the global's live value, not a remembered false.
assert!(o.apply(&base).gamepad_forwarding);
}
/// Stats verbosity Off must survive `apply` — it is a legitimate override, and going
/// through `set_stats_verbosity` keeps `show_stats` in sync in that direction too.
#[test]
+252 -9
View File
@@ -41,6 +41,9 @@ pub struct SessionParams {
pub display_hdr: Option<punktfunk_core::quic::HdrMeta>,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Run the uplink through the platform's echo cancellation ([`Settings::echo_cancel`]).
/// Ignored when `mic_enabled` is false; `PUNKTFUNK_NO_AEC=1` overrides it off.
pub echo_cancel: bool,
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool,
@@ -78,6 +81,30 @@ pub struct SessionParams {
/// above; it rides along so the stats overlay can answer "which profile am I on?" without
/// re-reading any store (design/client-settings-profiles.md §5.2).
pub profile: Option<String>,
/// Advertise `quic::CLIENT_CAP_PHASE_LOCK`: this embedder's presenter has REAL on-glass
/// latch stamps (`VK_KHR_present_wait`) and will feed [`latch_grid`](Self::latch_grid),
/// so the pump sends the ~1 Hz `PhaseReport`s the host phase-locks its capture tick to
/// (design/phase-locked-capture.md — previously Apple/Android only). Never set without
/// present timing: the host arms on report receipt, but the Hello should say what the
/// client actually does.
pub phase_lock: bool,
/// The presenter-written latch grid the pump's reports are computed from.
pub latch_grid: Arc<LatchGrid>,
}
/// The presenter's display-latch grid, shared presenter → pump (the `force_software`
/// pattern in the other direction): the presenter's 1 Hz present-timing fold writes a
/// recent on-glass latch instant plus the panel period; the pump's stats window folds its
/// per-AU arrival stamps against them into the ~1 Hz `PhaseReport`. All zeros until the
/// first fold — and forever when present timing isn't available — so the pump simply
/// stays quiet then.
#[derive(Default)]
pub struct LatchGrid {
/// A recent on-glass latch instant (client `CLOCK_REALTIME` ns — the same domain as
/// the AU arrival stamps). Any grid point works; the report extrapolates forward.
pub anchor_ns: std::sync::atomic::AtomicU64,
/// The panel's latch period (ns). `0` = no grid yet.
pub period_ns: std::sync::atomic::AtomicU64,
}
/// The session pump's share of the unified stats window (design/stats-unification.md):
@@ -121,9 +148,31 @@ pub struct Stats {
/// received+lost (%). The OSD renders the counter line only when nonzero.
pub lost: u32,
pub lost_pct: f32,
/// Mic uplink frames this window: handed to the QUIC datagram send, and shed anywhere
/// client-side (queue-full at the producer + the pump's stale-oldest backlog governor —
/// see [`NativeClient::mic_stats`]). Both stay 0 while the mic is off OR muted (a mute
/// stops the sending, not the capture), so the OSD renders the mic line only while voice
/// is actually going out — the muted case has its own badge, which does not need stats on.
pub mic_sent: u32,
pub mic_dropped: u32,
/// The decode path frames actually took this window (`"vaapi"`/`"software"`, empty
/// until the first frame) — the OSD's trailing tag; tracks a mid-session fallback.
pub decoder: &'static str,
/// The encoder's CURRENT target bitrate (kbps): the Welcome resolve, then live per
/// `BitrateChanged` ack. What `mbps` (measured goodput) is judged AGAINST — a user
/// staring at "19 Mb/s" can't otherwise tell "the encoder is capped at 20" from "my
/// 200 Mb/s ask was honoured and this scene is cheap" (the gap that let the
/// settings-drop bug ship four releases). `0` = an old host that never reported one.
pub target_kbps: u32,
/// Automatic bitrate is armed (ABR moves `target_kbps` on its own) — the OSD tags the
/// target `(auto)` so a moving figure reads as policy, not a broken setting.
pub auto_rate: bool,
/// The host resolved full-chroma 4:4:4 for this session (`Welcome::chroma_format`).
pub chroma_444: bool,
/// This session ADVERTISED `VIDEO_CAP_444` (the Settings "Full chroma" opt-in): with
/// `chroma_444` false, the host declined — the OSD says so instead of leaving the
/// switch's effect unobservable.
pub asked_444: bool,
}
/// Frames the pump keeps waiting for their 0xCF host timing (pts → capture→received µs).
@@ -160,10 +209,61 @@ pub enum SessionEvent {
Stats(Stats),
}
/// The in-stream microphone mute (B4), shared between the embedder's toggle (a keyboard chord
/// in the presenter) and the capture callback that reads it every quantum.
///
/// Two flags, not one, so the indicator can never lie: `live` is raised by the pump only once
/// the uplink is actually running, so a session whose mic is off in Settings — or whose capture
/// device failed to open — reports "no mic here" and the chord is a documented no-op instead of
/// silently latching a mute nothing implements. Per session by design: the mute is a moment
/// ("don't send the doorbell"), not a preference, so it is never persisted and every new
/// session starts unmuted.
#[derive(Clone, Default)]
pub struct MicControl {
muted: Arc<AtomicBool>,
live: Arc<AtomicBool>,
}
impl MicControl {
/// True when this session has a running uplink to mute at all.
pub fn live(&self) -> bool {
self.live.load(Ordering::Relaxed)
}
/// True when the user has muted a uplink that exists — what the OSD indicator draws.
pub fn muted(&self) -> bool {
self.live() && self.muted.load(Ordering::Relaxed)
}
/// Flip the mute. `Some(now_muted)` when it applied, `None` when this session has no
/// uplink (the caller says so rather than pretending something happened).
pub fn toggle(&self) -> Option<bool> {
if !self.live() {
return None;
}
let next = !self.muted.load(Ordering::Relaxed);
self.muted.store(next, Ordering::Relaxed);
Some(next)
}
/// The capture side's handle on the flag (the streamer reads it per quantum).
fn flag(&self) -> Arc<AtomicBool> {
self.muted.clone()
}
/// The pump's report that the uplink came up (or went away).
fn set_live(&self, live: bool) {
self.live.store(live, Ordering::Relaxed);
}
}
pub struct SessionHandle {
pub events: async_channel::Receiver<SessionEvent>,
pub frames: async_channel::Receiver<DecodedFrame>,
pub stop: Arc<AtomicBool>,
/// The in-stream mic mute. Inert (`live()` false) until the pump has the uplink running,
/// and for the whole session when the mic is off in Settings.
pub mic: MicControl,
/// The pump thread. A Vulkan-Video pump SUBMITS to the shared device's decode
/// queue — the presenter must join this before any `vkDeviceWaitIdle`/teardown
/// (external-sync rule over every device queue).
@@ -176,14 +276,17 @@ pub fn start(params: SessionParams) -> SessionHandle {
let (frame_tx, frame_rx) = async_channel::bounded(2);
let stop = Arc::new(AtomicBool::new(false));
let stop_w = stop.clone();
let mic = MicControl::default();
let mic_w = mic.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-session".into())
.spawn(move || pump(params, ev_tx, frame_tx, stop_w))
.spawn(move || pump(params, ev_tx, frame_tx, stop_w, mic_w))
.expect("spawn session thread");
SessionHandle {
events: ev_rx,
frames: frame_rx,
stop,
mic,
thread: Some(thread),
}
}
@@ -236,6 +339,7 @@ fn pump(
ev_tx: async_channel::Sender<SessionEvent>,
frame_tx: async_channel::Sender<DecodedFrame>,
stop: Arc<AtomicBool>,
mic: MicControl,
) {
// PUNKTFUNK_PREFER_PYROWAVE=1 — the Phase-2 lab opt-in for the wired-LAN wavelet codec
// (a Settings toggle is the Phase-3 productization). Riding `preferred_codec` is exactly
@@ -267,11 +371,17 @@ fn pump(
// This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an
// A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600).
punktfunk_core::client::display_hdr_env_override().or(params.display_hdr),
if params.cursor_forward {
// CURSOR: this embedder renders the host cursor locally in desktop mouse mode.
// PHASE_LOCK: the presenter has real latch stamps and the pump reports them below.
(if params.cursor_forward {
punktfunk_core::quic::CLIENT_CAP_CURSOR
} else {
0
},
}) | (if params.phase_lock {
punktfunk_core::quic::CLIENT_CAP_PHASE_LOCK
} else {
0
}),
// Slice-progressive delivery: off — this presenter feeds FFmpeg whole AUs; a partial
// avcodec feed path can flip it later.
false,
@@ -314,11 +424,31 @@ fn pump(
// Build the decoder for the codec the host resolved (never assume HEVC), honoring the
// Settings backend preference (auto/vaapi/software).
let codec_id = crate::video::ffmpeg_codec_id(connector.codec);
tracing::info!(
?codec_id,
welcome_codec = connector.codec,
"negotiated video codec"
);
// The WIRE codec is the negotiated truth; the FFmpeg id is meaningful only where
// FFmpeg decodes it. `ffmpeg_codec_id`'s fallthrough maps every unknown wire bit —
// PyroWave included — to HEVC, so logging it unconditionally claimed
// `codec_id=HEVC` for wavelet sessions that never touch FFmpeg at all.
let codec = match connector.codec {
punktfunk_core::quic::CODEC_H264 => "H264",
punktfunk_core::quic::CODEC_HEVC => "HEVC",
punktfunk_core::quic::CODEC_AV1 => "AV1",
punktfunk_core::quic::CODEC_PYROWAVE => "PyroWave",
_ => "unknown",
};
if connector.codec == punktfunk_core::quic::CODEC_PYROWAVE {
tracing::info!(
codec,
welcome_codec = connector.codec,
"negotiated video codec"
);
} else {
tracing::info!(
codec,
?codec_id,
welcome_codec = connector.codec,
"negotiated video codec"
);
}
// A negotiated PyroWave session decodes on the presenter's device, no FFmpeg —
// reachable only through the explicit preference above (resolve_codec never
// auto-picks the bit), so failing loudly here is failing an opted-in experiment.
@@ -361,6 +491,12 @@ fn pump(
}
};
let force_software = params.force_software.clone();
// Session-constant stats facts (design/stats-unification.md): what the target figure is
// judged against and whether the 4:4:4 opt-in was honoured. `target_kbps` itself is read
// live per window — an Automatic session's ABR moves it.
let auto_rate = connector.wants_decode_latency();
let chroma_444 = connector.chroma_format == punktfunk_core::quic::CHROMA_IDC_444;
let asked_444 = params.video_caps & punktfunk_core::quic::VIDEO_CAP_444 != 0;
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected). Audio runs on its own
// thread (one puller per plane), blocking on the audio queue like the Apple client.
@@ -379,18 +515,29 @@ fn pump(
.ok()
})
.flatten();
// The uplink, and with it the mute the embedder's chord drives. `set_live` is what makes
// the chord (and its indicator) real: a mic turned off in Settings, or a capture device
// that wouldn't open, leaves it false and the chord stays an honest no-op.
let _mic = params
.mic_enabled
.then(|| {
audio::MicStreamer::spawn(connector.clone())
audio::MicStreamer::spawn(connector.clone(), mic.flag(), params.echo_cancel)
.map_err(|e| tracing::warn!(error = %e, "mic uplink disabled"))
.ok()
})
.flatten();
mic.set_live(_mic.is_some());
// Live host↔client clock offset: loaded per frame (Relaxed) so mid-stream re-syncs (an NTP
// step, drift) keep the capture-clock latency stats honest — never cached at session start.
let clock_offset_live = connector.clock_offset_shared();
// Phase-lock (advertised above): every received AU's arrival stamp, folded per stats
// window against the presenter's latch grid into the ~1 Hz PhaseReport. Desktop
// sessions receive whole AUs only (no frame parts), so every arrival counts — the
// reference reporters (Apple/Android) sample the same signal. 256 ≈ 2 s at 120 Hz.
let latch_grid = params.latch_grid.clone();
let mut phase_arrivals: Vec<u64> = Vec::new();
let mut last_applied_phase: Option<i32> = None;
// PUNKTFUNK_DEBUG_RECONFIGURE=WxH@HZ:SECS — lab lever: request ONE mid-stream mode
// switch N seconds in, so a headless session (no window manager to drag a window in)
// can exercise the resize path deterministically — host pipeline rebuild, decoder
@@ -434,6 +581,9 @@ fn pump(
let mut dec_path: &'static str = "";
// The stats window keeps its own drop cursor — the OSD shows the per-window delta.
let mut window_dropped = connector.frames_dropped();
// Mic uplink cursor (same per-window diffing): a healthy 10 ms-frame mic reads ~100
// sent/s; a nonzero drop delta is the queue shedding backlog (see NativeClient::mic_stats).
let mut window_mic = connector.mic_stats();
let mut last_kf_req: Option<Instant> = None;
// Freeze-until-reanchor: the shared post-loss gate ([`punktfunk_core::reanchor::ReanchorGate`]).
// Armed on any loss signal (frame-index gap, dropped-count climb, decoder wedge/demotion), it
@@ -480,6 +630,9 @@ fn pump(
} else {
now_ns()
};
if params.phase_lock && phase_arrivals.len() < 256 {
phase_arrivals.push(received_ns);
}
// fps / goodput count every received AU (spec), decoded or not.
frames_n += 1;
bytes_n += frame.data.len() as u64;
@@ -733,6 +886,19 @@ fn pump(
// host never emits any — the deque fills to its cap and the OSD keeps the
// combined `host+network` stage.
while let Ok(t) = connector.next_host_timing(Duration::ZERO) {
// Phase-lock closed loop: the host's applied grid offset rides the 0xCF tail.
// Log transitions so an on-glass run can watch the controller engage/settle
// (the Android reporter's parity log).
if params.phase_lock
&& t.applied_phase_ns.is_some()
&& t.applied_phase_ns != last_applied_phase
{
last_applied_phase = t.applied_phase_ns;
tracing::info!(
applied_phase_ns = t.applied_phase_ns.unwrap_or(0),
"host phase-lock: applied capture-grid offset"
);
}
if let Some(i) = pending_split.iter().position(|(p, _)| *p == t.pts_ns) {
let (_, hn_us) = pending_split.remove(i).unwrap();
host_us_win.push(t.host_us as u64);
@@ -775,6 +941,41 @@ fn pump(
}
if window_start.elapsed() >= Duration::from_secs(1) {
// Phase-lock report (~1 Hz, riding the stats window — the reference reporters'
// cadence): this window's arrival leads before the presenter's latch grid,
// folded with the SHARED circular statistic (the host controller was tuned
// against it). Quiet until the presenter has a grid (period 0 — no
// present-timing samples yet) or the window is thin (< 8 arrivals —
// `circular_latch` declines). 1 ms uncertainty = Apple/Android parity.
if params.phase_lock {
let period = latch_grid.period_ns.load(Ordering::Relaxed);
let anchor = latch_grid.anchor_ns.load(Ordering::Relaxed);
if period > 0 && anchor > 0 {
let leads_us: Vec<u64> = phase_arrivals
.iter()
.map(|a| {
((anchor as i128 - *a as i128).rem_euclid(period as i128) / 1000) as u64
})
.collect();
if let Some((lead_ns, coherence)) =
punktfunk_core::phase::circular_latch(&leads_us, period as i64)
{
// Extrapolate the (possibly ~1 s old) anchor to the next latch at
// or after now, then express it on the host clock.
let (now, p, a) = (now_ns() as i128, period as i128, anchor as i128);
let k = ((now - a).max(0) + p - 1) / p;
let offset = clock_offset_live.load(Ordering::Relaxed) as i128;
connector.report_phase(
(a + k * p + offset).max(0) as u64,
period.min(u32::MAX as u64) as u32,
1_000_000,
lead_ns.min(u32::MAX as u64) as u32,
coherence,
);
}
}
phase_arrivals.clear();
}
let secs = window_start.elapsed().as_secs_f32();
let (hn_p50, _) = window_percentiles(&mut hostnet_us);
let (dec_p50, _) = window_percentiles(&mut decode_us);
@@ -789,6 +990,12 @@ fn pump(
let (pace_p50, _) = window_percentiles(&mut pace_us_win);
let lost = dropped.saturating_sub(window_dropped) as u32;
window_dropped = dropped;
let mic_now = connector.mic_stats();
let mic_sent = mic_now.sent.saturating_sub(window_mic.sent) as u32;
let mic_dropped = (mic_now.dropped_full + mic_now.dropped_stale)
.saturating_sub(window_mic.dropped_full + window_mic.dropped_stale)
as u32;
window_mic = mic_now;
tracing::debug!(
fps = frames_n,
hostnet_p50_us = hn_p50,
@@ -800,6 +1007,8 @@ fn pump(
pace_p50_us = pace_p50,
decode_p50_us = dec_p50,
lost,
mic_sent,
mic_dropped,
total_frames,
"stream window"
);
@@ -822,7 +1031,13 @@ fn pump(
} else {
0.0
},
mic_sent,
mic_dropped,
decoder: dec_path,
target_kbps: connector.current_bitrate_kbps(),
auto_rate,
chroma_444,
asked_444,
}));
window_start = Instant::now();
frames_n = 0;
@@ -844,6 +1059,10 @@ fn pump(
"session ended"
);
stop.store(true, Ordering::SeqCst);
// The uplink is about to be dropped with the rest of this frame — stop claiming a mute
// surface, so an embedder still holding the handle through its end path (browse mode
// returns to the console with it) can't draw a muted mic that no longer exists.
mic.set_live(false);
if let Some(t) = audio_thread {
let _ = t.join(); // exits within its 100 ms pull timeout once `stop` is set
}
@@ -954,4 +1173,28 @@ mod tests {
assert!(parse_debug_reconfigure(bad).is_none(), "{bad:?} parsed");
}
}
/// The mute is inert until the pump reports a live uplink — a session without a mic must
/// answer "nothing to mute" rather than latching a mute and drawing the indicator.
#[test]
fn mic_mute_is_a_no_op_without_an_uplink() {
let mic = MicControl::default();
assert!(!mic.live());
assert_eq!(mic.toggle(), None, "no uplink, nothing to toggle");
assert!(!mic.muted(), "and nothing to show");
mic.set_live(true);
assert_eq!(mic.toggle(), Some(true));
assert!(mic.muted());
// The capture side reads the same flag the toggle writes.
assert!(mic.flag().load(Ordering::Relaxed));
assert_eq!(mic.toggle(), Some(false));
assert!(!mic.muted());
// A mute that outlives its uplink stops being shown (session end clears `live`).
assert_eq!(mic.toggle(), Some(true));
mic.set_live(false);
assert!(!mic.muted());
assert_eq!(mic.toggle(), None);
}
}
+547 -19
View File
@@ -14,6 +14,7 @@ use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::endpoint;
use serde::{Deserialize, Serialize};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
pub fn config_dir() -> Result<PathBuf> {
@@ -268,8 +269,40 @@ impl KnownHosts {
self.hosts.iter().find(|h| h.fp_hex == fp_hex)
}
/// The record an address-keyed lookup resolves to, by index (so callers that go on to
/// mutate the store don't fight the borrow checker).
///
/// One address cannot host two live identities at once, but the store can still hold more
/// than one record claiming `addr:port`: an fp-less placeholder waiting for its first
/// ceremony, or — before [`KnownHosts::upsert_trusted`] existed — a re-keyed host whose new
/// record was appended beside the dead one. Resolving that positionally is what turned a
/// host reinstall into a permanent lockout: the dead pin, written first, won every later
/// connect, including right after a successful re-pair.
///
/// So the rule is "the newest trust decision wins": a real fingerprint beats a placeholder,
/// and among real ones the LAST record — records are only ever appended by an explicit
/// trust decision, so the last one is the most recent thing the user actually authorised.
/// That is a lookup order, never an authorisation: whichever record this picks, the pin it
/// yields still has to match the certificate the host presents, or the connect fails closed.
pub fn index_by_addr(&self, addr: &str, port: u16) -> Option<usize> {
let mut best: Option<usize> = None;
for (i, h) in self.hosts.iter().enumerate() {
if h.addr != addr || h.port != port {
continue;
}
let better = match best {
None => true,
Some(b) => !h.fp_hex.is_empty() || self.hosts[b].fp_hex.is_empty(),
};
if better {
best = Some(i);
}
}
best
}
pub fn find_by_addr(&self, addr: &str, port: u16) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.addr == addr && h.port == port)
self.index_by_addr(addr, port).map(|i| &self.hosts[i])
}
/// Forget the entry with this fingerprint. Returns true if one was removed (the user
@@ -322,6 +355,70 @@ impl KnownHosts {
self.hosts.push(entry);
}
}
/// [`upsert`](Self::upsert) for an **authorised trust decision** — a PIN ceremony, a
/// delegated approval, a TOFU accept, a headless pair — which additionally retires every
/// other record claiming the same `addr:port`.
///
/// `upsert` alone keys on the fingerprint, deliberately: that is how a host which moved
/// address keeps its record and the fields the user set on it. The cost was that a host
/// which changed IDENTITY — a reinstall, a wiped `ProgramData`, a re-key — matched nothing
/// and got a SECOND record appended for the address it already had, and every later
/// connect then pinned the dead fingerprint from the older one. No way out from the UI,
/// and re-pairing didn't help: the ceremony succeeded and appended yet another record.
///
/// A record retired here carries what describes the BOX rather than the identity onto the
/// record that survives — its MAC, its OS chain, the profile bound to it, its pinned cards,
/// when it was last used — so a reinstall doesn't quietly cost the user their setup.
/// Deliberately NOT carried: `paired` and `clipboard_sync`, which are decisions about one
/// specific certificate and have to be made again for a new one, and the stable record id
/// (a deep link written from the retired record falls through to the `host=` recovery the
/// link grammar already specifies, rather than silently pointing at a new identity).
///
/// **Only trust decisions may call this.** Everything that merely LEARNS something about a
/// host — a rediscovery, the wake path's address re-key — stays on plain `upsert`: those
/// are driven by unauthenticated mDNS, and letting an advert delete a saved host by
/// claiming its address would trade this bug for a much worse one.
pub fn upsert_trusted(&mut self, entry: KnownHost) {
let (addr, port, fp_hex) = (entry.addr.clone(), entry.port, entry.fp_hex.clone());
self.upsert(entry);
// Nothing to supersede *with*: an fp-less record is a placeholder, not an identity.
if fp_hex.is_empty() {
return;
}
let (keep, retired): (Vec<KnownHost>, Vec<KnownHost>) = std::mem::take(&mut self.hosts)
.into_iter()
.partition(|h| !(h.addr == addr && h.port == port && h.fp_hex != fp_hex));
self.hosts = keep;
if retired.is_empty() {
return;
}
let Some(h) = self.hosts.iter_mut().find(|h| h.fp_hex == fp_hex) else {
return;
};
for old in retired {
tracing::info!(
addr = %addr, port,
retired_fp = %old.fp_hex, kept_fp = %fp_hex,
"host re-keyed — retiring the superseded record for this address"
);
if h.mac.is_empty() {
h.mac = old.mac;
}
if h.os.is_empty() {
h.os = old.os;
}
if h.profile_id.is_none() {
h.profile_id = old.profile_id;
}
if h.pinned_profiles.is_empty() {
h.pinned_profiles = old.pinned_profiles;
}
if h.last_used.is_none() {
h.last_used = old.last_used;
}
}
}
}
/// Load-upsert-save in one step — the pin every trust decision (TOFU accept, PIN
@@ -332,7 +429,10 @@ pub fn persist_host(name: &str, addr: &str, port: u16, fp_hex: &str, paired: boo
// so every user-set field (clipboard, profile binding, pins) must arrive as "not carried"
// — `upsert` then leaves an existing host's own settings alone. A hand-written literal
// here is how those fields would get silently reset on the next re-pair.
known.upsert(KnownHost {
//
// `upsert_trusted`, not `upsert`: this IS the authorised decision, so it is also the point
// at which a host that re-keyed retires its own dead record for this address.
known.upsert_trusted(KnownHost {
name: name.to_string(),
addr: addr.to_string(),
port,
@@ -366,6 +466,23 @@ pub fn forget_placeholder(addr: &str, port: u16) {
}
}
/// The record [`learn_mac`]/[`learn_os`] should write what an advert taught them onto:
/// the fingerprint match if there is one, else whatever the address resolves to. Fingerprint
/// FIRST — a single pass that took "either" would hand a stale record at the same address the
/// data the live host advertised, purely because it came earlier in the file.
fn learn_target<'a>(
known: &'a mut KnownHosts,
fp_hex: &str,
addr: &str,
port: u16,
) -> Option<&'a mut KnownHost> {
let i = (!fp_hex.is_empty())
.then(|| known.hosts.iter().position(|h| h.fp_hex == fp_hex))
.flatten()
.or_else(|| known.index_by_addr(addr, port))?;
known.hosts.get_mut(i)
}
/// Learn/refresh a saved host's Wake-on-LAN MAC(s) from its live advert (called while the host
/// is online, matched by fingerprint or address). No-op — and no disk write — when unchanged, so
/// the hosts page can call it on every discovery tick without churning the store.
@@ -374,11 +491,7 @@ pub fn learn_mac(fp_hex: &str, addr: &str, port: u16, mac: &[String]) {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known
.hosts
.iter_mut()
.find(|h| (!fp_hex.is_empty() && h.fp_hex == fp_hex) || (h.addr == addr && h.port == port))
else {
let Some(h) = learn_target(&mut known, fp_hex, addr, port) else {
return;
};
if h.mac == mac {
@@ -396,11 +509,7 @@ pub fn learn_os(fp_hex: &str, addr: &str, port: u16, os: &str) {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known
.hosts
.iter_mut()
.find(|h| (!fp_hex.is_empty() && h.fp_hex == fp_hex) || (h.addr == addr && h.port == port))
else {
let Some(h) = learn_target(&mut known, fp_hex, addr, port) else {
return;
};
if h.os == os {
@@ -679,6 +788,45 @@ impl MouseMode {
}
}
/// Presentation intent — what the presenter optimizes for
/// (design/desktop-presentation-rebuild.md; the Apple/Android clients' shared
/// `present_priority`/`smooth_buffer` pair). Stored stringly in
/// [`Settings::present_priority`] + [`Settings::smooth_buffer`]; resolved with
/// [`PresentPriority::resolve`], whose rules match the Android reference
/// (`decode/presenter.rs`): anything but an explicit `"smooth"` is latency, and a
/// smooth buffer outside 1..=3 (including 0 = Automatic) becomes 2.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum PresentPriority {
/// Every frame presents the moment the display can take it; a network hiccup is an
/// occasional repeated or skipped frame. The default.
Latency,
/// A small frame buffer (13 frames) evens out network/decode jitter, at the
/// buffer's worth of added display latency.
Smooth { buffer: u8 },
}
impl PresentPriority {
/// The shared cross-client resolution rule — pure, so every embedder agrees on what
/// a foreign profile's values mean.
pub fn resolve(name: &str, buffer: u8) -> PresentPriority {
if name == "smooth" {
PresentPriority::Smooth {
buffer: if (1..=3).contains(&buffer) { buffer } else { 2 },
}
} else {
PresentPriority::Latency
}
}
/// Frames the smoothing store holds; `0` = newest-wins (the latency intent).
pub fn fifo_capacity(self) -> u8 {
match self {
PresentPriority::Latency => 0,
PresentPriority::Smooth { buffer } => buffer,
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
@@ -700,6 +848,21 @@ pub struct Settings {
/// container `#[serde(default)]`.
pub render_scale: f64,
pub gamepad: String,
/// Forward this device's controllers to the host at all. Default ON — that was the
/// unconditional behaviour before this became a setting.
///
/// Off is for the couch whose controller reaches the host by some *other* route: a USB
/// passthrough tool (VirtualHere and friends), or a pad simply plugged into the host
/// itself. Leaving forwarding on there gives the host two controllers for one pair of
/// hands, and games read both.
///
/// It is deliberately stronger than "send no input": with it off the client never
/// *opens* the controller, and opening is what grabs the hardware (SDL's HIDAPI drivers
/// take the hidraw node) — a held device is one a passthrough tool cannot bind. Menu
/// navigation in the launcher still opens the active pad, and the session releases it;
/// see [`crate::gamepad::GamepadService::set_forwarding`].
#[serde(default = "default_true")]
pub gamepad_forwarding: bool,
/// Stable identity (`vid:pid:name`, see `PadInfo::key`) of the physical controller
/// forwarded as pad 0; empty = automatic (most recently connected). Applied to the
/// gamepad service at startup so the choice survives restarts.
@@ -727,6 +890,15 @@ pub struct Settings {
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Run the mic uplink through the platform's echo cancellation (the Apple/Android clients'
/// "Echo cancellation" toggle, same `echo_cancel` key). On Linux that means preferring an
/// echo-cancelled PipeWire source; on Windows, asking WASAPI for the Communications stream
/// category so the endpoint's own canceller engages. Default ON — without it, a laptop
/// speaker playing the host's audio is heard by this device's mic and sent straight back.
/// Only meaningful while `mic_enabled`. `PUNKTFUNK_NO_AEC=1` overrides it off (see
/// `audio::aec_enabled`). `default` so pre-existing stores load with it on.
#[serde(default = "default_true")]
pub echo_cancel: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + playback layout.
pub audio_channels: u8,
@@ -757,6 +929,32 @@ pub struct Settings {
/// `default = true`: the Linux stores never carried this and always advertised.
#[serde(default = "default_true")]
pub hdr_enabled: bool,
/// Presentation intent: `"latency"` (default) or `"smooth"` — the Apple/Android
/// clients' shared `present_priority` profile key, resolved with
/// [`PresentPriority::resolve`] (via [`Settings::present_priority`]). Anything
/// unknown reads as latency, so a newer client's future value degrades safely.
#[serde(default = "default_present_priority")]
pub present_priority: String,
/// Smoothness buffer size in frames: `0` = Automatic (resolves to 2), else 13.
/// Only meaningful under `present_priority = "smooth"` (the shared `smooth_buffer`
/// key). Each buffered frame absorbs about one refresh of jitter and adds one
/// refresh of display latency.
#[serde(default)]
pub smooth_buffer: u8,
/// Tear-free presentation (default ON = today's behavior: MAILBOX, FIFO fallback).
/// Off asks for a tearing present mode (IMMEDIATE) for the lowest possible latch
/// latency — best-effort: platforms/drivers without tearing silently stay tear-free
/// and the active mode is visible in the detailed stats. The shared `vsync` profile
/// key; the desktop default differs from macOS's (`false` there) deliberately —
/// sync-off means something different on each platform, the key is the contract.
#[serde(default = "default_true")]
pub vsync: bool,
/// Let a variable-refresh display follow the stream cadence: prefers the present
/// mode that drives VRR panels directly when fullscreen. Inert on fixed-refresh
/// displays (detection is measured from on-glass timestamps, not queried). The
/// shared `allow_vrr` profile key. Default ON, like the Apple client.
#[serde(default = "default_true")]
pub allow_vrr: bool,
/// Legacy on/off for the stats overlay — superseded by `stats_verbosity` but kept
/// written in sync (`set_stats_verbosity`) so pre-tier binaries reading the same
/// file keep working. `alias`: the pre-unification WinUI shell (≤ 0.8.4) persisted
@@ -785,9 +983,10 @@ pub struct Settings {
#[serde(default)]
pub invert_scroll: bool,
/// Playback endpoint for stream audio — on Linux the PipeWire `node.name` the
/// playback stream targets (`target.object`); empty = the session default (the
/// Apple client's Speaker picker). The session maps it onto `PUNKTFUNK_AUDIO_SINK`.
/// Ignored on Windows until the WASAPI endpoint leg exists.
/// playback stream targets (`target.object`); on Windows the WASAPI `IMMDevice`
/// endpoint id; empty = the OS default (the Apple client's Speaker picker). The
/// session maps it onto `PUNKTFUNK_AUDIO_SINK`. A picked endpoint that's gone
/// falls back to the default on both OSes.
#[serde(default)]
pub speaker_device: String,
/// Capture endpoint for the mic uplink (same semantics as `speaker_device`;
@@ -807,6 +1006,14 @@ pub struct Settings {
/// the user will be looking at. `0` = never stored → the 1280×720 default.
pub last_window_w: u32,
pub last_window_h: u32,
/// Settings keys this build doesn't model (a newer client's field), carried through a
/// load→save round-trip untouched — [`crate::profiles::SettingsOverlay`]'s `extra`
/// pattern extended to the globals. Without it, every whole-file writer of this store
/// (two shells, the console settings screen, the session's resize callback, Decky)
/// running as an OLDER binary silently drops what a newer one persisted. Empty on
/// every existing store, and an empty map serializes to nothing, so files don't churn.
#[serde(flatten)]
pub extra: BTreeMap<String, serde_json::Value>,
}
fn default_codec() -> String {
@@ -821,6 +1028,10 @@ fn default_mouse_mode() -> String {
"capture".into()
}
fn default_present_priority() -> String {
"latency".into()
}
fn default_true() -> bool {
true
}
@@ -852,6 +1063,12 @@ impl Settings {
MouseMode::from_name(&self.mouse_mode)
}
/// The presentation intent for this session (the resolved
/// `present_priority` × `smooth_buffer` pair).
pub fn present_priority(&self) -> PresentPriority {
PresentPriority::resolve(&self.present_priority, self.smooth_buffer)
}
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() {
@@ -876,18 +1093,24 @@ impl Default for Settings {
bitrate_kbps: 0,
render_scale: 1.0,
gamepad: "auto".into(),
gamepad_forwarding: true,
forward_pad: String::new(),
compositor: "auto".into(),
touch_mode: "trackpad".into(),
mouse_mode: "capture".into(),
inhibit_shortcuts: true,
mic_enabled: false,
echo_cancel: true,
audio_channels: 2,
codec: "auto".into(),
decoder: "auto".into(),
adapter: String::new(),
enable_444: false,
hdr_enabled: true,
present_priority: "latency".into(),
smooth_buffer: 0,
vsync: true,
allow_vrr: true,
show_stats: true,
stats_verbosity: None,
fullscreen_on_stream: true,
@@ -899,6 +1122,7 @@ impl Default for Settings {
match_window: false,
last_window_w: 0,
last_window_h: 0,
extra: BTreeMap::new(),
}
}
}
@@ -960,10 +1184,9 @@ pub fn effective_settings(
) -> (Settings, Option<StreamProfile>) {
let base = Settings::load();
let catalog = ProfilesFile::load();
let bound = KnownHosts::load()
.hosts
.iter()
.find(|h| h.addr == addr && h.port == port)
let known = KnownHosts::load();
let bound = known
.find_by_addr(addr, port)
.and_then(|h| h.profile_id.clone());
match resolve_profile(&catalog, bound.as_deref(), one_off) {
@@ -1003,6 +1226,12 @@ fn resolve_profile(
mod tests {
use super::*;
/// A 64-hex fingerprint of one repeated digit — readable in an assertion, and distinct
/// per letter, which is all the known-hosts tests need one to be.
fn fp(c: char) -> String {
std::iter::repeat_n(c, 64).collect()
}
/// A settings file predating the touch-input model loads as `trackpad` (the shipped
/// default), and the name round-trips through the enum both ways.
#[test]
@@ -1020,6 +1249,43 @@ mod tests {
}
}
/// A settings file predating the presentation cluster loads with the shipped
/// defaults (latency intent, Automatic buffer, tear-free, VRR allowed), and the
/// resolution rules match the Apple/Android reference: anything but an explicit
/// `"smooth"` is latency, and a smooth buffer outside 1..=3 becomes 2.
#[test]
fn settings_presentation_defaults_and_resolution() {
let old = r#"{"width":1280,"height":720,"gamepad":"auto","compositor":"auto"}"#;
let s: Settings = serde_json::from_str(old).unwrap();
assert_eq!(s.present_priority, "latency");
assert_eq!(s.smooth_buffer, 0);
assert!(s.vsync);
assert!(s.allow_vrr);
assert_eq!(s.present_priority(), PresentPriority::Latency);
assert_eq!(
PresentPriority::resolve("smooth", 0),
PresentPriority::Smooth { buffer: 2 },
"Automatic resolves to 2"
);
assert_eq!(
PresentPriority::resolve("smooth", 3),
PresentPriority::Smooth { buffer: 3 }
);
assert_eq!(
PresentPriority::resolve("smooth", 9),
PresentPriority::Smooth { buffer: 2 },
"out-of-range pins to the Automatic resolution"
);
assert_eq!(
PresentPriority::resolve("balanced-from-the-future", 2),
PresentPriority::Latency,
"unknown intents degrade to latency"
);
assert_eq!(PresentPriority::Latency.fifo_capacity(), 0);
assert_eq!(PresentPriority::Smooth { buffer: 3 }.fifo_capacity(), 3);
}
/// A pre-`forward_pad` settings file (≤ 0.5.0) loads with the pin on automatic.
#[test]
fn settings_forward_pad_defaults_empty() {
@@ -1063,6 +1329,31 @@ mod tests {
assert_eq!(s.forward_pad, "");
assert!(s.fullscreen_on_stream);
assert!(!s.library_enabled);
// Echo cancellation post-dates every stored file: it must load ON, or an upgrade
// would silently turn a user's echo protection off.
assert!(s.echo_cancel);
}
/// A key this build doesn't model (a newer client's setting) survives a load→save
/// round trip instead of being dropped by the next whole-file write — the same
/// contract `SettingsOverlay.extra` gives profiles. And when there are no unknown
/// keys, the flatten map adds nothing, so existing files don't churn.
#[test]
fn settings_unknown_keys_survive_round_trip() {
let newer = r#"{"width":1920,"height":1080,"frob_mode":"fancy","frob_level":3}"#;
let s: Settings = serde_json::from_str(newer).unwrap();
assert_eq!((s.width, s.height), (1920, 1080));
assert_eq!(
s.extra.get("frob_mode").and_then(|v| v.as_str()),
Some("fancy")
);
let out = serde_json::to_string(&s).unwrap();
assert!(out.contains(r#""frob_mode":"fancy""#), "{out}");
assert!(out.contains(r#""frob_level":3"#), "{out}");
// No unknown keys → no artifact of the passthrough field in the file.
let plain = serde_json::to_string(&Settings::default()).unwrap();
assert!(!plain.contains("extra"), "{plain}");
assert!(!plain.contains("frob"), "{plain}");
}
/// Stats-tier resolution: a pre-tier store falls back to `show_stats` (off → Off,
@@ -1220,6 +1511,243 @@ mod tests {
assert_eq!(k.hosts[0].pinned_profiles, vec!["dddddddddddd".to_string()]);
}
/// A host that regenerated its identity (reinstall, wiped ProgramData, re-key) ends up with
/// ONE record for its address — the live one. This is the `.173` lockout: `upsert` keys on
/// the fingerprint, so the re-paired host used to be appended beside the dead record, and
/// every later connect pinned the dead one — forever, re-pairing included.
#[test]
fn upsert_trusted_supersedes_a_rekeyed_host() {
let (dead, live) = (fp('c'), fp('a'));
let mut k = KnownHosts {
hosts: vec![KnownHost {
name: "ENRICOS-DESKTOP (local)".into(),
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: dead.clone(),
paired: true,
last_used: Some(1000),
mac: vec!["aa:bb:cc:dd:ee:ff".into()],
os: "windows".into(),
clipboard_sync: true,
profile_id: Some("aaaaaaaaaaaa".into()),
pinned_profiles: vec!["bbbbbbbbbbbb".into()],
id: Some("11111111-2222-4333-8444-555555555555".into()),
}],
};
// The re-pair: same box, same address, a certificate the client has never seen.
k.upsert_trusted(KnownHost {
name: "127.0.0.1".into(),
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: live.clone(),
paired: true,
..Default::default()
});
assert_eq!(k.hosts.len(), 1);
let h = &k.hosts[0];
assert_eq!(h.fp_hex, live);
// …and the address now resolves to the live pin, which is the whole bug.
assert_eq!(k.find_by_addr("127.0.0.1", 9777).unwrap().fp_hex, live);
assert!(k.find_by_fp(&dead).is_none());
// What describes the BOX rides along, so a reinstall doesn't cost the user their setup.
assert_eq!(h.mac, vec!["aa:bb:cc:dd:ee:ff".to_string()]);
assert_eq!(h.os, "windows");
assert_eq!(h.profile_id.as_deref(), Some("aaaaaaaaaaaa"));
assert_eq!(h.pinned_profiles, vec!["bbbbbbbbbbbb".to_string()]);
assert_eq!(h.last_used, Some(1000));
// What described the dead IDENTITY does not: the clipboard grant is a decision about
// one certificate, and the retired record's stable id must not follow a new one.
assert!(!h.clipboard_sync);
assert_ne!(
h.id.as_deref(),
Some("11111111-2222-4333-8444-555555555555")
);
}
/// The case fingerprint-keying exists for still works through the trusted path: a host that
/// only MOVED keeps its one record, its `paired` bit and everything the user set on it —
/// including the clipboard grant and the stable id, which a same-identity re-pair must not
/// disturb (that would be the fix trading one silent reset for another).
#[test]
fn upsert_trusted_keeps_a_host_that_only_moved_address() {
let same = fp('a');
let mut k = KnownHosts {
hosts: vec![KnownHost {
name: "Desk".into(),
addr: "192.168.1.50".into(),
port: 9777,
fp_hex: same.clone(),
paired: true,
clipboard_sync: true,
profile_id: Some("aaaaaaaaaaaa".into()),
id: Some("11111111-2222-4333-8444-555555555555".into()),
..Default::default()
}],
};
k.upsert_trusted(KnownHost {
name: "Desk".into(),
addr: "192.168.1.51".into(),
port: 9777,
fp_hex: same.clone(),
paired: false, // must not demote
..Default::default()
});
assert_eq!(k.hosts.len(), 1);
let h = &k.hosts[0];
assert_eq!(h.addr, "192.168.1.51");
assert!(h.paired);
assert!(h.clipboard_sync);
assert_eq!(h.profile_id.as_deref(), Some("aaaaaaaaaaaa"));
assert_eq!(
h.id.as_deref(),
Some("11111111-2222-4333-8444-555555555555")
);
}
/// Superseding is scoped to the address the decision was made for, and only ever runs off
/// one: a trust decision for `.51` leaves a different host saved at `.50` alone, and an
/// fp-less save (a manual entry, `--add-host` without `--fp`) retires nothing at all — it
/// carries no identity to supersede anything WITH.
#[test]
fn upsert_trusted_leaves_other_addresses_and_placeholders_alone() {
let mut k = KnownHosts {
hosts: vec![
KnownHost {
name: "Other box".into(),
addr: "192.168.1.50".into(),
port: 9777,
fp_hex: fp('c'),
paired: true,
..Default::default()
},
// Same address, DIFFERENT port: a distinct endpoint, not a duplicate.
KnownHost {
name: "Second host".into(),
addr: "192.168.1.51".into(),
port: 9778,
fp_hex: fp('d'),
paired: true,
..Default::default()
},
],
};
k.upsert_trusted(KnownHost {
name: "New box".into(),
addr: "192.168.1.51".into(),
port: 9777,
fp_hex: fp('a'),
paired: true,
..Default::default()
});
assert_eq!(k.hosts.len(), 3);
assert_eq!(
k.find_by_addr("192.168.1.50", 9777).unwrap().fp_hex,
fp('c')
);
assert_eq!(
k.find_by_addr("192.168.1.51", 9778).unwrap().fp_hex,
fp('d')
);
// An fp-less save alongside a real record: nothing is retired, and the address still
// resolves to the record that HAS a pin.
k.upsert_trusted(KnownHost {
name: "Typed by hand".into(),
addr: "192.168.1.50".into(),
port: 9777,
..Default::default()
});
assert_eq!(k.hosts.len(), 4);
assert_eq!(
k.find_by_addr("192.168.1.50", 9777).unwrap().fp_hex,
fp('c')
);
}
/// A store that ALREADY holds the duplicate (every client shipped so far can have written
/// one) connects again on the next connect, before any re-pair: an address resolves to the
/// newest trust decision for it, not to whichever record happens to sit first in the file.
/// Nothing is deleted at load — which record is live isn't knowable there, and guessing
/// wrong would throw away the good one; the retirement waits for the next trust decision.
#[test]
fn a_duplicated_store_resolves_to_the_newest_record() {
let (dead, live) = (fp('c'), fp('a'));
let mut k = KnownHosts {
hosts: vec![
KnownHost {
name: "ENRICOS-DESKTOP (local)".into(),
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: dead.clone(),
paired: true,
last_used: Some(9999), // the stale record is the one that HAS connected
..Default::default()
},
KnownHost {
name: "127.0.0.1".into(),
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: live.clone(),
paired: true,
..Default::default()
},
],
};
assert_eq!(k.find_by_addr("127.0.0.1", 9777).unwrap().fp_hex, live);
// Loading is non-destructive: both records are still there to be looked up by pin.
assert!(k.find_by_fp(&dead).is_some());
// A placeholder appended later never displaces a real pin.
k.hosts.push(KnownHost {
addr: "127.0.0.1".into(),
port: 9777,
..Default::default()
});
assert_eq!(k.find_by_addr("127.0.0.1", 9777).unwrap().fp_hex, live);
// …and the next trust decision cleans the store up.
k.upsert_trusted(KnownHost {
name: "127.0.0.1".into(),
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: live.clone(),
paired: true,
..Default::default()
});
assert_eq!(k.hosts.len(), 1);
assert_eq!(k.hosts[0].fp_hex, live);
}
/// An advert's learned MAC/OS lands on the record it identified, not on a stale namesake
/// at the same address that merely came first in the file.
#[test]
fn learn_target_prefers_the_fingerprint_match() {
let (dead, live) = (fp('c'), fp('a'));
let mut k = KnownHosts {
hosts: vec![
KnownHost {
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: dead.clone(),
..Default::default()
},
KnownHost {
addr: "127.0.0.1".into(),
port: 9777,
fp_hex: live.clone(),
..Default::default()
},
],
};
learn_target(&mut k, &live, "127.0.0.1", 9777).unwrap().os = "windows".into();
assert_eq!(k.find_by_fp(&live).unwrap().os, "windows");
assert_eq!(k.find_by_fp(&dead).unwrap().os, "");
// No fingerprint to go on (an advert that carries none) → the address's own answer.
learn_target(&mut k, "", "127.0.0.1", 9777).unwrap().os = "linux".into();
assert_eq!(k.find_by_fp(&live).unwrap().os, "linux");
assert_eq!(k.find_by_fp(&dead).unwrap().os, "");
// An advert for a host this store has never seen writes nothing.
assert!(learn_target(&mut k, &fp('e'), "10.0.0.9", 9777).is_none());
}
/// Pins render in card order, deduplicated, with deleted profiles simply gone — a pin is
/// presentation state, so a dangling one is never an error surface.
#[test]
+15 -1
View File
@@ -99,6 +99,18 @@ pub struct Status {
/// Why the check couldn't complete. `update_available` is always false when set.
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
/// The feed answered, but this channel has **no release published yet** — an expected
/// state rather than a malfunction, so a caller can say so plainly instead of showing a
/// raw "HTTP 404".
///
/// Deliberately NOT symmetric with the host's `UpdateStatus`, which clears `last_error`
/// for this case: there the consumer is a human reading a console, and a red "last check
/// failed" on an empty feed is the bug being fixed. Here the consumer is a shell script
/// reading an exit code, so `error` stays set and `--check-update` keeps returning 1.
/// An empty channel is not evidence that this build is current, and a mistyped
/// `PUNKTFUNK_UPDATE_FEED` is indistinguishable from one out here.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub not_published: bool,
}
/// The Ed25519 keys trusted for update manifests — pinned once in [`pf_update_check`] so the
@@ -302,6 +314,7 @@ pub fn check(current: &str) -> Status {
opt_in_hint: opt_in_would_help(kind, caps).then(opt_in_hint),
notes_url: String::new(),
error: None,
not_published: false,
};
let (apply, applier) = apply_route(kind, caps);
status.apply = apply;
@@ -320,7 +333,8 @@ pub fn check(current: &str) -> Status {
) {
Ok(m) => m,
Err(e) => {
status.error = Some(e);
status.not_published = e.is_not_published();
status.error = Some(e.to_string());
return status;
}
};
+124 -2
View File
@@ -110,6 +110,21 @@ pub struct VkVideoFrame {
pub decode_done_value: u64,
pub width: u32,
pub height: u32,
/// The decode POOL's allocated extent (`AVHWFramesContext.width`/`.height`) — the
/// CODED picture size (rounded up to the codec's macroblock alignment, then to the
/// driver's Vulkan picture-access granularity), so it is `>=` `width`/`height`. At
/// 1080p the pool is 1088 rows tall: 1080 is not a multiple of 16.
///
/// The presenter samples this image with NORMALIZED coordinates, so it needs both
/// numbers — `width`/`height` is what to display, `coded_*` is what the texture
/// actually spans. Sampling `0..1` without the ratio stretches the alignment padding
/// into view; because encoders fill those rows by replicating the picture's last
/// line, that reads as the bottom row smeared over the final few rows of the image
/// (field report 2026-07-31). Same class as the D3D11VA source-rect clamp in
/// `crate::video_d3d11`, which shows as a green bar there only because DXVA padding
/// is left uninitialized rather than replicated.
pub coded_width: u32,
pub coded_height: u32,
pub color: ColorDesc,
/// Intra keyframe (IDR/I): the stream's re-anchor point. The pump resumes display on
/// one after suppressing the concealed frames a reference loss leaves in its wake (on
@@ -306,6 +321,88 @@ pub fn ffmpeg_codec_id(wire: u8) -> ffmpeg::codec::Id {
}
}
/// Select a decoder for `codec_id` that can actually drive `hw_pix_fmt` through
/// `hw_device_ctx` — the open-time capability check every hardware backend needs.
///
/// `avcodec_find_decoder(id)` is NOT that: it returns the registry's FIRST decoder for
/// the id, and upstream orders the native `av1` decoder LAST on purpose ("hwaccel hooks
/// only, so prefer external decoders" — allcodecs.c), behind libdav1d/libaom. The ID
/// lookup therefore hands every AV1 session a pure software decoder that silently
/// ignores `hw_device_ctx` and never calls `get_format`; each frame then fails the
/// backend's hw-format guard and the session burns the demotion ladder MID-STREAM
/// (~1 s per rung — field-logged as 68 Vulkan fails → D3D11VA → 102 fails → software,
/// ~3 s of black) instead of failing here at open in milliseconds. H.264/HEVC never hit
/// this only because their native decoders happen to be registered first.
///
/// The walk mirrors what `avcodec_find_decoder` would do, restricted to decoders whose
/// `avcodec_get_hw_config` advertises the wanted surface via
/// `AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX` — registry order still wins among those,
/// so H.264/HEVC keep selecting exactly the decoder they always did. The error names
/// the decoders that WERE found, so a log reader can tell "this build has no AV1
/// hwaccel at all" from "no AV1 decoder exists, period".
pub(crate) fn find_hw_decoder(
codec_id: ffmpeg::codec::Id,
hw_pix_fmt: ffmpeg::ffi::AVPixelFormat,
) -> Result<*const ffmpeg::ffi::AVCodec> {
use ffmpeg::ffi;
let want: ffi::AVCodecID = codec_id.into();
let mut found: Vec<String> = Vec::new();
// SAFETY: `av_codec_iterate` walks libav's static codec registry (`opaque` is its
// cursor) and returns static `AVCodec`s; `avcodec_get_hw_config` only reads the
// codec's own static hw-config table, NULL-terminated by returning null past the end.
unsafe {
let mut opaque = std::ptr::null_mut();
loop {
let codec = ffi::av_codec_iterate(&mut opaque);
if codec.is_null() {
break;
}
if (*codec).id != want || ffi::av_codec_is_decoder(codec) == 0 {
continue;
}
for i in 0.. {
let cfg = ffi::avcodec_get_hw_config(codec, i);
if cfg.is_null() {
break;
}
if (*cfg).methods & ffi::AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX as i32 != 0
&& (*cfg).pix_fmt == hw_pix_fmt
{
return Ok(codec);
}
}
found.push(
std::ffi::CStr::from_ptr((*codec).name)
.to_string_lossy()
.into_owned(),
);
}
}
if found.is_empty() {
bail!("no {codec_id:?} decoder in this FFmpeg build");
}
bail!(
"no {codec_id:?} decoder in this FFmpeg build can drive {hw_pix_fmt:?} via \
hw_device_ctx (found: {})",
found.join(", ")
);
}
/// The name of a registry `AVCodec` (`(*codec).name`), owned — the field every decode
/// log carries so `decoder="av1"` vs `decoder="libdav1d"` is one glance, not a debugger.
///
/// # Safety
/// `codec` must point to a registered `AVCodec` (their `name` is a static NUL-terminated
/// string, valid for the process).
pub(crate) unsafe fn codec_name(codec: *const ffmpeg::ffi::AVCodec) -> String {
// SAFETY: caller guarantees a registered AVCodec; `name` is its static C string.
unsafe {
std::ffi::CStr::from_ptr((*codec).name)
.to_string_lossy()
.into_owned()
}
}
/// The `quic` codec bitfield this client can decode — whatever FFmpeg has a decoder for (HEVC/H.264
/// always; AV1 when built in). Advertised to the host so it never emits a codec we can't decode.
pub fn decodable_codecs() -> u8 {
@@ -420,7 +517,11 @@ impl Decoder {
vaapi_tried = true;
match VaapiDecoder::new(codec_id) {
Ok(v) => {
tracing::info!(?codec_id, "VAAPI hardware decode active (zero-copy dmabuf)");
tracing::info!(
?codec_id,
decoder = v.name(),
"VAAPI hardware decode active (zero-copy dmabuf)"
);
return done(Backend::Vaapi(v));
}
Err(e) => {
@@ -455,6 +556,7 @@ impl Decoder {
Ok(d) => {
tracing::info!(
?codec_id,
decoder = d.name(),
"D3D11VA hardware decode active (shared-texture hand-off)"
);
return done(Backend::D3d11va(d));
@@ -475,6 +577,7 @@ impl Decoder {
Ok(v) => {
tracing::info!(
?codec_id,
decoder = v.name(),
"Vulkan Video hardware decode active (presenter-shared device)"
);
return done(Backend::Vulkan(v));
@@ -505,7 +608,11 @@ impl Decoder {
if choice != "software" && choice != "vulkan" && !vaapi_tried {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
tracing::info!(?codec_id, "VAAPI hardware decode active (zero-copy dmabuf)");
tracing::info!(
?codec_id,
decoder = v.name(),
"VAAPI hardware decode active (zero-copy dmabuf)"
);
return done(Backend::Vaapi(v));
}
Err(e) => {
@@ -533,6 +640,7 @@ impl Decoder {
Ok(d) => {
tracing::info!(
?codec_id,
decoder = d.name(),
"D3D11VA hardware decode active (shared-texture hand-off)"
);
return done(Backend::D3d11va(d));
@@ -709,6 +817,7 @@ impl Decoder {
match VaapiDecoder::new(self.codec_id) {
Ok(v) => {
tracing::warn!(error = %e, fails = self.vaapi_fails,
decoder = v.name(),
"Vulkan Video decode failing repeatedly — demoting to VAAPI");
self.backend = Backend::Vaapi(v);
self.vaapi_fails = 0;
@@ -730,6 +839,7 @@ impl Decoder {
) {
Ok(d) => {
tracing::warn!(error = %e, fails = self.vaapi_fails,
decoder = d.name(),
"Vulkan Video decode failing repeatedly — demoting to D3D11VA");
self.backend = Backend::D3d11va(d);
self.vaapi_fails = 0;
@@ -876,6 +986,10 @@ pub struct VulkanDecodeDevice {
/// features). The bundle now exists even without it — Windows D3D11 interop rides the
/// same struct — so consumers gate the FFmpeg-Vulkan decoder on THIS, not on `Some`.
pub video_decode: bool,
/// The presenter has REAL on-glass present timing (`VK_KHR_present_wait` — its
/// `PresentTimer` runs). Gates the `CLIENT_CAP_PHASE_LOCK` advertisement: without a
/// true latch stamp the desktop has no latch grid and must not claim the cap.
pub present_timing: bool,
/// PyroWave decode (the wired-LAN wavelet codec) is usable: Vulkan 1.3 + the compute
/// features its kernels need were present AND enabled at device creation
/// (`shaderInt16`, `storageBuffer8BitAccess`, subgroup size control). Gates the
@@ -950,6 +1064,9 @@ pub(crate) fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32>
Some(match sw {
AV_PIX_FMT_NV12 => fourcc(b'N', b'V', b'1', b'2'),
AV_PIX_FMT_P010LE => fourcc(b'P', b'0', b'1', b'0'),
// Full-chroma 4:4:4 semi-planar (HEVC RExt decode on drivers that export it as
// two planes) — the presenter imports the full-size chroma plane like any other.
AV_PIX_FMT_NV24 => fourcc(b'N', b'V', b'2', b'4'),
_ => return None,
})
}
@@ -984,6 +1101,7 @@ mod tests {
queue_families: Vec::new(),
pyrowave_decode: false,
video_decode: true,
present_timing: false,
d3d11_import: false,
d3d11_hdr10: false,
adapter_luid: None,
@@ -1024,6 +1142,10 @@ mod tests {
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV12),
Some(0x3231_564e)
);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV24),
Some(0x3432_564e)
);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_RGBA),
None
+115 -5
View File
@@ -552,6 +552,10 @@ pub(crate) struct D3d11vaDecoder {
/// ([`crate::video::VulkanDecodeDevice::d3d11_hdr10`]) — PQ streams get the HDR
/// pass-through ring; without it they keep the tonemap-to-sRGB ring.
hdr10_out: bool,
/// The selected decoder's registry name (`(*codec).name`) — `"av1"` vs `"libdav1d"`
/// is the difference between hardware decode and a silent CPU fallback, so every
/// log a field report leans on carries it.
name: String,
}
// SAFETY: the libav pointers are this decoder's own allocations (freed once in `Drop`) and the COM
@@ -609,10 +613,16 @@ impl D3d11vaDecoder {
if !d3d11va_decode_supported(hw_device.as_ptr()) {
bail!("GPU can't create the D3D11VA decode surface pool");
}
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
bail!("no {codec_id:?} decoder");
}
// NOT `avcodec_find_decoder`: the ID lookup returns the registry's FIRST
// decoder, and for AV1 that is libdav1d (upstream orders the hwaccel-only
// native decoder last) — a software decoder that silently ignores
// `hw_device_ctx` and fails every frame's D3D11-format guard mid-stream,
// even when the DXVA profile + pool probes above all passed. Select by
// capability instead: the first decoder that can drive AV_PIX_FMT_D3D11
// via hw_device_ctx, or fail here at open.
let codec =
crate::video::find_hw_decoder(codec_id, ffi::AVPixelFormat::AV_PIX_FMT_D3D11)?;
let name = crate::video::codec_name(codec);
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device.as_ptr());
(*ctx).get_format = Some(get_format_d3d11);
@@ -638,10 +648,16 @@ impl D3d11vaDecoder {
video_context1,
ring: None,
hdr10_out,
name,
})
}
}
/// The selected decoder's registry name (e.g. `"av1"`) — see the field doc.
pub(crate) fn name(&self) -> &str {
&self.name
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<D3d11Frame>> {
use ffmpeg::ffi;
// SAFETY: `packet`/`frame`/`ctx` are this decoder's own allocations, live for its whole
@@ -830,6 +846,7 @@ impl D3d11vaDecoder {
src_desc.Height,
index,
color.is_pq(),
&self.name,
);
Ok(D3d11Frame {
width,
@@ -883,7 +900,15 @@ impl Drop for D3d11vaDecoder {
/// One-time dump of the first decoded surface's layout — the forensics for a new GPU/driver.
/// `tex_*` is the DXVA-aligned decode surface (>= the frame); the gap is the padding the
/// stream source rect excludes.
fn log_layout_once(width: u32, height: u32, tex_w: u32, tex_h: u32, index: u32, pq: bool) {
fn log_layout_once(
width: u32,
height: u32,
tex_w: u32,
tex_h: u32,
index: u32,
pq: bool,
decoder: &str,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
@@ -894,7 +919,92 @@ fn log_layout_once(width: u32, height: u32, tex_w: u32, tex_h: u32, index: u32,
tex_h,
slice = index,
pq,
decoder,
"D3D11VA first frame"
);
}
}
/// This desktop's HDR colour volume (`IDXGIOutput6::GetDesc1`) → the Hello's
/// `display_hdr`, so the host's virtual-display EDID matches THIS panel instead of its
/// generic defaults (host apps then tone-map to the real glass). `pos` picks the output
/// containing that desktop point — the `--window-pos` monitor, where the stream window
/// will open; no `pos` or no match falls back to the output holding the desktop origin
/// (the primary). Returns `None` when that output's advanced color is off (an SDR
/// colorspace): claiming an HDR volume for a desktop that won't present HDR would steer
/// host tone mapping wrong, and the host's EDID defaults are the honest answer there.
/// (`PUNKTFUNK_CLIENT_PEAK_NITS` still overrides whatever this reports — see
/// `punktfunk_core::client::display_hdr_env_override`.)
pub fn display_hdr_volume(pos: Option<(i32, i32)>) -> Option<punktfunk_core::quic::HdrMeta> {
use windows::Win32::dxgi::{IDXGIOutput6, DXGI_OUTPUT_DESC1};
// SAFETY: plain DXGI factory creation — no arguments to get wrong; the returned
// interface is owned by this scope and dropped with it.
let factory: IDXGIFactory1 = unsafe { CreateDXGIFactory1() }.ok()?;
let mut fallback: Option<DXGI_OUTPUT_DESC1> = None;
for a in 0.. {
// SAFETY: read-only enumeration on the live factory; the returned adapter is
// owned by this scope.
let Ok(adapter) = (unsafe { factory.EnumAdapters1(a) }) else {
break;
};
for o in 0.. {
// Out-pointer convention in this windows-rs rev (no retval annotation).
let mut output: Option<windows::Win32::dxgi::IDXGIOutput> = None;
// SAFETY: read-only enumeration on the live adapter, writing a local
// out-pointer that outlives the call.
if unsafe { adapter.EnumOutputs(o, &mut output) }.ok().is_err() {
break;
}
let Some(output) = output else {
break;
};
let Ok(out6) = output.cast::<IDXGIOutput6>() else {
continue; // pre-1809 DXGI — no advanced-color facts to read
};
let mut desc = DXGI_OUTPUT_DESC1::default();
// SAFETY: fills a local, correctly-sized DXGI_OUTPUT_DESC1 that outlives
// the call; the interface is live (owned just above).
if unsafe { out6.GetDesc1(&mut desc) }.ok().is_err() {
continue;
}
let r = desc.DesktopCoordinates;
let contains =
|x: i32, y: i32| x >= r.left && x < r.right && y >= r.top && y < r.bottom;
if let Some((x, y)) = pos {
if contains(x, y) {
return hdr_meta_from_output(&desc);
}
}
if fallback.is_none() || contains(0, 0) {
fallback = Some(desc);
}
}
}
hdr_meta_from_output(&fallback?)
}
/// The ST.2086 shape of one output's colour facts; `None` for an SDR colorspace.
fn hdr_meta_from_output(
d: &windows::Win32::dxgi::DXGI_OUTPUT_DESC1,
) -> Option<punktfunk_core::quic::HdrMeta> {
use windows::Win32::dxgi::DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020;
if d.ColorSpace != DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 {
return None;
}
// Chromaticity → 1/50000 units; luminance → 0.0001 cd/m² units (the HdrMeta contract).
let c = |v: [f32; 2]| {
[
(v[0] * 50_000.0).round().clamp(0.0, 65_535.0) as u16,
(v[1] * 50_000.0).round().clamp(0.0, 65_535.0) as u16,
]
};
Some(punktfunk_core::quic::HdrMeta {
// ST.2086 primary order is G, B, R (see the HdrMeta docs); DXGI reports R/G/B.
display_primaries: [c(d.GreenPrimary), c(d.BluePrimary), c(d.RedPrimary)],
white_point: c(d.WhitePoint),
max_display_mastering_luminance: (f64::from(d.MaxLuminance) * 10_000.0) as u32,
min_display_mastering_luminance: (f64::from(d.MinLuminance) * 10_000.0) as u32,
max_cll: d.MaxLuminance.round().clamp(0.0, 65_535.0) as u16,
max_fall: d.MaxFullFrameLuminance.round().clamp(0.0, 65_535.0) as u16,
})
}
@@ -34,6 +34,12 @@ impl SoftwareDecoder {
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open video decoder")?;
// Every construction site (session open, preference, mid-stream demotion) says
// which decoder actually opened: for AV1 the ID lookup means libdav1d here —
// deliberately (fastest CPU path; the native `av1` decoder has no software
// path at all) — and the name in the log is what keeps that distinguishable
// from the hardware lanes' capability-selected decoders.
tracing::info!(?codec_id, decoder = codec.name(), "software decoder opened");
Ok(SoftwareDecoder { decoder, sws: None })
}
+22 -6
View File
@@ -46,6 +46,10 @@ pub(crate) struct VaapiDecoder {
hw_device: AvBuffer,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
/// The selected decoder's registry name (`(*codec).name`) — `"av1"` vs `"libdav1d"`
/// is the difference between hardware decode and a silent CPU fallback, so every
/// log a field report leans on carries it.
name: String,
}
// SAFETY: the three raw pointers (`ctx`, `packet`, `frame`) are allocations this decoder makes in
@@ -80,11 +84,15 @@ impl VaapiDecoder {
// Owned from here: every `bail!` below drops it, so none of them unref by hand.
let hw_device = AvBuffer::from_raw(hw_device)
.context("av_hwdevice_ctx_create(VAAPI) gave no device")?;
// The negotiated codec's decoder id (av_codec_id maps 1:1 from ffmpeg::codec::Id).
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
bail!("no {codec_id:?} decoder");
}
// NOT `avcodec_find_decoder`: the ID lookup returns the registry's FIRST
// decoder, and for AV1 that is libdav1d (upstream orders the hwaccel-only
// native decoder last) — a software decoder that silently ignores
// `hw_device_ctx` and fails every frame's VAAPI-format guard mid-stream.
// Select by capability instead: the first decoder that can drive
// AV_PIX_FMT_VAAPI via hw_device_ctx, or fail here at open.
let codec =
crate::video::find_hw_decoder(codec_id, ffi::AVPixelFormat::AV_PIX_FMT_VAAPI)?;
let name = crate::video::codec_name(codec);
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device.as_ptr());
(*ctx).get_format = Some(pick_vaapi);
@@ -109,10 +117,16 @@ impl VaapiDecoder {
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
name,
})
}
}
/// The selected decoder's registry name (e.g. `"av1"`) — see the field doc.
pub(crate) fn name(&self) -> &str {
&self.name
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<DmabufFrame>> {
use ffmpeg::ffi;
// SAFETY: `packet`/`frame`/`ctx` are this decoder's own allocations, live for its whole
@@ -207,7 +221,7 @@ impl VaapiDecoder {
// a single modifier for the texture.
let modifier = d.objects[0].format_modifier;
log_descriptor_once(d, sw_format, fourcc, modifier);
log_descriptor_once(d, sw_format, fourcc, modifier, &self.name);
Ok(DmabufFrame {
width: (*self.frame).width as u32,
@@ -233,6 +247,7 @@ fn log_descriptor_once(
sw: ffmpeg_next::ffi::AVPixelFormat,
fourcc: u32,
modifier: u64,
decoder: &str,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
@@ -250,6 +265,7 @@ fn log_descriptor_once(
nb_layers = d.nb_layers,
?layers,
modifier = format_args!("{:#018x}", modifier),
decoder,
"VAAPI dmabuf descriptor layout (first frame)"
);
}
+67 -5
View File
@@ -33,6 +33,10 @@ pub(crate) struct VulkanDecoder {
/// (resolved through the same get_proc_addr chain FFmpeg uses).
wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores,
vk_device: pf_ffvk::VkDevice,
/// The selected decoder's registry name (`(*codec).name`) — `"av1"` vs `"libdav1d"`
/// is the difference between hardware decode and a silent CPU fallback, so every
/// log a field report leans on carries it.
name: String,
/// Storage `AVVulkanDeviceContext` points into (extension string arrays + the
/// feature chain) — FFmpeg reads the extension lists past init (frames-context
/// setup keys code paths off them), so this lives exactly as long as `hw_device`.
@@ -245,10 +249,15 @@ impl VulkanDecoder {
}
let vk_device = (*hwctx).act_dev;
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
bail!("no {codec_id:?} decoder");
}
// NOT `avcodec_find_decoder`: the ID lookup returns the registry's FIRST
// decoder, and for AV1 that is libdav1d (upstream orders the hwaccel-only
// native decoder last) — a software decoder that silently ignores
// `hw_device_ctx` and fails every frame's Vulkan-format guard mid-stream.
// Select by capability instead: the first decoder that can drive
// AV_PIX_FMT_VULKAN via hw_device_ctx, or fail here at open.
let codec =
crate::video::find_hw_decoder(codec_id, ffi::AVPixelFormat::AV_PIX_FMT_VULKAN)?;
let name = crate::video::codec_name(codec);
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device.as_ptr());
(*ctx).get_format = Some(pick_vulkan);
@@ -270,11 +279,17 @@ impl VulkanDecoder {
frame: ffi::av_frame_alloc(),
wait_semaphores,
vk_device,
name,
_ctx_storage: store,
})
}
}
/// The selected decoder's registry name (e.g. `"av1"`) — see the field doc.
pub(crate) fn name(&self) -> &str {
&self.name
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<VkVideoFrame>> {
use ffmpeg::ffi;
// SAFETY: `packet`/`frame`/`ctx` are this decoder's own allocations, live for its whole
@@ -347,10 +362,16 @@ impl VulkanDecoder {
}
let fc = (*hwfc_ref).data as *mut ffi::AVHWFramesContext;
let sw = (*fc).sw_format;
// The 2-plane layouts the presenter's CSC can sample: 4:2:0 (NV12/P010) and
// full-chroma 4:4:4 (NV24/P410 — HEVC RExt decode, semi-planar like all
// NVDEC output). The presenter's `vkframe_plane_formats` table is the final
// authority; anything else bails here so the session demotes cleanly.
if sw != ffi::AVPixelFormat::AV_PIX_FMT_NV12
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P010LE
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_NV24
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P410LE
{
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010 only)");
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010/NV24/P410 only)");
}
let vkfc = (*fc).hwctx as *const pf_ffvk::AVVulkanFramesContext;
let vk_format = (*vkfc).format[0] as i32;
@@ -376,6 +397,14 @@ impl VulkanDecoder {
// sem_value was last written by the decode submission on THIS thread.
let timeline_sem = (*vkf).sem[0] as u64;
let decode_done_value = (*vkf).sem_value[0];
log_layout_once(
(*self.frame).width,
(*self.frame).height,
(*fc).width,
(*fc).height,
sw,
&self.name,
);
Ok(VkVideoFrame {
vkframe: vkf as usize,
frames_ctx: fc as usize,
@@ -386,6 +415,13 @@ impl VulkanDecoder {
decode_done_value,
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
// The pool extent, not the frame's: `avcodec_get_hw_frames_parameters`
// sizes it from `coded_width`/`coded_height` and FFmpeg's Vulkan layer
// rounds that up again to the driver's picture-access granularity. The
// `max` is defensive — a pool SMALLER than the frame would mean sampling
// past the surface, so degrade to "no crop" rather than trust it.
coded_width: ((*fc).width.max((*self.frame).width)) as u32,
coded_height: ((*fc).height.max((*self.frame).height)) as u32,
color: ColorDesc::from_raw(self.frame),
keyframe: frame_is_keyframe(self.frame),
guard: DrmFrameGuard(clone),
@@ -394,6 +430,32 @@ impl VulkanDecoder {
}
}
/// One-time dump of the first decoded frame's layout — the forensics for a new GPU/driver.
/// `pool_*` is the allocated decode surface (`>=` the frame); the gap is the alignment
/// padding the presenter's UV scale excludes. The D3D11VA path logs the same pair.
fn log_layout_once(
width: i32,
height: i32,
pool_w: i32,
pool_h: i32,
sw: ffmpeg::ffi::AVPixelFormat,
decoder: &str,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
tracing::info!(
width,
height,
pool_w,
pool_h,
?sw,
decoder,
"Vulkan Video first frame"
);
}
}
impl Drop for VulkanDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
+304 -7
View File
@@ -19,35 +19,66 @@ use skia_safe::{Canvas, Rect};
enum RowId {
Resolution,
Refresh,
RenderScale,
Bitrate,
Compositor,
Codec,
Decoder,
Hdr,
Chroma444,
PresentPriority,
SmoothBuffer,
Vsync,
AllowVrr,
Audio,
Mic,
EchoCancel,
PadForward,
Pad,
PadType,
Touch,
Mouse,
InvertScroll,
Shortcuts,
Stats,
Fullscreen,
AutoWake,
Library,
}
const ROWS: [RowId; 14] = [
// The couch-relevant subset grew 2026-07-31: this screen is the ONLY settings editor in
// Gaming Mode, so a field it omits is simply unreachable there (render scale, 4:4:4,
// scroll/shortcut behavior, fullscreen-on-stream, auto-wake, the library toggle and echo
// cancellation all were). Still deliberately smaller than the desktop dialogs — device
// pickers (GPU/speaker/mic) and the profile catalog stay desktop-only.
const ROWS: [RowId; 27] = [
RowId::Resolution,
RowId::Refresh,
RowId::RenderScale,
RowId::Bitrate,
RowId::Compositor,
RowId::Codec,
RowId::Decoder,
RowId::Hdr,
RowId::Chroma444,
RowId::PresentPriority,
RowId::SmoothBuffer,
RowId::Vsync,
RowId::AllowVrr,
RowId::Audio,
RowId::Mic,
RowId::EchoCancel,
RowId::PadForward,
RowId::Pad,
RowId::PadType,
RowId::Touch,
RowId::Mouse,
RowId::InvertScroll,
RowId::Shortcuts,
RowId::Stats,
RowId::Fullscreen,
RowId::AutoWake,
RowId::Library,
];
const RESOLUTIONS: [(u32, u32); 6] = [
@@ -59,6 +90,8 @@ const RESOLUTIONS: [(u32, u32); 6] = [
(3840, 2160),
];
const REFRESH: [u32; 5] = [0, 30, 60, 90, 120];
/// Mirrors [`punktfunk_core::render_scale::PRESETS`] (and the desktop pickers).
const RENDER_SCALES: [f64; 9] = [0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0];
const BITRATES: [u32; 7] = [0, 5_000, 10_000, 20_000, 30_000, 50_000, 80_000];
const COMPOSITORS: [(&str, &str); 5] = [
("auto", "Automatic"),
@@ -76,13 +109,35 @@ const CODECS: [(&str, &str); 5] = [
// selected when the host supports it too; anything else falls back to HEVC.
("pyrowave", "PyroWave (wired LAN)"),
];
// Per-OS hardware rungs, like the shells' pickers: the console ships on Windows too
// (`punktfunk-session --browse`), where "vaapi" is a dead option that ALSO hid the real
// hardware path (d3d11va) — `Decoder::new` has no VAAPI branch there.
#[cfg(not(windows))]
const DECODERS: [(&str, &str); 4] = [
("auto", "Automatic"),
("vulkan", "Vulkan Video"),
("vaapi", "VAAPI"),
("software", "Software"),
];
#[cfg(windows)]
const DECODERS: [(&str, &str); 4] = [
("auto", "Automatic"),
("vulkan", "Vulkan Video"),
("d3d11va", "Direct3D 11"),
("software", "Software"),
];
const AUDIO: [(u8, &str); 3] = [(2, "Stereo"), (6, "5.1"), (8, "7.1")];
/// Presentation intent — the `present_priority` key shared with the Apple and Android
/// clients, so one profile reads the same on every device.
const PRESENT_PRIORITIES: [(&str, &str); 2] =
[("latency", "Lowest latency"), ("smooth", "Smoothness")];
/// Smoothness buffer depth in frames; `0` = Automatic (resolves to 2).
const SMOOTH_BUFFERS: [(u8, &str); 4] = [
(0, "Automatic"),
(1, "1 frame"),
(2, "2 frames"),
(3, "3 frames"),
];
const PAD_TYPES: [(&str, &str); 6] = [
("auto", "Automatic"),
("xbox360", "Xbox 360"),
@@ -114,6 +169,15 @@ impl SettingsScreen {
return None;
}
let (msg, pulse) = self.list.menu(ev, ROWS.len());
// Rebase the shell-lifetime snapshot on the file before an adjust-then-save: this
// screen is one of the settings file's several whole-file writers (profiles.rs
// documents the no-merge debt), and adjusting a stale snapshot would silently
// revert what another writer (a session's match-window persist, a desktop shell)
// stored while the console was open. Only on the mutating events — a cursor move
// shouldn't touch the disk.
if matches!(msg, ListMsg::Adjust(_) | ListMsg::Activate) {
*ctx.settings = pf_client_core::trust::Settings::load();
}
match msg {
ListMsg::Adjust(delta) => {
let changed = adjust(ROWS[self.list.cursor], delta, false, ctx);
@@ -179,6 +243,18 @@ impl SettingsScreen {
fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
let s = &ctx.settings;
// Several rows follow another: echo cancellation only means anything while the mic
// streams, the pad rows only while any controller is forwarded at all, and the
// smoothness buffer only while that intent is chosen. All go dim and inert otherwise
// — the same relationship the desktop shells draw by greying a row out (they hide the
// buffer row entirely; a fixed row list can't, and a row that vanished mid-list would
// move everything under the cursor).
let enabled = match id {
RowId::EchoCancel => s.mic_enabled,
RowId::Pad | RowId::PadType => s.gamepad_forwarding,
RowId::SmoothBuffer => s.present_priority == "smooth",
_ => true,
};
let (header, label, value): (Option<&'static str>, &str, String) = match id {
RowId::Resolution => (
Some("Stream"),
@@ -200,6 +276,17 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
format!("{} Hz", s.refresh_hz)
},
),
RowId::RenderScale => (
None,
"Render scale",
if s.render_scale == 1.0 {
"Native".into()
} else if s.render_scale > 1.0 {
format!("{}× (supersample)", s.render_scale)
} else {
format!("{}×", s.render_scale)
},
),
RowId::Bitrate => (
None,
"Bitrate",
@@ -221,6 +308,23 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
),
RowId::Decoder => (None, "Decoder", label_for(&DECODERS, &s.decoder).into()),
RowId::Hdr => (None, "10-bit HDR", on_off(s.hdr_enabled).into()),
RowId::Chroma444 => (None, "Full chroma (4:4:4)", on_off(s.enable_444).into()),
RowId::PresentPriority => (
Some("Presentation"),
"Prioritize",
label_for(&PRESENT_PRIORITIES, &s.present_priority).into(),
),
RowId::SmoothBuffer => (
None,
"Smoothness buffer",
SMOOTH_BUFFERS
.iter()
.find(|(v, _)| *v == s.smooth_buffer)
.map_or("Automatic", |(_, l)| l)
.into(),
),
RowId::Vsync => (None, "V-Sync", on_off(s.vsync).into()),
RowId::AllowVrr => (None, "Follow variable refresh", on_off(s.allow_vrr).into()),
RowId::Audio => (
Some("Audio"),
"Audio channels",
@@ -231,8 +335,14 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
.into(),
),
RowId::Mic => (None, "Microphone", on_off(s.mic_enabled).into()),
RowId::Pad => (
RowId::EchoCancel => (None, "Echo cancellation", on_off(s.echo_cancel).into()),
RowId::PadForward => (
Some("Controller"),
"Forward controllers",
on_off(s.gamepad_forwarding).into(),
),
RowId::Pad => (
None,
"Use controller",
if s.forward_pad.is_empty() {
"Automatic".into()
@@ -254,20 +364,33 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
s.touch_mode().label().into(),
),
RowId::Mouse => (None, "Mouse mode", s.mouse_mode().label().into()),
RowId::InvertScroll => (None, "Invert scroll", on_off(s.invert_scroll).into()),
RowId::Shortcuts => (
None,
"Capture system shortcuts",
on_off(s.inhibit_shortcuts).into(),
),
RowId::Stats => (
Some("Interface"),
"Statistics overlay",
s.stats_verbosity().label().into(),
),
RowId::Fullscreen => (
None,
"Start streams fullscreen",
on_off(s.fullscreen_on_stream).into(),
),
RowId::AutoWake => (None, "Wake hosts automatically", on_off(s.auto_wake).into()),
RowId::Library => (None, "Game library", on_off(s.library_enabled).into()),
};
RowSpec {
header,
label: label.into(),
value: Some(value),
value_dim: false,
value_dim: !enabled,
caret: false,
adjustable: true,
enabled: true,
adjustable: enabled,
enabled,
}
}
@@ -278,6 +401,10 @@ fn detail(id: RowId) -> &'static str {
Match window follows this window, including mid-stream resizes."
}
RowId::Refresh => "Native follows the display this window is on.",
RowId::RenderScale => {
"The host renders larger or smaller than the stream mode and this window \
resamples above 1× supersamples, below saves bandwidth."
}
RowId::Bitrate => "Automatic uses the host's default (20 Mbps).",
RowId::Compositor => {
"Which compositor drives the virtual output — honored only if available on the host."
@@ -287,8 +414,43 @@ fn detail(id: RowId) -> &'static str {
RowId::Hdr => {
"HDR10 — engages when the host sends HDR content and this display supports it."
}
RowId::Chroma444 => {
"Full-colour video: crisp small text and thin lines, at more bandwidth. \
Needs an NVIDIA host (NVENC) or the PyroWave codec other encoders \
stream 4:2:0 and the session falls back silently."
}
RowId::PresentPriority => {
"Lowest latency shows each frame the moment the display can take it — a \
network hiccup becomes an occasional repeated or skipped frame. Smoothness \
buffers a little to even those out."
}
RowId::SmoothBuffer => {
"Frames held back before showing. Each one absorbs about a refresh of network \
hiccup and adds a refresh of delay. Automatic holds two."
}
RowId::Vsync => {
"Tear-free. Off removes the wait for the screen's refresh — the lowest \
possible delay, at the cost of visible tearing. Not every driver offers it; \
the stats overlay names the mode actually in use."
}
RowId::AllowVrr => {
"On a VRR screen, let the panel refresh in step with the stream instead of on \
a fixed cadence. Applies to fullscreen sessions; harmless on a fixed screen."
}
RowId::Audio => "The speaker layout requested from the host.",
RowId::Mic => "Send this device's microphone to the host's virtual mic.",
RowId::Mic => {
"Send this device's microphone to the host's virtual mic. \
Ctrl+Alt+Shift+V mutes and unmutes it while streaming."
}
RowId::EchoCancel => {
"Stops the host's audio, playing from this device's speakers, being picked up \
and sent back. Turn it off if your microphone already runs its own processing."
}
RowId::PadForward => {
"Send controllers connected to this device to the host. Turn it off when your \
controller already reaches the host another way USB passthrough such as \
VirtualHere, or a pad plugged into the host so games don't see two of them."
}
RowId::Pad => "Which pad is forwarded to the host, as player 1.",
RowId::PadType => "The virtual pad the host creates — Automatic matches this controller.",
RowId::Touch => {
@@ -300,10 +462,21 @@ fn detail(id: RowId) -> &'static str {
for games), Desktop leaves it free and sends absolute positions. \
Ctrl+Alt+Shift+M switches live while streaming."
}
RowId::InvertScroll => "Reverses the wheel and trackpad scroll direction sent to the host.",
RowId::Shortcuts => {
"Alt+Tab, Super and friends reach the host while input is captured. \
Off, they act on this device instead."
}
RowId::Stats => {
"How much the overlay shows: Compact (one line) → Normal → Detailed. \
Ctrl+Alt+Shift+S cycles it live while streaming."
}
RowId::Fullscreen => "Streams open fullscreen instead of windowed.",
RowId::AutoWake => {
"Send Wake-on-LAN to a sleeping host before connecting. Turn off for hosts \
reached over a VPN, where the wake wait only adds delay."
}
RowId::Library => "Show paired hosts' game libraries (tap a title to stream it).",
}
}
@@ -348,6 +521,13 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
let cur = REFRESH.iter().position(|r| *r == s.refresh_hz);
step_option(cur, REFRESH.len(), delta, wrap).map(|i| s.refresh_hz = REFRESH[i])
}
RowId::RenderScale => {
// Exact float compare is fine: every writer (here and the desktop pickers)
// stores one of these literals; a hand-edited oddball snaps to the first step.
let cur = RENDER_SCALES.iter().position(|v| *v == s.render_scale);
step_option(cur, RENDER_SCALES.len(), delta, wrap)
.map(|i| s.render_scale = RENDER_SCALES[i])
}
RowId::Bitrate => {
let cur = BITRATES.iter().position(|b| *b == s.bitrate_kbps);
step_option(cur, BITRATES.len(), delta, wrap).map(|i| s.bitrate_kbps = BITRATES[i])
@@ -356,12 +536,46 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
RowId::Codec => step_str(&CODECS, &mut s.codec, delta, wrap),
RowId::Decoder => step_str(&DECODERS, &mut s.decoder, delta, wrap),
RowId::Hdr => toggle(&mut s.hdr_enabled, delta, wrap),
RowId::Chroma444 => toggle(&mut s.enable_444, delta, wrap),
RowId::PresentPriority => {
let cur = PRESENT_PRIORITIES
.iter()
.position(|(v, _)| *v == s.present_priority);
step_option(cur, PRESENT_PRIORITIES.len(), delta, wrap)
.map(|i| s.present_priority = PRESENT_PRIORITIES[i].0.to_string())
}
// Inert unless smoothness is chosen — a boundary thud, matching the dimmed row.
RowId::SmoothBuffer => {
if s.present_priority == "smooth" {
let cur = SMOOTH_BUFFERS
.iter()
.position(|(v, _)| *v == s.smooth_buffer);
step_option(cur, SMOOTH_BUFFERS.len(), delta, wrap)
.map(|i| s.smooth_buffer = SMOOTH_BUFFERS[i].0)
} else {
None
}
}
RowId::Vsync => toggle(&mut s.vsync, delta, wrap),
RowId::AllowVrr => toggle(&mut s.allow_vrr, delta, wrap),
RowId::Audio => {
let cur = AUDIO.iter().position(|(v, _)| *v == s.audio_channels);
step_option(cur, AUDIO.len(), delta, wrap).map(|i| s.audio_channels = AUDIO[i].0)
}
RowId::Mic => toggle(&mut s.mic_enabled, delta, wrap),
// Inert while the mic is off — a boundary thud, matching what the dimmed row shows.
RowId::EchoCancel => {
if s.mic_enabled {
toggle(&mut s.echo_cancel, delta, wrap)
} else {
None
}
}
RowId::PadForward => toggle(&mut s.gamepad_forwarding, delta, wrap),
RowId::Pad => {
if !s.gamepad_forwarding {
return false;
}
// Automatic first, then every connected pad by stable key.
let keys: Vec<String> = std::iter::once(String::new())
.chain(ctx.pads.iter().map(|p| p.key.clone()))
@@ -369,7 +583,12 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
let cur = keys.iter().position(|c| *c == s.forward_pad);
step_option(cur, keys.len(), delta, wrap).map(|i| s.forward_pad = keys[i].clone())
}
RowId::PadType => step_str(&PAD_TYPES, &mut s.gamepad, delta, wrap),
RowId::PadType => {
if !s.gamepad_forwarding {
return false;
}
step_str(&PAD_TYPES, &mut s.gamepad, delta, wrap)
}
RowId::Touch => {
let cur = TouchMode::ALL.iter().position(|m| *m == s.touch_mode());
step_option(cur, TouchMode::ALL.len(), delta, wrap)
@@ -380,6 +599,8 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, MouseMode::ALL.len(), delta, wrap)
.map(|i| s.mouse_mode = MouseMode::ALL[i].as_name().to_string())
}
RowId::InvertScroll => toggle(&mut s.invert_scroll, delta, wrap),
RowId::Shortcuts => toggle(&mut s.inhibit_shortcuts, delta, wrap),
RowId::Stats => {
let cur = StatsVerbosity::ALL
.iter()
@@ -387,6 +608,9 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, StatsVerbosity::ALL.len(), delta, wrap)
.map(|i| s.set_stats_verbosity(StatsVerbosity::ALL[i]))
}
RowId::Fullscreen => toggle(&mut s.fullscreen_on_stream, delta, wrap),
RowId::AutoWake => toggle(&mut s.auto_wake, delta, wrap),
RowId::Library => toggle(&mut s.library_enabled, delta, wrap),
}
.is_some()
}
@@ -494,6 +718,79 @@ mod tests {
assert!(!ctx.settings.mic_enabled);
}
/// Echo cancellation follows the microphone: inert and dimmed while the mic is off, live
/// the moment it goes on. A row that silently accepted a change nobody could act on would
/// be the same lie as an enabled-looking control.
#[test]
fn echo_cancellation_follows_the_microphone() {
let (mut settings, pads) = ctx_parts();
settings.mic_enabled = false;
assert!(settings.echo_cancel, "it ships on");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
assert!(!row_spec(RowId::EchoCancel, &ctx).enabled);
assert!(
!adjust(RowId::EchoCancel, -1, false, &mut ctx),
"mic off = thud"
);
assert!(!adjust(RowId::EchoCancel, 1, true, &mut ctx), "A too");
assert!(ctx.settings.echo_cancel, "and nothing was written");
ctx.settings.mic_enabled = true;
assert!(row_spec(RowId::EchoCancel, &ctx).enabled);
assert!(adjust(RowId::EchoCancel, -1, false, &mut ctx));
assert!(!ctx.settings.echo_cancel);
assert!(adjust(RowId::EchoCancel, 1, true, &mut ctx));
assert!(ctx.settings.echo_cancel);
}
/// The smoothness buffer follows the presentation intent, exactly as echo cancellation
/// follows the mic: dimmed and inert under Lowest latency (where holding frames means
/// nothing), live under Smoothness. The desktop shells hide the row instead; a fixed
/// row list dims it, because a row vanishing mid-list would shift everything under the
/// cursor.
#[test]
fn smoothness_buffer_follows_the_intent() {
let (mut settings, pads) = ctx_parts();
assert_eq!(settings.present_priority, "latency", "the shipped default");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
assert!(!row_spec(RowId::SmoothBuffer, &ctx).enabled);
assert!(
!adjust(RowId::SmoothBuffer, 1, false, &mut ctx),
"latency intent = thud"
);
assert_eq!(ctx.settings.smooth_buffer, 0, "and nothing was written");
// Stepping the intent to Smoothness brings the buffer row to life.
assert!(adjust(RowId::PresentPriority, 1, false, &mut ctx));
assert_eq!(ctx.settings.present_priority, "smooth");
assert!(row_spec(RowId::SmoothBuffer, &ctx).enabled);
assert!(adjust(RowId::SmoothBuffer, 1, false, &mut ctx));
assert_eq!(ctx.settings.smooth_buffer, 1);
// The intent wraps back and the row goes inert again.
assert!(adjust(RowId::PresentPriority, -1, false, &mut ctx));
assert_eq!(ctx.settings.present_priority, "latency");
assert!(!row_spec(RowId::SmoothBuffer, &ctx).enabled);
}
#[test]
fn touch_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
+58 -1
View File
@@ -48,6 +48,9 @@ struct Drawn {
height: u32,
stats: Option<String>,
hint: Option<String>,
/// The mic-mute badge is up. Part of the damage key like everything else here — the badge
/// is static once drawn, so a muted stream still re-renders nothing per frame.
mic_muted: bool,
/// The UI scale this was drawn at, in percent — part of the damage key so dragging the window
/// to a differently-scaled monitor re-renders the chrome at the new size instead of keeping
/// the stale one (the text is identical, so nothing else here would notice).
@@ -390,7 +393,12 @@ impl Overlay for SkiaOverlay {
// spinning through the damage gate; `+ 1` keeps an active resize's step nonzero
// even on its first frame (phase 0) so the guard below doesn't skip it.
let resize_step = resize_phase.map_or(0, |p| (p * 120.0) as u16 + 1);
if ctx.stats.is_none() && ctx.hint.is_none() && banner_step == 0 && resize_step == 0 {
if ctx.stats.is_none()
&& ctx.hint.is_none()
&& !ctx.mic_muted
&& banner_step == 0
&& resize_step == 0
{
self.drawn = Drawn::default(); // forget content so re-show re-renders
return Ok(None);
}
@@ -402,6 +410,7 @@ impl Overlay for SkiaOverlay {
height: ctx.height,
stats: ctx.stats.map(str::to_owned),
hint: ctx.hint.map(str::to_owned),
mic_muted: ctx.mic_muted,
scale_pct: (scale * 100.0).round() as u16,
banner_step,
resize_step,
@@ -437,6 +446,11 @@ impl Overlay for SkiaOverlay {
if let Some(stats) = &want.stats {
draw_osd_panel(canvas, font, stats, ctx.width, scale);
}
// Top-RIGHT, so it never collides with the stats panel or the bottom pill: the badge
// has to stay readable at every stats tier, including Off.
if want.mic_muted {
draw_mic_muted_badge(canvas, font, ctx.width, scale);
}
if let Some(hint) = &want.hint {
draw_hint_pill(canvas, font, hint, ctx.width, ctx.height, 1.0, scale);
} else if banner_step > 0 {
@@ -660,6 +674,49 @@ fn draw_osd_panel(canvas: &Canvas, base_font: &Font, text: &str, width: u32, sca
}
}
/// The mic-mute badge: a dot in the error colour plus the words, on the same translucent pill
/// as the rest of the chrome, pinned to the TOP-RIGHT corner.
///
/// It is drawn from `FrameCtx::mic_muted` alone — not from the stats text — because it must
/// survive the stats overlay being Off, which is where most people leave it. Words, not a
/// glyph: the chrome font is a system monospace resolved at runtime and cannot be relied on to
/// carry a crossed-out microphone. Persistent by design (no fade): a fading indicator answers
/// "did the chord register?" but not "am I muted right now?", and the second question is the
/// one that matters ten minutes later.
fn draw_mic_muted_badge(canvas: &Canvas, base_font: &Font, width: u32, scale: f32) {
const LABEL: &str = "Microphone muted";
// Short line — it fits any window the stream runs in, so it takes the display scale as-is.
let font = &chrome_font(base_font, scale);
let (_, metrics) = font.metrics();
let line_h = metrics.descent - metrics.ascent;
let (pad_x, pad_y) = (base::PILL_PAD_X * scale, base::PILL_PAD_Y * scale);
let dot_r = 4.0 * scale;
let dot_gap = 8.0 * scale;
let text_w = font.measure_str(LABEL, None).0;
let w = text_w + 2.0 * dot_r + dot_gap + 2.0 * pad_x;
let h = line_h + 2.0 * pad_y;
let margin = base::OSD_MARGIN * scale;
let (x, y) = (width as f32 - w - margin, margin);
canvas.draw_rrect(
RRect::new_rect_xy(Rect::from_xywh(x, y, w, h), h / 2.0, h / 2.0),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.62), None),
);
canvas.draw_circle(
Point::new(x + pad_x + dot_r, y + h / 2.0),
dot_r,
&Paint::new(crate::theme::ERROR, None),
);
canvas.draw_str(
LABEL,
Point::new(
x + pad_x + 2.0 * dot_r + dot_gap,
y + pad_y - metrics.ascent,
),
font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.92), None),
);
}
/// The mid-stream-resize cover: a full-screen dark scrim, the shared rotating spinner, and
/// a "Resizing…" label centered over it — so the host's 0.32 s virtual-display + encoder
/// rebuild reads as a deliberate pause rather than the stream stretching to the changed
+5 -3
View File
@@ -6,7 +6,7 @@
use crate::anim::{approach, Spring, TRAY_C, TRAY_K};
use crate::library::{BUMP_C, BUMP_K};
use crate::theme::{brand, white, Fonts, PanelStroke, BRAND, FAINT, W, WHITE};
use crate::theme::{brand, white, Fonts, PanelStroke, BRAND, DIM, FAINT, W, WHITE};
use pf_client_core::gamepad::{MenuDir, MenuEvent, MenuPulse};
use skia_safe::{Canvas, Paint, Path, RRect, Rect};
@@ -35,7 +35,9 @@ pub(crate) struct RowSpec {
pub caret: bool,
/// Show chevrons while focused (left/right steps the value).
pub adjustable: bool,
/// Action rows render dimmed when not yet actionable.
/// Rows render dimmed when they aren't actionable: an action row's centered label loses
/// its brand tint, a value row's label greys — the look for a setting that depends on
/// another one being on (Echo cancellation under Microphone).
pub enabled: bool,
}
@@ -229,7 +231,7 @@ impl MenuList {
baseline,
W::SemiBold,
16.0 * k,
WHITE,
if row.enabled { WHITE } else { DIM },
);
let value = row.value.as_deref().unwrap_or_default();
let vcolor = if row.value_dim {
+5 -4
View File
@@ -117,10 +117,11 @@ pub(super) fn resolve_split_mode(bit_depth: u8, pixel_rate: u64) -> u32 {
/// deserves a `warn`, a default being tuned an `info`. Callers LATCH this once next to their
/// resolved subframe state (an env re-read at reconfigure would violate the "open and
/// reconfigure present identical init params" invariant).
/// Linux-cfg'd like its ONLY caller (the `nvenc_cuda` query_caps latch) — Windows sessions have
/// `subframe == forced` by construction (env opt-in only) and never consult this; without the
/// cfg it is dead code on every Windows leg (item-level dead_code, the recurring trap).
#[cfg(target_os = "linux")]
/// Both direct-SDK backends latch it now: Linux at the `nvenc_cuda` query_caps latch, Windows at
/// session init since sub-frame defaults on there too (it used to be env opt-in only, so
/// `subframe == forced` held by construction and the item was Linux-cfg'd to avoid being dead
/// code on the Windows leg — the recurring item-level `dead_code` trap).
#[cfg(any(target_os = "linux", windows))]
pub(super) fn subframe_env_forced() -> bool {
matches!(
std::env::var("PUNKTFUNK_NVENC_SUBFRAME").as_deref(),
+22 -9
View File
@@ -45,7 +45,7 @@
use super::nvenc_core::{
apply_low_latency_config, build_init_params, cached_ceiling, codec_guid, plan_range_recovery,
resolve_slices, resolve_split_mode, resolve_split_subframe, resolve_subframe, store_ceiling,
CeilingKey, LowLatencyConfig, NvStatusExt, RangePlan,
subframe_env_forced, CeilingKey, LowLatencyConfig, NvStatusExt, RangePlan,
};
use super::nvenc_status;
use super::{AuChunk, ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
@@ -588,6 +588,10 @@ pub struct NvencD3d11Encoder {
input_ring_depth: Option<usize>,
/// `NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT` from the caps probe — gates the async retrieve mode.
async_supported: bool,
/// `NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK` from the caps probe — gates the DEFAULT-on
/// sub-frame readback (the Linux backend's rule since its Phase 3; Windows joined after the
/// 2026-07-31 on-glass A/B), so a GPU without it never has sub-frame forced by default.
subframe_cap: bool,
/// (bitstream, mapped input resource to unmap after retrieval, pts_ns, recovery-anchor) per
/// in-flight encode. The fourth field tags the first frame encoded after a successful
/// [`invalidate_ref_frames`](Encoder::invalidate_ref_frames) — the clean re-anchor P-frame the
@@ -748,6 +752,7 @@ impl NvencD3d11Encoder {
async_rt: None,
input_ring_depth: None,
async_supported: false,
subframe_cap: false,
pending: VecDeque::new(),
frame_idx: 0,
force_kf: false,
@@ -922,6 +927,7 @@ impl NvencD3d11Encoder {
nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_CUSTOM_VBV_BUF_SIZE,
);
let async_enc = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT);
let subframe = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK);
let _ = (api().destroy_encoder)(enc);
// Reject an over-range mode with a clear message instead of an opaque InvalidParam.
@@ -955,10 +961,12 @@ impl NvencD3d11Encoder {
self.rfi_supported = rfi != 0;
self.custom_vbv = custom_vbv != 0;
self.async_supported = async_enc != 0;
self.subframe_cap = subframe != 0;
tracing::info!(
rfi = self.rfi_supported,
custom_vbv = self.custom_vbv,
async_encode = self.async_supported,
subframe_readback = self.subframe_cap,
max = %format!("{wmax}x{hmax}"),
ten_bit = ten_bit != 0,
"NVENC capabilities probed"
@@ -1152,11 +1160,14 @@ impl NvencD3d11Encoder {
// VIDEO_CAP_MULTI_SLICE / Moonlight slices-per-frame client gets real slices.
// `PUNKTFUNK_NVENC_SLICES` stays the operator override in both directions.
self.slices = resolve_slices(self.codec, 4.min(self.max_slices));
// Split × sub-frame arbitration (Phase 8) before the ladder/ceiling key. On Windows
// sub-frame is env-opt-in only, so resolved == forced by construction.
let subframe_req = resolve_subframe(false);
// Split × sub-frame arbitration (Phase 8) before the ladder/ceiling key. Sub-frame
// defaults ON where the GPU advertises SUBFRAME_READBACK (Linux parity; validated by
// the 2026-07-31 .173 on-glass A/B — no regression, and slice-progressive clients
// gain the encode/wire overlap); `PUNKTFUNK_NVENC_SUBFRAME` stays the tri-state
// operator escape in both directions.
let subframe_req = resolve_subframe(self.subframe_cap);
let (split_mode, subframe_req) =
resolve_split_subframe(self.codec, split_mode, subframe_req, subframe_req);
resolve_split_subframe(self.codec, split_mode, subframe_req, subframe_env_forced());
// Find the highest bitrate the GPU's codec LEVEL accepts and CLAMP to it. NVENC rejects
// `initialize_encoder` (InvalidParam) when the bitrate exceeds the level ceiling (e.g. a
// 1 Gbps request on HEVC). Strategy: try the requested rate; if the only problem is a forced
@@ -1318,8 +1329,7 @@ impl NvencD3d11Encoder {
self.session_async = use_async;
// Sub-frame chunked poll (P2f, the Windows leg of the slice pipeline): sync
// retrieve only — chunked poll is a depth-1 sync feature; the async retrieve's
// thread owns the bitstream lock. Sub-frame write itself stays env-gated
// (`PUNKTFUNK_NVENC_SUBFRAME=1`) until the Windows on-glass A/B validates it.
// thread owns the bitstream lock.
self.subframe_chunks = self.slices >= 2 && subframe_req && !use_async;
if self.subframe_chunks {
tracing::info!(
@@ -1659,8 +1669,11 @@ impl Encoder for NvencD3d11Encoder {
let anchor = std::mem::take(&mut self.pending_anchor) && flags == 0;
// Submit-time IDR intent: chunked poll must flag an AU's EARLY chunks before the
// driver reports `pictureType` (only the finishing lock sees it). Exact under
// P-only + infinite GOP: IDRs happen only when forced.
let idr_hint = flags != 0;
// P-only + infinite GOP: IDRs happen only when forced — or on the session-opening
// frame, which NVENC emits as an IDR regardless of pic flags (the Linux twin's
// `is_idr`; without the `opening` term frame 1's early chunks went out unflagged
// and the divergence WARN fired at every session start).
let idr_hint = flags != 0 || opening;
let mut pic = nv::NV_ENC_PIC_PARAMS {
version: nv::NV_ENC_PIC_PARAMS_VER,
inputWidth: self.width,
@@ -669,6 +669,11 @@ impl GamepadManager {
/// Service every pad's FF protocol; `send(index, low, high)` is invoked for each pad whose
/// mixed rumble level changed. Call frequently (games block in `EVIOCSFF` until answered).
pub fn pump_rumble(&mut self, mut send: impl FnMut(u16, u16, u16)) {
// Finish any unplug whose removal frame only armed the grace — the producer sends that
// frame once, so without this the uinput node would outlive the controller. The swept
// mask is discarded because this manager keeps no per-index sibling state (the pads mix
// rumble internally); if that ever changes, consume it like the other two backends do.
self.slots.reap();
for (i, pad) in self.slots.iter_mut() {
if let Some((low, high)) = pad.pump_ff() {
send(i as u16, low, high);

Some files were not shown because too many files have changed in this diff Show More