Compare commits

...

26 Commits

Author SHA1 Message Date
enricobuehler 87e7c82cbc chore(release): bump workspace version to 0.17.2
android / android (push) Has started running
arch / build-publish (push) Has started running
ci / rust (push) Has started running
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Waiting to run
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Waiting to run
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Waiting to run
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Waiting to run
windows / build (aarch64-pc-windows-msvc) (push) Waiting to run
windows / build (x86_64-pc-windows-msvc) (push) Waiting to run
windows-host / package (push) Has started running
audit / bun-audit (push) Failing after 18s
ci / bench (push) Has started running
ci / web (push) Successful in 54s
deb / build-publish (push) Has started running
ci / docs-site (push) Successful in 1m2s
deb / build-publish-host (push) Has started running
apple / swift (push) Successful in 1m23s
apple / screenshots (push) Waiting to run
release / apple (push) Has started running
audit / cargo-audit (push) Successful in 2m44s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 32s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m9s
flatpak / build-publish (push) Has started running
docker / deploy-docs (push) Waiting to run
2026-07-21 23:33:53 +02:00
enricobuehler b2e3d1b540 fix(steamdeck/install): add cmake dep + acquire sudo in preflight
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m52s
apple / swift (push) Successful in 1m22s
android / android (push) Successful in 13m4s
ci / bench (push) Successful in 8m2s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
deb / build-publish (push) Successful in 9m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m14s
apple / screenshots (push) Successful in 6m51s
arch / build-publish (push) Successful in 18m54s
docker / deploy-docs (push) Successful in 27s
ci / rust (push) Successful in 19m17s
deb / build-publish-host (push) Successful in 9m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
pyrowave-sys's build.rs runs cmake to build the vendored PyroWave C library,
but cmake wasn't in the distrobox apt list — so `cargo build -p punktfunk-host`
dies on every SteamOS host with "is cmake not installed?". Add it.

Also move sudo acquisition from step 4 (after the ~15-min build) into preflight
and keep the timestamp warm across the build. Before, a non-interactive run
(`ssh host 'bash install.sh'`) silently skipped the UDP-buffer/udev/vhci-hcd/
input-group tuning AND enable-linger — a "successful" host with no gamepad
passthrough, lossy streaming, and no headless persistence (dies on logout).
Now sudo is prompted once up front (authorize and walk away), and a no-TTY /
no-password run fails loudly before the build instead of silently at the end.

Validated end-to-end on SteamOS 3.8 in a VM with Radeon 780M passthrough:
full host + web build, all planes listening, VAAPI render node, headless
auto-start via linger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehler d36bec6e9d feat(core+host): negotiate ChaCha20-Poly1305 as the session cipher for soft-AES clients
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Failing after 13s
audit / cargo-audit (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Successful in 10m1s
windows-host / package (push) Successful in 11m20s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m39s
Lifts the ~100 Mbps decrypt ceiling on clients without hardware AES — the
armv7 soft-AES targets (webOS TVs), where AES-128-GCM resolves to fixsliced
software AES + software GHASH (~50-100 cpb) while ChaCha20-Poly1305's ARX
construction runs ~10-17 cpb portable, a 4-7x lift that PyroWave-on-TV needs
(design/chacha20-session-cipher.md).

Phase 1 (core crypto, no wire change): SessionKey merges cipher choice and
key material (invalid combinations unrepresentable, zeroize + redacted-Debug
discipline kept); SessionCrypto dispatches both aead-0.5 ciphers per call —
the salt||seq nonce scheme, per-direction salts, seq-as-AAD and replay
window carry over verbatim (same 96-bit nonce / 16-byte tag, const-asserted).
Config.key becomes SessionKey; validate's zero-key rejection follows the
active variant. The C ABI keeps its fixed 16-byte key mapped to AES — no
ABI_VERSION bump.

Phase 2 (negotiation): VIDEO_CAP_CHACHA20 (0x40) — support-plus-request in
one bit, the VIDEO_CAP_444 precedent. Welcome grows cipher@68 +
key_chacha@69..101, emitted only when non-zero so an AES session's Welcome
stays byte-identical to the pre-cipher form; decode is fail-closed (short
key or unknown id -> Err, never a silent AES fallback). No WIRE_VERSION
bump; downgrade resistance inherited from the pinned-TLS control channel.

Phase 3 (host): grant only when the client advertised the bit and the
PUNKTFUNK_CHACHA20 kill-switch (default on, documented) allows; fresh
32-byte per-session key from the same RNG discipline, legacy key field
stays independently random; resolved cipher logged at session start.

Verification: seal/open suites parameterized over both ciphers + a
cross-cipher tamper case; Welcome roundtrip/truncation/fail-closed tests;
ChaCha lossy-loopback soak (loss/replay is cipher-independent); bench
gains _chacha20 series (AES ids unchanged for CI history) — host-side
sealing line-rate-trivial on both x86 (~640 MiB/s) and Apple Silicon
(~535 MiB/s). punktfunk-probe drives the interop matrix via
PUNKTFUNK_CLIENT_CHACHA20=1 and logs the negotiated cipher.

Phase 4 (pf-webos pin bump + unconditional cap bit) follows the next
core release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehler abc54a7d13 fix(decky): actually ship shortcut artwork + Steam Input layout in the published plugin
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m58s
The CI workflow assembled the store zip by hand and silently drifted from
scripts/package.sh when 60d46530 added assets/ and controller_config/ — every
published build (canary and stable through 0.17.1) was missing both, so the
Steam shortcut a fresh install creates had no images and no controller layout
(shortcut_art() found nothing to send; apply_controller_config failed with
template-missing). CI now stages through scripts/package.sh — the plugin file
list lives in one place — and a backstop verifies the runtime-loaded pieces are
in the zip before publishing.

Frontend hardening for installs poisoned by those builds: applyArtwork stamped
its per-appId ART_VERSION marker even when zero assets were applied, so
shipping the files alone would never heal existing shortcuts — only mark done
when something actually landed, and bump ART_VERSION 2→3 to force one re-apply.
Same guard for the controller-config marker (ok with zero account configset
dirs is not done), plus one deferred artwork retry for the fresh-AddShortcut
registration race setShortcutHidden already defers around.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehler 309e37f1e1 fix(host/linux): scope the GPU clock pin to live client sessions
The PUNKTFUNK_PIN_CLOCKS clock pin (AMD power_dpm_force_performance_level=high / NVIDIA nvmlDeviceSetGpuLockedClocks) was armed once at host start and released only at host exit, so the box's clocks stayed pinned the whole time the host ran — even with no client connected.

Move the pin behind a box-wide refcount (gpuclocks::session_pin()) shared across both streaming planes (native + GameStream): it arms on the first live client and releases on the last disconnect. on_host_start() now only installs the process-scoped NVIDIA P2-cap application profile. No behavior change when PUNKTFUNK_PIN_CLOCKS is unset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehler 56d21f9445 fix(client/wol): retry the QUIC dial across the connect budget so wake-from-sleep launches survive
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 40s
ci / bench (push) Successful in 6m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish (push) Successful in 9m54s
release / apple (push) Successful in 9m5s
deb / build-publish-host (push) Successful in 9m35s
flatpak / build-publish (push) Failing after 8m19s
android / android (push) Successful in 16m57s
apple / screenshots (push) Successful in 6m29s
arch / build-publish (push) Successful in 17m35s
windows-host / package (push) Successful in 18m46s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
docker / deploy-docs (push) Successful in 26s
ci / rust (push) Successful in 25m24s
Launching from the Decky plugin against a sleeping host fired the
Wake-on-LAN packet and dialed immediately — but the dial was a single
quinn attempt that gave up after the transport's 8 s idle window, far
shorter than a suspend-to-RAM resume. The host woke to find the client
already gone (its retransmitted Initials show up host-side as
"QUIC accept failed error=timed out"), the stream shortcut exited, and
the launch looked cancelled.

- punktfunk-core: dial in a loop until the embedder's connect budget is
  spent — short 3 s attempts keep the Initial cadence dense, so the
  connect lands within ~a second of the host's network returning. Only
  silence is retried: pin mismatches, typed rejections, and any real
  answer surface immediately, and the shutdown flag stops the loop.
- decky: stretch the budget to 75 s when a magic packet actually went
  out (PF_CONNECT_TIMEOUT → --connect-timeout via the wrapper), so the
  retry window covers the resume; an awake host still connects in <1 s.
- host: a clean close before the control handshake is a reachability
  probe (hosts-page online pips), not a failed session — log it at
  debug instead of WARN "session ended with error", which buried the
  real failures in the reporter's log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:26:10 +02:00
enricobuehler f36d13e371 fix(steamdeck/install): prompt for sudo instead of requiring passwordless
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m5s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m43s
docker / deploy-docs (push) Successful in 48s
deb / build-publish-host (push) Successful in 10m9s
android / android (push) Successful in 18m18s
arch / build-publish (push) Successful in 19m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m16s
ci / rust (push) Successful in 28m27s
The system-tuning block (UDP buffers, gamepad udev rule, vhci-hcd autoload,
input group) was gated on `sudo -n true` — passwordless sudo — so on a stock
SteamOS box (the deck account needs a password) the whole block was silently
skipped: /dev/uhid stayed root-only so the gamepad degraded to an Xbox 360 pad,
and the UDP buffers stayed at the 416 KB default. Video was unaffected because
Game Mode/gamescope needs none of it, which masked the skip on-glass.

- install.sh + update.sh: acquire sudo interactively (`sudo -v` prompt on a TTY)
  instead of requiring passwordless; skip only when there's no TTY or auth fails,
  and then print the exact manual block plus a `passwd` hint — a stock SteamOS
  'deck' account has no password, so sudo can't work until one is set.
- update.sh: also retrofit the UDP-buffer sysctl for older/skipped installs, and
  loudly nag to reboot when the input group was just added (a --user restart does
  not pick up the new group; only a fresh login does).
- steamos-host.md: note this step prompts for the sudo password.

Root is unavoidable: the udev rule, input group, and vhci-hcd module are all
kernel operations with no user-space alternative.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:56:20 +02:00
enricobuehler aacd610b68 fix(steamdeck/install): set up controller passthrough + surface the required reboot
ci / docs-site (push) Successful in 1m12s
ci / web (push) Successful in 1m13s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m35s
apple / screenshots (push) Successful in 6m13s
deb / build-publish (push) Successful in 10m21s
docker / deploy-docs (push) Successful in 34s
deb / build-publish-host (push) Successful in 9m58s
android / android (push) Successful in 17m7s
arch / build-publish (push) Successful in 17m47s
ci / rust (push) Successful in 19m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m2s
A fresh SteamOS host install failed on-glass two ways: KWin denied Desktop-mode
capture (zkde_screencast_unstable_v1) so every session died in the pipeline build,
and the native Steam Deck pad degraded to an Xbox 360 controller. Both stem from
setup that only a fresh login applies — the host user-service keeps the
pre-install group set and KWin reads its .desktop grant at session start — plus a
missing vhci-hcd autoload the pad's USB/IP transport needs (the deb/rpm/arch
packages ship it; the Deck installer did not).

- install.sh: install punktfunk-modules.conf (+ modprobe vhci-hcd now) for the
  native Deck controller transport, seed the KDE RemoteDesktop grant
  (kde-authorized) for Desktop-mode input, and print a loud "reboot before
  streaming" notice when a relogin is actually required (input group added or the
  .desktop grant first installed).
- update.sh: retrofit the udev rule, vhci-hcd autoload, input group, and grant so
  existing installs pick them up on the next update without a reinstall.
- steamos-host.md: document the first-install reboot and the native controller
  passthrough requirements (input group + vhci-hcd + client-side Steam Input Off).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:22:08 +02:00
enricobuehler 081ff64087 docs: explain how to actually run the VirtualHere passthrough example
apple / swift (push) Successful in 1m22s
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m39s
apple / screenshots (push) Successful in 6m38s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
deb / build-publish (push) Successful in 9m0s
deb / build-publish-host (push) Successful in 9m20s
windows-host / package (push) Successful in 9m46s
android / android (push) Successful in 18m37s
arch / build-publish (push) Successful in 18m48s
docker / deploy-docs (push) Successful in 28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m44s
ci / rust (push) Successful in 26m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m2s
The VirtualHere DualSense recipe linked the `virtualhere-dualsense.ts` SDK
example as a deployable recipe but skipped everything needed to run it: that
VirtualHere is a server (couch) + client (host) pair, and that the example —
like every example — imports `../src/index.js`, which only resolves inside the
SDK repo. A user copying it out had no way to know they must
`bun add @punktfunk/host` and swap that import.

- automation.md: rewrite the recipe into a full walkthrough — the two-sided
  VirtualHere setup, the `-t` verbs, the zero-code hooks version, and a new
  scripted section with exact deploy steps, env vars, and running it as a
  service (its own SIGTERM handler makes `systemctl stop` release the pad).
- sdk/README.md: say how to run an example in-repo vs deployed on a host.
- virtualhere-dualsense.ts: header note on the import swap + service setup,
  since that file is where the doc link lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:28:23 +02:00
enricobuehler 675fd24cce chore(release): bump workspace version to 0.17.1
audit / cargo-audit (push) Successful in 3m22s
audit / bun-audit (push) Failing after 13s
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m12s
apple / screenshots (push) Successful in 6m55s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m55s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m39s
ci / rust (push) Successful in 24m5s
decky / build-publish (push) Successful in 30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
android-screenshots / screenshots (push) Successful in 3m34s
linux-client-screenshots / screenshots (push) Successful in 8m32s
release / apple (push) Successful in 11m16s
deb / build-publish-host (push) Successful in 12m51s
deb / build-publish (push) Successful in 12m57s
docker / deploy-docs (push) Successful in 27s
web-screenshots / screenshots (push) Successful in 3m13s
windows-host / package (push) Successful in 14m6s
android / android (push) Successful in 15m33s
arch / build-publish (push) Successful in 16m3s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m43s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m10s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 23m16s
flatpak / build-publish (push) Successful in 7m9s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:37:32 +02:00
enricobuehler 0e60e58cab perf(apple): windowed macOS presents — off-main transactional commit, surface prototype, safe-present setting
ci / docs-site (push) Successful in 52s
ci / web (push) Successful in 56s
decky / build-publish (push) Successful in 21s
apple / swift (push) Successful in 1m24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m53s
release / apple (push) Successful in 9m19s
deb / build-publish (push) Successful in 12m27s
arch / build-publish (push) Successful in 12m52s
docker / deploy-docs (push) Successful in 26s
deb / build-publish-host (push) Successful in 13m18s
apple / screenshots (push) Successful in 6m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m51s
android / android (push) Successful in 18m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m15s
ci / rust (push) Successful in 27m38s
Task 1 (latency): the transactional present now commits from the RENDER
thread inside an explicit CATransaction + flush() instead of hopping to
main. The present harness (2026-07-21, 240 Hz Studio, full-size window)
measured the main hop batching presents at runloop-iteration rate when
main is busy — an active implicit transaction NESTS the explicit one —
which is the field's presents=55 @ fps=240 / display_p50 18.6 ms.
Off-main commits measured immune to main churn: glass p50 ~10 ms,
cadence a clean 4.17 ms at 240. flush() is load-bearing: without it the
render thread's own implicit transaction (drawableSize/colour writes)
swallows the explicit commit and NOTHING reaches glass — the harness
reproduced the exact freeze (every present dropped). Legacy main-hop
kept as PUNKTFUNK_TXN_PRESENT=main for field A/B. New pf-windowed
Console line (subsystem io.unom.punktfunk, category presenter)
decomposes the issue side per second.

Task 1 lever D: the f407f418 IOSurface contents-swap path is
resurrected format-aware as WindowedPresentMode.surface — bgra8 SDR /
rgba16Float+PQ-tagged HDR pool, swap committed off-main from the
completion handler; EDR anchored by the metal layer underneath.
Env-only prototype (PUNKTFUNK_WINDOWED_PRESENT=surface) until the HDR
composite is eyeballed on glass.

Task 2 (setting): punktfunk.windowedSafePresent (default ON =
transactional mitigation; OFF = the fast async path whose panic the
saga is about — the caption says so in plain words). Rows in the touch
Settings (Presentation) and gamepad settings, macOS-only.
PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface overrides for dev
A/B. maximumDrawableCount stays 3 — the harness measured 2 starving the
render loop (172/s) and worsening glass p50.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:58:47 +02:00
enricobuehler 71faf38a85 fix(apple): stats os_log used %d for 64-bit Swift Int — macOS 26 drops the line
Swift `Int` is 64-bit (Foundation infers %lld); the stats mirror's format
string used %d (32-bit C int) for fps/presents/lost. macOS 26's strict
String(format:) validator rejects the mismatch and drops the whole line
(the error also cascades onto the float args, mis-blaming floor_p50). Use
%lld for the three Int fields. Pre-existing, unrelated to the DCP work;
surfaced while reading presenter logs during the panic fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:14:41 +02:00
enricobuehler f49d38826b fix(apple): windowed macOS DCP panic via presentsWithTransaction — keep real HDR, drop the SDR IOSurface path
The windowed-macOS "mismatched swapID's" @UnifiedPipeline.cpp kernel
panic is the CAMetalLayer ASYNC image queue (commandBuffer.present,
mandatory with displaySyncEnabled=false) diverging from WindowServer's
compositor on a high-refresh composited session — it survived glass
pacing (PyroWave, 2026-07-18) and hit windowed HEVC on the same 240 Hz
Mac Studio (2026-07-21), so it's the async queue itself, at any codec
or pacing.

f407f418's mitigation routed windowed PyroWave through a BGRA8 IOSurface
layer.contents pool, which cannot carry HDR — extending it to HEVC would
have tone-mapped windowed HDR down to SDR. Instead, present the drawable
through a Core Animation transaction (CAMetalLayer.presentsWithTransaction)
for every windowed session: commit, waitUntilScheduled, then drawable.present()
inside a CATransaction, so the swap commits with the layer tree and stays
in lockstep with the compositor instead of racing it. The full
rgba16Float / PQ / EDR render path is untouched — windowed HDR is real
HDR again. Fullscreen keeps the async path (direct scanout, lowest latency,
no panic there).

Removes the IOSurface surface pool, its surfaceLayer sibling, and the
macOS windowed PQ->SDR tone-map (tvOS keeps its no-headroom tone-map).
Net -150 lines. swift build + 169 tests green. Needs on-glass soak on
the 240 Hz Mac Studio (kernel race — only real hardware confirms).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:45:11 +02:00
enricobuehler 42198eb1b6 feat(linux/vulkan-encode): default-on native NV12 capture + RGB true-extent
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 25s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / bench (push) Successful in 5m59s
apple / screenshots (push) Successful in 6m17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m16s
deb / build-publish (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 13m40s
windows-host / package (push) Successful in 10m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m52s
android / android (push) Successful in 16m18s
arch / build-publish (push) Successful in 16m20s
docker / deploy-docs (push) Successful in 30s
ci / rust (push) Successful in 19m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m19s
Flip both zero-copy levers from opt-in to default:

- The capture negotiation now PREFERS gamescope's producer-side NV12 pod by
  default (PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation).
  Codec-aware gating rides a new OutputFormat::nv12_native ->
  ZeroCopyPolicy::native_nv12_session edge resolved by the host facade from
  the session plan (pf_encode::linux_native_nv12_ok): only H265/AV1 sessions
  whose backend can open the raw Vulkan Video encoder ever see the NV12 pod
  -- an H264/Moonlight session (libav VAAPI, which would misread the
  two-plane buffer) keeps today's BGRx negotiation, as do the
  GameStream-resolve and portal-mirror paths, PyroWave, and NVENC prefs.
- RGB-direct's unaligned modes default to the true-extent direct import
  (PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded-copy staging).
  Guarded-tested on Van Gogh with the kernel journal watched: clean, and at
  5.38 ms p50 the fastest 1080p encode path measured on that hardware. The
  EFC only exists on Mesa >= 26, where the codedExtent-driven session_init
  padding is guaranteed (verified back to Mesa 24.2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehler b4fbc94e36 feat(linux/vulkan-encode): PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT bring-up gate
Opt-in alternative to RGB-direct's padded-copy staging at unaligned modes:
direct-import the visible-size capture and declare the TRUE-SIZE source
codedExtent. RADV has derived the VCN session_init firmware padding from
srcPictureResource.codedExtent since Mesa 24.2, so the EFC is told the source
lacks the alignment rows and the hardware edge-extends them internally --
which also reframes the 2026-07-20 field GPU reset: that crash passed the
ALIGNED extent as the source codedExtent (firmware padding zero) over a
visible-size buffer. Off by default until a guarded live test proves the EFC
front-end honors the padding like the YUV fetch path does; the padded-copy
staging remains the shipped behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehler 004f0cacb8 perf(linux/vulkan-encode): true-size headers make native NV12 zero-copy at every mode
Drop the padded-copy staging for producer-native NV12 and direct-import the
visible-size buffer at unaligned modes (1080p) too. Safe by the driver's own
contract rather than by allocation luck: native sessions now author the H265
SPS / AV1 sequence header at the RENDER size and pass the matching codedExtent
on every picture resource. RADV rounds the bitstream SPS up itself (with a
conformance window -- radv_video_patch_encode_session_parameters, per the
VK_KHR_video_encode_h265 proposal's "implementations may override" clause) and
programs the VCN session_init with the true extent plus nonzero firmware
padding, so the hardware edge-extends the alignment rows internally and never
fetches past the source's real extent -- the exact mechanism VAAPI/radeonsi has
always used to encode 1080p. The driver-emitted header NALs already carry the
patched SPS, so the wire format is unchanged.

The CSC and RGB-direct paths keep the app-aligned convention (their sources
genuinely cover the aligned extent); RGB padded-copy staging is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehler 49c6dd00c9 fix(presenter): CPU frames never take the HDR10 passthrough surface
audit / bun-audit (push) Failing after 15s
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 58s
apple / swift (push) Successful in 1m22s
audit / cargo-audit (push) Successful in 2m20s
decky / build-publish (push) Successful in 32s
ci / bench (push) Successful in 7m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m18s
release / apple (push) Successful in 9m28s
deb / build-publish-host (push) Successful in 10m32s
deb / build-publish (push) Successful in 11m48s
docker / deploy-docs (push) Successful in 23s
android / android (push) Successful in 14m36s
arch / build-publish (push) Successful in 15m4s
windows-host / package (push) Successful in 16m11s
flatpak / build-publish (push) Failing after 8m19s
apple / screenshots (push) Successful in 6m40s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m11s
ci / rust (push) Successful in 19m38s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m39s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m36s
Software decode uploads swscale RGBA with no CSC/tonemap pass. On the new
desktop-compositor HDR10 surfaces (GNOME 48 / Plasma 6 + Mesa >= 25.1 offer
ST2084 even on SDR desktops) that sRGB-encoded content was composed as PQ —
the field-reported psychedelic picture (Fedora clients decode HEVC in
software because stock Mesa strips it; reproduced + fix live-validated on
GNOME Wayland: mode-0 soup -> HDR→SDR washed-out-but-correct).

The CPU lane keeps the SDR swapchain and its known untonemapped-PQ gap; a
real PQ→sRGB pass for CPU frames is the follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler e7c6c96e5f fix(presenter): set the Wayland app_id so the session window gets its icon
SDL_APP_ID = io.unom.Punktfunk — compositors match it against the shipped
.desktop for the window icon; without it KWin/mutter show the generic Wayland
icon (field-noticed on the Deck). Linux analog of win32 AppUserModelID.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler da134ad045 feat(presenter): PUNKTFUNK_HDR10=0 refuses the HDR10 swapchain (SDR tonemap pin)
GNOME 48 / Plasma 6 with Mesa >= 25.1 now offer HDR10_ST2084 surfaces even on
SDR desktops, silently flipping PQ streams into the mode-0 passthrough lane —
previously unreachable on desktop compositors and never validated there. The
hatch pins the shader tonemap for diagnosis and as the field escape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler 428bd2519c test(qsv): converter→ring-profile-P010→encoder live e2e (the RTV-written seam)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler 87d7eceabf fix(encode/qsv): write the colour VUI unconditionally + Intel-lane colour diagnostics
The native QSV encoder only attached mfxExtVideoSignalInfo on HDR sessions —
an SDR stream carried no colour description at all (NVENC writes 709-limited
unconditionally; qsv.rs now mirrors it).

Diagnostics for the field-reported blue/magenta Intel-host colours:
- hdr-p010-selftest takes an optional WxH + GPU vendor (intel|nvidia|amd) and
  prints which adapter it tested — 64x64 on the default GPU proved nothing on
  dual-GPU boxes, and the field heights (1080/1400) are not 16-aligned.
- pf-capture: ignored live test running the converter at 1920x1080 pinned to
  the Intel adapter.
- pf-encode: qsv_live_p010_1080_colorbars_dump — known P010 bars through the
  UNALIGNED-height ingest copy (1080 src -> align16 1088 pool, the seam no
  640x480 test exercises) to Main10 HEVC, dumped for off-box decode checks.
- dxgi.rs: drop the stale 'falls back to the R10 path' claim (no such path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler 1d4795666e feat(linux/vulkan-encode): opt-in gamescope producer-native NV12 encode source
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
decky / build-publish (push) Successful in 19s
apple / swift (push) Successful in 1m21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m53s
apple / screenshots (push) Successful in 6m16s
windows-host / package (push) Successful in 9m38s
deb / build-publish (push) Successful in 9m55s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m52s
android / android (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 14m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m0s
ci / rust (push) Successful in 28m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m41s
With PUNKTFUNK_PIPEWIRE_NV12=1 (bring-up gate), the PipeWire negotiation
offers an NV12 LINEAR DMA-BUF pod (BT.709 limited pinned MANDATORY) ahead
of the BGRx one, and gamescope's producer-side RGB->YUV pass replaces the
host CSC entirely: the encoder imports the two-plane buffer as a profiled
VIDEO_ENCODE_SRC image and the VCN encodes it directly. Contributed
measurements: encode p99 2.9 ms (from ~4-4.5 ms via EFC RGB-direct),
60 fps capture, 0 send drops.

Hardening on top of the contributed patch:
- unaligned modes (1080p!) stage through a padded aligned NV12 copy (edge
  rows/columns duplicated, transfer-only) instead of direct-importing the
  visible-size buffer -- a direct import would make the VCN read past the
  producer allocation, the exact OOB class behind the 2026-07-20 field GPU
  reset; the encode extents return to the aligned coded extent everywhere
- the UV plane layout honors the producer's plane-1 chunk (offset/stride)
  when the SPA buffer carries one (same-BO verified by inode), with the
  contiguous-plane contract as fallback
- PyroWave sessions are excluded from the gate (their Vulkan compute CSC
  ingests packed RGB), and a native-NV12 session that resolves to libav
  VAAPI (H264 codec, PUNKTFUNK_VULKAN_ENCODE=0, feature off, or a failed
  Vulkan open) refuses at open instead of streaming garbage chroma
- pad staging images carry TRANSFER_SRC (the width-padding pass self-copies
  the staging image -- previously missing on 1366-wide modes)
- metadata-cursor one-shot warn (parity with RGB-direct) and a padded-NV12
  PUNKTFUNK_PERF split label; the padded RGB copy gets its timestamps too

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 11:57:42 +02:00
enricobuehler 58b27acbd5 fix(vdisplay/session): scrub the dead desktop's WAYLAND_DISPLAY from the user-manager env
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m14s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 27s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m44s
apple / screenshots (push) Successful in 6m18s
android / android (push) Successful in 13m0s
docker / deploy-docs (push) Successful in 27s
deb / build-publish (push) Successful in 13m36s
deb / build-publish-host (push) Successful in 14m44s
windows-host / package (push) Successful in 16m32s
arch / build-publish (push) Successful in 17m49s
ci / rust (push) Successful in 19m58s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m42s
settle_desktop_portal pushes the live desktop's WAYLAND_DISPLAY into the
systemd --user manager (import-environment) so a re-activated portal inherits
the right session — but the import PERSISTS after that desktop dies. Every
later user unit then inherits the stale socket, including
gamescope-session.target: gamescope sees WAYLAND_DISPLAY, runs NESTED, and
aborts with "Failed to connect to wayland socket: wayland-0" — observed live
on a Deck, where Game Mode could not start at all (autologin crash-looped)
until the var was unset by hand.

Two-layer fix, mirroring the protection launch_session already gives its
transient unit:
- observe_session_instance: when a desktop compositor instance goes away,
  unset WAYLAND_DISPLAY/DISPLAY from the manager env (a desktop bounce is
  harmless — the next portal settle re-imports).
- write_steamos_dropin: UnsetEnvironment=DISPLAY WAYLAND_DISPLAY on the
  takeover drop-in, unit-scoped belt-and-suspenders for host-initiated
  restarts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:06:55 +02:00
enricobuehler 333c8979e9 fix(vdisplay/gamescope): capture-loss rebuild probes must not steal the box session
Observed live on a Deck host: the user switched Game Mode → Desktop, KDE came
up in 0.8 s — and the capture-loss rebuild's FIRST detection ran inside that
gap, still said Gaming, and its gamescope re-acquire restarted
gamescope-session.target. On SteamOS that steals the seat: the user was
yanked out of the KDE session they had just chosen, the two session managers
fought (job canceled / dependency failed), no gamescope node appeared within
30 s, and the stream died at the 40 s rebuild budget.

A rebuild probe acting on possibly-stale detection must never stop, relaunch,
or take over box sessions. New pf_vdisplay::rebuild_probe_scope (RAII,
counted): while held, the gamescope managed/takeover create paths only attach
to a live node and fail fast otherwise — no stop_autologin_sessions, no
session-plus relaunch, no gamescope-session.target restart. The capture-loss
loop holds it for the first 4 s after a loss (the detection-ambiguity
window); after that, destructive rebuilds are allowed again, so a genuine
switch INTO Game Mode still gets its headless takeover. Probe failures are
instant, so the loop now paces itself at ~2 Hz instead of spinning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:00:02 +02:00
enricobuehler 46d09ed973 fix(host/linux): stop burning retry budget on doomed pipeline builds
Two session-transition stalls found live on a SteamOS Deck host, one cause:
the pipeline retry loop couldn't tell a wait-it-out failure from a
retry-now-and-it-works one.

- Gamescope first-frame race: a PipeWire stream connected while gamescope
  re-initializes its headless takeover negotiates a format, reaches
  Streaming, and never receives a buffer — while a fresh connect delivers
  within ~0.5 s. Every gamescope bring-up ate the full 10 s first-frame
  budget on attempt 1 (17 s bring-ups; KWin: 1.2 s). The retry loop's FIRST
  attempt now waits 2.5 s (Capturer::next_frame_within), so the reconnect
  that fixes the race happens at ~3 s. Later attempts keep the patient 10 s
  — the documented 30-60 s Big Picture cold start still fits the budget.

- Capture-loss rebuild vs session switch: the rebuild loop re-detects the
  active session between build_pipeline_with_retry calls, but each call
  burned 8 attempts (~13 s) against a compositor that no longer exists (a
  Desktop→Gaming switch spent 13 of its 27 s retrying gone-KWin). The
  capture-loss path now passes max_attempts=2, turning the outer loop into
  ~1 s detect-and-retry cycles that follow the box to the new session.

The resize path's direct build_pipeline call keeps the default budget (no
retry wrapper there to absorb an early bail).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:34:49 +02:00
enricobuehler 946f1b3d69 fix(steamdeck): build the host with vulkan-encode, and wire up what it needs to actually engage
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m43s
apple / screenshots (push) Successful in 6m32s
deb / build-publish (push) Successful in 9m57s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m46s
android / android (push) Successful in 13m40s
deb / build-publish-host (push) Successful in 13m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m42s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m59s
ci / rust (push) Successful in 26m41s
The SteamOS scripts built the host featureless, so PlatformAuto could never
pick the Vulkan Video backend (default-on since 84329205) and every session
silently ran libav VAAPI — unlike the deb/arch/rpm/sysext builds, which all
pass the feature. Three gaps, found live on a Deck host:

- install.sh/update.sh: cargo build with --features punktfunk-host/vulkan-encode
  (matches the packaged builds; pure-Rust ash, no new system dep).
- host.env: RADV_PERFTEST=video_encode — Van Gogh RADV still gates
  VK_KHR_video_encode_* behind it, so without it the backend can't open and
  falls back to VAAPI. Harmless where encode is exposed by default.
- io.unom.Punktfunk.Host.desktop (Exec rewritten to this install's binary):
  KWin only grants zkde_screencast_unstable_v1/org_kde_kwin_fake_input to an
  exe a .desktop authorizes — without it Desktop-Mode capture fails outright
  and a mid-stream Game↔Desktop switch strands the stream on the old backend.

update.sh retrofits the last two idempotently so existing installs get them
without a reinstall.

Validated on a SteamOS 3.8.14 Deck (Van Gogh): Vulkan HEVC opened on both the
KWin and gamescope paths at 5120x1440 — in-place ABR reconfigure (no IDR, no
encoder rebuild) vs the VAAPI path's full teardown per ABR step, and Desktop
capture came up without a re-login.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:14:31 +02:00
63 changed files with 3220 additions and 765 deletions
+24 -30
View File
@@ -6,17 +6,12 @@
# #
# The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT # The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT
# need the Decky CLI (which requires Docker + rust-nightly only to compile native backends). # need the Decky CLI (which requires Docker + rust-nightly only to compile native backends).
# We build the frontend with pnpm and assemble the store-layout zip by hand: # We build the frontend with pnpm and stage the store-layout tree with the SAME script local
# # builds use (clients/decky/scripts/package.sh) — the plugin's file list lives in exactly ONE
# punktfunk.zip # place, so a file added there (bin/, assets/, controller_config/, …) can never be silently
# punktfunk/ <- single top-level dir == plugin.json "name" # missing from the published build. (Hand-assembling the zip here is how the shipped plugin
# plugin.json [required] # lost the shortcut artwork + Steam Input layout for a while.) CI only adds `update.json` on
# package.json [required; CI stamps "version" — Decky reads the installed version here] # top: the {channel, manifest} pointer the plugin's self-update check polls.
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
# #
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel # SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed # `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
@@ -90,28 +85,27 @@ jobs:
- name: Assemble store-layout zip - name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }} working-directory: ${{ gitea.workspace }}
run: | run: |
apt-get update && apt-get install -y --no-install-recommends zip >/dev/null # node:22-bookworm ships python3 (a package.sh dep) but not zip; install both anyway
STAGE="$RUNNER_TEMP/decky" # so an image change can't silently break the build.
DEST="$STAGE/$PLUGIN" apt-get update && apt-get install -y --no-install-recommends zip python3 >/dev/null
rm -rf "$STAGE"; mkdir -p "$DEST/dist" "$DEST/bin" # Stage the canonical plugin tree (dist/, main.py, bin/, assets/, controller_config/,
cp clients/decky/plugin.json "$DEST/" # LICENSE, …) with the same script local/sideload builds use — see the header comment.
cp clients/decky/package.json "$DEST/" # Runs AFTER the version stamp, so the staged package.json carries $VERSION.
cp clients/decky/main.py "$DEST/" bash clients/decky/scripts/package.sh
cp clients/decky/dist/index.js "$DEST/dist/" DEST="clients/decky/out/$PLUGIN"
cp clients/decky/README.md "$DEST/" # CI-only addition: the self-update channel pointer the backend reads (main.py
# The stream-launch wrapper (target of the Steam shortcut); keep it executable # check_update). It points at THIS channel's manifest.json (published below); that
# (runner_info() also re-chmods at runtime in case the zip/extract drops the bit). # manifest in turn points at the immutable per-version zip, so its sha256 stays valid
cp clients/decky/bin/punktfunkrun.sh "$DEST/bin/" # across future alias re-uploads.
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json" printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" ) ( cd clients/decky/out && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip" ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip" unzip -l "$RUNNER_TEMP/punktfunk.zip"
# Backstop against packaging drift: the runtime-loaded pieces MUST be in the zip.
for f in main.py dist/index.js bin/punktfunkrun.sh assets/grid.png \
controller_config/punktfunk.vdf update.json; do
unzip -l "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN/$f" >/dev/null || { echo "MISSING $f" >&2; exit 1; }
done
# The update manifest the plugin polls: the immutable per-version artifact + its # The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on # sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias). # mismatch — so it MUST be the per-version URL, never the mutable alias).
Generated
+65 -27
View File
@@ -656,6 +656,30 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]] [[package]]
name = "ciborium" name = "ciborium"
version = "0.2.2" version = "0.2.2"
@@ -691,6 +715,7 @@ checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [ dependencies = [
"crypto-common", "crypto-common",
"inout", "inout",
"zeroize",
] ]
[[package]] [[package]]
@@ -2159,7 +2184,7 @@ dependencies = [
[[package]] [[package]]
name = "latency-probe" name = "latency-probe"
version = "0.17.0" version = "0.17.2"
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
@@ -2264,7 +2289,7 @@ dependencies = [
[[package]] [[package]]
name = "libvpl-sys" name = "libvpl-sys"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -2299,7 +2324,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]] [[package]]
name = "loss-harness" name = "loss-harness"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"punktfunk-core", "punktfunk-core",
] ]
@@ -2788,7 +2813,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]] [[package]]
name = "pf-capture" name = "pf-capture"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2808,7 +2833,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-client-core" name = "pf-client-core"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2832,7 +2857,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-clipboard" name = "pf-clipboard"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2850,7 +2875,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-console-ui" name = "pf-console-ui"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2871,7 +2896,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-encode" name = "pf-encode"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2881,6 +2906,7 @@ dependencies = [
"libvpl-sys", "libvpl-sys",
"nvidia-video-codec-sdk", "nvidia-video-codec-sdk",
"openh264", "openh264",
"pf-capture",
"pf-frame", "pf-frame",
"pf-gpu", "pf-gpu",
"pf-host-config", "pf-host-config",
@@ -2894,7 +2920,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-ffvk" name = "pf-ffvk"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"ash", "ash",
"bindgen", "bindgen",
@@ -2903,7 +2929,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-frame" name = "pf-frame"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"libc", "libc",
@@ -2915,7 +2941,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-gpu" name = "pf-gpu"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-host-config", "pf-host-config",
@@ -2929,11 +2955,11 @@ dependencies = [
[[package]] [[package]]
name = "pf-host-config" name = "pf-host-config"
version = "0.17.0" version = "0.17.2"
[[package]] [[package]]
name = "pf-inject" name = "pf-inject"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2961,14 +2987,14 @@ dependencies = [
[[package]] [[package]]
name = "pf-paths" name = "pf-paths"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "pf-presenter" name = "pf-presenter"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2983,7 +3009,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-vdisplay" name = "pf-vdisplay"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -3013,7 +3039,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-win-display" name = "pf-win-display"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-paths", "pf-paths",
@@ -3025,7 +3051,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-zerocopy" name = "pf-zerocopy"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3136,6 +3162,17 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]] [[package]]
name = "polyval" name = "polyval"
version = "0.6.2" version = "0.6.2"
@@ -3221,7 +3258,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-android" name = "punktfunk-client-android"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"jni", "jni",
@@ -3237,7 +3274,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-linux" name = "punktfunk-client-linux"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -3253,7 +3290,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-session" name = "punktfunk-client-session"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-client-core", "pf-client-core",
@@ -3268,7 +3305,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-windows" name = "punktfunk-client-windows"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"async-channel", "async-channel",
"ffmpeg-next", "ffmpeg-next",
@@ -3287,11 +3324,12 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-core" name = "punktfunk-core"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"bytes", "bytes",
"cbindgen", "cbindgen",
"chacha20poly1305",
"criterion", "criterion",
"fec-rs", "fec-rs",
"hmac", "hmac",
@@ -3318,7 +3356,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-host" name = "punktfunk-host"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
@@ -3402,7 +3440,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-probe" name = "punktfunk-probe"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"mdns-sd", "mdns-sd",
@@ -3416,7 +3454,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-tray" name = "punktfunk-tray"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ksni", "ksni",
@@ -3439,7 +3477,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]] [[package]]
name = "pyrowave-sys" name = "pyrowave-sys"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" } ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package] [workspace.package]
version = "0.17.0" version = "0.17.2"
edition = "2021" edition = "2021"
rust-version = "1.82" rust-version = "1.82"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0", "name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0" "identifier": "MIT OR Apache-2.0"
}, },
"version": "0.17.0" "version": "0.17.2"
}, },
"paths": { "paths": {
"/api/v1/clients": { "/api/v1/clients": {
@@ -705,8 +705,11 @@ final class SessionModel: ObservableObject {
// captured before the 2026-07 floor policy); the appended trio carries the // captured before the 2026-07 floor policy); the appended trio carries the
// measured OS present floor and the floor-shaved values the HUD displays. // measured OS present floor and the floor-shaved values the HUD displays.
let line = String( let line = String(
format: "fps=%d presents=%d e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f " // Swift Int is 64-bit %lld, NOT %d (which is a 32-bit C int); macOS 26's
+ "decode_p50=%.1f display_p50=%.1f lost=%d " // strict String(format:) validator rejects the %d/Int mismatch and drops
// the whole line (a cascade error that also mis-blames the float args).
format: "fps=%lld presents=%lld e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%lld "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f", + "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f",
frames, frames,
displayWindow?.count ?? 0, displayWindow?.count ?? 0,
@@ -43,6 +43,9 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.presentPriority) private var presentPriority = @AppStorage(DefaultsKey.presentPriority) private var presentPriority =
SettingsOptions.presentPriorityDefault SettingsOptions.presentPriorityDefault
@AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.windowedSafePresent) private var windowedSafePresent = true
#endif
#if os(iOS) #if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false @AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
#endif #endif
@@ -345,6 +348,22 @@ struct GamepadSettingsView: View {
detail: "Turn off to use the touch interface even with a controller connected.", detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled), value: $gamepadUIEnabled),
] ]
#if os(macOS)
// The windowed safe-present toggle slots in after "Smoothness buffer" (staying inside
// the Video group) macOS only, mirroring the touch SettingsView's Presentation row
// (the DCP swapID-panic mitigation; see DefaultsKey.windowedSafePresent).
if let at = list.firstIndex(where: { $0.id == "smoothBuffer" }) {
list.insert(
toggleRow(
id: "windowedSafePresent", icon: "macwindow.badge.plus",
label: "Safe windowed presentation",
detail: "Windowed streams present in step with the compositor — avoids a "
+ "macOS display-driver crash on high-refresh displays, at a small "
+ "latency cost. Fullscreen always uses the fastest path.",
value: $windowedSafePresent),
at: at + 1)
}
#endif
#if os(iOS) #if os(iOS)
// The device-rumble mirror slots in after "Controller type" (staying inside the // The device-rumble mirror slots in after "Controller type" (staying inside the
// Controller group the next row carries the "Interface" header). iPhone only in // Controller group the next row carries the "Interface" header). iPhone only in
@@ -300,6 +300,18 @@ extension SettingsView {
+ "of added latency. Off shows frames as soon as they're ready.") { + "of added latency. Off shows frames as soon as they're ready.") {
Toggle("V-Sync", isOn: $vsync) Toggle("V-Sync", isOn: $vsync)
} }
// The DCP swapID-panic mitigation's user handle (see DefaultsKey.windowedSafePresent
// for the saga). Default ON: turning it off re-arms a WHOLE-MACHINE kernel panic on
// affected setups, so the caption says so in plain words.
described(windowedSafePresent
? "Windowed streams present in step with the system compositor — avoids a macOS "
+ "display-driver crash seen on high-refresh displays, at a small latency "
+ "cost. Fullscreen always uses the fastest path."
: "Windowed streams use the fastest present path. On some high-refresh setups "
+ "this can crash macOS itself (kernel panic) — turn back on if your Mac "
+ "restarts during windowed streaming.") {
Toggle("Safe windowed presentation", isOn: $windowedSafePresent)
}
#endif #endif
} }
} }
@@ -40,6 +40,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false @AppStorage(DefaultsKey.vsync) var vsync = false
@AppStorage(DefaultsKey.windowedSafePresent) var windowedSafePresent = true
#endif #endif
#if !os(tvOS) #if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true @AppStorage(DefaultsKey.allowVRR) var allowVRR = true
@@ -23,6 +23,34 @@ import os
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter") private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
#if os(macOS)
/// HOW a windowed (composited) macOS session pushes finished frames to glass the DCP
/// "mismatched swapID's" kernel-panic saga's mechanism picker. Fullscreen always presents
/// `async` (direct-scanout promotion, lowest latency, no panic reports there); the windowed
/// mechanism is resolved per session by SessionPresenter (user setting +
/// PUNKTFUNK_WINDOWED_PRESENT env override) and routed here via `setWindowedPresent`.
///
/// - `async`: the CAMetalLayer image queue (`commandBuffer.present`) the fastest composited
/// path and the PANIC TRIGGER on high-refresh displays (the out-of-band swaps race
/// WindowServer's compositor; it survived glass pacing and every codec).
/// - `transaction`: `CAMetalLayer.presentsWithTransaction` the swap commits WITH the layer
/// tree, in lockstep with the compositor (Apple's documented remedy; validated no-panic on
/// the 240 Hz repro machine). The present is committed from the RENDER thread inside an
/// explicit CATransaction + flush see `encodePresent` for why that beats the original
/// main-thread hop.
/// - `surface`: no image queue at all render into a pooled IOSurface and swap it into a plain
/// CALayer's `contents` (the f407f418 PyroWave mitigation, resurrected format-aware:
/// rgba16Float + PQ tagging keeps HDR). WindowServer treats it as ordinary layer damage on
/// its own composite cadence. PROTOTYPE: whether the compositor honors PQ/EDR for plain-layer
/// IOSurface contents still needs an on-glass eyeball the metal layer stays underneath with
/// `wantsExtendedDynamicRangeContent` as the EDR anchor.
enum WindowedPresentMode: String, Sendable {
case async
case transaction
case surface
}
#endif
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the /// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors /// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the /// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
@@ -198,8 +226,8 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.) // in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
// The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084 // The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff // EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map the // BT.709 OETF. Used by the tvOS biplanar tone-map and the tvOS planar (PyroWave) tone-map (the
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only. // no-HDR-headroom fallback). macOS keeps real HDR windowed now see `WindowedPresentMode`.
static inline float3 pqToSdr(float3 pq) { static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0; const float m1 = 2610.0/16384.0;
const float m2 = 78.84375; const float m2 = 78.84375;
@@ -230,8 +258,7 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows // PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave // fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions // stream must land on an 8-bit surface: tvOS without HDR headroom. The passthrough planar
// (the IOSurface present path the DCP-panic mitigation is BGRA8). The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the // HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation). // layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]], fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
@@ -259,21 +286,51 @@ public final class MetalVideoPresenter {
public let layer: CAMetalLayer public let layer: CAMetalLayer
#if os(macOS) #if os(macOS)
/// The WINDOWED-mode PyroWave present target: a plain CALayer sized like `layer` (installed /// WINDOWED-mode present coordination the macOS DCP KERNEL PANIC mitigation.
/// as a sibling ABOVE it), fed IOSurfaces via `contents` inside ordinary CATransactions.
/// ///
/// Why this exists the macOS DCP KERNEL PANIC ("mismatched swapID's" @UnifiedPipeline.cpp, /// The panic ("mismatched swapID's" @UnifiedPipeline.cpp, WindowServer dies, machine reboots):
/// WindowServer dies, machine reboots): out-of-band CAMetalLayer image-queue swaps into a /// the CAMetalLayer's ASYNCHRONOUS image queue (`commandBuffer.present(drawable)` an
/// COMPOSITED (windowed) session race WindowServer's own swap submissions on high-refresh /// out-of-band flip, mandatory with `displaySyncEnabled=false`) diverges from WindowServer's
/// displays, and the race survives glass pacing a fully serialized one-in-flight present /// compositor on a high-refresh COMPOSITED (windowed) session the compositor's notion of the
/// stream still panicked a 240 Hz Mac Studio (2026-07-18, twice). So in windowed mode we stop /// current swap and the layer's queued swap disagree, and the DCP asserts. It survived glass
/// using the image queue entirely and present the way video players do: render the planar CSC /// pacing: a fully serialized one-in-flight present stream still panicked a 240 Hz Mac Studio
/// into an IOSurface pool and swap `contents` on main WindowServer treats it as ordinary /// (2026-07-18, PyroWave), and a windowed HEVC session panicked the same machine 2026-07-21
/// damage on its own composite cadence, coalescing faster-than-refresh updates instead of /// so it is the async image queue itself, at any pacing or codec, not a present rate.
/// latching queue swaps mid-cycle. Fullscreen keeps the CAMetalLayer path (direct-scanout ///
/// promotion, no compositing, no panic reports). Contents updates are transparent to the /// The fix keeps the full render path (rgba16Float / PQ / EDR real HDR is preserved) and
/// layer below when nil, so flipping modes just covers/uncovers the metal layer. /// only changes HOW the drawable is presented: `CAMetalLayer.presentsWithTransaction`. With it
public let surfaceLayer: CALayer = { /// set, we don't hand the drawable to the command buffer; we commit, wait until scheduled, then
/// call `drawable.present()` INSIDE a CATransaction the present is enrolled in Core
/// Animation's transaction and committed together with the layer tree, so the swap stays in
/// lockstep with the compositor instead of racing it (Apple's documented remedy for Metal
/// presentation drifting out of sync with CA). Fullscreen keeps the async path (direct-scanout
/// promotion, lowest latency, no compositor and no panic reports there).
///
/// 2026-07-21 latency rework: the mitigation MECHANISM is now a three-way pick
/// (`WindowedPresentMode`) and the transactional present commits from the RENDER thread
/// see `encodePresent`. Staged under `stagingLock` (main pushes it via
/// `setComposited``setWindowedPresent`); the render thread drains it and toggles the layer
/// property + present style. `Active` is the render-thread copy so the layer property flips
/// exactly once per mode change.
private var windowedPresentStaged: WindowedPresentMode = .async
private var windowedPresentActive: WindowedPresentMode = .async
/// PUNKTFUNK_TXN_PRESENT=main the ORIGINAL transactional present (commit
/// waitUntilScheduled hop to the MAIN thread and present inside its CATransaction), kept
/// as a field A/B lever. The default is the render-thread commit: the present harness
/// (2026-07-21, this saga) measured the main hop landing a runloop turn late on a busy main
/// thread, and an ACTIVE implicit transaction there NESTS the explicit one presents batch
/// at runloop-iteration rate (the field's presents=55 @ fps=240, display_p50 18.6 ms).
/// Off-main commits measured immune to main-thread churn (~10 ms glass p50 at 240 Hz
/// full-size vs 14+ ms under a churned main hop).
private let txnPresentOnMain =
ProcessInfo.processInfo.environment["PUNKTFUNK_TXN_PRESENT"] == "main"
/// The WINDOWED-mode `surface` present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it by SessionPresenter), fed IOSurfaces via `contents` inside explicit
/// CATransactions. Transparent (nil contents) whenever surface mode is off, so the metal
/// layer below shows through. See `WindowedPresentMode.surface`.
let surfaceLayer: CALayer = {
let l = CALayer() let l = CALayer()
l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout
l.isOpaque = true l.isOpaque = true
@@ -281,8 +338,8 @@ public final class MetalVideoPresenter {
return l return l
}() }()
/// One IOSurface-backed render target of the windowed present pool. All pool state is /// One IOSurface-backed render target of the windowed surface-present pool. All pool state
/// RENDER-THREAD confined; only the immutable surface refs cross to main (contents swap). /// is RENDER-THREAD confined; only the immutable surface refs cross threads (contents swap).
private struct SurfaceSlot { private struct SurfaceSlot {
let surface: IOSurfaceRef let surface: IOSurfaceRef
let texture: MTLTexture let texture: MTLTexture
@@ -292,15 +349,52 @@ public final class MetalVideoPresenter {
private var surfacePool: [SurfaceSlot] = [] private var surfacePool: [SurfaceSlot] = []
private var surfacePoolSize: CGSize = .zero private var surfacePoolSize: CGSize = .zero
private var surfacePoolHDR = false
private var surfaceSeq: UInt64 = 0 private var surfaceSeq: UInt64 = 0
/// Index of the slot most recently handed to the layer never rewritten next, even if its /// Index of the slot most recently handed to the layer never rewritten next, even if its
/// use count already dropped (the compositor may still be scanning out the previous frame). /// use count already dropped (the compositor may still be scanning out the previous frame).
private var lastHandedOff: Int? private var lastHandedOff: Int?
/// Staged (under `stagingLock`, like every cross-thread input): the hosting view's windowed
/// vs fullscreen state, pushed from main via `setSurfacePresents`. Drained in `renderPlanar`. /// Once-per-second decomposition of the ACTIVE windowed present path (the field-diagnosis
private var surfacePresentsStaged = false /// half of the DCP-latency work): scheduled/completed wait + commit/flush cost per present,
/// Render-thread copy, so pool teardown happens exactly once on a mode flip. /// and how many presents/swaps were issued. The pf-present line shows the GLASS side
private var surfacePresentsActive = false /// (latchMs / dropped); this shows the ISSUE side. Logged via `presenterLog` only while a
/// windowed mechanism is active (zero cost fullscreen). Lock-guarded: transaction mode
/// records from the render thread, surface mode from Metal completion threads.
private final class WindowedPresentDiag: @unchecked Sendable {
private let lock = NSLock()
private var presents = 0
private var schedMs: [Double] = []
private var commitMs: [Double] = []
private var last = CACurrentMediaTime()
func record(schedMs sched: Double, commitMs commit: Double, mode: WindowedPresentMode) {
lock.lock()
presents += 1
schedMs.append(sched)
commitMs.append(commit)
let now = CACurrentMediaTime()
guard now - last >= 1 else {
lock.unlock()
return
}
last = now
let sSched = schedMs.sorted()
let sCommit = commitMs.sorted()
let line = String(
format: "pf-windowed mode=%@ presents=%d schedMs p50=%.2f max=%.2f "
+ "commitMs p50=%.2f max=%.2f",
mode.rawValue, presents, sSched[sSched.count / 2], sSched.last ?? 0,
sCommit[sCommit.count / 2], sCommit.last ?? 0)
presents = 0
schedMs.removeAll(keepingCapacity: true)
commitMs.removeAll(keepingCapacity: true)
lock.unlock()
presenterLog.info("\(line, privacy: .public)")
}
}
private let windowedDiag = WindowedPresentDiag()
#endif #endif
private let device: MTLDevice private let device: MTLDevice
@@ -316,7 +410,7 @@ public final class MetalVideoPresenter {
private let pipelinePlanar: MTLRenderPipelineState private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour /// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm /// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents). /// bgra8; tvOS without HDR headroom).
private let pipelinePlanarHDR: MTLRenderPipelineState private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache? private var textureCache: CVMetalTextureCache?
@@ -591,13 +685,14 @@ public final class MetalVideoPresenter {
} }
#if os(macOS) #if os(macOS)
/// Park the windowed-vs-fullscreen present routing (MAIN thread the hosting view pushes its /// Park the windowed present mechanism (MAIN thread the hosting view pushes its window
/// window state on every layout). true = PyroWave frames present via `surfaceLayer` contents /// state on every layout; SessionPresenter resolves the mechanism per session). `.async` =
/// (the DCP swapID-panic mitigation see `surfaceLayer`); false = the CAMetalLayer path. /// FULLSCREEN (or the user opted out of the mitigation): the image queue. `.transaction` /
/// `.surface` = COMPOSITED (windowed) mitigation mechanisms see `WindowedPresentMode`.
/// Applied by the render thread on the next frame, like every other staged value here. /// Applied by the render thread on the next frame, like every other staged value here.
public func setSurfacePresents(_ on: Bool) { func setWindowedPresent(_ mode: WindowedPresentMode) {
stagingLock.lock() stagingLock.lock()
surfacePresentsStaged = on windowedPresentStaged = mode
stagingLock.unlock() stagingLock.unlock()
} }
#endif #endif
@@ -734,36 +829,12 @@ public final class MetalVideoPresenter {
) -> Bool { ) -> Bool {
stagingLock.lock() stagingLock.lock()
let targetFromLayout = drawableTarget let targetFromLayout = drawableTarget
#if os(macOS)
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock() stagingLock.unlock()
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path; // A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader). // including macOS windowed sessions, which keep real HDR (the DCP mitigation is the
#if os(macOS) // transactional present in `encodePresent`, not a colour downgrade).
configure(hdr: planes.pq && !surfaceMode)
#else
configure(hdr: planes.pq) configure(hdr: planes.pq)
#endif
var csc = planes.csc var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
surfacePresentsActive = surfaceMode
presenterLog.info(
"stage2: windowed surface presents \(surfaceMode ? "ON" : "OFF", privacy: .public) (PyroWave DCP-panic mitigation)")
if !surfaceMode {
// Back to the metal path (fullscreen): drop the pool at 5K it holds >100 MB,
// and re-entering windowed mode rebuilds it in one frame.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
}
if surfaceMode {
return renderPlanarToSurface(
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still) // PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps // 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format). // in-shader instead (the pipeline must match the drawable's pixel format).
@@ -792,118 +863,6 @@ public final class MetalVideoPresenter {
} }
} }
#if os(macOS)
/// The windowed-mode present tail (see `surfaceLayer` for why this path exists): render the
/// planar CSC into a pooled IOSurface and hand it to `surfaceLayer.contents` on MAIN inside a
/// plain CATransaction an ordinary damaged-layer update on WindowServer's own composite
/// cadence, no CAMetalLayer image-queue swap anywhere. `presentAtMediaTime` doesn't apply
/// (the compositor paces); `onPresented` fires after the contents swap is committed, stamped
/// with CLOCK_REALTIME then the closest observable analogue of "reached glass" here (the
/// composite follows within a refresh, so the meters' display stage reads slightly optimistic).
private func renderPlanarToSurface(
_ planes: WaveletPlanes, targetFromLayout: CGSize, csc: inout CscUniform,
onPresented: ((Int64?) -> Void)?
) -> Bool {
let decodedSize = CGSize(width: planes.width, height: planes.height)
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
ensureSurfacePool(size: targetSize)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let keepAlive: [Any] = [planes.y, planes.cb, planes.cr]
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // ring textures pinned until the GPU finished sampling
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
onPresented?(
Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size` 4 BGRA8 IOSurface render targets (one on glass, one queued
/// in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the pool empty;
/// the caller returns false and the ring's putBack + display-link retry take over.
private func ensureSurfacePool(size: CGSize) {
guard size != surfacePoolSize else { return }
surfacePool.removeAll()
surfacePoolSize = size
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// 256-byte row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerRow = ((w * 4) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: 4,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one /// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule /// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback. /// the present and the on-glass callback.
@@ -936,6 +895,36 @@ public final class MetalVideoPresenter {
#if DEBUG #if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize) logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
#endif #endif
#if os(macOS)
// Windowed (composited) the DCP swapID-panic mitigation mechanism (see
// `WindowedPresentMode`). Toggle the layer property BEFORE vending a drawable so the
// vend matches how it will be presented; drained here on the render thread, flipped
// exactly once per mode change.
stagingLock.lock()
let windowedMode = windowedPresentStaged
stagingLock.unlock()
if windowedMode != windowedPresentActive {
windowedPresentActive = windowedMode
layer.presentsWithTransaction = windowedMode == .transaction
if windowedMode != .surface, !surfacePool.isEmpty {
// Leaving surface mode (fullscreen entry / mechanism A/B): drop the pool at 5K
// it holds >100 MB, and re-entering rebuilds it in one frame. SessionPresenter
// clears the surface layer's contents on main.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
presenterLog.info(
"stage2: windowed present mode \(windowedMode.rawValue, privacy: .public) (DCP swapID-panic mitigation)")
}
if windowedMode == .surface {
// No image queue at all: render into a pooled IOSurface and swap it into the
// sibling layer's contents. The drawable/queue tail below never runs.
return encodeToSurface(
targetSize: targetSize, pipeline: pipeline, onPresented: onPresented,
keepAlive: keepAlive, bind: bind)
}
#endif
if let providedDrawable, if let providedDrawable,
providedDrawable.texture.pixelFormat != layer.pixelFormat { providedDrawable.texture.pixelFormat != layer.pixelFormat {
return false // config outran the vend (HDR flip) next vend has the new format return false // config outran the vend (HDR flip) next vend has the new format
@@ -974,6 +963,61 @@ public final class MetalVideoPresenter {
} }
#endif #endif
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
#if os(macOS)
if windowedPresentActive == .transaction {
// Windowed DCP mitigation: present the drawable THROUGH a Core Animation transaction
// (`presentsWithTransaction`, set above) instead of the async image queue, so the swap
// commits with the layer tree and stays in lockstep with the compositor (no out-of-band
// flip to race WindowServer's swaps). Wait until the GPU work is scheduled (contents
// will be ready p50 ~0.1 ms), then present inside an EXPLICIT CATransaction ON THIS
// RENDER THREAD and `flush()`. `presentAtMediaTime` does not apply the transaction
// paces.
//
// Threading history, because BOTH failure modes shipped or nearly shipped:
// A bare `present()` from this thread (no transaction) never flushes nothing
// commits a runloop-less thread's implicit transaction, so drawables are never
// released; after maximumDrawableCount vends `nextDrawable()` blocks forever and
// the stream FREEZES (the fullscreenwindowed switch did exactly this).
// The explicit begin/commit alone is NOT enough either: this thread has an ACTIVE
// implicit transaction (the layer mutations above drawableSize/colour created
// it), so the explicit transaction NESTS inside it and its commit defers to the
// implicit one that never comes. The harness reproduced the exact freeze: every
// present reported presentedTime=0, nothing reached glass. `CATransaction.flush()`
// pushes the implicit transaction (present included) to the render server NOW.
// The original fix hopped to MAIN and presented there correct, but slow in the
// field (presents=55 @ fps=240, display_p50 18.6 ms on the 240 Hz Studio): each
// present lands a runloop turn late, and main's own implicit transaction batches
// enrolled presents at runloop-iteration rate. Kept as PUNKTFUNK_TXN_PRESENT=main.
// The off-main commit measured immune to main-thread churn in the harness
// (2026-07-21: glass p50 ~10 ms at 240 Hz full-size, cadence a clean 4.17 ms).
commandBuffer.commit()
let schedStart = CACurrentMediaTime()
commandBuffer.waitUntilScheduled()
let schedMs = (CACurrentMediaTime() - schedStart) * 1000
let commitStart = CACurrentMediaTime()
if txnPresentOnMain {
let presentedDrawable = drawable
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
presentedDrawable.present()
CATransaction.commit()
}
} else {
CATransaction.begin()
CATransaction.setDisableActions(true)
drawable.present()
CATransaction.commit()
CATransaction.flush()
}
windowedDiag.record(
schedMs: schedMs, commitMs: (CACurrentMediaTime() - commitStart) * 1000,
mode: .transaction)
return true
}
#endif
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment); // Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing. // immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime { if let presentAtMediaTime {
@@ -981,12 +1025,146 @@ public final class MetalVideoPresenter {
} else { } else {
commandBuffer.present(drawable) commandBuffer.present(drawable)
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
commandBuffer.commit() commandBuffer.commit()
return true return true
} }
#if os(macOS)
/// The WINDOWED `surface` present tail (see `WindowedPresentMode.surface`): render with the
/// same per-frame pipeline into a pooled IOSurface and hand it to `surfaceLayer.contents`
/// from the command buffer's COMPLETION handler, inside an explicit CATransaction + flush
/// (the same off-main commit discipline as the transactional present an ordinary
/// damaged-layer update on WindowServer's own composite cadence, no image queue anywhere).
/// RENDER THREAD. `onPresented` is stamped right after the contents swap commits the
/// closest observable analogue of "reached glass" here (the composite follows within a
/// refresh, so the display-stage meters read slightly OPTIMISTIC in this mode).
///
/// The pool tracks `hdrActive`: bgra8 for SDR, rgba16Float tagged BT.2100 PQ for HDR
/// `configure` already ran, so the caller's `pipeline` attachment format always matches.
/// HDR OPEN RISK (why this whole mode is a prototype): whether the compositor honors the
/// PQ tag + EDR for plain-CALayer IOSurface contents needs an on-glass eyeball; the metal
/// layer underneath keeps `wantsExtendedDynamicRangeContent` as the EDR anchor (the harness
/// measured the display's EDR headroom engaging with this arrangement).
private func encodeToSurface(
targetSize: CGSize, pipeline: MTLRenderPipelineState,
onPresented: ((Int64?) -> Void)?,
keepAlive: [Any], bind: (MTLRenderCommandEncoder) -> Void
) -> Bool {
ensureSurfacePool(size: targetSize, hdr: hdrActive)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipeline)
bind(encoder)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let diag = windowedDiag
let commitStamp = CACurrentMediaTime()
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // sources pinned until the GPU finished sampling
let completedAt = CACurrentMediaTime()
// Swap on THIS Metal completion thread: explicit transaction + flush, so the commit
// reaches the render server now, independent of main (completion handlers for one
// queue fire in execution order, so swaps can't reorder).
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
CATransaction.flush()
diag.record(
schedMs: (completedAt - commitStamp) * 1000,
commitMs: (CACurrentMediaTime() - completedAt) * 1000, mode: .surface)
onPresented?(Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size`/`hdr` 4 IOSurface render targets (one on glass, one
/// committed in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the
/// pool empty; the caller returns false and the ring's putBack + display-link retry take
/// over.
private func ensureSurfacePool(size: CGSize, hdr: Bool) {
guard size != surfacePoolSize || hdr != surfacePoolHDR else { return }
surfacePool.removeAll()
surfacePoolSize = size
surfacePoolHDR = hdr
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// rgba16Float (8 B/px) carries the PQ-encoded HDR samples; bgra8 the SDR ones. 256-byte
// row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerElement = hdr ? 8 : 4
let bytesPerRow = ((w * bytesPerElement) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: bytesPerElement,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: hdr
? kCVPixelFormatType_64RGBAHalf : kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: hdr ? .rgba16Float : .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
if hdr, let name = CGColorSpace(name: CGColorSpace.itur_2100_PQ)?.name {
// Tag the surface BT.2100 PQ so the compositor interprets the half-float
// samples as PQ-encoded HDR (the CALayer-contents analogue of the metal
// layer's colorspace).
IOSurfaceSetValue(surface, "IOSurfaceColorSpace" as CFString, name)
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
// The EDR request rides the SURFACE layer too (its contents are what composite); the
// metal layer underneath keeps its own from configureColor as the anchor. Layer flags
// are committed by the next swap's transaction flush.
surfaceLayer.wantsExtendedDynamicRangeContent = hdr
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the /// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained. /// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture( private func makeTexture(
@@ -142,20 +142,17 @@ enum PresentPriority: Equatable {
final class SessionPresenter { final class SessionPresenter {
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2 /// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
/// default, macOS PyroWave sessions ALSO get glass gating a kernel-panic mitigation, not a /// default, macOS PyroWave sessions ALSO get glass gating for SMOOTHNESS, not as the panic
/// latency tweak. macOS's DCP panics ("mismatched swapID's" @UnifiedPipeline.cpp, the whole /// fix (that is the windowed transactional present see `setComposited`). PyroWave's wavelet
/// machine dies) when WindowServer's swap submissions race, and the reliable trigger is /// decode is near-instant Metal compute, so a network clump presents within the same
/// out-of-band CAMetalLayer presents (displaySyncEnabled=false mandatory for us, see /// millisecond, and it is the codec that sustains stream rates above the panel's refresh; the
/// MetalVideoPresenter's init) arriving faster than the compositor latches them in a /// glass gate admits one presented-but-undisplayed swap at a time (serialized on the on-glass
/// COMPOSITED (windowed) session. Arrival pacing does exactly that with PyroWave: the wavelet /// callback, 100 ms stale backstop) so those bursts coalesce in the newest-wins ring instead
/// decode is near-instant Metal compute, so a network clump of frames presents within the /// of flooding the queue. (Glass pacing was ALSO the original DCP-panic mitigation attempt
/// same millisecond, and PyroWave is the codec that sustains stream rates above the panel's /// disproven: a fully serialized stream still panicked, which is why the real fix moved to the
/// refresh. The glass gate admits one presented-but-undisplayed swap at a time (serialized on /// present mechanism.) An explicit stage-2 pick (setting/env) still forces arrival pacing
/// the on-glass callback, 100 ms stale backstop), which removes the racing pattern outright; /// that A/B lever must stay honest. VideoToolbox codecs keep arrival pacing: decode latency
/// frames the panel couldn't have shown anyway coalesce in the newest-wins ring. An explicit /// spaces their presents.
/// stage-2 pick (setting/env) still forces arrival pacing that A/B lever must stay honest.
/// VideoToolbox codecs keep arrival pacing: decode latency spaces their presents, and years
/// of stage-2 defaults there predate any panic report.
static func pacing( static func pacing(
for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec
) -> PresentPacing { ) -> PresentPacing {
@@ -178,10 +175,25 @@ final class SessionPresenter {
/// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing /// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing
/// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate. /// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate.
/// ///
#if os(macOS)
/// Resolve the windowed (composited) present MECHANISM for this session the DCP
/// swapID-panic mitigation picker (see `WindowedPresentMode`). The
/// `PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface` env lever wins (dev A/B);
/// otherwise the user's safe-present setting: ON/unset `.transaction` (the validated
/// mitigation), OFF `.async` (the fast pre-mitigation path the panic returns on
/// affected high-refresh setups; the Settings caption says so). `.surface` is currently
/// env-only (prototype HDR-composite verification owed). Fullscreen always presents
/// async regardless (`setComposited`). Internal (not private) for unit tests.
static func windowedPresentMode(setting: Bool?, env: String?) -> WindowedPresentMode {
if let env, let mode = WindowedPresentMode(rawValue: env) { return mode }
return (setting ?? true) ? .transaction : .async
}
#endif
/// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder /// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder
/// stays reproducible on-device; macOS is pinned to 1, env ignored glass pacing exists /// stays reproducible on-device; macOS is pinned to 1, env ignored a deeper gate only builds
/// there as the DCP swapID kernel-panic mitigation (see `pacing`), and STRICT present /// a standing queue (see above), and macOS glass pacing exists for PyroWave smoothness
/// serialization is its point. Internal (not private) for unit tests. /// (see `pacing`), where depth 1 is the point. Internal (not private) for unit tests.
static func gateDepth(env: String?) -> Int { static func gateDepth(env: String?) -> Int {
#if os(macOS) #if os(macOS)
return 1 return 1
@@ -196,10 +208,16 @@ final class SessionPresenter {
private var stage2Link: CADisplayLink? private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer? private var metalLayer: CAMetalLayer?
#if os(macOS) #if os(macOS)
/// The windowed-mode PyroWave present target (sibling above `metalLayer`) and the last /// The windowed present MECHANISM this session runs while composited (resolved once per
/// routing pushed to the pipeline see `setComposited`. Main-thread only, like all of this. /// session in `start` the user's safe-present setting + the PUNKTFUNK_WINDOWED_PRESENT
/// dev override) and the routing last pushed to the pipeline see `setComposited` (the DCP
/// swapID-panic mitigation). Main-thread only, like all of this.
private var windowedMode: WindowedPresentMode = .transaction
private var windowedPresentApplied: WindowedPresentMode = .async
/// The windowed `surface` present target (sibling above `metalLayer`, transparent while
/// unused) installed whenever stage-2 runs so a mechanism flip never has to mutate the
/// layer tree mid-session.
private var surfaceLayer: CALayer? private var surfaceLayer: CALayer?
private var surfacePresentsActive = false
#endif #endif
private var connection: PunktfunkConnection? private var connection: PunktfunkConnection?
/// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's /// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's
@@ -283,11 +301,17 @@ final class SessionPresenter {
baseLayer.addSublayer(metal) baseLayer.addSublayer(metal)
metalLayer = metal metalLayer = metal
#if os(macOS) #if os(macOS)
// The windowed-PyroWave present target sits ABOVE the metal layer: transparent (nil windowedPresentApplied = .async
// contents) while the metal path presents, covering it while surface presents run. // Resolve THIS session's windowed mechanism once (setting + dev env lever)
// `setComposited` routes between it and fullscreen-async from every layout.
windowedMode = Self.windowedPresentMode(
setting: UserDefaults.standard.object(
forKey: DefaultsKey.windowedSafePresent) as? Bool,
env: ProcessInfo.processInfo.environment["PUNKTFUNK_WINDOWED_PRESENT"])
// The surface present target sits ABOVE the metal layer: transparent (nil contents)
// unless the surface mechanism actually presents, covering it while it does.
baseLayer.addSublayer(pipeline.surfaceLayer) baseLayer.addSublayer(pipeline.surfaceLayer)
surfaceLayer = pipeline.surfaceLayer surfaceLayer = pipeline.surfaceLayer
surfacePresentsActive = false
#endif #endif
stage2 = pipeline stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger // The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
@@ -432,19 +456,23 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
/// Route presents for the window's composited state (MAIN thread the view pushes it on /// Route presents for the window's composited state (MAIN thread the view pushes it on
/// every layout, which fullscreen transitions always trigger). PyroWave sessions in a /// every layout, which fullscreen transitions always trigger). A COMPOSITED (windowed)
/// COMPOSITED (windowed) session present via `surfaceLayer` contents instead of the /// session presents through this session's resolved mitigation mechanism (`windowedMode`
/// CAMetalLayer image queue the DCP "mismatched swapID's" kernel-panic mitigation (see /// transactional by default, see `windowedPresentMode`) instead of the async image queue
/// `MetalVideoPresenter.surfaceLayer`; the metal-swap race survives glass pacing, so pacing /// the DCP "mismatched swapID's" kernel-panic mitigation (see `MetalVideoPresenter`; the
/// alone was not enough). VT codecs keep the metal path: no panic reports there, and their /// async-swap race survives glass pacing, so pacing alone was not enough). ALL codecs:
/// HDR/EDR presentation has no surface-contents equivalent wired. /// PyroWave hit it 2026-07-18 and windowed HEVC hit the same 240 Hz Mac Studio 2026-07-21
/// it is the async image queue itself, not any codec or present rate. Fullscreen keeps the
/// async path (direct scanout, lowest latency, no panic there). The full HDR/EDR render
/// path is preserved in every mechanism.
func setComposited(_ composited: Bool) { func setComposited(_ composited: Bool) {
guard let stage2, let connection else { return } guard let stage2 else { return }
let wantsSurface = composited && connection.videoCodec == .pyrowave let mode: WindowedPresentMode = composited ? windowedMode : .async
guard wantsSurface != surfacePresentsActive else { return } guard mode != windowedPresentApplied else { return }
surfacePresentsActive = wantsSurface let wasSurface = windowedPresentApplied == .surface
stage2.setSurfacePresents(wantsSurface) windowedPresentApplied = mode
if !wantsSurface { stage2.setWindowedPresent(mode)
if wasSurface {
// Uncover the metal layer NOW (its last drawable is still attached, so fullscreen // Uncover the metal layer NOW (its last drawable is still attached, so fullscreen
// entry shows the previous frame until the next present no black flash). // entry shows the previous frame until the next present no black flash).
CATransaction.begin() CATransaction.begin()
@@ -471,7 +499,7 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
surfaceLayer?.removeFromSuperlayer() surfaceLayer?.removeFromSuperlayer()
surfaceLayer = nil surfaceLayer = nil
surfacePresentsActive = false windowedPresentApplied = .async
#endif #endif
connection = nil connection = nil
} }
@@ -1114,15 +1114,15 @@ public final class Stage2Pipeline {
} }
#if os(macOS) #if os(macOS)
/// The windowed-mode PyroWave present target (see `MetalVideoPresenter.surfaceLayer` the /// Forward the windowed present mechanism (MAIN thread see
/// DCP swapID-panic mitigation). The hosting view installs it as a sibling above `layer`. /// `MetalVideoPresenter.setWindowedPresent`, the DCP swapID-panic mitigation).
public var surfaceLayer: CALayer { presenter.surfaceLayer } func setWindowedPresent(_ mode: WindowedPresentMode) {
presenter.setWindowedPresent(mode)
/// Forward the windowed-vs-fullscreen present routing (MAIN thread see
/// `MetalVideoPresenter.setSurfacePresents`).
public func setSurfacePresents(_ on: Bool) {
presenter.setSurfacePresents(on)
} }
/// The windowed `surface` present target the hosting SessionPresenter installs as a sibling
/// ABOVE `layer` (transparent while unused see `MetalVideoPresenter.surfaceLayer`).
var surfaceLayer: CALayer { presenter.surfaceLayer }
#endif #endif
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen` /// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
@@ -700,9 +700,9 @@ public final class StreamLayerView: NSView {
private func layoutPresenter() { private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1) presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
// Present routing tracks the window's composited state (fullscreen transitions always // Present routing tracks the window's composited state (fullscreen transitions always
// re-layout, so this stays current): windowed PyroWave presents via surface contents // re-layout, so this stays current): a windowed session presents through a Core Animation
// the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). A view // transaction the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited).
// not yet in a window counts as composited (the safe default). // A view not yet in a window counts as composited (the safe default).
presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false)) presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false))
// Feed the follower only once in a window (backing scale is real then) and with real // Feed the follower only once in a window (backing scale is real then) and with real
// bounds a pre-window layout would report point-sized dimensions. // bounds a pre-window layout would report point-sized dimensions.
@@ -70,6 +70,16 @@ public enum DefaultsKey {
/// (lowest latency the default, OFF). Resolved once per session; /// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header. /// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync" public static let vsync = "punktfunk.vsync"
/// macOS: present WINDOWED sessions in lockstep with the system compositor (the DCP
/// "mismatched swapID's" kernel-panic mitigation see SessionPresenter.windowedPresentMode
/// and the MetalVideoPresenter saga notes). ON/unset (the default): windowed presents ride
/// a Core Animation transaction validated panic-free on the 240 Hz repro machine, at a
/// small display-latency cost vs the raw path. OFF: windowed sessions keep the fast async
/// image queue ON AFFECTED SETUPS (high-refresh displays) THAT PATH KERNEL-PANICS THE
/// WHOLE MAC, which is why the default is ON. Fullscreen always presents async (fast path)
/// regardless. Resolved once per session; PUNKTFUNK_WINDOWED_PRESENT=async|transaction|
/// surface overrides it for dev A/B.
public static let windowedSafePresent = "punktfunk.windowedSafePresent"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor, /// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical /// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off, /// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
@@ -316,6 +316,41 @@ final class PresentPacingTests: XCTestCase {
SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline) SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline)
} }
// MARK: - Windowed present mechanism (the macOS DCP swapID-panic mitigation picker)
#if os(macOS)
/// The safe-present setting: ON/unset the validated transactional mitigation; an explicit
/// OFF the fast async path (the user accepted the affected-setup panic risk). The
/// PUNKTFUNK_WINDOWED_PRESENT env lever overrides both ways, `surface` is env-only (the
/// prototype mechanism), and garbage/empty env values are "unset", not an override.
func testWindowedPresentModeResolution() {
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: nil), .transaction,
"unset defaults to the panic mitigation")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: nil), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: nil), .async,
"an explicit opt-out gets the fast async path")
// The dev env lever wins over the setting, both directions.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "async"), .async)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "transaction"),
.transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "surface"), .surface,
"the surface prototype is reachable via env only")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "surface"), .surface)
// Garbage/empty env = unset.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: "garbage"), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: ""), .async)
}
#endif
// MARK: - Glass-gate depth // MARK: - Glass-gate depth
/// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue /// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue
+11 -2
View File
@@ -16,6 +16,8 @@
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games) # PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect) # PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990) # PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_CONNECT_TIMEOUT connect budget in seconds (optional; the plugin stretches it after
# firing Wake-on-LAN so the connect survives the host's resume)
# PF_APPID flatpak app id (default io.unom.Punktfunk) # PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH) # PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
# #
@@ -61,10 +63,17 @@ if [ -z "${PF_HOST:-}" ]; then
echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2 echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2
exit 2 exit 2
fi fi
# Trailing args shared by both streaming execs. A stretched connect budget rides along when the
# plugin set one (it just fired Wake-on-LAN, so the host may still be resuming); an older flatpak
# without --connect-timeout ignores the flag harmlessly (hand-scanned argv).
set -- --fullscreen
if [ -n "${PF_CONNECT_TIMEOUT:-}" ]; then
set -- --connect-timeout "$PF_CONNECT_TIMEOUT" "$@"
fi
if [ -n "${PF_LAUNCH:-}" ]; then if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title. # A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" "$@"
fi fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" "$@"
+32 -7
View File
@@ -70,7 +70,9 @@ function setShortcutHidden(appId: number, hidden: boolean): void {
}; };
// Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId). // Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId).
const ART_VERSION = 2; // v3: CI zips through 0.17.1 shipped no assets/ at all, yet v2 was still recorded as applied
// on those installs — the bump makes them re-apply once on the first build that has the files.
const ART_VERSION = 3;
function artKey(appId: number): string { function artKey(appId: number): string {
return `punktfunk:shortcutArt:${appId}`; return `punktfunk:shortcutArt:${appId}`;
} }
@@ -79,7 +81,7 @@ function artKey(appId: number): string {
* Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per * Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per
* appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call. * appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call.
*/ */
async function applyArtwork(appId: number): Promise<void> { async function applyArtwork(appId: number, isRetry = false): Promise<void> {
try { try {
if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) { if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) {
return; return;
@@ -91,16 +93,29 @@ async function applyArtwork(appId: number): Promise<void> {
[art.logo, 2], [art.logo, 2],
[art.gridwide, 3], [art.gridwide, 3],
]; ];
let applied = false;
for (const [data, assetType] of assets) { for (const [data, assetType] of assets) {
if (data) { if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType); await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
applied = true;
} }
} }
if (art.icon_path) { if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path); SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
applied = true;
} }
// Only record "done" when something actually landed — a plugin build whose assets/ is
// missing/empty must keep retrying on later mounts instead of poisoning the marker.
if (applied) {
localStorage.setItem(artKey(appId), `${ART_VERSION}`); localStorage.setItem(artKey(appId), `${ART_VERSION}`);
}
} catch (e) { } catch (e) {
// A shortcut fresh out of AddShortcut may not be registered yet (the same race
// setShortcutHidden defers around) — one deferred second attempt, then leave it to
// the next mount.
if (!isRetry) {
setTimeout(() => void applyArtwork(appId, true), 2500);
}
console.warn("punktfunk: shortcut artwork not applied", e); console.warn("punktfunk: shortcut artwork not applied", e);
} }
} }
@@ -157,7 +172,9 @@ async function ensureControllerConfig(): Promise<void> {
return; return;
} }
const r = await applyControllerConfig(SHORTCUT_NAME); const r = await applyControllerConfig(SHORTCUT_NAME);
if (r?.ok) { // `ok` alone isn't done: with zero account configset dirs (fresh Steam) the backend
// succeeds without pointing any account at the template — keep retrying until one lands.
if (r?.ok && (r.applied ?? []).some((a) => a.startsWith("configset:"))) {
localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`); localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`);
} else { } else {
console.warn("punktfunk: controller config not fully applied", r); console.warn("punktfunk: controller config not fully applied", r);
@@ -283,13 +300,21 @@ export async function launchStream(
opts: LaunchOpts = {}, opts: LaunchOpts = {},
): Promise<void> { ): Promise<void> {
// Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now // Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now
// so it races with the shortcut setup (near-zero added latency), and awaited just before RunGame. // so it races with the shortcut setup (near-zero added latency); its outcome is needed below
// (the connect budget), and RunGame follows the await either way, so nothing is slower for it.
// Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is // Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is
// known), and the connect that follows has its own retry window, so a failure never blocks launch. // known), and the connect that follows has its own retry window, so a failure never blocks launch.
const waking = wake(host, port).catch(() => ({ ok: false })); const waking = wake(host, port).catch(() => ({ ok: false }));
const { appId, runner } = await ensureStreamShortcut(); const [{ appId, runner }, woke] = await Promise.all([ensureStreamShortcut(), waking]);
const target = port && port !== 9777 ? `${host}:${port}` : host; const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`]; const env = [`PF_HOST=${target}`];
// A magic packet actually went out (a MAC was known), so the host may be mid-resume from
// suspend — that takes far longer than the client's default 15 s connect budget. Stretch the
// budget so the client's wake-tolerant dial keeps retrying across the resume; against an
// already-awake host the connect still lands in under a second, so this costs nothing.
if (woke.ok) {
env.push("PF_CONNECT_TIMEOUT=75");
}
if (opts.browse) { if (opts.browse) {
env.push("PF_BROWSE=1"); env.push("PF_BROWSE=1");
if (opts.mgmt) { if (opts.mgmt) {
@@ -303,9 +328,9 @@ export async function launchStream(
env.push(`PF_LAUNCH=${opts.launchId}`); env.push(`PF_LAUNCH=${opts.launchId}`);
} }
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper // KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_* from the environment. // script rides behind it as an argument and reads PF_* from the environment. The wake was
// awaited above, so the magic packet is out before the connect attempt.
SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`); SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
await waking; // ensure the magic packet is out before the connect attempt
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100); SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
} }
+12
View File
@@ -498,6 +498,13 @@ async fn session(args: Args) -> Result<()> {
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() { if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444; caps |= punktfunk_core::quic::VIDEO_CAP_444;
} }
// PUNKTFUNK_CLIENT_CHACHA20=1 advertises VIDEO_CAP_CHACHA20 — drives the
// host's ChaCha20-Poly1305 session-cipher resolution (the soft-AES armv7
// negotiation, design/chacha20-session-cipher.md §7) without a webOS build;
// the negotiated cipher is reported in the welcome log line below.
if std::env::var_os("PUNKTFUNK_CLIENT_CHACHA20").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_CHACHA20;
}
caps caps
}, },
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the // `--audio-channels` (default stereo); the probe multistream-decodes + validates the
@@ -535,6 +542,11 @@ async fn session(args: Args) -> Result<()> {
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444, chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format, chroma_format_idc = welcome.chroma_format,
codec = codec_ext(welcome.codec), codec = codec_ext(welcome.codec),
cipher = if welcome.cipher == punktfunk_core::quic::CIPHER_CHACHA20_POLY1305 {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
"session offer" "session offer"
); );
+16
View File
@@ -24,6 +24,16 @@ use pf_frame::DmabufFrame;
pub trait Capturer: Send { pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>; fn next_frame(&mut self) -> Result<CapturedFrame>;
/// [`next_frame`](Self::next_frame) with a caller-chosen first-frame budget instead of the
/// backend's default. The pipeline retry loop shortens its FIRST attempt's wait: a PipeWire
/// stream connected while gamescope re-inits its headless takeover can negotiate a format,
/// reach `Streaming`, and still never receive a buffer — a fresh connect then delivers within
/// ~0.5 s, so waiting out the full default budget on a doomed stream just delays the retry
/// that fixes it. Backends without an internal wait budget ignore it (the default delegates).
fn next_frame_within(&mut self, _budget: std::time::Duration) -> Result<CapturedFrame> {
self.next_frame()
}
/// Non-blocking: the freshest frame available since the last call, or `None` if none has /// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default /// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal /// just produces a frame each call — fine for instant synthetic sources; the portal
@@ -249,6 +259,12 @@ pub struct ZeroCopyPolicy {
/// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only /// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only
/// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`. /// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`.
pub pyrowave_session: bool, pub pyrowave_session: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (the Linux raw Vulkan
/// Video backend on an H265/AV1 session — resolved by the host facade via
/// `pf_encode::linux_native_nv12_ok`). Gates whether the negotiation PREFERS gamescope's
/// producer-side NV12 pod: libav VAAPI (H264's backend) would misread the two-plane buffer,
/// so H264/GameStream/PyroWave sessions must never see NV12 frames.
pub native_nv12_session: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc, /// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA, /// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise. /// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
+155 -49
View File
@@ -299,29 +299,11 @@ fn spawn_pipewire(
impl Capturer for PortalCapturer { impl Capturer for PortalCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> { fn next_frame(&mut self) -> Result<CapturedFrame> {
// First frame can lag behind format negotiation; later frames arrive at ~fps. Wait in self.frame_within(Duration::from_secs(10))
// short slices so a GPU-import poison (worker death) fails the capture within ~0.5 s
// instead of sitting out the full first-frame budget.
let deadline = std::time::Instant::now() + Duration::from_secs(10);
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e),
}
} }
fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
self.frame_within(budget)
} }
fn supports_arrival_wait(&self) -> bool { fn supports_arrival_wait(&self) -> bool {
@@ -417,9 +399,41 @@ impl Capturer for PortalCapturer {
} }
impl PortalCapturer { impl PortalCapturer {
/// The [`Capturer::next_frame`] budget expired (or the thread ended) — turn it into the /// The blocking first-frame wait behind [`Capturer::next_frame`] /
/// diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged. /// [`Capturer::next_frame_within`]. First frame can lag behind format negotiation; later
fn next_frame_timed_out(&self, err: RecvTimeoutError) -> Result<CapturedFrame> { /// frames arrive at ~fps. Wait in short slices so a GPU-import poison (worker death) fails
/// the capture within ~0.5 s instead of sitting out the full first-frame budget.
fn frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
let deadline = std::time::Instant::now() + budget;
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e, budget),
}
}
}
/// The [`frame_within`](Self::frame_within) budget expired (or the thread ended) — turn it
/// into the diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(
&self,
err: RecvTimeoutError,
budget: Duration,
) -> Result<CapturedFrame> {
let within = budget.as_secs_f32();
match err { match err {
RecvTimeoutError::Timeout => { RecvTimeoutError::Timeout => {
// Split the two black-screen root causes apart so the operator gets a cause, not // Split the two black-screen root causes apart so the operator gets a cause, not
@@ -427,9 +441,10 @@ impl PortalCapturer {
// not (no acceptable format / node never emitted a param)? // not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) { if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiated but no buffers \ "no PipeWire frame within {within}s (node {}): format negotiated but no \
arrived the compositor produced no frames (virtual output idle/unmapped, \ buffers arrived the compositor produced no frames (virtual output \
or capture never started)", idle/unmapped, capture never started, or a stream bound during a \
compositor (re)start that will never deliver a reconnect fixes that)",
self.node_id self.node_id
)) ))
} else if self.hdr_offer { } else if self.hdr_offer {
@@ -440,10 +455,10 @@ impl PortalCapturer {
// auto-reconnects) negotiates SDR instead of re-running this same timeout. // auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed(); super::note_hdr_capture_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \ "no PipeWire frame within {within}s (node {}): the compositor never \
the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored monitor in \ accepted the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored \
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \ monitor in HDR mode on GNOME 50+? Downgrading this host to SDR capture; \
to stream SDR", reconnect to stream SDR",
self.node_id self.node_id
)) ))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() { } else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
@@ -452,14 +467,15 @@ impl PortalCapturer {
// retries on the CPU offer instead of failing this same negotiation forever. // retries on the CPU offer instead of failing this same negotiation forever.
pf_zerocopy::note_vaapi_dmabuf_failed(); pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \ "no PipeWire frame within {within}s (node {}): the compositor never \
the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \ accepted the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this \
CPU capture path; the pipeline rebuild will renegotiate without dmabuf", host to the CPU capture path; the pipeline rebuild will renegotiate \
without dmabuf",
self.node_id self.node_id
)) ))
} else { } else {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiation never \ "no PipeWire frame within {within}s (node {}): format negotiation never \
completed the compositor offered no format this consumer accepts \ completed the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param", (pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id self.node_id
@@ -824,6 +840,7 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba, VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb, VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr, VideoFormat::BGR => PixelFormat::Bgr,
VideoFormat::NV12 => PixelFormat::Nv12,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by // The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020). // the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10, VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
@@ -989,10 +1006,10 @@ mod pipewire {
.into_inner()) .into_inner())
} }
/// Build a BGRx dmabuf `EnumFormat` pod advertising the EGL-importable `modifiers` as a /// Build a LINEAR/modifier DMA-BUF `EnumFormat` pod. Packed BGRx is the existing import path;
/// mandatory enum Choice; the compositor fixates to one of them that it can allocate, which /// NV12 is gamescope's producer-side RGB→YUV path (opt-in during bring-up).
/// we read back in `param_changed`.
fn build_dmabuf_format( fn build_dmabuf_format(
format: VideoFormat,
modifiers: &[u64], modifiers: &[u64],
preferred: Option<(u32, u32, u32)>, preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> { ) -> Result<Vec<u8>> {
@@ -1003,7 +1020,7 @@ mod pipewire {
pw::spa::param::ParamType::EnumFormat, pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video), pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw), pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, VideoFormat::BGRx), pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
pw::spa::pod::property!( pw::spa::pod::property!(
FormatProperties::VideoSize, FormatProperties::VideoSize,
Choice, Choice,
@@ -1032,6 +1049,22 @@ mod pipewire {
pw::spa::utils::Fraction { num: 240, denom: 1 } pw::spa::utils::Fraction { num: 240, denom: 1 }
), ),
); );
if format == VideoFormat::NV12 {
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorMatrix,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_MATRIX_BT709,
)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorRange,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_RANGE_16_235,
)),
});
}
obj.properties.push(pw::spa::pod::Property { obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier, key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY, flags: pw::spa::pod::PropertyFlags::MANDATORY,
@@ -1604,8 +1637,8 @@ mod pipewire {
} }
} }
// VAAPI zero-copy passthrough: hand the raw dmabuf straight to the encoder, which imports // Raw DMA-BUF passthrough: packed RGB is imported for GPU CSC; producer-native NV12 can
// it into a VA surface and does RGB→NV12 on the GPU video engine. No CUDA importer here. // be consumed by the Vulkan Video encoder without another color conversion.
if ud.vaapi_passthrough { if ud.vaapi_passthrough {
if let Some(fmt) = ud.format { if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf { if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
@@ -1613,9 +1646,41 @@ mod pipewire {
let chunk = datas[0].chunk(); let chunk = datas[0].chunk();
let offset = chunk.offset(); let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32; let stride = chunk.stride().max(0) as u32;
// Native NV12 usually arrives as a two-plane SPA buffer over ONE buffer
// object; plane 1's chunk carries the REAL UV offset/stride (compositors
// may align the Y plane before UV). Pass it through instead of assuming
// contiguity. Each spa_data holds its own (dup'd) fd, so BO identity is
// by inode, not fd number; a genuinely two-BO frame cannot travel through
// the single-fd import — drop it with a diagnosis instead of streaming
// garbage chroma.
let plane1 =
if fmt == PixelFormat::Nv12 && datas.len() >= 2 && datas[1].fd() > 0 {
// SAFETY: zeroed `libc::stat` is a valid POD initializer; both fds are
// owned by the live PipeWire buffer for this callback, and `fstat`
// only writes the out-param structs, whose fields are read only after
// the `== 0` success checks.
let same_bo = unsafe {
let mut s0: libc::stat = std::mem::zeroed();
let mut s1: libc::stat = std::mem::zeroed();
libc::fstat(datas[0].fd() as i32, &mut s0) == 0
&& libc::fstat(datas[1].fd() as i32, &mut s1) == 0
&& (s0.st_dev, s0.st_ino) == (s1.st_dev, s1.st_ino)
};
if !same_bo {
warn_once(
"NV12 planes live in different buffer objects — frames \
dropped (single-fd import only)",
);
return;
}
let c1 = datas[1].chunk();
Some((c1.offset(), c1.stride().max(0) as u32))
} else {
None
};
// dup the fd so it survives the SPA buffer recycle — the encode thread // dup the fd so it survives the SPA buffer recycle — the encode thread
// imports it. (Content stability across the brief map+CSC window relies on // imports it. Content stability across the brief import/encode window relies
// the compositor's buffer-pool depth, like any zero-copy capture.) // on the compositor's buffer-pool depth, like any zero-copy capture.
// SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid // SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid
// for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd, // for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd,
// touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1). // touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1).
@@ -1642,9 +1707,10 @@ mod pipewire {
modifier: ud.modifier, modifier: ud.modifier,
offset, offset,
stride, stride,
plane1,
}), }),
// Cursor-as-metadata: the encoder blends this into its owned VA // Cursor-as-metadata is blended only by RGB→NV12 backends. Gamescope
// surface (raw dmabuf never touched). // embeds its pointer in the produced pixels, so native NV12 has none.
cursor: ud.cursor.overlay(), cursor: ud.cursor.overlay(),
}); });
static ONCE: std::sync::atomic::AtomicBool = static ONCE: std::sync::atomic::AtomicBool =
@@ -1655,7 +1721,12 @@ mod pipewire {
h, h,
modifier = ud.modifier, modifier = ud.modifier,
fourcc = format_args!("{:#010x}", fourcc), fourcc = format_args!("{:#010x}", fourcc),
"zero-copy: handing the raw dmabuf to the encoder (GPU import + CSC)" source = if fmt == PixelFormat::Nv12 {
"producer-native NV12"
} else {
"packed RGB (encoder GPU CSC)"
},
"zero-copy: handing the raw DMA-BUF to the encoder"
); );
} }
return; return;
@@ -2015,6 +2086,28 @@ mod pipewire {
// PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw // PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw
// dmabuf straight to the encoder. // dmabuf straight to the encoder.
let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough; let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough;
// Producer-side NV12 (default-on; PUNKTFUNK_PIPEWIRE_NV12=0 escapes): gamescope offers a
// one-fd LINEAR NV12 image when the consumer asks — its compositor pass does the RGB→YUV,
// and the Vulkan Video encoder imports the buffer as its encode source directly (no host
// CSC at all). `native_nv12_session` restricts this to sessions whose encoder can ingest
// it (Linux vulkan-encode H265/AV1 — never H264/libav-VAAPI, GameStream-resolve, or
// PyroWave, whose Vulkan compute CSC ingests packed RGB only). Raw passthrough is
// required because the CUDA importer expects packed RGB, and 4:4:4/HDR must not be
// silently subsampled/downconverted. Non-NV12 compositors (KWin/GNOME) simply match the
// packed-RGB fallback pod.
let prefer_native_nv12 = std::env::var("PUNKTFUNK_PIPEWIRE_NV12").as_deref() != Ok("0")
&& policy.native_nv12_session
&& backend_is_vaapi
&& vaapi_passthrough
&& !policy.pyrowave_session
&& !want_444
&& !want_hdr;
if prefer_native_nv12 {
tracing::info!(
"zero-copy: preferring gamescope producer-side NV12 LINEAR DMA-BUF (no host \
RGB CSC; PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation)"
);
}
// Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR // Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR
// (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via // (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via
// CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only: // CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only:
@@ -2054,7 +2147,9 @@ mod pipewire {
tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path"); tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path");
} else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() { } else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() {
tracing::info!( tracing::info!(
"zero-copy: advertising LINEAR dmabuf for direct VAAPI import (GPU CSC)" native_nv12_preferred = prefer_native_nv12,
"zero-copy: advertising LINEAR DMA-BUF for encoder import (native NV12 first \
when enabled, packed RGB fallback)"
); );
} else if want_dmabuf && !vaapi_passthrough { } else if want_dmabuf && !vaapi_passthrough {
tracing::info!( tracing::info!(
@@ -2394,7 +2489,18 @@ mod pipewire {
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?, build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
] ]
} else if want_dmabuf { } else if want_dmabuf {
vec![build_dmabuf_format(&modifiers, preferred)?] let mut pods = Vec::with_capacity(if prefer_native_nv12 { 2 } else { 1 });
if prefer_native_nv12 {
// First compatible consumer pod wins. Gamescope advertises NV12 and BGRx; pinning
// BT.709 limited here selects its RGB→NV12 shader with our bitstream colorimetry.
pods.push(build_dmabuf_format(VideoFormat::NV12, &[0], preferred)?);
}
pods.push(build_dmabuf_format(
VideoFormat::BGRx,
&modifiers,
preferred,
)?);
pods
} else { } else {
vec![serialize_pod(obj)?] vec![serialize_pod(obj)?]
}; };
+209 -12
View File
@@ -308,8 +308,9 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV /// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This /// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
/// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`] /// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`]
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format so the caller can fall /// surfaces a clear error if `CreateRenderTargetView` rejects the plane format. (There is no runtime
/// back to the existing R10 path. /// fallback — the error propagates through `try_consume` and ends the session; the "R10 path" the
/// original design referenced was never kept.)
pub(crate) struct HdrP010Converter { pub(crate) struct HdrP010Converter {
vs: ID3D11VertexShader, vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader, ps_y: ID3D11PixelShader,
@@ -737,14 +738,157 @@ fn p010_reference(r: f64, g: f64, b: f64) -> (f64, f64, f64) {
/// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL. /// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL.
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn hdr_p010_selftest() -> Result<()> { pub fn hdr_p010_selftest() -> Result<()> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_HARDWARE; hdr_p010_selftest_at(64, 64, None)
use windows::Win32::Graphics::Dxgi::IDXGIAdapter; }
// 64x64, even dims. A 4x4 grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → /// [`hdr_p010_selftest`] at an arbitrary even size and (optionally) on a specific GPU vendor
// exact chroma comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus /// (PCI vendor id, e.g. `0x8086` Intel / `0x10de` NVIDIA / `0x1002` AMD). The size matters on
// a couple of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only). /// top of the 64×64 default because the field sessions run at capture resolutions whose height
const W: u32 = 64; /// is NOT 16-aligned (1080 → the encoder's align16 pool seam) and a driver may treat the planar
const H: u32 = 64; /// RTVs differently at real sizes; the vendor pin matters on dual-GPU boxes where the default
/// adapter is not the one the session encodes on.
/// Test support (used by pf-encode's live e2e): the 8 sRGB colour bars (white/yellow/cyan/green/
/// magenta/red/blue/black, sRGB 1.0 = scRGB 1.0 = 80 nits) as a w×h FP16 scRGB texture on the
/// adapter with `luid`, converted through the REAL [`HdrP010Converter`] into a P010 texture with
/// **`BIND_RENDER_TARGET` only, `MiscFlags` 0 — the exact bind profile of the IDD out-ring** (the
/// CPU-upload encoder tests can't use that profile, so only this path exercises "RTV-written P010
/// → encoder ingest copy"). Returns `(device, p010)`; expected decoded codes per bar are the
/// bars_pq2020 fixture's: (490,512,512) (478,423,518) (464,525,473) (450,432,476) (350,584,585)
/// (325,448,598) (226,650,535) (64,512,512).
#[cfg(target_os = "windows")]
#[doc(hidden)]
pub fn hdr_p010_convert_bars_on_luid(
luid: [u8; 8],
w: u32,
h: u32,
) -> Result<(ID3D11Device, ID3D11Texture2D)> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("bars pattern needs even non-zero dimensions, got {w}x{h}");
}
// sRGB primaries at full/zero channels: sRGB EOTF(1.0)=1.0, (0)=0 → the scRGB pattern is
// pure 0/1 floats and the PQ/BT.2020 reference codes above are exact.
const BARS: [(f32, f32, f32); 8] = [
(1.0, 1.0, 1.0),
(1.0, 1.0, 0.0),
(0.0, 1.0, 1.0),
(0.0, 1.0, 0.0),
(1.0, 0.0, 1.0),
(1.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
(0.0, 0.0, 0.0),
];
let bar_w = (w / 8).max(1) as usize;
let mut fp16 = vec![0u16; (w * h * 4) as usize];
for y in 0..h as usize {
for x in 0..w as usize {
let (r, g, b) = BARS[(x / bar_w).min(7)];
let i = (y * w as usize + x) * 4;
fp16[i] = f32_to_f16(r);
fp16[i + 1] = f32_to_f16(g);
fp16[i + 2] = f32_to_f16(b);
fp16[i + 3] = f32_to_f16(1.0);
}
}
// SAFETY: same single-device/single-thread contract as `hdr_p010_selftest_at`; the FP16
// initial-data Vec outlives the synchronous CreateTexture2D; the returned COM handles own
// their references.
unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("dxgi factory")?;
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).context("adapter by luid")?;
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice(luid) for bars convert")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
let src_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R16G16B16A16_FLOAT,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: fp16.as_ptr() as *const c_void,
SysMemPitch: w * 8,
SysMemSlicePitch: 0,
};
let mut src_tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&src_desc, Some(&init), Some(&mut src_tex))
.context("CreateTexture2D(fp16 bars)")?;
let src_tex = src_tex.context("null src tex")?;
let mut src_srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&src_tex, None, Some(&mut src_srv))
.context("CreateShaderResourceView(fp16 bars)")?;
let src_srv = src_srv.context("null src srv")?;
// The IDD out-ring's exact profile: P010, RENDER_TARGET only, MiscFlags 0.
let p010_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
..Default::default()
};
let mut p010: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&p010_desc, None, Some(&mut p010))
.context("CreateTexture2D(P010 bars dst)")?;
let p010 = p010.context("null p010 tex")?;
let conv = HdrP010Converter::new(&device)?;
conv.convert(&device, &context, &src_srv, &p010, w, h)?;
Ok((device, p010))
}
}
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("hdr-p010-selftest needs even non-zero dimensions, got {w}x{h}");
}
// A grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → exact chroma
// comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus a couple
// of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
#[allow(non_snake_case)]
let (W, H) = (w, h);
const BLK: u32 = 16; const BLK: u32 = 16;
// (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values. // (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values.
let named: [(&str, f32, f32, f32); 8] = [ let named: [(&str, f32, f32, f32); 8] = [
@@ -797,12 +941,36 @@ pub fn hdr_p010_selftest() -> Result<()> {
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are // `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below. // proven individually at the `read_u16` closure below.
unsafe { unsafe {
// Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test). // Device on the requested vendor's adapter (dual-GPU boxes encode on a specific one), else
// the default hardware GPU. Always says which adapter ran — a PASS is only meaningful for
// the GPU it actually tested.
let adapter: Option<IDXGIAdapter> = match vendor {
None => None,
Some(want) => {
let factory: IDXGIFactory1 = CreateDXGIFactory1().context("dxgi factory")?;
let mut found = None;
for i in 0.. {
let Ok(a) = factory.EnumAdapters(i) else {
break;
};
let desc = a.GetDesc().context("adapter desc")?;
if desc.VendorId == want {
found = Some(a);
break;
}
}
Some(found.with_context(|| format!("no adapter with vendor id {want:#x}"))?)
}
};
let mut device: Option<ID3D11Device> = None; let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None; let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice( D3D11CreateDevice(
None::<&IDXGIAdapter>, adapter.as_ref(),
D3D_DRIVER_TYPE_HARDWARE, if adapter.is_some() {
D3D_DRIVER_TYPE_UNKNOWN
} else {
D3D_DRIVER_TYPE_HARDWARE
},
HMODULE::default(), HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]), Some(&[D3D_FEATURE_LEVEL_11_0]),
@@ -814,6 +982,22 @@ pub fn hdr_p010_selftest() -> Result<()> {
.context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?; .context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?;
let device = device.context("null device")?; let device = device.context("null device")?;
let context = context.context("null context")?; let context = context.context("null context")?;
{
let dxgi: windows::Win32::Graphics::Dxgi::IDXGIDevice =
device.cast().context("device -> IDXGIDevice")?;
let desc = dxgi.GetAdapter().context("GetAdapter")?.GetDesc()?;
let name = String::from_utf16_lossy(
&desc.Description[..desc
.Description
.iter()
.position(|&c| c == 0)
.unwrap_or(desc.Description.len())],
);
println!(
"adapter: {name} (vendor {:#06x}, luid {:08x}:{:08x})",
desc.VendorId, desc.AdapterLuid.HighPart, desc.AdapterLuid.LowPart
);
}
// Source FP16 texture (initialized) + SRV. // Source FP16 texture (initialized) + SRV.
let src_desc = D3D11_TEXTURE2D_DESC { let src_desc = D3D11_TEXTURE2D_DESC {
@@ -1175,3 +1359,16 @@ impl VideoConverter {
blt.context("VideoProcessorBlt") blt.context("VideoProcessorBlt")
} }
} }
#[cfg(test)]
mod hdr_selftests {
/// LIVE (needs the GPU): [`super::hdr_p010_selftest_at`] at the field capture size — 1080 is
/// NOT 16-aligned, and the planar-RTV write path is driver-specific per vendor. Pinned to the
/// Intel adapter (`0x8086`), so it runs on the Intel validation boxes and errors out cleanly
/// ("no adapter") elsewhere. `cargo test -p pf-capture -- --ignored hdr_p010 --nocapture`.
#[test]
#[ignore]
fn hdr_p010_selftest_intel_1080_live() {
super::hdr_p010_selftest_at(1920, 1080, Some(0x8086)).expect("hdr p010 selftest @1080");
}
}
+5
View File
@@ -25,6 +25,11 @@ tracing = "0.1"
# A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`). # A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`).
tracing-subscriber = { version = "0.3", features = ["env-filter"] } tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[target.'cfg(target_os = "windows")'.dev-dependencies]
# The QSV live e2e drives the REAL HdrP010Converter output (an RTV-written, ring-profile P010
# texture) into the encoder — the one seam the CPU-upload tests can't reach.
pf-capture = { path = "../pf-capture" }
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies] [target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
# Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms. # Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms.
openh264 = "0.9" openh264 = "0.9"
+24 -6
View File
@@ -37,9 +37,11 @@ pub(crate) fn fourcc_to_vk(fourcc: u32) -> Option<vk::Format> {
const AR24: u32 = 0x3432_5241; // ARGB8888 const AR24: u32 = 0x3432_5241; // ARGB8888
const XB24: u32 = 0x3432_4258; // XBGR8888 const XB24: u32 = 0x3432_4258; // XBGR8888
const AB24: u32 = 0x3432_4241; // ABGR8888 const AB24: u32 = 0x3432_4241; // ABGR8888
const NV12: u32 = 0x3231_564e; // DRM_FORMAT_NV12
match fourcc { match fourcc {
XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM), XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM),
XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM), XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM),
NV12 => Some(vk::Format::G8_B8R8_2PLANE_420_UNORM),
_ => None, _ => None,
} }
} }
@@ -90,9 +92,10 @@ pub(crate) unsafe fn import_rgb_dmabuf(
) )
} }
/// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list /// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list.
/// (chained into the image create) — the RGB-direct encode path imports the captured buffer /// Despite the historical name, this also imports gamescope's one-fd LINEAR NV12: the UV
/// as a profiled `VIDEO_ENCODE_SRC` image instead of a sampled one. /// subresource layout comes from the producer's plane-1 chunk when it reported one, falling
/// back to the shared-stride contiguous-plane contract.
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn import_rgb_dmabuf_as( pub(crate) unsafe fn import_rgb_dmabuf_as(
device: &ash::Device, device: &ash::Device,
@@ -108,12 +111,27 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
use std::os::fd::IntoRawFd; use std::os::fd::IntoRawFd;
let fmt = fourcc_to_vk(d.fourcc) let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?; .with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
let plane = [vk::SubresourceLayout::default() let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
d.offset as u64 + d.stride as u64 * ch as u64,
d.stride as u64,
));
vec![
vk::SubresourceLayout::default()
.offset(d.offset as u64) .offset(d.offset as u64)
.row_pitch(d.stride as u64)]; .row_pitch(d.stride as u64),
vk::SubresourceLayout::default()
.offset(uv_offset)
.row_pitch(uv_stride),
]
} else {
vec![vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)]
};
let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default() let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(d.modifier) .drm_format_modifier(d.modifier)
.plane_layouts(&plane); .plane_layouts(&planes);
let mut ext = vk::ExternalMemoryImageCreateInfo::default() let mut ext = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT); .handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
let mut ci = vk::ImageCreateInfo::default() let mut ci = vk::ImageCreateInfo::default()
+461 -100
View File
@@ -14,7 +14,7 @@ use super::vk_util::{color_range, find_mem, make_plain_image, make_view, pixel_t
use crate::{Codec, EncodedFrame, Encoder, EncoderCaps}; use crate::{Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result}; use anyhow::{bail, Context, Result};
use ash::vk; use ash::vk;
use pf_frame::{CapturedFrame, FramePayload}; use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::ffi::c_void; use std::ffi::c_void;
use std::os::fd::AsRawFd; use std::os::fd::AsRawFd;
@@ -84,17 +84,36 @@ fn rgb_request() -> Option<bool> {
} }
} }
/// True-extent RGB-direct at unaligned modes (default ON; `PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0`
/// restores the padded-copy staging): direct-import the visible-size capture with the TRUE-SIZE
/// source `codedExtent` — RADV derives nonzero VCN firmware padding from it, so the EFC is told
/// the source lacks the alignment rows (see [`RgbDirect::true_extent`]). Guarded-tested on Van
/// Gogh 2026-07-21 (kernel clean, and the fastest 1080p encode path measured); the EFC only
/// exists on Mesa ≥ 26, where the `codedExtent`-driven `session_init` is guaranteed.
fn rgb_true_extent_request() -> bool {
std::env::var("PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT").as_deref() != Ok("0")
}
/// Live RGB-direct session config: the chroma-siting bits the session was created with /// Live RGB-direct session config: the chroma-siting bits the session was created with
/// (chosen from what the driver advertises — see [`probe_rgb_direct`]). /// (chosen from what the driver advertises — see [`probe_rgb_direct`]).
struct RgbDirect { struct RgbDirect {
x_offset: u32, // vk_valve_rgb::CHROMA_OFFSET_* x_offset: u32, // vk_valve_rgb::CHROMA_OFFSET_*
y_offset: u32, y_offset: u32,
/// The mode is not 64x16-aligned, so the captured buffer cannot be the encode source /// The mode is not 64x16-aligned, so the captured buffer cannot be the encode source
/// directly (the EFC would read past it — the 2026-07-20 field GPU hang). Instead each /// under the session's ALIGNED source extent (the EFC read past it — the 2026-07-20 field
/// frame is copied into a per-slot ALIGNED BGRA staging image with the edge rows/columns /// GPU hang, when the source `codedExtent` was the aligned size and RADV therefore derived
/// duplicated into the padding (transfer-only, no shader) and encoded from there. Aligned /// ZERO firmware padding). Each frame is copied into a per-slot ALIGNED BGRA staging image
/// modes keep the true zero-copy import. /// with the edge rows/columns duplicated into the padding (transfer-only, no shader) and
/// encoded from there. Aligned modes keep the true zero-copy import.
padded: bool, padded: bool,
/// The default unaligned-mode source strategy (`PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0` falls
/// back to `padded`): direct-import the visible-size buffer and pass the TRUE-SIZE source
/// `codedExtent` — RADV then programs nonzero firmware padding from it (Mesa ≥ 24.2
/// derives `session_init` padding from `srcPictureResource.codedExtent`; see
/// [`VulkanVideoEncoder::native_nv12`]), telling the VCN the source lacks the alignment
/// rows, which the hardware edge-extends internally. The session/SPS/DPB stay app-aligned.
/// Guarded-tested on Van Gogh (kernel clean; fastest 1080p path measured).
true_extent: bool,
} }
/// Stack storage for a complete rgb-chained video profile. Profiled image creation AFTER open /// Stack storage for a complete rgb-chained video profile. Profiled image creation AFTER open
@@ -155,6 +174,51 @@ impl RgbProfileStack {
} }
} }
/// Non-RGB video profile rebuilt for native NV12 DMA-BUF imports. Image creation after `open`
/// must carry a profile identical by value to the session profile.
struct NativeProfileStack {
usage: vk::VideoEncodeUsageInfoKHR<'static>,
h265: vk::VideoEncodeH265ProfileInfoKHR<'static>,
av1: super::vk_av1_encode::VideoEncodeAV1ProfileInfoKHR,
profile: vk::VideoProfileInfoKHR<'static>,
}
impl NativeProfileStack {
fn new(codec_op: vk::VideoCodecOperationFlagsKHR) -> Self {
use super::vk_av1_encode as av1b;
Self {
usage: vk::VideoEncodeUsageInfoKHR::default()
.video_usage_hints(vk::VideoEncodeUsageFlagsKHR::STREAMING)
.video_content_hints(vk::VideoEncodeContentFlagsKHR::RENDERED)
.tuning_mode(vk::VideoEncodeTuningModeKHR::ULTRA_LOW_LATENCY),
h265: vk::VideoEncodeH265ProfileInfoKHR::default().std_profile_idc(
vk::native::StdVideoH265ProfileIdc_STD_VIDEO_H265_PROFILE_IDC_MAIN,
),
av1: av1b::VideoEncodeAV1ProfileInfoKHR {
s_type: av1b::stype(av1b::ST_PROFILE_INFO),
p_next: std::ptr::null(),
std_profile: vk::native::StdVideoAV1Profile_STD_VIDEO_AV1_PROFILE_MAIN,
},
profile: vk::VideoProfileInfoKHR::default()
.video_codec_operation(codec_op)
.chroma_subsampling(vk::VideoChromaSubsamplingFlagsKHR::TYPE_420)
.luma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8)
.chroma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8),
}
}
fn wire(&mut self, av1: bool) -> &vk::VideoProfileInfoKHR<'static> {
if av1 {
self.av1.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.av1 as *const _ as *const c_void;
} else {
self.h265.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.h265 as *const _ as *const c_void;
}
&self.profile
}
}
/// The Vulkan codec-operation bit for our codec selection (shared by open and the per-import /// The Vulkan codec-operation bit for our codec selection (shared by open and the per-import
/// profile rebuilds — the two must agree, profile identity is by value). /// profile rebuilds — the two must agree, profile identity is by value).
fn codec_op_for(av1: bool) -> vk::VideoCodecOperationFlagsKHR { fn codec_op_for(av1: bool) -> vk::VideoCodecOperationFlagsKHR {
@@ -173,12 +237,12 @@ enum SrcAcquire {
/// CSC path: `nv12_src` was written by this frame's compute batch (GENERAL layout; the /// CSC path: `nv12_src` was written by this frame's compute batch (GENERAL layout; the
/// csc_sem orders the queues). /// csc_sem orders the queues).
CscGeneral, CscGeneral,
/// RGB-direct, first use of a dmabuf import: acquire from the foreign producer /// First use of a DMA-BUF imported directly as the video source: acquire from the foreign
/// (UNDEFINED preserves the modifier-tiled bytes) with a FOREIGN→encode-family transfer. /// producer (UNDEFINED preserves modifier-backed bytes) with a FOREIGN→encode-family transfer.
RgbFresh, DmabufFresh,
/// RGB-direct, cached import: the image is already VIDEO_ENCODE_SRC; visibility-only /// Cached direct-source import: already VIDEO_ENCODE_SRC; visibility-only barrier for the
/// barrier for the producer's out-of-band rewrite of the bytes. /// producer's out-of-band rewrite of the bytes.
RgbCached, DmabufCached,
/// RGB-direct CPU upload: the compute queue copied the staging buffer in (semaphore /// RGB-direct CPU upload: the compute queue copied the staging buffer in (semaphore
/// ordered); transition TRANSFER_DST → VIDEO_ENCODE_SRC. /// ordered); transition TRANSFER_DST → VIDEO_ENCODE_SRC.
CpuUpload, CpuUpload,
@@ -376,18 +440,33 @@ pub struct VulkanVideoEncoder {
/// `ENCODE_QUALITY_LEVEL` control and baked into the session-parameters object (the spec /// `ENCODE_QUALITY_LEVEL` control and baked into the session-parameters object (the spec
/// requires the two to match). /// requires the two to match).
quality_level: u32, quality_level: u32,
/// `PUNKTFUNK_PERF` CSC/encode split: >0 ⇒ per-frame GPU timestamps are recorded around the /// `PUNKTFUNK_PERF` pre-encode split: >0 ⇒ per-frame GPU timestamps bracket either the
/// compute batch and a sampled `csc_us` line is logged; the value is the device timestamp /// RGB→NV12 compute batch or the native-NV12 padded copy. The measured duration is logged
/// period in ns/tick. 0.0 ⇒ off (env unset, or the compute family has no timestamp support). /// separately from the host's fence wait; 0.0 means disabled or unsupported.
ts_period_ns: f64, ts_period_ns: f64,
perf_at: std::time::Instant, // last sampled csc_us log (2 s cadence) perf_at: std::time::Instant,
/// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames /// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames
/// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does /// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does
/// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked /// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked
/// into the video session). /// into the video session).
rgb: Option<RgbDirect>, rgb: Option<RgbDirect>,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct session (EFC cannot /// Producer supplied native NV12 rather than packed RGB. EVERY mode encodes the imported
/// composite it — the cursor will be missing from the stream until the CSC path is used). /// visible-size buffer directly — safely, because native sessions use TRUE-SIZE headers:
/// the SPS/sequence header is authored at the render size and every picture resource's
/// `codedExtent` matches it, so RADV programs the VCN with `session_init` extent = the true
/// size and a nonzero `padding_width/height`, and the FIRMWARE edge-extends the alignment
/// padding internally (radv_video_enc.c `radv_enc_session_init`; the driver also rounds the
/// bitstream SPS up itself and compensates with a conformance window —
/// `radv_video_patch_encode_session_parameters`, per the VK_KHR_video_encode_h265 proposal's
/// "implementations may override" clause). The source is never read past its extent — unlike
/// the app-aligned-SPS convention the CSC/RGB paths use, where the coded extent is 64x16-
/// aligned and an undersized direct source is the OOB-read class behind the 2026-07-20 field
/// GPU reset (those paths keep their aligned-size sources/staging).
native_nv12: bool,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct or native-NV12 session
/// (neither has a compositing stage — the cursor will be missing from the stream until the
/// CSC path is used).
warned_cursor: bool, warned_cursor: bool,
/// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video /// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video
/// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it /// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it
@@ -422,14 +501,24 @@ impl VulkanVideoEncoder {
/// (B2). `PUNKTFUNK_VULKAN_RGB_DIRECT` overrides both ways (see [`rgb_request`]). /// (B2). `PUNKTFUNK_VULKAN_RGB_DIRECT` overrides both ways (see [`rgb_request`]).
pub fn open( pub fn open(
codec: Codec, codec: Codec,
format: PixelFormat,
width: u32, width: u32,
height: u32, height: u32,
fps: u32, fps: u32,
bitrate_bps: u64, bitrate_bps: u64,
cursor_blend: bool, cursor_blend: bool,
) -> Result<Self> { ) -> Result<Self> {
let want_rgb = rgb_request().unwrap_or(!cursor_blend); let native_nv12 = format == PixelFormat::Nv12;
Self::open_opts(codec, width, height, fps, bitrate_bps, want_rgb) let want_rgb = !native_nv12 && rgb_request().unwrap_or(!cursor_blend);
Self::open_opts_inner(
codec,
width,
height,
fps,
bitrate_bps,
want_rgb,
native_nv12,
)
} }
/// `open` with the RGB-direct request explicit instead of read from the env — the smoke /// `open` with the RGB-direct request explicit instead of read from the env — the smoke
@@ -443,6 +532,18 @@ impl VulkanVideoEncoder {
fps: u32, fps: u32,
bitrate_bps: u64, bitrate_bps: u64,
want_rgb: bool, want_rgb: bool,
) -> Result<Self> {
Self::open_opts_inner(codec, width, height, fps, bitrate_bps, want_rgb, false)
}
fn open_opts_inner(
codec: Codec,
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
want_rgb: bool,
native_nv12: bool,
) -> Result<Self> { ) -> Result<Self> {
if !matches!(codec, Codec::H265 | Codec::Av1) { if !matches!(codec, Codec::H265 | Codec::Av1) {
bail!("vulkan-encode backend supports HEVC + AV1 only (got {codec:?})"); bail!("vulkan-encode backend supports HEVC + AV1 only (got {codec:?})");
@@ -464,6 +565,7 @@ impl VulkanVideoEncoder {
fps.max(1), fps.max(1),
bitrate_bps.max(1_000_000), bitrate_bps.max(1_000_000),
want_rgb, want_rgb,
native_nv12,
) )
} }
} }
@@ -478,6 +580,7 @@ impl VulkanVideoEncoder {
fps: u32, fps: u32,
bitrate: u64, bitrate: u64,
want_rgb: bool, want_rgb: bool,
native_nv12: bool,
) -> Result<Self> { ) -> Result<Self> {
use super::vk_av1_encode as av1b; use super::vk_av1_encode as av1b;
use super::vk_valve_rgb as vrgb; use super::vk_valve_rgb as vrgb;
@@ -551,33 +654,50 @@ impl VulkanVideoEncoder {
0.0 0.0
}; };
// RGB-direct (EFC) resolution — BEFORE the profile is built, because an active rgb // Resolve the encode source before building the profile: EFC RGB conversion changes
// session changes the profile identity (the rgb-conversion struct rides the chain) and // profile identity; producer-native NV12 uses the ordinary 4:2:0 profile.
// the session's picture format. The probe runs unconditionally: its verdict is the
// field telemetry that decides where B2 can default this on.
let rgb_probe = probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1);
// ALIGNMENT GATE (field GPU-hang, 2026-07-20): the coded extent is 64x16-aligned but the
// captured dmabuf is only the REAL mode size — handing it to the encoder as the direct
// source makes the VCN's EFC read the alignment-padding rows PAST the end of the buffer.
// At 1920x1080 (coded 1088) that is 8 rows = 61 KB of out-of-bounds reads per frame:
// deterministic VM protection faults, vcn_enc ring timeouts, and — through the stall
// watchdog's rebuild-and-refault loop — a full MODE1 GPU reset with VRAM loss. The CSC
// shader absorbs the padding by clamping reads and duplicating the edge; RGB-direct has
// no such stage. Mode select: an aligned mode (720p/1440p/4K) encodes the imported
// buffer directly (true zero-copy); an unaligned one (1080p!) goes through the
// padded-copy staging image (see [`RgbDirect::padded`]) — transfer-only, still no
// compute CSC.
let aligned = rw == w && rh == h; let aligned = rw == w && rh == h;
let rgb_probe = if native_nv12 {
Err("not-probed(native NV12 source selected)")
} else {
probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1)
};
let rgb_cfg: Option<RgbDirect> = match (&rgb_probe, want_rgb) { let rgb_cfg: Option<RgbDirect> = match (&rgb_probe, want_rgb) {
(Ok((x, y)), true) => Some(RgbDirect { (Ok((x, y)), true) => {
let true_extent = !aligned && rgb_true_extent_request();
Some(RgbDirect {
x_offset: *x, x_offset: *x,
y_offset: *y, y_offset: *y,
padded: !aligned, padded: !aligned && !true_extent,
}), true_extent,
})
}
_ => None, _ => None,
}; };
if native_nv12 {
tracing::info!(
native_nv12 = "active(direct-import)",
source_width = rw,
source_height = rh,
fw_padding_width = w - rw,
fw_padding_height = h - rh,
"vulkan-encode: producer-native NV12 encode source (true-size headers: the \
driver aligns the bitstream SPS itself and the firmware edge-extends the \
padding the source is never read past its extent)"
);
} else {
tracing::info!( tracing::info!(
rgb_direct = match (&rgb_probe, want_rgb, &rgb_cfg) { rgb_direct = match (&rgb_probe, want_rgb, &rgb_cfg) {
(
_,
_,
Some(RgbDirect {
true_extent: true, ..
}),
) =>
"active(true-extent: unaligned mode, direct import with the true-size \
source codedExtent RADV firmware padding covers the alignment rows; \
PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded copy)",
(_, _, Some(RgbDirect { padded: false, .. })) => "active", (_, _, Some(RgbDirect { padded: false, .. })) => "active",
(_, _, Some(RgbDirect { padded: true, .. })) => (_, _, Some(RgbDirect { padded: true, .. })) =>
"active(padded-copy: mode is not 64x16-aligned — staging blit + edge \ "active(padded-copy: mode is not 64x16-aligned — staging blit + edge \
@@ -586,11 +706,11 @@ impl VulkanVideoEncoder {
"available(off: PUNKTFUNK_VULKAN_RGB_DIRECT=0, or a cursor-blend session \ "available(off: PUNKTFUNK_VULKAN_RGB_DIRECT=0, or a cursor-blend session \
=1 forces)", =1 forces)",
(Err(e), _, None) => e, (Err(e), _, None) => e,
// (Ok, wanted) always builds Some above.
(Ok(_), true, None) => unreachable!("rgb gate and cfg disagree"), (Ok(_), true, None) => unreachable!("rgb gate and cfg disagree"),
}, },
"vulkan-encode: EFC RGB-direct encode source (design/vulkan-rgb-direct-encode.md)" "vulkan-encode: EFC RGB-direct encode source (design/vulkan-rgb-direct-encode.md)"
); );
}
// the encode profile — H265 Main, or AV1 Main; chained raw and uniformly (vendored AV1 + // the encode profile — H265 Main, or AV1 Main; chained raw and uniformly (vendored AV1 +
// rgb structs can't `push_next`, and the chain must match [`RgbProfileStack::wire`]'s // rgb structs can't `push_next`, and the chain must match [`RgbProfileStack::wire`]'s
@@ -820,13 +940,19 @@ impl VulkanVideoEncoder {
// ---- session parameters + header framing (HEVC: VPS/SPS/PPS on keyframes; AV1: a // ---- session parameters + header framing (HEVC: VPS/SPS/PPS on keyframes; AV1: a
// temporal-delimiter OBU per frame + a sequence-header OBU on keyframes) ---- // temporal-delimiter OBU per frame + a sequence-header OBU on keyframes) ----
// Native NV12 authors TRUE-SIZE headers (SPS/seq at the render size, no app-side
// conformance window): RADV rounds the bitstream SPS up itself and, keyed off the
// matching true-size codedExtent, programs the VCN with firmware padding so the source
// is never read past its extent. The CSC/RGB paths keep the app-aligned convention
// (their sources genuinely cover the aligned extent).
let (hdr_w, hdr_h) = if native_nv12 { (rw, rh) } else { (w, h) };
let (params, header, frame_prefix) = if av1 { let (params, header, frame_prefix) = if av1 {
build_parameters_av1( build_parameters_av1(
&device, &device,
&vq_dev, &vq_dev,
session, session,
w, hdr_w,
h, hdr_h,
rw, rw,
rh, rh,
av1_caps.max_level, av1_caps.max_level,
@@ -839,8 +965,8 @@ impl VulkanVideoEncoder {
&vq_dev, &vq_dev,
&venc_dev, &venc_dev,
session, session,
w, hdr_w,
h, hdr_h,
rw, rw,
rh, rh,
quality_level, quality_level,
@@ -996,9 +1122,14 @@ impl VulkanVideoEncoder {
compute_pool, compute_pool,
bs_size, bs_size,
sampler, sampler,
ts_period_ns > 0.0 && rgb_cfg.is_none(), ts_period_ns > 0.0
rgb_cfg.is_none(), && ((rgb_cfg.is_none() && !native_nv12)
rgb_cfg.as_ref().is_some_and(|c| c.padded), || rgb_cfg.as_ref().is_some_and(|c| c.padded)),
rgb_cfg.is_none() && !native_nv12,
rgb_cfg
.as_ref()
.is_some_and(|c| c.padded)
.then_some(vk::Format::B8G8R8A8_UNORM),
guard.frames.last_mut().expect("frame just pushed"), guard.frames.last_mut().expect("frame just pushed"),
)?; )?;
} }
@@ -1052,6 +1183,7 @@ impl VulkanVideoEncoder {
ts_period_ns, ts_period_ns,
perf_at: std::time::Instant::now(), perf_at: std::time::Instant::now(),
rgb: rgb_cfg, rgb: rgb_cfg,
native_nv12,
warned_cursor: false, warned_cursor: false,
pending_bitrate: None, pending_bitrate: None,
width: w, width: w,
@@ -1200,15 +1332,31 @@ impl VulkanVideoEncoder {
} }
} }
/// Import a packed-RGB dmabuf as a VkImage (explicit DRM modifier). CSC sessions import it /// Import a DMA-BUF VkImage with usage/profile matching this session's source mode. Native
/// SAMPLED (the compute shader reads it); RGB-direct sessions import it as a profiled /// NV12 and aligned RGB-direct are profiled `VIDEO_ENCODE_SRC` images. Padded RGB-direct
/// `VIDEO_ENCODE_SRC` — the buffer IS the encode source. Caller destroys. /// imports the producer allocation as transfer-source only.
unsafe fn import_dmabuf( unsafe fn import_dmabuf(
&self, &self,
d: &pf_frame::DmabufFrame, d: &pf_frame::DmabufFrame,
cw: u32, cw: u32,
ch: u32, ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> { ) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
if self.native_nv12 {
let mut ps = NativeProfileStack::new(codec_op_for(self.codec == Codec::Av1));
let profile = *ps.wire(self.codec == Codec::Av1);
let arr = [profile];
let mut plist = vk::VideoProfileListInfoKHR::default().profiles(&arr);
return super::vk_util::import_rgb_dmabuf_as(
&self.device,
&self.ext_fd,
&self.mem_props,
d,
cw,
ch,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR,
Some(&mut plist),
);
}
if self.rgb.as_ref().is_some_and(|r| r.padded) { if self.rgb.as_ref().is_some_and(|r| r.padded) {
// Padded-copy mode: the import is only ever a transfer SOURCE (the blit into the // Padded-copy mode: the import is only ever a transfer SOURCE (the blit into the
// aligned staging image) — plain TRANSFER_SRC, no video profile involved. // aligned staging image) — plain TRANSFER_SRC, no video profile involved.
@@ -1503,8 +1651,21 @@ impl VulkanVideoEncoder {
setup_idx = (setup_idx + 1) % DPB_SLOTS as usize; setup_idx = (setup_idx + 1) % DPB_SLOTS as usize;
} }
// ---- 2..4 diverge by encode source; the RGB-direct twin returns through the shared // ---- 2..4 diverge by encode source; native NV12 and RGB-direct return through the
// bookkeeping tail (design/vulkan-rgb-direct-encode.md B1) ---- // shared bookkeeping tail ----
if self.native_nv12 {
self.record_submit_nv12(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping(
slot,
frame.pts_ns,
wire,
is_idr,
recovery,
setup_idx,
poc,
);
return Ok(());
}
if self.rgb.is_some() { if self.rgb.is_some() {
self.record_submit_rgb(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?; self.record_submit_rgb(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping( self.post_submit_bookkeeping(
@@ -1831,12 +1992,20 @@ impl VulkanVideoEncoder {
Ok(()) Ok(())
} }
/// Padded-copy blit (unaligned-mode RGB-direct): record — into `compute_cmd` — the visible /// Padded-copy blit (unaligned-mode RGB-direct or native NV12): record — into `compute_cmd`
/// frame copy from the imported capture image into the aligned staging image, plus the edge /// — the visible frame copy from the imported capture image into the aligned staging image,
/// duplication into the 64x16 padding (the same edge semantics the CSC shader implements /// plus the edge duplication into the 64x16 padding (the same edge semantics the CSC shader
/// with clamped reads). Transfer-only, no shader. The staging image lives in GENERAL — it /// implements with clamped reads). Transfer-only, no shader. The staging image lives in
/// is both copy dst and, for the right-column pass, copy src — and the encode acquires it /// GENERAL — it is both copy dst and, for the right-column pass, copy src — and the encode
/// via [`SrcAcquire::CscGeneral`] (content produced on the compute queue, csc_sem-ordered). /// acquires it via [`SrcAcquire::CscGeneral`] (content produced on the compute queue,
/// csc_sem-ordered).
///
/// `planes` lists the copy aspects with their subsampling divisor — `[(COLOR, 1)]` for
/// packed RGB, `[(PLANE_0, 1), (PLANE_1, 2)]` for NV12 (multi-planar copy regions are in
/// each plane's own coordinate space; barriers on non-disjoint images stay COLOR-aspect).
/// Every divisor must divide the visible and aligned extents (4:2:0 frames are even, the
/// coded extent is 64x16-aligned).
#[allow(clippy::too_many_arguments)]
unsafe fn record_pad_blit( unsafe fn record_pad_blit(
&self, &self,
dev: &ash::Device, dev: &ash::Device,
@@ -1844,6 +2013,8 @@ impl VulkanVideoEncoder {
src: vk::Image, src: vk::Image,
src_fresh: bool, src_fresh: bool,
pad: vk::Image, pad: vk::Image,
ts_pool: vk::QueryPool,
planes: &[(vk::ImageAspectFlags, u32)],
) -> Result<()> { ) -> Result<()> {
let (rw, rh) = (self.render_w, self.render_h); let (rw, rh) = (self.render_w, self.render_h);
let (w, h) = (self.width, self.height); let (w, h) = (self.width, self.height);
@@ -1852,6 +2023,10 @@ impl VulkanVideoEncoder {
&vk::CommandBufferBeginInfo::default() &vk::CommandBufferBeginInfo::default()
.flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT), .flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
)?; )?;
if self.ts_period_ns > 0.0 {
dev.cmd_reset_query_pool(compute_cmd, ts_pool, 0, 2);
dev.cmd_write_timestamp2(compute_cmd, vk::PipelineStageFlags2::NONE, ts_pool, 0);
}
// Acquire the imported capture buffer for transfer reads (FOREIGN hand-off on first // Acquire the imported capture buffer for transfer reads (FOREIGN hand-off on first
// import — UNDEFINED preserves the modifier-tiled bytes — visibility-only afterwards), // import — UNDEFINED preserves the modifier-tiled bytes — visibility-only afterwards),
// and the staging image for transfer writes (prior contents discarded). // and the staging image for transfer writes (prior contents discarded).
@@ -1893,10 +2068,11 @@ impl VulkanVideoEncoder {
compute_cmd, compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[src_acq, pad_dst]), &vk::DependencyInfo::default().image_memory_barriers(&[src_acq, pad_dst]),
); );
let region =
|aspect: vk::ImageAspectFlags, sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| {
let layers = vk::ImageSubresourceLayers::default() let layers = vk::ImageSubresourceLayers::default()
.aspect_mask(vk::ImageAspectFlags::COLOR) .aspect_mask(aspect)
.layer_count(1); .layer_count(1);
let region = |sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| {
vk::ImageCopy::default() vk::ImageCopy::default()
.src_subresource(layers) .src_subresource(layers)
.dst_subresource(layers) .dst_subresource(layers)
@@ -1908,11 +2084,16 @@ impl VulkanVideoEncoder {
depth: 1, depth: 1,
}) })
}; };
// Pass 1 — from the capture: the visible region, then each bottom padding row as a // Pass 1 — from the capture, per plane: the visible region, then each bottom padding row
// copy of the last visible row (1080p: 8 such rows). One call, disjoint regions. // as a copy of the last visible row (1080p: 8 luma + 4 chroma rows). One call, disjoint
let mut regions = vec![region(0, 0, 0, 0, rw, rh)]; // regions.
let mut regions = Vec::new();
for &(aspect, div) in planes {
let (rw, rh, h) = (rw / div, rh / div, h / div);
regions.push(region(aspect, 0, 0, 0, 0, rw, rh));
for y in rh..h { for y in rh..h {
regions.push(region(0, rh as i32 - 1, 0, y as i32, rw, 1)); regions.push(region(aspect, 0, rh as i32 - 1, 0, y as i32, rw, 1));
}
} }
dev.cmd_copy_image( dev.cmd_copy_image(
compute_cmd, compute_cmd,
@@ -1942,9 +2123,13 @@ impl VulkanVideoEncoder {
compute_cmd, compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[self_dep]), &vk::DependencyInfo::default().image_memory_barriers(&[self_dep]),
); );
let cols: Vec<vk::ImageCopy> = (rw..w) let mut cols = Vec::new();
.map(|x| region(rw as i32 - 1, 0, x as i32, 0, 1, h)) for &(aspect, div) in planes {
.collect(); let (rw, w, h) = (rw / div, w / div, h / div);
for x in rw..w {
cols.push(region(aspect, rw as i32 - 1, 0, x as i32, 0, 1, h));
}
}
dev.cmd_copy_image( dev.cmd_copy_image(
compute_cmd, compute_cmd,
pad, pad,
@@ -1954,10 +2139,112 @@ impl VulkanVideoEncoder {
&cols, &cols,
); );
} }
if self.ts_period_ns > 0.0 {
dev.cmd_write_timestamp2(
compute_cmd,
vk::PipelineStageFlags2::ALL_COMMANDS,
ts_pool,
1,
);
}
dev.end_command_buffer(compute_cmd)?; dev.end_command_buffer(compute_cmd)?;
Ok(()) Ok(())
} }
/// Producer-native NV12 submit: import the producer's visible-size buffer directly as the
/// encode source. Safe at every mode because native sessions run true-size headers — the
/// picture resources' codedExtent equals the source extent and the VCN firmware edge-extends
/// the alignment padding internally (see the [`Self::native_nv12`] field docs).
#[allow(clippy::too_many_arguments)]
unsafe fn record_submit_nv12(
&mut self,
slot: usize,
frame: &CapturedFrame,
is_idr: bool,
recovery: bool,
ref_slot: usize,
setup_idx: usize,
poc: i32,
) -> Result<()> {
if frame.format != PixelFormat::Nv12 {
bail!(
"vulkan-encode (native NV12): negotiated NV12 but received {:?}",
frame.format
);
}
if frame.width != self.render_w || frame.height != self.render_h {
bail!(
"vulkan-encode (native NV12): frame {}x{} != mode {}x{}",
frame.width,
frame.height,
self.render_w,
self.render_h
);
}
if frame.width % 2 != 0 || frame.height % 2 != 0 {
bail!("vulkan-encode (native NV12): 4:2:0 frame dimensions must be even");
}
let FramePayload::Dmabuf(d) = &frame.payload else {
bail!("vulkan-encode (native NV12): producer frame is not a DMA-BUF");
};
if d.fourcc != pf_frame::drm_fourcc(PixelFormat::Nv12).expect("NV12 FourCC") {
bail!(
"vulkan-encode (native NV12): DMA-BUF FourCC {:#x} is not NV12",
d.fourcc
);
}
if d.modifier != 0 {
bail!(
"vulkan-encode (native NV12): only LINEAR is supported, got modifier {:#x}",
d.modifier
);
}
// No compositing stage exists here (like RGB-direct/EFC): gamescope embeds its pointer
// in the produced pixels, but any other NV12 producer's metadata cursor would be lost —
// say so once instead of silently.
if frame.cursor.is_some() && !self.warned_cursor {
self.warned_cursor = true;
tracing::warn!(
"cursor bitmap on a native-NV12 session — nothing composites it; the cursor \
will be missing from the stream (unset PUNKTFUNK_PIPEWIRE_NV12 for \
metadata-cursor captures)"
);
}
let dev = self.device.clone();
let cmd = self.frames[slot].cmd;
let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf;
// The frame-size check above proved the buffer covers the (true-size) coded extent —
// the direct import is the encode source at every mode.
let (src_img, src_view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acquire = if fresh {
SrcAcquire::DmabufFresh
} else {
SrcAcquire::DmabufCached
};
if self.codec == Codec::Av1 {
self.record_coding_av1(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, recovery,
ref_slot, setup_idx, poc,
)?;
} else {
self.record_coding_h265(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, ref_slot,
setup_idx, poc,
)?;
}
dev.reset_fences(&[fence])?;
// The whole frame is one submit: the encoder reads the imported NV12 directly.
let ecmds = [cmd];
dev.queue_submit(
self.encode_queue,
&[vk::SubmitInfo::default().command_buffers(&ecmds)],
fence,
)?;
Ok(())
}
/// RGB-direct twin of [`record_submit`]'s steps 24 (step 1 and the bookkeeping tail are /// RGB-direct twin of [`record_submit`]'s steps 24 (step 1 and the bookkeeping tail are
/// shared): resolve the RGB encode source — the imported capture dmabuf itself, or the CPU /// shared): resolve the RGB encode source — the imported capture dmabuf itself, or the CPU
/// staging upload — record the encode, and submit. There is no compute CSC: the VCN EFC /// staging upload — record the encode, and submit. There is no compute CSC: the VCN EFC
@@ -1981,6 +2268,7 @@ impl VulkanVideoEncoder {
let fence = self.frames[slot].fence; let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool; let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf; let bs_buf = self.frames[slot].bs_buf;
let ts_pool = self.frames[slot].ts_pool;
// EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so // EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so
// once instead of silently losing the pointer (gamescope, the flagship, embeds it). // once instead of silently losing the pointer (gamescope, the flagship, embeds it).
if frame.cursor.is_some() && !self.warned_cursor { if frame.cursor.is_some() && !self.warned_cursor {
@@ -1995,25 +2283,33 @@ impl VulkanVideoEncoder {
let (src_img, src_view, acquire, compute_active) = match &frame.payload { let (src_img, src_view, acquire, compute_active) = match &frame.payload {
FramePayload::Dmabuf(d) if !padded => { FramePayload::Dmabuf(d) if !padded => {
// Defense in depth for the OOB class the alignment gate closes at open: the // Defense in depth for the OOB class the alignment gate closes at open: the
// imported buffer must cover the FULL coded extent, or the EFC reads past it // imported buffer must cover the source extent the encode declares — the FULL
// (VM faults → VCN ring hang → GPU reset, the 2026-07-20 field report). A // aligned coded extent normally (or the EFC reads past it: VM faults → VCN
// mismatched frame (mid-flight mode change, odd capture) errors out here and // ring hang → GPU reset, the 2026-07-20 field report), the render size in
// takes the encoder-rebuild path instead of faulting the GPU. // true-extent mode (where the declared source codedExtent shrinks with it and
if frame.width != self.width || frame.height != self.height { // RADV's firmware padding covers the alignment rows). A mismatched frame
// (mid-flight mode change, odd capture) errors out here and takes the
// encoder-rebuild path instead of faulting the GPU.
let (need_w, need_h) = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
(self.render_w, self.render_h)
} else {
(self.width, self.height)
};
if frame.width != need_w || frame.height != need_h {
bail!( bail!(
"vulkan-encode (rgb-direct): frame {}x{} does not cover the coded \ "vulkan-encode (rgb-direct): frame {}x{} does not cover the declared \
extent {}x{} refusing an out-of-bounds encode source", source extent {}x{} refusing an out-of-bounds encode source",
frame.width, frame.width,
frame.height, frame.height,
self.width, need_w,
self.height need_h
); );
} }
let (img, view, fresh) = self.import_cached(d, frame.width, frame.height)?; let (img, view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acq = if fresh { let acq = if fresh {
SrcAcquire::RgbFresh SrcAcquire::DmabufFresh
} else { } else {
SrcAcquire::RgbCached SrcAcquire::DmabufCached
}; };
(img, view, acq, false) (img, view, acq, false)
} }
@@ -2036,7 +2332,15 @@ impl VulkanVideoEncoder {
let (img, _view, fresh) = self.import_cached(d, frame.width, frame.height)?; let (img, _view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let pad_img = self.frames[slot].pad_img; let pad_img = self.frames[slot].pad_img;
let pad_view = self.frames[slot].pad_view; let pad_view = self.frames[slot].pad_view;
self.record_pad_blit(&dev, compute_cmd, img, fresh, pad_img)?; self.record_pad_blit(
&dev,
compute_cmd,
img,
fresh,
pad_img,
ts_pool,
&[(vk::ImageAspectFlags::COLOR, 1)],
)?;
// The staging image ends the blit in GENERAL with the csc_sem ordering the // The staging image ends the blit in GENERAL with the csc_sem ordering the
// hand-off — exactly the CscGeneral acquire contract. // hand-off — exactly the CscGeneral acquire contract.
(pad_img, pad_view, SrcAcquire::CscGeneral, true) (pad_img, pad_view, SrcAcquire::CscGeneral, true)
@@ -2240,13 +2544,13 @@ impl VulkanVideoEncoder {
.src_stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS) .src_stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::GENERAL), .old_layout(vk::ImageLayout::GENERAL),
SrcAcquire::RgbFresh => src_base SrcAcquire::DmabufFresh => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE) .src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::UNDEFINED) .old_layout(vk::ImageLayout::UNDEFINED)
.src_queue_family_index(vk::QUEUE_FAMILY_FOREIGN_EXT) .src_queue_family_index(vk::QUEUE_FAMILY_FOREIGN_EXT)
.dst_queue_family_index(self.encode_family), .dst_queue_family_index(self.encode_family),
SrcAcquire::RgbCached => src_base SrcAcquire::DmabufCached => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE) .src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::VIDEO_ENCODE_SRC_KHR), .old_layout(vk::ImageLayout::VIDEO_ENCODE_SRC_KHR),
@@ -2284,9 +2588,29 @@ impl VulkanVideoEncoder {
poc: i32, poc: i32,
) -> Result<()> { ) -> Result<()> {
use ash::vk::native as h; use ash::vk::native as h;
let ext2d = vk::Extent2D { // Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
// pairs with their aligned SPS), the render size for native NV12's true-size headers.
let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width, width: self.width,
height: self.height, height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
}; };
let ref_poc = if is_idr { 0 } else { self.slot_poc[ref_slot] }; let ref_poc = if is_idr { 0 } else { self.slot_poc[ref_slot] };
@@ -2458,7 +2782,7 @@ impl VulkanVideoEncoder {
} }
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty()); dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default() let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d) .coded_extent(src_extent)
.image_view_binding(src_view); .image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default() let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf) .dst_buffer(bs_buf)
@@ -2502,9 +2826,29 @@ impl VulkanVideoEncoder {
) -> Result<()> { ) -> Result<()> {
use super::vk_av1_encode as av1; use super::vk_av1_encode as av1;
use ash::vk::native as h; use ash::vk::native as h;
let ext2d = vk::Extent2D { // Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
// pairs with their aligned SPS), the render size for native NV12's true-size headers.
let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width, width: self.width,
height: self.height, height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
}; };
// ---- required AV1 frame sub-structs (single tile; no CDEF/LR/segmentation/global-motion) ---- // ---- required AV1 frame sub-structs (single tile; no CDEF/LR/segmentation/global-motion) ----
@@ -2726,7 +3070,7 @@ impl VulkanVideoEncoder {
} }
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty()); dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default() let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d) .coded_extent(src_extent)
.image_view_binding(src_view); .image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default() let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf) .dst_buffer(bs_buf)
@@ -2784,10 +3128,13 @@ impl VulkanVideoEncoder {
); );
} }
let (off, len) = (off64 as usize, len64 as usize); let (off, len) = (off64 as usize, len64 as usize);
// PUNKTFUNK_PERF CSC split (best-effort): the fence signaled, so the compute batch that // PUNKTFUNK_PERF pre-encode split (best-effort): the fence signaled, so the compute/transfer
// wrote these timestamps completed long ago — WAIT is a formality. Sampled to one log // timestamps are available. This is a device duration, not permission to label the
// line per ~2 s; `wait_us` in the pump's stage perf minus this ≈ the ASIC encode. // remaining host fence wait as pure VCN time; queueing and synchronization remain in it.
if self.ts_period_ns > 0.0 && f.ts_pool != vk::QueryPool::null() { if self.ts_period_ns > 0.0
&& f.ts_pool != vk::QueryPool::null()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{
let mut ts = [0u64; 2]; let mut ts = [0u64; 2];
if dev if dev
.get_query_pool_results( .get_query_pool_results(
@@ -2797,16 +3144,26 @@ impl VulkanVideoEncoder {
vk::QueryResultFlags::TYPE_64 | vk::QueryResultFlags::WAIT, vk::QueryResultFlags::TYPE_64 | vk::QueryResultFlags::WAIT,
) )
.is_ok() .is_ok()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{ {
self.perf_at = std::time::Instant::now(); self.perf_at = std::time::Instant::now();
let csc_us = (ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0; let pre_encode_us =
(ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0;
if self.rgb.as_ref().is_some_and(|r| r.padded) {
tracing::info!( tracing::info!(
csc_us = format!("{csc_us:.0}"), rgb_copy_us = format!("{pre_encode_us:.0}"),
au_bytes = len, au_bytes = len,
"vulkan-encode split (sampled): csc=GPU compute batch (import barriers + \ "vulkan-encode split (sampled): padded RGB copy device time before EFC; \
CSC + plane copies); ASIC encode stage-perf wait_us csc" remaining fence wait still includes queue synchronization + RGBYUV EFC \
+ video encode"
); );
} else {
tracing::info!(
csc_us = format!("{pre_encode_us:.0}"),
au_bytes = len,
"vulkan-encode split (sampled): RGB→NV12 compute batch device time; \
remaining fence wait still includes queue synchronization + video encode"
);
}
} }
} }
let f = &self.frames[slot]; let f = &self.frames[slot];
@@ -3412,26 +3769,30 @@ unsafe fn make_frame(
sampler: vk::Sampler, sampler: vk::Sampler,
with_ts: bool, with_ts: bool,
csc: bool, csc: bool,
rgb_pad: bool, pad_fmt: Option<vk::Format>,
f: &mut Frame, f: &mut Frame,
) -> Result<()> { ) -> Result<()> {
// "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`). // "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`).
f.cursor_serial = u64::MAX; f.cursor_serial = u64::MAX;
// Padded-copy RGB staging (unaligned-mode RGB-direct): aligned BGRA encode-src filled by a // Padded-copy staging (unaligned-mode RGB-direct or native NV12): an aligned encode-src in
// transfer blit each frame — concurrent compute (copy) + encode (source read). // the session's picture format, filled by a transfer blit each frame — concurrent compute
if rgb_pad { // (copy) + encode (source read). TRANSFER_SRC because the width-padding pass self-copies the
// staging image's own last visible column (see `record_pad_blit`).
if let Some(fmt) = pad_fmt {
(f.pad_img, f.pad_mem) = make_video_image( (f.pad_img, f.pad_mem) = make_video_image(
device, device,
mem_props, mem_props,
vk::Format::B8G8R8A8_UNORM, fmt,
w, w,
h, h,
1, 1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR | vk::ImageUsageFlags::TRANSFER_DST, vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR
| vk::ImageUsageFlags::TRANSFER_DST
| vk::ImageUsageFlags::TRANSFER_SRC,
profile_list, profile_list,
fams, fams,
)?; )?;
f.pad_view = make_view(device, f.pad_img, vk::Format::B8G8R8A8_UNORM, 0)?; f.pad_view = make_view(device, f.pad_img, fmt, 0)?;
} }
// RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no // RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no
// cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the // cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the
+257 -5
View File
@@ -478,10 +478,14 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b b
}); });
// HDR signalling (10-bit sessions are the HDR path on Windows — same coupling as NVENC): // Colour signalling, written UNCONDITIONALLY (mirrors nvenc_core.rs): the input is already
// BT.2020/PQ colour description + the source's mastering/CLL grade at every IDR. // CSC'd to a specific matrix — BT.709 limited for SDR (the capture-side VideoConverter),
// BT.2020 PQ for HDR (HdrP010Converter) — so the stream must say so. An SDR stream without a
// colour description leaves the choice to the decoder's "unspecified" default, and
// Moonlight/third-party/Android-vendor decoders default to 601 at sub-HD → mis-rendered
// colours. (10-bit sessions are the HDR path on Windows — same coupling as NVENC.)
let hdr = cfg.ten_bit && cfg.codec != Codec::H264; let hdr = cfg.ten_bit && cfg.codec != Codec::H264;
let vsi = hdr.then(|| { let vsi = {
// SAFETY: all-zero is valid; header stamped below. // SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() }); let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() });
b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32; b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32;
@@ -489,11 +493,17 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b.VideoFormat = 5; // unspecified b.VideoFormat = 5; // unspecified
b.VideoFullRange = 0; b.VideoFullRange = 0;
b.ColourDescriptionPresent = 1; b.ColourDescriptionPresent = 1;
if hdr {
b.ColourPrimaries = 9; // BT.2020 b.ColourPrimaries = 9; // BT.2020
b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ) b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ)
b.MatrixCoefficients = 9; // BT.2020 non-constant b.MatrixCoefficients = 9; // BT.2020 non-constant
b } else {
}); b.ColourPrimaries = 1; // BT.709
b.TransferCharacteristics = 1; // BT.709
b.MatrixCoefficients = 1; // BT.709
}
Some(b)
};
let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| { let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| {
// SAFETY: all-zero is valid; header stamped below. // SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> = let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> =
@@ -1994,4 +2004,246 @@ mod tests {
"the bitrate retarget emitted a keyframe (StartNewSequence leak)" "the bitrate retarget emitted a keyframe (StartNewSequence leak)"
); );
} }
/// FULL-CHAIN colour check at the field capture size: a known P010 colour-bar source at
/// 1920x1080 — whose height is NOT 16-aligned, so the ingest `CopySubresourceRegion` copies
/// into a 1920x1088 runtime pool surface whose chroma plane sits at a DIFFERENT row offset
/// than the source's (the seam no 640x480 test exercises) — encoded to Main10 HEVC and
/// dumped to `%TEMP%\pf_qsv_1080_bars.h265` for off-box decode verification against the
/// same codes. On-box this asserts stream shape; the pixel verdict needs a decoder.
#[test]
fn qsv_live_p010_1080_colorbars_dump() {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE,
D3D11_SDK_VERSION, D3D11_SUBRESOURCE_DATA, D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_P010, DXGI_SAMPLE_DESC};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
// (Y, Cb, Cr) 10-bit limited codes for the 8 sRGB bars white/yellow/cyan/green/magenta/
// red/blue/black at 80-nit SDR white under PQ/BT.2020 — the same math as pf-capture's
// `p010_reference` (and the bars_pq2020 client fixture). Stored MSB-aligned (`<<6`).
const BARS: [(u16, u16, u16); 8] = [
(490, 512, 512),
(478, 423, 518),
(464, 525, 473),
(450, 432, 476),
(350, 584, 585),
(325, 448, 598),
(226, 650, 535),
(64, 512, 512),
];
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
// P010 initial data: plane 0 = H rows of W u16 luma; plane 1 = H/2 rows of W u16
// (interleaved Cb,Cr pairs), same pitch. Bars are vertical: bar index = x / (W/8).
let bar_w = (W / 8) as usize;
let mut init = vec![0u16; (W as usize) * (H as usize + H as usize / 2)];
for y in 0..H as usize {
for x in 0..W as usize {
init[y * W as usize + x] = BARS[(x / bar_w).min(7)].0 << 6;
}
}
let chroma_base = (W as usize) * (H as usize);
for cy in 0..(H as usize / 2) {
for cx in 0..(W as usize / 2) {
let (_, cb, cr) = BARS[((cx * 2) / bar_w).min(7)];
init[chroma_base + cy * W as usize + cx * 2] = cb << 6;
init[chroma_base + cy * W as usize + cx * 2 + 1] = cr << 6;
}
}
// SAFETY: self-contained harness on one thread/device (same contract as `drive_live`);
// the initial-data pointer outlives the synchronous CreateTexture2D that reads it.
let (device, tex) = unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(imp.luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(imp.luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().expect("dxgi factory");
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).expect("intel adapter");
let mut device = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
windows::Win32::Foundation::HMODULE::default(),
Default::default(),
None,
D3D11_SDK_VERSION,
Some(&mut device),
None,
None,
)
.expect("d3d11 device on intel adapter");
let device: ID3D11Device = device.expect("device");
let desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
CPUAccessFlags: 0,
MiscFlags: 0,
};
let data = D3D11_SUBRESOURCE_DATA {
pSysMem: init.as_ptr() as *const std::ffi::c_void,
SysMemPitch: W * 2,
SysMemSlicePitch: 0,
};
let mut t: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&data), Some(&mut t))
.expect("bar texture");
(device.clone(), t.expect("texture"))
};
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
let mut keyframes = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
assert!(keyframes >= 1, "expected an IDR in the dump");
let path = std::env::temp_dir().join("pf_qsv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {} AUs ({} bytes, {keyframes} keyframes) to {}",
aus,
stream.len(),
path.display()
);
}
/// The PRODUCTION host chain minus the IDD ring: the REAL `HdrP010Converter` renders the 8
/// sRGB bars into a ring-profile P010 texture (`BIND_RENDER_TARGET` only — RTV-written, not
/// CPU-uploaded) on the VPL implementation's own adapter, and THAT texture goes through the
/// unaligned-height ingest copy into a Main10 encode. Dumped to
/// `%TEMP%\pf_qsv_conv_1080_bars.h265`; expected decode codes = the bars_pq2020 fixture set
/// (see `hdr_p010_convert_bars_on_luid`).
#[test]
fn qsv_live_hdr_converter_e2e_1080_dump() {
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
let (device, tex) = pf_capture::dxgi::hdr_p010_convert_bars_on_luid(imp.luid, W, H)
.expect("converter bars");
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
let path = std::env::temp_dir().join("pf_qsv_conv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {aus} AUs ({} bytes) to {}",
stream.len(),
path.display()
);
}
} }
+49
View File
@@ -330,6 +330,7 @@ fn open_video_backend(
{ {
match vulkan_video::VulkanVideoEncoder::open( match vulkan_video::VulkanVideoEncoder::open(
codec, codec,
format,
width, width,
height, height,
fps, fps,
@@ -344,12 +345,32 @@ fn open_video_backend(
); );
return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan")); return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan"));
} }
// Native NV12 (PUNKTFUNK_PIPEWIRE_NV12 capture) has no VAAPI fallback:
// libav's dmabuf lane would import the two-plane buffer as packed RGB
// (silent garbage) and its CPU lane bails per frame — die crisply instead.
Err(e) if format == PixelFormat::Nv12 => {
return Err(e.context(
"Vulkan Video open failed on a native-NV12 capture \
no VAAPI fallback exists; set PUNKTFUNK_PIPEWIRE_NV12=0 to \
restore the packed-RGB negotiation",
));
}
Err(e) => tracing::warn!( Err(e) => tracing::warn!(
error = %format!("{e:#}"), error = %format!("{e:#}"),
"Vulkan Video encode open failed — falling back to libav VAAPI" "Vulkan Video encode open failed — falling back to libav VAAPI"
), ),
} }
} }
// Same rule when the Vulkan backend was never eligible (H264 session,
// PUNKTFUNK_VULKAN_ENCODE=0, or a build without the feature).
if format == PixelFormat::Nv12 {
anyhow::bail!(
"native NV12 capture requires the Vulkan Video encoder (HEVC/AV1 \
session, --features vulkan-encode, PUNKTFUNK_VULKAN_ENCODE not 0) this \
session resolved to libav VAAPI; set PUNKTFUNK_PIPEWIRE_NV12=0 to restore \
the packed-RGB negotiation"
);
}
vaapi::VaapiEncoder::open( vaapi::VaapiEncoder::open(
codec, codec,
format, format,
@@ -390,6 +411,7 @@ fn open_video_backend(
} }
vulkan_video::VulkanVideoEncoder::open( vulkan_video::VulkanVideoEncoder::open(
codec, codec,
format,
width, width,
height, height,
fps, fps,
@@ -819,6 +841,33 @@ fn vulkan_encode_enabled() -> bool {
.unwrap_or(true) .unwrap_or(true)
} }
/// Whether THIS session's encoder can ingest a producer-native NV12 capture: only the raw
/// Vulkan Video backend does (libav VAAPI would misread the two-plane buffer as packed RGB —
/// [`open_video`] refuses the combination), so the session's codec must be one it encodes and
/// the backend must be eligible to open. The host facade threads the verdict into the capture
/// negotiation (`OutputFormat::nv12_native` → `ZeroCopyPolicy::native_nv12_session`), which
/// then PREFERS gamescope's producer-side NV12 pod (default-on; `PUNKTFUNK_PIPEWIRE_NV12=0`
/// escapes at the capture gate).
#[cfg(target_os = "linux")]
pub fn linux_native_nv12_ok(codec: Codec) -> bool {
#[cfg(feature = "vulkan-encode")]
{
matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
// NVENC/PyroWave prefs never open the Vulkan Video backend; every other pref
// (auto/vaapi/amd/intel/vulkan) tries it first on AMD/Intel — see [`open_video`].
&& !matches!(
pf_host_config::config().encoder_pref.as_str(),
"nvenc" | "nvidia" | "cuda" | "pyrowave"
)
}
#[cfg(not(feature = "vulkan-encode"))]
{
let _ = codec;
false
}
}
/// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA /// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA
/// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT /// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT
/// create a CUDA context (that would allocate GPU state on every host that merely *might* be /// create a CUDA context (that would allocate GPU state on every host that merely *might* be
+31 -10
View File
@@ -105,13 +105,15 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888 Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888 Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888 Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// Linux native NV12 capture (gamescope PipeWire): one LINEAR dmabuf with contiguous Y then
// interleaved UV, exposed under DRM_FORMAT_NV12.
Nv12 => drm_fourcc_code(b"NV12"),
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020). // The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010 X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010 X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path. // 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on // Rgb10a2/P010 are Windows formats; Yuv444 is OUR convert output, never a capture source.
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format. Rgb | Bgr | Rgb10a2 | P010 | Yuv444 => return None,
Rgb | Bgr | Rgb10a2 | Nv12 | P010 | Yuv444 => return None,
}) })
} }
@@ -144,6 +146,12 @@ pub struct OutputFormat {
/// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the /// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the
/// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture). /// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture).
pub pyrowave: bool, pub pyrowave: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (Linux raw Vulkan Video
/// backend on an H265/AV1 session — see `pf_encode::linux_native_nv12_ok`). The Linux capture
/// negotiation only offers gamescope the NV12 pod when this is set: libav VAAPI (the H264
/// codec's backend, and the fallback family) would misread the two-plane buffer as packed
/// RGB. Always `false` on Windows (the IDD-push capturer owns its own formats).
pub nv12_native: bool,
} }
impl OutputFormat { impl OutputFormat {
@@ -161,6 +169,11 @@ impl OutputFormat {
chroma_444: false, chroma_444: false,
// GameStream never negotiates PyroWave (native punktfunk/1 only). // GameStream never negotiates PyroWave (native punktfunk/1 only).
pyrowave: false, pyrowave: false,
// Conservative: the GameStream + spike paths don't resolve the codec here, and a
// Moonlight client may negotiate H264 (whose VAAPI backend can't ingest NV12) — so
// they never prefer the producer-native NV12 pod. The punktfunk/1 plane opts in via
// `SessionPlan::output_format()`, which knows the codec.
nv12_native: false,
} }
} }
} }
@@ -201,18 +214,26 @@ pub struct CapturedFrame {
pub cursor: Option<CursorOverlay>, pub cursor: Option<CursorOverlay>,
} }
/// A captured frame still living in a single-plane packed-RGB dmabuf (the VAAPI zero-copy path). /// A captured frame still living in a DMA-BUF. Packed RGB uses one plane. Native Linux NV12
/// (gamescope PipeWire) travels in ONE fd: Y starts at `offset`, and the interleaved UV plane
/// lives at `plane1`'s offset/stride when the producer reported them — else at the contiguous
/// fallback `offset + stride * frame_height` with the shared `stride`.
///
/// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be /// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be
/// imported into a VA surface there without the compositor's buffer being closed underneath it. /// imported there without the compositor's buffer being closed underneath it. Content stability
/// (Content stability across the brief import window relies on the compositor's buffer pool depth, /// across the brief import window relies on the compositor's buffer pool depth, like any zero-copy
/// same as any zero-copy capture — the VAAPI importer copies into its own NV12 surface promptly.) /// capture.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub struct DmabufFrame { pub struct DmabufFrame {
pub fd: std::os::fd::OwnedFd, pub fd: std::os::fd::OwnedFd,
/// DRM FourCC of the packed-RGB plane (e.g. `XR24` for BGRx). /// DRM FourCC (`XR24` for BGRx, `NV12` for native 4:2:0).
pub fourcc: u32, pub fourcc: u32,
/// DRM format modifier the compositor allocated (0 = LINEAR). /// DRM format modifier the compositor allocated (0 = LINEAR).
pub modifier: u64, pub modifier: u64,
/// Second-plane `(offset, stride)` within the SAME fd, when the producer reported one (the
/// PipeWire buffer's plane-1 chunk — NV12's interleaved UV). `None` falls back to the
/// contiguous-plane contract above. Always `None` for single-plane packed RGB.
pub plane1: Option<(u32, u32)>,
pub offset: u32, pub offset: u32,
pub stride: u32, pub stride: u32,
} }
@@ -225,8 +246,8 @@ pub enum FramePayload {
/// The dmabuf has already been imported + copied into this owned device buffer. /// The dmabuf has already been imported + copied into this owned device buffer.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
Cuda(pf_zerocopy::DeviceBuffer), Cuda(pf_zerocopy::DeviceBuffer),
/// A raw packed-RGB dmabuf — the AMD/Intel (VAAPI) zero-copy path. The encoder imports it into /// A raw DMA-BUF: packed RGB for the existing GPU CSC paths, or native NV12 from a producer
/// a VA surface and does RGB→NV12 on the GPU video engine (no host CSC, no upload). /// such as gamescope. The encoder imports it without a host copy.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
Dmabuf(DmabufFrame), Dmabuf(DmabufFrame),
/// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame. /// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame.
+17
View File
@@ -63,6 +63,13 @@ pub struct HostConfig {
/// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0. /// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0.
/// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth). /// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool, pub four_four_four: bool,
/// `PUNKTFUNK_CHACHA20` — host policy gate for the negotiated ChaCha20-Poly1305 session
/// cipher (design/chacha20-session-cipher.md). **Default ON** (pure rollout safety — perf-only,
/// both AEADs are full-strength): the host merely *allows* it — a session only seals with
/// ChaCha when the client advertised `VIDEO_CAP_CHACHA20` (set by soft-AES armv7 clients,
/// e.g. webOS TVs, whose GCM decrypt caps at ~100 Mbps); everyone else stays AES-128-GCM.
/// `PUNKTFUNK_CHACHA20=0`/`false`/`off`/`no` disables.
pub chacha20: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation. /// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool, pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic). /// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -147,6 +154,16 @@ impl HostConfig {
) )
}) })
.unwrap_or(true), .unwrap_or(true),
// Default ON, explicit-off grammar (the client's VIDEO_CAP_CHACHA20 bit is the real
// per-session switch; see the field doc).
chacha20: val("PUNKTFUNK_CHACHA20")
.map(|s| {
!matches!(
s.trim().to_ascii_lowercase().as_str(),
"0" | "false" | "off" | "no"
)
})
.unwrap_or(true),
perf: flag("PUNKTFUNK_PERF"), perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"), video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"), compositor: val("PUNKTFUNK_COMPOSITOR"),
+5
View File
@@ -333,6 +333,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// bottom-right corner (the reported bug). The menu/library is keyboard+gamepad-driven // bottom-right corner (the reported bug). The menu/library is keyboard+gamepad-driven
// and consumes no mouse, so nothing wanted these synthetic events anyway. // and consumes no mouse, so nothing wanted these synthetic events anyway.
sdl3::hint::set("SDL_TOUCH_MOUSE_EVENTS", "0"); sdl3::hint::set("SDL_TOUCH_MOUSE_EVENTS", "0");
// The Wayland `app_id` (and X11 WM_CLASS) — compositors match it against
// io.unom.Punktfunk.desktop for the window/taskbar icon. Without it SDL uses a generic
// identity and the session window gets the default-Wayland icon (the Linux analog of
// the AppUserModelID adoption above).
sdl3::hint::set("SDL_APP_ID", "io.unom.Punktfunk");
let sdl = sdl3::init().context("SDL init")?; let sdl = sdl3::init().context("SDL init")?;
let video = sdl.video().context("SDL video")?; let video = sdl.video().context("SDL video")?;
let events = sdl.event().context("SDL events")?; let events = sdl.event().context("SDL events")?;
+9 -1
View File
@@ -30,9 +30,17 @@ impl Presenter {
// switch modes before anything touches this frame. Only where the surface // switch modes before anything touches this frame. Only where the surface
// offers HDR10 — otherwise PQ stays on the SDR swapchain and the CSC shader // offers HDR10 — otherwise PQ stays on the SDR swapchain and the CSC shader
// tonemaps (mode 1). // tonemaps (mode 1).
//
// CPU frames NEVER take the HDR10 surface: software decode uploads swscale RGBA with
// no CSC/tonemap pass, so on a mode-0 swapchain that sRGB-encoded content would be
// composed as PQ — the field-reported psychedelic cyan/magenta picture (reproduced
// 2026-07-21: Fedora-class client, no hw HEVC decode, GNOME/Mesa offering HDR10 even
// on an SDR desktop). On the SDR swapchain the same frames are merely untonemapped
// (washed out) — wrong in the known, benign way until the CPU lane grows a real
// PQ→sRGB pass.
let frame_pq = match &input { let frame_pq = match &input {
FrameInput::Redraw => None, FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f.color.is_pq()), FrameInput::Cpu(_) => Some(false),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => Some(d.color.is_pq()), FrameInput::Dmabuf(d) => Some(d.color.is_pq()),
FrameInput::VkFrame(v) => Some(v.color.is_pq()), FrameInput::VkFrame(v) => Some(v.color.is_pq()),
+9
View File
@@ -617,6 +617,15 @@ pub(super) fn pick_formats(
surface: vk::SurfaceKHR, surface: vk::SurfaceKHR,
colorspace_ext: bool, colorspace_ext: bool,
) -> Result<(vk::SurfaceFormatKHR, Option<vk::SurfaceFormatKHR>)> { ) -> Result<(vk::SurfaceFormatKHR, Option<vk::SurfaceFormatKHR>)> {
// `PUNKTFUNK_HDR10=0` (explicit-off grammar) refuses the HDR10/ST.2084 swapchain outright,
// pinning PQ streams to the shader tonemap on an SDR surface. Two reasons this exists:
// desktop compositors newly offer HDR10 even on SDR desktops (GNOME 48 / Plasma 6 with
// Mesa ≥ 25.1 — a lane that otherwise engages silently), and it is the A/B lever that
// splits "HDR10 passthrough composes wrong" from "the decoded planes are wrong" in the
// field without rebuilding anything.
let colorspace_ext = colorspace_ext
&& !std::env::var("PUNKTFUNK_HDR10")
.is_ok_and(|v| matches!(v.as_str(), "0" | "false" | "off" | "no"));
let formats = unsafe { surface_i.get_physical_device_surface_formats(pdev, surface) }?; let formats = unsafe { surface_i.get_physical_device_surface_formats(pdev, surface) }?;
let mut sdr = None; let mut sdr = None;
for want in [vk::Format::B8G8R8A8_UNORM, vk::Format::R8G8B8A8_UNORM] { for want in [vk::Format::B8G8R8A8_UNORM, vk::Format::R8G8B8A8_UNORM] {
+28
View File
@@ -254,6 +254,34 @@ pub fn detect() -> Result<Compositor> {
} }
} }
/// Attach-only probes: while any scope is held, backend `create` paths must not stop, relaunch,
/// or take over box sessions — they may only attach to an already-live output, and fail fast
/// otherwise. The capture-loss rebuild holds one for its first seconds: right after a capture
/// loss the active-session detection can be STALE (a Game→Desktop switch observed live: the
/// probe's gamescope re-acquire restarted `gamescope-session.target` and yanked the user out of
/// the KDE session they had just switched to). A counter, so overlapping scopes compose.
static REBUILD_PROBES: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
/// RAII scope marking pipeline builds as attach-only probes (see [`rebuild_probe_active`]).
pub struct RebuildProbeScope(());
pub fn rebuild_probe_scope() -> RebuildProbeScope {
REBUILD_PROBES.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
RebuildProbeScope(())
}
impl Drop for RebuildProbeScope {
fn drop(&mut self) {
REBUILD_PROBES.fetch_sub(1, std::sync::atomic::Ordering::SeqCst);
}
}
/// Is any [`rebuild_probe_scope`] active? Destructive session operations (stop/relaunch/
/// takeover-restart) must be skipped while true.
pub fn rebuild_probe_active() -> bool {
REBUILD_PROBES.load(std::sync::atomic::Ordering::SeqCst) > 0
}
/// Open the virtual-display driver for `compositor`. /// Open the virtual-display driver for `compositor`.
pub fn open(compositor: Compositor) -> Result<Box<dyn VirtualDisplay>> { pub fn open(compositor: Compositor) -> Result<Box<dyn VirtualDisplay>> {
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
@@ -325,6 +325,29 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
if steamos_session_present() { if steamos_session_present() {
return create_managed_session_steamos(mode); return create_managed_session_steamos(mode);
} }
// Attach-only rebuild probe: reuse a live same-mode session, but NEVER stop/relaunch box
// sessions — right after a capture loss the caller's session detection can be stale, and a
// destructive rebuild here would fight the session the user just switched to.
if crate::rebuild_probe_active() {
let guard = MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner());
let same_mode = guard.as_ref().is_some_and(|s| {
s.width == mode.width && s.height == mode.height && s.refresh_hz == mode.refresh_hz
});
if same_mode {
if let Some(node_id) = find_gamescope_node() {
point_injector_at_eis();
tracing::info!(
node_id,
"gamescope session: attach-only probe reusing live node"
);
return Ok(managed_output(node_id, mode));
}
}
return Err(anyhow!(
"gamescope session has no attachable live node — attach-only rebuild probe refuses \
to stop/relaunch box sessions (re-detection follows the live session)"
));
}
// Steam is single-instance: if the box autologged into gaming mode on a physical display (the // Steam is single-instance: if the box autologged into gaming mode on a physical display (the
// Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and // Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and
// renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by // renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by
@@ -607,12 +630,17 @@ fn write_steamos_dropin(shim_dir: &std::path::Path, mode: Mode) -> Result<()> {
if let Some(parent) = path.parent() { if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).with_context(|| format!("mkdir {}", parent.display()))?; std::fs::create_dir_all(parent).with_context(|| format!("mkdir {}", parent.display()))?;
} }
// UnsetEnvironment: the same headless-must-not-attach armor `launch_session` gives its
// transient unit — the manager env can carry a stale desktop DISPLAY/WAYLAND_DISPLAY (from a
// portal settle), and gamescope would abort trying to attach to it instead of becoming the
// display server. Unit-scoped belt-and-suspenders on top of the observe_session_instance scrub.
let body = format!( let body = format!(
"[Service]\n\ "[Service]\n\
Environment=PATH={shim}:/usr/bin:/bin:/usr/local/bin\n\ Environment=PATH={shim}:/usr/bin:/bin:/usr/local/bin\n\
Environment=PF_W={w}\n\ Environment=PF_W={w}\n\
Environment=PF_H={h}\n\ Environment=PF_H={h}\n\
Environment=PF_HZ={hz}\n", Environment=PF_HZ={hz}\n\
UnsetEnvironment=DISPLAY WAYLAND_DISPLAY\n",
shim = shim_dir.display(), shim = shim_dir.display(),
w = mode.width, w = mode.width,
h = mode.height, h = mode.height,
@@ -650,6 +678,16 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
} }
*guard = None; // tracked session lost its node — fall through to a clean restart *guard = None; // tracked session lost its node — fall through to a clean restart
} }
// Attach-only rebuild probe: the reuse path above may attach, but a restart of the session
// target is out of bounds — observed live on a Deck: a stale post-capture-loss detection made
// this restart steal the seat back from the KDE session the user had just switched to.
if crate::rebuild_probe_active() {
return Err(anyhow!(
"gamescope has no live node and this is an attach-only rebuild probe — refusing to \
restart {STEAMOS_SESSION_TARGET} (the box may be mid-switch to another session; \
re-detection follows it)"
));
}
let shim_dir = write_headless_shim()?; let shim_dir = write_headless_shim()?;
write_steamos_dropin(&shim_dir, mode)?; write_steamos_dropin(&shim_dir, mode)?;
systemctl_user(&["daemon-reload"]); systemctl_user(&["daemon-reload"]);
@@ -57,6 +57,13 @@ pub fn observe_session_instance(active: &ActiveSession) {
if let Some(old) = compositor_for_kind(prev.0) { if let Some(old) = compositor_for_kind(prev.0) {
registry::invalidate_backend(old.id()); registry::invalidate_backend(old.id());
} }
// The dead desktop's socket vars may still sit in the systemd --user manager env
// ([`settle_desktop_portal`]'s import-environment) — scrub them NOW, or the next
// `gamescope-session.target` start inherits a stale WAYLAND_DISPLAY and gamescope
// runs NESTED against the dead desktop socket instead of becoming the display
// server ("Failed to connect to wayland socket: wayland-0" — kept a Deck's Game
// Mode from starting at all, observed live 2026-07-21).
scrub_desktop_manager_env();
} }
let epoch = bump_session_epoch(); let epoch = bump_session_epoch();
tracing::info!( tracing::info!(
@@ -70,6 +77,23 @@ pub fn observe_session_instance(active: &ActiveSession) {
*last = Some(cur); *last = Some(cur);
} }
/// Counterpart to [`settle_desktop_portal`]'s `import-environment`: drop the desktop session's
/// socket vars from the systemd `--user` manager env once that desktop instance is GONE. They
/// persist in the manager otherwise, and every later user unit inherits them — including
/// `gamescope-session.target`, whose gamescope then aborts trying to attach to the dead desktop
/// socket. Best-effort; the D-Bus activation env has no unset op, but gamescope-session is
/// systemd-started, so the manager scrub is the one that matters. (A desktop restart re-imports
/// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.)
#[cfg(target_os = "linux")]
fn scrub_desktop_manager_env() {
let _ = std::process::Command::new("systemctl")
.args(["--user", "unset-environment", "WAYLAND_DISPLAY", "DISPLAY"])
.status();
}
#[cfg(not(target_os = "linux"))]
fn scrub_desktop_manager_env() {}
/// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die /// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die
/// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are /// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are
/// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]). /// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]).
+5
View File
@@ -33,6 +33,11 @@ reed-solomon-simd = "3.1" # GF(2^16) Leopard-RS, SIMD, O(n log n) — the w
# NOT interoperable.) See vendor/fec-rs/LICENSE (BSD-2-Clause). # NOT interoperable.) See vendor/fec-rs/LICENSE (BSD-2-Clause).
fec-rs = { path = "vendor/fec-rs" } fec-rs = { path = "vendor/fec-rs" }
aes-gcm = "0.10" # AES-128-GCM session crypto, matches GameStream aes-gcm = "0.10" # AES-128-GCM session crypto, matches GameStream
# ChaCha20-Poly1305 session crypto, negotiated by clients without hardware AES (the soft-AES
# armv7 targets — webOS TVs — where GCM caps decrypt at ~100 Mbps; ARX runs 4-7x faster there).
# Same RustCrypto `aead 0.5` generation as aes-gcm: identical trait/nonce/tag shapes, pure Rust,
# cross-compiles like aes-gcm (no cmake). See design/chacha20-session-cipher.md.
chacha20poly1305 = "0.10"
zerocopy = { version = "0.8", features = ["derive"] } zerocopy = { version = "0.8", features = ["derive"] }
bytes = "1" bytes = "1"
socket2 = { version = "0.6", features = [ socket2 = { version = "0.6", features = [
+22 -12
View File
@@ -1,7 +1,8 @@
//! Tier-1 microbenchmarks for the punktfunk/1 hot path — GPU-free, so they run in normal CI. //! Tier-1 microbenchmarks for the punktfunk/1 hot path — GPU-free, so they run in normal CI.
//! //!
//! Two layers: //! Two layers:
//! - `crypto/*` — the isolated AES-128-GCM primitives on one ~MTU shard. //! - `crypto/*` — the isolated AEAD primitives (AES-128-GCM + the negotiated
//! ChaCha20-Poly1305) on one ~MTU shard.
//! - `pipeline/*`— a whole frame through the real per-frame path end to end over the in-process //! - `pipeline/*`— a whole frame through the real per-frame path end to end over the in-process
//! loopback transport: FEC encode → AES-GCM seal → packetize → (loopback) → reassemble → //! loopback transport: FEC encode → AES-GCM seal → packetize → (loopback) → reassemble →
//! FEC decode → open. This is what a throughput/latency regression in the core would show up in. //! FEC decode → open. This is what a throughput/latency regression in the core would show up in.
@@ -11,11 +12,11 @@
use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput}; use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionCrypto; use punktfunk_core::crypto::{SessionCrypto, SessionKey};
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
use punktfunk_core::transport::loopback_pair; use punktfunk_core::transport::loopback_pair;
const TAG_LEN: usize = 16; // AES-GCM authentication tag const TAG_LEN: usize = 16; // AEAD authentication tag (GCM and Poly1305 share the size)
const SHARD: usize = punktfunk_core::config::mtu1500_shard_payload(); // one MTU-safe data shard const SHARD: usize = punktfunk_core::config::mtu1500_shard_payload(); // one MTU-safe data shard
fn cfg(role: Role, scheme: FecScheme) -> Config { fn cfg(role: Role, scheme: FecScheme) -> Config {
@@ -38,21 +39,29 @@ fn cfg(role: Role, scheme: FecScheme) -> Config {
shard_payload: SHARD, shard_payload: SHARD,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: true, // bench the real path — crypto is always on for punktfunk/1 encrypt: true, // bench the real path — crypto is always on for punktfunk/1
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4], salt: [1, 2, 3, 4],
loopback_drop_period: 0, // throughput run: no induced loss (loss-harness covers recovery) loopback_drop_period: 0, // throughput run: no induced loss (loss-harness covers recovery)
} }
} }
fn bench_crypto(c: &mut Criterion) { fn bench_crypto(c: &mut Criterion) {
let host = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Host); let mut g = c.benchmark_group("crypto");
let client = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Client); g.throughput(Throughput::Bytes(SHARD as u64));
// Both negotiated session AEADs. On the x86 / Apple Silicon this runs on, both must be
// line-rate-trivial — the chacha20 series is the host-side sealing-cost check for the
// negotiated soft-AES-armv7 path (design/chacha20-session-cipher.md §7). The AES series
// keeps its unsuffixed names so the CI regression compare retains its history.
for (suffix, key) in [
("", SessionKey::Aes128Gcm([7u8; 16])),
("_chacha20", SessionKey::ChaCha20Poly1305([7u8; 32])),
] {
let host = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Host);
let client = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Client);
let payload = vec![0xABu8; SHARD]; let payload = vec![0xABu8; SHARD];
let sealed = host.seal(0, &payload).unwrap(); let sealed = host.seal(0, &payload).unwrap();
let mut g = c.benchmark_group("crypto"); g.bench_function(format!("seal{suffix}"), |b| {
g.throughput(Throughput::Bytes(SHARD as u64));
g.bench_function("seal", |b| {
let mut seq = 0u64; let mut seq = 0u64;
b.iter(|| { b.iter(|| {
let ct = host.seal(seq, black_box(&payload)).unwrap(); let ct = host.seal(seq, black_box(&payload)).unwrap();
@@ -60,7 +69,7 @@ fn bench_crypto(c: &mut Criterion) {
black_box(ct) black_box(ct)
}) })
}); });
g.bench_function("seal_in_place", |b| { g.bench_function(format!("seal_in_place{suffix}"), |b| {
let mut seq = 0u64; let mut seq = 0u64;
let mut buf = vec![0xABu8; SHARD + TAG_LEN]; let mut buf = vec![0xABu8; SHARD + TAG_LEN];
b.iter(|| { b.iter(|| {
@@ -68,10 +77,10 @@ fn bench_crypto(c: &mut Criterion) {
seq += 1; seq += 1;
}) })
}); });
g.bench_function("open", |b| { g.bench_function(format!("open{suffix}"), |b| {
b.iter(|| black_box(client.open(0, black_box(&sealed)).unwrap())) b.iter(|| black_box(client.open(0, black_box(&sealed)).unwrap()))
}); });
g.bench_function("open_in_place", |b| { g.bench_function(format!("open_in_place{suffix}"), |b| {
// In-place open consumes the buffer, so each iteration restores the ciphertext first — // In-place open consumes the buffer, so each iteration restores the ciphertext first —
// one memcpy, mirroring what the recv ring does when the next datagram lands in the slot. // one memcpy, mirroring what the recv ring does when the next datagram lands in the slot.
let mut buf = sealed.clone(); let mut buf = sealed.clone();
@@ -80,6 +89,7 @@ fn bench_crypto(c: &mut Criterion) {
black_box(client.open_in_place(0, &mut buf).unwrap()); black_box(client.open_in_place(0, &mut buf).unwrap());
}) })
}); });
}
g.finish(); g.finish();
} }
+5 -1
View File
@@ -11,6 +11,7 @@
//! - Panics never cross the boundary: every entry point is wrapped in `catch_unwind`. //! - Panics never cross the boundary: every entry point is wrapped in `catch_unwind`.
use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use crate::crypto::SessionKey;
use crate::error::PunktfunkStatus; use crate::error::PunktfunkStatus;
use crate::input::InputEvent; use crate::input::InputEvent;
use crate::reanchor::{GateVerdict, ReanchorGate}; use crate::reanchor::{GateVerdict, ReanchorGate};
@@ -94,7 +95,10 @@ impl PunktfunkConfig {
shard_payload: self.shard_payload as usize, shard_payload: self.shard_payload as usize,
max_frame_bytes, max_frame_bytes,
encrypt: self.encrypt != 0, encrypt: self.encrypt != 0,
key: self.key, // The C ABI keeps its fixed 16-byte key and always selects AES-128-GCM — no
// ABI_VERSION bump. Raw-`Config` C embedders can't negotiate ChaCha; the Swift/
// Kotlin clients are aarch64 with AES CE and never want it.
key: SessionKey::Aes128Gcm(self.key),
salt: self.salt, salt: self.salt,
loopback_drop_period: self.loopback_drop_period, loopback_drop_period: self.loopback_drop_period,
}; };
+1
View File
@@ -393,6 +393,7 @@ impl NativeClient {
launch, launch,
pin, pin,
identity, identity,
connect_timeout: timeout,
frames: frame_chan_w, frames: frame_chan_w,
audio_tx, audio_tx,
rumble_tx, rumble_tx,
@@ -32,21 +32,65 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
identity.as_ref().map(|(c, k)| (c.as_str(), k.as_str())), identity.as_ref().map(|(c, k)| (c.as_str(), k.as_str())),
); );
let ep = ep.map_err(|e| PunktfunkError::Io(std::io::Error::other(e.to_string())))?; let ep = ep.map_err(|e| PunktfunkError::Io(std::io::Error::other(e.to_string())))?;
let conn = ep // Dial with retry across the connect budget, not a single attempt: one quinn dial gives
// up after the transport's idle window (~8 s of silence), which is shorter than a
// suspend-to-RAM resume — the Steam Deck flow fires Wake-on-LAN and connects
// immediately, so the host is still waking while the first Initials go out, and a
// single-shot dial died just before the host came up. Short attempts keep the Initial
// cadence dense (quinn's per-attempt retransmits back off toward multi-second gaps), so
// the connect lands within ~a second of the host's network returning. Only SILENCE is
// retried: a host that answers and rejects us (pin mismatch, ALPN/version, typed close)
// must surface immediately, and the embedder's shutdown flag (budget expiry in
// `connect`, or a user cancel) stops the loop between attempts.
const DIAL_ATTEMPT: std::time::Duration = std::time::Duration::from_secs(3);
// Redial headroom: leave room for the control handshake (Hello/Welcome/clock sync)
// after a late dial success, so it still completes inside the embedder's budget.
const CONTROL_HEADROOM: std::time::Duration = std::time::Duration::from_secs(2);
let start = tokio::time::Instant::now();
let deadline = start + args.connect_timeout;
let redial_until = start + args.connect_timeout.saturating_sub(CONTROL_HEADROOM);
let conn = loop {
let connecting = ep
.connect(remote, "punktfunk") .connect(remote, "punktfunk")
.map_err(|_| PunktfunkError::InvalidArg("connect"))? .map_err(|_| PunktfunkError::InvalidArg("connect"))?;
.await // Cap the attempt to the remaining budget so a success never lands after the
.map_err(|e| { // embedder's `ready_rx` wait has already given up and flagged a teardown.
let now = tokio::time::Instant::now();
let attempt = DIAL_ATTEMPT.min(deadline.saturating_duration_since(now));
let gave_up = || {
tokio::time::Instant::now() >= redial_until
|| shutdown.load(std::sync::atomic::Ordering::SeqCst)
};
match tokio::time::timeout(attempt, connecting).await {
Ok(Ok(conn)) => break conn,
Ok(Err(e)) => {
// A pin mismatch surfaces as a TLS failure; report it as a crypto error so // A pin mismatch surfaces as a TLS failure; report it as a crypto error so
// the embedder can distinguish "wrong host identity" from plain IO trouble. // the embedder can distinguish "wrong host identity" from plain IO trouble.
let fp_mismatch = let fp_mismatch = pin.is_some()
pin.is_some() && observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true); && observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true);
if fp_mismatch { if fp_mismatch {
PunktfunkError::Crypto return Err(PunktfunkError::Crypto);
} else {
PunktfunkError::Io(std::io::Error::other(e.to_string()))
} }
})?; // The transport's own idle expiry — the host never answered — is the one
// retryable outcome; everything else is a real answer or a local failure.
let host_silent = matches!(e, quinn::ConnectionError::TimedOut);
if !host_silent {
return Err(PunktfunkError::Io(std::io::Error::other(e.to_string())));
}
if gave_up() {
return Err(PunktfunkError::Timeout);
}
}
// Attempt window elapsed with the host still silent; dropping `connecting`
// abandoned that dial — go again unless the budget is spent.
Err(_) => {
if gave_up() {
return Err(PunktfunkError::Timeout);
}
}
}
tracing::debug!(%remote, "host silent — re-dialing (wake/resume tolerant connect)");
};
let fingerprint = observed.lock().unwrap().unwrap_or([0u8; 32]); let fingerprint = observed.lock().unwrap().unwrap_or([0u8; 32]);
// The rest of the handshake runs in an inner future so a failure can consult // The rest of the handshake runs in an inner future so a failure can consult
// `conn.close_reason()`: a host that turned us away with a typed application close // `conn.close_reason()`: a host that turned us away with a typed application close
@@ -25,6 +25,10 @@ pub(crate) struct WorkerArgs {
pub(crate) launch: Option<String>, pub(crate) launch: Option<String>,
pub(crate) pin: Option<[u8; 32]>, pub(crate) pin: Option<[u8; 32]>,
pub(crate) identity: Option<(String, String)>, pub(crate) identity: Option<(String, String)>,
/// The embedder's connect budget (the same value `connect` bounds `ready_rx` with): the
/// dial loop re-dials a silent host within it, so a host still resuming from Wake-on-LAN
/// is caught the moment its network comes back instead of failing on the first attempt.
pub(crate) connect_timeout: std::time::Duration,
pub(crate) frames: Arc<FrameChannel>, pub(crate) frames: Arc<FrameChannel>,
pub(crate) audio_tx: SyncSender<AudioPacket>, pub(crate) audio_tx: SyncSender<AudioPacket>,
pub(crate) rumble_tx: SyncSender<RumbleUpdate>, pub(crate) rumble_tx: SyncSender<RumbleUpdate>,
+15 -6
View File
@@ -1,5 +1,6 @@
//! Session configuration and protocol/FEC parameters. //! Session configuration and protocol/FEC parameters.
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
use crate::packet::{CRYPTO_OVERHEAD, HEADER_LEN, MAX_DATAGRAM_BYTES}; use crate::packet::{CRYPTO_OVERHEAD, HEADER_LEN, MAX_DATAGRAM_BYTES};
use zeroize::Zeroize; use zeroize::Zeroize;
@@ -355,9 +356,11 @@ pub struct Config {
/// hostile/corrupt headers; see [`Session`](crate::session::Session)). /// hostile/corrupt headers; see [`Session`](crate::session::Session)).
pub max_frame_bytes: usize, pub max_frame_bytes: usize,
pub encrypt: bool, pub encrypt: bool,
/// AES-128 session key established during pairing. MUST be unique per session when /// The negotiated session AEAD + its key, established during pairing/handshake —
/// AES-128-GCM for every peer by default, ChaCha20-Poly1305 when the client negotiated it
/// (soft-AES armv7 targets; see [`SessionKey`]). MUST be unique per session when
/// `encrypt` is set (see the nonce-uniqueness contract in [`crate::crypto`]). /// `encrypt` is set (see the nonce-uniqueness contract in [`crate::crypto`]).
pub key: [u8; 16], pub key: SessionKey,
/// Per-session nonce salt, established alongside `key` during pairing. MUST be /// Per-session nonce salt, established alongside `key` during pairing. MUST be
/// unique per (key, session). /// unique per (key, session).
pub salt: [u8; 4], pub salt: [u8; 4],
@@ -382,7 +385,8 @@ impl std::fmt::Debug for Config {
.field("shard_payload", &self.shard_payload) .field("shard_payload", &self.shard_payload)
.field("max_frame_bytes", &self.max_frame_bytes) .field("max_frame_bytes", &self.max_frame_bytes)
.field("encrypt", &self.encrypt) .field("encrypt", &self.encrypt)
.field("key", &"<redacted>") // SessionKey's own Debug redacts the material but keeps the cipher choice visible.
.field("key", &self.key)
.field("salt", &"<redacted>") .field("salt", &"<redacted>")
.field("loopback_drop_period", &self.loopback_drop_period) .field("loopback_drop_period", &self.loopback_drop_period)
.finish() .finish()
@@ -426,7 +430,7 @@ impl Config {
"max_frame_bytes too large for this shard/block configuration (block count overflows u16)", "max_frame_bytes too large for this shard/block configuration (block count overflows u16)",
)); ));
} }
if self.encrypt && self.key == [0u8; 16] { if self.encrypt && self.key.is_zero() {
return Err(PunktfunkError::InvalidArg( return Err(PunktfunkError::InvalidArg(
"encrypt requires a non-zero session key (see crypto nonce-uniqueness contract)", "encrypt requires a non-zero session key (see crypto nonce-uniqueness contract)",
)); ));
@@ -449,7 +453,7 @@ impl Config {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 64 * 1024 * 1024, max_frame_bytes: 64 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
@@ -465,7 +469,12 @@ mod tests {
let mut c = Config::p1_defaults(Role::Host); let mut c = Config::p1_defaults(Role::Host);
c.encrypt = true; // key is still all-zero c.encrypt = true; // key is still all-zero
assert!(c.validate().is_err()); assert!(c.validate().is_err());
c.key = [1u8; 16]; c.key = SessionKey::Aes128Gcm([1u8; 16]);
assert!(c.validate().is_ok());
// The rejection follows whichever cipher variant is active.
c.key = SessionKey::ChaCha20Poly1305([0u8; 32]);
assert!(c.validate().is_err());
c.key = SessionKey::ChaCha20Poly1305([1u8; 32]);
assert!(c.validate().is_ok()); assert!(c.validate().is_ok());
} }
+194 -34
View File
@@ -1,9 +1,10 @@
//! AES-128-GCM session sealing, matching GameStream's video crypto in P1. //! Session sealing with the negotiated AEAD — AES-128-GCM (matching GameStream's video
//! crypto in P1) by default, ChaCha20-Poly1305 (RFC 8439) for clients without hardware AES.
//! //!
//! ## Nonce uniqueness (the GCM safety requirement) //! ## Nonce uniqueness (the AEAD safety requirement)
//! //!
//! The 96-bit nonce is `salt (4 bytes) || sequence (8 bytes, big-endian)`. Reusing a //! The 96-bit nonce is `salt (4 bytes) || sequence (8 bytes, big-endian)`. Reusing a
//! `(key, nonce)` pair under AES-GCM is catastrophic, so two precautions apply: //! `(key, nonce)` pair is catastrophic under either AEAD, so two precautions apply:
//! //!
//! 1. **Per-direction salts.** Host and client share one `key` and `salt`, and each //! 1. **Per-direction salts.** Host and client share one `key` and `salt`, and each
//! counts its sequence from 0. To stop the host's video stream and the client's input //! counts its sequence from 0. To stop the host's video stream and the client's input
@@ -17,17 +18,96 @@
//! The sequence number is also passed as AEAD associated data, so tampering with the //! The sequence number is also passed as AEAD associated data, so tampering with the
//! on-wire sequence is detected (the tag check fails) rather than silently shifting the //! on-wire sequence is detected (the tag check fails) rather than silently shifting the
//! nonce. Note: this layer does not provide anti-replay — see `Session`. //! nonce. Note: this layer does not provide anti-replay — see `Session`.
//!
//! ## Why two ciphers
//!
//! Both AEADs are full-strength; the choice (negotiated via `Welcome::cipher`) is purely a
//! performance one. On targets without hardware AES — the soft-AES armv7 clients (webOS TVs) —
//! GCM's fixsliced AES + software GHASH costs ~50100 cycles/byte and caps decrypt at
//! ~100 Mbps, while ChaCha20-Poly1305's ARX construction runs ~1017 cycles/byte in portable
//! software (design/chacha20-session-cipher.md). Same 96-bit nonce, 16-byte tag, and AAD
//! shape, so the entire nonce discipline above carries over verbatim.
use crate::config::Role; use crate::config::Role;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
use aes_gcm::aead::{Aead, AeadInPlace, KeyInit, Payload}; use aes_gcm::aead::{Aead, AeadInPlace, KeyInit, Payload};
use aes_gcm::{Aes128Gcm, Key, Nonce}; use aes_gcm::{Aes128Gcm, Key, Nonce};
use chacha20poly1305::ChaCha20Poly1305;
use zeroize::Zeroize;
/// 16-byte AEAD authentication tag appended by GCM. /// 16-byte AEAD authentication tag appended by either session cipher.
pub const TAG_LEN: usize = 16; pub const TAG_LEN: usize = 16;
// The wire (CRYPTO_OVERHEAD) and every in-place split assume both negotiated AEADs append
// exactly TAG_LEN bytes — a different-tag cipher can never slip in behind this constant.
const _: () = assert!(std::mem::size_of::<aes_gcm::Tag>() == TAG_LEN);
const _: () = assert!(std::mem::size_of::<chacha20poly1305::Tag>() == TAG_LEN);
/// The negotiated session AEAD together with its key material — merged so the invalid state
/// (a ChaCha cipher with an AES-sized key, or vice versa) is unrepresentable. AES-128-GCM is
/// the default every peer speaks; ChaCha20-Poly1305 is granted to clients that advertised
/// [`VIDEO_CAP_CHACHA20`](crate::quic::VIDEO_CAP_CHACHA20) (the soft-AES armv7 targets —
/// see the module docs). 256 bits for ChaCha is what RFC 8439 requires.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum SessionKey {
Aes128Gcm([u8; 16]),
ChaCha20Poly1305([u8; 32]),
}
impl SessionKey {
/// Canonical lowercase cipher name for session-start logs.
pub fn cipher_name(&self) -> &'static str {
match self {
SessionKey::Aes128Gcm(_) => "aes-128-gcm",
SessionKey::ChaCha20Poly1305(_) => "chacha20-poly1305",
}
}
/// True when the key material is all zeros — the pairing-layer footgun `Config::validate`
/// rejects when encryption is on (see the nonce-uniqueness contract in the module docs).
pub fn is_zero(&self) -> bool {
match self {
SessionKey::Aes128Gcm(k) => k == &[0u8; 16],
SessionKey::ChaCha20Poly1305(k) => k == &[0u8; 32],
}
}
}
/// Key material never appears in logs, whichever variant is active — only the cipher choice
/// (`Config`'s hand-written `Debug` relies on this).
impl std::fmt::Debug for SessionKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SessionKey::Aes128Gcm(_) => f.write_str("Aes128Gcm(<redacted>)"),
SessionKey::ChaCha20Poly1305(_) => f.write_str("ChaCha20Poly1305(<redacted>)"),
}
}
}
/// Same zeroize-on-drop discipline the raw key array had (`Config`'s `Drop`).
impl Zeroize for SessionKey {
fn zeroize(&mut self) {
match self {
SessionKey::Aes128Gcm(k) => k.zeroize(),
SessionKey::ChaCha20Poly1305(k) => k.zeroize(),
}
}
}
/// The two negotiated AEADs behind one seal/open surface. Both are the same RustCrypto
/// `aead 0.5` generation (identical trait shapes, nonce/tag types), so each call below is a
/// two-arm match right next to the cipher work itself.
// AES's precomputed round keys (~0.7 KB) dwarf ChaCha's 32-byte state, but there is exactly
// one long-lived `SessionCrypto` per session — boxing the variant would trade that one-off
// slack for a pointer chase on every per-datagram seal/open.
#[allow(clippy::large_enum_variant)]
enum Cipher {
Aes128Gcm(Aes128Gcm),
ChaCha20Poly1305(ChaCha20Poly1305),
}
pub struct SessionCrypto { pub struct SessionCrypto {
cipher: Aes128Gcm, cipher: Cipher,
/// Salt for nonces we seal with (our direction). /// Salt for nonces we seal with (our direction).
send_salt: [u8; 4], send_salt: [u8; 4],
/// Salt for nonces we open with (the peer's direction). /// Salt for nonces we open with (the peer's direction).
@@ -35,11 +115,18 @@ pub struct SessionCrypto {
} }
impl SessionCrypto { impl SessionCrypto {
pub fn new(key: &[u8; 16], salt: [u8; 4], role: Role) -> Self { pub fn new(key: &SessionKey, salt: [u8; 4], role: Role) -> Self {
let key = Key::<Aes128Gcm>::from_slice(key); let cipher = match key {
SessionKey::Aes128Gcm(k) => {
Cipher::Aes128Gcm(Aes128Gcm::new(Key::<Aes128Gcm>::from_slice(k)))
}
SessionKey::ChaCha20Poly1305(k) => Cipher::ChaCha20Poly1305(ChaCha20Poly1305::new(
Key::<ChaCha20Poly1305>::from_slice(k),
)),
};
let own = direction(role); let own = direction(role);
SessionCrypto { SessionCrypto {
cipher: Aes128Gcm::new(key), cipher,
send_salt: dir_salt(salt, own), send_salt: dir_salt(salt, own),
recv_salt: dir_salt(salt, own ^ 1), recv_salt: dir_salt(salt, own ^ 1),
} }
@@ -49,14 +136,15 @@ impl SessionCrypto {
/// authenticated as associated data. /// authenticated as associated data.
pub fn seal(&self, seq: u64, plaintext: &[u8]) -> Result<Vec<u8>> { pub fn seal(&self, seq: u64, plaintext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.send_salt, seq); let nonce = nonce(self.send_salt, seq);
self.cipher let aad = seq.to_be_bytes();
.encrypt( let payload = Payload {
Nonce::from_slice(&nonce),
Payload {
msg: plaintext, msg: plaintext,
aad: &seq.to_be_bytes(), aad: &aad,
}, };
) match &self.cipher {
Cipher::Aes128Gcm(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
Cipher::ChaCha20Poly1305(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
}
.map_err(|_| PunktfunkError::Crypto) .map_err(|_| PunktfunkError::Crypto)
} }
@@ -69,9 +157,15 @@ impl SessionCrypto {
let nonce = nonce(self.send_salt, seq); let nonce = nonce(self.send_salt, seq);
let split = buf.len() - TAG_LEN; let split = buf.len() - TAG_LEN;
let (plaintext, tag_slot) = buf.split_at_mut(split); let (plaintext, tag_slot) = buf.split_at_mut(split);
let tag = self let aad = seq.to_be_bytes();
.cipher let tag = match &self.cipher {
.encrypt_in_place_detached(Nonce::from_slice(&nonce), &seq.to_be_bytes(), plaintext) Cipher::Aes128Gcm(c) => {
c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
Cipher::ChaCha20Poly1305(c) => {
c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
}
.map_err(|_| PunktfunkError::Crypto)?; .map_err(|_| PunktfunkError::Crypto)?;
tag_slot.copy_from_slice(&tag); tag_slot.copy_from_slice(&tag);
Ok(()) Ok(())
@@ -80,20 +174,21 @@ impl SessionCrypto {
/// Open `ciphertext || tag` for sequence `seq` (also bound as associated data). /// Open `ciphertext || tag` for sequence `seq` (also bound as associated data).
pub fn open(&self, seq: u64, ciphertext: &[u8]) -> Result<Vec<u8>> { pub fn open(&self, seq: u64, ciphertext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.recv_salt, seq); let nonce = nonce(self.recv_salt, seq);
self.cipher let aad = seq.to_be_bytes();
.decrypt( let payload = Payload {
Nonce::from_slice(&nonce),
Payload {
msg: ciphertext, msg: ciphertext,
aad: &seq.to_be_bytes(), aad: &aad,
}, };
) match &self.cipher {
Cipher::Aes128Gcm(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
Cipher::ChaCha20Poly1305(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
}
.map_err(|_| PunktfunkError::Crypto) .map_err(|_| PunktfunkError::Crypto)
} }
/// Open in place, no per-packet allocation: `buf` holds `[ciphertext .. ][tag]` on entry and /// Open in place, no per-packet allocation: `buf` holds `[ciphertext .. ][tag]` on entry and
/// the plaintext in its first `buf.len() - TAG_LEN` bytes on success (returned as the length) /// the plaintext in its first `buf.len() - TAG_LEN` bytes on success (returned as the length)
/// — byte-identical to `open`, just written in place. GCM verifies the tag *before* /// — byte-identical to `open`, just written in place. Both AEADs verify the tag *before*
/// decrypting, so on failure `buf` still holds the ciphertext (the caller drops the packet /// decrypting, so on failure `buf` still holds the ciphertext (the caller drops the packet
/// either way). The hot-path receiver (`Session::poll_frame`) uses this to avoid the `Vec` /// either way). The hot-path receiver (`Session::poll_frame`) uses this to avoid the `Vec`
/// that `open`'s convenience API allocates for every datagram at line rate — the receive /// that `open`'s convenience API allocates for every datagram at line rate — the receive
@@ -105,13 +200,21 @@ impl SessionCrypto {
let nonce = nonce(self.recv_salt, seq); let nonce = nonce(self.recv_salt, seq);
let split = buf.len() - TAG_LEN; let split = buf.len() - TAG_LEN;
let (ciphertext, tag) = buf.split_at_mut(split); let (ciphertext, tag) = buf.split_at_mut(split);
self.cipher let aad = seq.to_be_bytes();
.decrypt_in_place_detached( match &self.cipher {
Cipher::Aes128Gcm(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce), Nonce::from_slice(&nonce),
&seq.to_be_bytes(), &aad,
ciphertext, ciphertext,
aes_gcm::Tag::from_slice(tag), aes_gcm::Tag::from_slice(tag),
) ),
Cipher::ChaCha20Poly1305(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce),
&aad,
ciphertext,
chacha20poly1305::Tag::from_slice(tag),
),
}
.map_err(|_| PunktfunkError::Crypto)?; .map_err(|_| PunktfunkError::Crypto)?;
Ok(split) Ok(split)
} }
@@ -145,6 +248,13 @@ pub fn random_key() -> [u8; 16] {
k k
} }
/// Generate a fresh random ChaCha20-Poly1305 session key (RFC 8439's 256-bit size).
pub fn random_key32() -> [u8; 32] {
let mut k = [0u8; 32];
rand::RngCore::fill_bytes(&mut rand::rng(), &mut k);
k
}
/// Generate a fresh random per-session nonce salt. /// Generate a fresh random per-session nonce salt.
pub fn random_salt() -> [u8; 4] { pub fn random_salt() -> [u8; 4] {
let mut s = [0u8; 4]; let mut s = [0u8; 4];
@@ -156,9 +266,17 @@ pub fn random_salt() -> [u8; 4] {
mod tests { mod tests {
use super::*; use super::*;
/// One fresh key per negotiated cipher — every sealing test below must hold for both.
fn both_keys() -> [SessionKey; 2] {
[
SessionKey::Aes128Gcm(random_key()),
SessionKey::ChaCha20Poly1305(random_key32()),
]
}
#[test] #[test]
fn seal_open_roundtrip_cross_direction() { fn seal_open_roundtrip_cross_direction() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -175,10 +293,11 @@ mod tests {
// open its own outbound packet → distinct nonce spaces per direction. // open its own outbound packet → distinct nonce spaces per direction.
assert!(host.open(42, &sealed).is_err()); assert!(host.open(42, &sealed).is_err());
} }
}
#[test] #[test]
fn directions_use_distinct_nonce_spaces() { fn directions_use_distinct_nonce_spaces() {
let key = random_key(); for key in both_keys() {
let salt = [0u8; 4]; // even an all-zero base salt must separate the directions let salt = [0u8; 4]; // even an all-zero base salt must separate the directions
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -188,10 +307,11 @@ mod tests {
client.seal(0, b"abc").unwrap() client.seal(0, b"abc").unwrap()
); );
} }
}
#[test] #[test]
fn open_in_place_matches_open_and_rejects_tampering() { fn open_in_place_matches_open_and_rejects_tampering() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -221,10 +341,11 @@ mod tests {
let mut runt = vec![0u8; TAG_LEN - 1]; let mut runt = vec![0u8; TAG_LEN - 1];
assert!(client.open_in_place(0, &mut runt).is_err()); assert!(client.open_in_place(0, &mut runt).is_err());
} }
}
#[test] #[test]
fn seal_in_place_matches_seal_and_opens() { fn seal_in_place_matches_seal_and_opens() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -245,4 +366,43 @@ mod tests {
assert_eq!(client.open(7, &buf).unwrap(), msg); assert_eq!(client.open(7, &buf).unwrap(), msg);
} }
} }
}
#[test]
fn ciphers_are_not_interchangeable() {
// A packet sealed under one AEAD must not open under the other — negotiation skew has
// to fail loudly (a tag mismatch), never decode garbage. The ChaCha key repeats the AES
// key bytes so even overlapping key material can't accidentally interoperate.
let salt = random_salt();
let aes = SessionKey::Aes128Gcm([7u8; 16]);
let chacha = SessionKey::ChaCha20Poly1305([7u8; 32]);
let sealed = SessionCrypto::new(&aes, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&chacha, salt, Role::Client)
.open(1, &sealed)
.is_err());
let sealed = SessionCrypto::new(&chacha, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&aes, salt, Role::Client)
.open(1, &sealed)
.is_err());
}
#[test]
fn session_key_zero_check_and_debug_redaction() {
assert!(SessionKey::Aes128Gcm([0u8; 16]).is_zero());
assert!(SessionKey::ChaCha20Poly1305([0u8; 32]).is_zero());
assert!(!SessionKey::Aes128Gcm([1u8; 16]).is_zero());
assert!(!SessionKey::ChaCha20Poly1305([1u8; 32]).is_zero());
// Key bytes must never reach a log, whichever variant — only the cipher choice.
for key in both_keys() {
let dbg = format!("{key:?}");
assert!(dbg.contains("<redacted>"), "{dbg}");
}
let mut k = SessionKey::ChaCha20Poly1305([9u8; 32]);
k.zeroize();
assert!(k.is_zero());
}
} }
+3 -2
View File
@@ -1,6 +1,7 @@
use super::reassemble::LOSS_WINDOW_NS; use super::reassemble::LOSS_WINDOW_NS;
use super::*; use super::*;
use crate::config::{Config, FecScheme}; use crate::config::{Config, FecScheme};
use crate::crypto::SessionKey;
use crate::fec::coder_for; use crate::fec::coder_for;
use crate::stats::StatsCounters; use crate::stats::StatsCounters;
use zerocopy::{FromBytes, IntoBytes}; use zerocopy::{FromBytes, IntoBytes};
@@ -182,7 +183,7 @@ fn explicit_frame_index_is_stamped_and_internal_counter_untouched() {
shard_payload: 16, shard_payload: 16,
max_frame_bytes: 4096, max_frame_bytes: 4096,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
}; };
@@ -292,7 +293,7 @@ fn e2e_config(scheme: FecScheme, fec_percent: u8) -> Config {
shard_payload: 16, shard_payload: 16,
max_frame_bytes: 4096, max_frame_bytes: 4096,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
+13
View File
@@ -44,6 +44,17 @@ pub const VIDEO_CAP_PROBE_SEQ: u8 = 0x10;
/// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so /// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so
/// the fallback is zero-risk. /// the fallback is zero-risk.
pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20; pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20;
/// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
/// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
/// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
/// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
/// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
/// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
/// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
/// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
/// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
/// control channel, so there is no downgrade surface.
pub const VIDEO_CAP_CHACHA20: u8 = 0x40;
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] /// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder /// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -225,6 +236,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_HEVC, codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD, host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD,
cipher: 0,
key_chacha: None,
}; };
let got = Welcome::decode(&w.encode()).unwrap(); let got = Welcome::decode(&w.encode()).unwrap();
assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD); assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD);
+152 -3
View File
@@ -4,6 +4,7 @@ use super::*;
use crate::config::{ use crate::config::{
CompositorPref, Config, FecConfig, FecScheme, GamepadPref, Mode, ProtocolPhase, Role, CompositorPref, Config, FecConfig, FecScheme, GamepadPref, Mode, ProtocolPhase, Role,
}; };
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
/// `client → host`: open the session, requesting a display mode (the host creates its /// `client → host`: open the session, requesting a display mode (the host creates its
@@ -107,6 +108,13 @@ pub const HELLO_NAME_MAX: usize = 64;
/// (`steam:<appid>` / `custom:<12 hex>`); the cap just bounds an attacker-controlled field. /// (`steam:<appid>` / `custom:<12 hex>`); the cap just bounds an attacker-controlled field.
pub const HELLO_LAUNCH_MAX: usize = 128; pub const HELLO_LAUNCH_MAX: usize = 128;
/// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
/// only one pre-cipher builds know).
pub const CIPHER_AES_128_GCM: u8 = 0;
/// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
/// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
pub const CIPHER_CHACHA20_POLY1305: u8 = 1;
/// `host → client`: the complete session offer. /// `host → client`: the complete session offer.
#[derive(Clone, Copy, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Welcome { pub struct Welcome {
@@ -173,6 +181,22 @@ pub struct Welcome {
/// per-transition events otherwise). Appended after `codec` as a single trailing byte; an /// per-transition events otherwise). Appended after `codec` as a single trailing byte; an
/// older host that omits it decodes to `0` (no capabilities — legacy events only). /// older host that omits it decodes to `0` (no capabilities — legacy events only).
pub host_caps: u8, pub host_caps: u8,
/// The session AEAD the data plane seals with — [`CIPHER_AES_128_GCM`] (`0`, the default
/// every peer speaks) or [`CIPHER_CHACHA20_POLY1305`] (`1`). The host sets `1` ONLY toward
/// a client that advertised [`VIDEO_CAP_CHACHA20`] (the soft-AES armv7 targets). Appended
/// after `host_caps` at offset 68 and — unlike the earlier trailing fields — emitted only
/// when non-zero, so an AES session's Welcome stays **byte-identical** to the pre-cipher
/// wire form; an older host omits it (→ `0`, AES). Decode is fail-closed: an unknown id is
/// an `Err`, never a silent AES fallback — the host only picks a cipher this client
/// advertised, so an unknown id reaching us is a bug, and falling back would yield an
/// undecryptable session with a confusing failure signature.
pub cipher: u8,
/// The 256-bit ChaCha20-Poly1305 session key (RFC 8439 requires the full 32 bytes; wire
/// cost is once per handshake) — present iff `cipher == 1`, at offsets 69..101. The legacy
/// 16-byte `key` keeps its offset and stays independently random, so nothing downstream
/// ever observes an all-zero key. Decode rejects `cipher == 1` with fewer than 32 key
/// bytes following.
pub key_chacha: Option<[u8; 32]>,
} }
/// `client → host`: data plane is bound, begin streaming. /// `client → host`: data plane is bound, begin streaming.
@@ -366,6 +390,21 @@ impl Welcome {
b.push(self.codec); b.push(self.codec);
// Host input caps at offset 67 — older clients stop before this → 0 (legacy input only). // Host input caps at offset 67 — older clients stop before this → 0 (legacy input only).
b.push(self.host_caps); b.push(self.host_caps);
// Session cipher at offset 68 + the 32-byte ChaCha key at 69..101 — emitted ONLY when a
// non-default cipher was negotiated, so an AES session's Welcome stays byte-identical
// to the pre-cipher wire form. The host only sets cipher toward a client that
// advertised VIDEO_CAP_CHACHA20, so an old client never sees these bytes at all.
debug_assert_eq!(
self.cipher == CIPHER_CHACHA20_POLY1305,
self.key_chacha.is_some(),
"key_chacha present iff cipher == 1"
);
if self.cipher != CIPHER_AES_128_GCM {
b.push(self.cipher);
if let Some(k) = &self.key_chacha {
b.extend_from_slice(k);
}
}
b b
} }
@@ -374,8 +413,10 @@ impl Welcome {
// scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45] // scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45]
// salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59] // salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59]
// bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63] // bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63]
// chroma_format[64] audio_channels[65] codec[66] (everything from compositor on is an // chroma_format[64] audio_channels[65] codec[66] host_caps[67] cipher[68]
// optional trailing byte; an older host stops earlier). // key_chacha[69..101] (everything from compositor on is an optional trailing byte; an
// older host stops earlier; cipher/key_chacha are present only when ChaCha was
// negotiated).
if b.len() < 53 || &b[0..4] != MAGIC { if b.len() < 53 || &b[0..4] != MAGIC {
return Err(PunktfunkError::InvalidArg("bad Welcome")); return Err(PunktfunkError::InvalidArg("bad Welcome"));
} }
@@ -385,6 +426,24 @@ impl Welcome {
key.copy_from_slice(&b[29..45]); key.copy_from_slice(&b[29..45]);
let mut salt = [0u8; 4]; let mut salt = [0u8; 4];
salt.copy_from_slice(&b[45..49]); salt.copy_from_slice(&b[45..49]);
// Session cipher at 68 — absent on an older host → AES-128-GCM. Fail-closed on
// anything else: `cipher == 1` with fewer than 32 key bytes must be an error (a silent
// AES fallback would yield an undecryptable session with a confusing failure
// signature), and an unknown id (≥ 2) reaching us is a bug — a host only picks a
// cipher this client advertised — never a legitimate negotiation.
let cipher = b.get(68).copied().unwrap_or(CIPHER_AES_128_GCM);
let key_chacha = match cipher {
CIPHER_AES_128_GCM => None,
CIPHER_CHACHA20_POLY1305 => {
let bytes = b
.get(69..101)
.ok_or(PunktfunkError::InvalidArg("bad Welcome"))?;
let mut k = [0u8; 32];
k.copy_from_slice(bytes);
Some(k)
}
_ => return Err(PunktfunkError::InvalidArg("bad Welcome")),
};
Ok(Welcome { Ok(Welcome {
abi_version: u32at(4), abi_version: u32at(4),
udp_port: u16at(8), udp_port: u16at(8),
@@ -452,6 +511,8 @@ impl Welcome {
// Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state // Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state
// snapshots; the client keeps sending legacy per-transition events). // snapshots; the client keeps sending legacy per-transition events).
host_caps: b.get(67).copied().unwrap_or(0), host_caps: b.get(67).copied().unwrap_or(0),
cipher,
key_chacha,
}) })
} }
@@ -462,7 +523,12 @@ impl Welcome {
c.fec = self.fec; c.fec = self.fec;
c.shard_payload = self.shard_payload as usize; c.shard_payload = self.shard_payload as usize;
c.encrypt = self.encrypt; c.encrypt = self.encrypt;
c.key = self.key; // The negotiated AEAD: the ChaCha key when cipher == 1 (guaranteed present by decode —
// the `(1, None)` shape is unreachable off the wire), the legacy AES key otherwise.
c.key = match (self.cipher, self.key_chacha) {
(CIPHER_CHACHA20_POLY1305, Some(k)) => SessionKey::ChaCha20Poly1305(k),
_ => SessionKey::Aes128Gcm(self.key),
};
c.salt = self.salt; c.salt = self.salt;
// Client-side reassembler ceiling: p1_defaults' 64 MiB hostile-header memory bound is // Client-side reassembler ceiling: p1_defaults' 64 MiB hostile-header memory bound is
// ~10x larger than any real access unit. Derive it from the negotiated rate instead: // ~10x larger than any real access unit. Derive it from the negotiated rate instead:
@@ -531,6 +597,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, // exercise a non-default codec through the roundtrip codec: CODEC_H264, // exercise a non-default codec through the roundtrip
host_caps: HOST_CAP_GAMEPAD_STATE, host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
}; };
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w); assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
@@ -564,6 +632,81 @@ mod tests {
assert!(derived > (8 << 20) && derived < (64 << 20)); assert!(derived > (8 << 20) && derived < (64 << 20));
} }
#[test]
fn welcome_cipher_negotiation_wire_and_back_compat() {
use crate::crypto::SessionKey;
let base = Welcome {
abi_version: 2,
udp_port: 7000,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 20,
max_data_per_block: 4096,
},
shard_payload: 1200,
encrypt: true,
key: [7u8; 16],
salt: [9, 8, 7, 6],
frames: 0,
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 50_000,
bit_depth: 8,
color: ColorInfo::SDR_BT709,
chroma_format: CHROMA_IDC_420,
audio_channels: 2,
codec: CODEC_HEVC,
host_caps: 0,
cipher: CIPHER_AES_128_GCM,
key_chacha: None,
};
// An AES session's Welcome is byte-identical to the pre-cipher wire form (68 bytes) —
// the old-client × new-host interop guarantee.
let enc = base.encode();
assert_eq!(enc.len(), 68);
assert_eq!(Welcome::decode(&enc).unwrap(), base);
// ChaCha roundtrip: cipher byte at 68, the 32-byte key at 69..101.
let k32: [u8; 32] = core::array::from_fn(|i| i as u8 + 1);
let cha = Welcome {
cipher: CIPHER_CHACHA20_POLY1305,
key_chacha: Some(k32),
..base
};
let cenc = cha.encode();
assert_eq!(cenc.len(), 68 + 1 + 32);
assert_eq!(Welcome::decode(&cenc).unwrap(), cha);
// A truncated old-host Welcome (no cipher byte) decodes to the AES default.
let old_host = Welcome::decode(&cenc[..68]).unwrap();
assert_eq!(old_host.cipher, CIPHER_AES_128_GCM);
assert_eq!(old_host.key_chacha, None);
// cipher == 1 with a missing / short key → Err, fail-closed (a silent AES fallback
// would yield an undecryptable session with a confusing failure signature).
assert!(Welcome::decode(&cenc[..69]).is_err());
assert!(Welcome::decode(&cenc[..100]).is_err());
// An unknown cipher id (≥ 2) → Err: the host only picks a cipher we advertised, so an
// unknown id reaching us is a bug, never a legitimate negotiation.
let mut bad = cenc.clone();
bad[68] = 2;
assert!(Welcome::decode(&bad).is_err());
// session_config maps both variants onto the data-plane key, and both validate.
let aes_cfg = base.session_config(Role::Client);
assert_eq!(aes_cfg.key, SessionKey::Aes128Gcm([7u8; 16]));
aes_cfg.validate().expect("AES config validates");
let cha_cfg = cha.session_config(Role::Client);
assert_eq!(cha_cfg.key, SessionKey::ChaCha20Poly1305(k32));
cha_cfg.validate().expect("ChaCha config validates");
}
#[test] #[test]
fn codec_negotiation_and_back_compat() { fn codec_negotiation_and_back_compat() {
// resolve_codec precedence (HEVC > AV1 > H.264), no preference (0). // resolve_codec precedence (HEVC > AV1 > H.264), no preference (0).
@@ -656,6 +799,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_PYROWAVE, codec: CODEC_PYROWAVE,
host_caps: 0, host_caps: 0,
cipher: 0,
key_chacha: None,
} }
.encode(), .encode(),
) )
@@ -726,6 +871,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, codec: CODEC_H264,
host_caps: 0, host_caps: 0,
cipher: 0,
key_chacha: None,
} }
.encode(), .encode(),
) )
@@ -831,6 +978,8 @@ mod tests {
audio_channels: 6, // 5.1 — exercises the non-default trailing byte audio_channels: 6, // 5.1 — exercises the non-default trailing byte
codec: CODEC_HEVC, codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE, host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
}; };
let wenc = w.encode(); let wenc = w.encode();
assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps
+3 -2
View File
@@ -781,6 +781,7 @@ impl Session {
mod wire_equivalence_tests { mod wire_equivalence_tests {
use super::*; use super::*;
use crate::config::{FecConfig, FecScheme, ProtocolPhase}; use crate::config::{FecConfig, FecScheme, ProtocolPhase};
use crate::crypto::SessionKey;
use crate::transport::loopback_pair; use crate::transport::loopback_pair;
fn host_cfg(scheme: FecScheme, fec_percent: u8, encrypt: bool) -> Config { fn host_cfg(scheme: FecScheme, fec_percent: u8, encrypt: bool) -> Config {
@@ -798,7 +799,7 @@ mod wire_equivalence_tests {
shard_payload: 64, shard_payload: 64,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt, encrypt,
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [3, 1, 4, 1], salt: [3, 1, 4, 1],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
@@ -930,7 +931,7 @@ mod wire_equivalence_tests {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
}; };
+26 -1
View File
@@ -5,6 +5,7 @@
use proptest::prelude::*; use proptest::prelude::*;
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionKey;
use punktfunk_core::fec::coder_for; use punktfunk_core::fec::coder_for;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
@@ -25,7 +26,7 @@ fn config(role: Role, scheme: FecScheme, encrypt: bool, drop_period: u32) -> Con
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt, encrypt,
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4], salt: [1, 2, 3, 4],
loopback_drop_period: drop_period, loopback_drop_period: drop_period,
} }
@@ -101,6 +102,30 @@ fn encrypted_stream_recovers_under_loss() {
assert_eq!(stats.frames_completed, frames.len() as u64); assert_eq!(stats.frames_completed, frames.len() as u64);
} }
/// The negotiated ChaCha20-Poly1305 session cipher through the same lossy full-stream path:
/// loss/replay behavior is cipher-independent (the replay window keys off the authenticated
/// seq), so recovery must be byte-identical to the AES run above.
#[test]
fn chacha20_encrypted_stream_recovers_under_loss() {
let frames = sample_frames();
let mk = |role| {
let mut c = config(role, FecScheme::Gf16, true, 8);
c.key = SessionKey::ChaCha20Poly1305([7u8; 32]);
c
};
let (host_tp, client_tp) = loopback_pair(8, 0);
let mut host = Session::new(mk(Role::Host), Box::new(host_tp)).unwrap();
let mut client = Session::new(mk(Role::Client), Box::new(client_tp)).unwrap();
for (i, frame) in frames.iter().enumerate() {
host.submit_frame(frame, i as u64 * 1_000_000, 0).unwrap();
let got = client
.poll_frame()
.expect("frame should recover despite loss");
assert_eq!(&got.data, frame, "frame {i} mismatched after recovery");
}
assert!(client.stats().fec_recovered_shards > 0);
}
#[test] #[test]
fn lossless_stream_is_exact() { fn lossless_stream_is_exact() {
let frames = sample_frames(); let frames = sample_frames();
+9 -3
View File
@@ -26,7 +26,10 @@ pub use pf_capture::{dxgi, synthetic_nv12};
/// capture→encode cycle). Resolved here (the host facade) and threaded in, so the edge stays one-way /// capture→encode cycle). Resolved here (the host facade) and threaded in, so the edge stays one-way
/// (plan §2.4 / §W6). /// (plan §2.4 / §W6).
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
fn zero_copy_policy(pyrowave_session: bool) -> pf_capture::ZeroCopyPolicy { fn zero_copy_policy(
pyrowave_session: bool,
native_nv12_session: bool,
) -> pf_capture::ZeroCopyPolicy {
let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi(); let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi();
// The raw-dmabuf passthrough serves a PyroWave session on ANY vendor (the wavelet encoder's // The raw-dmabuf passthrough serves a PyroWave session on ANY vendor (the wavelet encoder's
// own Vulkan device imports the dmabuf) — per-session from the negotiated codec, plus the // own Vulkan device imports the dmabuf) — per-session from the negotiated codec, plus the
@@ -56,6 +59,7 @@ fn zero_copy_policy(pyrowave_session: bool) -> pf_capture::ZeroCopyPolicy {
backend_is_gpu: crate::encode::resolved_backend_is_gpu(), backend_is_gpu: crate::encode::resolved_backend_is_gpu(),
pyrowave_session, pyrowave_session,
pyrowave_modifiers, pyrowave_modifiers,
native_nv12_session,
} }
} }
@@ -70,7 +74,9 @@ pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> {
let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei; let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei;
// Monitor mirrors never carry the native PyroWave plane (GameStream protocol) — per-session // Monitor mirrors never carry the native PyroWave plane (GameStream protocol) — per-session
// passthrough is virtual-output-only; the global encoder-pref lever still applies inside. // passthrough is virtual-output-only; the global encoder-pref lever still applies inside.
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy(false)) // Native NV12 stays off too: the mirror path doesn't resolve the codec here, and the desktop
// compositors it mirrors (GNOME/KWin) don't produce NV12 anyway.
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy(false, false))
} }
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
@@ -101,7 +107,7 @@ pub fn capture_virtual_output(
vout.keepalive, vout.keepalive,
want.gpu, want.gpu,
want.chroma_444, want.chroma_444,
zero_copy_policy(want.pyrowave), zero_copy_policy(want.pyrowave, want.nv12_native),
) )
} }
@@ -86,6 +86,12 @@ pub fn start(
crate::events::emit(crate::events::EventKind::ClientConnected { crate::events::emit(crate::events::EventKind::ClientConnected {
client: event_client.clone(), client: event_client.clone(),
}); });
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock
// floor while this compat-plane stream runs, refcounted with every other live session
// across both planes. Released when the closure exits (stream stopped) — so idle clocks
// aren't pinned between Moonlight sessions. No-op off Linux / when the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
let result = run( let result = run(
cfg, cfg,
app.as_ref(), app.as_ref(),
+96 -20
View File
@@ -2,11 +2,15 @@
//! plan Tier 1B): the driver's adaptive P-state ramps clocks down between bursty encode frames, //! plan Tier 1B): the driver's adaptive P-state ramps clocks down between bursty encode frames,
//! so every frame re-pays a spin-up. This is NOT theoretical — measured on the 780M (VCN 4), //! so every frame re-pays a spin-up. This is NOT theoretical — measured on the 780M (VCN 4),
//! a 1440p HEVC encode takes ~4.4 ms/frame with the clocks hot (120 fps pacing) but ~8 ms/frame //! a 1440p HEVC encode takes ~4.4 ms/frame with the clocks hot (120 fps pacing) but ~8 ms/frame
//! at a 60 fps duty cycle: the sag doubles per-frame encode latency. //! at a 60 fps duty cycle: the sag doubles per-frame encode latency. So the pin is worth holding
//! only while a client is actively streaming: it is armed on the first live client and released
//! when the last one disconnects (refcounted across both streaming planes — see [`session_pin`]),
//! leaving the driver's idle power management alone the rest of the time.
//! //!
//! **AMD** (`PUNKTFUNK_PIN_CLOCKS=1`, root-gated by sysfs ownership): write `high` into each //! **AMD** (`PUNKTFUNK_PIN_CLOCKS=1`, root-gated by sysfs ownership): write `high` into each
//! amdgpu card's `power_dpm_force_performance_level` for the host lifetime, restoring the prior //! amdgpu card's `power_dpm_force_performance_level` while a client is streaming, restoring the
//! value on exit. Non-root gets EACCES → logged once with the privilege recipe. Deliberately //! prior value when the last client disconnects. Non-root gets EACCES → logged once with the
//! privilege recipe. Deliberately
//! opt-in: it defeats power management box-wide and is wrong on battery (Steam Deck!). //! opt-in: it defeats power management box-wide and is wrong on battery (Steam Deck!).
//! //!
//! **NVIDIA** — two independent halves, both no-ops off NVIDIA: //! **NVIDIA** — two independent halves, both no-ops off NVIDIA:
@@ -28,14 +32,16 @@
//! while leaving boost headroom — NVIDIA's own latency guidance is "raise the floor, don't pin //! while leaving boost headroom — NVIDIA's own latency guidance is "raise the floor, don't pin
//! the max" (locking above base just gets throttled; a max pin only burns idle watts). Non-root //! the max" (locking above base just gets throttled; a max pin only burns idle watts). Non-root
//! callers get `NVML_ERROR_NO_PERMISSION` — logged once with the privilege recipe, then the //! callers get `NVML_ERROR_NO_PERMISSION` — logged once with the privilege recipe, then the
//! host runs unpinned. The pin is undone on drop (host exit); after a crash it persists until //! host runs unpinned. The pin is undone on drop (when the last client disconnects); after a
//! driver reload/reboot, which the reset-before-pin on the next start self-heals. Deliberately //! crash it persists until driver reload/reboot, which the reset-before-pin on the next arm
//! self-heals. Deliberately
//! NOT default-on: it defeats idle downclocking for the whole box and is wrong on //! NOT default-on: it defeats idle downclocking for the whole box and is wrong on
//! battery-powered hosts. //! battery-powered hosts.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program). // Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use std::os::raw::{c_char, c_int, c_uint, c_void}; use std::os::raw::{c_char, c_int, c_uint, c_void};
use std::sync::{Mutex, OnceLock};
/// `nvmlDevice_t` — an opaque driver handle. /// `nvmlDevice_t` — an opaque driver handle.
type NvmlDevice = *mut c_void; type NvmlDevice = *mut c_void;
@@ -133,8 +139,9 @@ struct AmdPin {
restore: String, restore: String,
} }
/// Host-lifetime guard: holds the armed clock pins (NVML floor and/or amdgpu perf level) and /// Holds the armed clock pins (NVML floor and/or amdgpu perf level) and undoes them on drop. Owned
/// undoes them on drop. /// by the [`session_pin`] refcount: constructed when the first live client arms the pin, dropped
/// (clocks restored) when the last one disconnects.
pub struct ClockGuard { pub struct ClockGuard {
nvml: Option<NvmlPin>, nvml: Option<NvmlPin>,
amd: Vec<AmdPin>, amd: Vec<AmdPin>,
@@ -142,9 +149,10 @@ pub struct ClockGuard {
// SAFETY: `ClockGuard` holds opaque NVML device handles + resolved fn pointers from the loaded // SAFETY: `ClockGuard` holds opaque NVML device handles + resolved fn pointers from the loaded
// driver library (plus plain sysfs paths/strings). NVML is documented thread-safe, the handles are // driver library (plus plain sysfs paths/strings). NVML is documented thread-safe, the handles are
// plain driver tokens with no thread affinity, and the guard is only ever *moved* (held in `main`, // plain driver tokens with no thread affinity, and the guard is only ever *moved* (into the
// dropped once at exit) and used through `&mut`/ownership — never shared. Transfer across threads // `pin_refcount` mutex when armed, taken back out and dropped when the last client disconnects) and
// is therefore sound. // used through exclusive ownership behind that mutex — never shared. Transfer across threads is
// therefore sound.
unsafe impl Send for ClockGuard {} unsafe impl Send for ClockGuard {}
impl Drop for ClockGuard { impl Drop for ClockGuard {
@@ -182,22 +190,89 @@ impl Drop for ClockGuard {
} }
/// Startup hook for the host subcommands (`serve` / `punktfunk1-host`): install the NVIDIA P2-cap /// Startup hook for the host subcommands (`serve` / `punktfunk1-host`): install the NVIDIA P2-cap
/// application profile and, when `PUNKTFUNK_PIN_CLOCKS` is set, arm the vendor clock pin (NVML /// application profile — the process-scoped, no-root half of the NVIDIA lever, which the driver
/// core-clock floor / amdgpu `high` performance level). Returns the guard keeping the pins for /// only acts on once the host holds a live CUDA/NVENC context (i.e. during a session).
/// the host lifetime. `None` when nothing was armed. ///
pub fn on_host_start() -> Option<ClockGuard> { /// The vendor clock *pin* is deliberately NOT armed here anymore: held for the whole host lifetime
/// it kept the box's clocks hot even with no client connected. It is now refcounted per live client
/// via [`session_pin`] (armed on both streaming planes), so idle clocks are left to the driver's
/// power management until someone actually streams.
pub fn on_host_start() {
if nvidia_present() { if nvidia_present() {
ensure_cuda_perf_profile(); ensure_cuda_perf_profile();
} }
}
/// The box-wide clock-pin refcount, shared across BOTH streaming planes (native + GameStream): the
/// vendor pin is a single global GPU setting, so N concurrent sessions share ONE pin — armed when
/// the first client goes live, released when the last one leaves.
struct PinRefcount {
/// Number of live [`SessionClockPin`] handles.
live: usize,
/// The armed pins, present iff `live > 0` and something was actually pinnable.
guard: Option<ClockGuard>,
}
fn pin_refcount() -> &'static Mutex<PinRefcount> {
static STATE: OnceLock<Mutex<PinRefcount>> = OnceLock::new();
STATE.get_or_init(|| {
Mutex::new(PinRefcount {
live: 0,
guard: None,
})
})
}
/// RAII handle that keeps the box-wide clock pin armed while it is alive. Obtain one per live client
/// session on either plane via [`session_pin`]; when the last outstanding handle drops, the pin is
/// released and the driver's idle downclocking resumes. A no-op handle when `PUNKTFUNK_PIN_CLOCKS`
/// is unset (the opt-in gate) — the refcount only ticks for sessions that asked for pinning.
pub struct SessionClockPin {
/// Whether this handle actually incremented the refcount (false = opt-in gate off → no-op).
counted: bool,
}
/// Arm the box-wide clock pin for one live client session, refcounted across every plane. Returns
/// an RAII handle; the pin is released when the last handle drops. A no-op (returns immediately,
/// touches no GPU state) unless `PUNKTFUNK_PIN_CLOCKS` is set.
pub fn session_pin() -> SessionClockPin {
if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") { if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") {
return None; return SessionClockPin { counted: false };
} }
let mut state = pin_refcount().lock().unwrap();
state.live += 1;
if state.live == 1 {
// 0 → 1: the first live client — arm the vendor pin (reset-before-pin inside `pin_nvidia`
// heals a stale pin from a crashed previous run).
let nvml = if nvidia_present() { pin_nvidia() } else { None }; let nvml = if nvidia_present() { pin_nvidia() } else { None };
let amd = pin_amdgpu(); let amd = pin_amdgpu();
if nvml.is_none() && amd.is_empty() { state.guard = if nvml.is_none() && amd.is_empty() {
return None; None // nothing pinnable (no perms / no supported GPU) — the session streams unpinned
} } else {
Some(ClockGuard { nvml, amd }) Some(ClockGuard { nvml, amd })
};
}
SessionClockPin { counted: true }
}
impl Drop for SessionClockPin {
fn drop(&mut self) {
if !self.counted {
return;
}
// Take the guard out under the lock but drop it *outside* — releasing the pin does NVML +
// sysfs I/O we don't want to hold the refcount lock across.
let release = {
let mut state = pin_refcount().lock().unwrap();
state.live = state.live.saturating_sub(1);
if state.live == 0 {
state.guard.take()
} else {
None
}
};
drop(release);
}
} }
/// Force every amdgpu card's DPM performance level to `high` for the session — the encode-latency /// Force every amdgpu card's DPM performance level to `high` for the session — the encode-latency
@@ -238,7 +313,7 @@ fn pin_amdgpu() -> Vec<AmdPin> {
card = %name, card = %name,
was = %prev, was = %prev,
"amdgpu performance level pinned to high (encode clock sag removed) — \ "amdgpu performance level pinned to high (encode clock sag removed) — \
restored on host exit" restored when the last client disconnects"
); );
pins.push(AmdPin { pins.push(AmdPin {
path, path,
@@ -327,7 +402,8 @@ fn pin_nvidia() -> Option<NvmlPin> {
} }
tracing::info!( tracing::info!(
devices = pinned.len(), devices = pinned.len(),
"NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released on host exit" "NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released when the last \
client disconnects"
); );
Some(NvmlPin { nvml, pinned }) Some(NvmlPin { nvml, pinned })
} }
+37 -8
View File
@@ -246,14 +246,17 @@ fn real_main() -> Result<()> {
crate::capture::dxgi::install_gpu_pref_hook(); crate::capture::dxgi::install_gpu_pref_hook();
} }
// NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile and, // NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile. The
// under PUNKTFUNK_PIN_CLOCKS, hold the NVML core-clock floor for the host lifetime (reset on // vendor clock *pin* (PUNKTFUNK_PIN_CLOCKS) is no longer held for the host lifetime — it is
// exit via the guard's Drop). No-op off NVIDIA / on the tool subcommands. // armed per live client via `gpuclocks::session_pin()` on both streaming planes, so idle clocks
// are left alone while nobody is connected. No-op off NVIDIA / on the tool subcommands.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
let _nv_clocks = match args.first().map(String::as_str) { if matches!(
Some("serve") | Some("punktfunk1-host") => gpuclocks::on_host_start(), args.first().map(String::as_str),
_ => None, Some("serve") | Some("punktfunk1-host")
}; ) {
gpuclocks::on_host_start();
}
match args.first().map(String::as_str) { match args.first().map(String::as_str) {
// The host: the native punktfunk/1 plane + management API by default (secure), and — with // The host: the native punktfunk/1 plane + management API by default (secure), and — with
@@ -322,7 +325,33 @@ fn real_main() -> Result<()> {
// `PUNKTFUNK_HDR_SHADER_P010` colour math without green-screening a live HDR stream. Prints // `PUNKTFUNK_HDR_SHADER_P010` colour math without green-screening a live HDR stream. Prints
// PASS/FAIL + max Y/Cb/Cr error. // PASS/FAIL + max Y/Cb/Cr error.
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
Some("hdr-p010-selftest") => crate::capture::dxgi::hdr_p010_selftest(), Some("hdr-p010-selftest") => {
// Optional args: a `WxH` size (default 64x64 — pass the real capture size: heights
// like 1080 are NOT 16-aligned and exercise a different driver path) and a GPU
// vendor (`intel`|`nvidia`|`amd` — dual-GPU boxes otherwise test the default
// adapter, which may not be the one that encodes).
let mut size = (64u32, 64u32);
let mut vendor = None;
for a in args.iter().skip(2) {
match a.as_str() {
"intel" => vendor = Some(0x8086),
"nvidia" => vendor = Some(0x10de),
"amd" => vendor = Some(0x1002),
s => {
let parsed = s
.split_once('x')
.and_then(|(w, h)| Some((w.parse().ok()?, h.parse().ok()?)));
match parsed {
Some(wh) => size = wh,
None => anyhow::bail!(
"hdr-p010-selftest: unrecognized arg {s:?} (want WxH or intel|nvidia|amd)"
),
}
}
}
}
crate::capture::dxgi::hdr_p010_selftest_at(size.0, size.1, vendor)
}
// Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently // Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently
// in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for // in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for
// HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my // HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my
+37 -6
View File
@@ -439,7 +439,11 @@ pub(crate) async fn serve(
) )
.await .await
{ {
Ok(()) => tracing::info!(%peer, "session complete"), Ok(Served::Session) => tracing::info!(%peer, "session complete"),
Ok(Served::ProbeClose) => tracing::debug!(
%peer,
"closed before the control handshake (reachability probe)"
),
Err(e) => { Err(e) => {
tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error") tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error")
} }
@@ -629,6 +633,15 @@ type AudioCapSlot = Arc<std::sync::Mutex<Option<Box<dyn crate::audio::AudioCaptu
/// connection (the host stops waiting at once). /// connection (the host stops waiting at once).
const PENDING_APPROVAL_WAIT: std::time::Duration = std::time::Duration::from_secs(180); const PENDING_APPROVAL_WAIT: std::time::Duration = std::time::Duration::from_secs(180);
/// How a served connection ended. A peer that completes the QUIC handshake and closes cleanly
/// (code 0) without ever opening the control stream is a reachability probe (the clients'
/// hosts-page "online" pips / `--reachable`) or an abandoned connect — routine, and logged
/// quietly: as a WARN it buried the real failures in a wake-on-LAN triage log.
enum Served {
Session,
ProbeClose,
}
/// One client session: handshake → input/audio planes → data plane until done/disconnect. /// One client session: handshake → input/audio planes → data plane until done/disconnect.
/// Everything torn down on return (RAII: virtual output, encoder, threads via channel close). /// Everything torn down on return (RAII: virtual output, encoder, threads via channel close).
/// A connection whose first message is a PairRequest runs the pairing ceremony instead. /// A connection whose first message is a PairRequest runs the pairing ceremony instead.
@@ -651,14 +664,23 @@ async fn serve_session(
// parked knock can't hold a streaming slot. `sem` is the pool it re-acquires from. // parked knock can't hold a streaming slot. `sem` is the pool it re-acquires from.
mut permit: tokio::sync::OwnedSemaphorePermit, mut permit: tokio::sync::OwnedSemaphorePermit,
sem: Arc<tokio::sync::Semaphore>, sem: Arc<tokio::sync::Semaphore>,
) -> Result<()> { ) -> Result<Served> {
let peer = conn.remote_address(); let peer = conn.remote_address();
// First message decides what this connection is: a pairing ceremony or a session. // First message decides what this connection is: a pairing ceremony or a session.
let (mut send, mut recv) = tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi()) let (mut send, mut recv) = match tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi())
.await .await
.map_err(|_| anyhow!("control stream timeout"))? .map_err(|_| anyhow!("control stream timeout"))?
.context("accept control stream")?; {
// A clean close before any control stream: a reachability probe / abandoned connect,
// not a failed session (see [`Served::ProbeClose`]).
Err(quinn::ConnectionError::ApplicationClosed(ref ac))
if ac.error_code == quinn::VarInt::from_u32(0) =>
{
return Ok(Served::ProbeClose);
}
r => r.context("accept control stream")?,
};
let first = tokio::time::timeout(HANDSHAKE_TIMEOUT, io::read_msg(&mut recv)) let first = tokio::time::timeout(HANDSHAKE_TIMEOUT, io::read_msg(&mut recv))
.await .await
.map_err(|_| anyhow!("first message timeout"))??; .map_err(|_| anyhow!("first message timeout"))??;
@@ -709,7 +731,9 @@ async fn serve_session(
} }
*last = Some(std::time::Instant::now()); *last = Some(std::time::Instant::now());
} }
return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin).await; return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin)
.await
.map(|()| Served::Session);
} }
// Pairing gate for a session Hello (a PairRequest was handled above). Lifted OUT of the // Pairing gate for a session Hello (a PairRequest was handled above). Lifted OUT of the
@@ -1175,6 +1199,13 @@ async fn serve_session(
launch: hello.launch.clone(), launch: hello.launch.clone(),
plane: crate::events::Plane::Native, plane: crate::events::Plane::Native,
}); });
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock floor for
// as long as THIS session streams, refcounted with every other live session across both planes.
// RAII like the marker above — armed on the first live client, released when the last one
// disconnects, so idle clocks aren't pinned while nobody is connected. No-op off Linux / when
// the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
// The session's launch, threaded into the data plane. Windows carries the store-qualified id // The session's launch, threaded into the data plane. Windows carries the store-qualified id
// (spawned into the interactive user session once capture is live); other hosts resolve the id // (spawned into the interactive user session once capture is live); other hosts resolve the id
// to its shell command HERE against the host's own library — a client can only ever pick an // to its shell command HERE against the host's own library — a client can only ever pick an
@@ -1392,7 +1423,7 @@ async fn serve_session(
// host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the // host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the
// TV's gaming session back so it's the default when no one is streaming. // TV's gaming session back so it's the default when no one is streaming.
crate::vdisplay::restore_managed_session(); crate::vdisplay::restore_managed_session();
result result.map(|()| Served::Session)
} }
/// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails /// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails
@@ -354,6 +354,28 @@ pub(super) async fn negotiate(
// just follow. // just follow.
let mut salt = [0u8; 4]; let mut salt = [0u8; 4];
rand::thread_rng().fill_bytes(&mut salt); rand::thread_rng().fill_bytes(&mut salt);
// Session AEAD: ChaCha20-Poly1305 when the client asked for it (VIDEO_CAP_CHACHA20 — the
// soft-AES armv7 targets, whose GCM decrypt caps at ~100 Mbps) and the operator
// kill-switch allows (PUNKTFUNK_CHACHA20, default on — pure rollout safety; perf-only,
// both AEADs are full-strength). The fresh-per-session discipline above applies to this
// key identically; the legacy 16-byte `key` stays independently random so nothing
// downstream ever observes an all-zero key.
let client_wants_chacha = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_CHACHA20 != 0;
let chacha = client_wants_chacha && pf_host_config::config().chacha20;
let key_chacha = chacha.then(|| {
let mut k = [0u8; 32];
rand::thread_rng().fill_bytes(&mut k);
k
});
tracing::info!(
cipher = if chacha {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
client_wants_chacha,
"session cipher"
);
let welcome = Welcome { let welcome = Welcome {
abi_version: punktfunk_core::WIRE_VERSION, abi_version: punktfunk_core::WIRE_VERSION,
udp_port, udp_port,
@@ -423,6 +445,16 @@ pub(super) async fn negotiate(
} else { } else {
0 0
}, },
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
// pre-cipher wire form. The host's own data plane picks the cipher up via
// `welcome.session_config` — no other host change.
cipher: if chacha {
punktfunk_core::quic::CIPHER_CHACHA20_POLY1305
} else {
punktfunk_core::quic::CIPHER_AES_128_GCM
},
key_chacha,
}; };
io::write_msg(send, &welcome.encode()).await?; io::write_msg(send, &welcome.encode()).await?;
bringup.mark("welcome"); bringup.mark("welcome");
+74 -12
View File
@@ -1111,6 +1111,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan, plan,
&quit, &quit,
&stop, &stop,
8,
Some(bringup.as_ref()), Some(bringup.as_ref()),
)?; )?;
// Setup done — the IDD-push setup lock releases as the guard leaves this arm's scope, // Setup done — the IDD-push setup lock releases as the guard leaves this arm's scope,
@@ -1426,6 +1427,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan, plan,
&quit, &quit,
&stop, &stop,
8,
None, None,
)?; )?;
Ok((new_vd, pipe)) Ok((new_vd, pipe))
@@ -1522,6 +1524,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
bit_depth, bit_depth,
plan, plan,
&quit, &quit,
// No first-frame shortening here: this direct call has no retry wrapper to
// absorb an early bail, and the resize source is a live compositor (the
// takeover race doesn't apply) — keep the patient default.
None,
Some(resize_trace.as_ref()), Some(resize_trace.as_ref()),
) { ) {
Ok(next_pipe) => { Ok(next_pipe) => {
@@ -1878,7 +1884,15 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// connected, frozen on the last frame, and the stream resumes when the new output // connected, frozen on the last frame, and the stream resumes when the new output
// appears — no reconnect. // appears — no reconnect.
const REBUILD_BUDGET: std::time::Duration = std::time::Duration::from_secs(40); const REBUILD_BUDGET: std::time::Duration = std::time::Duration::from_secs(40);
let rebuild_deadline = std::time::Instant::now() + REBUILD_BUDGET; // Attach-only holdoff: for the first seconds after a capture loss the session
// detection can be STALE (the new session isn't up yet), and a rebuild acting on
// a stale "Gaming" answer restarts gamescope-session.target — which on SteamOS
// steals the seat back from the session the user just switched to (observed
// live). While the holdoff lasts, builds run under a vdisplay rebuild-probe
// scope: attach to live outputs only, never stop/relaunch/take over sessions.
const PROBE_HOLDOFF: std::time::Duration = std::time::Duration::from_secs(4);
let loss_at = std::time::Instant::now();
let rebuild_deadline = loss_at + REBUILD_BUDGET;
let (new_cap, new_enc, new_frame, new_interval, new_node_id, new_display_gen) = loop { let (new_cap, new_enc, new_frame, new_interval, new_node_id, new_display_gen) = loop {
// Follow the active session unless an explicit PUNKTFUNK_COMPOSITOR pin forbids // Follow the active session unless an explicit PUNKTFUNK_COMPOSITOR pin forbids
// retargeting (then we stick to the pinned backend and just rebuild it). // retargeting (then we stick to the pinned backend and just rebuild it).
@@ -1912,6 +1926,8 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
} }
} }
} }
let _probe = (loss_at.elapsed() < PROBE_HOLDOFF)
.then(crate::vdisplay::rebuild_probe_scope);
match build_pipeline_with_retry( match build_pipeline_with_retry(
&mut vd, &mut vd,
cur_mode, cur_mode,
@@ -1920,6 +1936,16 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan, plan,
&quit, &quit,
&stop, &stop,
// 1, not 8: this loop re-detects the active session per iteration — short
// inner cycles are what let it FOLLOW a session switch instead of burning
// retries against a compositor that no longer exists. One attempt per
// cycle also keeps every probe on the SHORT first-frame window (attempt 1
// = 2.5 s): a patient 10 s attempt here just waits on a stale backend
// (observed live: it made a Game→Desktop switch 20 s instead of ~9 —
// the winning KWin rebuild took 0.7 s once detection caught up). The
// slow-new-session case is the OUTER loop's job (40 s budget, fresh
// 2.5 s probes until the new compositor delivers).
1,
None, None,
) { ) {
Ok(p) => break p, Ok(p) => break p,
@@ -1932,6 +1958,9 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
} }
tracing::warn!(error = %format!("{e2:#}"), tracing::warn!(error = %format!("{e2:#}"),
"capture lost — new session not up yet, retrying"); "capture lost — new session not up yet, retrying");
// Probe failures are instant (attach-only bail) — pace the loop so
// re-detection runs at ~2 Hz instead of spinning.
std::thread::sleep(std::time::Duration::from_millis(500));
} }
} }
}; };
@@ -2655,6 +2684,7 @@ pub(super) fn prepare_display(
plan, plan,
quit, quit,
stop, stop,
8,
Some(trace), Some(trace),
)?; )?;
Ok(PreparedDisplay { vd, pipeline }) Ok(PreparedDisplay { vd, pipeline })
@@ -2679,15 +2709,22 @@ fn build_pipeline_with_retry(
plan: crate::session_plan::SessionPlan, plan: crate::session_plan::SessionPlan,
quit: &Arc<AtomicBool>, quit: &Arc<AtomicBool>,
stop: &Arc<AtomicBool>, stop: &Arc<AtomicBool>,
// Retry budget: 8 everywhere EXCEPT the capture-loss rebuild (2). That path wraps this call
// in its own outer loop that RE-DETECTS the active session between calls — during a
// Gaming↔Desktop switch the old compositor is simply gone, so burning 8 attempts (~13 s)
// against its dead socket only delays following the box to the session that replaced it
// (observed live: a Desktop→Gaming switch spent 13 of its 27 s retrying gone-KWin).
max_attempts: u32,
// Transition trace (P0.1): `Some` for the traced builds (bring-up, resize); each stage stamps // Transition trace (P0.1): `Some` for the traced builds (bring-up, resize); each stage stamps
// once (first crossing) so the retry loop can pass it through unconditionally. // once (first crossing) so the retry loop can pass it through unconditionally.
trace: Option<&crate::bringup::Trace>, trace: Option<&crate::bringup::Trace>,
) -> Result<Pipeline> { ) -> Result<Pipeline> {
// ~10s first-frame wait per attempt. 8 gives a ~90s budget for the SLOW case: a host-managed // ~10s first-frame wait per attempt (attempt 1: see FIRST_ATTEMPT_FRAME_BUDGET below). 8
// gamescope session cold-starting Steam Big Picture (the SteamOS/Bazzite takeover) can take // gives a ~80s budget for the SLOW case: a host-managed gamescope session cold-starting Steam
// 30-60s to produce its first frame, and a first-connect timeout would tear down the warm // Big Picture (the SteamOS/Bazzite takeover) can take 30-60s to produce its first frame, and
// session (forcing another cold start on reconnect). A genuinely permanent failure still fails // a first-connect timeout would tear down the warm session (forcing another cold start on
// fast via `is_permanent_build_error`; only transient "no frame yet" retries consume the budget. // reconnect). A genuinely permanent failure still fails fast via `is_permanent_build_error`;
// only transient "no frame yet" retries consume the budget.
// IDD-push only: HOLD one monitor lease across all build attempts. A failed attempt's capturer // IDD-push only: HOLD one monitor lease across all build attempts. A failed attempt's capturer
// drop releases ITS lease, but this held lease keeps the shared monitor Active (refs >= 1), so the // drop releases ITS lease, but this held lease keeps the shared monitor Active (refs >= 1), so the
// next attempt's `vd.create` JOINS it (refcount++) instead of finding it Lingering and tripping the // next attempt's `vd.create` JOINS it (refcount++) instead of finding it Lingering and tripping the
@@ -2705,9 +2742,16 @@ fn build_pipeline_with_retry(
} else { } else {
None None
}; };
const MAX_ATTEMPTS: u32 = 8; // Attempt 1 waits only briefly for the first frame: a PipeWire stream connected while
// gamescope re-initializes its headless takeover negotiates a format and reaches `Streaming`
// but never receives a buffer — a FRESH connect then delivers within ~0.5 s (observed on
// SteamOS: every gamescope bring-up burned the full 10 s on attempt 1, then attempt 2 got
// frames instantly → 17 s bring-ups). Healthy compositors deliver the first frame well inside
// this window (KWin ~0.3 s), and the genuinely-slow cold start above still gets the patient
// 10 s window on every later attempt.
const FIRST_ATTEMPT_FRAME_BUDGET: std::time::Duration = std::time::Duration::from_millis(2500);
let mut backoff = std::time::Duration::from_millis(500); let mut backoff = std::time::Duration::from_millis(500);
for attempt in 1..=MAX_ATTEMPTS { for attempt in 1..=max_attempts {
// The client is gone (connection closed → `stop`): every further attempt only churns the // The client is gone (connection closed → `stop`): every further attempt only churns the
// box for a session no one is watching — on a Bazzite takeover that means SIGKILLing and // box for a session no one is watching — on a Bazzite takeover that means SIGKILLing and
// relaunching the box's Steam session once per attempt for minutes (the .181 storm // relaunching the box's Steam session once per attempt for minutes (the .181 storm
@@ -2719,7 +2763,17 @@ fn build_pipeline_with_retry(
attempt - 1 attempt - 1
); );
} }
match build_pipeline(vd, mode, bitrate_kbps, bit_depth, plan, quit, trace) { let first_frame_budget = (attempt == 1).then_some(FIRST_ATTEMPT_FRAME_BUDGET);
match build_pipeline(
vd,
mode,
bitrate_kbps,
bit_depth,
plan,
quit,
first_frame_budget,
trace,
) {
Ok(pipe) => { Ok(pipe) => {
if attempt > 1 { if attempt > 1 {
tracing::info!(attempt, "pipeline up after retry"); tracing::info!(attempt, "pipeline up after retry");
@@ -2729,7 +2783,7 @@ fn build_pipeline_with_retry(
Err(e) => { Err(e) => {
let chain = format!("{e:#}"); let chain = format!("{e:#}");
let permanent = is_permanent_build_error(&chain); let permanent = is_permanent_build_error(&chain);
if permanent || attempt == MAX_ATTEMPTS { if permanent || attempt == max_attempts {
let why = if permanent { let why = if permanent {
"permanent" "permanent"
} else { } else {
@@ -2741,7 +2795,7 @@ fn build_pipeline_with_retry(
} }
tracing::warn!( tracing::warn!(
attempt, attempt,
max = MAX_ATTEMPTS, max = max_attempts,
backoff_ms = backoff.as_millis() as u64, backoff_ms = backoff.as_millis() as u64,
error = %chain, error = %chain,
"pipeline build failed — retrying" "pipeline build failed — retrying"
@@ -2802,6 +2856,10 @@ fn build_pipeline(
bit_depth: u8, bit_depth: u8,
plan: crate::session_plan::SessionPlan, plan: crate::session_plan::SessionPlan,
quit: &Arc<AtomicBool>, quit: &Arc<AtomicBool>,
// First-frame wait override (`None` = the backend's default 10 s): the retry loop shortens
// its FIRST attempt so a stream stuck in the gamescope takeover race fails over to the
// reconnect that fixes it (see FIRST_ATTEMPT_FRAME_BUDGET in `build_pipeline_with_retry`).
first_frame_budget: Option<std::time::Duration>,
// Transition trace (P0.1): stamps the build's stages (display acquire, capture attach, first // Transition trace (P0.1): stamps the build's stages (display acquire, capture attach, first
// frame, encoder open) into the bring-up/resize timeline. `None` on untraced rebuilds. // frame, encoder open) into the bring-up/resize timeline. `None` on untraced rebuilds.
trace: Option<&crate::bringup::Trace>, trace: Option<&crate::bringup::Trace>,
@@ -2857,7 +2915,11 @@ fn build_pipeline(
t.mark("capture_attached"); t.mark("capture_attached");
} }
capturer.set_active(true); capturer.set_active(true);
let frame = match capturer.next_frame().context("first frame") { let first = match first_frame_budget {
Some(budget) => capturer.next_frame_within(budget),
None => capturer.next_frame(),
};
let frame = match first.context("first frame") {
Ok(f) => f, Ok(f) => f,
Err(e) => { Err(e) => {
// A reused kept display was dead — invalidate it so the next attempt creates fresh (A2). // A reused kept display was dead — invalidate it so the next attempt creates fresh (A2).
@@ -179,6 +179,13 @@ impl SessionPlan {
// Vulkan device; on Linux the capture facade flips the zero-copy policy to the // Vulkan device; on Linux the capture facade flips the zero-copy policy to the
// raw-dmabuf passthrough (see above). // raw-dmabuf passthrough (see above).
pyrowave: self.codec == crate::encode::Codec::PyroWave, pyrowave: self.codec == crate::encode::Codec::PyroWave,
// Producer-native NV12 (gamescope) is consumable only by the Linux Vulkan Video
// backend — resolved HERE from the plan's codec so the capturer never reaches back
// into encode (the same one-way edge as `gpu` above).
#[cfg(target_os = "linux")]
nv12_native: crate::encode::linux_native_nv12_ok(self.codec),
#[cfg(not(target_os = "linux"))]
nv12_native: false,
} }
} }
} }
+60 -9
View File
@@ -168,12 +168,29 @@ The canonical "decide, don't just observe" pattern — approve pairing from your
## Recipe: full controller passthrough (VirtualHere) ## Recipe: full controller passthrough (VirtualHere)
To get a controller's *native* features on the host — DualSense gyro, touchpad, adaptive To get a controller's *native* features on the host — DualSense gyro, touchpad, adaptive
triggers, USB rumble — instead of the emulated pad, share the physical device from the couch with triggers, USB rumble — instead of the emulated pad, hand the physical device from the couch to the
[VirtualHere](https://www.virtualhere.com/) (USB-over-IP) and bind it to the host only while a host over [VirtualHere](https://www.virtualhere.com/) (USB-over-IP), bound only while a client is
client is connected. The couch runs the VirtualHere **server** (sharing the pad); the host runs connected so the couch keeps its own controller the rest of the time.
the VirtualHere **client** and this automation drives its `-t` IPC.
Zero-code, bracketed on the stream with two hooks: **The two sides.** VirtualHere is a server/client pair, and you run both:
- **Server — on the couch** (where the pad is physically plugged in). Run the VirtualHere USB
Server there; it shares the pad on the LAN. Leave it running.
- **Client — on the host** (where the game and this automation run). Install the VirtualHere
Client (as a service, or the tray app). It auto-discovers the couch's shared pad on the same
LAN; across subnets, add it once with `<VH_CLIENT> -t "MANUAL HUB ADD,<couch-ip>:7575"`. The
client binary is `vhclientx86_64` on Linux, `vhui64.exe` on Windows, `vhclientosx` on macOS,
`vhclientarm64` on ARM Linux.
The client's `-t` flag is a one-shot IPC to the already-running client: `-t LIST` prints every
visible device with its address (`server.port`, e.g. `couch-deck.11`); `-t "USE,<addr>"` mounts it
onto the host; `-t "STOP USING,<addr>"` hands it back. The automation just brackets
`USE` / `STOP USING` around a session.
### Zero-code: two hooks
Bracket it on the stream with two [hooks](#hooks-hooksjson) — mount when video starts, release
when it stops. This is all most setups need:
```json ```json
{ {
@@ -184,8 +201,42 @@ Zero-code, bracketed on the stream with two hooks:
} }
``` ```
`couch-deck.11` is the device's VirtualHere address (`vhclientx86_64 -t LIST`); same-LAN setups `couch-deck.11` is the device's VirtualHere address from `vhclientx86_64 -t LIST`.
auto-discover it, otherwise `MANUAL HUB ADD,<couch-ip>:7575` once. For a version that resolves the
device by name, filters to one couch, and releases the pad on a clean shutdown, see the ### Scripted: resolve by name, release on shutdown
The hooks above hard-code the address, and can strand the pad on the host if it's stopped
mid-stream (the `stream.stopped` hook never fires). The
[`virtualhere-dualsense.ts`](https://git.unom.io/unom/punktfunk/src/branch/main/sdk/examples/virtualhere-dualsense.ts) [`virtualhere-dualsense.ts`](https://git.unom.io/unom/punktfunk/src/branch/main/sdk/examples/virtualhere-dualsense.ts)
SDK example. SDK example is the robust version: it resolves the device by **name substring** (survives the
address changing), can filter to **one couch** in a multi-client setup, and **releases the pad on
a clean shutdown** (`systemctl stop`, ^C) so the couch always gets its controller back.
It's a standalone [`@punktfunk/host` script](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
— run it in its own directory. The one edit is its import: the in-repo copy imports from
`../src/index.js`; outside the repo it's the published package:
```sh
mkdir ~/punktfunk-scripts && cd ~/punktfunk-scripts
bun init -y
bun add @punktfunk/host # point the @punktfunk scope at the registry first — see the SDK README
# save the example as virtualhere-dualsense.ts, and change its first import to the package:
# - import { connect } from "../src/index.js";
# + import { connect } from "@punktfunk/host";
VH_DEVICE=DualSense bun virtualhere-dualsense.ts
```
`VH_DEVICE` is required — a VirtualHere address (`couch-deck.11`) or a device-name substring
(`DualSense`). Optional: `VH_CLIENT` overrides the client binary (default `vhclientx86_64`);
`VH_ONLY_CLIENT` binds only for one punktfunk client label. Running on the host box the SDK needs
no token or URL — it reads the host's loopback credentials itself (see
[Connection resolution](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#connection-resolution)).
Keep it running as a
[systemd user unit](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
(its default `SIGTERM` triggers the script's own release step — so `systemctl stop` hands the pad
back), or drop it under the [scripting runner](/docs/plugins) with your other units.
> The example brackets on `client.connected` / `client.disconnected` — the pad returns to the
> couch the moment they disconnect. Switch to `stream.started` / `stream.stopped` if you'd rather
> pass it through only while video is actually flowing; both are noted in the file's header.
+1
View File
@@ -90,6 +90,7 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
| `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. | | `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. |
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. | | `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. |
| `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. | | `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
| `PUNKTFUNK_CHACHA20` | `1` · `0` *(default on)* | ChaCha20-Poly1305 session encryption for clients without hardware AES (old ARM TVs, e.g. webOS), lifting their ~100 Mbps software-AES decrypt ceiling. **On by default** on the host; a session uses it only when the client requests it — everyone else stays on AES-GCM. Purely a performance choice (both ciphers are full-strength); set `0` to force AES-GCM for all sessions. |
| `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. | | `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. |
| `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. | | `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. |
| `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. | | `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. |
+18 -2
View File
@@ -58,8 +58,10 @@ It is idempotent — safe to re-run. In one pass it:
1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain, 1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain,
2. builds `punktfunk-host` (and the web console), 2. builds `punktfunk-host` (and the web console),
3. writes config to `~/.config/punktfunk/` (a generated web-console login password), 3. writes config to `~/.config/punktfunk/` (a generated web-console login password),
4. raises the UDP socket buffers to 32 MB and adds you to the `input` group (needs `sudo`; skipped 4. raises the UDP socket buffers to 32 MB, installs the gamepad udev rule + the `vhci-hcd` autoload
with a warning if unavailable), and adds you to the `input` group (virtual gamepads / **native Steam Deck controller passthrough**),
and seeds the KDE RemoteDesktop grant for Desktop-mode input — this step **prompts for your `sudo`
password** (a stock Steam Deck requires one; without it gamepad passthrough and the UDP tuning are skipped),
5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger, 5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger,
so they run without a login session). so they run without a login session).
@@ -82,6 +84,14 @@ When it finishes it prints the web-console URL and how to pair.
> If you only ever use native clients, install with `--no-gamestream` for a host with no GameStream > If you only ever use native clients, install with `--no-gamestream` for a host with no GameStream
> surface at all. > surface at all.
> **First install — reboot once before streaming.** KWin only authorizes Desktop-mode screen capture
> on a fresh session, and the new `input` group (native Steam Deck controller passthrough) only takes
> effect on a new login — so after the **first** install, **reboot the Deck** (a re-run that changes
> nothing doesn't need it). Streaming **Game Mode** with a generic Xbox pad works right away; **Desktop
> capture and the native Steam Deck controller need the reboot.** If a client connects and every
> session ends with `KWin does not expose zkde_screencast_unstable_v1` or the pad shows up as an Xbox
> 360 controller, you haven't rebooted yet.
## 3. Pair a device ## 3. Pair a device
By default the host **requires PIN pairing** (secure). Two ways to pair: By default the host **requires PIN pairing** (secure). Two ways to pair:
@@ -128,6 +138,12 @@ bash ~/punktfunk/scripts/steamdeck/update.sh
thrash the managed session. Pick one mode per session. thrash the managed session. Pick one mode per session.
- **Keep the device awake.** On handhelds, Game Mode auto-suspends on idle, which drops the host off - **Keep the device awake.** On handhelds, Game Mode auto-suspends on idle, which drops the host off
the network mid stream — disable auto-suspend (Settings → Power) for a headless host. the network mid stream — disable auto-suspend (Settings → Power) for a headless host.
- **Native Steam Deck controller passthrough** presents the client's pad as a real Steam Deck
controller (paddles, trackpads, gyro) via a virtual USB device — that needs the `input` group and the
`vhci-hcd` module live, so it only works **after the first-install reboot** above; until then the pad
degrades to a generic Xbox 360 controller (still fully playable). If you're streaming *to* another
Steam Deck, also set Steam Input to **Off** for Punktfunk on that Deck — see
[Stream to a Steam Deck](/docs/steam-deck).
- **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to - **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to
start after an update, just re-run `update.sh` to rebuild against the new base. start after an update, just re-run `update.sh` to rebuild against the new base.
- Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md). - Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md).
+27 -1
View File
@@ -272,7 +272,7 @@
#define INBOUND_REQ_FLAG 2147483648 #define INBOUND_REQ_FLAG 2147483648
#endif #endif
// 16-byte AEAD authentication tag appended by GCM. // 16-byte AEAD authentication tag appended by either session cipher.
#define TAG_LEN 16 #define TAG_LEN 16
// Wire tag distinguishing an input datagram from a video packet. // Wire tag distinguishing an input datagram from a video packet.
@@ -465,6 +465,20 @@
#define VIDEO_CAP_STREAMED_AU 32 #define VIDEO_CAP_STREAMED_AU 32
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
// control channel, so there is no downgrade surface.
#define VIDEO_CAP_CHACHA20 64
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] // [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder // (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -855,6 +869,18 @@
#define HELLO_LAUNCH_MAX 128 #define HELLO_LAUNCH_MAX 128
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
// only one pre-cipher builds know).
#define CIPHER_AES_128_GCM 0
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
#define CIPHER_CHACHA20_POLY1305 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// Type byte of [`PairRequest`]. // Type byte of [`PairRequest`].
#define MSG_PAIR_REQUEST 16 #define MSG_PAIR_REQUEST 16
+106 -11
View File
@@ -48,6 +48,9 @@ BIN="$TARGET_DIR/release/punktfunk-host"
CONFIG="$HOME/.config/punktfunk" CONFIG="$HOME/.config/punktfunk"
UNITS="$HOME/.config/systemd/user" UNITS="$HOME/.config/systemd/user"
XRD="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}" XRD="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"
# Set when this run does something that only a fresh login picks up (input-group add, first-time
# KWin .desktop grant). Drives the loud "reboot before streaming" note in the summary.
NEED_RELOGIN=0
# --- 0. preflight ---------------------------------------------------------- # --- 0. preflight ----------------------------------------------------------
log "Preflight" log "Preflight"
@@ -66,6 +69,33 @@ fi
DISTROBOX="$(command -v distrobox)" # baked into the web unit (may be /usr/bin or ~/.local/bin) DISTROBOX="$(command -v distrobox)" # baked into the web unit (may be /usr/bin or ~/.local/bin)
ok "distrobox: $DISTROBOX" ok "distrobox: $DISTROBOX"
# --- acquire sudo up front (before the ~15-min build) ----------------------
# Steps 4-5 (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger) need root. Prompt NOW,
# not after the build — so you authorize once and walk away, and a non-interactive run fails LOUDLY
# here instead of silently skipping the tuning at the very end. A stock SteamOS 'deck' has no
# password, so sudo can't work until you set one.
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo is needed once (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger):"
if sudo -v; then
SUDO_OK=1
# keep the sudo timestamp warm across the long build so steps 4-5 don't re-prompt / expire
( while sudo -n -v 2>/dev/null; do sleep 50; done ) &
_pf_sudo_keepalive=$!
trap '[ -n "${_pf_sudo_keepalive:-}" ] && kill "$_pf_sudo_keepalive" 2>/dev/null || true' EXIT
fi
fi
if [ "$SUDO_OK" != 1 ]; then
if [ -t 0 ]; then
warn "No sudo — a stock SteamOS 'deck' account has no password. Set one and re-run: passwd"
else
warn "No TTY for the sudo prompt (non-interactive run) — system tuning + linger will be SKIPPED."
warn "Run in Konsole or an interactive 'ssh -t' session (or pre-authorize sudo) to enable them."
fi
fi
# --- 1. build container + toolchain --------------------------------------- # --- 1. build container + toolchain ---------------------------------------
log "Build container '$BOX' ($BOX_IMAGE)" log "Build container '$BOX' ($BOX_IMAGE)"
if distrobox list 2>/dev/null | awk -F'|' '{gsub(/ /,"",$2); print $2}' | grep -qx "$BOX"; then if distrobox list 2>/dev/null | awk -F'|' '{gsub(/ /,"",$2); print $2}' | grep -qx "$BOX"; then
@@ -84,7 +114,7 @@ set -e
export DEBIAN_FRONTEND=noninteractive export DEBIAN_FRONTEND=noninteractive
sudo apt-get update -qq sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends \ sudo apt-get install -y -qq --no-install-recommends \
build-essential pkg-config clang curl git ca-certificates \ build-essential pkg-config clang cmake curl git ca-certificates \
libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev libswscale-dev libavdevice-dev \ libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev libswscale-dev libavdevice-dev \
libpipewire-0.3-dev libspa-0.2-dev \ libpipewire-0.3-dev libspa-0.2-dev \
libgbm-dev libegl-dev libgl-dev libdrm-dev libva-dev \ libgbm-dev libegl-dev libgl-dev libdrm-dev libva-dev \
@@ -101,10 +131,13 @@ ok "build deps ready"
# --- 2. build host (+ web) ------------------------------------------------- # --- 2. build host (+ web) -------------------------------------------------
log "Building punktfunk-host (release) — first build is slow (~10-15 min)" log "Building punktfunk-host (release) — first build is slow (~10-15 min)"
# vulkan-encode matches the packaged builds (deb/arch): the raw Vulkan Video HEVC/AV1 backend
# (real RFI loss recovery). Pure-Rust ash — no extra system dep. A featureless hand build would
# silently fall back to libav VAAPI.
distrobox enter "$BOX" -- bash -lc " distrobox enter "$BOX" -- bash -lc "
set -e set -e
export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR' export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'
cd '$SRC' && cargo build -r -p punktfunk-host cd '$SRC' && cargo build -r -p punktfunk-host --features punktfunk-host/vulkan-encode
" "
[ -x "$BIN" ] || die "build did not produce $BIN" [ -x "$BIN" ] || die "build did not produce $BIN"
ok "host binary: $BIN" ok "host binary: $BIN"
@@ -126,8 +159,12 @@ mkdir -p "$CONFIG"
if [ ! -f "$CONFIG/host.env" ]; then if [ ! -f "$CONFIG/host.env" ]; then
cat > "$CONFIG/host.env" <<'EOF' cat > "$CONFIG/host.env" <<'EOF'
# punktfunk Steam Deck host config (sourced by the punktfunk-host user service). # punktfunk Steam Deck host config (sourced by the punktfunk-host user service).
# Auto encoder: VAAPI on the Deck's AMD GPU, NVENC on NVIDIA. # Auto encoder: Vulkan Video (or VAAPI fallback) on the Deck's AMD GPU, NVENC on NVIDIA.
PUNKTFUNK_ENCODER=auto PUNKTFUNK_ENCODER=auto
# Van Gogh (LCD/OLED Deck) RADV still gates VK_KHR_video_encode_* behind this perftest flag;
# without it the Vulkan backend can't open and sessions fall back to libav VAAPI. Harmless on
# GPUs where encode is exposed by default.
RADV_PERFTEST=video_encode
# The host auto-detects the live session (Game Mode gamescope / Desktop KDE) per connect. # The host auto-detects the live session (Game Mode gamescope / Desktop KDE) per connect.
# Override the compositor only if detection misbehaves: PUNKTFUNK_COMPOSITOR=gamescope # Override the compositor only if detection misbehaves: PUNKTFUNK_COMPOSITOR=gamescope
EOF EOF
@@ -136,6 +173,34 @@ else
ok "host.env exists (left as-is)" ok "host.env exists (left as-is)"
fi fi
# KWin authorization for Desktop-Mode streaming (and mid-stream Game↔Desktop switches): KWin
# resolves a connecting client's /proc/<pid>/exe against a .desktop `Exec=` and only then grants
# the restricted Wayland globals it lists (see packaging/linux/io.unom.Punktfunk.Host.desktop).
# Exec must therefore be THIS install's binary path, not the packaged /usr/bin one. KWin reads
# grants at session start — after first install, restart the Desktop session (Game Mode and back).
DESKTOP_DST="$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
# First-time install of the grant: KWin only reads it at session start, so a fresh login is required
# before Desktop-mode capture works. A re-run that just rewrites it needs no relogin.
[ -f "$DESKTOP_DST" ] || NEED_RELOGIN=1
mkdir -p "$HOME/.local/share/applications"
sed "s|^Exec=.*|Exec=$BIN|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" > "$DESKTOP_DST"
ok "KWin desktop-capture authorization (io.unom.Punktfunk.Host.desktop → $BIN)"
# KDE Desktop-mode INPUT: a normal Plasma login lacks the RemoteDesktop portal grant the host's libei
# input path needs, so it would pop an "Allow remote control?" dialog a headless host can't answer.
# Seed it once (per-user, no root) — mirrors packaging/bazzite/kde-desktop-setup.sh. Game Mode
# (gamescope) needs none of this; the .desktop above already grants org_kde_kwin_fake_input.
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ -s "$GRANT_DST" ]; then
ok "KDE RemoteDesktop grant already present"
elif [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
systemctl --user restart xdg-permission-store 2>/dev/null || true
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
if [ "$WITH_WEB" = 1 ] && [ ! -f "$CONFIG/web.env" ]; then if [ "$WITH_WEB" = 1 ] && [ ! -f "$CONFIG/web.env" ]; then
# Random login password + session secret for the web console, generated once. # Random login password + session secret for the web console, generated once.
# `|| true` swallows the SIGPIPE `tr` takes when `head` closes the pipe (pipefail would abort). # `|| true` swallows the SIGPIPE `tr` takes when `head` closes the pipe (pipefail would abort).
@@ -151,9 +216,11 @@ else
[ "$WITH_WEB" = 1 ] && ok "web.env exists (login password unchanged)" [ "$WITH_WEB" = 1 ] && ok "web.env exists (login password unchanged)"
fi fi
# --- 4. system tuning (needs sudo; skipped gracefully if unavailable) ------ # --- 4. system tuning (needs sudo: UDP buffers + gamepad udev rule + vhci-hcd + input group) --------
log "System tuning (UDP buffers + input group) — needs sudo" log "System tuning (UDP buffers + gamepad rules + vhci-hcd + input group)"
if sudo -n true 2>/dev/null; then # sudo was acquired up front in preflight (SUDO_OK) so this never stalls behind the long build; a
# skip here (no password / no TTY) was already reported loudly there.
if [ "$SUDO_OK" = 1 ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' \ printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' \
| sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null | sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
@@ -161,13 +228,34 @@ if sudo -n true 2>/dev/null; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules && sudo udevadm trigger || true sudo udevadm control --reload-rules && sudo udevadm trigger || true
ok "installed udev rule (virtual gamepads)" ok "installed udev rule (virtual gamepads + native Steam Deck controller)"
fi
# vhci-hcd: the usbip transport that makes the virtual Steam Deck pad a *real* USB device so Steam
# Input adopts it (else it degrades to plain UHID, which Steam ignores — "no controller appears").
# Persist the autoload AND load it now so passthrough works without waiting for a reboot.
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || warn "could not load vhci-hcd now (loads on next boot) — needed for the native Steam Deck pad"
ok "vhci-hcd autoload installed (native Steam Deck controller transport)"
fi
if id -nG "$USER" | grep -qw input; then
ok "already in the 'input' group"
else
sudo usermod -aG input "$USER"
NEED_RELOGIN=1
warn "added $USER to the 'input' group (applies on next login)"
fi fi
id -nG "$USER" | grep -qw input || { sudo usermod -aG input "$USER"; warn "added $USER to 'input' group — log out/in (or reboot) for gamepad support"; }
else else
warn "passwordless sudo unavailable — skipping UDP-buffer + udev tuning." warn "no usable sudo — SKIPPED system tuning. Gamepad passthrough + clean streaming need root (udev"
warn "Without it, high-bitrate streaming drops packets. Apply manually later:" warn "rule, 'input' group, vhci-hcd, UDP buffers) — there is no user-space way to do these."
warn " echo -e 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf && sudo sysctl --system" warn "A stock SteamOS 'deck' account has NO password, so sudo can't work until you set one:"
warn " passwd # set a sudo password once, then re-run this script"
warn "Or apply it by hand (then reboot):"
warn " sudo install -m644 $SRC/scripts/60-punktfunk.rules /etc/udev/rules.d/ &&"
warn " sudo install -m644 $SRC/scripts/punktfunk-modules.conf /etc/modules-load.d/punktfunk.conf &&"
warn " sudo usermod -aG input $USER &&"
warn " printf 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432\\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf &&"
warn " sudo sysctl --system && sudo udevadm control --reload-rules && sudo udevadm trigger"
fi fi
# --- 5. systemd user services --------------------------------------------- # --- 5. systemd user services ---------------------------------------------
@@ -248,3 +336,10 @@ else
echo " • Pairing required (secure default). From a client, pick this host and enter the PIN the host shows." echo " • Pairing required (secure default). From a client, pick this host and enter the PIN the host shows."
fi fi
echo " • Update later: bash $SRC/scripts/steamdeck/update.sh" echo " • Update later: bash $SRC/scripts/steamdeck/update.sh"
if [ "$NEED_RELOGIN" = 1 ]; then
echo
warn "ONE MORE STEP before streaming — reboot the Deck (or fully log out and back in)."
echo " KWin only authorizes Desktop-mode screen capture on a fresh session, and the new 'input'"
echo " group (native Steam Deck controller passthrough) only applies to a new login. Streaming"
echo " Game Mode with a generic Xbox pad works now; Desktop capture + the native Deck pad need the reboot."
fi
+65 -1
View File
@@ -21,7 +21,8 @@ if [ "${1:-}" = "--pull" ]; then
fi fi
log "Rebuilding host (release)" log "Rebuilding host (release)"
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'; cd '$SRC' && cargo build -r -p punktfunk-host" # vulkan-encode matches the packaged builds (deb/arch) — see install.sh.
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'; cd '$SRC' && cargo build -r -p punktfunk-host --features punktfunk-host/vulkan-encode"
ok "host rebuilt" ok "host rebuilt"
if [ "$WEB" = 1 ]; then if [ "$WEB" = 1 ]; then
log "Rebuilding web console" log "Rebuilding web console"
@@ -29,6 +30,69 @@ if [ "$WEB" = 1 ]; then
ok "web rebuilt" ok "web rebuilt"
fi fi
# Retrofit config that install.sh now writes but older installs predate (both idempotent):
# RADV_PERFTEST — Van Gogh RADV still gates VK_KHR_video_encode_* behind it; without it the
# Vulkan backend can't open and sessions silently fall back to libav VAAPI. The KWin .desktop —
# KWin only grants the restricted capture/input globals to the exe a .desktop authorizes.
HOST_ENV="$HOME/.config/punktfunk/host.env"
if [ -f "$HOST_ENV" ] && ! grep -q '^RADV_PERFTEST=' "$HOST_ENV"; then
printf '\n# Van Gogh RADV gates VK_KHR_video_encode_* behind this (Vulkan Video encode).\nRADV_PERFTEST=video_encode\n' >> "$HOST_ENV"
ok "host.env: added RADV_PERFTEST=video_encode"
fi
mkdir -p "$HOME/.local/share/applications"
sed "s|^Exec=.*|Exec=$TARGET_DIR/release/punktfunk-host|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" \
> "$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
ok "KWin desktop-capture authorization refreshed"
# Retrofit the system bits install.sh now sets up but older installs predate (idempotent). vhci-hcd =
# usbip transport for the native Steam Deck pad; 60-punktfunk.rules = /dev/uhid + vhci access; input
# group = uhid write; the kde-authorized grant (per-user, no root) = Desktop-mode input. A stock Deck
# needs a sudo PASSWORD, so PROMPT for it rather than silently skipping (skipping = gamepads stay dead).
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo needs your password to (re)apply the gamepad udev rule, vhci-hcd, input group, and UDP buffers:"
sudo -v && SUDO_OK=1 || true
fi
if [ "$SUDO_OK" = 1 ]; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules >/dev/null 2>&1 || true
sudo udevadm trigger >/dev/null 2>&1 || true
ok "gamepad udev rule ensured"
fi
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || true
ok "vhci-hcd autoload ensured (native Steam Deck controller)"
fi
# UDP buffers: older installs (or sudo-skipped ones) still run the stock 416 KB cap.
if [ ! -f /etc/sysctl.d/99-punktfunk-net.conf ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null 2>&1 || true
ok "UDP socket buffers raised to 32 MB (persisted)"
fi
if id -nG "$USER" | grep -qw input; then :; else
sudo usermod -aG input "$USER"
warn "added $USER to the 'input' group — REBOOT (or log out/in) for it to apply"
fi
else
warn "no usable sudo — SKIPPED gamepad/udev/vhci/UDP tuning (all root-only; no user-space alternative)."
warn "A stock SteamOS 'deck' account has NO password — set one with 'passwd', then re-run. Gamepads stay"
warn "Xbox-360 until this runs and you reboot."
fi
echo
warn "If the controller still shows as an Xbox 360 pad, REBOOT the Deck once — the 'input' group and the"
warn "vhci-hcd module only become live for the host service on a fresh login."
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ ! -s "$GRANT_DST" ] && [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
log "Restarting services" log "Restarting services"
systemctl --user restart punktfunk-host.service systemctl --user restart punktfunk-host.service
ok "punktfunk-host restarted" ok "punktfunk-host restarted"
+4 -1
View File
@@ -89,7 +89,10 @@ A complexity ladder in [`examples/`](./examples) — start at the top:
4. [`couch-preset.effect.ts`](./examples/couch-preset.effect.ts) — **advanced, Effect-native**: only if you're composing Effect programs. 4. [`couch-preset.effect.ts`](./examples/couch-preset.effect.ts) — **advanced, Effect-native**: only if you're composing Effect programs.
Examples 13 are the plain Promise facade and cover most automation; you only need example 4's Examples 13 are the plain Promise facade and cover most automation; you only need example 4's
Effect surface for composed, interruptible programs. Run any with `bun examples/<file>.ts`. Effect surface for composed, interruptible programs. Run any **in the repo** with
`bun examples/<file>.ts`. To **deploy** one on a host, install the package into its own directory
(`bun add @punktfunk/host`) and change its `../src/…` import to `@punktfunk/host` — see
[Running a single script as a service](#running-a-single-script-as-a-service).
Plus a real-world recipe: Plus a real-world recipe:
+5
View File
@@ -15,6 +15,11 @@
// VH_DEVICE=couch-deck.11 bun examples/virtualhere-dualsense.ts # address from `-t LIST` // VH_DEVICE=couch-deck.11 bun examples/virtualhere-dualsense.ts # address from `-t LIST`
// VH_DEVICE=DualSense bun examples/virtualhere-dualsense.ts # …or match by name substring // VH_DEVICE=DualSense bun examples/virtualhere-dualsense.ts # …or match by name substring
// //
// To run this *outside* the repo (the normal case), drop it in its own dir, `bun add
// @punktfunk/host`, and change the import below from `../src/index.js` to `@punktfunk/host`; then
// keep it alive as a systemd user unit (its SIGTERM release hands the pad back on `systemctl
// stop`). Full walkthrough: docs → Events & hooks → "full controller passthrough (VirtualHere)".
//
// Env: VH_DEVICE required — a VirtualHere address (`server.port`) or a device-name substring. // Env: VH_DEVICE required — a VirtualHere address (`server.port`) or a device-name substring.
// VH_CLIENT client binary. Default `vhclientx86_64` (Linux); Windows `vhui64.exe`, // VH_CLIENT client binary. Default `vhclientx86_64` (Linux); Windows `vhui64.exe`,
// macOS `vhclientosx`, ARM Linux `vhclientarm64`. // macOS `vhclientosx`, ARM Linux `vhclientarm64`.
+2 -1
View File
@@ -6,6 +6,7 @@
//! harness adds `tc netem` jitter/reorder on the UDP path. //! harness adds `tc netem` jitter/reorder on the UDP path.
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionKey;
use punktfunk_core::error::PunktfunkError; use punktfunk_core::error::PunktfunkError;
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
use punktfunk_core::transport::loopback_pair; use punktfunk_core::transport::loopback_pair;
@@ -25,7 +26,7 @@ fn config(role: Role, scheme: FecScheme, drop_period: u32) -> Config {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: drop_period, loopback_drop_period: drop_period,
} }