Compare commits

...

29 Commits

Author SHA1 Message Date
enricobuehler 1efb8aef74 feat(core+host+client): cursor channel — remote-desktop sweep M2a+M2b
ci / web (pull_request) Successful in 1m8s
ci / docs-site (pull_request) Successful in 1m10s
apple / swift (pull_request) Successful in 1m23s
apple / screenshots (pull_request) Has been skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Failing after 6m44s
ci / bench (pull_request) Successful in 7m24s
android / android (pull_request) Failing after 7m47s
ci / rust (pull_request) Failing after 8m36s
windows / build (x86_64-pc-windows-msvc) (pull_request) Failing after 4m36s
The host cursor stops riding the video and becomes a real OS cursor on
the client (the Parsec/RDP model): pointer feel no longer pays the
capture→encode→network→decode→present round trip.

Wire (M2a):
- Hello grows a client_caps trailing byte (CLIENT_CAP_CURSOR) after the
  fixed display_hdr block — presence disambiguated by remaining length,
  which caps the post-HDR tail at 27 bytes (documented); Welcome answers
  HOST_CAP_CURSOR (capable-and-asked, the 444/clipboard precedent).
- CursorShape (0x50, control stream): serial + dims + hotspot + straight
  RGBA, ≤120px/side so the u16 frame always fits (128² would overshoot);
  client caches by serial — re-showing a known shape costs 14 bytes, not
  a bitmap (RDP pointer-cache for free).
- CursorState (0xD0 datagram): serial + visible/relative_hint flags +
  position, sent once per encode-loop tick — latest-wins, self-healing
  under loss, no refresh timer. relative_hint is reserved for M3.
- Client core: two new planes (control-task + datagram-task arms) →
  next_cursor_shape/next_cursor_state; connect() grows client_caps
  (C ABI passes 0 until the v11 cursor poll fns exist).

Host (M2b, Linux portal only):
- handshake::cursor_forward is THE predicate (client asked ∧ Linux ∧
  compositor ≠ gamescope) — Welcome bit and session wiring both read it.
- SessionPlan.cursor_blend goes false for a forwarding session; the
  encode loop ticks a CursorForwarder every iteration: shape-serial diff
  → control-task bridge (mirrors probe_result), state datagram → conn.
- CursorOverlay/capture CursorState carry the hotspot through
  (nearest-neighbor downscale backstop for XL cursors, unit-tested).

Presenter:
- CursorChannel drains both planes per loop iteration; shapes become
  SDL color cursors (from_surface + hotspot), applied while the desktop
  mouse model is engaged; visibility follows the host; capture/released
  hands back the system cursor. Sessions advertise the cap when they
  START in desktop mode.

Verified on .21: fmt + clippy -D warnings (7 crates) + tests green
(core 218 incl. new wire roundtrips, host 245 incl. e2e + forwarder
downscale tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 01:09:51 +02:00
enricobuehler 12df1388de feat(apple/input): desktop (absolute) mouse mode on macOS — remote-desktop sweep M1
ci / web (pull_request) Successful in 53s
ci / docs-site (pull_request) Successful in 53s
apple / swift (pull_request) Successful in 1m21s
apple / screenshots (pull_request) Has been skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 4m15s
ci / bench (pull_request) Successful in 5m35s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 1m40s
android / android (pull_request) Successful in 11m14s
ci / rust (pull_request) Successful in 21m49s
Folds the parked client-side-cursor machinery (cursorMode auto/always/
never, hidden while disabled) into the cross-client mouse model:
MouseInputMode capture|desktop under DefaultsKey.mouseMode, picked in
Settings ▸ Keyboard & mouse (macOS), resolved at session start and
gated off on gamescope hosts (relative-only EIS).

Desktop model = the un-neutered absolute path with the SDL cursor
policy: pointer never disassociated (enters/leaves the stream freely),
monitor forwards letterboxed absolute positions, and the local cursor
hides only while over the view via an invisible-cursor rect (the
host's composited cursor is the one you see; AppKit manages the rect,
so no hide/unhide balancing). ⌘⇧C becomes ⌃⌥⇧M — the same chord as
the SDL clients — flipping the model live with an atomic
release/re-engage.

Verified: swift build + full test suites green (macOS arm64).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 00:03:19 +02:00
enricobuehler 7295ae70f9 feat(client/input): desktop (absolute) mouse mode — remote-desktop sweep M1
New physical-mouse model beside capture: Desktop leaves the pointer
uncaptured and sends absolute positions through the letterbox
(MouseMoveAbs — every host injector already consumes it). Capture
stays the default and the game model.

- pf-client-core: MouseMode (capture|desktop) persisted in Settings,
  default capture so existing stores are unchanged.
- pf-presenter: Capture grows a desktop model — latest-wins pending
  abs position coalesced per loop iteration (same 1000 Hz discipline
  as relative), flushed before clicks/keys/wheel so they land where
  the cursor is; Ctrl+Alt+Shift+M flips the model live; local cursor
  stays hidden over the window (the host's composited cursor is the
  one you see until the M2 cursor channel); Windows keyboard grab
  only engages for capture (a desktop stream is something you
  Alt-Tab away from).
- gamescope gating: its EIS is relative-only, so desktop mode is
  pinned off there (resolved_compositor), with a log note.
- Settings surfaces: GTK row (dynamic caption), WinUI combo,
  console-UI row + step test, capture-hint line.

Verified: fmt + clippy -D warnings + tests (33/40/19) on Linux .21;
clippy -D warnings for all five crates incl. punktfunk-client-windows
native on .173.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 23:52:16 +02:00
enricobuehler 87e7c82cbc chore(release): bump workspace version to 0.17.2
audit / bun-audit (push) Failing after 18s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
apple / swift (push) Successful in 1m23s
audit / cargo-audit (push) Successful in 2m44s
ci / bench (push) Successful in 6m26s
apple / screenshots (push) Successful in 6m38s
ci / rust (push) Successful in 24m53s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m10s
decky / build-publish (push) Successful in 22s
android-screenshots / screenshots (push) Successful in 3m7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m38s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9m3s
release / apple (push) Successful in 10m50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m41s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m4s
deb / build-publish-host (push) Successful in 12m44s
android / android (push) Successful in 14m35s
docker / deploy-docs (push) Successful in 23s
arch / build-publish (push) Successful in 15m35s
web-screenshots / screenshots (push) Successful in 3m1s
linux-client-screenshots / screenshots (push) Successful in 7m31s
windows-host / package (push) Successful in 13m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m24s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m32s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m39s
flatpak / build-publish (push) Successful in 6m40s
2026-07-21 23:33:53 +02:00
enricobuehler b2e3d1b540 fix(steamdeck/install): add cmake dep + acquire sudo in preflight
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m52s
apple / swift (push) Successful in 1m22s
android / android (push) Successful in 13m4s
ci / bench (push) Successful in 8m2s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
deb / build-publish (push) Successful in 9m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m14s
apple / screenshots (push) Successful in 6m51s
arch / build-publish (push) Successful in 18m54s
docker / deploy-docs (push) Successful in 27s
ci / rust (push) Successful in 19m17s
deb / build-publish-host (push) Successful in 9m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
pyrowave-sys's build.rs runs cmake to build the vendored PyroWave C library,
but cmake wasn't in the distrobox apt list — so `cargo build -p punktfunk-host`
dies on every SteamOS host with "is cmake not installed?". Add it.

Also move sudo acquisition from step 4 (after the ~15-min build) into preflight
and keep the timestamp warm across the build. Before, a non-interactive run
(`ssh host 'bash install.sh'`) silently skipped the UDP-buffer/udev/vhci-hcd/
input-group tuning AND enable-linger — a "successful" host with no gamepad
passthrough, lossy streaming, and no headless persistence (dies on logout).
Now sudo is prompted once up front (authorize and walk away), and a no-TTY /
no-password run fails loudly before the build instead of silently at the end.

Validated end-to-end on SteamOS 3.8 in a VM with Radeon 780M passthrough:
full host + web build, all planes listening, VAAPI render node, headless
auto-start via linger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehler d36bec6e9d feat(core+host): negotiate ChaCha20-Poly1305 as the session cipher for soft-AES clients
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Failing after 13s
audit / cargo-audit (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Successful in 10m1s
windows-host / package (push) Successful in 11m20s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m39s
Lifts the ~100 Mbps decrypt ceiling on clients without hardware AES — the
armv7 soft-AES targets (webOS TVs), where AES-128-GCM resolves to fixsliced
software AES + software GHASH (~50-100 cpb) while ChaCha20-Poly1305's ARX
construction runs ~10-17 cpb portable, a 4-7x lift that PyroWave-on-TV needs
(design/chacha20-session-cipher.md).

Phase 1 (core crypto, no wire change): SessionKey merges cipher choice and
key material (invalid combinations unrepresentable, zeroize + redacted-Debug
discipline kept); SessionCrypto dispatches both aead-0.5 ciphers per call —
the salt||seq nonce scheme, per-direction salts, seq-as-AAD and replay
window carry over verbatim (same 96-bit nonce / 16-byte tag, const-asserted).
Config.key becomes SessionKey; validate's zero-key rejection follows the
active variant. The C ABI keeps its fixed 16-byte key mapped to AES — no
ABI_VERSION bump.

Phase 2 (negotiation): VIDEO_CAP_CHACHA20 (0x40) — support-plus-request in
one bit, the VIDEO_CAP_444 precedent. Welcome grows cipher@68 +
key_chacha@69..101, emitted only when non-zero so an AES session's Welcome
stays byte-identical to the pre-cipher form; decode is fail-closed (short
key or unknown id -> Err, never a silent AES fallback). No WIRE_VERSION
bump; downgrade resistance inherited from the pinned-TLS control channel.

Phase 3 (host): grant only when the client advertised the bit and the
PUNKTFUNK_CHACHA20 kill-switch (default on, documented) allows; fresh
32-byte per-session key from the same RNG discipline, legacy key field
stays independently random; resolved cipher logged at session start.

Verification: seal/open suites parameterized over both ciphers + a
cross-cipher tamper case; Welcome roundtrip/truncation/fail-closed tests;
ChaCha lossy-loopback soak (loss/replay is cipher-independent); bench
gains _chacha20 series (AES ids unchanged for CI history) — host-side
sealing line-rate-trivial on both x86 (~640 MiB/s) and Apple Silicon
(~535 MiB/s). punktfunk-probe drives the interop matrix via
PUNKTFUNK_CLIENT_CHACHA20=1 and logs the negotiated cipher.

Phase 4 (pf-webos pin bump + unconditional cap bit) follows the next
core release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehler abc54a7d13 fix(decky): actually ship shortcut artwork + Steam Input layout in the published plugin
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m58s
The CI workflow assembled the store zip by hand and silently drifted from
scripts/package.sh when 60d46530 added assets/ and controller_config/ — every
published build (canary and stable through 0.17.1) was missing both, so the
Steam shortcut a fresh install creates had no images and no controller layout
(shortcut_art() found nothing to send; apply_controller_config failed with
template-missing). CI now stages through scripts/package.sh — the plugin file
list lives in one place — and a backstop verifies the runtime-loaded pieces are
in the zip before publishing.

Frontend hardening for installs poisoned by those builds: applyArtwork stamped
its per-appId ART_VERSION marker even when zero assets were applied, so
shipping the files alone would never heal existing shortcuts — only mark done
when something actually landed, and bump ART_VERSION 2→3 to force one re-apply.
Same guard for the controller-config marker (ok with zero account configset
dirs is not done), plus one deferred artwork retry for the fresh-AddShortcut
registration race setShortcutHidden already defers around.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehler 309e37f1e1 fix(host/linux): scope the GPU clock pin to live client sessions
The PUNKTFUNK_PIN_CLOCKS clock pin (AMD power_dpm_force_performance_level=high / NVIDIA nvmlDeviceSetGpuLockedClocks) was armed once at host start and released only at host exit, so the box's clocks stayed pinned the whole time the host ran — even with no client connected.

Move the pin behind a box-wide refcount (gpuclocks::session_pin()) shared across both streaming planes (native + GameStream): it arms on the first live client and releases on the last disconnect. on_host_start() now only installs the process-scoped NVIDIA P2-cap application profile. No behavior change when PUNKTFUNK_PIN_CLOCKS is unset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehler 56d21f9445 fix(client/wol): retry the QUIC dial across the connect budget so wake-from-sleep launches survive
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 40s
ci / bench (push) Successful in 6m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish (push) Successful in 9m54s
release / apple (push) Successful in 9m5s
deb / build-publish-host (push) Successful in 9m35s
flatpak / build-publish (push) Failing after 8m19s
android / android (push) Successful in 16m57s
apple / screenshots (push) Successful in 6m29s
arch / build-publish (push) Successful in 17m35s
windows-host / package (push) Successful in 18m46s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
docker / deploy-docs (push) Successful in 26s
ci / rust (push) Successful in 25m24s
Launching from the Decky plugin against a sleeping host fired the
Wake-on-LAN packet and dialed immediately — but the dial was a single
quinn attempt that gave up after the transport's 8 s idle window, far
shorter than a suspend-to-RAM resume. The host woke to find the client
already gone (its retransmitted Initials show up host-side as
"QUIC accept failed error=timed out"), the stream shortcut exited, and
the launch looked cancelled.

- punktfunk-core: dial in a loop until the embedder's connect budget is
  spent — short 3 s attempts keep the Initial cadence dense, so the
  connect lands within ~a second of the host's network returning. Only
  silence is retried: pin mismatches, typed rejections, and any real
  answer surface immediately, and the shutdown flag stops the loop.
- decky: stretch the budget to 75 s when a magic packet actually went
  out (PF_CONNECT_TIMEOUT → --connect-timeout via the wrapper), so the
  retry window covers the resume; an awake host still connects in <1 s.
- host: a clean close before the control handshake is a reachability
  probe (hosts-page online pips), not a failed session — log it at
  debug instead of WARN "session ended with error", which buried the
  real failures in the reporter's log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:26:10 +02:00
enricobuehler f36d13e371 fix(steamdeck/install): prompt for sudo instead of requiring passwordless
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m5s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m43s
docker / deploy-docs (push) Successful in 48s
deb / build-publish-host (push) Successful in 10m9s
android / android (push) Successful in 18m18s
arch / build-publish (push) Successful in 19m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m16s
ci / rust (push) Successful in 28m27s
The system-tuning block (UDP buffers, gamepad udev rule, vhci-hcd autoload,
input group) was gated on `sudo -n true` — passwordless sudo — so on a stock
SteamOS box (the deck account needs a password) the whole block was silently
skipped: /dev/uhid stayed root-only so the gamepad degraded to an Xbox 360 pad,
and the UDP buffers stayed at the 416 KB default. Video was unaffected because
Game Mode/gamescope needs none of it, which masked the skip on-glass.

- install.sh + update.sh: acquire sudo interactively (`sudo -v` prompt on a TTY)
  instead of requiring passwordless; skip only when there's no TTY or auth fails,
  and then print the exact manual block plus a `passwd` hint — a stock SteamOS
  'deck' account has no password, so sudo can't work until one is set.
- update.sh: also retrofit the UDP-buffer sysctl for older/skipped installs, and
  loudly nag to reboot when the input group was just added (a --user restart does
  not pick up the new group; only a fresh login does).
- steamos-host.md: note this step prompts for the sudo password.

Root is unavoidable: the udev rule, input group, and vhci-hcd module are all
kernel operations with no user-space alternative.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:56:20 +02:00
enricobuehler aacd610b68 fix(steamdeck/install): set up controller passthrough + surface the required reboot
ci / docs-site (push) Successful in 1m12s
ci / web (push) Successful in 1m13s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m35s
apple / screenshots (push) Successful in 6m13s
deb / build-publish (push) Successful in 10m21s
docker / deploy-docs (push) Successful in 34s
deb / build-publish-host (push) Successful in 9m58s
android / android (push) Successful in 17m7s
arch / build-publish (push) Successful in 17m47s
ci / rust (push) Successful in 19m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m2s
A fresh SteamOS host install failed on-glass two ways: KWin denied Desktop-mode
capture (zkde_screencast_unstable_v1) so every session died in the pipeline build,
and the native Steam Deck pad degraded to an Xbox 360 controller. Both stem from
setup that only a fresh login applies — the host user-service keeps the
pre-install group set and KWin reads its .desktop grant at session start — plus a
missing vhci-hcd autoload the pad's USB/IP transport needs (the deb/rpm/arch
packages ship it; the Deck installer did not).

- install.sh: install punktfunk-modules.conf (+ modprobe vhci-hcd now) for the
  native Deck controller transport, seed the KDE RemoteDesktop grant
  (kde-authorized) for Desktop-mode input, and print a loud "reboot before
  streaming" notice when a relogin is actually required (input group added or the
  .desktop grant first installed).
- update.sh: retrofit the udev rule, vhci-hcd autoload, input group, and grant so
  existing installs pick them up on the next update without a reinstall.
- steamos-host.md: document the first-install reboot and the native controller
  passthrough requirements (input group + vhci-hcd + client-side Steam Input Off).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:22:08 +02:00
enricobuehler 081ff64087 docs: explain how to actually run the VirtualHere passthrough example
apple / swift (push) Successful in 1m22s
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m39s
apple / screenshots (push) Successful in 6m38s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
deb / build-publish (push) Successful in 9m0s
deb / build-publish-host (push) Successful in 9m20s
windows-host / package (push) Successful in 9m46s
android / android (push) Successful in 18m37s
arch / build-publish (push) Successful in 18m48s
docker / deploy-docs (push) Successful in 28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m44s
ci / rust (push) Successful in 26m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m2s
The VirtualHere DualSense recipe linked the `virtualhere-dualsense.ts` SDK
example as a deployable recipe but skipped everything needed to run it: that
VirtualHere is a server (couch) + client (host) pair, and that the example —
like every example — imports `../src/index.js`, which only resolves inside the
SDK repo. A user copying it out had no way to know they must
`bun add @punktfunk/host` and swap that import.

- automation.md: rewrite the recipe into a full walkthrough — the two-sided
  VirtualHere setup, the `-t` verbs, the zero-code hooks version, and a new
  scripted section with exact deploy steps, env vars, and running it as a
  service (its own SIGTERM handler makes `systemctl stop` release the pad).
- sdk/README.md: say how to run an example in-repo vs deployed on a host.
- virtualhere-dualsense.ts: header note on the import swap + service setup,
  since that file is where the doc link lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:28:23 +02:00
enricobuehler 675fd24cce chore(release): bump workspace version to 0.17.1
audit / cargo-audit (push) Successful in 3m22s
audit / bun-audit (push) Failing after 13s
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m12s
apple / screenshots (push) Successful in 6m55s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m55s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m39s
ci / rust (push) Successful in 24m5s
decky / build-publish (push) Successful in 30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
android-screenshots / screenshots (push) Successful in 3m34s
linux-client-screenshots / screenshots (push) Successful in 8m32s
release / apple (push) Successful in 11m16s
deb / build-publish-host (push) Successful in 12m51s
deb / build-publish (push) Successful in 12m57s
docker / deploy-docs (push) Successful in 27s
web-screenshots / screenshots (push) Successful in 3m13s
windows-host / package (push) Successful in 14m6s
android / android (push) Successful in 15m33s
arch / build-publish (push) Successful in 16m3s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m43s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m10s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 23m16s
flatpak / build-publish (push) Successful in 7m9s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:37:32 +02:00
enricobuehler 0e60e58cab perf(apple): windowed macOS presents — off-main transactional commit, surface prototype, safe-present setting
ci / docs-site (push) Successful in 52s
ci / web (push) Successful in 56s
decky / build-publish (push) Successful in 21s
apple / swift (push) Successful in 1m24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m53s
release / apple (push) Successful in 9m19s
deb / build-publish (push) Successful in 12m27s
arch / build-publish (push) Successful in 12m52s
docker / deploy-docs (push) Successful in 26s
deb / build-publish-host (push) Successful in 13m18s
apple / screenshots (push) Successful in 6m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m51s
android / android (push) Successful in 18m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m15s
ci / rust (push) Successful in 27m38s
Task 1 (latency): the transactional present now commits from the RENDER
thread inside an explicit CATransaction + flush() instead of hopping to
main. The present harness (2026-07-21, 240 Hz Studio, full-size window)
measured the main hop batching presents at runloop-iteration rate when
main is busy — an active implicit transaction NESTS the explicit one —
which is the field's presents=55 @ fps=240 / display_p50 18.6 ms.
Off-main commits measured immune to main churn: glass p50 ~10 ms,
cadence a clean 4.17 ms at 240. flush() is load-bearing: without it the
render thread's own implicit transaction (drawableSize/colour writes)
swallows the explicit commit and NOTHING reaches glass — the harness
reproduced the exact freeze (every present dropped). Legacy main-hop
kept as PUNKTFUNK_TXN_PRESENT=main for field A/B. New pf-windowed
Console line (subsystem io.unom.punktfunk, category presenter)
decomposes the issue side per second.

Task 1 lever D: the f407f418 IOSurface contents-swap path is
resurrected format-aware as WindowedPresentMode.surface — bgra8 SDR /
rgba16Float+PQ-tagged HDR pool, swap committed off-main from the
completion handler; EDR anchored by the metal layer underneath.
Env-only prototype (PUNKTFUNK_WINDOWED_PRESENT=surface) until the HDR
composite is eyeballed on glass.

Task 2 (setting): punktfunk.windowedSafePresent (default ON =
transactional mitigation; OFF = the fast async path whose panic the
saga is about — the caption says so in plain words). Rows in the touch
Settings (Presentation) and gamepad settings, macOS-only.
PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface overrides for dev
A/B. maximumDrawableCount stays 3 — the harness measured 2 starving the
render loop (172/s) and worsening glass p50.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:58:47 +02:00
enricobuehler 71faf38a85 fix(apple): stats os_log used %d for 64-bit Swift Int — macOS 26 drops the line
Swift `Int` is 64-bit (Foundation infers %lld); the stats mirror's format
string used %d (32-bit C int) for fps/presents/lost. macOS 26's strict
String(format:) validator rejects the mismatch and drops the whole line
(the error also cascades onto the float args, mis-blaming floor_p50). Use
%lld for the three Int fields. Pre-existing, unrelated to the DCP work;
surfaced while reading presenter logs during the panic fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:14:41 +02:00
enricobuehler f49d38826b fix(apple): windowed macOS DCP panic via presentsWithTransaction — keep real HDR, drop the SDR IOSurface path
The windowed-macOS "mismatched swapID's" @UnifiedPipeline.cpp kernel
panic is the CAMetalLayer ASYNC image queue (commandBuffer.present,
mandatory with displaySyncEnabled=false) diverging from WindowServer's
compositor on a high-refresh composited session — it survived glass
pacing (PyroWave, 2026-07-18) and hit windowed HEVC on the same 240 Hz
Mac Studio (2026-07-21), so it's the async queue itself, at any codec
or pacing.

f407f418's mitigation routed windowed PyroWave through a BGRA8 IOSurface
layer.contents pool, which cannot carry HDR — extending it to HEVC would
have tone-mapped windowed HDR down to SDR. Instead, present the drawable
through a Core Animation transaction (CAMetalLayer.presentsWithTransaction)
for every windowed session: commit, waitUntilScheduled, then drawable.present()
inside a CATransaction, so the swap commits with the layer tree and stays
in lockstep with the compositor instead of racing it. The full
rgba16Float / PQ / EDR render path is untouched — windowed HDR is real
HDR again. Fullscreen keeps the async path (direct scanout, lowest latency,
no panic there).

Removes the IOSurface surface pool, its surfaceLayer sibling, and the
macOS windowed PQ->SDR tone-map (tvOS keeps its no-headroom tone-map).
Net -150 lines. swift build + 169 tests green. Needs on-glass soak on
the 240 Hz Mac Studio (kernel race — only real hardware confirms).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:45:11 +02:00
enricobuehler 42198eb1b6 feat(linux/vulkan-encode): default-on native NV12 capture + RGB true-extent
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 25s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / bench (push) Successful in 5m59s
apple / screenshots (push) Successful in 6m17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m16s
deb / build-publish (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 13m40s
windows-host / package (push) Successful in 10m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m52s
android / android (push) Successful in 16m18s
arch / build-publish (push) Successful in 16m20s
docker / deploy-docs (push) Successful in 30s
ci / rust (push) Successful in 19m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m19s
Flip both zero-copy levers from opt-in to default:

- The capture negotiation now PREFERS gamescope's producer-side NV12 pod by
  default (PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation).
  Codec-aware gating rides a new OutputFormat::nv12_native ->
  ZeroCopyPolicy::native_nv12_session edge resolved by the host facade from
  the session plan (pf_encode::linux_native_nv12_ok): only H265/AV1 sessions
  whose backend can open the raw Vulkan Video encoder ever see the NV12 pod
  -- an H264/Moonlight session (libav VAAPI, which would misread the
  two-plane buffer) keeps today's BGRx negotiation, as do the
  GameStream-resolve and portal-mirror paths, PyroWave, and NVENC prefs.
- RGB-direct's unaligned modes default to the true-extent direct import
  (PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded-copy staging).
  Guarded-tested on Van Gogh with the kernel journal watched: clean, and at
  5.38 ms p50 the fastest 1080p encode path measured on that hardware. The
  EFC only exists on Mesa >= 26, where the codedExtent-driven session_init
  padding is guaranteed (verified back to Mesa 24.2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehler b4fbc94e36 feat(linux/vulkan-encode): PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT bring-up gate
Opt-in alternative to RGB-direct's padded-copy staging at unaligned modes:
direct-import the visible-size capture and declare the TRUE-SIZE source
codedExtent. RADV has derived the VCN session_init firmware padding from
srcPictureResource.codedExtent since Mesa 24.2, so the EFC is told the source
lacks the alignment rows and the hardware edge-extends them internally --
which also reframes the 2026-07-20 field GPU reset: that crash passed the
ALIGNED extent as the source codedExtent (firmware padding zero) over a
visible-size buffer. Off by default until a guarded live test proves the EFC
front-end honors the padding like the YUV fetch path does; the padded-copy
staging remains the shipped behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehler 004f0cacb8 perf(linux/vulkan-encode): true-size headers make native NV12 zero-copy at every mode
Drop the padded-copy staging for producer-native NV12 and direct-import the
visible-size buffer at unaligned modes (1080p) too. Safe by the driver's own
contract rather than by allocation luck: native sessions now author the H265
SPS / AV1 sequence header at the RENDER size and pass the matching codedExtent
on every picture resource. RADV rounds the bitstream SPS up itself (with a
conformance window -- radv_video_patch_encode_session_parameters, per the
VK_KHR_video_encode_h265 proposal's "implementations may override" clause) and
programs the VCN session_init with the true extent plus nonzero firmware
padding, so the hardware edge-extends the alignment rows internally and never
fetches past the source's real extent -- the exact mechanism VAAPI/radeonsi has
always used to encode 1080p. The driver-emitted header NALs already carry the
patched SPS, so the wire format is unchanged.

The CSC and RGB-direct paths keep the app-aligned convention (their sources
genuinely cover the aligned extent); RGB padded-copy staging is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehler 49c6dd00c9 fix(presenter): CPU frames never take the HDR10 passthrough surface
audit / bun-audit (push) Failing after 15s
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 58s
apple / swift (push) Successful in 1m22s
audit / cargo-audit (push) Successful in 2m20s
decky / build-publish (push) Successful in 32s
ci / bench (push) Successful in 7m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m18s
release / apple (push) Successful in 9m28s
deb / build-publish-host (push) Successful in 10m32s
deb / build-publish (push) Successful in 11m48s
docker / deploy-docs (push) Successful in 23s
android / android (push) Successful in 14m36s
arch / build-publish (push) Successful in 15m4s
windows-host / package (push) Successful in 16m11s
flatpak / build-publish (push) Failing after 8m19s
apple / screenshots (push) Successful in 6m40s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m11s
ci / rust (push) Successful in 19m38s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m39s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m36s
Software decode uploads swscale RGBA with no CSC/tonemap pass. On the new
desktop-compositor HDR10 surfaces (GNOME 48 / Plasma 6 + Mesa >= 25.1 offer
ST2084 even on SDR desktops) that sRGB-encoded content was composed as PQ —
the field-reported psychedelic picture (Fedora clients decode HEVC in
software because stock Mesa strips it; reproduced + fix live-validated on
GNOME Wayland: mode-0 soup -> HDR→SDR washed-out-but-correct).

The CPU lane keeps the SDR swapchain and its known untonemapped-PQ gap; a
real PQ→sRGB pass for CPU frames is the follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler e7c6c96e5f fix(presenter): set the Wayland app_id so the session window gets its icon
SDL_APP_ID = io.unom.Punktfunk — compositors match it against the shipped
.desktop for the window icon; without it KWin/mutter show the generic Wayland
icon (field-noticed on the Deck). Linux analog of win32 AppUserModelID.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler da134ad045 feat(presenter): PUNKTFUNK_HDR10=0 refuses the HDR10 swapchain (SDR tonemap pin)
GNOME 48 / Plasma 6 with Mesa >= 25.1 now offer HDR10_ST2084 surfaces even on
SDR desktops, silently flipping PQ streams into the mode-0 passthrough lane —
previously unreachable on desktop compositors and never validated there. The
hatch pins the shader tonemap for diagnosis and as the field escape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler 428bd2519c test(qsv): converter→ring-profile-P010→encoder live e2e (the RTV-written seam)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler 87d7eceabf fix(encode/qsv): write the colour VUI unconditionally + Intel-lane colour diagnostics
The native QSV encoder only attached mfxExtVideoSignalInfo on HDR sessions —
an SDR stream carried no colour description at all (NVENC writes 709-limited
unconditionally; qsv.rs now mirrors it).

Diagnostics for the field-reported blue/magenta Intel-host colours:
- hdr-p010-selftest takes an optional WxH + GPU vendor (intel|nvidia|amd) and
  prints which adapter it tested — 64x64 on the default GPU proved nothing on
  dual-GPU boxes, and the field heights (1080/1400) are not 16-aligned.
- pf-capture: ignored live test running the converter at 1920x1080 pinned to
  the Intel adapter.
- pf-encode: qsv_live_p010_1080_colorbars_dump — known P010 bars through the
  UNALIGNED-height ingest copy (1080 src -> align16 1088 pool, the seam no
  640x480 test exercises) to Main10 HEVC, dumped for off-box decode checks.
- dxgi.rs: drop the stale 'falls back to the R10 path' claim (no such path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehler 1d4795666e feat(linux/vulkan-encode): opt-in gamescope producer-native NV12 encode source
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
decky / build-publish (push) Successful in 19s
apple / swift (push) Successful in 1m21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m53s
apple / screenshots (push) Successful in 6m16s
windows-host / package (push) Successful in 9m38s
deb / build-publish (push) Successful in 9m55s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m52s
android / android (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 14m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m0s
ci / rust (push) Successful in 28m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m41s
With PUNKTFUNK_PIPEWIRE_NV12=1 (bring-up gate), the PipeWire negotiation
offers an NV12 LINEAR DMA-BUF pod (BT.709 limited pinned MANDATORY) ahead
of the BGRx one, and gamescope's producer-side RGB->YUV pass replaces the
host CSC entirely: the encoder imports the two-plane buffer as a profiled
VIDEO_ENCODE_SRC image and the VCN encodes it directly. Contributed
measurements: encode p99 2.9 ms (from ~4-4.5 ms via EFC RGB-direct),
60 fps capture, 0 send drops.

Hardening on top of the contributed patch:
- unaligned modes (1080p!) stage through a padded aligned NV12 copy (edge
  rows/columns duplicated, transfer-only) instead of direct-importing the
  visible-size buffer -- a direct import would make the VCN read past the
  producer allocation, the exact OOB class behind the 2026-07-20 field GPU
  reset; the encode extents return to the aligned coded extent everywhere
- the UV plane layout honors the producer's plane-1 chunk (offset/stride)
  when the SPA buffer carries one (same-BO verified by inode), with the
  contiguous-plane contract as fallback
- PyroWave sessions are excluded from the gate (their Vulkan compute CSC
  ingests packed RGB), and a native-NV12 session that resolves to libav
  VAAPI (H264 codec, PUNKTFUNK_VULKAN_ENCODE=0, feature off, or a failed
  Vulkan open) refuses at open instead of streaming garbage chroma
- pad staging images carry TRANSFER_SRC (the width-padding pass self-copies
  the staging image -- previously missing on 1366-wide modes)
- metadata-cursor one-shot warn (parity with RGB-direct) and a padded-NV12
  PUNKTFUNK_PERF split label; the padded RGB copy gets its timestamps too

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 11:57:42 +02:00
enricobuehler 58b27acbd5 fix(vdisplay/session): scrub the dead desktop's WAYLAND_DISPLAY from the user-manager env
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m14s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 27s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m44s
apple / screenshots (push) Successful in 6m18s
android / android (push) Successful in 13m0s
docker / deploy-docs (push) Successful in 27s
deb / build-publish (push) Successful in 13m36s
deb / build-publish-host (push) Successful in 14m44s
windows-host / package (push) Successful in 16m32s
arch / build-publish (push) Successful in 17m49s
ci / rust (push) Successful in 19m58s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m42s
settle_desktop_portal pushes the live desktop's WAYLAND_DISPLAY into the
systemd --user manager (import-environment) so a re-activated portal inherits
the right session — but the import PERSISTS after that desktop dies. Every
later user unit then inherits the stale socket, including
gamescope-session.target: gamescope sees WAYLAND_DISPLAY, runs NESTED, and
aborts with "Failed to connect to wayland socket: wayland-0" — observed live
on a Deck, where Game Mode could not start at all (autologin crash-looped)
until the var was unset by hand.

Two-layer fix, mirroring the protection launch_session already gives its
transient unit:
- observe_session_instance: when a desktop compositor instance goes away,
  unset WAYLAND_DISPLAY/DISPLAY from the manager env (a desktop bounce is
  harmless — the next portal settle re-imports).
- write_steamos_dropin: UnsetEnvironment=DISPLAY WAYLAND_DISPLAY on the
  takeover drop-in, unit-scoped belt-and-suspenders for host-initiated
  restarts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:06:55 +02:00
enricobuehler 333c8979e9 fix(vdisplay/gamescope): capture-loss rebuild probes must not steal the box session
Observed live on a Deck host: the user switched Game Mode → Desktop, KDE came
up in 0.8 s — and the capture-loss rebuild's FIRST detection ran inside that
gap, still said Gaming, and its gamescope re-acquire restarted
gamescope-session.target. On SteamOS that steals the seat: the user was
yanked out of the KDE session they had just chosen, the two session managers
fought (job canceled / dependency failed), no gamescope node appeared within
30 s, and the stream died at the 40 s rebuild budget.

A rebuild probe acting on possibly-stale detection must never stop, relaunch,
or take over box sessions. New pf_vdisplay::rebuild_probe_scope (RAII,
counted): while held, the gamescope managed/takeover create paths only attach
to a live node and fail fast otherwise — no stop_autologin_sessions, no
session-plus relaunch, no gamescope-session.target restart. The capture-loss
loop holds it for the first 4 s after a loss (the detection-ambiguity
window); after that, destructive rebuilds are allowed again, so a genuine
switch INTO Game Mode still gets its headless takeover. Probe failures are
instant, so the loop now paces itself at ~2 Hz instead of spinning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:00:02 +02:00
enricobuehler 46d09ed973 fix(host/linux): stop burning retry budget on doomed pipeline builds
Two session-transition stalls found live on a SteamOS Deck host, one cause:
the pipeline retry loop couldn't tell a wait-it-out failure from a
retry-now-and-it-works one.

- Gamescope first-frame race: a PipeWire stream connected while gamescope
  re-initializes its headless takeover negotiates a format, reaches
  Streaming, and never receives a buffer — while a fresh connect delivers
  within ~0.5 s. Every gamescope bring-up ate the full 10 s first-frame
  budget on attempt 1 (17 s bring-ups; KWin: 1.2 s). The retry loop's FIRST
  attempt now waits 2.5 s (Capturer::next_frame_within), so the reconnect
  that fixes the race happens at ~3 s. Later attempts keep the patient 10 s
  — the documented 30-60 s Big Picture cold start still fits the budget.

- Capture-loss rebuild vs session switch: the rebuild loop re-detects the
  active session between build_pipeline_with_retry calls, but each call
  burned 8 attempts (~13 s) against a compositor that no longer exists (a
  Desktop→Gaming switch spent 13 of its 27 s retrying gone-KWin). The
  capture-loss path now passes max_attempts=2, turning the outer loop into
  ~1 s detect-and-retry cycles that follow the box to the new session.

The resize path's direct build_pipeline call keeps the default budget (no
retry wrapper there to absorb an early bail).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:34:49 +02:00
enricobuehler 946f1b3d69 fix(steamdeck): build the host with vulkan-encode, and wire up what it needs to actually engage
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m43s
apple / screenshots (push) Successful in 6m32s
deb / build-publish (push) Successful in 9m57s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m46s
android / android (push) Successful in 13m40s
deb / build-publish-host (push) Successful in 13m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m42s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m59s
ci / rust (push) Successful in 26m41s
The SteamOS scripts built the host featureless, so PlatformAuto could never
pick the Vulkan Video backend (default-on since 84329205) and every session
silently ran libav VAAPI — unlike the deb/arch/rpm/sysext builds, which all
pass the feature. Three gaps, found live on a Deck host:

- install.sh/update.sh: cargo build with --features punktfunk-host/vulkan-encode
  (matches the packaged builds; pure-Rust ash, no new system dep).
- host.env: RADV_PERFTEST=video_encode — Van Gogh RADV still gates
  VK_KHR_video_encode_* behind it, so without it the backend can't open and
  falls back to VAAPI. Harmless where encode is exposed by default.
- io.unom.Punktfunk.Host.desktop (Exec rewritten to this install's binary):
  KWin only grants zkde_screencast_unstable_v1/org_kde_kwin_fake_input to an
  exe a .desktop authorizes — without it Desktop-Mode capture fails outright
  and a mid-stream Game↔Desktop switch strands the stream on the old backend.

update.sh retrofits the last two idempotently so existing installs get them
without a reinstall.

Validated on a SteamOS 3.8.14 Deck (Van Gogh): Vulkan HEVC opened on both the
KWin and gamescope paths at 5120x1440 — in-place ABR reconfigure (no IDR, no
encoder rebuild) vs the VAAPI path's full teardown per ABR step, and Desktop
capture came up without a re-login.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:14:31 +02:00
83 changed files with 4544 additions and 853 deletions
+24 -30
View File
@@ -6,17 +6,12 @@
# #
# The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT # The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT
# need the Decky CLI (which requires Docker + rust-nightly only to compile native backends). # need the Decky CLI (which requires Docker + rust-nightly only to compile native backends).
# We build the frontend with pnpm and assemble the store-layout zip by hand: # We build the frontend with pnpm and stage the store-layout tree with the SAME script local
# # builds use (clients/decky/scripts/package.sh) — the plugin's file list lives in exactly ONE
# punktfunk.zip # place, so a file added there (bin/, assets/, controller_config/, …) can never be silently
# punktfunk/ <- single top-level dir == plugin.json "name" # missing from the published build. (Hand-assembling the zip here is how the shipped plugin
# plugin.json [required] # lost the shortcut artwork + Steam Input layout for a while.) CI only adds `update.json` on
# package.json [required; CI stamps "version" — Decky reads the installed version here] # top: the {channel, manifest} pointer the plugin's self-update check polls.
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
# #
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel # SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed # `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
@@ -90,28 +85,27 @@ jobs:
- name: Assemble store-layout zip - name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }} working-directory: ${{ gitea.workspace }}
run: | run: |
apt-get update && apt-get install -y --no-install-recommends zip >/dev/null # node:22-bookworm ships python3 (a package.sh dep) but not zip; install both anyway
STAGE="$RUNNER_TEMP/decky" # so an image change can't silently break the build.
DEST="$STAGE/$PLUGIN" apt-get update && apt-get install -y --no-install-recommends zip python3 >/dev/null
rm -rf "$STAGE"; mkdir -p "$DEST/dist" "$DEST/bin" # Stage the canonical plugin tree (dist/, main.py, bin/, assets/, controller_config/,
cp clients/decky/plugin.json "$DEST/" # LICENSE, …) with the same script local/sideload builds use — see the header comment.
cp clients/decky/package.json "$DEST/" # Runs AFTER the version stamp, so the staged package.json carries $VERSION.
cp clients/decky/main.py "$DEST/" bash clients/decky/scripts/package.sh
cp clients/decky/dist/index.js "$DEST/dist/" DEST="clients/decky/out/$PLUGIN"
cp clients/decky/README.md "$DEST/" # CI-only addition: the self-update channel pointer the backend reads (main.py
# The stream-launch wrapper (target of the Steam shortcut); keep it executable # check_update). It points at THIS channel's manifest.json (published below); that
# (runner_info() also re-chmods at runtime in case the zip/extract drops the bit). # manifest in turn points at the immutable per-version zip, so its sha256 stays valid
cp clients/decky/bin/punktfunkrun.sh "$DEST/bin/" # across future alias re-uploads.
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json" printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" ) ( cd clients/decky/out && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip" ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip" unzip -l "$RUNNER_TEMP/punktfunk.zip"
# Backstop against packaging drift: the runtime-loaded pieces MUST be in the zip.
for f in main.py dist/index.js bin/punktfunkrun.sh assets/grid.png \
controller_config/punktfunk.vdf update.json; do
unzip -l "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN/$f" >/dev/null || { echo "MISSING $f" >&2; exit 1; }
done
# The update manifest the plugin polls: the immutable per-version artifact + its # The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on # sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias). # mismatch — so it MUST be the per-version URL, never the mutable alias).
Generated
+65 -27
View File
@@ -656,6 +656,30 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]] [[package]]
name = "ciborium" name = "ciborium"
version = "0.2.2" version = "0.2.2"
@@ -691,6 +715,7 @@ checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [ dependencies = [
"crypto-common", "crypto-common",
"inout", "inout",
"zeroize",
] ]
[[package]] [[package]]
@@ -2159,7 +2184,7 @@ dependencies = [
[[package]] [[package]]
name = "latency-probe" name = "latency-probe"
version = "0.17.0" version = "0.17.2"
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
@@ -2264,7 +2289,7 @@ dependencies = [
[[package]] [[package]]
name = "libvpl-sys" name = "libvpl-sys"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -2299,7 +2324,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]] [[package]]
name = "loss-harness" name = "loss-harness"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"punktfunk-core", "punktfunk-core",
] ]
@@ -2788,7 +2813,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]] [[package]]
name = "pf-capture" name = "pf-capture"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2808,7 +2833,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-client-core" name = "pf-client-core"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2832,7 +2857,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-clipboard" name = "pf-clipboard"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2850,7 +2875,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-console-ui" name = "pf-console-ui"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2871,7 +2896,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-encode" name = "pf-encode"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2881,6 +2906,7 @@ dependencies = [
"libvpl-sys", "libvpl-sys",
"nvidia-video-codec-sdk", "nvidia-video-codec-sdk",
"openh264", "openh264",
"pf-capture",
"pf-frame", "pf-frame",
"pf-gpu", "pf-gpu",
"pf-host-config", "pf-host-config",
@@ -2894,7 +2920,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-ffvk" name = "pf-ffvk"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"ash", "ash",
"bindgen", "bindgen",
@@ -2903,7 +2929,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-frame" name = "pf-frame"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"libc", "libc",
@@ -2915,7 +2941,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-gpu" name = "pf-gpu"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-host-config", "pf-host-config",
@@ -2929,11 +2955,11 @@ dependencies = [
[[package]] [[package]]
name = "pf-host-config" name = "pf-host-config"
version = "0.17.0" version = "0.17.2"
[[package]] [[package]]
name = "pf-inject" name = "pf-inject"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2961,14 +2987,14 @@ dependencies = [
[[package]] [[package]]
name = "pf-paths" name = "pf-paths"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "pf-presenter" name = "pf-presenter"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2983,7 +3009,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-vdisplay" name = "pf-vdisplay"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -3013,7 +3039,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-win-display" name = "pf-win-display"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-paths", "pf-paths",
@@ -3025,7 +3051,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-zerocopy" name = "pf-zerocopy"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3136,6 +3162,17 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]] [[package]]
name = "polyval" name = "polyval"
version = "0.6.2" version = "0.6.2"
@@ -3221,7 +3258,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-android" name = "punktfunk-client-android"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"jni", "jni",
@@ -3237,7 +3274,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-linux" name = "punktfunk-client-linux"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -3253,7 +3290,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-session" name = "punktfunk-client-session"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-client-core", "pf-client-core",
@@ -3268,7 +3305,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-windows" name = "punktfunk-client-windows"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"async-channel", "async-channel",
"ffmpeg-next", "ffmpeg-next",
@@ -3287,11 +3324,12 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-core" name = "punktfunk-core"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"bytes", "bytes",
"cbindgen", "cbindgen",
"chacha20poly1305",
"criterion", "criterion",
"fec-rs", "fec-rs",
"hmac", "hmac",
@@ -3318,7 +3356,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-host" name = "punktfunk-host"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
@@ -3402,7 +3440,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-probe" name = "punktfunk-probe"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"mdns-sd", "mdns-sd",
@@ -3416,7 +3454,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-tray" name = "punktfunk-tray"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ksni", "ksni",
@@ -3439,7 +3477,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]] [[package]]
name = "pyrowave-sys" name = "pyrowave-sys"
version = "0.17.0" version = "0.17.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" } ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package] [workspace.package]
version = "0.17.0" version = "0.17.2"
edition = "2021" edition = "2021"
rust-version = "1.82" rust-version = "1.82"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0", "name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0" "identifier": "MIT OR Apache-2.0"
}, },
"version": "0.17.0" "version": "0.17.2"
}, },
"paths": { "paths": {
"/api/v1/clients": { "/api/v1/clients": {
@@ -705,8 +705,11 @@ final class SessionModel: ObservableObject {
// captured before the 2026-07 floor policy); the appended trio carries the // captured before the 2026-07 floor policy); the appended trio carries the
// measured OS present floor and the floor-shaved values the HUD displays. // measured OS present floor and the floor-shaved values the HUD displays.
let line = String( let line = String(
format: "fps=%d presents=%d e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f " // Swift Int is 64-bit %lld, NOT %d (which is a 32-bit C int); macOS 26's
+ "decode_p50=%.1f display_p50=%.1f lost=%d " // strict String(format:) validator rejects the %d/Int mismatch and drops
// the whole line (a cascade error that also mis-blames the float args).
format: "fps=%lld presents=%lld e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%lld "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f", + "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f",
frames, frames,
displayWindow?.count ?? 0, displayWindow?.count ?? 0,
@@ -43,6 +43,9 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.presentPriority) private var presentPriority = @AppStorage(DefaultsKey.presentPriority) private var presentPriority =
SettingsOptions.presentPriorityDefault SettingsOptions.presentPriorityDefault
@AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.windowedSafePresent) private var windowedSafePresent = true
#endif
#if os(iOS) #if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false @AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
#endif #endif
@@ -345,6 +348,22 @@ struct GamepadSettingsView: View {
detail: "Turn off to use the touch interface even with a controller connected.", detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled), value: $gamepadUIEnabled),
] ]
#if os(macOS)
// The windowed safe-present toggle slots in after "Smoothness buffer" (staying inside
// the Video group) macOS only, mirroring the touch SettingsView's Presentation row
// (the DCP swapID-panic mitigation; see DefaultsKey.windowedSafePresent).
if let at = list.firstIndex(where: { $0.id == "smoothBuffer" }) {
list.insert(
toggleRow(
id: "windowedSafePresent", icon: "macwindow.badge.plus",
label: "Safe windowed presentation",
detail: "Windowed streams present in step with the compositor — avoids a "
+ "macOS display-driver crash on high-refresh displays, at a small "
+ "latency cost. Fullscreen always uses the fastest path.",
value: $windowedSafePresent),
at: at + 1)
}
#endif
#if os(iOS) #if os(iOS)
// The device-rumble mirror slots in after "Controller type" (staying inside the // The device-rumble mirror slots in after "Controller type" (staying inside the
// Controller group the next row carries the "Interface" header). iPhone only in // Controller group the next row carries the "Interface" header). iPhone only in
@@ -300,6 +300,18 @@ extension SettingsView {
+ "of added latency. Off shows frames as soon as they're ready.") { + "of added latency. Off shows frames as soon as they're ready.") {
Toggle("V-Sync", isOn: $vsync) Toggle("V-Sync", isOn: $vsync)
} }
// The DCP swapID-panic mitigation's user handle (see DefaultsKey.windowedSafePresent
// for the saga). Default ON: turning it off re-arms a WHOLE-MACHINE kernel panic on
// affected setups, so the caption says so in plain words.
described(windowedSafePresent
? "Windowed streams present in step with the system compositor — avoids a macOS "
+ "display-driver crash seen on high-refresh displays, at a small latency "
+ "cost. Fullscreen always uses the fastest path."
: "Windowed streams use the fastest present path. On some high-refresh setups "
+ "this can crash macOS itself (kernel panic) — turn back on if your Mac "
+ "restarts during windowed streaming.") {
Toggle("Safe windowed presentation", isOn: $windowedSafePresent)
}
#endif #endif
} }
} }
@@ -435,6 +447,14 @@ extension SettingsView {
/// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path). /// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path).
@ViewBuilder var inputSection: some View { @ViewBuilder var inputSection: some View {
Section("Keyboard & mouse") { Section("Keyboard & mouse") {
#if os(macOS)
described(mouseModeDescription) {
Picker("Mouse input", selection: $mouseMode) {
Text("Capture (games)").tag(MouseInputMode.capture.rawValue)
Text("Desktop (absolute)").tag(MouseInputMode.desktop.rawValue)
}
}
#endif
described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) { described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) {
Picker("Modifier keys", selection: $modifierLayout) { Picker("Modifier keys", selection: $modifierLayout) {
ForEach(ModifierLayout.allCases, id: \.self) { layout in ForEach(ModifierLayout.allCases, id: \.self) { layout in
@@ -447,6 +467,20 @@ extension SettingsView {
} }
} }
} }
#if os(macOS)
/// The SELECTED mouse model explained dynamic, like the touch-mode caption.
private var mouseModeDescription: String {
switch MouseInputMode(rawValue: mouseMode) ?? .capture {
case .capture:
return "The pointer locks to the stream and sends relative motion — best for "
+ "games. ⌃⌥⇧M switches live; applies from the next capture otherwise."
case .desktop:
return "The pointer moves freely in and out of the stream and sends absolute "
+ "positions — best for remote desktop work. Unavailable on gamescope hosts."
}
}
#endif
#endif #endif
// MARK: - Audio // MARK: - Audio
@@ -40,6 +40,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false @AppStorage(DefaultsKey.vsync) var vsync = false
@AppStorage(DefaultsKey.windowedSafePresent) var windowedSafePresent = true
#endif #endif
#if !os(tvOS) #if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true @AppStorage(DefaultsKey.allowVRR) var allowVRR = true
@@ -88,6 +89,7 @@ struct SettingsView: View {
@State var customMode = false @State var customMode = false
#endif #endif
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.mouseMode) var mouseMode = MouseInputMode.capture.rawValue
@AppStorage(DefaultsKey.speakerUID) var speakerUID = "" @AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = "" @AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0 @AppStorage(DefaultsKey.micChannel) var micChannel = 0
@@ -110,11 +110,11 @@ public final class InputCapture {
/// event itself is swallowed). Main queue. /// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)? public var onToggleCapture: (() -> Void)?
/// Fired on C (the client-side-cursor toggle flips between the captured/disassociated /// Fired on M (the mouse-model flip, capture desktop cross-client parity with the
/// relative path and the visible-cursor absolute path; detected here, like , so it works /// SDL clients' Ctrl+Alt+Shift+M; detected here, like , so it works regardless of the
/// regardless of the current capture state and the event itself is swallowed). macOS only; /// current capture state and the event itself is swallowed). macOS only; the
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue. /// absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)? public var onToggleMouseMode: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS /// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which /// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
@@ -245,13 +245,14 @@ public final class InputCapture {
self.onToggleCapture?() self.onToggleCapture?()
return nil return nil
} }
// C toggles the client-side cursor (visible-cursor absolute path vs the // M flips the mouse model (capture desktop the SDL clients' identical
// captured relative path). keyCode 8 = kVK_ANSI_C; layout-independent so it // chord). Detected in both capture states, like , so the model can be set
// fires the same on any keyboard. Suppress the C (latched like 's Esc) so it // before engaging. keyCode 46 = kVK_ANSI_M; layout-independent. Suppress the M
// doesn't type into the host, and swallow the event so it doesn't beep. // (latched like 's Esc) so it doesn't type into the host, and swallow the
if event.keyCode == 8 /* C */, flags == [.command, .shift] { // event so it doesn't beep.
self.suppressedVK = 0x43 // VK_C the same physical C is en route via GC if event.keyCode == 46 /* M */, flags == [.control, .option, .shift] {
self.onToggleCursor?() self.suppressedVK = 0x4D // VK_M the same physical M is en route via GC
self.onToggleMouseMode?()
return nil return nil
} }
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other // The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
@@ -0,0 +1,12 @@
/// How a physical mouse drives the host the cross-client mouse model (the SDL clients'
/// `MouseMode` / `Settings::mouse_mode`, design/remote-desktop-sweep.md M1). Stored stringly
/// under `DefaultsKey.mouseMode`.
public enum MouseInputMode: String, CaseIterable, Sendable {
/// Pointer capture (disassociated, hidden cursor, relative deltas) the game model,
/// and the default: the only cursor you see is the host's.
case capture
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion is forwarded as absolute positions through the letterbox. The remote desktop
/// model. Requires a host injector with absolute support (not gamescope).
case desktop
}
@@ -23,6 +23,34 @@ import os
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter") private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
#if os(macOS)
/// HOW a windowed (composited) macOS session pushes finished frames to glass the DCP
/// "mismatched swapID's" kernel-panic saga's mechanism picker. Fullscreen always presents
/// `async` (direct-scanout promotion, lowest latency, no panic reports there); the windowed
/// mechanism is resolved per session by SessionPresenter (user setting +
/// PUNKTFUNK_WINDOWED_PRESENT env override) and routed here via `setWindowedPresent`.
///
/// - `async`: the CAMetalLayer image queue (`commandBuffer.present`) the fastest composited
/// path and the PANIC TRIGGER on high-refresh displays (the out-of-band swaps race
/// WindowServer's compositor; it survived glass pacing and every codec).
/// - `transaction`: `CAMetalLayer.presentsWithTransaction` the swap commits WITH the layer
/// tree, in lockstep with the compositor (Apple's documented remedy; validated no-panic on
/// the 240 Hz repro machine). The present is committed from the RENDER thread inside an
/// explicit CATransaction + flush see `encodePresent` for why that beats the original
/// main-thread hop.
/// - `surface`: no image queue at all render into a pooled IOSurface and swap it into a plain
/// CALayer's `contents` (the f407f418 PyroWave mitigation, resurrected format-aware:
/// rgba16Float + PQ tagging keeps HDR). WindowServer treats it as ordinary layer damage on
/// its own composite cadence. PROTOTYPE: whether the compositor honors PQ/EDR for plain-layer
/// IOSurface contents still needs an on-glass eyeball the metal layer stays underneath with
/// `wantsExtendedDynamicRangeContent` as the EDR anchor.
enum WindowedPresentMode: String, Sendable {
case async
case transaction
case surface
}
#endif
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the /// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors /// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the /// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
@@ -198,8 +226,8 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.) // in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
// The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084 // The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff // EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map the // BT.709 OETF. Used by the tvOS biplanar tone-map and the tvOS planar (PyroWave) tone-map (the
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only. // no-HDR-headroom fallback). macOS keeps real HDR windowed now see `WindowedPresentMode`.
static inline float3 pqToSdr(float3 pq) { static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0; const float m1 = 2610.0/16384.0;
const float m2 = 78.84375; const float m2 = 78.84375;
@@ -230,8 +258,7 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows // PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave // fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions // stream must land on an 8-bit surface: tvOS without HDR headroom. The passthrough planar
// (the IOSurface present path the DCP-panic mitigation is BGRA8). The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the // HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation). // layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]], fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
@@ -259,21 +286,51 @@ public final class MetalVideoPresenter {
public let layer: CAMetalLayer public let layer: CAMetalLayer
#if os(macOS) #if os(macOS)
/// The WINDOWED-mode PyroWave present target: a plain CALayer sized like `layer` (installed /// WINDOWED-mode present coordination the macOS DCP KERNEL PANIC mitigation.
/// as a sibling ABOVE it), fed IOSurfaces via `contents` inside ordinary CATransactions.
/// ///
/// Why this exists the macOS DCP KERNEL PANIC ("mismatched swapID's" @UnifiedPipeline.cpp, /// The panic ("mismatched swapID's" @UnifiedPipeline.cpp, WindowServer dies, machine reboots):
/// WindowServer dies, machine reboots): out-of-band CAMetalLayer image-queue swaps into a /// the CAMetalLayer's ASYNCHRONOUS image queue (`commandBuffer.present(drawable)` an
/// COMPOSITED (windowed) session race WindowServer's own swap submissions on high-refresh /// out-of-band flip, mandatory with `displaySyncEnabled=false`) diverges from WindowServer's
/// displays, and the race survives glass pacing a fully serialized one-in-flight present /// compositor on a high-refresh COMPOSITED (windowed) session the compositor's notion of the
/// stream still panicked a 240 Hz Mac Studio (2026-07-18, twice). So in windowed mode we stop /// current swap and the layer's queued swap disagree, and the DCP asserts. It survived glass
/// using the image queue entirely and present the way video players do: render the planar CSC /// pacing: a fully serialized one-in-flight present stream still panicked a 240 Hz Mac Studio
/// into an IOSurface pool and swap `contents` on main WindowServer treats it as ordinary /// (2026-07-18, PyroWave), and a windowed HEVC session panicked the same machine 2026-07-21
/// damage on its own composite cadence, coalescing faster-than-refresh updates instead of /// so it is the async image queue itself, at any pacing or codec, not a present rate.
/// latching queue swaps mid-cycle. Fullscreen keeps the CAMetalLayer path (direct-scanout ///
/// promotion, no compositing, no panic reports). Contents updates are transparent to the /// The fix keeps the full render path (rgba16Float / PQ / EDR real HDR is preserved) and
/// layer below when nil, so flipping modes just covers/uncovers the metal layer. /// only changes HOW the drawable is presented: `CAMetalLayer.presentsWithTransaction`. With it
public let surfaceLayer: CALayer = { /// set, we don't hand the drawable to the command buffer; we commit, wait until scheduled, then
/// call `drawable.present()` INSIDE a CATransaction the present is enrolled in Core
/// Animation's transaction and committed together with the layer tree, so the swap stays in
/// lockstep with the compositor instead of racing it (Apple's documented remedy for Metal
/// presentation drifting out of sync with CA). Fullscreen keeps the async path (direct-scanout
/// promotion, lowest latency, no compositor and no panic reports there).
///
/// 2026-07-21 latency rework: the mitigation MECHANISM is now a three-way pick
/// (`WindowedPresentMode`) and the transactional present commits from the RENDER thread
/// see `encodePresent`. Staged under `stagingLock` (main pushes it via
/// `setComposited``setWindowedPresent`); the render thread drains it and toggles the layer
/// property + present style. `Active` is the render-thread copy so the layer property flips
/// exactly once per mode change.
private var windowedPresentStaged: WindowedPresentMode = .async
private var windowedPresentActive: WindowedPresentMode = .async
/// PUNKTFUNK_TXN_PRESENT=main the ORIGINAL transactional present (commit
/// waitUntilScheduled hop to the MAIN thread and present inside its CATransaction), kept
/// as a field A/B lever. The default is the render-thread commit: the present harness
/// (2026-07-21, this saga) measured the main hop landing a runloop turn late on a busy main
/// thread, and an ACTIVE implicit transaction there NESTS the explicit one presents batch
/// at runloop-iteration rate (the field's presents=55 @ fps=240, display_p50 18.6 ms).
/// Off-main commits measured immune to main-thread churn (~10 ms glass p50 at 240 Hz
/// full-size vs 14+ ms under a churned main hop).
private let txnPresentOnMain =
ProcessInfo.processInfo.environment["PUNKTFUNK_TXN_PRESENT"] == "main"
/// The WINDOWED-mode `surface` present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it by SessionPresenter), fed IOSurfaces via `contents` inside explicit
/// CATransactions. Transparent (nil contents) whenever surface mode is off, so the metal
/// layer below shows through. See `WindowedPresentMode.surface`.
let surfaceLayer: CALayer = {
let l = CALayer() let l = CALayer()
l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout
l.isOpaque = true l.isOpaque = true
@@ -281,8 +338,8 @@ public final class MetalVideoPresenter {
return l return l
}() }()
/// One IOSurface-backed render target of the windowed present pool. All pool state is /// One IOSurface-backed render target of the windowed surface-present pool. All pool state
/// RENDER-THREAD confined; only the immutable surface refs cross to main (contents swap). /// is RENDER-THREAD confined; only the immutable surface refs cross threads (contents swap).
private struct SurfaceSlot { private struct SurfaceSlot {
let surface: IOSurfaceRef let surface: IOSurfaceRef
let texture: MTLTexture let texture: MTLTexture
@@ -292,15 +349,52 @@ public final class MetalVideoPresenter {
private var surfacePool: [SurfaceSlot] = [] private var surfacePool: [SurfaceSlot] = []
private var surfacePoolSize: CGSize = .zero private var surfacePoolSize: CGSize = .zero
private var surfacePoolHDR = false
private var surfaceSeq: UInt64 = 0 private var surfaceSeq: UInt64 = 0
/// Index of the slot most recently handed to the layer never rewritten next, even if its /// Index of the slot most recently handed to the layer never rewritten next, even if its
/// use count already dropped (the compositor may still be scanning out the previous frame). /// use count already dropped (the compositor may still be scanning out the previous frame).
private var lastHandedOff: Int? private var lastHandedOff: Int?
/// Staged (under `stagingLock`, like every cross-thread input): the hosting view's windowed
/// vs fullscreen state, pushed from main via `setSurfacePresents`. Drained in `renderPlanar`. /// Once-per-second decomposition of the ACTIVE windowed present path (the field-diagnosis
private var surfacePresentsStaged = false /// half of the DCP-latency work): scheduled/completed wait + commit/flush cost per present,
/// Render-thread copy, so pool teardown happens exactly once on a mode flip. /// and how many presents/swaps were issued. The pf-present line shows the GLASS side
private var surfacePresentsActive = false /// (latchMs / dropped); this shows the ISSUE side. Logged via `presenterLog` only while a
/// windowed mechanism is active (zero cost fullscreen). Lock-guarded: transaction mode
/// records from the render thread, surface mode from Metal completion threads.
private final class WindowedPresentDiag: @unchecked Sendable {
private let lock = NSLock()
private var presents = 0
private var schedMs: [Double] = []
private var commitMs: [Double] = []
private var last = CACurrentMediaTime()
func record(schedMs sched: Double, commitMs commit: Double, mode: WindowedPresentMode) {
lock.lock()
presents += 1
schedMs.append(sched)
commitMs.append(commit)
let now = CACurrentMediaTime()
guard now - last >= 1 else {
lock.unlock()
return
}
last = now
let sSched = schedMs.sorted()
let sCommit = commitMs.sorted()
let line = String(
format: "pf-windowed mode=%@ presents=%d schedMs p50=%.2f max=%.2f "
+ "commitMs p50=%.2f max=%.2f",
mode.rawValue, presents, sSched[sSched.count / 2], sSched.last ?? 0,
sCommit[sCommit.count / 2], sCommit.last ?? 0)
presents = 0
schedMs.removeAll(keepingCapacity: true)
commitMs.removeAll(keepingCapacity: true)
lock.unlock()
presenterLog.info("\(line, privacy: .public)")
}
}
private let windowedDiag = WindowedPresentDiag()
#endif #endif
private let device: MTLDevice private let device: MTLDevice
@@ -316,7 +410,7 @@ public final class MetalVideoPresenter {
private let pipelinePlanar: MTLRenderPipelineState private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour /// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm /// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents). /// bgra8; tvOS without HDR headroom).
private let pipelinePlanarHDR: MTLRenderPipelineState private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache? private var textureCache: CVMetalTextureCache?
@@ -591,13 +685,14 @@ public final class MetalVideoPresenter {
} }
#if os(macOS) #if os(macOS)
/// Park the windowed-vs-fullscreen present routing (MAIN thread the hosting view pushes its /// Park the windowed present mechanism (MAIN thread the hosting view pushes its window
/// window state on every layout). true = PyroWave frames present via `surfaceLayer` contents /// state on every layout; SessionPresenter resolves the mechanism per session). `.async` =
/// (the DCP swapID-panic mitigation see `surfaceLayer`); false = the CAMetalLayer path. /// FULLSCREEN (or the user opted out of the mitigation): the image queue. `.transaction` /
/// `.surface` = COMPOSITED (windowed) mitigation mechanisms see `WindowedPresentMode`.
/// Applied by the render thread on the next frame, like every other staged value here. /// Applied by the render thread on the next frame, like every other staged value here.
public func setSurfacePresents(_ on: Bool) { func setWindowedPresent(_ mode: WindowedPresentMode) {
stagingLock.lock() stagingLock.lock()
surfacePresentsStaged = on windowedPresentStaged = mode
stagingLock.unlock() stagingLock.unlock()
} }
#endif #endif
@@ -734,36 +829,12 @@ public final class MetalVideoPresenter {
) -> Bool { ) -> Bool {
stagingLock.lock() stagingLock.lock()
let targetFromLayout = drawableTarget let targetFromLayout = drawableTarget
#if os(macOS)
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock() stagingLock.unlock()
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path; // A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader). // including macOS windowed sessions, which keep real HDR (the DCP mitigation is the
#if os(macOS) // transactional present in `encodePresent`, not a colour downgrade).
configure(hdr: planes.pq && !surfaceMode)
#else
configure(hdr: planes.pq) configure(hdr: planes.pq)
#endif
var csc = planes.csc var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
surfacePresentsActive = surfaceMode
presenterLog.info(
"stage2: windowed surface presents \(surfaceMode ? "ON" : "OFF", privacy: .public) (PyroWave DCP-panic mitigation)")
if !surfaceMode {
// Back to the metal path (fullscreen): drop the pool at 5K it holds >100 MB,
// and re-entering windowed mode rebuilds it in one frame.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
}
if surfaceMode {
return renderPlanarToSurface(
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still) // PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps // 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format). // in-shader instead (the pipeline must match the drawable's pixel format).
@@ -792,118 +863,6 @@ public final class MetalVideoPresenter {
} }
} }
#if os(macOS)
/// The windowed-mode present tail (see `surfaceLayer` for why this path exists): render the
/// planar CSC into a pooled IOSurface and hand it to `surfaceLayer.contents` on MAIN inside a
/// plain CATransaction an ordinary damaged-layer update on WindowServer's own composite
/// cadence, no CAMetalLayer image-queue swap anywhere. `presentAtMediaTime` doesn't apply
/// (the compositor paces); `onPresented` fires after the contents swap is committed, stamped
/// with CLOCK_REALTIME then the closest observable analogue of "reached glass" here (the
/// composite follows within a refresh, so the meters' display stage reads slightly optimistic).
private func renderPlanarToSurface(
_ planes: WaveletPlanes, targetFromLayout: CGSize, csc: inout CscUniform,
onPresented: ((Int64?) -> Void)?
) -> Bool {
let decodedSize = CGSize(width: planes.width, height: planes.height)
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
ensureSurfacePool(size: targetSize)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let keepAlive: [Any] = [planes.y, planes.cb, planes.cr]
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // ring textures pinned until the GPU finished sampling
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
onPresented?(
Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size` 4 BGRA8 IOSurface render targets (one on glass, one queued
/// in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the pool empty;
/// the caller returns false and the ring's putBack + display-link retry take over.
private func ensureSurfacePool(size: CGSize) {
guard size != surfacePoolSize else { return }
surfacePool.removeAll()
surfacePoolSize = size
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// 256-byte row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerRow = ((w * 4) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: 4,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one /// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule /// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback. /// the present and the on-glass callback.
@@ -936,6 +895,36 @@ public final class MetalVideoPresenter {
#if DEBUG #if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize) logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
#endif #endif
#if os(macOS)
// Windowed (composited) the DCP swapID-panic mitigation mechanism (see
// `WindowedPresentMode`). Toggle the layer property BEFORE vending a drawable so the
// vend matches how it will be presented; drained here on the render thread, flipped
// exactly once per mode change.
stagingLock.lock()
let windowedMode = windowedPresentStaged
stagingLock.unlock()
if windowedMode != windowedPresentActive {
windowedPresentActive = windowedMode
layer.presentsWithTransaction = windowedMode == .transaction
if windowedMode != .surface, !surfacePool.isEmpty {
// Leaving surface mode (fullscreen entry / mechanism A/B): drop the pool at 5K
// it holds >100 MB, and re-entering rebuilds it in one frame. SessionPresenter
// clears the surface layer's contents on main.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
presenterLog.info(
"stage2: windowed present mode \(windowedMode.rawValue, privacy: .public) (DCP swapID-panic mitigation)")
}
if windowedMode == .surface {
// No image queue at all: render into a pooled IOSurface and swap it into the
// sibling layer's contents. The drawable/queue tail below never runs.
return encodeToSurface(
targetSize: targetSize, pipeline: pipeline, onPresented: onPresented,
keepAlive: keepAlive, bind: bind)
}
#endif
if let providedDrawable, if let providedDrawable,
providedDrawable.texture.pixelFormat != layer.pixelFormat { providedDrawable.texture.pixelFormat != layer.pixelFormat {
return false // config outran the vend (HDR flip) next vend has the new format return false // config outran the vend (HDR flip) next vend has the new format
@@ -974,6 +963,61 @@ public final class MetalVideoPresenter {
} }
#endif #endif
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
#if os(macOS)
if windowedPresentActive == .transaction {
// Windowed DCP mitigation: present the drawable THROUGH a Core Animation transaction
// (`presentsWithTransaction`, set above) instead of the async image queue, so the swap
// commits with the layer tree and stays in lockstep with the compositor (no out-of-band
// flip to race WindowServer's swaps). Wait until the GPU work is scheduled (contents
// will be ready p50 ~0.1 ms), then present inside an EXPLICIT CATransaction ON THIS
// RENDER THREAD and `flush()`. `presentAtMediaTime` does not apply the transaction
// paces.
//
// Threading history, because BOTH failure modes shipped or nearly shipped:
// A bare `present()` from this thread (no transaction) never flushes nothing
// commits a runloop-less thread's implicit transaction, so drawables are never
// released; after maximumDrawableCount vends `nextDrawable()` blocks forever and
// the stream FREEZES (the fullscreenwindowed switch did exactly this).
// The explicit begin/commit alone is NOT enough either: this thread has an ACTIVE
// implicit transaction (the layer mutations above drawableSize/colour created
// it), so the explicit transaction NESTS inside it and its commit defers to the
// implicit one that never comes. The harness reproduced the exact freeze: every
// present reported presentedTime=0, nothing reached glass. `CATransaction.flush()`
// pushes the implicit transaction (present included) to the render server NOW.
// The original fix hopped to MAIN and presented there correct, but slow in the
// field (presents=55 @ fps=240, display_p50 18.6 ms on the 240 Hz Studio): each
// present lands a runloop turn late, and main's own implicit transaction batches
// enrolled presents at runloop-iteration rate. Kept as PUNKTFUNK_TXN_PRESENT=main.
// The off-main commit measured immune to main-thread churn in the harness
// (2026-07-21: glass p50 ~10 ms at 240 Hz full-size, cadence a clean 4.17 ms).
commandBuffer.commit()
let schedStart = CACurrentMediaTime()
commandBuffer.waitUntilScheduled()
let schedMs = (CACurrentMediaTime() - schedStart) * 1000
let commitStart = CACurrentMediaTime()
if txnPresentOnMain {
let presentedDrawable = drawable
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
presentedDrawable.present()
CATransaction.commit()
}
} else {
CATransaction.begin()
CATransaction.setDisableActions(true)
drawable.present()
CATransaction.commit()
CATransaction.flush()
}
windowedDiag.record(
schedMs: schedMs, commitMs: (CACurrentMediaTime() - commitStart) * 1000,
mode: .transaction)
return true
}
#endif
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment); // Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing. // immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime { if let presentAtMediaTime {
@@ -981,12 +1025,146 @@ public final class MetalVideoPresenter {
} else { } else {
commandBuffer.present(drawable) commandBuffer.present(drawable)
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
commandBuffer.commit() commandBuffer.commit()
return true return true
} }
#if os(macOS)
/// The WINDOWED `surface` present tail (see `WindowedPresentMode.surface`): render with the
/// same per-frame pipeline into a pooled IOSurface and hand it to `surfaceLayer.contents`
/// from the command buffer's COMPLETION handler, inside an explicit CATransaction + flush
/// (the same off-main commit discipline as the transactional present an ordinary
/// damaged-layer update on WindowServer's own composite cadence, no image queue anywhere).
/// RENDER THREAD. `onPresented` is stamped right after the contents swap commits the
/// closest observable analogue of "reached glass" here (the composite follows within a
/// refresh, so the display-stage meters read slightly OPTIMISTIC in this mode).
///
/// The pool tracks `hdrActive`: bgra8 for SDR, rgba16Float tagged BT.2100 PQ for HDR
/// `configure` already ran, so the caller's `pipeline` attachment format always matches.
/// HDR OPEN RISK (why this whole mode is a prototype): whether the compositor honors the
/// PQ tag + EDR for plain-CALayer IOSurface contents needs an on-glass eyeball; the metal
/// layer underneath keeps `wantsExtendedDynamicRangeContent` as the EDR anchor (the harness
/// measured the display's EDR headroom engaging with this arrangement).
private func encodeToSurface(
targetSize: CGSize, pipeline: MTLRenderPipelineState,
onPresented: ((Int64?) -> Void)?,
keepAlive: [Any], bind: (MTLRenderCommandEncoder) -> Void
) -> Bool {
ensureSurfacePool(size: targetSize, hdr: hdrActive)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipeline)
bind(encoder)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let diag = windowedDiag
let commitStamp = CACurrentMediaTime()
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // sources pinned until the GPU finished sampling
let completedAt = CACurrentMediaTime()
// Swap on THIS Metal completion thread: explicit transaction + flush, so the commit
// reaches the render server now, independent of main (completion handlers for one
// queue fire in execution order, so swaps can't reorder).
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
CATransaction.flush()
diag.record(
schedMs: (completedAt - commitStamp) * 1000,
commitMs: (CACurrentMediaTime() - completedAt) * 1000, mode: .surface)
onPresented?(Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size`/`hdr` 4 IOSurface render targets (one on glass, one
/// committed in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the
/// pool empty; the caller returns false and the ring's putBack + display-link retry take
/// over.
private func ensureSurfacePool(size: CGSize, hdr: Bool) {
guard size != surfacePoolSize || hdr != surfacePoolHDR else { return }
surfacePool.removeAll()
surfacePoolSize = size
surfacePoolHDR = hdr
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// rgba16Float (8 B/px) carries the PQ-encoded HDR samples; bgra8 the SDR ones. 256-byte
// row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerElement = hdr ? 8 : 4
let bytesPerRow = ((w * bytesPerElement) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: bytesPerElement,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: hdr
? kCVPixelFormatType_64RGBAHalf : kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: hdr ? .rgba16Float : .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
if hdr, let name = CGColorSpace(name: CGColorSpace.itur_2100_PQ)?.name {
// Tag the surface BT.2100 PQ so the compositor interprets the half-float
// samples as PQ-encoded HDR (the CALayer-contents analogue of the metal
// layer's colorspace).
IOSurfaceSetValue(surface, "IOSurfaceColorSpace" as CFString, name)
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
// The EDR request rides the SURFACE layer too (its contents are what composite); the
// metal layer underneath keeps its own from configureColor as the anchor. Layer flags
// are committed by the next swap's transaction flush.
surfaceLayer.wantsExtendedDynamicRangeContent = hdr
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the /// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained. /// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture( private func makeTexture(
@@ -142,20 +142,17 @@ enum PresentPriority: Equatable {
final class SessionPresenter { final class SessionPresenter {
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2 /// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
/// default, macOS PyroWave sessions ALSO get glass gating a kernel-panic mitigation, not a /// default, macOS PyroWave sessions ALSO get glass gating for SMOOTHNESS, not as the panic
/// latency tweak. macOS's DCP panics ("mismatched swapID's" @UnifiedPipeline.cpp, the whole /// fix (that is the windowed transactional present see `setComposited`). PyroWave's wavelet
/// machine dies) when WindowServer's swap submissions race, and the reliable trigger is /// decode is near-instant Metal compute, so a network clump presents within the same
/// out-of-band CAMetalLayer presents (displaySyncEnabled=false mandatory for us, see /// millisecond, and it is the codec that sustains stream rates above the panel's refresh; the
/// MetalVideoPresenter's init) arriving faster than the compositor latches them in a /// glass gate admits one presented-but-undisplayed swap at a time (serialized on the on-glass
/// COMPOSITED (windowed) session. Arrival pacing does exactly that with PyroWave: the wavelet /// callback, 100 ms stale backstop) so those bursts coalesce in the newest-wins ring instead
/// decode is near-instant Metal compute, so a network clump of frames presents within the /// of flooding the queue. (Glass pacing was ALSO the original DCP-panic mitigation attempt
/// same millisecond, and PyroWave is the codec that sustains stream rates above the panel's /// disproven: a fully serialized stream still panicked, which is why the real fix moved to the
/// refresh. The glass gate admits one presented-but-undisplayed swap at a time (serialized on /// present mechanism.) An explicit stage-2 pick (setting/env) still forces arrival pacing
/// the on-glass callback, 100 ms stale backstop), which removes the racing pattern outright; /// that A/B lever must stay honest. VideoToolbox codecs keep arrival pacing: decode latency
/// frames the panel couldn't have shown anyway coalesce in the newest-wins ring. An explicit /// spaces their presents.
/// stage-2 pick (setting/env) still forces arrival pacing that A/B lever must stay honest.
/// VideoToolbox codecs keep arrival pacing: decode latency spaces their presents, and years
/// of stage-2 defaults there predate any panic report.
static func pacing( static func pacing(
for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec
) -> PresentPacing { ) -> PresentPacing {
@@ -178,10 +175,25 @@ final class SessionPresenter {
/// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing /// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing
/// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate. /// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate.
/// ///
#if os(macOS)
/// Resolve the windowed (composited) present MECHANISM for this session the DCP
/// swapID-panic mitigation picker (see `WindowedPresentMode`). The
/// `PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface` env lever wins (dev A/B);
/// otherwise the user's safe-present setting: ON/unset `.transaction` (the validated
/// mitigation), OFF `.async` (the fast pre-mitigation path the panic returns on
/// affected high-refresh setups; the Settings caption says so). `.surface` is currently
/// env-only (prototype HDR-composite verification owed). Fullscreen always presents
/// async regardless (`setComposited`). Internal (not private) for unit tests.
static func windowedPresentMode(setting: Bool?, env: String?) -> WindowedPresentMode {
if let env, let mode = WindowedPresentMode(rawValue: env) { return mode }
return (setting ?? true) ? .transaction : .async
}
#endif
/// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder /// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder
/// stays reproducible on-device; macOS is pinned to 1, env ignored glass pacing exists /// stays reproducible on-device; macOS is pinned to 1, env ignored a deeper gate only builds
/// there as the DCP swapID kernel-panic mitigation (see `pacing`), and STRICT present /// a standing queue (see above), and macOS glass pacing exists for PyroWave smoothness
/// serialization is its point. Internal (not private) for unit tests. /// (see `pacing`), where depth 1 is the point. Internal (not private) for unit tests.
static func gateDepth(env: String?) -> Int { static func gateDepth(env: String?) -> Int {
#if os(macOS) #if os(macOS)
return 1 return 1
@@ -196,10 +208,16 @@ final class SessionPresenter {
private var stage2Link: CADisplayLink? private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer? private var metalLayer: CAMetalLayer?
#if os(macOS) #if os(macOS)
/// The windowed-mode PyroWave present target (sibling above `metalLayer`) and the last /// The windowed present MECHANISM this session runs while composited (resolved once per
/// routing pushed to the pipeline see `setComposited`. Main-thread only, like all of this. /// session in `start` the user's safe-present setting + the PUNKTFUNK_WINDOWED_PRESENT
/// dev override) and the routing last pushed to the pipeline see `setComposited` (the DCP
/// swapID-panic mitigation). Main-thread only, like all of this.
private var windowedMode: WindowedPresentMode = .transaction
private var windowedPresentApplied: WindowedPresentMode = .async
/// The windowed `surface` present target (sibling above `metalLayer`, transparent while
/// unused) installed whenever stage-2 runs so a mechanism flip never has to mutate the
/// layer tree mid-session.
private var surfaceLayer: CALayer? private var surfaceLayer: CALayer?
private var surfacePresentsActive = false
#endif #endif
private var connection: PunktfunkConnection? private var connection: PunktfunkConnection?
/// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's /// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's
@@ -283,11 +301,17 @@ final class SessionPresenter {
baseLayer.addSublayer(metal) baseLayer.addSublayer(metal)
metalLayer = metal metalLayer = metal
#if os(macOS) #if os(macOS)
// The windowed-PyroWave present target sits ABOVE the metal layer: transparent (nil windowedPresentApplied = .async
// contents) while the metal path presents, covering it while surface presents run. // Resolve THIS session's windowed mechanism once (setting + dev env lever)
// `setComposited` routes between it and fullscreen-async from every layout.
windowedMode = Self.windowedPresentMode(
setting: UserDefaults.standard.object(
forKey: DefaultsKey.windowedSafePresent) as? Bool,
env: ProcessInfo.processInfo.environment["PUNKTFUNK_WINDOWED_PRESENT"])
// The surface present target sits ABOVE the metal layer: transparent (nil contents)
// unless the surface mechanism actually presents, covering it while it does.
baseLayer.addSublayer(pipeline.surfaceLayer) baseLayer.addSublayer(pipeline.surfaceLayer)
surfaceLayer = pipeline.surfaceLayer surfaceLayer = pipeline.surfaceLayer
surfacePresentsActive = false
#endif #endif
stage2 = pipeline stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger // The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
@@ -432,19 +456,23 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
/// Route presents for the window's composited state (MAIN thread the view pushes it on /// Route presents for the window's composited state (MAIN thread the view pushes it on
/// every layout, which fullscreen transitions always trigger). PyroWave sessions in a /// every layout, which fullscreen transitions always trigger). A COMPOSITED (windowed)
/// COMPOSITED (windowed) session present via `surfaceLayer` contents instead of the /// session presents through this session's resolved mitigation mechanism (`windowedMode`
/// CAMetalLayer image queue the DCP "mismatched swapID's" kernel-panic mitigation (see /// transactional by default, see `windowedPresentMode`) instead of the async image queue
/// `MetalVideoPresenter.surfaceLayer`; the metal-swap race survives glass pacing, so pacing /// the DCP "mismatched swapID's" kernel-panic mitigation (see `MetalVideoPresenter`; the
/// alone was not enough). VT codecs keep the metal path: no panic reports there, and their /// async-swap race survives glass pacing, so pacing alone was not enough). ALL codecs:
/// HDR/EDR presentation has no surface-contents equivalent wired. /// PyroWave hit it 2026-07-18 and windowed HEVC hit the same 240 Hz Mac Studio 2026-07-21
/// it is the async image queue itself, not any codec or present rate. Fullscreen keeps the
/// async path (direct scanout, lowest latency, no panic there). The full HDR/EDR render
/// path is preserved in every mechanism.
func setComposited(_ composited: Bool) { func setComposited(_ composited: Bool) {
guard let stage2, let connection else { return } guard let stage2 else { return }
let wantsSurface = composited && connection.videoCodec == .pyrowave let mode: WindowedPresentMode = composited ? windowedMode : .async
guard wantsSurface != surfacePresentsActive else { return } guard mode != windowedPresentApplied else { return }
surfacePresentsActive = wantsSurface let wasSurface = windowedPresentApplied == .surface
stage2.setSurfacePresents(wantsSurface) windowedPresentApplied = mode
if !wantsSurface { stage2.setWindowedPresent(mode)
if wasSurface {
// Uncover the metal layer NOW (its last drawable is still attached, so fullscreen // Uncover the metal layer NOW (its last drawable is still attached, so fullscreen
// entry shows the previous frame until the next present no black flash). // entry shows the previous frame until the next present no black flash).
CATransaction.begin() CATransaction.begin()
@@ -471,7 +499,7 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
surfaceLayer?.removeFromSuperlayer() surfaceLayer?.removeFromSuperlayer()
surfaceLayer = nil surfaceLayer = nil
surfacePresentsActive = false windowedPresentApplied = .async
#endif #endif
connection = nil connection = nil
} }
@@ -1114,15 +1114,15 @@ public final class Stage2Pipeline {
} }
#if os(macOS) #if os(macOS)
/// The windowed-mode PyroWave present target (see `MetalVideoPresenter.surfaceLayer` the /// Forward the windowed present mechanism (MAIN thread see
/// DCP swapID-panic mitigation). The hosting view installs it as a sibling above `layer`. /// `MetalVideoPresenter.setWindowedPresent`, the DCP swapID-panic mitigation).
public var surfaceLayer: CALayer { presenter.surfaceLayer } func setWindowedPresent(_ mode: WindowedPresentMode) {
presenter.setWindowedPresent(mode)
/// Forward the windowed-vs-fullscreen present routing (MAIN thread see
/// `MetalVideoPresenter.setSurfacePresents`).
public func setSurfacePresents(_ on: Bool) {
presenter.setSurfacePresents(on)
} }
/// The windowed `surface` present target the hosting SessionPresenter installs as a sibling
/// ABOVE `layer` (transparent while unused see `MetalVideoPresenter.surfaceLayer`).
var surfaceLayer: CALayer { presenter.surfaceLayer }
#endif #endif
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen` /// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
@@ -38,10 +38,11 @@ private let streamInputDebug =
/// dragged deltas become the relative motion StreamLayerView forwards), and hide it. /// dragged deltas become the relative motion StreamLayerView forwards), and hide it.
/// hide/unhide and associate are balanced via `captured`. /// hide/unhide and associate are balanced via `captured`.
/// ///
/// In CLIENT-SIDE-CURSOR mode (gamescope, whose capture carries no host cursor) this is a /// In the DESKTOP mouse model (absolute pointer, remote-desktop-sweep M1) this is a no-op:
/// no-op: the local cursor stays visible and free, and StreamLayerView forwards ABSOLUTE /// the pointer stays free (entering and leaving the stream at will) and StreamLayerView
/// positions instead the visible system cursor IS the on-screen cursor. `disassociate` /// forwards ABSOLUTE positions instead; the local cursor is hidden only while over the view
/// selects between the two; `release()` only undoes what `capture` actually did. /// (cursor rects). `disassociate` selects between the two; `release()` only undoes what
/// `capture` actually did.
private final class CursorCapture { private final class CursorCapture {
private var captured = false private var captured = false
/// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode), /// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode),
@@ -207,14 +208,17 @@ public final class StreamLayerView: NSView {
/// forwarded). Main-thread only. /// forwarded). Main-thread only.
public private(set) var captured = false public private(set) var captured = false
/// Client-side-cursor mode: when true the local system cursor stays VISIBLE over the /// Desktop (absolute) mouse model remote-desktop-sweep M1: when true the pointer is
/// stream and the mouse monitor forwards ABSOLUTE positions (the visible cursor is the /// never disassociated (it enters and leaves the stream freely) and the mouse monitor
/// on-screen cursor gamescope draws none, so no double cursor); when false the existing /// forwards ABSOLUTE positions through the letterbox; the local cursor is hidden only
/// captured/disassociated relative path runs unchanged. Initialized at session start from /// while over this view (cursor rects the host's composited cursor, tracking our
/// the `cursorMode` setting + the host's resolved compositor, toggled live by C. A live /// sends, is the one you see) and reappears the moment it leaves. When false the
/// flip re-engages capture in the new mode so disassociation + the abs/rel choice swap /// captured/disassociated relative path runs unchanged. Initialized at session start
/// atomically. Main-thread only. /// from the `mouseMode` setting gated by the host's resolved compositor (gamescope's
private var cursorVisible = false /// EIS is relative-only absolute sends would be dropped, so it pins to capture);
/// flipped live by M. A live flip re-engages capture in the new model so
/// disassociation + the abs/rel choice swap atomically. Main-thread only.
private var desktopMouse = false
/// One-shot auto-engage request (stream start, trust confirmed) attempted as soon /// One-shot auto-engage request (stream start, trust confirmed) attempted as soon
/// as the view is in a window with real bounds, then dropped, so it can never fire /// as the view is in a window with real bounds, then dropped, so it can never fire
/// surprisingly later (e.g. on a resize). /// surprisingly later (e.g. on a resize).
@@ -440,9 +444,9 @@ public final class StreamLayerView: NSView {
// If the cursor grab is refused (e.g. the reactivating click arrives before the app is // If the cursor grab is refused (e.g. the reactivating click arrives before the app is
// frontmost), stay released so the NEXT click retries never latch captured=true over // frontmost), stay released so the NEXT click retries never latch captured=true over
// a free cursor, which would make mouseDown's `!captured` guard reject every later click. // a free cursor, which would make mouseDown's `!captured` guard reject every later click.
// In client-side-cursor mode there is no grab (the cursor stays visible) capture // In the desktop mouse model there is no grab (the pointer stays free) capture
// always engages and the monitor forwards absolute positions instead. // always engages and the monitor forwards absolute positions instead.
guard cursorCapture.capture(in: self, disassociate: !cursorVisible) else { return } guard cursorCapture.capture(in: self, disassociate: !desktopMouse) else { return }
inputCapture?.setForwarding(true, suppressClick: fromClick) inputCapture?.setForwarding(true, suppressClick: fromClick)
// Install AFTER the warp + setForwarding: the engage warp generates no forwarded // Install AFTER the warp + setForwarding: the engage warp generates no forwarded
// delta (the monitor isn't up yet), and the engage click's suppression latch is // delta (the monitor isn't up yet), and the engage click's suppression latch is
@@ -450,6 +454,7 @@ public final class StreamLayerView: NSView {
installMouseMonitor() installMouseMonitor()
captured = true captured = true
window?.makeFirstResponder(self) window?.makeFirstResponder(self)
window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages
notifyCaptureChange(true) notifyCaptureChange(true)
} }
@@ -459,9 +464,28 @@ public final class StreamLayerView: NSView {
cursorCapture.release() cursorCapture.release()
inputCapture?.setForwarding(false) inputCapture?.setForwarding(false)
captured = false captured = false
window?.invalidateCursorRects(for: self)
notifyCaptureChange(false) notifyCaptureChange(false)
} }
/// A fully transparent cursor for the desktop mouse model's hide-over-view rect
/// an empty 1×1 image draws nothing.
private static let invisibleCursor = NSCursor(
image: NSImage(size: NSSize(width: 1, height: 1)), hotSpot: .zero)
/// Desktop mouse model: the local cursor is hidden while over the stream (the host's
/// composited cursor, tracking our absolute sends, is the one you see) and reappears
/// the moment it leaves the view AppKit applies/removes the rect's cursor for us,
/// so there is no hide/unhide balancing to get wrong. Capture model instead hides
/// globally via `CursorCapture` (the pointer can't leave the view there).
override public func resetCursorRects() {
if captured && desktopMouse {
addCursorRect(bounds, cursor: Self.invisibleCursor)
} else {
super.resetCursorRects()
}
}
/// A single local monitor for motion + buttons, installed only while captured. A local /// A single local monitor for motion + buttons, installed only while captured. A local
/// monitor is more robust than view overrides for relative motion: it sidesteps the /// monitor is more robust than view overrides for relative motion: it sidesteps the
/// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and /// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and
@@ -473,12 +497,12 @@ public final class StreamLayerView: NSView {
/// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're /// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're
/// inert locally. /// inert locally.
/// ///
/// In client-side-cursor mode the cursor is NOT frozen, so bare `.mouseMoved` events are /// In the desktop mouse model the cursor is NOT frozen, so bare `.mouseMoved` events are
/// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and /// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and
/// restore it on removal so absolute hover-motion keeps flowing without a click held. /// restore it on removal so absolute hover-motion keeps flowing without a click held.
private func installMouseMonitor() { private func installMouseMonitor() {
guard mouseEventMonitor == nil else { return } guard mouseEventMonitor == nil else { return }
if cursorVisible { if desktopMouse {
savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents
window?.acceptsMouseMovedEvents = true window?.acceptsMouseMovedEvents = true
} }
@@ -490,8 +514,8 @@ public final class StreamLayerView: NSView {
guard let self, self.captured, let ic = self.inputCapture else { return event } guard let self, self.captured, let ic = self.inputCapture else { return event }
switch event.type { switch event.type {
case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged: case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged:
if self.cursorVisible { if self.desktopMouse {
// Client-side cursor: forward the ABSOLUTE position (mapped through the // Desktop mouse model: forward the ABSOLUTE position (mapped through the
// aspect-fit letterbox into host pixels), the same path the iPad pointer // aspect-fit letterbox into host pixels), the same path the iPad pointer
// fallback uses. Events in the letterbox bars are dropped (nil host point). // fallback uses. Events in the letterbox bars are dropped (nil host point).
if let p = self.hostPoint(from: event) { if let p = self.hostPoint(from: event) {
@@ -609,14 +633,27 @@ public final class StreamLayerView: NSView {
// be a cursor trap with dead input. // be a cursor trap with dead input.
self?.releaseCapture() self?.releaseCapture()
} }
// C flips the client-side cursor live. Only the key window's stream owns it (same // M flips the mouse model (capture desktop) live the SDL clients' identical
// guard as the capture toggle). Re-engage capture in the new mode so disassociation // chord. Only the key window's stream owns it (same guard as the capture toggle).
// and the absolute/relative forwarding choice swap atomically releaseCapture restores // Re-engage capture in the new model so disassociation and the absolute/relative
// the old mode's grab (if any), engageCapture installs the new one. // forwarding choice swap atomically releaseCapture restores the old model's grab
// C would flip the client-side cursor live NEUTERED while the feature is disabled // (if any), engageCapture installs the new one. On a gamescope host the chord is a
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only // no-op: its EIS grants only a relative pointer, so the desktop model's absolute
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands. // sends would be silently dropped (pointer stuck = "all input dead").
capture.onToggleCursor = {} capture.onToggleMouseMode = { [weak self] in
guard let self, self.window?.isKeyWindow == true,
let conn = self.connection else { return }
guard conn.resolvedCompositor != .gamescope else {
streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only")
return
}
let wasCaptured = self.captured
if wasCaptured { self.releaseCapture() }
self.desktopMouse.toggle()
if wasCaptured { self.engageCapture(fromClick: false) }
self.window?.invalidateCursorRects(for: self)
streamInputLog.info("mouse mode: \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by // The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout. // the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in capture.onReleaseCapture = { [weak self] in
@@ -643,15 +680,18 @@ public final class StreamLayerView: NSView {
capture.start() capture.start()
inputCapture = capture inputCapture = capture
// Client-side cursor is TEMPORARILY DISABLED. It positions the host cursor with ABSOLUTE // Desktop (absolute) mouse model resolved at session start from the mouseMode
// events, but gamescope's input socket (EIS) grants only a relative pointer, so those are // setting, gated by the host's compositor: gamescope's input socket (EIS) grants
// silently dropped the pointer never moves and clicks/scroll land on the stuck position // only a relative pointer, so absolute sends would be silently dropped there
// (looks like "all input dead"). gamescope is exactly the compositor Auto enabled it for. // (pointer stuck = "all input dead") pinned to capture. M flips it live.
// Forced off until per-compositor gating (KWin/GNOME/Sway have absolute) or a synthetic- let mode = MouseInputMode(
// cursor-over-relative path lands; the resolution logic below is kept for that. See the rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? ""
// C handler (also neutered) and the cursorMode setting (hidden). ) ?? .capture
cursorVisible = false let absOK = connection.resolvedCompositor != .gamescope
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent) desktopMouse = mode == .desktop && absOK
if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
}
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2 // Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by // (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
@@ -700,9 +740,9 @@ public final class StreamLayerView: NSView {
private func layoutPresenter() { private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1) presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
// Present routing tracks the window's composited state (fullscreen transitions always // Present routing tracks the window's composited state (fullscreen transitions always
// re-layout, so this stays current): windowed PyroWave presents via surface contents // re-layout, so this stays current): a windowed session presents through a Core Animation
// the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). A view // transaction the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited).
// not yet in a window counts as composited (the safe default). // A view not yet in a window counts as composited (the safe default).
presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false)) presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false))
// Feed the follower only once in a window (backing scale is real then) and with real // Feed the follower only once in a window (backing scale is real then) and with real
// bounds a pre-window layout would report point-sized dimensions. // bounds a pre-window layout would report point-sized dimensions.
@@ -70,6 +70,16 @@ public enum DefaultsKey {
/// (lowest latency the default, OFF). Resolved once per session; /// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header. /// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync" public static let vsync = "punktfunk.vsync"
/// macOS: present WINDOWED sessions in lockstep with the system compositor (the DCP
/// "mismatched swapID's" kernel-panic mitigation see SessionPresenter.windowedPresentMode
/// and the MetalVideoPresenter saga notes). ON/unset (the default): windowed presents ride
/// a Core Animation transaction validated panic-free on the 240 Hz repro machine, at a
/// small display-latency cost vs the raw path. OFF: windowed sessions keep the fast async
/// image queue ON AFFECTED SETUPS (high-refresh displays) THAT PATH KERNEL-PANICS THE
/// WHOLE MAC, which is why the default is ON. Fullscreen always presents async (fast path)
/// regardless. Resolved once per session; PUNKTFUNK_WINDOWED_PRESENT=async|transaction|
/// surface overrides it for dev A/B.
public static let windowedSafePresent = "punktfunk.windowedSafePresent"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor, /// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical /// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off, /// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
@@ -84,8 +94,11 @@ public enum DefaultsKey {
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth. /// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444" public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts" public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never". /// Physical-mouse model (macOS): "capture" (pointer lock + relative, the default) or
public static let cursorMode = "punktfunk.cursorMode" /// "desktop" (uncaptured absolute pointer) the cross-client `mouse_mode`. Replaces the
/// never-shipped "punktfunk.cursorMode" (auto/always/never client-side-cursor setting,
/// which was hidden while disabled and had no readers).
public static let mouseMode = "punktfunk.mouseMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by /// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is /// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across /// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
@@ -316,6 +316,41 @@ final class PresentPacingTests: XCTestCase {
SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline) SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline)
} }
// MARK: - Windowed present mechanism (the macOS DCP swapID-panic mitigation picker)
#if os(macOS)
/// The safe-present setting: ON/unset the validated transactional mitigation; an explicit
/// OFF the fast async path (the user accepted the affected-setup panic risk). The
/// PUNKTFUNK_WINDOWED_PRESENT env lever overrides both ways, `surface` is env-only (the
/// prototype mechanism), and garbage/empty env values are "unset", not an override.
func testWindowedPresentModeResolution() {
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: nil), .transaction,
"unset defaults to the panic mitigation")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: nil), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: nil), .async,
"an explicit opt-out gets the fast async path")
// The dev env lever wins over the setting, both directions.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "async"), .async)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "transaction"),
.transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "surface"), .surface,
"the surface prototype is reachable via env only")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "surface"), .surface)
// Garbage/empty env = unset.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: "garbage"), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: ""), .async)
}
#endif
// MARK: - Glass-gate depth // MARK: - Glass-gate depth
/// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue /// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue
+11 -2
View File
@@ -16,6 +16,8 @@
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games) # PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect) # PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990) # PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_CONNECT_TIMEOUT connect budget in seconds (optional; the plugin stretches it after
# firing Wake-on-LAN so the connect survives the host's resume)
# PF_APPID flatpak app id (default io.unom.Punktfunk) # PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH) # PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
# #
@@ -61,10 +63,17 @@ if [ -z "${PF_HOST:-}" ]; then
echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2 echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2
exit 2 exit 2
fi fi
# Trailing args shared by both streaming execs. A stretched connect budget rides along when the
# plugin set one (it just fired Wake-on-LAN, so the host may still be resuming); an older flatpak
# without --connect-timeout ignores the flag harmlessly (hand-scanned argv).
set -- --fullscreen
if [ -n "${PF_CONNECT_TIMEOUT:-}" ]; then
set -- --connect-timeout "$PF_CONNECT_TIMEOUT" "$@"
fi
if [ -n "${PF_LAUNCH:-}" ]; then if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title. # A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" "$@"
fi fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" "$@"
+33 -8
View File
@@ -70,7 +70,9 @@ function setShortcutHidden(appId: number, hidden: boolean): void {
}; };
// Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId). // Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId).
const ART_VERSION = 2; // v3: CI zips through 0.17.1 shipped no assets/ at all, yet v2 was still recorded as applied
// on those installs — the bump makes them re-apply once on the first build that has the files.
const ART_VERSION = 3;
function artKey(appId: number): string { function artKey(appId: number): string {
return `punktfunk:shortcutArt:${appId}`; return `punktfunk:shortcutArt:${appId}`;
} }
@@ -79,7 +81,7 @@ function artKey(appId: number): string {
* Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per * Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per
* appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call. * appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call.
*/ */
async function applyArtwork(appId: number): Promise<void> { async function applyArtwork(appId: number, isRetry = false): Promise<void> {
try { try {
if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) { if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) {
return; return;
@@ -91,16 +93,29 @@ async function applyArtwork(appId: number): Promise<void> {
[art.logo, 2], [art.logo, 2],
[art.gridwide, 3], [art.gridwide, 3],
]; ];
let applied = false;
for (const [data, assetType] of assets) { for (const [data, assetType] of assets) {
if (data) { if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType); await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
applied = true;
} }
} }
if (art.icon_path) { if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path); SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
applied = true;
}
// Only record "done" when something actually landed — a plugin build whose assets/ is
// missing/empty must keep retrying on later mounts instead of poisoning the marker.
if (applied) {
localStorage.setItem(artKey(appId), `${ART_VERSION}`);
} }
localStorage.setItem(artKey(appId), `${ART_VERSION}`);
} catch (e) { } catch (e) {
// A shortcut fresh out of AddShortcut may not be registered yet (the same race
// setShortcutHidden defers around) — one deferred second attempt, then leave it to
// the next mount.
if (!isRetry) {
setTimeout(() => void applyArtwork(appId, true), 2500);
}
console.warn("punktfunk: shortcut artwork not applied", e); console.warn("punktfunk: shortcut artwork not applied", e);
} }
} }
@@ -157,7 +172,9 @@ async function ensureControllerConfig(): Promise<void> {
return; return;
} }
const r = await applyControllerConfig(SHORTCUT_NAME); const r = await applyControllerConfig(SHORTCUT_NAME);
if (r?.ok) { // `ok` alone isn't done: with zero account configset dirs (fresh Steam) the backend
// succeeds without pointing any account at the template — keep retrying until one lands.
if (r?.ok && (r.applied ?? []).some((a) => a.startsWith("configset:"))) {
localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`); localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`);
} else { } else {
console.warn("punktfunk: controller config not fully applied", r); console.warn("punktfunk: controller config not fully applied", r);
@@ -283,13 +300,21 @@ export async function launchStream(
opts: LaunchOpts = {}, opts: LaunchOpts = {},
): Promise<void> { ): Promise<void> {
// Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now // Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now
// so it races with the shortcut setup (near-zero added latency), and awaited just before RunGame. // so it races with the shortcut setup (near-zero added latency); its outcome is needed below
// (the connect budget), and RunGame follows the await either way, so nothing is slower for it.
// Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is // Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is
// known), and the connect that follows has its own retry window, so a failure never blocks launch. // known), and the connect that follows has its own retry window, so a failure never blocks launch.
const waking = wake(host, port).catch(() => ({ ok: false })); const waking = wake(host, port).catch(() => ({ ok: false }));
const { appId, runner } = await ensureStreamShortcut(); const [{ appId, runner }, woke] = await Promise.all([ensureStreamShortcut(), waking]);
const target = port && port !== 9777 ? `${host}:${port}` : host; const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`]; const env = [`PF_HOST=${target}`];
// A magic packet actually went out (a MAC was known), so the host may be mid-resume from
// suspend — that takes far longer than the client's default 15 s connect budget. Stretch the
// budget so the client's wake-tolerant dial keeps retrying across the resume; against an
// already-awake host the connect still lands in under a second, so this costs nothing.
if (woke.ok) {
env.push("PF_CONNECT_TIMEOUT=75");
}
if (opts.browse) { if (opts.browse) {
env.push("PF_BROWSE=1"); env.push("PF_BROWSE=1");
if (opts.mgmt) { if (opts.mgmt) {
@@ -303,9 +328,9 @@ export async function launchStream(
env.push(`PF_LAUNCH=${opts.launchId}`); env.push(`PF_LAUNCH=${opts.launchId}`);
} }
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper // KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_* from the environment. // script rides behind it as an argument and reads PF_* from the environment. The wake was
// awaited above, so the magic packet is out before the connect attempt.
SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`); SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
await waking; // ensure the magic packet is out before the connect attempt
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100); SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
} }
+31
View File
@@ -69,6 +69,14 @@ const TOUCH_MODE_CAPTIONS: &[&str] = &[
"The cursor jumps to your finger — a tap clicks there", "The cursor jumps to your finger — a tap clicks there",
"Real multi-touch reaches the host — for touch-native apps", "Real multi-touch reaches the host — for touch-native apps",
]; ];
/// Physical-mouse model values (persisted) + labels + dynamic captions — same idiom as
/// the touch rows. Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[&str] = &["capture", "desktop"];
const MOUSE_MODE_LABELS: &[&str] = &["Capture (games)", "Desktop (absolute)"];
const MOUSE_MODE_CAPTIONS: &[&str] = &[
"Pointer locks to the stream — relative motion, best for games",
"Pointer moves freely in and out — best for remote desktop work",
];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page. /// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page.
const APP_LICENSE: &str = concat!( const APP_LICENSE: &str = concat!(
@@ -542,6 +550,20 @@ pub fn show(
set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]); set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]);
}); });
} }
let mouse_row = ChoiceRow::new(
&dialog,
inline,
"Mouse input",
MOUSE_MODE_CAPTIONS[0],
MOUSE_MODE_LABELS,
);
{
let w = mouse_row.widget().clone();
mouse_row.connect_changed(move |i| {
let i = (i as usize).min(MOUSE_MODE_CAPTIONS.len() - 1);
set_row_subtitle(&w, MOUSE_MODE_CAPTIONS[i]);
});
}
let inhibit_row = adw::SwitchRow::builder() let inhibit_row = adw::SwitchRow::builder()
.title("Capture system shortcuts") .title("Capture system shortcuts")
.subtitle("Forward Alt+Tab, Super, … to the host while input is captured") .subtitle("Forward Alt+Tab, Super, … to the host while input is captured")
@@ -718,6 +740,12 @@ pub fn show(
touch_row.set_selected(touch_i as u32); touch_row.set_selected(touch_i as u32);
// set_selected never fires the changed hook, so seed the dynamic caption directly. // set_selected never fires the changed hook, so seed the dynamic caption directly.
set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]); set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]);
let mouse_i = MOUSE_MODES
.iter()
.position(|&m| m == s.mouse_mode)
.unwrap_or(0);
mouse_row.set_selected(mouse_i as u32);
set_row_subtitle(mouse_row.widget(), MOUSE_MODE_CAPTIONS[mouse_i]);
let comp_i = COMPOSITORS let comp_i = COMPOSITORS
.iter() .iter()
.position(|&c| c == s.compositor) .position(|&c| c == s.compositor)
@@ -788,6 +816,7 @@ pub fn show(
touch_group.add(touch_row.widget()); touch_group.add(touch_row.widget());
// Group titles are Pango markup — the ampersand must be an entity. // Group titles are Pango markup — the ampersand must be an entity.
let kbm_group = group("Keyboard &amp; mouse", ""); let kbm_group = group("Keyboard &amp; mouse", "");
kbm_group.add(mouse_row.widget());
kbm_group.add(&inhibit_row); kbm_group.add(&inhibit_row);
kbm_group.add(&invert_row); kbm_group.add(&invert_row);
input.add(&touch_group); input.add(&touch_group);
@@ -867,6 +896,8 @@ pub fn show(
} }
s.touch_mode = s.touch_mode =
TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string(); TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string();
s.mouse_mode =
MOUSE_MODES[(mouse_row.selected() as usize).min(MOUSE_MODES.len() - 1)].to_string();
s.forward_pad = chosen_pin.borrow().clone(); s.forward_pad = chosen_pin.borrow().clone();
s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)] s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)]
.to_string(); .to_string();
+12
View File
@@ -498,6 +498,13 @@ async fn session(args: Args) -> Result<()> {
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() { if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444; caps |= punktfunk_core::quic::VIDEO_CAP_444;
} }
// PUNKTFUNK_CLIENT_CHACHA20=1 advertises VIDEO_CAP_CHACHA20 — drives the
// host's ChaCha20-Poly1305 session-cipher resolution (the soft-AES armv7
// negotiation, design/chacha20-session-cipher.md §7) without a webOS build;
// the negotiated cipher is reported in the welcome log line below.
if std::env::var_os("PUNKTFUNK_CLIENT_CHACHA20").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_CHACHA20;
}
caps caps
}, },
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the // `--audio-channels` (default stereo); the probe multistream-decodes + validates the
@@ -535,6 +542,11 @@ async fn session(args: Args) -> Result<()> {
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444, chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format, chroma_format_idc = welcome.chroma_format,
codec = codec_ext(welcome.codec), codec = codec_ext(welcome.codec),
cipher = if welcome.cipher == punktfunk_core::quic::CIPHER_CHACHA20_POLY1305 {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
"session offer" "session offer"
); );
+1
View File
@@ -158,6 +158,7 @@ pub fn run(target: Option<&str>) -> u8 {
v => v, v => v,
}, },
touch_mode: settings_at_start.touch_mode(), touch_mode: settings_at_start.touch_mode(),
mouse_mode: settings_at_start.mouse_mode(),
invert_scroll: settings_at_start.invert_scroll, invert_scroll: settings_at_start.invert_scroll,
json_status, json_status,
on_connected: Some(Box::new(move |fingerprint: [u8; 32]| { on_connected: Some(Box::new(move |fingerprint: [u8; 32]| {
+6
View File
@@ -172,6 +172,11 @@ mod session_main {
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session // defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually. // pump) pins one manually.
display_hdr: None, display_hdr: None,
// The presenter renders the host cursor locally in desktop mouse mode (M2 cursor
// channel); capture-mode sessions keep the composited cursor, so only advertise
// when the session STARTS in desktop mode. The host gates further (Linux portal
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled, mic_enabled: settings.mic_enabled,
clipboard, clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter // The Settings preference (auto → VAAPI where it exists; the presenter
@@ -429,6 +434,7 @@ mod session_main {
v => v, v => v,
}, },
touch_mode: settings.touch_mode(), touch_mode: settings.touch_mode(),
mouse_mode: settings.mouse_mode(),
invert_scroll: settings.invert_scroll, invert_scroll: settings.invert_scroll,
json_status: true, json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| { on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
+18
View File
@@ -90,6 +90,13 @@ const TOUCH_MODES: &[(&str, &str)] = &[
("pointer", "Direct pointer"), ("pointer", "Direct pointer"),
("touch", "Touch passthrough"), ("touch", "Touch passthrough"),
]; ];
/// Physical-mouse presets: `(stored value, display label)` — capture (pointer lock,
/// relative, for games) vs desktop (uncaptured absolute pointer, for remote desktop
/// work). Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[(&str, &str)] = &[
("capture", "Capture (games)"),
("desktop", "Desktop (absolute)"),
];
/// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to /// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to
/// auto-detect when the choice is unavailable. Only meaningful against a Linux host. /// auto-detect when the choice is unavailable. Only meaningful against a Linux host.
const COMPOSITORS: &[(&str, &str)] = &[ const COMPOSITORS: &[(&str, &str)] = &[
@@ -394,6 +401,10 @@ pub(crate) fn settings_page(
let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| { let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| {
s.touch_mode = TOUCH_MODES[i].0.to_string(); s.touch_mode = TOUCH_MODES[i].0.to_string();
}); });
let (mouse_names, mouse_i) = presets(MOUSE_MODES, |v| *v == s.mouse_mode);
let mouse_combo = setting_combo(ctx, "Mouse input", mouse_names, mouse_i, |s, i| {
s.mouse_mode = MOUSE_MODES[i].0.to_string();
});
let invert_scroll_toggle = let invert_scroll_toggle =
setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| { setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| {
s.invert_scroll = on s.invert_scroll = on
@@ -542,6 +553,13 @@ pub(crate) fn settings_page(
out.extend(group( out.extend(group(
Some("Keyboard & mouse"), Some("Keyboard & mouse"),
vec![ vec![
described(
mouse_combo,
"Capture locks the pointer to the stream and sends relative motion — \
best for games. Desktop leaves the pointer free to enter and leave \
the stream and sends absolute positions best for remote desktop \
work. Ctrl+Alt+Shift+M switches live.",
),
described( described(
shortcuts_toggle, shortcuts_toggle,
"Alt+Tab, the Windows key and friends reach the host while the stream \ "Alt+Tab, the Windows key and friends reach the host while the stream \
+16
View File
@@ -24,6 +24,16 @@ use pf_frame::DmabufFrame;
pub trait Capturer: Send { pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>; fn next_frame(&mut self) -> Result<CapturedFrame>;
/// [`next_frame`](Self::next_frame) with a caller-chosen first-frame budget instead of the
/// backend's default. The pipeline retry loop shortens its FIRST attempt's wait: a PipeWire
/// stream connected while gamescope re-inits its headless takeover can negotiate a format,
/// reach `Streaming`, and still never receive a buffer — a fresh connect then delivers within
/// ~0.5 s, so waiting out the full default budget on a doomed stream just delays the retry
/// that fixes it. Backends without an internal wait budget ignore it (the default delegates).
fn next_frame_within(&mut self, _budget: std::time::Duration) -> Result<CapturedFrame> {
self.next_frame()
}
/// Non-blocking: the freshest frame available since the last call, or `None` if none has /// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default /// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal /// just produces a frame each call — fine for instant synthetic sources; the portal
@@ -249,6 +259,12 @@ pub struct ZeroCopyPolicy {
/// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only /// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only
/// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`. /// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`.
pub pyrowave_session: bool, pub pyrowave_session: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (the Linux raw Vulkan
/// Video backend on an H265/AV1 session — resolved by the host facade via
/// `pf_encode::linux_native_nv12_ok`). Gates whether the negotiation PREFERS gamescope's
/// producer-side NV12 pod: libav VAAPI (H264's backend) would misread the two-plane buffer,
/// so H264/GameStream/PyroWave sessions must never see NV12 frames.
pub native_nv12_session: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc, /// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA, /// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise. /// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
+164 -50
View File
@@ -299,29 +299,11 @@ fn spawn_pipewire(
impl Capturer for PortalCapturer { impl Capturer for PortalCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> { fn next_frame(&mut self) -> Result<CapturedFrame> {
// First frame can lag behind format negotiation; later frames arrive at ~fps. Wait in self.frame_within(Duration::from_secs(10))
// short slices so a GPU-import poison (worker death) fails the capture within ~0.5 s }
// instead of sitting out the full first-frame budget.
let deadline = std::time::Instant::now() + Duration::from_secs(10); fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
loop { self.frame_within(budget)
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e),
}
}
} }
fn supports_arrival_wait(&self) -> bool { fn supports_arrival_wait(&self) -> bool {
@@ -417,9 +399,41 @@ impl Capturer for PortalCapturer {
} }
impl PortalCapturer { impl PortalCapturer {
/// The [`Capturer::next_frame`] budget expired (or the thread ended) — turn it into the /// The blocking first-frame wait behind [`Capturer::next_frame`] /
/// diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged. /// [`Capturer::next_frame_within`]. First frame can lag behind format negotiation; later
fn next_frame_timed_out(&self, err: RecvTimeoutError) -> Result<CapturedFrame> { /// frames arrive at ~fps. Wait in short slices so a GPU-import poison (worker death) fails
/// the capture within ~0.5 s instead of sitting out the full first-frame budget.
fn frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
let deadline = std::time::Instant::now() + budget;
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e, budget),
}
}
}
/// The [`frame_within`](Self::frame_within) budget expired (or the thread ended) — turn it
/// into the diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(
&self,
err: RecvTimeoutError,
budget: Duration,
) -> Result<CapturedFrame> {
let within = budget.as_secs_f32();
match err { match err {
RecvTimeoutError::Timeout => { RecvTimeoutError::Timeout => {
// Split the two black-screen root causes apart so the operator gets a cause, not // Split the two black-screen root causes apart so the operator gets a cause, not
@@ -427,9 +441,10 @@ impl PortalCapturer {
// not (no acceptable format / node never emitted a param)? // not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) { if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiated but no buffers \ "no PipeWire frame within {within}s (node {}): format negotiated but no \
arrived the compositor produced no frames (virtual output idle/unmapped, \ buffers arrived the compositor produced no frames (virtual output \
or capture never started)", idle/unmapped, capture never started, or a stream bound during a \
compositor (re)start that will never deliver a reconnect fixes that)",
self.node_id self.node_id
)) ))
} else if self.hdr_offer { } else if self.hdr_offer {
@@ -440,10 +455,10 @@ impl PortalCapturer {
// auto-reconnects) negotiates SDR instead of re-running this same timeout. // auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed(); super::note_hdr_capture_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \ "no PipeWire frame within {within}s (node {}): the compositor never \
the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored monitor in \ accepted the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored \
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \ monitor in HDR mode on GNOME 50+? Downgrading this host to SDR capture; \
to stream SDR", reconnect to stream SDR",
self.node_id self.node_id
)) ))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() { } else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
@@ -452,14 +467,15 @@ impl PortalCapturer {
// retries on the CPU offer instead of failing this same negotiation forever. // retries on the CPU offer instead of failing this same negotiation forever.
pf_zerocopy::note_vaapi_dmabuf_failed(); pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \ "no PipeWire frame within {within}s (node {}): the compositor never \
the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \ accepted the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this \
CPU capture path; the pipeline rebuild will renegotiate without dmabuf", host to the CPU capture path; the pipeline rebuild will renegotiate \
without dmabuf",
self.node_id self.node_id
)) ))
} else { } else {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiation never \ "no PipeWire frame within {within}s (node {}): format negotiation never \
completed the compositor offered no format this consumer accepts \ completed the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param", (pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id self.node_id
@@ -824,6 +840,7 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba, VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb, VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr, VideoFormat::BGR => PixelFormat::Bgr,
VideoFormat::NV12 => PixelFormat::Nv12,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by // The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020). // the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10, VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
@@ -851,6 +868,10 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves, /// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change. /// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64, serial: u64,
/// The compositor-reported hotspot — carried on the overlay for the cursor-forward
/// channel (the blend path uses the pre-adjusted `x`/`y` and never reads it).
hot_x: i32,
hot_y: i32,
} }
impl CursorState { impl CursorState {
@@ -867,6 +888,8 @@ mod pipewire {
h: self.bh, h: self.bh,
rgba: self.rgba.clone(), rgba: self.rgba.clone(),
serial: self.serial, serial: self.serial,
hot_x: self.hot_x.max(0) as u32,
hot_y: self.hot_y.max(0) as u32,
}) })
} }
} }
@@ -989,10 +1012,10 @@ mod pipewire {
.into_inner()) .into_inner())
} }
/// Build a BGRx dmabuf `EnumFormat` pod advertising the EGL-importable `modifiers` as a /// Build a LINEAR/modifier DMA-BUF `EnumFormat` pod. Packed BGRx is the existing import path;
/// mandatory enum Choice; the compositor fixates to one of them that it can allocate, which /// NV12 is gamescope's producer-side RGB→YUV path (opt-in during bring-up).
/// we read back in `param_changed`.
fn build_dmabuf_format( fn build_dmabuf_format(
format: VideoFormat,
modifiers: &[u64], modifiers: &[u64],
preferred: Option<(u32, u32, u32)>, preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> { ) -> Result<Vec<u8>> {
@@ -1003,7 +1026,7 @@ mod pipewire {
pw::spa::param::ParamType::EnumFormat, pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video), pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw), pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, VideoFormat::BGRx), pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
pw::spa::pod::property!( pw::spa::pod::property!(
FormatProperties::VideoSize, FormatProperties::VideoSize,
Choice, Choice,
@@ -1032,6 +1055,22 @@ mod pipewire {
pw::spa::utils::Fraction { num: 240, denom: 1 } pw::spa::utils::Fraction { num: 240, denom: 1 }
), ),
); );
if format == VideoFormat::NV12 {
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorMatrix,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_MATRIX_BT709,
)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorRange,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_RANGE_16_235,
)),
});
}
obj.properties.push(pw::spa::pod::Property { obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier, key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY, flags: pw::spa::pod::PropertyFlags::MANDATORY,
@@ -1347,6 +1386,8 @@ mod pipewire {
cursor.visible = true; cursor.visible = true;
cursor.x = pos_x - hot_x; cursor.x = pos_x - hot_x;
cursor.y = pos_y - hot_y; cursor.y = pos_y - hot_y;
cursor.hot_x = hot_x;
cursor.hot_y = hot_y;
if bmp_off == 0 { if bmp_off == 0 {
// Position-only update — keep the cached bitmap. // Position-only update — keep the cached bitmap.
return; return;
@@ -1604,8 +1645,8 @@ mod pipewire {
} }
} }
// VAAPI zero-copy passthrough: hand the raw dmabuf straight to the encoder, which imports // Raw DMA-BUF passthrough: packed RGB is imported for GPU CSC; producer-native NV12 can
// it into a VA surface and does RGB→NV12 on the GPU video engine. No CUDA importer here. // be consumed by the Vulkan Video encoder without another color conversion.
if ud.vaapi_passthrough { if ud.vaapi_passthrough {
if let Some(fmt) = ud.format { if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf { if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
@@ -1613,9 +1654,41 @@ mod pipewire {
let chunk = datas[0].chunk(); let chunk = datas[0].chunk();
let offset = chunk.offset(); let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32; let stride = chunk.stride().max(0) as u32;
// Native NV12 usually arrives as a two-plane SPA buffer over ONE buffer
// object; plane 1's chunk carries the REAL UV offset/stride (compositors
// may align the Y plane before UV). Pass it through instead of assuming
// contiguity. Each spa_data holds its own (dup'd) fd, so BO identity is
// by inode, not fd number; a genuinely two-BO frame cannot travel through
// the single-fd import — drop it with a diagnosis instead of streaming
// garbage chroma.
let plane1 =
if fmt == PixelFormat::Nv12 && datas.len() >= 2 && datas[1].fd() > 0 {
// SAFETY: zeroed `libc::stat` is a valid POD initializer; both fds are
// owned by the live PipeWire buffer for this callback, and `fstat`
// only writes the out-param structs, whose fields are read only after
// the `== 0` success checks.
let same_bo = unsafe {
let mut s0: libc::stat = std::mem::zeroed();
let mut s1: libc::stat = std::mem::zeroed();
libc::fstat(datas[0].fd() as i32, &mut s0) == 0
&& libc::fstat(datas[1].fd() as i32, &mut s1) == 0
&& (s0.st_dev, s0.st_ino) == (s1.st_dev, s1.st_ino)
};
if !same_bo {
warn_once(
"NV12 planes live in different buffer objects — frames \
dropped (single-fd import only)",
);
return;
}
let c1 = datas[1].chunk();
Some((c1.offset(), c1.stride().max(0) as u32))
} else {
None
};
// dup the fd so it survives the SPA buffer recycle — the encode thread // dup the fd so it survives the SPA buffer recycle — the encode thread
// imports it. (Content stability across the brief map+CSC window relies on // imports it. Content stability across the brief import/encode window relies
// the compositor's buffer-pool depth, like any zero-copy capture.) // on the compositor's buffer-pool depth, like any zero-copy capture.
// SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid // SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid
// for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd, // for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd,
// touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1). // touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1).
@@ -1642,9 +1715,10 @@ mod pipewire {
modifier: ud.modifier, modifier: ud.modifier,
offset, offset,
stride, stride,
plane1,
}), }),
// Cursor-as-metadata: the encoder blends this into its owned VA // Cursor-as-metadata is blended only by RGB→NV12 backends. Gamescope
// surface (raw dmabuf never touched). // embeds its pointer in the produced pixels, so native NV12 has none.
cursor: ud.cursor.overlay(), cursor: ud.cursor.overlay(),
}); });
static ONCE: std::sync::atomic::AtomicBool = static ONCE: std::sync::atomic::AtomicBool =
@@ -1655,7 +1729,12 @@ mod pipewire {
h, h,
modifier = ud.modifier, modifier = ud.modifier,
fourcc = format_args!("{:#010x}", fourcc), fourcc = format_args!("{:#010x}", fourcc),
"zero-copy: handing the raw dmabuf to the encoder (GPU import + CSC)" source = if fmt == PixelFormat::Nv12 {
"producer-native NV12"
} else {
"packed RGB (encoder GPU CSC)"
},
"zero-copy: handing the raw DMA-BUF to the encoder"
); );
} }
return; return;
@@ -2015,6 +2094,28 @@ mod pipewire {
// PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw // PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw
// dmabuf straight to the encoder. // dmabuf straight to the encoder.
let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough; let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough;
// Producer-side NV12 (default-on; PUNKTFUNK_PIPEWIRE_NV12=0 escapes): gamescope offers a
// one-fd LINEAR NV12 image when the consumer asks — its compositor pass does the RGB→YUV,
// and the Vulkan Video encoder imports the buffer as its encode source directly (no host
// CSC at all). `native_nv12_session` restricts this to sessions whose encoder can ingest
// it (Linux vulkan-encode H265/AV1 — never H264/libav-VAAPI, GameStream-resolve, or
// PyroWave, whose Vulkan compute CSC ingests packed RGB only). Raw passthrough is
// required because the CUDA importer expects packed RGB, and 4:4:4/HDR must not be
// silently subsampled/downconverted. Non-NV12 compositors (KWin/GNOME) simply match the
// packed-RGB fallback pod.
let prefer_native_nv12 = std::env::var("PUNKTFUNK_PIPEWIRE_NV12").as_deref() != Ok("0")
&& policy.native_nv12_session
&& backend_is_vaapi
&& vaapi_passthrough
&& !policy.pyrowave_session
&& !want_444
&& !want_hdr;
if prefer_native_nv12 {
tracing::info!(
"zero-copy: preferring gamescope producer-side NV12 LINEAR DMA-BUF (no host \
RGB CSC; PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation)"
);
}
// Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR // Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR
// (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via // (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via
// CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only: // CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only:
@@ -2054,7 +2155,9 @@ mod pipewire {
tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path"); tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path");
} else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() { } else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() {
tracing::info!( tracing::info!(
"zero-copy: advertising LINEAR dmabuf for direct VAAPI import (GPU CSC)" native_nv12_preferred = prefer_native_nv12,
"zero-copy: advertising LINEAR DMA-BUF for encoder import (native NV12 first \
when enabled, packed RGB fallback)"
); );
} else if want_dmabuf && !vaapi_passthrough { } else if want_dmabuf && !vaapi_passthrough {
tracing::info!( tracing::info!(
@@ -2394,7 +2497,18 @@ mod pipewire {
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?, build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
] ]
} else if want_dmabuf { } else if want_dmabuf {
vec![build_dmabuf_format(&modifiers, preferred)?] let mut pods = Vec::with_capacity(if prefer_native_nv12 { 2 } else { 1 });
if prefer_native_nv12 {
// First compatible consumer pod wins. Gamescope advertises NV12 and BGRx; pinning
// BT.709 limited here selects its RGB→NV12 shader with our bitstream colorimetry.
pods.push(build_dmabuf_format(VideoFormat::NV12, &[0], preferred)?);
}
pods.push(build_dmabuf_format(
VideoFormat::BGRx,
&modifiers,
preferred,
)?);
pods
} else { } else {
vec![serialize_pod(obj)?] vec![serialize_pod(obj)?]
}; };
+209 -12
View File
@@ -308,8 +308,9 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV /// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This /// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
/// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`] /// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`]
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format so the caller can fall /// surfaces a clear error if `CreateRenderTargetView` rejects the plane format. (There is no runtime
/// back to the existing R10 path. /// fallback — the error propagates through `try_consume` and ends the session; the "R10 path" the
/// original design referenced was never kept.)
pub(crate) struct HdrP010Converter { pub(crate) struct HdrP010Converter {
vs: ID3D11VertexShader, vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader, ps_y: ID3D11PixelShader,
@@ -737,14 +738,157 @@ fn p010_reference(r: f64, g: f64, b: f64) -> (f64, f64, f64) {
/// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL. /// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL.
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn hdr_p010_selftest() -> Result<()> { pub fn hdr_p010_selftest() -> Result<()> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_HARDWARE; hdr_p010_selftest_at(64, 64, None)
use windows::Win32::Graphics::Dxgi::IDXGIAdapter; }
// 64x64, even dims. A 4x4 grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → /// [`hdr_p010_selftest`] at an arbitrary even size and (optionally) on a specific GPU vendor
// exact chroma comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus /// (PCI vendor id, e.g. `0x8086` Intel / `0x10de` NVIDIA / `0x1002` AMD). The size matters on
// a couple of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only). /// top of the 64×64 default because the field sessions run at capture resolutions whose height
const W: u32 = 64; /// is NOT 16-aligned (1080 → the encoder's align16 pool seam) and a driver may treat the planar
const H: u32 = 64; /// RTVs differently at real sizes; the vendor pin matters on dual-GPU boxes where the default
/// adapter is not the one the session encodes on.
/// Test support (used by pf-encode's live e2e): the 8 sRGB colour bars (white/yellow/cyan/green/
/// magenta/red/blue/black, sRGB 1.0 = scRGB 1.0 = 80 nits) as a w×h FP16 scRGB texture on the
/// adapter with `luid`, converted through the REAL [`HdrP010Converter`] into a P010 texture with
/// **`BIND_RENDER_TARGET` only, `MiscFlags` 0 — the exact bind profile of the IDD out-ring** (the
/// CPU-upload encoder tests can't use that profile, so only this path exercises "RTV-written P010
/// → encoder ingest copy"). Returns `(device, p010)`; expected decoded codes per bar are the
/// bars_pq2020 fixture's: (490,512,512) (478,423,518) (464,525,473) (450,432,476) (350,584,585)
/// (325,448,598) (226,650,535) (64,512,512).
#[cfg(target_os = "windows")]
#[doc(hidden)]
pub fn hdr_p010_convert_bars_on_luid(
luid: [u8; 8],
w: u32,
h: u32,
) -> Result<(ID3D11Device, ID3D11Texture2D)> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("bars pattern needs even non-zero dimensions, got {w}x{h}");
}
// sRGB primaries at full/zero channels: sRGB EOTF(1.0)=1.0, (0)=0 → the scRGB pattern is
// pure 0/1 floats and the PQ/BT.2020 reference codes above are exact.
const BARS: [(f32, f32, f32); 8] = [
(1.0, 1.0, 1.0),
(1.0, 1.0, 0.0),
(0.0, 1.0, 1.0),
(0.0, 1.0, 0.0),
(1.0, 0.0, 1.0),
(1.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
(0.0, 0.0, 0.0),
];
let bar_w = (w / 8).max(1) as usize;
let mut fp16 = vec![0u16; (w * h * 4) as usize];
for y in 0..h as usize {
for x in 0..w as usize {
let (r, g, b) = BARS[(x / bar_w).min(7)];
let i = (y * w as usize + x) * 4;
fp16[i] = f32_to_f16(r);
fp16[i + 1] = f32_to_f16(g);
fp16[i + 2] = f32_to_f16(b);
fp16[i + 3] = f32_to_f16(1.0);
}
}
// SAFETY: same single-device/single-thread contract as `hdr_p010_selftest_at`; the FP16
// initial-data Vec outlives the synchronous CreateTexture2D; the returned COM handles own
// their references.
unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("dxgi factory")?;
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).context("adapter by luid")?;
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice(luid) for bars convert")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
let src_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R16G16B16A16_FLOAT,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: fp16.as_ptr() as *const c_void,
SysMemPitch: w * 8,
SysMemSlicePitch: 0,
};
let mut src_tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&src_desc, Some(&init), Some(&mut src_tex))
.context("CreateTexture2D(fp16 bars)")?;
let src_tex = src_tex.context("null src tex")?;
let mut src_srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&src_tex, None, Some(&mut src_srv))
.context("CreateShaderResourceView(fp16 bars)")?;
let src_srv = src_srv.context("null src srv")?;
// The IDD out-ring's exact profile: P010, RENDER_TARGET only, MiscFlags 0.
let p010_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
..Default::default()
};
let mut p010: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&p010_desc, None, Some(&mut p010))
.context("CreateTexture2D(P010 bars dst)")?;
let p010 = p010.context("null p010 tex")?;
let conv = HdrP010Converter::new(&device)?;
conv.convert(&device, &context, &src_srv, &p010, w, h)?;
Ok((device, p010))
}
}
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("hdr-p010-selftest needs even non-zero dimensions, got {w}x{h}");
}
// A grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → exact chroma
// comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus a couple
// of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
#[allow(non_snake_case)]
let (W, H) = (w, h);
const BLK: u32 = 16; const BLK: u32 = 16;
// (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values. // (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values.
let named: [(&str, f32, f32, f32); 8] = [ let named: [(&str, f32, f32, f32); 8] = [
@@ -797,12 +941,36 @@ pub fn hdr_p010_selftest() -> Result<()> {
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are // `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below. // proven individually at the `read_u16` closure below.
unsafe { unsafe {
// Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test). // Device on the requested vendor's adapter (dual-GPU boxes encode on a specific one), else
// the default hardware GPU. Always says which adapter ran — a PASS is only meaningful for
// the GPU it actually tested.
let adapter: Option<IDXGIAdapter> = match vendor {
None => None,
Some(want) => {
let factory: IDXGIFactory1 = CreateDXGIFactory1().context("dxgi factory")?;
let mut found = None;
for i in 0.. {
let Ok(a) = factory.EnumAdapters(i) else {
break;
};
let desc = a.GetDesc().context("adapter desc")?;
if desc.VendorId == want {
found = Some(a);
break;
}
}
Some(found.with_context(|| format!("no adapter with vendor id {want:#x}"))?)
}
};
let mut device: Option<ID3D11Device> = None; let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None; let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice( D3D11CreateDevice(
None::<&IDXGIAdapter>, adapter.as_ref(),
D3D_DRIVER_TYPE_HARDWARE, if adapter.is_some() {
D3D_DRIVER_TYPE_UNKNOWN
} else {
D3D_DRIVER_TYPE_HARDWARE
},
HMODULE::default(), HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]), Some(&[D3D_FEATURE_LEVEL_11_0]),
@@ -814,6 +982,22 @@ pub fn hdr_p010_selftest() -> Result<()> {
.context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?; .context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?;
let device = device.context("null device")?; let device = device.context("null device")?;
let context = context.context("null context")?; let context = context.context("null context")?;
{
let dxgi: windows::Win32::Graphics::Dxgi::IDXGIDevice =
device.cast().context("device -> IDXGIDevice")?;
let desc = dxgi.GetAdapter().context("GetAdapter")?.GetDesc()?;
let name = String::from_utf16_lossy(
&desc.Description[..desc
.Description
.iter()
.position(|&c| c == 0)
.unwrap_or(desc.Description.len())],
);
println!(
"adapter: {name} (vendor {:#06x}, luid {:08x}:{:08x})",
desc.VendorId, desc.AdapterLuid.HighPart, desc.AdapterLuid.LowPart
);
}
// Source FP16 texture (initialized) + SRV. // Source FP16 texture (initialized) + SRV.
let src_desc = D3D11_TEXTURE2D_DESC { let src_desc = D3D11_TEXTURE2D_DESC {
@@ -1175,3 +1359,16 @@ impl VideoConverter {
blt.context("VideoProcessorBlt") blt.context("VideoProcessorBlt")
} }
} }
#[cfg(test)]
mod hdr_selftests {
/// LIVE (needs the GPU): [`super::hdr_p010_selftest_at`] at the field capture size — 1080 is
/// NOT 16-aligned, and the planar-RTV write path is driver-specific per vendor. Pinned to the
/// Intel adapter (`0x8086`), so it runs on the Intel validation boxes and errors out cleanly
/// ("no adapter") elsewhere. `cargo test -p pf-capture -- --ignored hdr_p010 --nocapture`.
#[test]
#[ignore]
fn hdr_p010_selftest_intel_1080_live() {
super::hdr_p010_selftest_at(1920, 1080, Some(0x8086)).expect("hdr p010 selftest @1080");
}
}
+12
View File
@@ -44,6 +44,13 @@ pub struct SessionParams {
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The /// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`. /// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool, pub clipboard: bool,
/// Advertise `quic::CLIENT_CAP_CURSOR`: this embedder renders the host cursor locally
/// (the presenter's cursor channel, design/remote-desktop-sweep.md M2), so the host may
/// stop compositing the pointer into the video. Only set when the embedder actually
/// draws it (the SDL presenter in desktop mouse mode) — a session that advertises it
/// without rendering streams with NO visible cursor. The host answers `HOST_CAP_CURSOR`
/// when its capture can forward (Linux portal, not gamescope/Windows).
pub cursor_forward: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see /// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
/// `video::Decoder::new`). /// `video::Decoder::new`).
pub decoder: String, pub decoder: String,
@@ -255,6 +262,11 @@ fn pump(
// This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an // This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an
// A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600). // A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600).
punktfunk_core::client::display_hdr_env_override().or(params.display_hdr), punktfunk_core::client::display_hdr_env_override().or(params.display_hdr),
if params.cursor_forward {
punktfunk_core::quic::CLIENT_CAP_CURSOR
} else {
0
},
params.launch.clone(), params.launch.clone(),
params.pin, params.pin,
Some(params.identity), Some(params.identity),
+57
View File
@@ -456,6 +456,48 @@ impl TouchMode {
} }
} }
/// How a physical mouse drives the host — the desktop-sweep mouse model
/// (design/remote-desktop-sweep.md M1). Stored stringly in [`Settings::mouse_mode`] so the
/// file stays readable; parsed with [`MouseMode::from_name`].
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum MouseMode {
/// Pointer lock (relative deltas, hidden cursor) — the game model, and the default:
/// the only cursor you see is the host's.
Capture,
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion goes on the wire as absolute positions through the letterbox. The remote
/// desktop model. Requires a host injector with absolute support (not gamescope).
Desktop,
}
impl MouseMode {
/// Cycle/picker order (also the settings pickers' option order).
pub const ALL: [MouseMode; 2] = [MouseMode::Capture, MouseMode::Desktop];
/// Parse the persisted name, defaulting to `Capture` for unset/unknown values.
pub fn from_name(s: &str) -> MouseMode {
match s {
"desktop" => MouseMode::Desktop,
_ => MouseMode::Capture,
}
}
/// The persisted name (the inverse of [`from_name`](Self::from_name)).
pub fn as_name(self) -> &'static str {
match self {
MouseMode::Capture => "capture",
MouseMode::Desktop => "desktop",
}
}
pub fn label(self) -> &'static str {
match self {
MouseMode::Capture => "Capture (games)",
MouseMode::Desktop => "Desktop (absolute)",
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file /// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time. /// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)] #[derive(Clone, Serialize, Deserialize)]
@@ -490,6 +532,12 @@ pub struct Settings {
/// stores load as trackpad. /// stores load as trackpad.
#[serde(default = "default_touch_mode")] #[serde(default = "default_touch_mode")]
pub touch_mode: String, pub touch_mode: String,
/// How a physical mouse drives the host: a [`MouseMode`] name — `"capture"` (default,
/// pointer lock + relative) or `"desktop"` (uncaptured absolute pointer). Read at
/// connect via [`Settings::mouse_mode`]. `default` so pre-existing stores load as
/// capture — today's behavior.
#[serde(default = "default_mouse_mode")]
pub mouse_mode: String,
/// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured. /// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured.
pub inhibit_shortcuts: bool, pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source. /// Stream the default microphone to the host's virtual mic source.
@@ -577,6 +625,10 @@ fn default_touch_mode() -> String {
"trackpad".into() "trackpad".into()
} }
fn default_mouse_mode() -> String {
"capture".into()
}
fn default_true() -> bool { fn default_true() -> bool {
true true
} }
@@ -604,6 +656,10 @@ impl Settings {
TouchMode::from_name(&self.touch_mode) TouchMode::from_name(&self.touch_mode)
} }
pub fn mouse_mode(&self) -> MouseMode {
MouseMode::from_name(&self.mouse_mode)
}
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto). /// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 { pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() { match self.codec.as_str() {
@@ -631,6 +687,7 @@ impl Default for Settings {
forward_pad: String::new(), forward_pad: String::new(),
compositor: "auto".into(), compositor: "auto".into(),
touch_mode: "trackpad".into(), touch_mode: "trackpad".into(),
mouse_mode: "capture".into(),
inhibit_shortcuts: true, inhibit_shortcuts: true,
mic_enabled: false, mic_enabled: false,
audio_channels: 2, audio_channels: 2,
+42 -2
View File
@@ -10,7 +10,7 @@ use crate::screens::{Ctx, Outbox};
use crate::theme::{Fonts, DIM, W}; use crate::theme::{Fonts, DIM, W};
use crate::widgets::{ListMsg, MenuList, RowSpec}; use crate::widgets::{ListMsg, MenuList, RowSpec};
use pf_client_core::gamepad::{MenuEvent, MenuPulse}; use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use skia_safe::{Canvas, Rect}; use skia_safe::{Canvas, Rect};
/// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale /// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale
@@ -29,10 +29,11 @@ enum RowId {
Pad, Pad,
PadType, PadType,
Touch, Touch,
Mouse,
Stats, Stats,
} }
const ROWS: [RowId; 13] = [ const ROWS: [RowId; 14] = [
RowId::Resolution, RowId::Resolution,
RowId::Refresh, RowId::Refresh,
RowId::Bitrate, RowId::Bitrate,
@@ -45,6 +46,7 @@ const ROWS: [RowId; 13] = [
RowId::Pad, RowId::Pad,
RowId::PadType, RowId::PadType,
RowId::Touch, RowId::Touch,
RowId::Mouse,
RowId::Stats, RowId::Stats,
]; ];
@@ -251,6 +253,7 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
"Touch mode", "Touch mode",
s.touch_mode().label().into(), s.touch_mode().label().into(),
), ),
RowId::Mouse => (None, "Mouse mode", s.mouse_mode().label().into()),
RowId::Stats => ( RowId::Stats => (
Some("Interface"), Some("Interface"),
"Statistics overlay", "Statistics overlay",
@@ -292,6 +295,11 @@ fn detail(id: RowId) -> &'static str {
"How the touchscreen drives the host: Trackpad (relative cursor), \ "How the touchscreen drives the host: Trackpad (relative cursor), \
Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)." Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)."
} }
RowId::Mouse => {
"How a physical mouse drives the host: Capture locks the pointer (relative, \
for games), Desktop leaves it free and sends absolute positions. \
Ctrl+Alt+Shift+M switches live while streaming."
}
RowId::Stats => { RowId::Stats => {
"How much the overlay shows: Compact (one line) → Normal → Detailed. \ "How much the overlay shows: Compact (one line) → Normal → Detailed. \
Ctrl+Alt+Shift+S cycles it live while streaming." Ctrl+Alt+Shift+S cycles it live while streaming."
@@ -367,6 +375,11 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, TouchMode::ALL.len(), delta, wrap) step_option(cur, TouchMode::ALL.len(), delta, wrap)
.map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string()) .map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string())
} }
RowId::Mouse => {
let cur = MouseMode::ALL.iter().position(|m| *m == s.mouse_mode());
step_option(cur, MouseMode::ALL.len(), delta, wrap)
.map(|i| s.mouse_mode = MouseMode::ALL[i].as_name().to_string())
}
RowId::Stats => { RowId::Stats => {
let cur = StatsVerbosity::ALL let cur = StatsVerbosity::ALL
.iter() .iter()
@@ -510,6 +523,33 @@ mod tests {
assert_eq!(ctx.settings.touch_mode, "trackpad"); assert_eq!(ctx.settings.touch_mode, "trackpad");
} }
#[test]
fn mouse_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
assert_eq!(settings.mouse_mode, "capture");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
// Capture → Desktop, then a step past the end is a boundary.
assert!(
!adjust(RowId::Mouse, -1, false, &mut ctx),
"already first = thud"
);
assert!(adjust(RowId::Mouse, 1, false, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "desktop");
assert!(!adjust(RowId::Mouse, 1, false, &mut ctx), "last = thud");
// A wraps back to the first.
assert!(adjust(RowId::Mouse, 1, true, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "capture");
}
#[test] #[test]
fn unknown_value_snaps_to_first() { fn unknown_value_snaps_to_first() {
let (mut settings, pads) = ctx_parts(); let (mut settings, pads) = ctx_parts();
+5
View File
@@ -25,6 +25,11 @@ tracing = "0.1"
# A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`). # A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`).
tracing-subscriber = { version = "0.3", features = ["env-filter"] } tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[target.'cfg(target_os = "windows")'.dev-dependencies]
# The QSV live e2e drives the REAL HdrP010Converter output (an RTV-written, ring-profile P010
# texture) into the encoder — the one seam the CPU-upload tests can't reach.
pf-capture = { path = "../pf-capture" }
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies] [target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
# Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms. # Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms.
openh264 = "0.9" openh264 = "0.9"
+25 -7
View File
@@ -37,9 +37,11 @@ pub(crate) fn fourcc_to_vk(fourcc: u32) -> Option<vk::Format> {
const AR24: u32 = 0x3432_5241; // ARGB8888 const AR24: u32 = 0x3432_5241; // ARGB8888
const XB24: u32 = 0x3432_4258; // XBGR8888 const XB24: u32 = 0x3432_4258; // XBGR8888
const AB24: u32 = 0x3432_4241; // ABGR8888 const AB24: u32 = 0x3432_4241; // ABGR8888
const NV12: u32 = 0x3231_564e; // DRM_FORMAT_NV12
match fourcc { match fourcc {
XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM), XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM),
XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM), XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM),
NV12 => Some(vk::Format::G8_B8R8_2PLANE_420_UNORM),
_ => None, _ => None,
} }
} }
@@ -90,9 +92,10 @@ pub(crate) unsafe fn import_rgb_dmabuf(
) )
} }
/// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list /// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list.
/// (chained into the image create) — the RGB-direct encode path imports the captured buffer /// Despite the historical name, this also imports gamescope's one-fd LINEAR NV12: the UV
/// as a profiled `VIDEO_ENCODE_SRC` image instead of a sampled one. /// subresource layout comes from the producer's plane-1 chunk when it reported one, falling
/// back to the shared-stride contiguous-plane contract.
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn import_rgb_dmabuf_as( pub(crate) unsafe fn import_rgb_dmabuf_as(
device: &ash::Device, device: &ash::Device,
@@ -108,12 +111,27 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
use std::os::fd::IntoRawFd; use std::os::fd::IntoRawFd;
let fmt = fourcc_to_vk(d.fourcc) let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?; .with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
let plane = [vk::SubresourceLayout::default() let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
.offset(d.offset as u64) let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
.row_pitch(d.stride as u64)]; d.offset as u64 + d.stride as u64 * ch as u64,
d.stride as u64,
));
vec![
vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64),
vk::SubresourceLayout::default()
.offset(uv_offset)
.row_pitch(uv_stride),
]
} else {
vec![vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)]
};
let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default() let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(d.modifier) .drm_format_modifier(d.modifier)
.plane_layouts(&plane); .plane_layouts(&planes);
let mut ext = vk::ExternalMemoryImageCreateInfo::default() let mut ext = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT); .handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
let mut ci = vk::ImageCreateInfo::default() let mut ci = vk::ImageCreateInfo::default()
+498 -137
View File
@@ -14,7 +14,7 @@ use super::vk_util::{color_range, find_mem, make_plain_image, make_view, pixel_t
use crate::{Codec, EncodedFrame, Encoder, EncoderCaps}; use crate::{Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result}; use anyhow::{bail, Context, Result};
use ash::vk; use ash::vk;
use pf_frame::{CapturedFrame, FramePayload}; use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::ffi::c_void; use std::ffi::c_void;
use std::os::fd::AsRawFd; use std::os::fd::AsRawFd;
@@ -84,17 +84,36 @@ fn rgb_request() -> Option<bool> {
} }
} }
/// True-extent RGB-direct at unaligned modes (default ON; `PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0`
/// restores the padded-copy staging): direct-import the visible-size capture with the TRUE-SIZE
/// source `codedExtent` — RADV derives nonzero VCN firmware padding from it, so the EFC is told
/// the source lacks the alignment rows (see [`RgbDirect::true_extent`]). Guarded-tested on Van
/// Gogh 2026-07-21 (kernel clean, and the fastest 1080p encode path measured); the EFC only
/// exists on Mesa ≥ 26, where the `codedExtent`-driven `session_init` is guaranteed.
fn rgb_true_extent_request() -> bool {
std::env::var("PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT").as_deref() != Ok("0")
}
/// Live RGB-direct session config: the chroma-siting bits the session was created with /// Live RGB-direct session config: the chroma-siting bits the session was created with
/// (chosen from what the driver advertises — see [`probe_rgb_direct`]). /// (chosen from what the driver advertises — see [`probe_rgb_direct`]).
struct RgbDirect { struct RgbDirect {
x_offset: u32, // vk_valve_rgb::CHROMA_OFFSET_* x_offset: u32, // vk_valve_rgb::CHROMA_OFFSET_*
y_offset: u32, y_offset: u32,
/// The mode is not 64x16-aligned, so the captured buffer cannot be the encode source /// The mode is not 64x16-aligned, so the captured buffer cannot be the encode source
/// directly (the EFC would read past it — the 2026-07-20 field GPU hang). Instead each /// under the session's ALIGNED source extent (the EFC read past it — the 2026-07-20 field
/// frame is copied into a per-slot ALIGNED BGRA staging image with the edge rows/columns /// GPU hang, when the source `codedExtent` was the aligned size and RADV therefore derived
/// duplicated into the padding (transfer-only, no shader) and encoded from there. Aligned /// ZERO firmware padding). Each frame is copied into a per-slot ALIGNED BGRA staging image
/// modes keep the true zero-copy import. /// with the edge rows/columns duplicated into the padding (transfer-only, no shader) and
/// encoded from there. Aligned modes keep the true zero-copy import.
padded: bool, padded: bool,
/// The default unaligned-mode source strategy (`PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0` falls
/// back to `padded`): direct-import the visible-size buffer and pass the TRUE-SIZE source
/// `codedExtent` — RADV then programs nonzero firmware padding from it (Mesa ≥ 24.2
/// derives `session_init` padding from `srcPictureResource.codedExtent`; see
/// [`VulkanVideoEncoder::native_nv12`]), telling the VCN the source lacks the alignment
/// rows, which the hardware edge-extends internally. The session/SPS/DPB stay app-aligned.
/// Guarded-tested on Van Gogh (kernel clean; fastest 1080p path measured).
true_extent: bool,
} }
/// Stack storage for a complete rgb-chained video profile. Profiled image creation AFTER open /// Stack storage for a complete rgb-chained video profile. Profiled image creation AFTER open
@@ -155,6 +174,51 @@ impl RgbProfileStack {
} }
} }
/// Non-RGB video profile rebuilt for native NV12 DMA-BUF imports. Image creation after `open`
/// must carry a profile identical by value to the session profile.
struct NativeProfileStack {
usage: vk::VideoEncodeUsageInfoKHR<'static>,
h265: vk::VideoEncodeH265ProfileInfoKHR<'static>,
av1: super::vk_av1_encode::VideoEncodeAV1ProfileInfoKHR,
profile: vk::VideoProfileInfoKHR<'static>,
}
impl NativeProfileStack {
fn new(codec_op: vk::VideoCodecOperationFlagsKHR) -> Self {
use super::vk_av1_encode as av1b;
Self {
usage: vk::VideoEncodeUsageInfoKHR::default()
.video_usage_hints(vk::VideoEncodeUsageFlagsKHR::STREAMING)
.video_content_hints(vk::VideoEncodeContentFlagsKHR::RENDERED)
.tuning_mode(vk::VideoEncodeTuningModeKHR::ULTRA_LOW_LATENCY),
h265: vk::VideoEncodeH265ProfileInfoKHR::default().std_profile_idc(
vk::native::StdVideoH265ProfileIdc_STD_VIDEO_H265_PROFILE_IDC_MAIN,
),
av1: av1b::VideoEncodeAV1ProfileInfoKHR {
s_type: av1b::stype(av1b::ST_PROFILE_INFO),
p_next: std::ptr::null(),
std_profile: vk::native::StdVideoAV1Profile_STD_VIDEO_AV1_PROFILE_MAIN,
},
profile: vk::VideoProfileInfoKHR::default()
.video_codec_operation(codec_op)
.chroma_subsampling(vk::VideoChromaSubsamplingFlagsKHR::TYPE_420)
.luma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8)
.chroma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8),
}
}
fn wire(&mut self, av1: bool) -> &vk::VideoProfileInfoKHR<'static> {
if av1 {
self.av1.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.av1 as *const _ as *const c_void;
} else {
self.h265.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.h265 as *const _ as *const c_void;
}
&self.profile
}
}
/// The Vulkan codec-operation bit for our codec selection (shared by open and the per-import /// The Vulkan codec-operation bit for our codec selection (shared by open and the per-import
/// profile rebuilds — the two must agree, profile identity is by value). /// profile rebuilds — the two must agree, profile identity is by value).
fn codec_op_for(av1: bool) -> vk::VideoCodecOperationFlagsKHR { fn codec_op_for(av1: bool) -> vk::VideoCodecOperationFlagsKHR {
@@ -173,12 +237,12 @@ enum SrcAcquire {
/// CSC path: `nv12_src` was written by this frame's compute batch (GENERAL layout; the /// CSC path: `nv12_src` was written by this frame's compute batch (GENERAL layout; the
/// csc_sem orders the queues). /// csc_sem orders the queues).
CscGeneral, CscGeneral,
/// RGB-direct, first use of a dmabuf import: acquire from the foreign producer /// First use of a DMA-BUF imported directly as the video source: acquire from the foreign
/// (UNDEFINED preserves the modifier-tiled bytes) with a FOREIGN→encode-family transfer. /// producer (UNDEFINED preserves modifier-backed bytes) with a FOREIGN→encode-family transfer.
RgbFresh, DmabufFresh,
/// RGB-direct, cached import: the image is already VIDEO_ENCODE_SRC; visibility-only /// Cached direct-source import: already VIDEO_ENCODE_SRC; visibility-only barrier for the
/// barrier for the producer's out-of-band rewrite of the bytes. /// producer's out-of-band rewrite of the bytes.
RgbCached, DmabufCached,
/// RGB-direct CPU upload: the compute queue copied the staging buffer in (semaphore /// RGB-direct CPU upload: the compute queue copied the staging buffer in (semaphore
/// ordered); transition TRANSFER_DST → VIDEO_ENCODE_SRC. /// ordered); transition TRANSFER_DST → VIDEO_ENCODE_SRC.
CpuUpload, CpuUpload,
@@ -376,18 +440,33 @@ pub struct VulkanVideoEncoder {
/// `ENCODE_QUALITY_LEVEL` control and baked into the session-parameters object (the spec /// `ENCODE_QUALITY_LEVEL` control and baked into the session-parameters object (the spec
/// requires the two to match). /// requires the two to match).
quality_level: u32, quality_level: u32,
/// `PUNKTFUNK_PERF` CSC/encode split: >0 ⇒ per-frame GPU timestamps are recorded around the /// `PUNKTFUNK_PERF` pre-encode split: >0 ⇒ per-frame GPU timestamps bracket either the
/// compute batch and a sampled `csc_us` line is logged; the value is the device timestamp /// RGB→NV12 compute batch or the native-NV12 padded copy. The measured duration is logged
/// period in ns/tick. 0.0 ⇒ off (env unset, or the compute family has no timestamp support). /// separately from the host's fence wait; 0.0 means disabled or unsupported.
ts_period_ns: f64, ts_period_ns: f64,
perf_at: std::time::Instant, // last sampled csc_us log (2 s cadence) perf_at: std::time::Instant,
/// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames /// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames
/// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does /// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does
/// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked /// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked
/// into the video session). /// into the video session).
rgb: Option<RgbDirect>, rgb: Option<RgbDirect>,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct session (EFC cannot /// Producer supplied native NV12 rather than packed RGB. EVERY mode encodes the imported
/// composite it — the cursor will be missing from the stream until the CSC path is used). /// visible-size buffer directly — safely, because native sessions use TRUE-SIZE headers:
/// the SPS/sequence header is authored at the render size and every picture resource's
/// `codedExtent` matches it, so RADV programs the VCN with `session_init` extent = the true
/// size and a nonzero `padding_width/height`, and the FIRMWARE edge-extends the alignment
/// padding internally (radv_video_enc.c `radv_enc_session_init`; the driver also rounds the
/// bitstream SPS up itself and compensates with a conformance window —
/// `radv_video_patch_encode_session_parameters`, per the VK_KHR_video_encode_h265 proposal's
/// "implementations may override" clause). The source is never read past its extent — unlike
/// the app-aligned-SPS convention the CSC/RGB paths use, where the coded extent is 64x16-
/// aligned and an undersized direct source is the OOB-read class behind the 2026-07-20 field
/// GPU reset (those paths keep their aligned-size sources/staging).
native_nv12: bool,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct or native-NV12 session
/// (neither has a compositing stage — the cursor will be missing from the stream until the
/// CSC path is used).
warned_cursor: bool, warned_cursor: bool,
/// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video /// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video
/// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it /// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it
@@ -422,14 +501,24 @@ impl VulkanVideoEncoder {
/// (B2). `PUNKTFUNK_VULKAN_RGB_DIRECT` overrides both ways (see [`rgb_request`]). /// (B2). `PUNKTFUNK_VULKAN_RGB_DIRECT` overrides both ways (see [`rgb_request`]).
pub fn open( pub fn open(
codec: Codec, codec: Codec,
format: PixelFormat,
width: u32, width: u32,
height: u32, height: u32,
fps: u32, fps: u32,
bitrate_bps: u64, bitrate_bps: u64,
cursor_blend: bool, cursor_blend: bool,
) -> Result<Self> { ) -> Result<Self> {
let want_rgb = rgb_request().unwrap_or(!cursor_blend); let native_nv12 = format == PixelFormat::Nv12;
Self::open_opts(codec, width, height, fps, bitrate_bps, want_rgb) let want_rgb = !native_nv12 && rgb_request().unwrap_or(!cursor_blend);
Self::open_opts_inner(
codec,
width,
height,
fps,
bitrate_bps,
want_rgb,
native_nv12,
)
} }
/// `open` with the RGB-direct request explicit instead of read from the env — the smoke /// `open` with the RGB-direct request explicit instead of read from the env — the smoke
@@ -443,6 +532,18 @@ impl VulkanVideoEncoder {
fps: u32, fps: u32,
bitrate_bps: u64, bitrate_bps: u64,
want_rgb: bool, want_rgb: bool,
) -> Result<Self> {
Self::open_opts_inner(codec, width, height, fps, bitrate_bps, want_rgb, false)
}
fn open_opts_inner(
codec: Codec,
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
want_rgb: bool,
native_nv12: bool,
) -> Result<Self> { ) -> Result<Self> {
if !matches!(codec, Codec::H265 | Codec::Av1) { if !matches!(codec, Codec::H265 | Codec::Av1) {
bail!("vulkan-encode backend supports HEVC + AV1 only (got {codec:?})"); bail!("vulkan-encode backend supports HEVC + AV1 only (got {codec:?})");
@@ -464,6 +565,7 @@ impl VulkanVideoEncoder {
fps.max(1), fps.max(1),
bitrate_bps.max(1_000_000), bitrate_bps.max(1_000_000),
want_rgb, want_rgb,
native_nv12,
) )
} }
} }
@@ -478,6 +580,7 @@ impl VulkanVideoEncoder {
fps: u32, fps: u32,
bitrate: u64, bitrate: u64,
want_rgb: bool, want_rgb: bool,
native_nv12: bool,
) -> Result<Self> { ) -> Result<Self> {
use super::vk_av1_encode as av1b; use super::vk_av1_encode as av1b;
use super::vk_valve_rgb as vrgb; use super::vk_valve_rgb as vrgb;
@@ -551,46 +654,63 @@ impl VulkanVideoEncoder {
0.0 0.0
}; };
// RGB-direct (EFC) resolution — BEFORE the profile is built, because an active rgb // Resolve the encode source before building the profile: EFC RGB conversion changes
// session changes the profile identity (the rgb-conversion struct rides the chain) and // profile identity; producer-native NV12 uses the ordinary 4:2:0 profile.
// the session's picture format. The probe runs unconditionally: its verdict is the
// field telemetry that decides where B2 can default this on.
let rgb_probe = probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1);
// ALIGNMENT GATE (field GPU-hang, 2026-07-20): the coded extent is 64x16-aligned but the
// captured dmabuf is only the REAL mode size — handing it to the encoder as the direct
// source makes the VCN's EFC read the alignment-padding rows PAST the end of the buffer.
// At 1920x1080 (coded 1088) that is 8 rows = 61 KB of out-of-bounds reads per frame:
// deterministic VM protection faults, vcn_enc ring timeouts, and — through the stall
// watchdog's rebuild-and-refault loop — a full MODE1 GPU reset with VRAM loss. The CSC
// shader absorbs the padding by clamping reads and duplicating the edge; RGB-direct has
// no such stage. Mode select: an aligned mode (720p/1440p/4K) encodes the imported
// buffer directly (true zero-copy); an unaligned one (1080p!) goes through the
// padded-copy staging image (see [`RgbDirect::padded`]) — transfer-only, still no
// compute CSC.
let aligned = rw == w && rh == h; let aligned = rw == w && rh == h;
let rgb_probe = if native_nv12 {
Err("not-probed(native NV12 source selected)")
} else {
probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1)
};
let rgb_cfg: Option<RgbDirect> = match (&rgb_probe, want_rgb) { let rgb_cfg: Option<RgbDirect> = match (&rgb_probe, want_rgb) {
(Ok((x, y)), true) => Some(RgbDirect { (Ok((x, y)), true) => {
x_offset: *x, let true_extent = !aligned && rgb_true_extent_request();
y_offset: *y, Some(RgbDirect {
padded: !aligned, x_offset: *x,
}), y_offset: *y,
padded: !aligned && !true_extent,
true_extent,
})
}
_ => None, _ => None,
}; };
tracing::info!( if native_nv12 {
rgb_direct = match (&rgb_probe, want_rgb, &rgb_cfg) { tracing::info!(
(_, _, Some(RgbDirect { padded: false, .. })) => "active", native_nv12 = "active(direct-import)",
(_, _, Some(RgbDirect { padded: true, .. })) => source_width = rw,
"active(padded-copy: mode is not 64x16-aligned — staging blit + edge \ source_height = rh,
duplication instead of the direct import)", fw_padding_width = w - rw,
(Ok(_), false, None) => fw_padding_height = h - rh,
"available(off: PUNKTFUNK_VULKAN_RGB_DIRECT=0, or a cursor-blend session \ "vulkan-encode: producer-native NV12 encode source (true-size headers: the \
=1 forces)", driver aligns the bitstream SPS itself and the firmware edge-extends the \
(Err(e), _, None) => e, padding the source is never read past its extent)"
// (Ok, wanted) always builds Some above. );
(Ok(_), true, None) => unreachable!("rgb gate and cfg disagree"), } else {
}, tracing::info!(
"vulkan-encode: EFC RGB-direct encode source (design/vulkan-rgb-direct-encode.md)" rgb_direct = match (&rgb_probe, want_rgb, &rgb_cfg) {
); (
_,
_,
Some(RgbDirect {
true_extent: true, ..
}),
) =>
"active(true-extent: unaligned mode, direct import with the true-size \
source codedExtent RADV firmware padding covers the alignment rows; \
PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded copy)",
(_, _, Some(RgbDirect { padded: false, .. })) => "active",
(_, _, Some(RgbDirect { padded: true, .. })) =>
"active(padded-copy: mode is not 64x16-aligned — staging blit + edge \
duplication instead of the direct import)",
(Ok(_), false, None) =>
"available(off: PUNKTFUNK_VULKAN_RGB_DIRECT=0, or a cursor-blend session \
=1 forces)",
(Err(e), _, None) => e,
(Ok(_), true, None) => unreachable!("rgb gate and cfg disagree"),
},
"vulkan-encode: EFC RGB-direct encode source (design/vulkan-rgb-direct-encode.md)"
);
}
// the encode profile — H265 Main, or AV1 Main; chained raw and uniformly (vendored AV1 + // the encode profile — H265 Main, or AV1 Main; chained raw and uniformly (vendored AV1 +
// rgb structs can't `push_next`, and the chain must match [`RgbProfileStack::wire`]'s // rgb structs can't `push_next`, and the chain must match [`RgbProfileStack::wire`]'s
@@ -820,13 +940,19 @@ impl VulkanVideoEncoder {
// ---- session parameters + header framing (HEVC: VPS/SPS/PPS on keyframes; AV1: a // ---- session parameters + header framing (HEVC: VPS/SPS/PPS on keyframes; AV1: a
// temporal-delimiter OBU per frame + a sequence-header OBU on keyframes) ---- // temporal-delimiter OBU per frame + a sequence-header OBU on keyframes) ----
// Native NV12 authors TRUE-SIZE headers (SPS/seq at the render size, no app-side
// conformance window): RADV rounds the bitstream SPS up itself and, keyed off the
// matching true-size codedExtent, programs the VCN with firmware padding so the source
// is never read past its extent. The CSC/RGB paths keep the app-aligned convention
// (their sources genuinely cover the aligned extent).
let (hdr_w, hdr_h) = if native_nv12 { (rw, rh) } else { (w, h) };
let (params, header, frame_prefix) = if av1 { let (params, header, frame_prefix) = if av1 {
build_parameters_av1( build_parameters_av1(
&device, &device,
&vq_dev, &vq_dev,
session, session,
w, hdr_w,
h, hdr_h,
rw, rw,
rh, rh,
av1_caps.max_level, av1_caps.max_level,
@@ -839,8 +965,8 @@ impl VulkanVideoEncoder {
&vq_dev, &vq_dev,
&venc_dev, &venc_dev,
session, session,
w, hdr_w,
h, hdr_h,
rw, rw,
rh, rh,
quality_level, quality_level,
@@ -996,9 +1122,14 @@ impl VulkanVideoEncoder {
compute_pool, compute_pool,
bs_size, bs_size,
sampler, sampler,
ts_period_ns > 0.0 && rgb_cfg.is_none(), ts_period_ns > 0.0
rgb_cfg.is_none(), && ((rgb_cfg.is_none() && !native_nv12)
rgb_cfg.as_ref().is_some_and(|c| c.padded), || rgb_cfg.as_ref().is_some_and(|c| c.padded)),
rgb_cfg.is_none() && !native_nv12,
rgb_cfg
.as_ref()
.is_some_and(|c| c.padded)
.then_some(vk::Format::B8G8R8A8_UNORM),
guard.frames.last_mut().expect("frame just pushed"), guard.frames.last_mut().expect("frame just pushed"),
)?; )?;
} }
@@ -1052,6 +1183,7 @@ impl VulkanVideoEncoder {
ts_period_ns, ts_period_ns,
perf_at: std::time::Instant::now(), perf_at: std::time::Instant::now(),
rgb: rgb_cfg, rgb: rgb_cfg,
native_nv12,
warned_cursor: false, warned_cursor: false,
pending_bitrate: None, pending_bitrate: None,
width: w, width: w,
@@ -1200,15 +1332,31 @@ impl VulkanVideoEncoder {
} }
} }
/// Import a packed-RGB dmabuf as a VkImage (explicit DRM modifier). CSC sessions import it /// Import a DMA-BUF VkImage with usage/profile matching this session's source mode. Native
/// SAMPLED (the compute shader reads it); RGB-direct sessions import it as a profiled /// NV12 and aligned RGB-direct are profiled `VIDEO_ENCODE_SRC` images. Padded RGB-direct
/// `VIDEO_ENCODE_SRC` — the buffer IS the encode source. Caller destroys. /// imports the producer allocation as transfer-source only.
unsafe fn import_dmabuf( unsafe fn import_dmabuf(
&self, &self,
d: &pf_frame::DmabufFrame, d: &pf_frame::DmabufFrame,
cw: u32, cw: u32,
ch: u32, ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> { ) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
if self.native_nv12 {
let mut ps = NativeProfileStack::new(codec_op_for(self.codec == Codec::Av1));
let profile = *ps.wire(self.codec == Codec::Av1);
let arr = [profile];
let mut plist = vk::VideoProfileListInfoKHR::default().profiles(&arr);
return super::vk_util::import_rgb_dmabuf_as(
&self.device,
&self.ext_fd,
&self.mem_props,
d,
cw,
ch,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR,
Some(&mut plist),
);
}
if self.rgb.as_ref().is_some_and(|r| r.padded) { if self.rgb.as_ref().is_some_and(|r| r.padded) {
// Padded-copy mode: the import is only ever a transfer SOURCE (the blit into the // Padded-copy mode: the import is only ever a transfer SOURCE (the blit into the
// aligned staging image) — plain TRANSFER_SRC, no video profile involved. // aligned staging image) — plain TRANSFER_SRC, no video profile involved.
@@ -1503,8 +1651,21 @@ impl VulkanVideoEncoder {
setup_idx = (setup_idx + 1) % DPB_SLOTS as usize; setup_idx = (setup_idx + 1) % DPB_SLOTS as usize;
} }
// ---- 2..4 diverge by encode source; the RGB-direct twin returns through the shared // ---- 2..4 diverge by encode source; native NV12 and RGB-direct return through the
// bookkeeping tail (design/vulkan-rgb-direct-encode.md B1) ---- // shared bookkeeping tail ----
if self.native_nv12 {
self.record_submit_nv12(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping(
slot,
frame.pts_ns,
wire,
is_idr,
recovery,
setup_idx,
poc,
);
return Ok(());
}
if self.rgb.is_some() { if self.rgb.is_some() {
self.record_submit_rgb(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?; self.record_submit_rgb(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping( self.post_submit_bookkeeping(
@@ -1831,12 +1992,20 @@ impl VulkanVideoEncoder {
Ok(()) Ok(())
} }
/// Padded-copy blit (unaligned-mode RGB-direct): record — into `compute_cmd` — the visible /// Padded-copy blit (unaligned-mode RGB-direct or native NV12): record — into `compute_cmd`
/// frame copy from the imported capture image into the aligned staging image, plus the edge /// — the visible frame copy from the imported capture image into the aligned staging image,
/// duplication into the 64x16 padding (the same edge semantics the CSC shader implements /// plus the edge duplication into the 64x16 padding (the same edge semantics the CSC shader
/// with clamped reads). Transfer-only, no shader. The staging image lives in GENERAL — it /// implements with clamped reads). Transfer-only, no shader. The staging image lives in
/// is both copy dst and, for the right-column pass, copy src — and the encode acquires it /// GENERAL — it is both copy dst and, for the right-column pass, copy src — and the encode
/// via [`SrcAcquire::CscGeneral`] (content produced on the compute queue, csc_sem-ordered). /// acquires it via [`SrcAcquire::CscGeneral`] (content produced on the compute queue,
/// csc_sem-ordered).
///
/// `planes` lists the copy aspects with their subsampling divisor — `[(COLOR, 1)]` for
/// packed RGB, `[(PLANE_0, 1), (PLANE_1, 2)]` for NV12 (multi-planar copy regions are in
/// each plane's own coordinate space; barriers on non-disjoint images stay COLOR-aspect).
/// Every divisor must divide the visible and aligned extents (4:2:0 frames are even, the
/// coded extent is 64x16-aligned).
#[allow(clippy::too_many_arguments)]
unsafe fn record_pad_blit( unsafe fn record_pad_blit(
&self, &self,
dev: &ash::Device, dev: &ash::Device,
@@ -1844,6 +2013,8 @@ impl VulkanVideoEncoder {
src: vk::Image, src: vk::Image,
src_fresh: bool, src_fresh: bool,
pad: vk::Image, pad: vk::Image,
ts_pool: vk::QueryPool,
planes: &[(vk::ImageAspectFlags, u32)],
) -> Result<()> { ) -> Result<()> {
let (rw, rh) = (self.render_w, self.render_h); let (rw, rh) = (self.render_w, self.render_h);
let (w, h) = (self.width, self.height); let (w, h) = (self.width, self.height);
@@ -1852,6 +2023,10 @@ impl VulkanVideoEncoder {
&vk::CommandBufferBeginInfo::default() &vk::CommandBufferBeginInfo::default()
.flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT), .flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
)?; )?;
if self.ts_period_ns > 0.0 {
dev.cmd_reset_query_pool(compute_cmd, ts_pool, 0, 2);
dev.cmd_write_timestamp2(compute_cmd, vk::PipelineStageFlags2::NONE, ts_pool, 0);
}
// Acquire the imported capture buffer for transfer reads (FOREIGN hand-off on first // Acquire the imported capture buffer for transfer reads (FOREIGN hand-off on first
// import — UNDEFINED preserves the modifier-tiled bytes — visibility-only afterwards), // import — UNDEFINED preserves the modifier-tiled bytes — visibility-only afterwards),
// and the staging image for transfer writes (prior contents discarded). // and the staging image for transfer writes (prior contents discarded).
@@ -1893,26 +2068,32 @@ impl VulkanVideoEncoder {
compute_cmd, compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[src_acq, pad_dst]), &vk::DependencyInfo::default().image_memory_barriers(&[src_acq, pad_dst]),
); );
let layers = vk::ImageSubresourceLayers::default() let region =
.aspect_mask(vk::ImageAspectFlags::COLOR) |aspect: vk::ImageAspectFlags, sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| {
.layer_count(1); let layers = vk::ImageSubresourceLayers::default()
let region = |sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| { .aspect_mask(aspect)
vk::ImageCopy::default() .layer_count(1);
.src_subresource(layers) vk::ImageCopy::default()
.dst_subresource(layers) .src_subresource(layers)
.src_offset(vk::Offset3D { x: sx, y: sy, z: 0 }) .dst_subresource(layers)
.dst_offset(vk::Offset3D { x: dx, y: dy, z: 0 }) .src_offset(vk::Offset3D { x: sx, y: sy, z: 0 })
.extent(vk::Extent3D { .dst_offset(vk::Offset3D { x: dx, y: dy, z: 0 })
width: cw, .extent(vk::Extent3D {
height: ch, width: cw,
depth: 1, height: ch,
}) depth: 1,
}; })
// Pass 1 — from the capture: the visible region, then each bottom padding row as a };
// copy of the last visible row (1080p: 8 such rows). One call, disjoint regions. // Pass 1 — from the capture, per plane: the visible region, then each bottom padding row
let mut regions = vec![region(0, 0, 0, 0, rw, rh)]; // as a copy of the last visible row (1080p: 8 luma + 4 chroma rows). One call, disjoint
for y in rh..h { // regions.
regions.push(region(0, rh as i32 - 1, 0, y as i32, rw, 1)); let mut regions = Vec::new();
for &(aspect, div) in planes {
let (rw, rh, h) = (rw / div, rh / div, h / div);
regions.push(region(aspect, 0, 0, 0, 0, rw, rh));
for y in rh..h {
regions.push(region(aspect, 0, rh as i32 - 1, 0, y as i32, rw, 1));
}
} }
dev.cmd_copy_image( dev.cmd_copy_image(
compute_cmd, compute_cmd,
@@ -1942,9 +2123,13 @@ impl VulkanVideoEncoder {
compute_cmd, compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[self_dep]), &vk::DependencyInfo::default().image_memory_barriers(&[self_dep]),
); );
let cols: Vec<vk::ImageCopy> = (rw..w) let mut cols = Vec::new();
.map(|x| region(rw as i32 - 1, 0, x as i32, 0, 1, h)) for &(aspect, div) in planes {
.collect(); let (rw, w, h) = (rw / div, w / div, h / div);
for x in rw..w {
cols.push(region(aspect, rw as i32 - 1, 0, x as i32, 0, 1, h));
}
}
dev.cmd_copy_image( dev.cmd_copy_image(
compute_cmd, compute_cmd,
pad, pad,
@@ -1954,10 +2139,112 @@ impl VulkanVideoEncoder {
&cols, &cols,
); );
} }
if self.ts_period_ns > 0.0 {
dev.cmd_write_timestamp2(
compute_cmd,
vk::PipelineStageFlags2::ALL_COMMANDS,
ts_pool,
1,
);
}
dev.end_command_buffer(compute_cmd)?; dev.end_command_buffer(compute_cmd)?;
Ok(()) Ok(())
} }
/// Producer-native NV12 submit: import the producer's visible-size buffer directly as the
/// encode source. Safe at every mode because native sessions run true-size headers — the
/// picture resources' codedExtent equals the source extent and the VCN firmware edge-extends
/// the alignment padding internally (see the [`Self::native_nv12`] field docs).
#[allow(clippy::too_many_arguments)]
unsafe fn record_submit_nv12(
&mut self,
slot: usize,
frame: &CapturedFrame,
is_idr: bool,
recovery: bool,
ref_slot: usize,
setup_idx: usize,
poc: i32,
) -> Result<()> {
if frame.format != PixelFormat::Nv12 {
bail!(
"vulkan-encode (native NV12): negotiated NV12 but received {:?}",
frame.format
);
}
if frame.width != self.render_w || frame.height != self.render_h {
bail!(
"vulkan-encode (native NV12): frame {}x{} != mode {}x{}",
frame.width,
frame.height,
self.render_w,
self.render_h
);
}
if frame.width % 2 != 0 || frame.height % 2 != 0 {
bail!("vulkan-encode (native NV12): 4:2:0 frame dimensions must be even");
}
let FramePayload::Dmabuf(d) = &frame.payload else {
bail!("vulkan-encode (native NV12): producer frame is not a DMA-BUF");
};
if d.fourcc != pf_frame::drm_fourcc(PixelFormat::Nv12).expect("NV12 FourCC") {
bail!(
"vulkan-encode (native NV12): DMA-BUF FourCC {:#x} is not NV12",
d.fourcc
);
}
if d.modifier != 0 {
bail!(
"vulkan-encode (native NV12): only LINEAR is supported, got modifier {:#x}",
d.modifier
);
}
// No compositing stage exists here (like RGB-direct/EFC): gamescope embeds its pointer
// in the produced pixels, but any other NV12 producer's metadata cursor would be lost —
// say so once instead of silently.
if frame.cursor.is_some() && !self.warned_cursor {
self.warned_cursor = true;
tracing::warn!(
"cursor bitmap on a native-NV12 session — nothing composites it; the cursor \
will be missing from the stream (unset PUNKTFUNK_PIPEWIRE_NV12 for \
metadata-cursor captures)"
);
}
let dev = self.device.clone();
let cmd = self.frames[slot].cmd;
let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf;
// The frame-size check above proved the buffer covers the (true-size) coded extent —
// the direct import is the encode source at every mode.
let (src_img, src_view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acquire = if fresh {
SrcAcquire::DmabufFresh
} else {
SrcAcquire::DmabufCached
};
if self.codec == Codec::Av1 {
self.record_coding_av1(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, recovery,
ref_slot, setup_idx, poc,
)?;
} else {
self.record_coding_h265(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, ref_slot,
setup_idx, poc,
)?;
}
dev.reset_fences(&[fence])?;
// The whole frame is one submit: the encoder reads the imported NV12 directly.
let ecmds = [cmd];
dev.queue_submit(
self.encode_queue,
&[vk::SubmitInfo::default().command_buffers(&ecmds)],
fence,
)?;
Ok(())
}
/// RGB-direct twin of [`record_submit`]'s steps 24 (step 1 and the bookkeeping tail are /// RGB-direct twin of [`record_submit`]'s steps 24 (step 1 and the bookkeeping tail are
/// shared): resolve the RGB encode source — the imported capture dmabuf itself, or the CPU /// shared): resolve the RGB encode source — the imported capture dmabuf itself, or the CPU
/// staging upload — record the encode, and submit. There is no compute CSC: the VCN EFC /// staging upload — record the encode, and submit. There is no compute CSC: the VCN EFC
@@ -1981,6 +2268,7 @@ impl VulkanVideoEncoder {
let fence = self.frames[slot].fence; let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool; let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf; let bs_buf = self.frames[slot].bs_buf;
let ts_pool = self.frames[slot].ts_pool;
// EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so // EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so
// once instead of silently losing the pointer (gamescope, the flagship, embeds it). // once instead of silently losing the pointer (gamescope, the flagship, embeds it).
if frame.cursor.is_some() && !self.warned_cursor { if frame.cursor.is_some() && !self.warned_cursor {
@@ -1995,25 +2283,33 @@ impl VulkanVideoEncoder {
let (src_img, src_view, acquire, compute_active) = match &frame.payload { let (src_img, src_view, acquire, compute_active) = match &frame.payload {
FramePayload::Dmabuf(d) if !padded => { FramePayload::Dmabuf(d) if !padded => {
// Defense in depth for the OOB class the alignment gate closes at open: the // Defense in depth for the OOB class the alignment gate closes at open: the
// imported buffer must cover the FULL coded extent, or the EFC reads past it // imported buffer must cover the source extent the encode declares — the FULL
// (VM faults → VCN ring hang → GPU reset, the 2026-07-20 field report). A // aligned coded extent normally (or the EFC reads past it: VM faults → VCN
// mismatched frame (mid-flight mode change, odd capture) errors out here and // ring hang → GPU reset, the 2026-07-20 field report), the render size in
// takes the encoder-rebuild path instead of faulting the GPU. // true-extent mode (where the declared source codedExtent shrinks with it and
if frame.width != self.width || frame.height != self.height { // RADV's firmware padding covers the alignment rows). A mismatched frame
// (mid-flight mode change, odd capture) errors out here and takes the
// encoder-rebuild path instead of faulting the GPU.
let (need_w, need_h) = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
(self.render_w, self.render_h)
} else {
(self.width, self.height)
};
if frame.width != need_w || frame.height != need_h {
bail!( bail!(
"vulkan-encode (rgb-direct): frame {}x{} does not cover the coded \ "vulkan-encode (rgb-direct): frame {}x{} does not cover the declared \
extent {}x{} refusing an out-of-bounds encode source", source extent {}x{} refusing an out-of-bounds encode source",
frame.width, frame.width,
frame.height, frame.height,
self.width, need_w,
self.height need_h
); );
} }
let (img, view, fresh) = self.import_cached(d, frame.width, frame.height)?; let (img, view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acq = if fresh { let acq = if fresh {
SrcAcquire::RgbFresh SrcAcquire::DmabufFresh
} else { } else {
SrcAcquire::RgbCached SrcAcquire::DmabufCached
}; };
(img, view, acq, false) (img, view, acq, false)
} }
@@ -2036,7 +2332,15 @@ impl VulkanVideoEncoder {
let (img, _view, fresh) = self.import_cached(d, frame.width, frame.height)?; let (img, _view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let pad_img = self.frames[slot].pad_img; let pad_img = self.frames[slot].pad_img;
let pad_view = self.frames[slot].pad_view; let pad_view = self.frames[slot].pad_view;
self.record_pad_blit(&dev, compute_cmd, img, fresh, pad_img)?; self.record_pad_blit(
&dev,
compute_cmd,
img,
fresh,
pad_img,
ts_pool,
&[(vk::ImageAspectFlags::COLOR, 1)],
)?;
// The staging image ends the blit in GENERAL with the csc_sem ordering the // The staging image ends the blit in GENERAL with the csc_sem ordering the
// hand-off — exactly the CscGeneral acquire contract. // hand-off — exactly the CscGeneral acquire contract.
(pad_img, pad_view, SrcAcquire::CscGeneral, true) (pad_img, pad_view, SrcAcquire::CscGeneral, true)
@@ -2240,13 +2544,13 @@ impl VulkanVideoEncoder {
.src_stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS) .src_stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::GENERAL), .old_layout(vk::ImageLayout::GENERAL),
SrcAcquire::RgbFresh => src_base SrcAcquire::DmabufFresh => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE) .src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::UNDEFINED) .old_layout(vk::ImageLayout::UNDEFINED)
.src_queue_family_index(vk::QUEUE_FAMILY_FOREIGN_EXT) .src_queue_family_index(vk::QUEUE_FAMILY_FOREIGN_EXT)
.dst_queue_family_index(self.encode_family), .dst_queue_family_index(self.encode_family),
SrcAcquire::RgbCached => src_base SrcAcquire::DmabufCached => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE) .src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::VIDEO_ENCODE_SRC_KHR), .old_layout(vk::ImageLayout::VIDEO_ENCODE_SRC_KHR),
@@ -2284,9 +2588,29 @@ impl VulkanVideoEncoder {
poc: i32, poc: i32,
) -> Result<()> { ) -> Result<()> {
use ash::vk::native as h; use ash::vk::native as h;
let ext2d = vk::Extent2D { // Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
width: self.width, // pairs with their aligned SPS), the render size for native NV12's true-size headers.
height: self.height, let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width,
height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
}; };
let ref_poc = if is_idr { 0 } else { self.slot_poc[ref_slot] }; let ref_poc = if is_idr { 0 } else { self.slot_poc[ref_slot] };
@@ -2458,7 +2782,7 @@ impl VulkanVideoEncoder {
} }
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty()); dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default() let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d) .coded_extent(src_extent)
.image_view_binding(src_view); .image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default() let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf) .dst_buffer(bs_buf)
@@ -2502,9 +2826,29 @@ impl VulkanVideoEncoder {
) -> Result<()> { ) -> Result<()> {
use super::vk_av1_encode as av1; use super::vk_av1_encode as av1;
use ash::vk::native as h; use ash::vk::native as h;
let ext2d = vk::Extent2D { // Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
width: self.width, // pairs with their aligned SPS), the render size for native NV12's true-size headers.
height: self.height, let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width,
height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
}; };
// ---- required AV1 frame sub-structs (single tile; no CDEF/LR/segmentation/global-motion) ---- // ---- required AV1 frame sub-structs (single tile; no CDEF/LR/segmentation/global-motion) ----
@@ -2726,7 +3070,7 @@ impl VulkanVideoEncoder {
} }
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty()); dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default() let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d) .coded_extent(src_extent)
.image_view_binding(src_view); .image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default() let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf) .dst_buffer(bs_buf)
@@ -2784,10 +3128,13 @@ impl VulkanVideoEncoder {
); );
} }
let (off, len) = (off64 as usize, len64 as usize); let (off, len) = (off64 as usize, len64 as usize);
// PUNKTFUNK_PERF CSC split (best-effort): the fence signaled, so the compute batch that // PUNKTFUNK_PERF pre-encode split (best-effort): the fence signaled, so the compute/transfer
// wrote these timestamps completed long ago — WAIT is a formality. Sampled to one log // timestamps are available. This is a device duration, not permission to label the
// line per ~2 s; `wait_us` in the pump's stage perf minus this ≈ the ASIC encode. // remaining host fence wait as pure VCN time; queueing and synchronization remain in it.
if self.ts_period_ns > 0.0 && f.ts_pool != vk::QueryPool::null() { if self.ts_period_ns > 0.0
&& f.ts_pool != vk::QueryPool::null()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{
let mut ts = [0u64; 2]; let mut ts = [0u64; 2];
if dev if dev
.get_query_pool_results( .get_query_pool_results(
@@ -2797,16 +3144,26 @@ impl VulkanVideoEncoder {
vk::QueryResultFlags::TYPE_64 | vk::QueryResultFlags::WAIT, vk::QueryResultFlags::TYPE_64 | vk::QueryResultFlags::WAIT,
) )
.is_ok() .is_ok()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{ {
self.perf_at = std::time::Instant::now(); self.perf_at = std::time::Instant::now();
let csc_us = (ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0; let pre_encode_us =
tracing::info!( (ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0;
csc_us = format!("{csc_us:.0}"), if self.rgb.as_ref().is_some_and(|r| r.padded) {
au_bytes = len, tracing::info!(
"vulkan-encode split (sampled): csc=GPU compute batch (import barriers + \ rgb_copy_us = format!("{pre_encode_us:.0}"),
CSC + plane copies); ASIC encode stage-perf wait_us csc" au_bytes = len,
); "vulkan-encode split (sampled): padded RGB copy device time before EFC; \
remaining fence wait still includes queue synchronization + RGBYUV EFC \
+ video encode"
);
} else {
tracing::info!(
csc_us = format!("{pre_encode_us:.0}"),
au_bytes = len,
"vulkan-encode split (sampled): RGB→NV12 compute batch device time; \
remaining fence wait still includes queue synchronization + video encode"
);
}
} }
} }
let f = &self.frames[slot]; let f = &self.frames[slot];
@@ -3412,26 +3769,30 @@ unsafe fn make_frame(
sampler: vk::Sampler, sampler: vk::Sampler,
with_ts: bool, with_ts: bool,
csc: bool, csc: bool,
rgb_pad: bool, pad_fmt: Option<vk::Format>,
f: &mut Frame, f: &mut Frame,
) -> Result<()> { ) -> Result<()> {
// "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`). // "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`).
f.cursor_serial = u64::MAX; f.cursor_serial = u64::MAX;
// Padded-copy RGB staging (unaligned-mode RGB-direct): aligned BGRA encode-src filled by a // Padded-copy staging (unaligned-mode RGB-direct or native NV12): an aligned encode-src in
// transfer blit each frame — concurrent compute (copy) + encode (source read). // the session's picture format, filled by a transfer blit each frame — concurrent compute
if rgb_pad { // (copy) + encode (source read). TRANSFER_SRC because the width-padding pass self-copies the
// staging image's own last visible column (see `record_pad_blit`).
if let Some(fmt) = pad_fmt {
(f.pad_img, f.pad_mem) = make_video_image( (f.pad_img, f.pad_mem) = make_video_image(
device, device,
mem_props, mem_props,
vk::Format::B8G8R8A8_UNORM, fmt,
w, w,
h, h,
1, 1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR | vk::ImageUsageFlags::TRANSFER_DST, vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR
| vk::ImageUsageFlags::TRANSFER_DST
| vk::ImageUsageFlags::TRANSFER_SRC,
profile_list, profile_list,
fams, fams,
)?; )?;
f.pad_view = make_view(device, f.pad_img, vk::Format::B8G8R8A8_UNORM, 0)?; f.pad_view = make_view(device, f.pad_img, fmt, 0)?;
} }
// RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no // RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no
// cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the // cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the
+260 -8
View File
@@ -478,10 +478,14 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b b
}); });
// HDR signalling (10-bit sessions are the HDR path on Windows — same coupling as NVENC): // Colour signalling, written UNCONDITIONALLY (mirrors nvenc_core.rs): the input is already
// BT.2020/PQ colour description + the source's mastering/CLL grade at every IDR. // CSC'd to a specific matrix — BT.709 limited for SDR (the capture-side VideoConverter),
// BT.2020 PQ for HDR (HdrP010Converter) — so the stream must say so. An SDR stream without a
// colour description leaves the choice to the decoder's "unspecified" default, and
// Moonlight/third-party/Android-vendor decoders default to 601 at sub-HD → mis-rendered
// colours. (10-bit sessions are the HDR path on Windows — same coupling as NVENC.)
let hdr = cfg.ten_bit && cfg.codec != Codec::H264; let hdr = cfg.ten_bit && cfg.codec != Codec::H264;
let vsi = hdr.then(|| { let vsi = {
// SAFETY: all-zero is valid; header stamped below. // SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() }); let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() });
b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32; b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32;
@@ -489,11 +493,17 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b.VideoFormat = 5; // unspecified b.VideoFormat = 5; // unspecified
b.VideoFullRange = 0; b.VideoFullRange = 0;
b.ColourDescriptionPresent = 1; b.ColourDescriptionPresent = 1;
b.ColourPrimaries = 9; // BT.2020 if hdr {
b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ) b.ColourPrimaries = 9; // BT.2020
b.MatrixCoefficients = 9; // BT.2020 non-constant b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ)
b b.MatrixCoefficients = 9; // BT.2020 non-constant
}); } else {
b.ColourPrimaries = 1; // BT.709
b.TransferCharacteristics = 1; // BT.709
b.MatrixCoefficients = 1; // BT.709
}
Some(b)
};
let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| { let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| {
// SAFETY: all-zero is valid; header stamped below. // SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> = let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> =
@@ -1994,4 +2004,246 @@ mod tests {
"the bitrate retarget emitted a keyframe (StartNewSequence leak)" "the bitrate retarget emitted a keyframe (StartNewSequence leak)"
); );
} }
/// FULL-CHAIN colour check at the field capture size: a known P010 colour-bar source at
/// 1920x1080 — whose height is NOT 16-aligned, so the ingest `CopySubresourceRegion` copies
/// into a 1920x1088 runtime pool surface whose chroma plane sits at a DIFFERENT row offset
/// than the source's (the seam no 640x480 test exercises) — encoded to Main10 HEVC and
/// dumped to `%TEMP%\pf_qsv_1080_bars.h265` for off-box decode verification against the
/// same codes. On-box this asserts stream shape; the pixel verdict needs a decoder.
#[test]
fn qsv_live_p010_1080_colorbars_dump() {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE,
D3D11_SDK_VERSION, D3D11_SUBRESOURCE_DATA, D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_P010, DXGI_SAMPLE_DESC};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
// (Y, Cb, Cr) 10-bit limited codes for the 8 sRGB bars white/yellow/cyan/green/magenta/
// red/blue/black at 80-nit SDR white under PQ/BT.2020 — the same math as pf-capture's
// `p010_reference` (and the bars_pq2020 client fixture). Stored MSB-aligned (`<<6`).
const BARS: [(u16, u16, u16); 8] = [
(490, 512, 512),
(478, 423, 518),
(464, 525, 473),
(450, 432, 476),
(350, 584, 585),
(325, 448, 598),
(226, 650, 535),
(64, 512, 512),
];
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
// P010 initial data: plane 0 = H rows of W u16 luma; plane 1 = H/2 rows of W u16
// (interleaved Cb,Cr pairs), same pitch. Bars are vertical: bar index = x / (W/8).
let bar_w = (W / 8) as usize;
let mut init = vec![0u16; (W as usize) * (H as usize + H as usize / 2)];
for y in 0..H as usize {
for x in 0..W as usize {
init[y * W as usize + x] = BARS[(x / bar_w).min(7)].0 << 6;
}
}
let chroma_base = (W as usize) * (H as usize);
for cy in 0..(H as usize / 2) {
for cx in 0..(W as usize / 2) {
let (_, cb, cr) = BARS[((cx * 2) / bar_w).min(7)];
init[chroma_base + cy * W as usize + cx * 2] = cb << 6;
init[chroma_base + cy * W as usize + cx * 2 + 1] = cr << 6;
}
}
// SAFETY: self-contained harness on one thread/device (same contract as `drive_live`);
// the initial-data pointer outlives the synchronous CreateTexture2D that reads it.
let (device, tex) = unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(imp.luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(imp.luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().expect("dxgi factory");
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).expect("intel adapter");
let mut device = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
windows::Win32::Foundation::HMODULE::default(),
Default::default(),
None,
D3D11_SDK_VERSION,
Some(&mut device),
None,
None,
)
.expect("d3d11 device on intel adapter");
let device: ID3D11Device = device.expect("device");
let desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
CPUAccessFlags: 0,
MiscFlags: 0,
};
let data = D3D11_SUBRESOURCE_DATA {
pSysMem: init.as_ptr() as *const std::ffi::c_void,
SysMemPitch: W * 2,
SysMemSlicePitch: 0,
};
let mut t: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&data), Some(&mut t))
.expect("bar texture");
(device.clone(), t.expect("texture"))
};
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
let mut keyframes = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
assert!(keyframes >= 1, "expected an IDR in the dump");
let path = std::env::temp_dir().join("pf_qsv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {} AUs ({} bytes, {keyframes} keyframes) to {}",
aus,
stream.len(),
path.display()
);
}
/// The PRODUCTION host chain minus the IDD ring: the REAL `HdrP010Converter` renders the 8
/// sRGB bars into a ring-profile P010 texture (`BIND_RENDER_TARGET` only — RTV-written, not
/// CPU-uploaded) on the VPL implementation's own adapter, and THAT texture goes through the
/// unaligned-height ingest copy into a Main10 encode. Dumped to
/// `%TEMP%\pf_qsv_conv_1080_bars.h265`; expected decode codes = the bars_pq2020 fixture set
/// (see `hdr_p010_convert_bars_on_luid`).
#[test]
fn qsv_live_hdr_converter_e2e_1080_dump() {
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
let (device, tex) = pf_capture::dxgi::hdr_p010_convert_bars_on_luid(imp.luid, W, H)
.expect("converter bars");
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
let path = std::env::temp_dir().join("pf_qsv_conv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {aus} AUs ({} bytes) to {}",
stream.len(),
path.display()
);
}
} }
+49
View File
@@ -330,6 +330,7 @@ fn open_video_backend(
{ {
match vulkan_video::VulkanVideoEncoder::open( match vulkan_video::VulkanVideoEncoder::open(
codec, codec,
format,
width, width,
height, height,
fps, fps,
@@ -344,12 +345,32 @@ fn open_video_backend(
); );
return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan")); return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan"));
} }
// Native NV12 (PUNKTFUNK_PIPEWIRE_NV12 capture) has no VAAPI fallback:
// libav's dmabuf lane would import the two-plane buffer as packed RGB
// (silent garbage) and its CPU lane bails per frame — die crisply instead.
Err(e) if format == PixelFormat::Nv12 => {
return Err(e.context(
"Vulkan Video open failed on a native-NV12 capture \
no VAAPI fallback exists; set PUNKTFUNK_PIPEWIRE_NV12=0 to \
restore the packed-RGB negotiation",
));
}
Err(e) => tracing::warn!( Err(e) => tracing::warn!(
error = %format!("{e:#}"), error = %format!("{e:#}"),
"Vulkan Video encode open failed — falling back to libav VAAPI" "Vulkan Video encode open failed — falling back to libav VAAPI"
), ),
} }
} }
// Same rule when the Vulkan backend was never eligible (H264 session,
// PUNKTFUNK_VULKAN_ENCODE=0, or a build without the feature).
if format == PixelFormat::Nv12 {
anyhow::bail!(
"native NV12 capture requires the Vulkan Video encoder (HEVC/AV1 \
session, --features vulkan-encode, PUNKTFUNK_VULKAN_ENCODE not 0) this \
session resolved to libav VAAPI; set PUNKTFUNK_PIPEWIRE_NV12=0 to restore \
the packed-RGB negotiation"
);
}
vaapi::VaapiEncoder::open( vaapi::VaapiEncoder::open(
codec, codec,
format, format,
@@ -390,6 +411,7 @@ fn open_video_backend(
} }
vulkan_video::VulkanVideoEncoder::open( vulkan_video::VulkanVideoEncoder::open(
codec, codec,
format,
width, width,
height, height,
fps, fps,
@@ -819,6 +841,33 @@ fn vulkan_encode_enabled() -> bool {
.unwrap_or(true) .unwrap_or(true)
} }
/// Whether THIS session's encoder can ingest a producer-native NV12 capture: only the raw
/// Vulkan Video backend does (libav VAAPI would misread the two-plane buffer as packed RGB —
/// [`open_video`] refuses the combination), so the session's codec must be one it encodes and
/// the backend must be eligible to open. The host facade threads the verdict into the capture
/// negotiation (`OutputFormat::nv12_native` → `ZeroCopyPolicy::native_nv12_session`), which
/// then PREFERS gamescope's producer-side NV12 pod (default-on; `PUNKTFUNK_PIPEWIRE_NV12=0`
/// escapes at the capture gate).
#[cfg(target_os = "linux")]
pub fn linux_native_nv12_ok(codec: Codec) -> bool {
#[cfg(feature = "vulkan-encode")]
{
matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
// NVENC/PyroWave prefs never open the Vulkan Video backend; every other pref
// (auto/vaapi/amd/intel/vulkan) tries it first on AMD/Intel — see [`open_video`].
&& !matches!(
pf_host_config::config().encoder_pref.as_str(),
"nvenc" | "nvidia" | "cuda" | "pyrowave"
)
}
#[cfg(not(feature = "vulkan-encode"))]
{
let _ = codec;
false
}
}
/// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA /// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA
/// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT /// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT
/// create a CUDA context (that would allocate GPU state on every host that merely *might* be /// create a CUDA context (that would allocate GPU state on every host that merely *might* be
+36 -10
View File
@@ -105,13 +105,15 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888 Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888 Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888 Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// Linux native NV12 capture (gamescope PipeWire): one LINEAR dmabuf with contiguous Y then
// interleaved UV, exposed under DRM_FORMAT_NV12.
Nv12 => drm_fourcc_code(b"NV12"),
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020). // The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010 X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010 X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path. // 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on // Rgb10a2/P010 are Windows formats; Yuv444 is OUR convert output, never a capture source.
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format. Rgb | Bgr | Rgb10a2 | P010 | Yuv444 => return None,
Rgb | Bgr | Rgb10a2 | Nv12 | P010 | Yuv444 => return None,
}) })
} }
@@ -144,6 +146,12 @@ pub struct OutputFormat {
/// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the /// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the
/// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture). /// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture).
pub pyrowave: bool, pub pyrowave: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (Linux raw Vulkan Video
/// backend on an H265/AV1 session — see `pf_encode::linux_native_nv12_ok`). The Linux capture
/// negotiation only offers gamescope the NV12 pod when this is set: libav VAAPI (the H264
/// codec's backend, and the fallback family) would misread the two-plane buffer as packed
/// RGB. Always `false` on Windows (the IDD-push capturer owns its own formats).
pub nv12_native: bool,
} }
impl OutputFormat { impl OutputFormat {
@@ -161,6 +169,11 @@ impl OutputFormat {
chroma_444: false, chroma_444: false,
// GameStream never negotiates PyroWave (native punktfunk/1 only). // GameStream never negotiates PyroWave (native punktfunk/1 only).
pyrowave: false, pyrowave: false,
// Conservative: the GameStream + spike paths don't resolve the codec here, and a
// Moonlight client may negotiate H264 (whose VAAPI backend can't ingest NV12) — so
// they never prefer the producer-native NV12 pod. The punktfunk/1 plane opts in via
// `SessionPlan::output_format()`, which knows the codec.
nv12_native: false,
} }
} }
} }
@@ -183,6 +196,11 @@ pub struct CursorOverlay {
pub rgba: std::sync::Arc<Vec<u8>>, pub rgba: std::sync::Arc<Vec<u8>>,
/// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves. /// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves.
pub serial: u64, pub serial: u64,
/// Hotspot (the pixel that IS the pointer position) within `w`×`h`. The blend paths ignore
/// it (`x`/`y` are already hotspot-adjusted); the cursor-forward channel ships it to the
/// client so a locally-drawn OS cursor points with the right pixel.
pub hot_x: u32,
pub hot_y: u32,
} }
/// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of /// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of
@@ -201,18 +219,26 @@ pub struct CapturedFrame {
pub cursor: Option<CursorOverlay>, pub cursor: Option<CursorOverlay>,
} }
/// A captured frame still living in a single-plane packed-RGB dmabuf (the VAAPI zero-copy path). /// A captured frame still living in a DMA-BUF. Packed RGB uses one plane. Native Linux NV12
/// (gamescope PipeWire) travels in ONE fd: Y starts at `offset`, and the interleaved UV plane
/// lives at `plane1`'s offset/stride when the producer reported them — else at the contiguous
/// fallback `offset + stride * frame_height` with the shared `stride`.
///
/// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be /// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be
/// imported into a VA surface there without the compositor's buffer being closed underneath it. /// imported there without the compositor's buffer being closed underneath it. Content stability
/// (Content stability across the brief import window relies on the compositor's buffer pool depth, /// across the brief import window relies on the compositor's buffer pool depth, like any zero-copy
/// same as any zero-copy capture — the VAAPI importer copies into its own NV12 surface promptly.) /// capture.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub struct DmabufFrame { pub struct DmabufFrame {
pub fd: std::os::fd::OwnedFd, pub fd: std::os::fd::OwnedFd,
/// DRM FourCC of the packed-RGB plane (e.g. `XR24` for BGRx). /// DRM FourCC (`XR24` for BGRx, `NV12` for native 4:2:0).
pub fourcc: u32, pub fourcc: u32,
/// DRM format modifier the compositor allocated (0 = LINEAR). /// DRM format modifier the compositor allocated (0 = LINEAR).
pub modifier: u64, pub modifier: u64,
/// Second-plane `(offset, stride)` within the SAME fd, when the producer reported one (the
/// PipeWire buffer's plane-1 chunk — NV12's interleaved UV). `None` falls back to the
/// contiguous-plane contract above. Always `None` for single-plane packed RGB.
pub plane1: Option<(u32, u32)>,
pub offset: u32, pub offset: u32,
pub stride: u32, pub stride: u32,
} }
@@ -225,8 +251,8 @@ pub enum FramePayload {
/// The dmabuf has already been imported + copied into this owned device buffer. /// The dmabuf has already been imported + copied into this owned device buffer.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
Cuda(pf_zerocopy::DeviceBuffer), Cuda(pf_zerocopy::DeviceBuffer),
/// A raw packed-RGB dmabuf — the AMD/Intel (VAAPI) zero-copy path. The encoder imports it into /// A raw DMA-BUF: packed RGB for the existing GPU CSC paths, or native NV12 from a producer
/// a VA surface and does RGB→NV12 on the GPU video engine (no host CSC, no upload). /// such as gamescope. The encoder imports it without a host copy.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
Dmabuf(DmabufFrame), Dmabuf(DmabufFrame),
/// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame. /// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame.
+17
View File
@@ -63,6 +63,13 @@ pub struct HostConfig {
/// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0. /// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0.
/// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth). /// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool, pub four_four_four: bool,
/// `PUNKTFUNK_CHACHA20` — host policy gate for the negotiated ChaCha20-Poly1305 session
/// cipher (design/chacha20-session-cipher.md). **Default ON** (pure rollout safety — perf-only,
/// both AEADs are full-strength): the host merely *allows* it — a session only seals with
/// ChaCha when the client advertised `VIDEO_CAP_CHACHA20` (set by soft-AES armv7 clients,
/// e.g. webOS TVs, whose GCM decrypt caps at ~100 Mbps); everyone else stays AES-128-GCM.
/// `PUNKTFUNK_CHACHA20=0`/`false`/`off`/`no` disables.
pub chacha20: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation. /// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool, pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic). /// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -147,6 +154,16 @@ impl HostConfig {
) )
}) })
.unwrap_or(true), .unwrap_or(true),
// Default ON, explicit-off grammar (the client's VIDEO_CAP_CHACHA20 bit is the real
// per-session switch; see the field doc).
chacha20: val("PUNKTFUNK_CHACHA20")
.map(|s| {
!matches!(
s.trim().to_ascii_lowercase().as_str(),
"0" | "false" | "off" | "no"
)
})
.unwrap_or(true),
perf: flag("PUNKTFUNK_PERF"), perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"), video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"), compositor: val("PUNKTFUNK_COMPOSITOR"),
+119
View File
@@ -0,0 +1,119 @@
//! Client-side cursor rendering (design/remote-desktop-sweep.md M2): the host forwards the
//! pointer's SHAPE (reliable control stream, cached by serial) and per-frame STATE (lossy
//! `0xD0` — position/visibility), and WE draw it as a real OS cursor — pointer feel stops
//! paying the video round-trip (the Parsec/RDP model). Active only when the session
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use std::collections::HashMap;
use std::time::Duration;
/// Shape serials cached at most — cursors cycle through a handful of shapes (arrow, I-beam,
/// resize…); a runaway host can't grow the map past this (the cache resets, shapes re-arrive
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → built OS cursor. An SDL `Cursor` must outlive its `set()`, so the cache owns
/// every shape ever applied this session (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, Cursor>,
/// The serial currently installed via `Cursor::set` (`None` = default/system cursor).
installed: Option<u32>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
impl CursorChannel {
pub fn new(connector: &NativeClient) -> CursorChannel {
let negotiated = connector.host_caps() & HOST_CAP_CURSOR != 0;
if negotiated {
tracing::info!("cursor channel negotiated — host cursor renders locally");
}
CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
state: None,
}
}
/// Whether the host forwards the cursor this session (it no longer composites one).
pub fn negotiated(&self) -> bool {
self.negotiated
}
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal.
pub fn pump(&mut self, connector: &NativeClient, mouse: &MouseUtil, desktop_active: bool) {
if !self.negotiated {
return;
}
while let Ok(shape) = connector.next_cursor_shape(Duration::ZERO) {
if self.shapes.len() >= SHAPE_CACHE_MAX {
// Degenerate host: reset — live shapes re-install via the serial-miss path.
self.shapes.clear();
self.installed = None;
}
let mut data = shape.rgba;
let built = sdl3::surface::Surface::from_data(
&mut data,
shape.w as u32,
shape.h as u32,
shape.w as u32 * 4,
sdl3::pixels::PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())
.and_then(|surf| {
Cursor::from_surface(&surf, shape.hot_x as i32, shape.hot_y as i32)
.map_err(|e| e.to_string())
});
match built {
Ok(cursor) => {
// A re-sent serial replaces its entry; force re-install if it's current.
if self.installed == Some(shape.serial) {
self.installed = None;
}
self.shapes.insert(shape.serial, cursor);
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
}
if !desktop_active {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
Cursor::from_system(SystemCursor::Arrow)
.map(|c| c.set())
.ok();
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some(st.serial) {
if let Some(cursor) = self.shapes.get(&st.serial) {
cursor.set();
self.installed = Some(st.serial);
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
}
// Visibility follows the host (a host app hid its pointer ⇒ ours hides too). Queried,
// not shadowed, so apply_capture's own show/hide calls can never desync us.
if mouse.is_cursor_showing() != st.visible() {
mouse.show_cursor(st.visible());
}
}
}
+63 -3
View File
@@ -14,10 +14,17 @@
//! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are //! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are
//! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would //! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would
//! otherwise send a datagram per event). //! otherwise send a datagram per event).
//!
//! The DESKTOP mouse model (design/remote-desktop-sweep.md M1) reuses this same engage/
//! release state but never locks the pointer: the local cursor moves freely (hidden over
//! the window — the host's composited cursor is the one you see) and motion goes on the
//! wire as absolute positions through the letterbox (`MouseMoveAbs`, latest-wins per loop
//! iteration). Requires a host injector with absolute support — gamescope's EIS is
//! relative-only, so sessions there are pinned to capture ([`Capture::new`] `abs_ok`).
use crate::keymap_sdl; use crate::keymap_sdl;
use crate::touch::{Abs, Act, Gestures}; use crate::touch::{Abs, Act, Gestures};
use pf_client_core::trust::TouchMode; use pf_client_core::trust::{MouseMode, TouchMode};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
@@ -41,6 +48,13 @@ pub struct Capture {
held_buttons: HashSet<u32>, held_buttons: HashSet<u32>,
/// Relative motion not yet on the wire, summed per loop iteration. /// Relative motion not yet on the wire, summed per loop iteration.
pending_rel: (i32, i32), pending_rel: (i32, i32),
/// Desktop-model position not yet on the wire, latest-wins per loop iteration.
pending_abs: Option<Abs>,
/// The desktop (absolute, uncaptured) mouse model is active. Flipped live by the
/// Ctrl+Alt+Shift+M chord; never true unless `abs_ok`.
desktop: bool,
/// The host injector accepts `MouseMoveAbs` (any compositor but gamescope).
abs_ok: bool,
/// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space — /// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space —
/// precision surfaces deliver sub-unit deltas; truncating each event drops the tail. /// precision surfaces deliver sub-unit deltas; truncating each event drops the tail.
scroll_acc: (f64, f64), scroll_acc: (f64, f64),
@@ -70,10 +84,14 @@ fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, fl
} }
impl Capture { impl Capture {
/// `abs_ok` = the host injector accepts absolute pointer events; without it the
/// desktop model is unavailable and `mouse_mode` silently resolves to capture.
pub fn new( pub fn new(
connector: Arc<NativeClient>, connector: Arc<NativeClient>,
touch_mode: TouchMode, touch_mode: TouchMode,
invert_scroll: bool, invert_scroll: bool,
mouse_mode: MouseMode,
abs_ok: bool,
) -> Capture { ) -> Capture {
Capture { Capture {
connector, connector,
@@ -82,6 +100,9 @@ impl Capture {
held_keys: HashSet::new(), held_keys: HashSet::new(),
held_buttons: HashSet::new(), held_buttons: HashSet::new(),
pending_rel: (0, 0), pending_rel: (0, 0),
pending_abs: None,
desktop: abs_ok && mouse_mode == MouseMode::Desktop,
abs_ok,
scroll_acc: (0.0, 0.0), scroll_acc: (0.0, 0.0),
touch_slots: HashMap::new(), touch_slots: HashMap::new(),
touch_mode, touch_mode,
@@ -94,6 +115,24 @@ impl Capture {
self.captured self.captured
} }
/// The desktop (absolute, uncaptured) mouse model is active.
pub fn desktop(&self) -> bool {
self.desktop
}
/// Flip capture ⇄ desktop (the Ctrl+Alt+Shift+M chord). `None` = the host can't take
/// absolute pointer events (gamescope), so the chord has nothing to offer; otherwise
/// the new desktop state. Motion gathered under the old model never crosses modes.
pub fn toggle_desktop(&mut self) -> Option<bool> {
if !self.abs_ok {
return None;
}
self.desktop = !self.desktop;
self.pending_rel = (0, 0);
self.pending_abs = None;
Some(self.desktop)
}
/// Whether a regained focus should re-engage: yes unless the user released /// Whether a regained focus should re-engage: yes unless the user released
/// deliberately (the chord keeps its meaning across an Alt-Tab). /// deliberately (the chord keeps its meaning across an Alt-Tab).
pub fn should_reengage(&self) -> bool { pub fn should_reengage(&self) -> bool {
@@ -117,6 +156,7 @@ impl Capture {
return false; return false;
} }
self.pending_rel = (0, 0); // never flush motion gathered while captured self.pending_rel = (0, 0); // never flush motion gathered while captured
self.pending_abs = None;
for vk in self.held_keys.drain() { for vk in self.held_keys.drain() {
send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0); send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0);
} }
@@ -132,22 +172,42 @@ impl Capture {
true true
} }
/// Forward the coalesced motion delta, if any — one datagram per loop iteration. /// Forward the coalesced motion, if any — one datagram per loop iteration. Only one
/// of the two stores is ever populated (the run loop routes by [`desktop`](Self::desktop)).
pub fn flush_motion(&mut self) { pub fn flush_motion(&mut self) {
let (dx, dy) = std::mem::take(&mut self.pending_rel); let (dx, dy) = std::mem::take(&mut self.pending_rel);
if dx != 0 || dy != 0 { if dx != 0 || dy != 0 {
send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0); send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0);
} }
if let Some(a) = self.pending_abs.take() {
send(
&self.connector,
InputKind::MouseMoveAbs,
0,
a.x,
a.y,
Self::touch_flags(a.w, a.h),
);
}
} }
/// Relative motion (SDL relative mouse mode delivers raw deltas while locked). /// Relative motion (SDL relative mouse mode delivers raw deltas while locked).
pub fn on_motion(&mut self, xrel: f32, yrel: f32) { pub fn on_motion(&mut self, xrel: f32, yrel: f32) {
if self.captured { if self.captured && !self.desktop {
self.pending_rel.0 += xrel as i32; self.pending_rel.0 += xrel as i32;
self.pending_rel.1 += yrel as i32; self.pending_rel.1 += yrel as i32;
} }
} }
/// Desktop-model motion: the cursor's position mapped into the letterboxed content
/// rect. Latest-wins — intermediate positions carry no information the final one
/// doesn't (unlike deltas, which must sum).
pub fn on_motion_abs(&mut self, abs: Abs) {
if self.captured && self.desktop {
self.pending_abs = Some(abs);
}
}
pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) { pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) {
if !self.captured { if !self.captured {
return; return;
+2
View File
@@ -16,6 +16,8 @@
#[cfg(any(target_os = "linux", windows))] #[cfg(any(target_os = "linux", windows))]
pub mod csc; pub mod csc;
#[cfg(any(target_os = "linux", windows))]
pub mod cursor;
#[cfg(windows)] #[cfg(windows)]
pub mod d3d11; pub mod d3d11;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
+119 -21
View File
@@ -20,11 +20,11 @@ use crate::vk::{FrameInput, Presenter};
use anyhow::{Context as _, Result}; use anyhow::{Context as _, Result};
use pf_client_core::gamepad::GamepadService; use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats}; use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use pf_client_core::video::VulkanDecodeDevice; use pf_client_core::video::VulkanDecodeDevice;
use pf_client_core::video::{DecodedFrame, DecodedImage}; use pf_client_core::video::{DecodedFrame, DecodedImage};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::config::Mode; use punktfunk_core::config::{CompositorPref, Mode};
use sdl3::event::{Event, WindowEvent}; use sdl3::event::{Event, WindowEvent};
use sdl3::keyboard::Mod; use sdl3::keyboard::Mod;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
@@ -48,6 +48,11 @@ pub struct SessionOpts {
/// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per /// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per
/// session — a mouse-only client leaves this at the default and never sees a finger. /// session — a mouse-only client leaves this at the default and never sees a finger.
pub touch_mode: TouchMode, pub touch_mode: TouchMode,
/// Physical-mouse model: `Capture` (pointer lock + relative, the default) or `Desktop`
/// (uncaptured absolute pointer — design/remote-desktop-sweep.md M1). Ctrl+Alt+Shift+M
/// flips it live; silently resolves to capture on hosts without absolute injection
/// (gamescope).
pub mouse_mode: MouseMode,
/// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]). /// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]).
pub invert_scroll: bool, pub invert_scroll: bool,
/// Emit the `{"ready":true}` stdout line after the first presented frame. /// Emit the `{"ready":true}` stdout line after the first presented frame.
@@ -228,6 +233,9 @@ struct StreamState {
/// window-normalized position must be re-based onto the content rect). `None` until /// window-normalized position must be re-based onto the content rect). `None` until
/// the first frame; touches before then have nothing to map onto and are dropped. /// the first frame; touches before then have nothing to map onto and are dropped.
last_video: Option<(u32, u32)>, last_video: Option<(u32, u32)>,
/// Client-side cursor rendering (M2 cursor channel) — created with the connector; inert
/// when the host didn't negotiate the channel.
cursor_chan: Option<crate::cursor::CursorChannel>,
} }
impl StreamState { impl StreamState {
@@ -258,6 +266,7 @@ impl StreamState {
frames: wake_rx, frames: wake_rx,
connector: None, connector: None,
capture: None, capture: None,
cursor_chan: None,
force_software, force_software,
canceled: false, canceled: false,
ready_announced: false, ready_announced: false,
@@ -333,6 +342,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// bottom-right corner (the reported bug). The menu/library is keyboard+gamepad-driven // bottom-right corner (the reported bug). The menu/library is keyboard+gamepad-driven
// and consumes no mouse, so nothing wanted these synthetic events anyway. // and consumes no mouse, so nothing wanted these synthetic events anyway.
sdl3::hint::set("SDL_TOUCH_MOUSE_EVENTS", "0"); sdl3::hint::set("SDL_TOUCH_MOUSE_EVENTS", "0");
// The Wayland `app_id` (and X11 WM_CLASS) — compositors match it against
// io.unom.Punktfunk.desktop for the window/taskbar icon. Without it SDL uses a generic
// identity and the session window gets the default-Wayland icon (the Linux analog of
// the AppUserModelID adoption above).
sdl3::hint::set("SDL_APP_ID", "io.unom.Punktfunk");
let sdl = sdl3::init().context("SDL init")?; let sdl = sdl3::init().context("SDL init")?;
let video = sdl.video().context("SDL video")?; let video = sdl.video().context("SDL video")?;
let events = sdl.event().context("SDL events")?; let events = sdl.event().context("SDL events")?;
@@ -485,7 +499,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
WindowEvent::FocusLost => { WindowEvent::FocusLost => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(false) { if cap.release(false) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
tracing::info!("focus lost — input released"); tracing::info!("focus lost — input released");
} }
} }
@@ -496,7 +510,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.should_reengage() { if cap.should_reengage() {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
tracing::info!("focus gained — input recaptured"); tracing::info!("focus gained — input recaptured");
} }
} }
@@ -532,20 +546,39 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.captured() { if cap.captured() {
cap.release(true); cap.release(true);
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} else { } else {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} }
tracing::info!(captured = cap.captured(), "chord: release/engage"); tracing::info!(captured = cap.captured(), "chord: release/engage");
} }
continue; continue;
} }
// Mouse model flip (capture ⇄ desktop) — applies immediately when
// engaged; a released stream just changes what the next engage does.
if chord && sc == Scancode::M {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
match cap.toggle_desktop() {
Some(desktop) => {
if cap.captured() {
apply_capture(&mut window, &mouse, true, desktop);
}
tracing::info!(desktop, "chord: mouse mode");
}
None => tracing::info!(
"chord: mouse mode — host has no absolute pointer \
(gamescope), staying captured"
),
}
}
continue;
}
if chord && sc == Scancode::D { if chord && sc == Scancode::D {
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("chord: disconnect"); tracing::info!("chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
// The pump emits Ended(None); the end path routes per mode. // The pump emits Ended(None); the end path routes per mode.
} }
continue; continue;
@@ -578,9 +611,34 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.on_key_up(sc); cap.on_key_up(sc);
} }
} }
Event::MouseMotion { xrel, yrel, .. } => { Event::MouseMotion {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { x, y, xrel, yrel, ..
cap.on_motion(xrel, yrel); } => {
if let Some(st) = stream.as_mut() {
let video = st.last_video;
if let Some(cap) = st.capture.as_mut() {
if cap.desktop() {
// Desktop model: the cursor's window position through the
// letterbox (same mapping as a pointer-mode finger).
// Before the first decoded frame there is nothing to map
// onto — dropped, like touch.
if let Some(video) = video {
let (lw, lh) = window.size();
let nx = x / lw.max(1) as f32;
let ny = y / lh.max(1) as f32;
let (ax, ay, aw, ah) =
finger_to_content(window.size_in_pixels(), video, nx, ny);
cap.on_motion_abs(Abs {
x: ax,
y: ay,
w: aw,
h: ah,
});
}
} else {
cap.on_motion(xrel, yrel);
}
}
} }
} }
Event::MouseButtonDown { mouse_btn, .. } => { Event::MouseButtonDown { mouse_btn, .. } => {
@@ -588,7 +646,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if !cap.captured() { if !cap.captured() {
// The engaging click is suppressed toward the host. // The engaging click is suppressed toward the host.
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} else { } else {
cap.on_button_down(mouse_btn); cap.on_button_down(mouse_btn);
} }
@@ -690,6 +748,17 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.flush_motion(); cap.flush_motion();
} }
// Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor —
// only meaningful in the desktop mouse model (capture's relative lock hides it).
if let Some(st) = stream.as_mut() {
if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) {
let desktop_active = st
.capture
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active);
}
}
// Text input follows the overlay's editing state (edge-triggered). // Text input follows the overlay's editing state (edge-triggered).
let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active()); let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active());
@@ -709,7 +778,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
while escape_rx.try_recv().is_ok() { while escape_rx.try_recv().is_ok() {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(true) { if cap.release(true) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
if fullscreen && !opts.fullscreen { if fullscreen && !opts.fullscreen {
@@ -722,7 +791,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("controller chord: disconnect"); tracing::info!("controller chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
@@ -813,10 +882,28 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
.ok(); .ok();
gamepad.attach(c.clone()); gamepad.attach(c.clone());
st.clock_offset = Some(c.clock_offset_shared()); st.clock_offset = Some(c.clock_offset_shared());
let mut cap = Capture::new(c.clone(), opts.touch_mode, opts.invert_scroll); // gamescope's EIS grants only a relative pointer — absolute sends
// would be dropped, so the desktop model is pinned off there. Auto
// (an older host that didn't say) stays allowed: Windows hosts and
// pre-Welcome-compositor Linux hosts both take absolute.
let abs_ok = c.resolved_compositor != CompositorPref::Gamescope;
if opts.mouse_mode == MouseMode::Desktop && !abs_ok {
tracing::info!(
"desktop mouse mode unavailable on a gamescope host \
(relative-only input) using capture"
);
}
let mut cap = Capture::new(
c.clone(),
opts.touch_mode,
opts.invert_scroll,
opts.mouse_mode,
abs_ok,
);
cap.engage(); // capture engages when the stream starts (ui_stream parity) cap.engage(); // capture engages when the stream starts (ui_stream parity)
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
st.capture = Some(cap); st.capture = Some(cap);
st.cursor_chan = Some(crate::cursor::CursorChannel::new(&c));
st.connector = Some(c); st.connector = Some(c);
if let Some(f) = opts.on_connected.as_mut() { if let Some(f) = opts.on_connected.as_mut() {
f(fingerprint); f(fingerprint);
@@ -865,7 +952,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = stream.take() { if let Some(st) = stream.take() {
st.shutdown(); st.shutdown();
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
if let Some(o) = overlay.as_mut() { if let Some(o) = overlay.as_mut() {
// A user-canceled dial ends silently — no error scene. // A user-canceled dial ends silently — no error scene.
if canceled { if canceled {
@@ -882,7 +969,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = &mut st.capture { if let Some(cap) = &mut st.capture {
cap.release(true); cap.release(true);
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
match &mode { match &mode {
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)), ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),
ModeCtl::Browse(_) => { ModeCtl::Browse(_) => {
@@ -1472,11 +1559,22 @@ impl ResizeIndicator {
/// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process /// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process
/// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor /// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor
/// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts). /// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts).
fn apply_capture(window: &mut sdl3::video::Window, mouse: &sdl3::mouse::MouseUtil, on: bool) { ///
mouse.set_relative_mouse_mode(window, on); /// The `desktop` mouse model never locks: the pointer roams (and leaves the window)
/// freely, the local cursor is hidden over the window — the host's composited cursor,
/// tracking our absolute sends, is the one you see (until the M2 cursor channel flips
/// who draws it) — and system chords stay local (a remote desktop is something you
/// Alt-Tab away from, not into). `desktop` only matters while `on`.
fn apply_capture(
window: &mut sdl3::video::Window,
mouse: &sdl3::mouse::MouseUtil,
on: bool,
desktop: bool,
) {
mouse.set_relative_mouse_mode(window, on && !desktop);
mouse.show_cursor(!on); mouse.show_cursor(!on);
#[cfg(windows)] #[cfg(windows)]
window.set_keyboard_grab(on); window.set_keyboard_grab(on && !desktop);
} }
/// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)? /// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)?
@@ -1594,7 +1692,7 @@ struct PresentedWindow {
/// The capture hints (`ui_stream` parity — the words the user reads while released). /// The capture hints (`ui_stream` parity — the words the user reads while released).
const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats"; Ctrl+Alt+Shift+M mouse mode · Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats";
const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave"; Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave";
+9 -1
View File
@@ -30,9 +30,17 @@ impl Presenter {
// switch modes before anything touches this frame. Only where the surface // switch modes before anything touches this frame. Only where the surface
// offers HDR10 — otherwise PQ stays on the SDR swapchain and the CSC shader // offers HDR10 — otherwise PQ stays on the SDR swapchain and the CSC shader
// tonemaps (mode 1). // tonemaps (mode 1).
//
// CPU frames NEVER take the HDR10 surface: software decode uploads swscale RGBA with
// no CSC/tonemap pass, so on a mode-0 swapchain that sRGB-encoded content would be
// composed as PQ — the field-reported psychedelic cyan/magenta picture (reproduced
// 2026-07-21: Fedora-class client, no hw HEVC decode, GNOME/Mesa offering HDR10 even
// on an SDR desktop). On the SDR swapchain the same frames are merely untonemapped
// (washed out) — wrong in the known, benign way until the CPU lane grows a real
// PQ→sRGB pass.
let frame_pq = match &input { let frame_pq = match &input {
FrameInput::Redraw => None, FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f.color.is_pq()), FrameInput::Cpu(_) => Some(false),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => Some(d.color.is_pq()), FrameInput::Dmabuf(d) => Some(d.color.is_pq()),
FrameInput::VkFrame(v) => Some(v.color.is_pq()), FrameInput::VkFrame(v) => Some(v.color.is_pq()),
+9
View File
@@ -617,6 +617,15 @@ pub(super) fn pick_formats(
surface: vk::SurfaceKHR, surface: vk::SurfaceKHR,
colorspace_ext: bool, colorspace_ext: bool,
) -> Result<(vk::SurfaceFormatKHR, Option<vk::SurfaceFormatKHR>)> { ) -> Result<(vk::SurfaceFormatKHR, Option<vk::SurfaceFormatKHR>)> {
// `PUNKTFUNK_HDR10=0` (explicit-off grammar) refuses the HDR10/ST.2084 swapchain outright,
// pinning PQ streams to the shader tonemap on an SDR surface. Two reasons this exists:
// desktop compositors newly offer HDR10 even on SDR desktops (GNOME 48 / Plasma 6 with
// Mesa ≥ 25.1 — a lane that otherwise engages silently), and it is the A/B lever that
// splits "HDR10 passthrough composes wrong" from "the decoded planes are wrong" in the
// field without rebuilding anything.
let colorspace_ext = colorspace_ext
&& !std::env::var("PUNKTFUNK_HDR10")
.is_ok_and(|v| matches!(v.as_str(), "0" | "false" | "off" | "no"));
let formats = unsafe { surface_i.get_physical_device_surface_formats(pdev, surface) }?; let formats = unsafe { surface_i.get_physical_device_surface_formats(pdev, surface) }?;
let mut sdr = None; let mut sdr = None;
for want in [vk::Format::B8G8R8A8_UNORM, vk::Format::R8G8B8A8_UNORM] { for want in [vk::Format::B8G8R8A8_UNORM, vk::Format::R8G8B8A8_UNORM] {
+28
View File
@@ -254,6 +254,34 @@ pub fn detect() -> Result<Compositor> {
} }
} }
/// Attach-only probes: while any scope is held, backend `create` paths must not stop, relaunch,
/// or take over box sessions — they may only attach to an already-live output, and fail fast
/// otherwise. The capture-loss rebuild holds one for its first seconds: right after a capture
/// loss the active-session detection can be STALE (a Game→Desktop switch observed live: the
/// probe's gamescope re-acquire restarted `gamescope-session.target` and yanked the user out of
/// the KDE session they had just switched to). A counter, so overlapping scopes compose.
static REBUILD_PROBES: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
/// RAII scope marking pipeline builds as attach-only probes (see [`rebuild_probe_active`]).
pub struct RebuildProbeScope(());
pub fn rebuild_probe_scope() -> RebuildProbeScope {
REBUILD_PROBES.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
RebuildProbeScope(())
}
impl Drop for RebuildProbeScope {
fn drop(&mut self) {
REBUILD_PROBES.fetch_sub(1, std::sync::atomic::Ordering::SeqCst);
}
}
/// Is any [`rebuild_probe_scope`] active? Destructive session operations (stop/relaunch/
/// takeover-restart) must be skipped while true.
pub fn rebuild_probe_active() -> bool {
REBUILD_PROBES.load(std::sync::atomic::Ordering::SeqCst) > 0
}
/// Open the virtual-display driver for `compositor`. /// Open the virtual-display driver for `compositor`.
pub fn open(compositor: Compositor) -> Result<Box<dyn VirtualDisplay>> { pub fn open(compositor: Compositor) -> Result<Box<dyn VirtualDisplay>> {
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
@@ -325,6 +325,29 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
if steamos_session_present() { if steamos_session_present() {
return create_managed_session_steamos(mode); return create_managed_session_steamos(mode);
} }
// Attach-only rebuild probe: reuse a live same-mode session, but NEVER stop/relaunch box
// sessions — right after a capture loss the caller's session detection can be stale, and a
// destructive rebuild here would fight the session the user just switched to.
if crate::rebuild_probe_active() {
let guard = MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner());
let same_mode = guard.as_ref().is_some_and(|s| {
s.width == mode.width && s.height == mode.height && s.refresh_hz == mode.refresh_hz
});
if same_mode {
if let Some(node_id) = find_gamescope_node() {
point_injector_at_eis();
tracing::info!(
node_id,
"gamescope session: attach-only probe reusing live node"
);
return Ok(managed_output(node_id, mode));
}
}
return Err(anyhow!(
"gamescope session has no attachable live node — attach-only rebuild probe refuses \
to stop/relaunch box sessions (re-detection follows the live session)"
));
}
// Steam is single-instance: if the box autologged into gaming mode on a physical display (the // Steam is single-instance: if the box autologged into gaming mode on a physical display (the
// Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and // Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and
// renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by // renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by
@@ -607,12 +630,17 @@ fn write_steamos_dropin(shim_dir: &std::path::Path, mode: Mode) -> Result<()> {
if let Some(parent) = path.parent() { if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).with_context(|| format!("mkdir {}", parent.display()))?; std::fs::create_dir_all(parent).with_context(|| format!("mkdir {}", parent.display()))?;
} }
// UnsetEnvironment: the same headless-must-not-attach armor `launch_session` gives its
// transient unit — the manager env can carry a stale desktop DISPLAY/WAYLAND_DISPLAY (from a
// portal settle), and gamescope would abort trying to attach to it instead of becoming the
// display server. Unit-scoped belt-and-suspenders on top of the observe_session_instance scrub.
let body = format!( let body = format!(
"[Service]\n\ "[Service]\n\
Environment=PATH={shim}:/usr/bin:/bin:/usr/local/bin\n\ Environment=PATH={shim}:/usr/bin:/bin:/usr/local/bin\n\
Environment=PF_W={w}\n\ Environment=PF_W={w}\n\
Environment=PF_H={h}\n\ Environment=PF_H={h}\n\
Environment=PF_HZ={hz}\n", Environment=PF_HZ={hz}\n\
UnsetEnvironment=DISPLAY WAYLAND_DISPLAY\n",
shim = shim_dir.display(), shim = shim_dir.display(),
w = mode.width, w = mode.width,
h = mode.height, h = mode.height,
@@ -650,6 +678,16 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
} }
*guard = None; // tracked session lost its node — fall through to a clean restart *guard = None; // tracked session lost its node — fall through to a clean restart
} }
// Attach-only rebuild probe: the reuse path above may attach, but a restart of the session
// target is out of bounds — observed live on a Deck: a stale post-capture-loss detection made
// this restart steal the seat back from the KDE session the user had just switched to.
if crate::rebuild_probe_active() {
return Err(anyhow!(
"gamescope has no live node and this is an attach-only rebuild probe — refusing to \
restart {STEAMOS_SESSION_TARGET} (the box may be mid-switch to another session; \
re-detection follows it)"
));
}
let shim_dir = write_headless_shim()?; let shim_dir = write_headless_shim()?;
write_steamos_dropin(&shim_dir, mode)?; write_steamos_dropin(&shim_dir, mode)?;
systemctl_user(&["daemon-reload"]); systemctl_user(&["daemon-reload"]);
@@ -57,6 +57,13 @@ pub fn observe_session_instance(active: &ActiveSession) {
if let Some(old) = compositor_for_kind(prev.0) { if let Some(old) = compositor_for_kind(prev.0) {
registry::invalidate_backend(old.id()); registry::invalidate_backend(old.id());
} }
// The dead desktop's socket vars may still sit in the systemd --user manager env
// ([`settle_desktop_portal`]'s import-environment) — scrub them NOW, or the next
// `gamescope-session.target` start inherits a stale WAYLAND_DISPLAY and gamescope
// runs NESTED against the dead desktop socket instead of becoming the display
// server ("Failed to connect to wayland socket: wayland-0" — kept a Deck's Game
// Mode from starting at all, observed live 2026-07-21).
scrub_desktop_manager_env();
} }
let epoch = bump_session_epoch(); let epoch = bump_session_epoch();
tracing::info!( tracing::info!(
@@ -70,6 +77,23 @@ pub fn observe_session_instance(active: &ActiveSession) {
*last = Some(cur); *last = Some(cur);
} }
/// Counterpart to [`settle_desktop_portal`]'s `import-environment`: drop the desktop session's
/// socket vars from the systemd `--user` manager env once that desktop instance is GONE. They
/// persist in the manager otherwise, and every later user unit inherits them — including
/// `gamescope-session.target`, whose gamescope then aborts trying to attach to the dead desktop
/// socket. Best-effort; the D-Bus activation env has no unset op, but gamescope-session is
/// systemd-started, so the manager scrub is the one that matters. (A desktop restart re-imports
/// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.)
#[cfg(target_os = "linux")]
fn scrub_desktop_manager_env() {
let _ = std::process::Command::new("systemctl")
.args(["--user", "unset-environment", "WAYLAND_DISPLAY", "DISPLAY"])
.status();
}
#[cfg(not(target_os = "linux"))]
fn scrub_desktop_manager_env() {}
/// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die /// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die
/// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are /// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are
/// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]). /// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]).
+5
View File
@@ -33,6 +33,11 @@ reed-solomon-simd = "3.1" # GF(2^16) Leopard-RS, SIMD, O(n log n) — the w
# NOT interoperable.) See vendor/fec-rs/LICENSE (BSD-2-Clause). # NOT interoperable.) See vendor/fec-rs/LICENSE (BSD-2-Clause).
fec-rs = { path = "vendor/fec-rs" } fec-rs = { path = "vendor/fec-rs" }
aes-gcm = "0.10" # AES-128-GCM session crypto, matches GameStream aes-gcm = "0.10" # AES-128-GCM session crypto, matches GameStream
# ChaCha20-Poly1305 session crypto, negotiated by clients without hardware AES (the soft-AES
# armv7 targets — webOS TVs — where GCM caps decrypt at ~100 Mbps; ARX runs 4-7x faster there).
# Same RustCrypto `aead 0.5` generation as aes-gcm: identical trait/nonce/tag shapes, pure Rust,
# cross-compiles like aes-gcm (no cmake). See design/chacha20-session-cipher.md.
chacha20poly1305 = "0.10"
zerocopy = { version = "0.8", features = ["derive"] } zerocopy = { version = "0.8", features = ["derive"] }
bytes = "1" bytes = "1"
socket2 = { version = "0.6", features = [ socket2 = { version = "0.6", features = [
+47 -37
View File
@@ -1,7 +1,8 @@
//! Tier-1 microbenchmarks for the punktfunk/1 hot path — GPU-free, so they run in normal CI. //! Tier-1 microbenchmarks for the punktfunk/1 hot path — GPU-free, so they run in normal CI.
//! //!
//! Two layers: //! Two layers:
//! - `crypto/*` — the isolated AES-128-GCM primitives on one ~MTU shard. //! - `crypto/*` — the isolated AEAD primitives (AES-128-GCM + the negotiated
//! ChaCha20-Poly1305) on one ~MTU shard.
//! - `pipeline/*`— a whole frame through the real per-frame path end to end over the in-process //! - `pipeline/*`— a whole frame through the real per-frame path end to end over the in-process
//! loopback transport: FEC encode → AES-GCM seal → packetize → (loopback) → reassemble → //! loopback transport: FEC encode → AES-GCM seal → packetize → (loopback) → reassemble →
//! FEC decode → open. This is what a throughput/latency regression in the core would show up in. //! FEC decode → open. This is what a throughput/latency regression in the core would show up in.
@@ -11,11 +12,11 @@
use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput}; use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionCrypto; use punktfunk_core::crypto::{SessionCrypto, SessionKey};
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
use punktfunk_core::transport::loopback_pair; use punktfunk_core::transport::loopback_pair;
const TAG_LEN: usize = 16; // AES-GCM authentication tag const TAG_LEN: usize = 16; // AEAD authentication tag (GCM and Poly1305 share the size)
const SHARD: usize = punktfunk_core::config::mtu1500_shard_payload(); // one MTU-safe data shard const SHARD: usize = punktfunk_core::config::mtu1500_shard_payload(); // one MTU-safe data shard
fn cfg(role: Role, scheme: FecScheme) -> Config { fn cfg(role: Role, scheme: FecScheme) -> Config {
@@ -38,48 +39,57 @@ fn cfg(role: Role, scheme: FecScheme) -> Config {
shard_payload: SHARD, shard_payload: SHARD,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: true, // bench the real path — crypto is always on for punktfunk/1 encrypt: true, // bench the real path — crypto is always on for punktfunk/1
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4], salt: [1, 2, 3, 4],
loopback_drop_period: 0, // throughput run: no induced loss (loss-harness covers recovery) loopback_drop_period: 0, // throughput run: no induced loss (loss-harness covers recovery)
} }
} }
fn bench_crypto(c: &mut Criterion) { fn bench_crypto(c: &mut Criterion) {
let host = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Host);
let client = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Client);
let payload = vec![0xABu8; SHARD];
let sealed = host.seal(0, &payload).unwrap();
let mut g = c.benchmark_group("crypto"); let mut g = c.benchmark_group("crypto");
g.throughput(Throughput::Bytes(SHARD as u64)); g.throughput(Throughput::Bytes(SHARD as u64));
g.bench_function("seal", |b| { // Both negotiated session AEADs. On the x86 / Apple Silicon this runs on, both must be
let mut seq = 0u64; // line-rate-trivial — the chacha20 series is the host-side sealing-cost check for the
b.iter(|| { // negotiated soft-AES-armv7 path (design/chacha20-session-cipher.md §7). The AES series
let ct = host.seal(seq, black_box(&payload)).unwrap(); // keeps its unsuffixed names so the CI regression compare retains its history.
seq += 1; for (suffix, key) in [
black_box(ct) ("", SessionKey::Aes128Gcm([7u8; 16])),
}) ("_chacha20", SessionKey::ChaCha20Poly1305([7u8; 32])),
}); ] {
g.bench_function("seal_in_place", |b| { let host = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Host);
let mut seq = 0u64; let client = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Client);
let mut buf = vec![0xABu8; SHARD + TAG_LEN]; let payload = vec![0xABu8; SHARD];
b.iter(|| { let sealed = host.seal(0, &payload).unwrap();
host.seal_in_place(seq, black_box(&mut buf)).unwrap();
seq += 1; g.bench_function(format!("seal{suffix}"), |b| {
}) let mut seq = 0u64;
}); b.iter(|| {
g.bench_function("open", |b| { let ct = host.seal(seq, black_box(&payload)).unwrap();
b.iter(|| black_box(client.open(0, black_box(&sealed)).unwrap())) seq += 1;
}); black_box(ct)
g.bench_function("open_in_place", |b| { })
// In-place open consumes the buffer, so each iteration restores the ciphertext first — });
// one memcpy, mirroring what the recv ring does when the next datagram lands in the slot. g.bench_function(format!("seal_in_place{suffix}"), |b| {
let mut buf = sealed.clone(); let mut seq = 0u64;
b.iter(|| { let mut buf = vec![0xABu8; SHARD + TAG_LEN];
buf.copy_from_slice(black_box(&sealed)); b.iter(|| {
black_box(client.open_in_place(0, &mut buf).unwrap()); host.seal_in_place(seq, black_box(&mut buf)).unwrap();
}) seq += 1;
}); })
});
g.bench_function(format!("open{suffix}"), |b| {
b.iter(|| black_box(client.open(0, black_box(&sealed)).unwrap()))
});
g.bench_function(format!("open_in_place{suffix}"), |b| {
// In-place open consumes the buffer, so each iteration restores the ciphertext first —
// one memcpy, mirroring what the recv ring does when the next datagram lands in the slot.
let mut buf = sealed.clone();
b.iter(|| {
buf.copy_from_slice(black_box(&sealed));
black_box(client.open_in_place(0, &mut buf).unwrap());
})
});
}
g.finish(); g.finish();
} }
+9 -1
View File
@@ -11,6 +11,7 @@
//! - Panics never cross the boundary: every entry point is wrapped in `catch_unwind`. //! - Panics never cross the boundary: every entry point is wrapped in `catch_unwind`.
use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use crate::crypto::SessionKey;
use crate::error::PunktfunkStatus; use crate::error::PunktfunkStatus;
use crate::input::InputEvent; use crate::input::InputEvent;
use crate::reanchor::{GateVerdict, ReanchorGate}; use crate::reanchor::{GateVerdict, ReanchorGate};
@@ -94,7 +95,10 @@ impl PunktfunkConfig {
shard_payload: self.shard_payload as usize, shard_payload: self.shard_payload as usize,
max_frame_bytes, max_frame_bytes,
encrypt: self.encrypt != 0, encrypt: self.encrypt != 0,
key: self.key, // The C ABI keeps its fixed 16-byte key and always selects AES-128-GCM — no
// ABI_VERSION bump. Raw-`Config` C embedders can't negotiate ChaCha; the Swift/
// Kotlin clients are aarch64 with AES CE and never want it.
key: SessionKey::Aes128Gcm(self.key),
salt: self.salt, salt: self.salt,
loopback_drop_period: self.loopback_drop_period, loopback_drop_period: self.loopback_drop_period,
}; };
@@ -1570,6 +1574,10 @@ unsafe fn connect_ex_impl(
// themselves (EDR / MediaCodec), so the host's EDID defaults are fine there. An `ex8` // themselves (EDR / MediaCodec), so the host's EDID defaults are fine there. An `ex8`
// variant can carry it if a passthrough embedder ever needs it. // variant can carry it if a passthrough embedder ever needs it.
None, None,
// No client_caps in the C ABI yet either: cursor-channel opt-in for Apple/Android
// arrives with the ABI v11 cursor poll fns — until an embedder can RENDER the
// forwarded cursor it must not ask the host to stop compositing it.
0,
launch, launch,
pin, pin,
identity, identity,
+50 -2
View File
@@ -42,8 +42,8 @@ pub use self::rumble::{ActuatorQuirks, RumbleCommand};
use self::control::{CtrlRequest, Negotiated}; use self::control::{CtrlRequest, Negotiated};
use self::frame_channel::{DecodeLatAcc, FrameChannel, FramePop}; use self::frame_channel::{DecodeLatAcc, FrameChannel, FramePop};
use self::planes::{ use self::planes::{
RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE, RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, CURSOR_SHAPE_QUEUE, CURSOR_STATE_QUEUE,
RUMBLE_QUEUE, HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE, RUMBLE_QUEUE,
}; };
use self::probe::ProbeState; use self::probe::ProbeState;
use self::pump::run_pump; use self::pump::run_pump;
@@ -93,6 +93,12 @@ pub struct NativeClient {
/// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises /// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises
/// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any). /// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any).
host_timing: Mutex<Receiver<crate::quic::HostTiming>>, host_timing: Mutex<Receiver<crate::quic::HostTiming>>,
/// Inbound cursor shapes (control-stream [`crate::quic::CursorShape`]) — only a session
/// that advertised [`quic::CLIENT_CAP_CURSOR`] against a [`quic::HOST_CAP_CURSOR`] host
/// ever receives any.
cursor_shape: Mutex<Receiver<crate::quic::CursorShape>>,
/// Inbound per-frame cursor state — `0xD0` datagrams (same negotiation gate as shapes).
cursor_state: Mutex<Receiver<crate::quic::CursorState>>,
input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>, input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>,
/// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker. /// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker.
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing /// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
@@ -316,6 +322,12 @@ impl NativeClient {
// display's EDID so host apps tone-map to the client's real panel; `None` = unknown/SDR // display's EDID so host apps tone-map to the client's real panel; `None` = unknown/SDR
// (the host keeps its built-in EDID defaults). See [`crate::quic::Hello::display_hdr`]. // (the host keeps its built-in EDID defaults). See [`crate::quic::Hello::display_hdr`].
display_hdr: Option<HdrMeta>, display_hdr: Option<HdrMeta>,
// Non-video client capabilities ([`crate::quic::Hello::client_caps`]) — set
// [`crate::quic::CLIENT_CAP_CURSOR`] ONLY if this embedder actually renders the host
// cursor locally (shape + state planes): the host stops compositing the pointer into
// the video for a session that advertises it, so a non-rendering embedder that sets it
// streams with NO visible cursor at all. `0` = today's composited behavior.
client_caps: u8,
launch: Option<String>, launch: Option<String>,
pin: Option<[u8; 32]>, pin: Option<[u8; 32]>,
identity: Option<(String, String)>, identity: Option<(String, String)>,
@@ -337,6 +349,10 @@ impl NativeClient {
let (clip_event_tx, clip_event_rx) = let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE); std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>(); let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>();
let (cursor_shape_tx, cursor_shape_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorShape>(CURSOR_SHAPE_QUEUE);
let (cursor_state_tx, cursor_state_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorState>(CURSOR_STATE_QUEUE);
let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>(); let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>();
let shutdown = Arc::new(AtomicBool::new(false)); let shutdown = Arc::new(AtomicBool::new(false));
let quit = Arc::new(AtomicBool::new(false)); let quit = Arc::new(AtomicBool::new(false));
@@ -390,9 +406,11 @@ impl NativeClient {
video_codecs, video_codecs,
preferred_codec, preferred_codec,
display_hdr, display_hdr,
client_caps,
launch, launch,
pin, pin,
identity, identity,
connect_timeout: timeout,
frames: frame_chan_w, frames: frame_chan_w,
audio_tx, audio_tx,
rumble_tx, rumble_tx,
@@ -400,6 +418,8 @@ impl NativeClient {
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx, host_timing_tx,
cursor_shape_tx,
cursor_state_tx,
input_rx, input_rx,
mic_rx, mic_rx,
rich_input_rx, rich_input_rx,
@@ -444,6 +464,8 @@ impl NativeClient {
hidout: Mutex::new(hidout_rx), hidout: Mutex::new(hidout_rx),
hdr_meta: Mutex::new(hdr_meta_rx), hdr_meta: Mutex::new(hdr_meta_rx),
host_timing: Mutex::new(host_timing_rx), host_timing: Mutex::new(host_timing_rx),
cursor_shape: Mutex::new(cursor_shape_rx),
cursor_state: Mutex::new(cursor_state_rx),
input_tx, input_tx,
mic_tx, mic_tx,
rich_input_tx, rich_input_tx,
@@ -891,6 +913,32 @@ impl NativeClient {
} }
} }
/// Pull the next host cursor shape (design/remote-desktop-sweep.md M2): RGBA bitmap +
/// hotspot, sent on pointer-bitmap change over the reliable control stream. The embedder
/// caches by `serial` and builds an OS cursor from it; [`NativeClient::next_cursor_state`]
/// references shapes by serial. Only a session that advertised
/// [`crate::quic::CLIENT_CAP_CURSOR`] against a capable host receives any. Same
/// timeout/closed semantics as [`NativeClient::next_hidout`].
pub fn next_cursor_shape(&self, timeout: Duration) -> Result<crate::quic::CursorShape> {
match self.cursor_shape.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-frame cursor state (`0xD0`): position, visibility and the M3
/// relative-mode hint, referencing a shape by serial. Latest-wins — an embedder should
/// drain the queue and apply only the newest. Same negotiation gate and timeout/closed
/// semantics as [`NativeClient::next_cursor_shape`].
pub fn next_cursor_state(&self, timeout: Duration) -> Result<crate::quic::CursorState> {
match self.cursor_state.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access /// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access
/// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network` /// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network`
/// split (`network = (received + clock_offset pts) host_us`); a stats consumer should /// split (`network = (received + clock_offset pts) host_us`); a stats consumer should
@@ -35,6 +35,16 @@ pub(crate) const HOST_TIMING_QUEUE: usize = 512;
/// a dropped fetch-request makes the serving stream time out and reset cleanly. /// a dropped fetch-request makes the serving stream time out and reset cleanly.
pub(crate) const CLIP_EVENT_QUEUE: usize = 32; pub(crate) const CLIP_EVENT_QUEUE: usize = 32;
/// Cursor-shape plane depth (control-stream [`crate::quic::CursorShape`], one per pointer-bitmap
/// change — human-paced). Overflow drops the newest (try_send); the next shape change or a
/// serial mismatch against `0xD0` state heals it visually within a shape-change period.
pub(crate) const CURSOR_SHAPE_QUEUE: usize = 8;
/// Cursor-state plane depth (`0xD0`, one datagram per captured frame). Latest-wins state — the
/// embedder drains per present; a tiny ring only bridges scheduling jitter. Overflow drops the
/// newest (try_send), healed by the very next frame's datagram.
pub(crate) const CURSOR_STATE_QUEUE: usize = 8;
/// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames). /// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames).
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct AudioPacket { pub struct AudioPacket {
+4
View File
@@ -51,6 +51,8 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx, host_timing_tx,
cursor_shape_tx,
cursor_state_tx,
input_rx, input_rx,
mut mic_rx, mut mic_rx,
mut rich_input_rx, mut rich_input_rx,
@@ -123,6 +125,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
clock_offset: clock_offset.clone(), clock_offset: clock_offset.clone(),
clock_gen: clock_gen.clone(), clock_gen: clock_gen.clone(),
clip_event_tx: clip_event_tx.clone(), clip_event_tx: clip_event_tx.clone(),
cursor_shape_tx,
} }
.run(), .run(),
); );
@@ -136,6 +139,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx, host_timing_tx,
cursor_state_tx,
)); ));
// Clipboard task: the fetch-stream accept loop (host pulls what we offered) + outbound fetches // Clipboard task: the fetch-stream accept loop (host pulls what we offered) + outbound fetches
@@ -21,6 +21,9 @@ pub(super) struct ControlTask {
/// Clipboard metadata events (ClipState/ClipOffer) feed the same event plane the /// Clipboard metadata events (ClipState/ClipOffer) feed the same event plane the
/// clipboard task uses for fetch data. /// clipboard task uses for fetch data.
pub(super) clip_event_tx: std::sync::mpsc::SyncSender<ClipEventCore>, pub(super) clip_event_tx: std::sync::mpsc::SyncSender<ClipEventCore>,
/// Host cursor shapes ([`CursorShape`], sent on pointer-bitmap change) → the embedder's
/// shape plane ([`NativeClient::next_cursor_shape`]).
pub(super) cursor_shape_tx: std::sync::mpsc::SyncSender<crate::quic::CursorShape>,
} }
impl ControlTask { impl ControlTask {
@@ -36,6 +39,7 @@ impl ControlTask {
clock_offset, clock_offset,
clock_gen, clock_gen,
clip_event_tx, clip_event_tx,
cursor_shape_tx,
} = self; } = self;
// Mid-stream clock re-sync (see [`ClockResync`]): a batch runs every // Mid-stream clock re-sync (see [`ClockResync`]): a batch runs every
// CLOCK_RESYNC_INTERVAL and whenever the pump asks (CtrlRequest::ClockResync after // CLOCK_RESYNC_INTERVAL and whenever the pump asks (CtrlRequest::ClockResync after
@@ -167,6 +171,10 @@ impl ControlTask {
seq: offer.seq, seq: offer.seq,
kinds: offer.kinds, kinds: offer.kinds,
}); });
} else if let Ok(shape) = crate::quic::CursorShape::decode(&msg) {
// Pointer bitmap changed (cursor channel, only when negotiated). try_send:
// an overflowing ring drops the newest shape — the next change resends.
let _ = cursor_shape_tx.try_send(shape);
} else { } else {
tracing::warn!( tracing::warn!(
tag = ?msg.first(), tag = ?msg.first(),
@@ -3,6 +3,9 @@
use super::*; use super::*;
// One parameter per demuxed plane — grouping them into a struct would just move the field
// list one hop away from the single call site.
#[allow(clippy::too_many_arguments)]
pub(super) async fn run( pub(super) async fn run(
conn: quinn::Connection, conn: quinn::Connection,
audio_tx: std::sync::mpsc::SyncSender<AudioPacket>, audio_tx: std::sync::mpsc::SyncSender<AudioPacket>,
@@ -11,6 +14,7 @@ pub(super) async fn run(
hidout_tx: std::sync::mpsc::SyncSender<crate::quic::HidOutput>, hidout_tx: std::sync::mpsc::SyncSender<crate::quic::HidOutput>,
hdr_meta_tx: std::sync::mpsc::SyncSender<crate::quic::HdrMeta>, hdr_meta_tx: std::sync::mpsc::SyncSender<crate::quic::HdrMeta>,
host_timing_tx: std::sync::mpsc::SyncSender<crate::quic::HostTiming>, host_timing_tx: std::sync::mpsc::SyncSender<crate::quic::HostTiming>,
cursor_state_tx: std::sync::mpsc::SyncSender<crate::quic::CursorState>,
) { ) {
// Per-pad reorder gate for v2 rumble envelopes (the seq analog of the host's gamepad-state // Per-pad reorder gate for v2 rumble envelopes (the seq analog of the host's gamepad-state
// gate): a datagram the network reordered must not roll a stopped motor back on. Legacy v1 // gate): a datagram the network reordered must not roll a stopped motor back on. Legacy v1
@@ -73,6 +77,11 @@ pub(super) async fn run(
let _ = host_timing_tx.try_send(t); let _ = host_timing_tx.try_send(t);
} }
} }
Some(&crate::quic::CURSOR_STATE_MAGIC) => {
if let Some(s) = crate::quic::decode_cursor_state_datagram(&d) {
let _ = cursor_state_tx.try_send(s);
}
}
_ => {} // unknown tag — a newer host; ignore _ => {} // unknown tag — a newer host; ignore
} }
} }
@@ -32,21 +32,65 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
identity.as_ref().map(|(c, k)| (c.as_str(), k.as_str())), identity.as_ref().map(|(c, k)| (c.as_str(), k.as_str())),
); );
let ep = ep.map_err(|e| PunktfunkError::Io(std::io::Error::other(e.to_string())))?; let ep = ep.map_err(|e| PunktfunkError::Io(std::io::Error::other(e.to_string())))?;
let conn = ep // Dial with retry across the connect budget, not a single attempt: one quinn dial gives
.connect(remote, "punktfunk") // up after the transport's idle window (~8 s of silence), which is shorter than a
.map_err(|_| PunktfunkError::InvalidArg("connect"))? // suspend-to-RAM resume — the Steam Deck flow fires Wake-on-LAN and connects
.await // immediately, so the host is still waking while the first Initials go out, and a
.map_err(|e| { // single-shot dial died just before the host came up. Short attempts keep the Initial
// A pin mismatch surfaces as a TLS failure; report it as a crypto error so // cadence dense (quinn's per-attempt retransmits back off toward multi-second gaps), so
// the embedder can distinguish "wrong host identity" from plain IO trouble. // the connect lands within ~a second of the host's network returning. Only SILENCE is
let fp_mismatch = // retried: a host that answers and rejects us (pin mismatch, ALPN/version, typed close)
pin.is_some() && observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true); // must surface immediately, and the embedder's shutdown flag (budget expiry in
if fp_mismatch { // `connect`, or a user cancel) stops the loop between attempts.
PunktfunkError::Crypto const DIAL_ATTEMPT: std::time::Duration = std::time::Duration::from_secs(3);
} else { // Redial headroom: leave room for the control handshake (Hello/Welcome/clock sync)
PunktfunkError::Io(std::io::Error::other(e.to_string())) // after a late dial success, so it still completes inside the embedder's budget.
const CONTROL_HEADROOM: std::time::Duration = std::time::Duration::from_secs(2);
let start = tokio::time::Instant::now();
let deadline = start + args.connect_timeout;
let redial_until = start + args.connect_timeout.saturating_sub(CONTROL_HEADROOM);
let conn = loop {
let connecting = ep
.connect(remote, "punktfunk")
.map_err(|_| PunktfunkError::InvalidArg("connect"))?;
// Cap the attempt to the remaining budget so a success never lands after the
// embedder's `ready_rx` wait has already given up and flagged a teardown.
let now = tokio::time::Instant::now();
let attempt = DIAL_ATTEMPT.min(deadline.saturating_duration_since(now));
let gave_up = || {
tokio::time::Instant::now() >= redial_until
|| shutdown.load(std::sync::atomic::Ordering::SeqCst)
};
match tokio::time::timeout(attempt, connecting).await {
Ok(Ok(conn)) => break conn,
Ok(Err(e)) => {
// A pin mismatch surfaces as a TLS failure; report it as a crypto error so
// the embedder can distinguish "wrong host identity" from plain IO trouble.
let fp_mismatch = pin.is_some()
&& observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true);
if fp_mismatch {
return Err(PunktfunkError::Crypto);
}
// The transport's own idle expiry — the host never answered — is the one
// retryable outcome; everything else is a real answer or a local failure.
let host_silent = matches!(e, quinn::ConnectionError::TimedOut);
if !host_silent {
return Err(PunktfunkError::Io(std::io::Error::other(e.to_string())));
}
if gave_up() {
return Err(PunktfunkError::Timeout);
}
} }
})?; // Attempt window elapsed with the host still silent; dropping `connecting`
// abandoned that dial — go again unless the budget is spent.
Err(_) => {
if gave_up() {
return Err(PunktfunkError::Timeout);
}
}
}
tracing::debug!(%remote, "host silent — re-dialing (wake/resume tolerant connect)");
};
let fingerprint = observed.lock().unwrap().unwrap_or([0u8; 32]); let fingerprint = observed.lock().unwrap().unwrap_or([0u8; 32]);
// The rest of the handshake runs in an inner future so a failure can consult // The rest of the handshake runs in an inner future so a failure can consult
// `conn.close_reason()`: a host that turned us away with a typed application close // `conn.close_reason()`: a host that turned us away with a typed application close
@@ -99,6 +143,10 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
// The client display's HDR volume → the host's virtual-display EDID (host apps // The client display's HDR volume → the host's virtual-display EDID (host apps
// tone-map to the client's real panel). `None` = unknown/SDR. // tone-map to the client's real panel). `None` = unknown/SDR.
display_hdr, display_hdr,
// NOT unconditional like HOST_TIMING above: CLIENT_CAP_CURSOR makes the host
// stop compositing the pointer, so only an embedder that actually renders the
// cursor locally may set it (the embedder decides, we pass through).
client_caps: args.client_caps,
} }
.encode(), .encode(),
) )
@@ -22,9 +22,14 @@ pub(crate) struct WorkerArgs {
pub(crate) video_codecs: u8, pub(crate) video_codecs: u8,
pub(crate) preferred_codec: u8, pub(crate) preferred_codec: u8,
pub(crate) display_hdr: Option<HdrMeta>, pub(crate) display_hdr: Option<HdrMeta>,
pub(crate) client_caps: u8,
pub(crate) launch: Option<String>, pub(crate) launch: Option<String>,
pub(crate) pin: Option<[u8; 32]>, pub(crate) pin: Option<[u8; 32]>,
pub(crate) identity: Option<(String, String)>, pub(crate) identity: Option<(String, String)>,
/// The embedder's connect budget (the same value `connect` bounds `ready_rx` with): the
/// dial loop re-dials a silent host within it, so a host still resuming from Wake-on-LAN
/// is caught the moment its network comes back instead of failing on the first attempt.
pub(crate) connect_timeout: std::time::Duration,
pub(crate) frames: Arc<FrameChannel>, pub(crate) frames: Arc<FrameChannel>,
pub(crate) audio_tx: SyncSender<AudioPacket>, pub(crate) audio_tx: SyncSender<AudioPacket>,
pub(crate) rumble_tx: SyncSender<RumbleUpdate>, pub(crate) rumble_tx: SyncSender<RumbleUpdate>,
@@ -34,6 +39,8 @@ pub(crate) struct WorkerArgs {
pub(crate) hidout_tx: SyncSender<HidOutput>, pub(crate) hidout_tx: SyncSender<HidOutput>,
pub(crate) hdr_meta_tx: SyncSender<HdrMeta>, pub(crate) hdr_meta_tx: SyncSender<HdrMeta>,
pub(crate) host_timing_tx: SyncSender<crate::quic::HostTiming>, pub(crate) host_timing_tx: SyncSender<crate::quic::HostTiming>,
pub(crate) cursor_shape_tx: SyncSender<crate::quic::CursorShape>,
pub(crate) cursor_state_tx: SyncSender<crate::quic::CursorState>,
pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>, pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>, pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>,
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>, pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
+15 -6
View File
@@ -1,5 +1,6 @@
//! Session configuration and protocol/FEC parameters. //! Session configuration and protocol/FEC parameters.
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
use crate::packet::{CRYPTO_OVERHEAD, HEADER_LEN, MAX_DATAGRAM_BYTES}; use crate::packet::{CRYPTO_OVERHEAD, HEADER_LEN, MAX_DATAGRAM_BYTES};
use zeroize::Zeroize; use zeroize::Zeroize;
@@ -355,9 +356,11 @@ pub struct Config {
/// hostile/corrupt headers; see [`Session`](crate::session::Session)). /// hostile/corrupt headers; see [`Session`](crate::session::Session)).
pub max_frame_bytes: usize, pub max_frame_bytes: usize,
pub encrypt: bool, pub encrypt: bool,
/// AES-128 session key established during pairing. MUST be unique per session when /// The negotiated session AEAD + its key, established during pairing/handshake —
/// AES-128-GCM for every peer by default, ChaCha20-Poly1305 when the client negotiated it
/// (soft-AES armv7 targets; see [`SessionKey`]). MUST be unique per session when
/// `encrypt` is set (see the nonce-uniqueness contract in [`crate::crypto`]). /// `encrypt` is set (see the nonce-uniqueness contract in [`crate::crypto`]).
pub key: [u8; 16], pub key: SessionKey,
/// Per-session nonce salt, established alongside `key` during pairing. MUST be /// Per-session nonce salt, established alongside `key` during pairing. MUST be
/// unique per (key, session). /// unique per (key, session).
pub salt: [u8; 4], pub salt: [u8; 4],
@@ -382,7 +385,8 @@ impl std::fmt::Debug for Config {
.field("shard_payload", &self.shard_payload) .field("shard_payload", &self.shard_payload)
.field("max_frame_bytes", &self.max_frame_bytes) .field("max_frame_bytes", &self.max_frame_bytes)
.field("encrypt", &self.encrypt) .field("encrypt", &self.encrypt)
.field("key", &"<redacted>") // SessionKey's own Debug redacts the material but keeps the cipher choice visible.
.field("key", &self.key)
.field("salt", &"<redacted>") .field("salt", &"<redacted>")
.field("loopback_drop_period", &self.loopback_drop_period) .field("loopback_drop_period", &self.loopback_drop_period)
.finish() .finish()
@@ -426,7 +430,7 @@ impl Config {
"max_frame_bytes too large for this shard/block configuration (block count overflows u16)", "max_frame_bytes too large for this shard/block configuration (block count overflows u16)",
)); ));
} }
if self.encrypt && self.key == [0u8; 16] { if self.encrypt && self.key.is_zero() {
return Err(PunktfunkError::InvalidArg( return Err(PunktfunkError::InvalidArg(
"encrypt requires a non-zero session key (see crypto nonce-uniqueness contract)", "encrypt requires a non-zero session key (see crypto nonce-uniqueness contract)",
)); ));
@@ -449,7 +453,7 @@ impl Config {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 64 * 1024 * 1024, max_frame_bytes: 64 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
@@ -465,7 +469,12 @@ mod tests {
let mut c = Config::p1_defaults(Role::Host); let mut c = Config::p1_defaults(Role::Host);
c.encrypt = true; // key is still all-zero c.encrypt = true; // key is still all-zero
assert!(c.validate().is_err()); assert!(c.validate().is_err());
c.key = [1u8; 16]; c.key = SessionKey::Aes128Gcm([1u8; 16]);
assert!(c.validate().is_ok());
// The rejection follows whichever cipher variant is active.
c.key = SessionKey::ChaCha20Poly1305([0u8; 32]);
assert!(c.validate().is_err());
c.key = SessionKey::ChaCha20Poly1305([1u8; 32]);
assert!(c.validate().is_ok()); assert!(c.validate().is_ok());
} }
+266 -106
View File
@@ -1,9 +1,10 @@
//! AES-128-GCM session sealing, matching GameStream's video crypto in P1. //! Session sealing with the negotiated AEAD — AES-128-GCM (matching GameStream's video
//! crypto in P1) by default, ChaCha20-Poly1305 (RFC 8439) for clients without hardware AES.
//! //!
//! ## Nonce uniqueness (the GCM safety requirement) //! ## Nonce uniqueness (the AEAD safety requirement)
//! //!
//! The 96-bit nonce is `salt (4 bytes) || sequence (8 bytes, big-endian)`. Reusing a //! The 96-bit nonce is `salt (4 bytes) || sequence (8 bytes, big-endian)`. Reusing a
//! `(key, nonce)` pair under AES-GCM is catastrophic, so two precautions apply: //! `(key, nonce)` pair is catastrophic under either AEAD, so two precautions apply:
//! //!
//! 1. **Per-direction salts.** Host and client share one `key` and `salt`, and each //! 1. **Per-direction salts.** Host and client share one `key` and `salt`, and each
//! counts its sequence from 0. To stop the host's video stream and the client's input //! counts its sequence from 0. To stop the host's video stream and the client's input
@@ -17,17 +18,96 @@
//! The sequence number is also passed as AEAD associated data, so tampering with the //! The sequence number is also passed as AEAD associated data, so tampering with the
//! on-wire sequence is detected (the tag check fails) rather than silently shifting the //! on-wire sequence is detected (the tag check fails) rather than silently shifting the
//! nonce. Note: this layer does not provide anti-replay — see `Session`. //! nonce. Note: this layer does not provide anti-replay — see `Session`.
//!
//! ## Why two ciphers
//!
//! Both AEADs are full-strength; the choice (negotiated via `Welcome::cipher`) is purely a
//! performance one. On targets without hardware AES — the soft-AES armv7 clients (webOS TVs) —
//! GCM's fixsliced AES + software GHASH costs ~50100 cycles/byte and caps decrypt at
//! ~100 Mbps, while ChaCha20-Poly1305's ARX construction runs ~1017 cycles/byte in portable
//! software (design/chacha20-session-cipher.md). Same 96-bit nonce, 16-byte tag, and AAD
//! shape, so the entire nonce discipline above carries over verbatim.
use crate::config::Role; use crate::config::Role;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
use aes_gcm::aead::{Aead, AeadInPlace, KeyInit, Payload}; use aes_gcm::aead::{Aead, AeadInPlace, KeyInit, Payload};
use aes_gcm::{Aes128Gcm, Key, Nonce}; use aes_gcm::{Aes128Gcm, Key, Nonce};
use chacha20poly1305::ChaCha20Poly1305;
use zeroize::Zeroize;
/// 16-byte AEAD authentication tag appended by GCM. /// 16-byte AEAD authentication tag appended by either session cipher.
pub const TAG_LEN: usize = 16; pub const TAG_LEN: usize = 16;
// The wire (CRYPTO_OVERHEAD) and every in-place split assume both negotiated AEADs append
// exactly TAG_LEN bytes — a different-tag cipher can never slip in behind this constant.
const _: () = assert!(std::mem::size_of::<aes_gcm::Tag>() == TAG_LEN);
const _: () = assert!(std::mem::size_of::<chacha20poly1305::Tag>() == TAG_LEN);
/// The negotiated session AEAD together with its key material — merged so the invalid state
/// (a ChaCha cipher with an AES-sized key, or vice versa) is unrepresentable. AES-128-GCM is
/// the default every peer speaks; ChaCha20-Poly1305 is granted to clients that advertised
/// [`VIDEO_CAP_CHACHA20`](crate::quic::VIDEO_CAP_CHACHA20) (the soft-AES armv7 targets —
/// see the module docs). 256 bits for ChaCha is what RFC 8439 requires.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum SessionKey {
Aes128Gcm([u8; 16]),
ChaCha20Poly1305([u8; 32]),
}
impl SessionKey {
/// Canonical lowercase cipher name for session-start logs.
pub fn cipher_name(&self) -> &'static str {
match self {
SessionKey::Aes128Gcm(_) => "aes-128-gcm",
SessionKey::ChaCha20Poly1305(_) => "chacha20-poly1305",
}
}
/// True when the key material is all zeros — the pairing-layer footgun `Config::validate`
/// rejects when encryption is on (see the nonce-uniqueness contract in the module docs).
pub fn is_zero(&self) -> bool {
match self {
SessionKey::Aes128Gcm(k) => k == &[0u8; 16],
SessionKey::ChaCha20Poly1305(k) => k == &[0u8; 32],
}
}
}
/// Key material never appears in logs, whichever variant is active — only the cipher choice
/// (`Config`'s hand-written `Debug` relies on this).
impl std::fmt::Debug for SessionKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SessionKey::Aes128Gcm(_) => f.write_str("Aes128Gcm(<redacted>)"),
SessionKey::ChaCha20Poly1305(_) => f.write_str("ChaCha20Poly1305(<redacted>)"),
}
}
}
/// Same zeroize-on-drop discipline the raw key array had (`Config`'s `Drop`).
impl Zeroize for SessionKey {
fn zeroize(&mut self) {
match self {
SessionKey::Aes128Gcm(k) => k.zeroize(),
SessionKey::ChaCha20Poly1305(k) => k.zeroize(),
}
}
}
/// The two negotiated AEADs behind one seal/open surface. Both are the same RustCrypto
/// `aead 0.5` generation (identical trait shapes, nonce/tag types), so each call below is a
/// two-arm match right next to the cipher work itself.
// AES's precomputed round keys (~0.7 KB) dwarf ChaCha's 32-byte state, but there is exactly
// one long-lived `SessionCrypto` per session — boxing the variant would trade that one-off
// slack for a pointer chase on every per-datagram seal/open.
#[allow(clippy::large_enum_variant)]
enum Cipher {
Aes128Gcm(Aes128Gcm),
ChaCha20Poly1305(ChaCha20Poly1305),
}
pub struct SessionCrypto { pub struct SessionCrypto {
cipher: Aes128Gcm, cipher: Cipher,
/// Salt for nonces we seal with (our direction). /// Salt for nonces we seal with (our direction).
send_salt: [u8; 4], send_salt: [u8; 4],
/// Salt for nonces we open with (the peer's direction). /// Salt for nonces we open with (the peer's direction).
@@ -35,11 +115,18 @@ pub struct SessionCrypto {
} }
impl SessionCrypto { impl SessionCrypto {
pub fn new(key: &[u8; 16], salt: [u8; 4], role: Role) -> Self { pub fn new(key: &SessionKey, salt: [u8; 4], role: Role) -> Self {
let key = Key::<Aes128Gcm>::from_slice(key); let cipher = match key {
SessionKey::Aes128Gcm(k) => {
Cipher::Aes128Gcm(Aes128Gcm::new(Key::<Aes128Gcm>::from_slice(k)))
}
SessionKey::ChaCha20Poly1305(k) => Cipher::ChaCha20Poly1305(ChaCha20Poly1305::new(
Key::<ChaCha20Poly1305>::from_slice(k),
)),
};
let own = direction(role); let own = direction(role);
SessionCrypto { SessionCrypto {
cipher: Aes128Gcm::new(key), cipher,
send_salt: dir_salt(salt, own), send_salt: dir_salt(salt, own),
recv_salt: dir_salt(salt, own ^ 1), recv_salt: dir_salt(salt, own ^ 1),
} }
@@ -49,15 +136,16 @@ impl SessionCrypto {
/// authenticated as associated data. /// authenticated as associated data.
pub fn seal(&self, seq: u64, plaintext: &[u8]) -> Result<Vec<u8>> { pub fn seal(&self, seq: u64, plaintext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.send_salt, seq); let nonce = nonce(self.send_salt, seq);
self.cipher let aad = seq.to_be_bytes();
.encrypt( let payload = Payload {
Nonce::from_slice(&nonce), msg: plaintext,
Payload { aad: &aad,
msg: plaintext, };
aad: &seq.to_be_bytes(), match &self.cipher {
}, Cipher::Aes128Gcm(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
) Cipher::ChaCha20Poly1305(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
.map_err(|_| PunktfunkError::Crypto) }
.map_err(|_| PunktfunkError::Crypto)
} }
/// Seal in place, no per-packet allocation: `buf` is laid out as `[plaintext .. ][TAG_LEN]` (the /// Seal in place, no per-packet allocation: `buf` is laid out as `[plaintext .. ][TAG_LEN]` (the
@@ -69,10 +157,16 @@ impl SessionCrypto {
let nonce = nonce(self.send_salt, seq); let nonce = nonce(self.send_salt, seq);
let split = buf.len() - TAG_LEN; let split = buf.len() - TAG_LEN;
let (plaintext, tag_slot) = buf.split_at_mut(split); let (plaintext, tag_slot) = buf.split_at_mut(split);
let tag = self let aad = seq.to_be_bytes();
.cipher let tag = match &self.cipher {
.encrypt_in_place_detached(Nonce::from_slice(&nonce), &seq.to_be_bytes(), plaintext) Cipher::Aes128Gcm(c) => {
.map_err(|_| PunktfunkError::Crypto)?; c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
Cipher::ChaCha20Poly1305(c) => {
c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
}
.map_err(|_| PunktfunkError::Crypto)?;
tag_slot.copy_from_slice(&tag); tag_slot.copy_from_slice(&tag);
Ok(()) Ok(())
} }
@@ -80,20 +174,21 @@ impl SessionCrypto {
/// Open `ciphertext || tag` for sequence `seq` (also bound as associated data). /// Open `ciphertext || tag` for sequence `seq` (also bound as associated data).
pub fn open(&self, seq: u64, ciphertext: &[u8]) -> Result<Vec<u8>> { pub fn open(&self, seq: u64, ciphertext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.recv_salt, seq); let nonce = nonce(self.recv_salt, seq);
self.cipher let aad = seq.to_be_bytes();
.decrypt( let payload = Payload {
Nonce::from_slice(&nonce), msg: ciphertext,
Payload { aad: &aad,
msg: ciphertext, };
aad: &seq.to_be_bytes(), match &self.cipher {
}, Cipher::Aes128Gcm(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
) Cipher::ChaCha20Poly1305(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
.map_err(|_| PunktfunkError::Crypto) }
.map_err(|_| PunktfunkError::Crypto)
} }
/// Open in place, no per-packet allocation: `buf` holds `[ciphertext .. ][tag]` on entry and /// Open in place, no per-packet allocation: `buf` holds `[ciphertext .. ][tag]` on entry and
/// the plaintext in its first `buf.len() - TAG_LEN` bytes on success (returned as the length) /// the plaintext in its first `buf.len() - TAG_LEN` bytes on success (returned as the length)
/// — byte-identical to `open`, just written in place. GCM verifies the tag *before* /// — byte-identical to `open`, just written in place. Both AEADs verify the tag *before*
/// decrypting, so on failure `buf` still holds the ciphertext (the caller drops the packet /// decrypting, so on failure `buf` still holds the ciphertext (the caller drops the packet
/// either way). The hot-path receiver (`Session::poll_frame`) uses this to avoid the `Vec` /// either way). The hot-path receiver (`Session::poll_frame`) uses this to avoid the `Vec`
/// that `open`'s convenience API allocates for every datagram at line rate — the receive /// that `open`'s convenience API allocates for every datagram at line rate — the receive
@@ -105,14 +200,22 @@ impl SessionCrypto {
let nonce = nonce(self.recv_salt, seq); let nonce = nonce(self.recv_salt, seq);
let split = buf.len() - TAG_LEN; let split = buf.len() - TAG_LEN;
let (ciphertext, tag) = buf.split_at_mut(split); let (ciphertext, tag) = buf.split_at_mut(split);
self.cipher let aad = seq.to_be_bytes();
.decrypt_in_place_detached( match &self.cipher {
Cipher::Aes128Gcm(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce), Nonce::from_slice(&nonce),
&seq.to_be_bytes(), &aad,
ciphertext, ciphertext,
aes_gcm::Tag::from_slice(tag), aes_gcm::Tag::from_slice(tag),
) ),
.map_err(|_| PunktfunkError::Crypto)?; Cipher::ChaCha20Poly1305(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce),
&aad,
ciphertext,
chacha20poly1305::Tag::from_slice(tag),
),
}
.map_err(|_| PunktfunkError::Crypto)?;
Ok(split) Ok(split)
} }
} }
@@ -145,6 +248,13 @@ pub fn random_key() -> [u8; 16] {
k k
} }
/// Generate a fresh random ChaCha20-Poly1305 session key (RFC 8439's 256-bit size).
pub fn random_key32() -> [u8; 32] {
let mut k = [0u8; 32];
rand::RngCore::fill_bytes(&mut rand::rng(), &mut k);
k
}
/// Generate a fresh random per-session nonce salt. /// Generate a fresh random per-session nonce salt.
pub fn random_salt() -> [u8; 4] { pub fn random_salt() -> [u8; 4] {
let mut s = [0u8; 4]; let mut s = [0u8; 4];
@@ -156,93 +266,143 @@ pub fn random_salt() -> [u8; 4] {
mod tests { mod tests {
use super::*; use super::*;
/// One fresh key per negotiated cipher — every sealing test below must hold for both.
fn both_keys() -> [SessionKey; 2] {
[
SessionKey::Aes128Gcm(random_key()),
SessionKey::ChaCha20Poly1305(random_key32()),
]
}
#[test] #[test]
fn seal_open_roundtrip_cross_direction() { fn seal_open_roundtrip_cross_direction() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
let msg = b"the quick brown fox"; let msg = b"the quick brown fox";
let sealed = host.seal(42, msg).unwrap(); // host -> client (video direction) let sealed = host.seal(42, msg).unwrap(); // host -> client (video direction)
assert_ne!(&sealed[..msg.len()], &msg[..]); // actually encrypted assert_ne!(&sealed[..msg.len()], &msg[..]); // actually encrypted
assert_eq!(sealed.len(), msg.len() + TAG_LEN); assert_eq!(sealed.len(), msg.len() + TAG_LEN);
assert_eq!(client.open(42, &sealed).unwrap(), msg); assert_eq!(client.open(42, &sealed).unwrap(), msg);
// Wrong sequence (nonce + AAD) → authentication failure. // Wrong sequence (nonce + AAD) → authentication failure.
assert!(client.open(43, &sealed).is_err()); assert!(client.open(43, &sealed).is_err());
// Direction separation: the host opens with the peer (client) salt, so it cannot // Direction separation: the host opens with the peer (client) salt, so it cannot
// open its own outbound packet → distinct nonce spaces per direction. // open its own outbound packet → distinct nonce spaces per direction.
assert!(host.open(42, &sealed).is_err()); assert!(host.open(42, &sealed).is_err());
}
} }
#[test] #[test]
fn directions_use_distinct_nonce_spaces() { fn directions_use_distinct_nonce_spaces() {
let key = random_key(); for key in both_keys() {
let salt = [0u8; 4]; // even an all-zero base salt must separate the directions let salt = [0u8; 4]; // even an all-zero base salt must separate the directions
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
// Same seq, same key, opposite directions → different ciphertext (no reuse). // Same seq, same key, opposite directions → different ciphertext (no reuse).
assert_ne!( assert_ne!(
host.seal(0, b"abc").unwrap(), host.seal(0, b"abc").unwrap(),
client.seal(0, b"abc").unwrap() client.seal(0, b"abc").unwrap()
); );
}
} }
#[test] #[test]
fn open_in_place_matches_open_and_rejects_tampering() { fn open_in_place_matches_open_and_rejects_tampering() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
for msg in [ for msg in [
&b""[..], &b""[..],
b"x", b"x",
b"the quick brown fox jumps over 13 lazy dogs!!", b"the quick brown fox jumps over 13 lazy dogs!!",
] { ] {
let sealed = host.seal(9, msg).unwrap(); let sealed = host.seal(9, msg).unwrap();
let mut buf = sealed.clone(); let mut buf = sealed.clone();
let n = client.open_in_place(9, &mut buf).unwrap(); let n = client.open_in_place(9, &mut buf).unwrap();
assert_eq!( assert_eq!(
&buf[..n], &buf[..n],
msg, msg,
"in-place open must be byte-identical to open" "in-place open must be byte-identical to open"
); );
// Wrong sequence (nonce + AAD) → authentication failure, like `open`. // Wrong sequence (nonce + AAD) → authentication failure, like `open`.
let mut buf = sealed.clone(); let mut buf = sealed.clone();
assert!(client.open_in_place(8, &mut buf).is_err()); assert!(client.open_in_place(8, &mut buf).is_err());
// A flipped ciphertext/tag bit → authentication failure. // A flipped ciphertext/tag bit → authentication failure.
let mut buf = sealed.clone(); let mut buf = sealed.clone();
let last = buf.len() - 1; let last = buf.len() - 1;
buf[last] ^= 1; buf[last] ^= 1;
assert!(client.open_in_place(9, &mut buf).is_err()); assert!(client.open_in_place(9, &mut buf).is_err());
}
// Shorter than a tag can't be a sealed packet at all.
let mut runt = vec![0u8; TAG_LEN - 1];
assert!(client.open_in_place(0, &mut runt).is_err());
} }
// Shorter than a tag can't be a sealed packet at all.
let mut runt = vec![0u8; TAG_LEN - 1];
assert!(client.open_in_place(0, &mut runt).is_err());
} }
#[test] #[test]
fn seal_in_place_matches_seal_and_opens() { fn seal_in_place_matches_seal_and_opens() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
for msg in [ for msg in [
&b""[..], &b""[..],
b"x", b"x",
b"the quick brown fox jumps over 13 lazy dogs!!", b"the quick brown fox jumps over 13 lazy dogs!!",
] { ] {
let reference = host.seal(7, msg).unwrap(); // ciphertext || tag let reference = host.seal(7, msg).unwrap(); // ciphertext || tag
// In-place: [plaintext .. ][TAG_LEN scratch]. // In-place: [plaintext .. ][TAG_LEN scratch].
let mut buf = msg.to_vec(); let mut buf = msg.to_vec();
buf.resize(msg.len() + TAG_LEN, 0); buf.resize(msg.len() + TAG_LEN, 0);
host.seal_in_place(7, &mut buf).unwrap(); host.seal_in_place(7, &mut buf).unwrap();
assert_eq!( assert_eq!(
buf, reference, buf, reference,
"in-place seal must be byte-identical to seal" "in-place seal must be byte-identical to seal"
); );
assert_eq!(client.open(7, &buf).unwrap(), msg); assert_eq!(client.open(7, &buf).unwrap(), msg);
}
} }
} }
#[test]
fn ciphers_are_not_interchangeable() {
// A packet sealed under one AEAD must not open under the other — negotiation skew has
// to fail loudly (a tag mismatch), never decode garbage. The ChaCha key repeats the AES
// key bytes so even overlapping key material can't accidentally interoperate.
let salt = random_salt();
let aes = SessionKey::Aes128Gcm([7u8; 16]);
let chacha = SessionKey::ChaCha20Poly1305([7u8; 32]);
let sealed = SessionCrypto::new(&aes, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&chacha, salt, Role::Client)
.open(1, &sealed)
.is_err());
let sealed = SessionCrypto::new(&chacha, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&aes, salt, Role::Client)
.open(1, &sealed)
.is_err());
}
#[test]
fn session_key_zero_check_and_debug_redaction() {
assert!(SessionKey::Aes128Gcm([0u8; 16]).is_zero());
assert!(SessionKey::ChaCha20Poly1305([0u8; 32]).is_zero());
assert!(!SessionKey::Aes128Gcm([1u8; 16]).is_zero());
assert!(!SessionKey::ChaCha20Poly1305([1u8; 32]).is_zero());
// Key bytes must never reach a log, whichever variant — only the cipher choice.
for key in both_keys() {
let dbg = format!("{key:?}");
assert!(dbg.contains("<redacted>"), "{dbg}");
}
let mut k = SessionKey::ChaCha20Poly1305([9u8; 32]);
k.zeroize();
assert!(k.is_zero());
}
} }
+3 -2
View File
@@ -1,6 +1,7 @@
use super::reassemble::LOSS_WINDOW_NS; use super::reassemble::LOSS_WINDOW_NS;
use super::*; use super::*;
use crate::config::{Config, FecScheme}; use crate::config::{Config, FecScheme};
use crate::crypto::SessionKey;
use crate::fec::coder_for; use crate::fec::coder_for;
use crate::stats::StatsCounters; use crate::stats::StatsCounters;
use zerocopy::{FromBytes, IntoBytes}; use zerocopy::{FromBytes, IntoBytes};
@@ -182,7 +183,7 @@ fn explicit_frame_index_is_stamped_and_internal_counter_untouched() {
shard_payload: 16, shard_payload: 16,
max_frame_bytes: 4096, max_frame_bytes: 4096,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
}; };
@@ -292,7 +293,7 @@ fn e2e_config(scheme: FecScheme, fec_percent: u8) -> Config {
shard_payload: 16, shard_payload: 16,
max_frame_bytes: 4096, max_frame_bytes: 4096,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
+31
View File
@@ -44,6 +44,17 @@ pub const VIDEO_CAP_PROBE_SEQ: u8 = 0x10;
/// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so /// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so
/// the fallback is zero-risk. /// the fallback is zero-risk.
pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20; pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20;
/// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
/// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
/// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
/// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
/// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
/// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
/// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
/// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
/// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
/// control channel, so there is no downgrade surface.
pub const VIDEO_CAP_CHACHA20: u8 = 0x40;
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] /// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder /// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -60,6 +71,24 @@ pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// trailing `host_caps` byte — no wire-layout change. /// trailing `host_caps` byte — no wire-layout change.
pub const HOST_CAP_CLIPBOARD: u8 = 0x02; pub const HOST_CAP_CLIPBOARD: u8 = 0x02;
/// [`Hello::client_caps`] bit: the client renders the host cursor LOCALLY
/// (design/remote-desktop-sweep.md M2). It consumes [`CursorShape`](super::control::CursorShape)
/// control messages (RGBA bitmap + hotspot, cached by serial) and per-frame
/// [`CursorState`](super::datagram::CursorState) `0xD0` datagrams (position/visibility), and
/// draws the pointer itself — so the host must STOP compositing the cursor into the video
/// (`SessionPlan.cursor_blend = false`) or the user sees it twice. Active only when the host
/// answers with [`HOST_CAP_CURSOR`] (capable-and-agreed, the 444/clipboard precedent); toward
/// an older or incapable host nothing changes.
pub const CLIENT_CAP_CURSOR: u8 = 0x01;
/// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
/// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
/// whose capture carries no cursor, and NOT Windows yet, where DWM composites into the IDD
/// frame). Set only when the client asked via [`CLIENT_CAP_CURSOR`]; when both bits agree the
/// host stops blending and ships [`CursorShape`](super::control::CursorShape) +
/// [`CursorState`](super::datagram::CursorState) instead.
pub const HOST_CAP_CURSOR: u8 = 0x04;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** /// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST /// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this. /// advertise this.
@@ -225,6 +254,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_HEVC, codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD, host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD,
cipher: 0,
key_chacha: None,
}; };
let got = Welcome::decode(&w.encode()).unwrap(); let got = Welcome::decode(&w.encode()).unwrap();
assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD); assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD);
+115
View File
@@ -783,6 +783,83 @@ impl ClipFetchHdr {
} }
} }
// --- Cursor channel (design/remote-desktop-sweep.md M2) --------------------------------------
// The host cursor, forwarded out-of-band so the CLIENT draws it as a real OS cursor (the
// Parsec/RDP model) instead of paying the video round-trip. Shape (rare, needs reliability)
// rides here on the control stream; per-frame position/visibility rides the lossy `0xD0`
// datagram plane ([`super::datagram::CursorState`]). Active only when the client's
// [`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) met the host's
// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR) — the host stops compositing then.
// ---------------------------------------------------------------------------------------------
/// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
pub const MSG_CURSOR_SHAPE: u8 = 0x50;
/// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
/// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
/// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
/// real cursors (typically ≤ 64 px, ≤ 96 px at HiDPI scale); the HOST downscales anything
/// larger before forwarding, so the cap is invisible to clients.
pub const CURSOR_SHAPE_MAX_SIDE: u16 = 120;
/// `host → client` ([`MSG_CURSOR_SHAPE`]): one cursor shape, sent when the pointer's bitmap
/// changes (never per-frame — [`super::datagram::CursorState`] carries the motion). The client
/// caches shapes by `serial` and re-installs a cached one without any bitmap crossing again
/// (the RDP pointer-cache idea for free: re-showing a known serial is a 14-byte
/// [`super::datagram::CursorState`], not a resend).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct CursorShape {
/// Bitmap identity — bumped by the host's capture layer only on shape change; position
/// moves keep the serial stable. [`super::datagram::CursorState::serial`] references it.
pub serial: u32,
/// Bitmap dimensions in pixels, `1..=`[`CURSOR_SHAPE_MAX_SIDE`] each.
pub w: u16,
pub h: u16,
/// Hotspot (the pixel that IS the pointer position), within `w`×`h`.
pub hot_x: u16,
pub hot_y: u16,
/// Straight-alpha RGBA8, exactly `w * h * 4` bytes, no padding.
pub rgba: Vec<u8>,
}
impl CursorShape {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] serial[5..9] w[9..11] h[11..13] hot_x[13..15] hot_y[15..17] rgba…
let mut b = Vec::with_capacity(17 + self.rgba.len());
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CURSOR_SHAPE);
b.extend_from_slice(&self.serial.to_le_bytes());
b.extend_from_slice(&self.w.to_le_bytes());
b.extend_from_slice(&self.h.to_le_bytes());
b.extend_from_slice(&self.hot_x.to_le_bytes());
b.extend_from_slice(&self.hot_y.to_le_bytes());
b.extend_from_slice(&self.rgba);
b
}
pub fn decode(b: &[u8]) -> Result<CursorShape> {
if b.len() < 17 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CURSOR_SHAPE {
return Err(PunktfunkError::InvalidArg("bad CursorShape"));
}
let u16at = |o: usize| u16::from_le_bytes([b[o], b[o + 1]]);
let (w, h) = (u16at(9), u16at(11));
if w == 0 || h == 0 || w > CURSOR_SHAPE_MAX_SIDE || h > CURSOR_SHAPE_MAX_SIDE {
return Err(PunktfunkError::InvalidArg("bad CursorShape dims"));
}
if b.len() != 17 + (w as usize) * (h as usize) * 4 {
return Err(PunktfunkError::InvalidArg("bad CursorShape len"));
}
Ok(CursorShape {
serial: u32::from_le_bytes(b[5..9].try_into().unwrap()),
w,
h,
hot_x: u16at(13),
hot_y: u16at(15),
rgba: b[17..].to_vec(),
})
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use crate::config::Mode; use crate::config::Mode;
@@ -1147,4 +1224,42 @@ mod tests {
assert!(ClipFetchHdr::decode(&[bytes.as_slice(), &[0]].concat()).is_err()); assert!(ClipFetchHdr::decode(&[bytes.as_slice(), &[0]].concat()).is_err());
assert!(ClipFetchHdr::decode(&bytes[..bytes.len() - 1]).is_err()); assert!(ClipFetchHdr::decode(&bytes[..bytes.len() - 1]).is_err());
} }
#[test]
fn cursor_shape_roundtrip() {
let s = CursorShape {
serial: 7,
w: 2,
h: 3,
hot_x: 1,
hot_y: 2,
rgba: (0..2 * 3 * 4).map(|i| i as u8).collect(),
};
assert_eq!(CursorShape::decode(&s.encode()).unwrap(), s);
// Max-side shape still fits the u16 control frame with headroom.
let side = CURSOR_SHAPE_MAX_SIDE;
let big = CursorShape {
serial: u32::MAX,
w: side,
h: side,
hot_x: side - 1,
hot_y: 0,
rgba: vec![0xAB; side as usize * side as usize * 4],
};
let bytes = big.encode();
assert!(bytes.len() <= u16::MAX as usize, "must fit a control frame");
assert_eq!(CursorShape::decode(&bytes).unwrap(), big);
// Rejections: zero / oversize dims, and a length that disagrees with them.
let mut zero = s.encode();
zero[9] = 0;
zero[10] = 0;
assert!(CursorShape::decode(&zero).is_err());
let mut oversize = s.encode();
oversize[9..11].copy_from_slice(&(CURSOR_SHAPE_MAX_SIDE + 1).to_le_bytes());
assert!(CursorShape::decode(&oversize).is_err());
let mut short = s.encode();
short.pop();
assert!(CursorShape::decode(&short).is_err());
// Distinct from the neighboring vocabulary.
assert!(ClipState::decode(&s.encode()).is_err());
}
} }
@@ -606,6 +606,75 @@ pub fn decode_host_timing_datagram(b: &[u8]) -> Option<HostTiming> {
}) })
} }
/// Cursor-state datagram tag, host → client (design/remote-desktop-sweep.md M2). Next tag after
/// [`HOST_TIMING_MAGIC`]. Sent once per captured frame while the cursor channel is negotiated
/// ([`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) ∧
/// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR)) — per-frame resend makes the plane
/// self-healing under loss (latest-wins, no refresh timer). The bitmap itself rides the
/// reliable control stream ([`CursorShape`](super::control::CursorShape)); this 14-byte
/// datagram only moves/hides the pointer.
pub const CURSOR_STATE_MAGIC: u8 = 0xD0;
/// [`CursorState::flags`] bit: the host cursor is visible.
pub const CURSOR_VISIBLE: u8 = 0x01;
/// [`CursorState::flags`] bit: a host app captured/hid the pointer — the client SHOULD run
/// relative/captured (M3 auto-flip; advisory, user override always wins).
pub const CURSOR_RELATIVE_HINT: u8 = 0x02;
/// Per-frame host-cursor state (position, visibility, mode hint). `x`/`y` are the pointer
/// position (hotspot point, not bitmap top-left) in the host OUTPUT's pixel space — the same
/// space the video mode describes, so the client maps through its letterbox exactly like it
/// maps touches, in reverse.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct CursorState {
/// The [`CursorShape`](super::control::CursorShape) serial this state refers to. A client
/// that has no cached shape for it keeps its previous cursor until the (reliable) shape
/// message lands — at worst one control-stream RTT of stale shape, never a wrong position.
pub serial: u32,
/// Bitfield of [`CURSOR_VISIBLE`] / [`CURSOR_RELATIVE_HINT`].
pub flags: u8,
pub x: i32,
pub y: i32,
}
impl CursorState {
pub fn visible(&self) -> bool {
self.flags & CURSOR_VISIBLE != 0
}
pub fn relative_hint(&self) -> bool {
self.flags & CURSOR_RELATIVE_HINT != 0
}
}
/// Wire length of a [`CURSOR_STATE_MAGIC`] datagram: tag + u32 serial + flags + 2 × i32 = 14.
const CURSOR_STATE_LEN: usize = 1 + 4 + 1 + 8;
/// Encode a [`CursorState`] into a [`CURSOR_STATE_MAGIC`] datagram.
pub fn encode_cursor_state_datagram(s: &CursorState) -> Vec<u8> {
let mut b = Vec::with_capacity(CURSOR_STATE_LEN);
b.push(CURSOR_STATE_MAGIC);
b.extend_from_slice(&s.serial.to_le_bytes());
b.push(s.flags);
b.extend_from_slice(&s.x.to_le_bytes());
b.extend_from_slice(&s.y.to_le_bytes());
b
}
/// Parse a [`CURSOR_STATE_MAGIC`] datagram → [`CursorState`]. `None` on bad tag or a short
/// buffer (the fixed length bounds every read before it happens; a longer buffer is tolerated
/// for append-extension, like 0xCF).
pub fn decode_cursor_state_datagram(b: &[u8]) -> Option<CursorState> {
if b.len() < CURSOR_STATE_LEN || b[0] != CURSOR_STATE_MAGIC {
return None;
}
Some(CursorState {
serial: u32::from_le_bytes(b[1..5].try_into().unwrap()),
flags: b[5],
x: i32::from_le_bytes(b[6..10].try_into().unwrap()),
y: i32::from_le_bytes(b[10..14].try_into().unwrap()),
})
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use crate::quic::*; use crate::quic::*;
@@ -922,4 +991,32 @@ mod tests {
) )
.is_none()); .is_none());
} }
#[test]
fn cursor_state_roundtrip() {
for (flags, x, y) in [
(CURSOR_VISIBLE, 0i32, 0i32),
(CURSOR_VISIBLE | CURSOR_RELATIVE_HINT, -5, 2160),
(0, i32::MIN, i32::MAX),
] {
let s = CursorState {
serial: 42,
flags,
x,
y,
};
let d = encode_cursor_state_datagram(&s);
assert_eq!(decode_cursor_state_datagram(&d), Some(s));
assert_eq!(s.visible(), flags & CURSOR_VISIBLE != 0);
assert_eq!(s.relative_hint(), flags & CURSOR_RELATIVE_HINT != 0);
// Append-extensible like 0xCF: a longer buffer still parses the known prefix.
let mut ext = d.clone();
ext.push(0xFF);
assert_eq!(decode_cursor_state_datagram(&ext), Some(s));
// Short / wrong tag are rejected before any read.
assert_eq!(decode_cursor_state_datagram(&d[..d.len() - 1]), None);
let mut bad = d.clone();
bad[0] = HOST_TIMING_MAGIC;
assert_eq!(decode_cursor_state_datagram(&bad), None);
}
}
} }
+268 -13
View File
@@ -4,6 +4,7 @@ use super::*;
use crate::config::{ use crate::config::{
CompositorPref, Config, FecConfig, FecScheme, GamepadPref, Mode, ProtocolPhase, Role, CompositorPref, Config, FecConfig, FecScheme, GamepadPref, Mode, ProtocolPhase, Role,
}; };
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
/// `client → host`: open the session, requesting a display mode (the host creates its /// `client → host`: open the session, requesting a display mode (the host creates its
@@ -82,6 +83,15 @@ pub struct Hello {
/// forcing the earlier placeholders. Omitted by older clients / when the client has no HDR /// forcing the earlier placeholders. Omitted by older clients / when the client has no HDR
/// display (decodes to `None` — the host keeps its built-in EDID defaults). /// display (decodes to `None` — the host keeps its built-in EDID defaults).
pub display_hdr: Option<HdrMeta>, pub display_hdr: Option<HdrMeta>,
/// Non-video client capabilities — a bitfield of [`CLIENT_CAP_CURSOR`] (the client renders
/// the host cursor locally; the host stops compositing it and forwards shape + state
/// instead). Appended as a single byte AFTER `display_hdr`; because that block is a fixed
/// [`super::datagram::HDR_META_BODY_LEN`]-byte optional with no placeholder form, presence is
/// disambiguated by REMAINING LENGTH at decode: fewer than `HDR_META_BODY_LEN` bytes after
/// `preferred_codec` ⇒ no HDR block, the tail bytes are the post-HDR fields directly. This
/// caps everything after `display_hdr` at `HDR_META_BODY_LEN 1` bytes total — document any
/// future field here and mind the budget. Omitted when zero and by older clients (→ `0`).
pub client_caps: u8,
} }
/// QUIC application error code a punktfunk/1 client closes the control connection with on a /// QUIC application error code a punktfunk/1 client closes the control connection with on a
@@ -107,6 +117,13 @@ pub const HELLO_NAME_MAX: usize = 64;
/// (`steam:<appid>` / `custom:<12 hex>`); the cap just bounds an attacker-controlled field. /// (`steam:<appid>` / `custom:<12 hex>`); the cap just bounds an attacker-controlled field.
pub const HELLO_LAUNCH_MAX: usize = 128; pub const HELLO_LAUNCH_MAX: usize = 128;
/// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
/// only one pre-cipher builds know).
pub const CIPHER_AES_128_GCM: u8 = 0;
/// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
/// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
pub const CIPHER_CHACHA20_POLY1305: u8 = 1;
/// `host → client`: the complete session offer. /// `host → client`: the complete session offer.
#[derive(Clone, Copy, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Welcome { pub struct Welcome {
@@ -173,6 +190,22 @@ pub struct Welcome {
/// per-transition events otherwise). Appended after `codec` as a single trailing byte; an /// per-transition events otherwise). Appended after `codec` as a single trailing byte; an
/// older host that omits it decodes to `0` (no capabilities — legacy events only). /// older host that omits it decodes to `0` (no capabilities — legacy events only).
pub host_caps: u8, pub host_caps: u8,
/// The session AEAD the data plane seals with — [`CIPHER_AES_128_GCM`] (`0`, the default
/// every peer speaks) or [`CIPHER_CHACHA20_POLY1305`] (`1`). The host sets `1` ONLY toward
/// a client that advertised [`VIDEO_CAP_CHACHA20`] (the soft-AES armv7 targets). Appended
/// after `host_caps` at offset 68 and — unlike the earlier trailing fields — emitted only
/// when non-zero, so an AES session's Welcome stays **byte-identical** to the pre-cipher
/// wire form; an older host omits it (→ `0`, AES). Decode is fail-closed: an unknown id is
/// an `Err`, never a silent AES fallback — the host only picks a cipher this client
/// advertised, so an unknown id reaching us is a bug, and falling back would yield an
/// undecryptable session with a confusing failure signature.
pub cipher: u8,
/// The 256-bit ChaCha20-Poly1305 session key (RFC 8439 requires the full 32 bytes; wire
/// cost is once per handshake) — present iff `cipher == 1`, at offsets 69..101. The legacy
/// 16-byte `key` keeps its offset and stays independently random, so nothing downstream
/// ever observes an all-zero key. Decode rejects `cipher == 1` with fewer than 32 key
/// bytes following.
pub key_chacha: Option<[u8; 32]>,
} }
/// `client → host`: data plane is bound, begin streaming. /// `client → host`: data plane is bound, begin streaming.
@@ -220,8 +253,13 @@ impl Hello {
let vcodecs_present = self.video_codecs != 0; let vcodecs_present = self.video_codecs != 0;
let pref_present = self.preferred_codec != 0; let pref_present = self.preferred_codec != 0;
let hdr_present = self.display_hdr.is_some(); let hdr_present = self.display_hdr.is_some();
let need_placeholders = let ccaps_present = self.client_caps != 0;
self.video_caps != 0 || ac_present || vcodecs_present || pref_present || hdr_present; let need_placeholders = self.video_caps != 0
|| ac_present
|| vcodecs_present
|| pref_present
|| hdr_present
|| ccaps_present;
match (&self.name, &self.launch) { match (&self.name, &self.launch) {
(None, None) if !need_placeholders => {} (None, None) if !need_placeholders => {}
(name, _) => { (name, _) => {
@@ -242,21 +280,27 @@ impl Hello {
b.push(self.video_caps); b.push(self.video_caps);
} }
// audio_channels: emitted when non-stereo OR a later field follows. // audio_channels: emitted when non-stereo OR a later field follows.
if ac_present || vcodecs_present || pref_present || hdr_present { if ac_present || vcodecs_present || pref_present || hdr_present || ccaps_present {
b.push(self.audio_channels); b.push(self.audio_channels);
} }
// video_codecs: emitted when non-zero OR a later field follows. // video_codecs: emitted when non-zero OR a later field follows.
if vcodecs_present || pref_present || hdr_present { if vcodecs_present || pref_present || hdr_present || ccaps_present {
b.push(self.video_codecs); b.push(self.video_codecs);
} }
// preferred_codec: emitted when non-zero OR display_hdr follows. // preferred_codec: emitted when non-zero OR a later field follows.
if pref_present || hdr_present { if pref_present || hdr_present || ccaps_present {
b.push(self.preferred_codec); b.push(self.preferred_codec);
} }
// display_hdr: fixed HDR_META_BODY_LEN-byte HdrMeta body. Last field; omitted when `None`. // display_hdr: fixed HDR_META_BODY_LEN-byte HdrMeta body; omitted when `None` even if
// later fields follow (no placeholder form — the decoder disambiguates by remaining
// length, which caps the post-HDR tail at HDR_META_BODY_LEN 1 bytes).
if let Some(m) = &self.display_hdr { if let Some(m) = &self.display_hdr {
super::datagram::write_hdr_meta_body(m, &mut b); super::datagram::write_hdr_meta_body(m, &mut b);
} }
// client_caps: single byte after the (optional) HDR block. Emitted when non-zero.
if ccaps_present {
b.push(self.client_caps);
}
b b
} }
@@ -322,9 +366,26 @@ impl Hello {
preferred_codec: b.get(tail + 3).copied().unwrap_or(0), preferred_codec: b.get(tail + 3).copied().unwrap_or(0),
// Optional trailing HdrMeta body (fixed length) — absent on an older client / a // Optional trailing HdrMeta body (fixed length) — absent on an older client / a
// client without an HDR display → `None` (the host keeps its EDID defaults). // client without an HDR display → `None` (the host keeps its EDID defaults).
display_hdr: b // Presence is decided by REMAINING LENGTH (there is no placeholder form for the
.get(tail + 4..tail + 4 + super::datagram::HDR_META_BODY_LEN) // fixed block): ≥ HDR_META_BODY_LEN bytes after `preferred_codec` ⇒ the block is
.map(super::datagram::read_hdr_meta_body), // there and post-HDR fields follow it; fewer ⇒ no block, the bytes ARE the post-HDR
// fields. Sound as long as the post-HDR tail stays under HDR_META_BODY_LEN bytes.
display_hdr: (b.len().saturating_sub(tail + 4) >= super::datagram::HDR_META_BODY_LEN)
.then(|| {
b.get(tail + 4..tail + 4 + super::datagram::HDR_META_BODY_LEN)
.map(super::datagram::read_hdr_meta_body)
})
.flatten(),
// client_caps: the byte after the HDR block when present, else directly at tail+4.
client_caps: {
let off = if b.len().saturating_sub(tail + 4) >= super::datagram::HDR_META_BODY_LEN
{
tail + 4 + super::datagram::HDR_META_BODY_LEN
} else {
tail + 4
};
b.get(off).copied().unwrap_or(0)
},
}) })
} }
} }
@@ -366,6 +427,21 @@ impl Welcome {
b.push(self.codec); b.push(self.codec);
// Host input caps at offset 67 — older clients stop before this → 0 (legacy input only). // Host input caps at offset 67 — older clients stop before this → 0 (legacy input only).
b.push(self.host_caps); b.push(self.host_caps);
// Session cipher at offset 68 + the 32-byte ChaCha key at 69..101 — emitted ONLY when a
// non-default cipher was negotiated, so an AES session's Welcome stays byte-identical
// to the pre-cipher wire form. The host only sets cipher toward a client that
// advertised VIDEO_CAP_CHACHA20, so an old client never sees these bytes at all.
debug_assert_eq!(
self.cipher == CIPHER_CHACHA20_POLY1305,
self.key_chacha.is_some(),
"key_chacha present iff cipher == 1"
);
if self.cipher != CIPHER_AES_128_GCM {
b.push(self.cipher);
if let Some(k) = &self.key_chacha {
b.extend_from_slice(k);
}
}
b b
} }
@@ -374,8 +450,10 @@ impl Welcome {
// scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45] // scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45]
// salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59] // salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59]
// bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63] // bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63]
// chroma_format[64] audio_channels[65] codec[66] (everything from compositor on is an // chroma_format[64] audio_channels[65] codec[66] host_caps[67] cipher[68]
// optional trailing byte; an older host stops earlier). // key_chacha[69..101] (everything from compositor on is an optional trailing byte; an
// older host stops earlier; cipher/key_chacha are present only when ChaCha was
// negotiated).
if b.len() < 53 || &b[0..4] != MAGIC { if b.len() < 53 || &b[0..4] != MAGIC {
return Err(PunktfunkError::InvalidArg("bad Welcome")); return Err(PunktfunkError::InvalidArg("bad Welcome"));
} }
@@ -385,6 +463,24 @@ impl Welcome {
key.copy_from_slice(&b[29..45]); key.copy_from_slice(&b[29..45]);
let mut salt = [0u8; 4]; let mut salt = [0u8; 4];
salt.copy_from_slice(&b[45..49]); salt.copy_from_slice(&b[45..49]);
// Session cipher at 68 — absent on an older host → AES-128-GCM. Fail-closed on
// anything else: `cipher == 1` with fewer than 32 key bytes must be an error (a silent
// AES fallback would yield an undecryptable session with a confusing failure
// signature), and an unknown id (≥ 2) reaching us is a bug — a host only picks a
// cipher this client advertised — never a legitimate negotiation.
let cipher = b.get(68).copied().unwrap_or(CIPHER_AES_128_GCM);
let key_chacha = match cipher {
CIPHER_AES_128_GCM => None,
CIPHER_CHACHA20_POLY1305 => {
let bytes = b
.get(69..101)
.ok_or(PunktfunkError::InvalidArg("bad Welcome"))?;
let mut k = [0u8; 32];
k.copy_from_slice(bytes);
Some(k)
}
_ => return Err(PunktfunkError::InvalidArg("bad Welcome")),
};
Ok(Welcome { Ok(Welcome {
abi_version: u32at(4), abi_version: u32at(4),
udp_port: u16at(8), udp_port: u16at(8),
@@ -452,6 +548,8 @@ impl Welcome {
// Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state // Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state
// snapshots; the client keeps sending legacy per-transition events). // snapshots; the client keeps sending legacy per-transition events).
host_caps: b.get(67).copied().unwrap_or(0), host_caps: b.get(67).copied().unwrap_or(0),
cipher,
key_chacha,
}) })
} }
@@ -462,7 +560,12 @@ impl Welcome {
c.fec = self.fec; c.fec = self.fec;
c.shard_payload = self.shard_payload as usize; c.shard_payload = self.shard_payload as usize;
c.encrypt = self.encrypt; c.encrypt = self.encrypt;
c.key = self.key; // The negotiated AEAD: the ChaCha key when cipher == 1 (guaranteed present by decode —
// the `(1, None)` shape is unreachable off the wire), the legacy AES key otherwise.
c.key = match (self.cipher, self.key_chacha) {
(CIPHER_CHACHA20_POLY1305, Some(k)) => SessionKey::ChaCha20Poly1305(k),
_ => SessionKey::Aes128Gcm(self.key),
};
c.salt = self.salt; c.salt = self.salt;
// Client-side reassembler ceiling: p1_defaults' 64 MiB hostile-header memory bound is // Client-side reassembler ceiling: p1_defaults' 64 MiB hostile-header memory bound is
// ~10x larger than any real access unit. Derive it from the negotiated rate instead: // ~10x larger than any real access unit. Derive it from the negotiated rate instead:
@@ -531,6 +634,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, // exercise a non-default codec through the roundtrip codec: CODEC_H264, // exercise a non-default codec through the roundtrip
host_caps: HOST_CAP_GAMEPAD_STATE, host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
}; };
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w); assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
@@ -564,6 +669,81 @@ mod tests {
assert!(derived > (8 << 20) && derived < (64 << 20)); assert!(derived > (8 << 20) && derived < (64 << 20));
} }
#[test]
fn welcome_cipher_negotiation_wire_and_back_compat() {
use crate::crypto::SessionKey;
let base = Welcome {
abi_version: 2,
udp_port: 7000,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 20,
max_data_per_block: 4096,
},
shard_payload: 1200,
encrypt: true,
key: [7u8; 16],
salt: [9, 8, 7, 6],
frames: 0,
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 50_000,
bit_depth: 8,
color: ColorInfo::SDR_BT709,
chroma_format: CHROMA_IDC_420,
audio_channels: 2,
codec: CODEC_HEVC,
host_caps: 0,
cipher: CIPHER_AES_128_GCM,
key_chacha: None,
};
// An AES session's Welcome is byte-identical to the pre-cipher wire form (68 bytes) —
// the old-client × new-host interop guarantee.
let enc = base.encode();
assert_eq!(enc.len(), 68);
assert_eq!(Welcome::decode(&enc).unwrap(), base);
// ChaCha roundtrip: cipher byte at 68, the 32-byte key at 69..101.
let k32: [u8; 32] = core::array::from_fn(|i| i as u8 + 1);
let cha = Welcome {
cipher: CIPHER_CHACHA20_POLY1305,
key_chacha: Some(k32),
..base
};
let cenc = cha.encode();
assert_eq!(cenc.len(), 68 + 1 + 32);
assert_eq!(Welcome::decode(&cenc).unwrap(), cha);
// A truncated old-host Welcome (no cipher byte) decodes to the AES default.
let old_host = Welcome::decode(&cenc[..68]).unwrap();
assert_eq!(old_host.cipher, CIPHER_AES_128_GCM);
assert_eq!(old_host.key_chacha, None);
// cipher == 1 with a missing / short key → Err, fail-closed (a silent AES fallback
// would yield an undecryptable session with a confusing failure signature).
assert!(Welcome::decode(&cenc[..69]).is_err());
assert!(Welcome::decode(&cenc[..100]).is_err());
// An unknown cipher id (≥ 2) → Err: the host only picks a cipher we advertised, so an
// unknown id reaching us is a bug, never a legitimate negotiation.
let mut bad = cenc.clone();
bad[68] = 2;
assert!(Welcome::decode(&bad).is_err());
// session_config maps both variants onto the data-plane key, and both validate.
let aes_cfg = base.session_config(Role::Client);
assert_eq!(aes_cfg.key, SessionKey::Aes128Gcm([7u8; 16]));
aes_cfg.validate().expect("AES config validates");
let cha_cfg = cha.session_config(Role::Client);
assert_eq!(cha_cfg.key, SessionKey::ChaCha20Poly1305(k32));
cha_cfg.validate().expect("ChaCha config validates");
}
#[test] #[test]
fn codec_negotiation_and_back_compat() { fn codec_negotiation_and_back_compat() {
// resolve_codec precedence (HEVC > AV1 > H.264), no preference (0). // resolve_codec precedence (HEVC > AV1 > H.264), no preference (0).
@@ -656,6 +836,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_PYROWAVE, codec: CODEC_PYROWAVE,
host_caps: 0, host_caps: 0,
cipher: 0,
key_chacha: None,
} }
.encode(), .encode(),
) )
@@ -684,6 +866,7 @@ mod tests {
video_codecs: CODEC_H264 | CODEC_HEVC, video_codecs: CODEC_H264 | CODEC_HEVC,
preferred_codec: CODEC_H264, preferred_codec: CODEC_H264,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
let enc = h.encode(); let enc = h.encode();
let dec = Hello::decode(&enc).unwrap(); let dec = Hello::decode(&enc).unwrap();
@@ -726,6 +909,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, codec: CODEC_H264,
host_caps: 0, host_caps: 0,
cipher: 0,
key_chacha: None,
} }
.encode(), .encode(),
) )
@@ -758,6 +943,7 @@ mod tests {
video_codecs: CODEC_H264 | CODEC_HEVC, // exercise the codec bitfield roundtrip video_codecs: CODEC_H264 | CODEC_HEVC, // exercise the codec bitfield roundtrip
preferred_codec: CODEC_HEVC, preferred_codec: CODEC_HEVC,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
assert_eq!(Hello::decode(&h.encode()).unwrap(), h); assert_eq!(Hello::decode(&h.encode()).unwrap(), h);
let s = Start { let s = Start {
@@ -788,6 +974,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
let enc = h.encode(); let enc = h.encode();
assert_eq!(enc.len(), 26); assert_eq!(enc.len(), 26);
@@ -831,6 +1018,8 @@ mod tests {
audio_channels: 6, // 5.1 — exercises the non-default trailing byte audio_channels: 6, // 5.1 — exercises the non-default trailing byte
codec: CODEC_HEVC, codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE, host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
}; };
let wenc = w.encode(); let wenc = w.encode();
assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps
@@ -903,6 +1092,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
let enc = base.encode(); let enc = base.encode();
assert_eq!( assert_eq!(
@@ -954,6 +1144,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
// launch alone (no name): a zero-length name placeholder keeps the offset deterministic. // launch alone (no name): a zero-length name placeholder keeps the offset deterministic.
let with_launch = Hello { let with_launch = Hello {
@@ -1013,6 +1204,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
// A real client-panel volume (P3 primaries, 800-nit peak, 0.05-nit floor, 400-nit FALL). // A real client-panel volume (P3 primaries, 800-nit peak, 0.05-nit floor, 400-nit FALL).
let vol = HdrMeta { let vol = HdrMeta {
@@ -1080,6 +1272,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
} }
.encode(); .encode();
assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair"); assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair");
@@ -1093,4 +1286,66 @@ mod tests {
.encode(); .encode();
assert!(Hello::decode(&pr).is_err()); assert!(Hello::decode(&pr).is_err());
} }
#[test]
fn hello_client_caps_roundtrip_and_back_compat() {
let base = Hello {
abi_version: 2,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 0,
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
};
let vol = HdrMeta {
display_primaries: [[13250, 34500], [7500, 3000], [34000, 16000]],
white_point: [15635, 16450],
max_display_mastering_luminance: 8_000_000,
min_display_mastering_luminance: 500,
max_cll: 0,
max_fall: 400,
};
// caps WITHOUT an HDR block: the single byte after preferred_codec (remaining < the
// fixed block length, so the decoder must NOT read it as a truncated HdrMeta).
let caps_only = Hello {
client_caps: CLIENT_CAP_CURSOR,
..base.clone()
};
assert_eq!(Hello::decode(&caps_only.encode()).unwrap(), caps_only);
// caps AND the HDR block: caps lands after the fixed block.
let both = Hello {
display_hdr: Some(vol),
client_caps: CLIENT_CAP_CURSOR,
..base.clone()
};
assert_eq!(Hello::decode(&both.encode()).unwrap(), both);
// HDR without caps stays byte-identical to the pre-caps wire form and decodes caps 0.
let hdr_only = Hello {
display_hdr: Some(vol),
..base.clone()
};
assert_eq!(Hello::decode(&hdr_only.encode()).unwrap(), hdr_only);
// An older client (no trailing byte at all) decodes to 0.
assert_eq!(Hello::decode(&base.encode()).unwrap().client_caps, 0);
// An older HOST reading a caps-bearing Hello: its decode simply never looks past the
// fields it knows — nothing before the caps byte moved.
let enc = both.encode();
assert_eq!(
Hello::decode(&enc[..enc.len() - 1]).unwrap(),
Hello {
client_caps: 0,
..both.clone()
}
);
}
} }
+3 -2
View File
@@ -781,6 +781,7 @@ impl Session {
mod wire_equivalence_tests { mod wire_equivalence_tests {
use super::*; use super::*;
use crate::config::{FecConfig, FecScheme, ProtocolPhase}; use crate::config::{FecConfig, FecScheme, ProtocolPhase};
use crate::crypto::SessionKey;
use crate::transport::loopback_pair; use crate::transport::loopback_pair;
fn host_cfg(scheme: FecScheme, fec_percent: u8, encrypt: bool) -> Config { fn host_cfg(scheme: FecScheme, fec_percent: u8, encrypt: bool) -> Config {
@@ -798,7 +799,7 @@ mod wire_equivalence_tests {
shard_payload: 64, shard_payload: 64,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt, encrypt,
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [3, 1, 4, 1], salt: [3, 1, 4, 1],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
@@ -930,7 +931,7 @@ mod wire_equivalence_tests {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
}; };
+26 -1
View File
@@ -5,6 +5,7 @@
use proptest::prelude::*; use proptest::prelude::*;
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionKey;
use punktfunk_core::fec::coder_for; use punktfunk_core::fec::coder_for;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
@@ -25,7 +26,7 @@ fn config(role: Role, scheme: FecScheme, encrypt: bool, drop_period: u32) -> Con
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt, encrypt,
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4], salt: [1, 2, 3, 4],
loopback_drop_period: drop_period, loopback_drop_period: drop_period,
} }
@@ -101,6 +102,30 @@ fn encrypted_stream_recovers_under_loss() {
assert_eq!(stats.frames_completed, frames.len() as u64); assert_eq!(stats.frames_completed, frames.len() as u64);
} }
/// The negotiated ChaCha20-Poly1305 session cipher through the same lossy full-stream path:
/// loss/replay behavior is cipher-independent (the replay window keys off the authenticated
/// seq), so recovery must be byte-identical to the AES run above.
#[test]
fn chacha20_encrypted_stream_recovers_under_loss() {
let frames = sample_frames();
let mk = |role| {
let mut c = config(role, FecScheme::Gf16, true, 8);
c.key = SessionKey::ChaCha20Poly1305([7u8; 32]);
c
};
let (host_tp, client_tp) = loopback_pair(8, 0);
let mut host = Session::new(mk(Role::Host), Box::new(host_tp)).unwrap();
let mut client = Session::new(mk(Role::Client), Box::new(client_tp)).unwrap();
for (i, frame) in frames.iter().enumerate() {
host.submit_frame(frame, i as u64 * 1_000_000, 0).unwrap();
let got = client
.poll_frame()
.expect("frame should recover despite loss");
assert_eq!(&got.data, frame, "frame {i} mismatched after recovery");
}
assert!(client.stats().fec_recovered_shards > 0);
}
#[test] #[test]
fn lossless_stream_is_exact() { fn lossless_stream_is_exact() {
let frames = sample_frames(); let frames = sample_frames();
+9 -3
View File
@@ -26,7 +26,10 @@ pub use pf_capture::{dxgi, synthetic_nv12};
/// capture→encode cycle). Resolved here (the host facade) and threaded in, so the edge stays one-way /// capture→encode cycle). Resolved here (the host facade) and threaded in, so the edge stays one-way
/// (plan §2.4 / §W6). /// (plan §2.4 / §W6).
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
fn zero_copy_policy(pyrowave_session: bool) -> pf_capture::ZeroCopyPolicy { fn zero_copy_policy(
pyrowave_session: bool,
native_nv12_session: bool,
) -> pf_capture::ZeroCopyPolicy {
let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi(); let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi();
// The raw-dmabuf passthrough serves a PyroWave session on ANY vendor (the wavelet encoder's // The raw-dmabuf passthrough serves a PyroWave session on ANY vendor (the wavelet encoder's
// own Vulkan device imports the dmabuf) — per-session from the negotiated codec, plus the // own Vulkan device imports the dmabuf) — per-session from the negotiated codec, plus the
@@ -56,6 +59,7 @@ fn zero_copy_policy(pyrowave_session: bool) -> pf_capture::ZeroCopyPolicy {
backend_is_gpu: crate::encode::resolved_backend_is_gpu(), backend_is_gpu: crate::encode::resolved_backend_is_gpu(),
pyrowave_session, pyrowave_session,
pyrowave_modifiers, pyrowave_modifiers,
native_nv12_session,
} }
} }
@@ -70,7 +74,9 @@ pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> {
let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei; let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei;
// Monitor mirrors never carry the native PyroWave plane (GameStream protocol) — per-session // Monitor mirrors never carry the native PyroWave plane (GameStream protocol) — per-session
// passthrough is virtual-output-only; the global encoder-pref lever still applies inside. // passthrough is virtual-output-only; the global encoder-pref lever still applies inside.
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy(false)) // Native NV12 stays off too: the mirror path doesn't resolve the codec here, and the desktop
// compositors it mirrors (GNOME/KWin) don't produce NV12 anyway.
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy(false, false))
} }
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
@@ -101,7 +107,7 @@ pub fn capture_virtual_output(
vout.keepalive, vout.keepalive,
want.gpu, want.gpu,
want.chroma_444, want.chroma_444,
zero_copy_policy(want.pyrowave), zero_copy_policy(want.pyrowave, want.nv12_native),
) )
} }
@@ -86,6 +86,12 @@ pub fn start(
crate::events::emit(crate::events::EventKind::ClientConnected { crate::events::emit(crate::events::EventKind::ClientConnected {
client: event_client.clone(), client: event_client.clone(),
}); });
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock
// floor while this compat-plane stream runs, refcounted with every other live session
// across both planes. Released when the closure exits (stream stopped) — so idle clocks
// aren't pinned between Moonlight sessions. No-op off Linux / when the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
let result = run( let result = run(
cfg, cfg,
app.as_ref(), app.as_ref(),
+98 -22
View File
@@ -2,11 +2,15 @@
//! plan Tier 1B): the driver's adaptive P-state ramps clocks down between bursty encode frames, //! plan Tier 1B): the driver's adaptive P-state ramps clocks down between bursty encode frames,
//! so every frame re-pays a spin-up. This is NOT theoretical — measured on the 780M (VCN 4), //! so every frame re-pays a spin-up. This is NOT theoretical — measured on the 780M (VCN 4),
//! a 1440p HEVC encode takes ~4.4 ms/frame with the clocks hot (120 fps pacing) but ~8 ms/frame //! a 1440p HEVC encode takes ~4.4 ms/frame with the clocks hot (120 fps pacing) but ~8 ms/frame
//! at a 60 fps duty cycle: the sag doubles per-frame encode latency. //! at a 60 fps duty cycle: the sag doubles per-frame encode latency. So the pin is worth holding
//! only while a client is actively streaming: it is armed on the first live client and released
//! when the last one disconnects (refcounted across both streaming planes — see [`session_pin`]),
//! leaving the driver's idle power management alone the rest of the time.
//! //!
//! **AMD** (`PUNKTFUNK_PIN_CLOCKS=1`, root-gated by sysfs ownership): write `high` into each //! **AMD** (`PUNKTFUNK_PIN_CLOCKS=1`, root-gated by sysfs ownership): write `high` into each
//! amdgpu card's `power_dpm_force_performance_level` for the host lifetime, restoring the prior //! amdgpu card's `power_dpm_force_performance_level` while a client is streaming, restoring the
//! value on exit. Non-root gets EACCES → logged once with the privilege recipe. Deliberately //! prior value when the last client disconnects. Non-root gets EACCES → logged once with the
//! privilege recipe. Deliberately
//! opt-in: it defeats power management box-wide and is wrong on battery (Steam Deck!). //! opt-in: it defeats power management box-wide and is wrong on battery (Steam Deck!).
//! //!
//! **NVIDIA** — two independent halves, both no-ops off NVIDIA: //! **NVIDIA** — two independent halves, both no-ops off NVIDIA:
@@ -28,14 +32,16 @@
//! while leaving boost headroom — NVIDIA's own latency guidance is "raise the floor, don't pin //! while leaving boost headroom — NVIDIA's own latency guidance is "raise the floor, don't pin
//! the max" (locking above base just gets throttled; a max pin only burns idle watts). Non-root //! the max" (locking above base just gets throttled; a max pin only burns idle watts). Non-root
//! callers get `NVML_ERROR_NO_PERMISSION` — logged once with the privilege recipe, then the //! callers get `NVML_ERROR_NO_PERMISSION` — logged once with the privilege recipe, then the
//! host runs unpinned. The pin is undone on drop (host exit); after a crash it persists until //! host runs unpinned. The pin is undone on drop (when the last client disconnects); after a
//! driver reload/reboot, which the reset-before-pin on the next start self-heals. Deliberately //! crash it persists until driver reload/reboot, which the reset-before-pin on the next arm
//! self-heals. Deliberately
//! NOT default-on: it defeats idle downclocking for the whole box and is wrong on //! NOT default-on: it defeats idle downclocking for the whole box and is wrong on
//! battery-powered hosts. //! battery-powered hosts.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program). // Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use std::os::raw::{c_char, c_int, c_uint, c_void}; use std::os::raw::{c_char, c_int, c_uint, c_void};
use std::sync::{Mutex, OnceLock};
/// `nvmlDevice_t` — an opaque driver handle. /// `nvmlDevice_t` — an opaque driver handle.
type NvmlDevice = *mut c_void; type NvmlDevice = *mut c_void;
@@ -133,8 +139,9 @@ struct AmdPin {
restore: String, restore: String,
} }
/// Host-lifetime guard: holds the armed clock pins (NVML floor and/or amdgpu perf level) and /// Holds the armed clock pins (NVML floor and/or amdgpu perf level) and undoes them on drop. Owned
/// undoes them on drop. /// by the [`session_pin`] refcount: constructed when the first live client arms the pin, dropped
/// (clocks restored) when the last one disconnects.
pub struct ClockGuard { pub struct ClockGuard {
nvml: Option<NvmlPin>, nvml: Option<NvmlPin>,
amd: Vec<AmdPin>, amd: Vec<AmdPin>,
@@ -142,9 +149,10 @@ pub struct ClockGuard {
// SAFETY: `ClockGuard` holds opaque NVML device handles + resolved fn pointers from the loaded // SAFETY: `ClockGuard` holds opaque NVML device handles + resolved fn pointers from the loaded
// driver library (plus plain sysfs paths/strings). NVML is documented thread-safe, the handles are // driver library (plus plain sysfs paths/strings). NVML is documented thread-safe, the handles are
// plain driver tokens with no thread affinity, and the guard is only ever *moved* (held in `main`, // plain driver tokens with no thread affinity, and the guard is only ever *moved* (into the
// dropped once at exit) and used through `&mut`/ownership — never shared. Transfer across threads // `pin_refcount` mutex when armed, taken back out and dropped when the last client disconnects) and
// is therefore sound. // used through exclusive ownership behind that mutex — never shared. Transfer across threads is
// therefore sound.
unsafe impl Send for ClockGuard {} unsafe impl Send for ClockGuard {}
impl Drop for ClockGuard { impl Drop for ClockGuard {
@@ -182,22 +190,89 @@ impl Drop for ClockGuard {
} }
/// Startup hook for the host subcommands (`serve` / `punktfunk1-host`): install the NVIDIA P2-cap /// Startup hook for the host subcommands (`serve` / `punktfunk1-host`): install the NVIDIA P2-cap
/// application profile and, when `PUNKTFUNK_PIN_CLOCKS` is set, arm the vendor clock pin (NVML /// application profile — the process-scoped, no-root half of the NVIDIA lever, which the driver
/// core-clock floor / amdgpu `high` performance level). Returns the guard keeping the pins for /// only acts on once the host holds a live CUDA/NVENC context (i.e. during a session).
/// the host lifetime. `None` when nothing was armed. ///
pub fn on_host_start() -> Option<ClockGuard> { /// The vendor clock *pin* is deliberately NOT armed here anymore: held for the whole host lifetime
/// it kept the box's clocks hot even with no client connected. It is now refcounted per live client
/// via [`session_pin`] (armed on both streaming planes), so idle clocks are left to the driver's
/// power management until someone actually streams.
pub fn on_host_start() {
if nvidia_present() { if nvidia_present() {
ensure_cuda_perf_profile(); ensure_cuda_perf_profile();
} }
}
/// The box-wide clock-pin refcount, shared across BOTH streaming planes (native + GameStream): the
/// vendor pin is a single global GPU setting, so N concurrent sessions share ONE pin — armed when
/// the first client goes live, released when the last one leaves.
struct PinRefcount {
/// Number of live [`SessionClockPin`] handles.
live: usize,
/// The armed pins, present iff `live > 0` and something was actually pinnable.
guard: Option<ClockGuard>,
}
fn pin_refcount() -> &'static Mutex<PinRefcount> {
static STATE: OnceLock<Mutex<PinRefcount>> = OnceLock::new();
STATE.get_or_init(|| {
Mutex::new(PinRefcount {
live: 0,
guard: None,
})
})
}
/// RAII handle that keeps the box-wide clock pin armed while it is alive. Obtain one per live client
/// session on either plane via [`session_pin`]; when the last outstanding handle drops, the pin is
/// released and the driver's idle downclocking resumes. A no-op handle when `PUNKTFUNK_PIN_CLOCKS`
/// is unset (the opt-in gate) — the refcount only ticks for sessions that asked for pinning.
pub struct SessionClockPin {
/// Whether this handle actually incremented the refcount (false = opt-in gate off → no-op).
counted: bool,
}
/// Arm the box-wide clock pin for one live client session, refcounted across every plane. Returns
/// an RAII handle; the pin is released when the last handle drops. A no-op (returns immediately,
/// touches no GPU state) unless `PUNKTFUNK_PIN_CLOCKS` is set.
pub fn session_pin() -> SessionClockPin {
if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") { if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") {
return None; return SessionClockPin { counted: false };
} }
let nvml = if nvidia_present() { pin_nvidia() } else { None }; let mut state = pin_refcount().lock().unwrap();
let amd = pin_amdgpu(); state.live += 1;
if nvml.is_none() && amd.is_empty() { if state.live == 1 {
return None; // 0 → 1: the first live client — arm the vendor pin (reset-before-pin inside `pin_nvidia`
// heals a stale pin from a crashed previous run).
let nvml = if nvidia_present() { pin_nvidia() } else { None };
let amd = pin_amdgpu();
state.guard = if nvml.is_none() && amd.is_empty() {
None // nothing pinnable (no perms / no supported GPU) — the session streams unpinned
} else {
Some(ClockGuard { nvml, amd })
};
}
SessionClockPin { counted: true }
}
impl Drop for SessionClockPin {
fn drop(&mut self) {
if !self.counted {
return;
}
// Take the guard out under the lock but drop it *outside* — releasing the pin does NVML +
// sysfs I/O we don't want to hold the refcount lock across.
let release = {
let mut state = pin_refcount().lock().unwrap();
state.live = state.live.saturating_sub(1);
if state.live == 0 {
state.guard.take()
} else {
None
}
};
drop(release);
} }
Some(ClockGuard { nvml, amd })
} }
/// Force every amdgpu card's DPM performance level to `high` for the session — the encode-latency /// Force every amdgpu card's DPM performance level to `high` for the session — the encode-latency
@@ -238,7 +313,7 @@ fn pin_amdgpu() -> Vec<AmdPin> {
card = %name, card = %name,
was = %prev, was = %prev,
"amdgpu performance level pinned to high (encode clock sag removed) — \ "amdgpu performance level pinned to high (encode clock sag removed) — \
restored on host exit" restored when the last client disconnects"
); );
pins.push(AmdPin { pins.push(AmdPin {
path, path,
@@ -327,7 +402,8 @@ fn pin_nvidia() -> Option<NvmlPin> {
} }
tracing::info!( tracing::info!(
devices = pinned.len(), devices = pinned.len(),
"NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released on host exit" "NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released when the last \
client disconnects"
); );
Some(NvmlPin { nvml, pinned }) Some(NvmlPin { nvml, pinned })
} }
+37 -8
View File
@@ -246,14 +246,17 @@ fn real_main() -> Result<()> {
crate::capture::dxgi::install_gpu_pref_hook(); crate::capture::dxgi::install_gpu_pref_hook();
} }
// NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile and, // NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile. The
// under PUNKTFUNK_PIN_CLOCKS, hold the NVML core-clock floor for the host lifetime (reset on // vendor clock *pin* (PUNKTFUNK_PIN_CLOCKS) is no longer held for the host lifetime — it is
// exit via the guard's Drop). No-op off NVIDIA / on the tool subcommands. // armed per live client via `gpuclocks::session_pin()` on both streaming planes, so idle clocks
// are left alone while nobody is connected. No-op off NVIDIA / on the tool subcommands.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
let _nv_clocks = match args.first().map(String::as_str) { if matches!(
Some("serve") | Some("punktfunk1-host") => gpuclocks::on_host_start(), args.first().map(String::as_str),
_ => None, Some("serve") | Some("punktfunk1-host")
}; ) {
gpuclocks::on_host_start();
}
match args.first().map(String::as_str) { match args.first().map(String::as_str) {
// The host: the native punktfunk/1 plane + management API by default (secure), and — with // The host: the native punktfunk/1 plane + management API by default (secure), and — with
@@ -322,7 +325,33 @@ fn real_main() -> Result<()> {
// `PUNKTFUNK_HDR_SHADER_P010` colour math without green-screening a live HDR stream. Prints // `PUNKTFUNK_HDR_SHADER_P010` colour math without green-screening a live HDR stream. Prints
// PASS/FAIL + max Y/Cb/Cr error. // PASS/FAIL + max Y/Cb/Cr error.
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
Some("hdr-p010-selftest") => crate::capture::dxgi::hdr_p010_selftest(), Some("hdr-p010-selftest") => {
// Optional args: a `WxH` size (default 64x64 — pass the real capture size: heights
// like 1080 are NOT 16-aligned and exercise a different driver path) and a GPU
// vendor (`intel`|`nvidia`|`amd` — dual-GPU boxes otherwise test the default
// adapter, which may not be the one that encodes).
let mut size = (64u32, 64u32);
let mut vendor = None;
for a in args.iter().skip(2) {
match a.as_str() {
"intel" => vendor = Some(0x8086),
"nvidia" => vendor = Some(0x10de),
"amd" => vendor = Some(0x1002),
s => {
let parsed = s
.split_once('x')
.and_then(|(w, h)| Some((w.parse().ok()?, h.parse().ok()?)));
match parsed {
Some(wh) => size = wh,
None => anyhow::bail!(
"hdr-p010-selftest: unrecognized arg {s:?} (want WxH or intel|nvidia|amd)"
),
}
}
}
}
crate::capture::dxgi::hdr_p010_selftest_at(size.0, size.1, vendor)
}
// Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently // Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently
// in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for // in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for
// HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my // HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my
+55 -6
View File
@@ -76,6 +76,8 @@ mod handshake;
/// The mid-stream control task (plan §W1); `serve_session` spawns `control::run` after the /// The mid-stream control task (plan §W1); `serve_session` spawns `control::run` after the
/// handshake to multiplex renegotiation / speed-test control messages onto the data-plane channels. /// handshake to multiplex renegotiation / speed-test control messages onto the data-plane channels.
mod control; mod control;
/// Cursor-forward channel (M2): the encode loop's shape/state emission.
mod cursor_fwd;
/// The capture→encode→send data plane (plan §W1); `serve_session` dispatches the synthetic or /// The capture→encode→send data plane (plan §W1); `serve_session` dispatches the synthetic or
/// virtual source here (`synthetic_stream` / `virtual_stream`) and hands the latter a /// virtual source here (`synthetic_stream` / `virtual_stream`) and hands the latter a
@@ -439,7 +441,11 @@ pub(crate) async fn serve(
) )
.await .await
{ {
Ok(()) => tracing::info!(%peer, "session complete"), Ok(Served::Session) => tracing::info!(%peer, "session complete"),
Ok(Served::ProbeClose) => tracing::debug!(
%peer,
"closed before the control handshake (reachability probe)"
),
Err(e) => { Err(e) => {
tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error") tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error")
} }
@@ -629,6 +635,15 @@ type AudioCapSlot = Arc<std::sync::Mutex<Option<Box<dyn crate::audio::AudioCaptu
/// connection (the host stops waiting at once). /// connection (the host stops waiting at once).
const PENDING_APPROVAL_WAIT: std::time::Duration = std::time::Duration::from_secs(180); const PENDING_APPROVAL_WAIT: std::time::Duration = std::time::Duration::from_secs(180);
/// How a served connection ended. A peer that completes the QUIC handshake and closes cleanly
/// (code 0) without ever opening the control stream is a reachability probe (the clients'
/// hosts-page "online" pips / `--reachable`) or an abandoned connect — routine, and logged
/// quietly: as a WARN it buried the real failures in a wake-on-LAN triage log.
enum Served {
Session,
ProbeClose,
}
/// One client session: handshake → input/audio planes → data plane until done/disconnect. /// One client session: handshake → input/audio planes → data plane until done/disconnect.
/// Everything torn down on return (RAII: virtual output, encoder, threads via channel close). /// Everything torn down on return (RAII: virtual output, encoder, threads via channel close).
/// A connection whose first message is a PairRequest runs the pairing ceremony instead. /// A connection whose first message is a PairRequest runs the pairing ceremony instead.
@@ -651,14 +666,23 @@ async fn serve_session(
// parked knock can't hold a streaming slot. `sem` is the pool it re-acquires from. // parked knock can't hold a streaming slot. `sem` is the pool it re-acquires from.
mut permit: tokio::sync::OwnedSemaphorePermit, mut permit: tokio::sync::OwnedSemaphorePermit,
sem: Arc<tokio::sync::Semaphore>, sem: Arc<tokio::sync::Semaphore>,
) -> Result<()> { ) -> Result<Served> {
let peer = conn.remote_address(); let peer = conn.remote_address();
// First message decides what this connection is: a pairing ceremony or a session. // First message decides what this connection is: a pairing ceremony or a session.
let (mut send, mut recv) = tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi()) let (mut send, mut recv) = match tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi())
.await .await
.map_err(|_| anyhow!("control stream timeout"))? .map_err(|_| anyhow!("control stream timeout"))?
.context("accept control stream")?; {
// A clean close before any control stream: a reachability probe / abandoned connect,
// not a failed session (see [`Served::ProbeClose`]).
Err(quinn::ConnectionError::ApplicationClosed(ref ac))
if ac.error_code == quinn::VarInt::from_u32(0) =>
{
return Ok(Served::ProbeClose);
}
r => r.context("accept control stream")?,
};
let first = tokio::time::timeout(HANDSHAKE_TIMEOUT, io::read_msg(&mut recv)) let first = tokio::time::timeout(HANDSHAKE_TIMEOUT, io::read_msg(&mut recv))
.await .await
.map_err(|_| anyhow!("first message timeout"))??; .map_err(|_| anyhow!("first message timeout"))??;
@@ -709,7 +733,9 @@ async fn serve_session(
} }
*last = Some(std::time::Instant::now()); *last = Some(std::time::Instant::now());
} }
return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin).await; return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin)
.await
.map(|()| Served::Session);
} }
// Pairing gate for a session Hello (a PairRequest was handled above). Lifted OUT of the // Pairing gate for a session Hello (a PairRequest was handled above). Lifted OUT of the
@@ -919,6 +945,15 @@ async fn serve_session(
// accepted ack as "the active mode is now X" and fixes itself; old clients just log it. // accepted ack as "the active mode is now X" and fixes itself; old clients just log it.
let (reconfig_result_tx, reconfig_result_rx) = let (reconfig_result_tx, reconfig_result_rx) =
tokio::sync::mpsc::unbounded_channel::<Reconfigured>(); tokio::sync::mpsc::unbounded_channel::<Reconfigured>();
// Cursor-forward bridge (M2): the encode loop diffs each frame's cursor serial and hands
// changed SHAPES here; the control task (the control stream's sole writer) sends them.
// Same shape as `probe_result_tx`. Wired even when the channel wasn't negotiated — it
// just never fires then.
let (cursor_shape_tx, cursor_shape_rx) =
tokio::sync::mpsc::unbounded_channel::<punktfunk_core::quic::CursorShape>();
// Negotiated cursor forwarding: MUST match the HOST_CAP_CURSOR bit the Welcome advertised
// (handshake::cursor_forward is the single predicate both read).
let cursor_forward = handshake::cursor_forward(hello.client_caps, compositor);
// Adaptive FEC: the control task maps each client LossReport to a recovery percent and publishes // Adaptive FEC: the control task maps each client LossReport to a recovery percent and publishes
// it here; the data-plane send loop reads + applies it per frame. Disabled (pinned) when // it here; the data-plane send loop reads + applies it per frame. Disabled (pinned) when
// PUNKTFUNK_FEC_PCT is set. Seeded with the session's starting FEC so it's a no-op until a report. // PUNKTFUNK_FEC_PCT is set. Seeded with the session's starting FEC so it's a no-op until a report.
@@ -954,6 +989,7 @@ async fn serve_session(
probe_tx, probe_tx,
probe_result_rx, probe_result_rx,
reconfig_result_rx, reconfig_result_rx,
cursor_shape_rx,
clip_enabled, clip_enabled,
clip, clip,
)); ));
@@ -1175,6 +1211,13 @@ async fn serve_session(
launch: hello.launch.clone(), launch: hello.launch.clone(),
plane: crate::events::Plane::Native, plane: crate::events::Plane::Native,
}); });
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock floor for
// as long as THIS session streams, refcounted with every other live session across both planes.
// RAII like the marker above — armed on the first live client, released when the last one
// disconnects, so idle clocks aren't pinned while nobody is connected. No-op off Linux / when
// the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
// The session's launch, threaded into the data plane. Windows carries the store-qualified id // The session's launch, threaded into the data plane. Windows carries the store-qualified id
// (spawned into the interactive user session once capture is live); other hosts resolve the id // (spawned into the interactive user session once capture is live); other hosts resolve the id
// to its shell command HERE against the host's own library — a client can only ever pick an // to its shell command HERE against the host's own library — a client can only ever pick an
@@ -1332,6 +1375,8 @@ async fn serve_session(
fec_target: fec_target_dp, fec_target: fec_target_dp,
conn: conn_stream, conn: conn_stream,
timing_conn, timing_conn,
cursor_forward,
cursor_shape_tx,
probe_seq, probe_seq,
streamed_au, streamed_au,
stats: stats_dp, stats: stats_dp,
@@ -1392,7 +1437,7 @@ async fn serve_session(
// host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the // host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the
// TV's gaming session back so it's the default when no one is streaming. // TV's gaming session back so it's the default when no one is streaming.
crate::vdisplay::restore_managed_session(); crate::vdisplay::restore_managed_session();
result result.map(|()| Served::Session)
} }
/// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails /// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails
@@ -1998,6 +2043,7 @@ mod tests {
0, // video_codecs (HEVC-only) 0, // video_codecs (HEVC-only)
0, // preferred_codec 0, // preferred_codec
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
None, // pin (TOFU) None, // pin (TOFU)
None, // identity (host doesn't require pairing) None, // identity (host doesn't require pairing)
@@ -2168,6 +2214,7 @@ mod tests {
0, // video_codecs (0 → HEVC-only) 0, // video_codecs (0 → HEVC-only)
0, // preferred_codec (auto) 0, // preferred_codec (auto)
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
None, // pin: TOFU — the operator's approval (not a PIN) authorizes this client None, // pin: TOFU — the operator's approval (not a PIN) authorizes this client
Some((cert, key)), Some((cert, key)),
@@ -2235,6 +2282,7 @@ mod tests {
0, // video_codecs 0, // video_codecs
0, // preferred_codec 0, // preferred_codec
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
None, None,
None, None,
@@ -2264,6 +2312,7 @@ mod tests {
0, // video_codecs 0, // video_codecs
0, // preferred_codec 0, // preferred_codec
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
Some(host_fp), Some(host_fp),
Some((cert.clone(), key.clone())), Some((cert.clone(), key.clone())),
@@ -30,6 +30,7 @@ pub(super) async fn run(
probe_tx: std::sync::mpsc::Sender<ProbeRequest>, probe_tx: std::sync::mpsc::Sender<ProbeRequest>,
mut probe_result_rx: tokio::sync::mpsc::UnboundedReceiver<ProbeResult>, mut probe_result_rx: tokio::sync::mpsc::UnboundedReceiver<ProbeResult>,
mut reconfig_result_rx: tokio::sync::mpsc::UnboundedReceiver<Reconfigured>, mut reconfig_result_rx: tokio::sync::mpsc::UnboundedReceiver<Reconfigured>,
mut cursor_shape_rx: tokio::sync::mpsc::UnboundedReceiver<punktfunk_core::quic::CursorShape>,
clip_enabled: Arc<AtomicBool>, clip_enabled: Arc<AtomicBool>,
clip: pf_clipboard::ClipCoord, clip: pf_clipboard::ClipCoord,
) { ) {
@@ -262,6 +263,15 @@ pub(super) async fn run(
break; break;
} }
} }
shape = cursor_shape_rx.recv() => {
// Cursor-forward bridge (M2): the encode loop diffed a new pointer bitmap.
// Rare (shape changes are human-paced); ≤ ~58 KiB fits the u16 frame by
// construction (cursor_fwd downscales).
let Some(shape) = shape else { break }; // data plane gone
if io::write_msg(&mut ctrl_send, &shape.encode()).await.is_err() {
break;
}
}
offer = clip_offer_rx.recv(), if !clip_offer_closed => { offer = clip_offer_rx.recv(), if !clip_offer_closed => {
// Host copied → the coordinator minted a `ClipOffer`; forward it to the client // Host copied → the coordinator minted a `ClipOffer`; forward it to the client
// (only while sync is on — a race with a just-received disable would otherwise // (only while sync is on — a race with a just-received disable would otherwise
@@ -0,0 +1,144 @@
//! Cursor-forward channel, host side (design/remote-desktop-sweep.md M2).
//!
//! When the session negotiated the cursor channel (client `CLIENT_CAP_CURSOR` met our
//! `HOST_CAP_CURSOR`), the encoder stops blending the pointer into the video
//! (`SessionPlan::cursor_blend = false`) and the encode loop forwards it out-of-band instead:
//! the SHAPE (bitmap + hotspot, rare) rides the reliable control stream via the control-task
//! bridge, per-tick STATE (position/visibility, 14 B) rides a lossy `0xD0` datagram — resent
//! every iteration so loss self-heals with no refresh timer.
use punktfunk_core::quic::{
encode_cursor_state_datagram, CursorShape, CursorState, CURSOR_SHAPE_MAX_SIDE, CURSOR_VISIBLE,
};
/// Per-session forward state, owned by the encode loop (the thread that binds frames).
pub(super) struct CursorForwarder {
/// Serial of the last shape handed to the control-task bridge (`None` = none yet).
sent_serial: Option<u64>,
/// Last visible pointer position (hotspot point, frame px) — held across hidden spans so
/// a hide still states WHERE the pointer was (the M3 reappear position).
last_pos: (i32, i32),
}
impl CursorForwarder {
pub(super) fn new() -> CursorForwarder {
CursorForwarder {
sent_serial: None,
last_pos: (0, 0),
}
}
/// Called once per encode-loop iteration with the bound frame's overlay (also on repeat
/// iterations — the state datagram is the plane's loss heal, so it goes out every tick).
/// `None` overlay = hidden pointer (or no bitmap yet): state only, `visible` clear.
pub(super) fn tick(
&mut self,
cursor: Option<&pf_frame::CursorOverlay>,
conn: &quinn::Connection,
shape_tx: &tokio::sync::mpsc::UnboundedSender<CursorShape>,
) {
let flags = match cursor {
Some(ov) => {
if self.sent_serial != Some(ov.serial) {
if let Some(shape) = shape_from_overlay(ov) {
// Bridge full ⇒ control task gone ⇒ session is tearing down anyway.
let _ = shape_tx.send(shape);
self.sent_serial = Some(ov.serial);
}
}
self.last_pos = (ov.x + ov.hot_x as i32, ov.y + ov.hot_y as i32);
CURSOR_VISIBLE
}
None => 0,
};
let state = CursorState {
serial: self.sent_serial.unwrap_or(0) as u32,
flags,
x: self.last_pos.0,
y: self.last_pos.1,
};
let _ = conn.send_datagram(encode_cursor_state_datagram(&state).into());
}
}
/// Build the wire shape from a capture overlay, integer-downscaling (nearest-neighbor) anything
/// over [`CURSOR_SHAPE_MAX_SIDE`] so the message always fits the u16-length control frame.
/// Real cursors are far under the cap — the scale path is a correctness backstop for XL
/// accessibility cursors, not a quality path. `None` on a malformed overlay (short buffer).
fn shape_from_overlay(ov: &pf_frame::CursorOverlay) -> Option<CursorShape> {
let px = (ov.w as usize).checked_mul(ov.h as usize)?.checked_mul(4)?;
if ov.w == 0 || ov.h == 0 || ov.rgba.len() < px {
return None;
}
let max = CURSOR_SHAPE_MAX_SIDE as u32;
let f = ov.w.max(ov.h).div_ceil(max).max(1);
let (w, h) = (ov.w.div_ceil(f), ov.h.div_ceil(f));
let rgba = if f == 1 {
ov.rgba.as_ref().clone()
} else {
let mut out = Vec::with_capacity((w * h * 4) as usize);
for y in 0..h {
for x in 0..w {
let (sx, sy) = ((x * f).min(ov.w - 1), (y * f).min(ov.h - 1));
let o = ((sy * ov.w + sx) * 4) as usize;
out.extend_from_slice(&ov.rgba[o..o + 4]);
}
}
out
};
Some(CursorShape {
serial: ov.serial as u32,
w: w as u16,
h: h as u16,
hot_x: (ov.hot_x / f).min(w - 1) as u16,
hot_y: (ov.hot_y / f).min(h - 1) as u16,
rgba,
})
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
fn overlay(w: u32, h: u32, hot: (u32, u32)) -> pf_frame::CursorOverlay {
pf_frame::CursorOverlay {
x: 10,
y: 20,
w,
h,
rgba: Arc::new((0..w * h * 4).map(|i| i as u8).collect()),
serial: 3,
hot_x: hot.0,
hot_y: hot.1,
}
}
#[test]
fn small_shape_passes_through() {
let s = shape_from_overlay(&overlay(32, 32, (4, 5))).unwrap();
assert_eq!((s.w, s.h, s.hot_x, s.hot_y, s.serial), (32, 32, 4, 5, 3));
assert_eq!(s.rgba.len(), 32 * 32 * 4);
// Encodes within the u16 control-frame cap.
assert!(s.encode().len() <= u16::MAX as usize);
}
#[test]
fn oversize_shape_downscales_with_hotspot() {
// 256² → f = ceil(256/120) = 3 → 86² (256.div_ceil(3)), hotspot scales with it.
let s = shape_from_overlay(&overlay(256, 256, (255, 0))).unwrap();
assert!(s.w <= CURSOR_SHAPE_MAX_SIDE && s.h <= CURSOR_SHAPE_MAX_SIDE);
assert_eq!(s.rgba.len(), s.w as usize * s.h as usize * 4);
assert!(s.hot_x < s.w && s.hot_y < s.h);
assert!(s.encode().len() <= u16::MAX as usize);
// The scaled message must decode (dims within the cap).
assert_eq!(CursorShape::decode(&s.encode()).unwrap(), s);
}
#[test]
fn short_buffer_rejected() {
let mut ov = overlay(8, 8, (0, 0));
ov.rgba = Arc::new(vec![0; 8]);
assert!(shape_from_overlay(&ov).is_none());
}
}
@@ -8,6 +8,22 @@
use super::*; use super::*;
/// Whether this session forwards the cursor out-of-band (design/remote-desktop-sweep.md M2):
/// the client asked ([`CLIENT_CAP_CURSOR`](punktfunk_core::quic::CLIENT_CAP_CURSOR)) AND the
/// capture path can deliver cursor metadata separately from the frame — today that is the
/// Linux portal `SPA_META_Cursor` path only: not gamescope (its capture paints no cursor at
/// all), not Windows (DWM composites into the IDD frame — M2c). THE single predicate: the
/// Welcome's `HOST_CAP_CURSOR` bit and the session's forwarding/blend-off wiring both read it,
/// so they can never disagree.
pub(super) fn cursor_forward(
client_caps: u8,
compositor: Option<crate::vdisplay::Compositor>,
) -> bool {
cfg!(target_os = "linux")
&& client_caps & punktfunk_core::quic::CLIENT_CAP_CURSOR != 0
&& compositor.is_some_and(|c| c != crate::vdisplay::Compositor::Gamescope)
}
/// Run the Hello→Welcome→Start negotiation. Borrows the control streams (the caller keeps them for /// Run the Hello→Welcome→Start negotiation. Borrows the control streams (the caller keeps them for
/// mid-stream renegotiation afterwards). `first` is the already-read first control message. /// mid-stream renegotiation afterwards). `first` is the already-read first control message.
#[allow(clippy::type_complexity, clippy::too_many_arguments)] #[allow(clippy::type_complexity, clippy::too_many_arguments)]
@@ -354,6 +370,28 @@ pub(super) async fn negotiate(
// just follow. // just follow.
let mut salt = [0u8; 4]; let mut salt = [0u8; 4];
rand::thread_rng().fill_bytes(&mut salt); rand::thread_rng().fill_bytes(&mut salt);
// Session AEAD: ChaCha20-Poly1305 when the client asked for it (VIDEO_CAP_CHACHA20 — the
// soft-AES armv7 targets, whose GCM decrypt caps at ~100 Mbps) and the operator
// kill-switch allows (PUNKTFUNK_CHACHA20, default on — pure rollout safety; perf-only,
// both AEADs are full-strength). The fresh-per-session discipline above applies to this
// key identically; the legacy 16-byte `key` stays independently random so nothing
// downstream ever observes an all-zero key.
let client_wants_chacha = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_CHACHA20 != 0;
let chacha = client_wants_chacha && pf_host_config::config().chacha20;
let key_chacha = chacha.then(|| {
let mut k = [0u8; 32];
rand::thread_rng().fill_bytes(&mut k);
k
});
tracing::info!(
cipher = if chacha {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
client_wants_chacha,
"session cipher"
);
let welcome = Welcome { let welcome = Welcome {
abi_version: punktfunk_core::WIRE_VERSION, abi_version: punktfunk_core::WIRE_VERSION,
udp_port, udp_port,
@@ -422,7 +460,25 @@ pub(super) async fn negotiate(
punktfunk_core::quic::HOST_CAP_CLIPBOARD punktfunk_core::quic::HOST_CAP_CLIPBOARD
} else { } else {
0 0
}
// Cursor channel granted (client asked + this capture path can deliver cursor
// metadata out of the frame) — the client turns its local renderer on ONLY when
// it sees this bit, and serve_session wires forwarding from the same predicate.
| if cursor_forward(hello.client_caps, compositor) {
punktfunk_core::quic::HOST_CAP_CURSOR
} else {
0
}, },
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
// pre-cipher wire form. The host's own data plane picks the cipher up via
// `welcome.session_config` — no other host change.
cipher: if chacha {
punktfunk_core::quic::CIPHER_CHACHA20_POLY1305
} else {
punktfunk_core::quic::CIPHER_AES_128_GCM
},
key_chacha,
}; };
io::write_msg(send, &welcome.encode()).await?; io::write_msg(send, &welcome.encode()).await?;
bringup.mark("welcome"); bringup.mark("welcome");
+101 -13
View File
@@ -938,6 +938,14 @@ pub(super) struct SessionContext {
/// thread emits one 0xCF datagram per AU (capture→sent µs) on it, so the client can split its /// thread emits one 0xCF datagram per AU (capture→sent µs) on it, so the client can split its
/// `host+network` latency stage. `None` = older client, no emission. /// `host+network` latency stage. `None` = older client, no emission.
pub(super) timing_conn: Option<quinn::Connection>, pub(super) timing_conn: Option<quinn::Connection>,
/// The session negotiated the cursor channel (design/remote-desktop-sweep.md M2 —
/// `handshake::cursor_forward`): the encoder does NOT blend the pointer into the video;
/// the encode loop forwards shape (via `cursor_shape_tx`) + per-tick `0xD0` state instead.
pub(super) cursor_forward: bool,
/// SHAPE bridge to the control task (the control stream's sole writer) — mirrors
/// `probe_result_tx`. Inert when `cursor_forward` is false.
pub(super) cursor_shape_tx:
tokio::sync::mpsc::UnboundedSender<punktfunk_core::quic::CursorShape>,
/// The client advertised [`punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ`]: speed-test bursts may /// The client advertised [`punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ`]: speed-test bursts may
/// run mid-session in the probe index space (its reassembler keeps a separate probe window). /// run mid-session in the probe index space (its reassembler keeps a separate probe window).
/// `false` = older client whose single-window reassembler would drop probe-space frames as /// `false` = older client whose single-window reassembler would drop probe-space frames as
@@ -987,7 +995,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
ctx.bit_depth, ctx.bit_depth,
ctx.chroma, ctx.chroma,
ctx.codec, ctx.codec,
ctx.compositor != pf_vdisplay::Compositor::Gamescope, // Blend the pointer into the video only where the capture HAS one (not gamescope) AND
// the client is not drawing it locally (the M2 cursor channel — blending too would
// show it twice).
ctx.compositor != pf_vdisplay::Compositor::Gamescope && !ctx.cursor_forward,
); );
// PyroWave rides the datagram-aligned wire mode (§4.4): every encoder this session opens // PyroWave rides the datagram-aligned wire mode (§4.4): every encoder this session opens
// packetizes at the negotiated shard payload, so a lost datagram costs blocks, not frames. // packetizes at the negotiated shard payload, so a lost datagram costs blocks, not frames.
@@ -1021,6 +1032,8 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
fec_target, fec_target,
conn, conn,
timing_conn, timing_conn,
cursor_forward,
cursor_shape_tx,
probe_seq, probe_seq,
streamed_au, streamed_au,
stats, stats,
@@ -1034,6 +1047,13 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// reverts to whole-AU sends without touching the encoder's slicing knobs). The third gate — // reverts to whole-AU sends without touching the encoder's slicing knobs). The third gate —
// whether the ENCODER actually chunks — is dynamic (`supports_chunked_poll`, per AU). // whether the ENCODER actually chunks — is dynamic (`supports_chunked_poll`, per AU).
let streamed_wire = streamed_au && std::env::var("PUNKTFUNK_STREAMED_AU").as_deref() != Ok("0"); let streamed_wire = streamed_au && std::env::var("PUNKTFUNK_STREAMED_AU").as_deref() != Ok("0");
// Cursor-forward state (M2): shape-serial diffing + the per-tick 0xD0 state send. The
// encoder was told not to blend (SessionPlan above), so from the first frame the client's
// locally-drawn cursor is the only one.
let mut cursor_fwd = cursor_forward.then(super::cursor_fwd::CursorForwarder::new);
if cursor_forward {
tracing::info!("cursor channel negotiated — forwarding shape/state, encoder blend off");
}
if streamed_wire { if streamed_wire {
tracing::info!( tracing::info!(
"client accepts streamed AUs (VIDEO_CAP_STREAMED_AU) — chunked encoder output \ "client accepts streamed AUs (VIDEO_CAP_STREAMED_AU) — chunked encoder output \
@@ -1111,6 +1131,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan, plan,
&quit, &quit,
&stop, &stop,
8,
Some(bringup.as_ref()), Some(bringup.as_ref()),
)?; )?;
// Setup done — the IDD-push setup lock releases as the guard leaves this arm's scope, // Setup done — the IDD-push setup lock releases as the guard leaves this arm's scope,
@@ -1426,6 +1447,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan, plan,
&quit, &quit,
&stop, &stop,
8,
None, None,
)?; )?;
Ok((new_vd, pipe)) Ok((new_vd, pipe))
@@ -1522,6 +1544,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
bit_depth, bit_depth,
plan, plan,
&quit, &quit,
// No first-frame shortening here: this direct call has no retry wrapper to
// absorb an early bail, and the resize source is a live compositor (the
// takeover race doesn't apply) — keep the patient default.
None,
Some(resize_trace.as_ref()), Some(resize_trace.as_ref()),
) { ) {
Ok(next_pipe) => { Ok(next_pipe) => {
@@ -1878,7 +1904,15 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// connected, frozen on the last frame, and the stream resumes when the new output // connected, frozen on the last frame, and the stream resumes when the new output
// appears — no reconnect. // appears — no reconnect.
const REBUILD_BUDGET: std::time::Duration = std::time::Duration::from_secs(40); const REBUILD_BUDGET: std::time::Duration = std::time::Duration::from_secs(40);
let rebuild_deadline = std::time::Instant::now() + REBUILD_BUDGET; // Attach-only holdoff: for the first seconds after a capture loss the session
// detection can be STALE (the new session isn't up yet), and a rebuild acting on
// a stale "Gaming" answer restarts gamescope-session.target — which on SteamOS
// steals the seat back from the session the user just switched to (observed
// live). While the holdoff lasts, builds run under a vdisplay rebuild-probe
// scope: attach to live outputs only, never stop/relaunch/take over sessions.
const PROBE_HOLDOFF: std::time::Duration = std::time::Duration::from_secs(4);
let loss_at = std::time::Instant::now();
let rebuild_deadline = loss_at + REBUILD_BUDGET;
let (new_cap, new_enc, new_frame, new_interval, new_node_id, new_display_gen) = loop { let (new_cap, new_enc, new_frame, new_interval, new_node_id, new_display_gen) = loop {
// Follow the active session unless an explicit PUNKTFUNK_COMPOSITOR pin forbids // Follow the active session unless an explicit PUNKTFUNK_COMPOSITOR pin forbids
// retargeting (then we stick to the pinned backend and just rebuild it). // retargeting (then we stick to the pinned backend and just rebuild it).
@@ -1912,6 +1946,8 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
} }
} }
} }
let _probe = (loss_at.elapsed() < PROBE_HOLDOFF)
.then(crate::vdisplay::rebuild_probe_scope);
match build_pipeline_with_retry( match build_pipeline_with_retry(
&mut vd, &mut vd,
cur_mode, cur_mode,
@@ -1920,6 +1956,16 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan, plan,
&quit, &quit,
&stop, &stop,
// 1, not 8: this loop re-detects the active session per iteration — short
// inner cycles are what let it FOLLOW a session switch instead of burning
// retries against a compositor that no longer exists. One attempt per
// cycle also keeps every probe on the SHORT first-frame window (attempt 1
// = 2.5 s): a patient 10 s attempt here just waits on a stale backend
// (observed live: it made a Game→Desktop switch 20 s instead of ~9 —
// the winning KWin rebuild took 0.7 s once detection caught up). The
// slow-new-session case is the OUTER loop's job (40 s budget, fresh
// 2.5 s probes until the new compositor delivers).
1,
None, None,
) { ) {
Ok(p) => break p, Ok(p) => break p,
@@ -1932,6 +1978,9 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
} }
tracing::warn!(error = %format!("{e2:#}"), tracing::warn!(error = %format!("{e2:#}"),
"capture lost — new session not up yet, retrying"); "capture lost — new session not up yet, retrying");
// Probe failures are instant (attach-only bail) — pace the loop so
// re-detection runs at ~2 Hz instead of spinning.
std::thread::sleep(std::time::Duration::from_millis(500));
} }
} }
}; };
@@ -1956,6 +2005,12 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
); );
} }
} }
// Cursor channel (M2): every iteration — new frame OR repeat — states the pointer
// (self-healing under datagram loss) and forwards a changed shape via the control
// bridge. `frame` is the newest bound frame either way.
if let Some(fwd) = cursor_fwd.as_mut() {
fwd.tick(frame.cursor.as_ref(), &conn, &cursor_shape_tx);
}
if perf && diag_at.elapsed() >= std::time::Duration::from_secs(2) { if perf && diag_at.elapsed() >= std::time::Duration::from_secs(2) {
let secs = diag_at.elapsed().as_secs_f64(); let secs = diag_at.elapsed().as_secs_f64();
tracing::info!( tracing::info!(
@@ -2655,6 +2710,7 @@ pub(super) fn prepare_display(
plan, plan,
quit, quit,
stop, stop,
8,
Some(trace), Some(trace),
)?; )?;
Ok(PreparedDisplay { vd, pipeline }) Ok(PreparedDisplay { vd, pipeline })
@@ -2679,15 +2735,22 @@ fn build_pipeline_with_retry(
plan: crate::session_plan::SessionPlan, plan: crate::session_plan::SessionPlan,
quit: &Arc<AtomicBool>, quit: &Arc<AtomicBool>,
stop: &Arc<AtomicBool>, stop: &Arc<AtomicBool>,
// Retry budget: 8 everywhere EXCEPT the capture-loss rebuild (2). That path wraps this call
// in its own outer loop that RE-DETECTS the active session between calls — during a
// Gaming↔Desktop switch the old compositor is simply gone, so burning 8 attempts (~13 s)
// against its dead socket only delays following the box to the session that replaced it
// (observed live: a Desktop→Gaming switch spent 13 of its 27 s retrying gone-KWin).
max_attempts: u32,
// Transition trace (P0.1): `Some` for the traced builds (bring-up, resize); each stage stamps // Transition trace (P0.1): `Some` for the traced builds (bring-up, resize); each stage stamps
// once (first crossing) so the retry loop can pass it through unconditionally. // once (first crossing) so the retry loop can pass it through unconditionally.
trace: Option<&crate::bringup::Trace>, trace: Option<&crate::bringup::Trace>,
) -> Result<Pipeline> { ) -> Result<Pipeline> {
// ~10s first-frame wait per attempt. 8 gives a ~90s budget for the SLOW case: a host-managed // ~10s first-frame wait per attempt (attempt 1: see FIRST_ATTEMPT_FRAME_BUDGET below). 8
// gamescope session cold-starting Steam Big Picture (the SteamOS/Bazzite takeover) can take // gives a ~80s budget for the SLOW case: a host-managed gamescope session cold-starting Steam
// 30-60s to produce its first frame, and a first-connect timeout would tear down the warm // Big Picture (the SteamOS/Bazzite takeover) can take 30-60s to produce its first frame, and
// session (forcing another cold start on reconnect). A genuinely permanent failure still fails // a first-connect timeout would tear down the warm session (forcing another cold start on
// fast via `is_permanent_build_error`; only transient "no frame yet" retries consume the budget. // reconnect). A genuinely permanent failure still fails fast via `is_permanent_build_error`;
// only transient "no frame yet" retries consume the budget.
// IDD-push only: HOLD one monitor lease across all build attempts. A failed attempt's capturer // IDD-push only: HOLD one monitor lease across all build attempts. A failed attempt's capturer
// drop releases ITS lease, but this held lease keeps the shared monitor Active (refs >= 1), so the // drop releases ITS lease, but this held lease keeps the shared monitor Active (refs >= 1), so the
// next attempt's `vd.create` JOINS it (refcount++) instead of finding it Lingering and tripping the // next attempt's `vd.create` JOINS it (refcount++) instead of finding it Lingering and tripping the
@@ -2705,9 +2768,16 @@ fn build_pipeline_with_retry(
} else { } else {
None None
}; };
const MAX_ATTEMPTS: u32 = 8; // Attempt 1 waits only briefly for the first frame: a PipeWire stream connected while
// gamescope re-initializes its headless takeover negotiates a format and reaches `Streaming`
// but never receives a buffer — a FRESH connect then delivers within ~0.5 s (observed on
// SteamOS: every gamescope bring-up burned the full 10 s on attempt 1, then attempt 2 got
// frames instantly → 17 s bring-ups). Healthy compositors deliver the first frame well inside
// this window (KWin ~0.3 s), and the genuinely-slow cold start above still gets the patient
// 10 s window on every later attempt.
const FIRST_ATTEMPT_FRAME_BUDGET: std::time::Duration = std::time::Duration::from_millis(2500);
let mut backoff = std::time::Duration::from_millis(500); let mut backoff = std::time::Duration::from_millis(500);
for attempt in 1..=MAX_ATTEMPTS { for attempt in 1..=max_attempts {
// The client is gone (connection closed → `stop`): every further attempt only churns the // The client is gone (connection closed → `stop`): every further attempt only churns the
// box for a session no one is watching — on a Bazzite takeover that means SIGKILLing and // box for a session no one is watching — on a Bazzite takeover that means SIGKILLing and
// relaunching the box's Steam session once per attempt for minutes (the .181 storm // relaunching the box's Steam session once per attempt for minutes (the .181 storm
@@ -2719,7 +2789,17 @@ fn build_pipeline_with_retry(
attempt - 1 attempt - 1
); );
} }
match build_pipeline(vd, mode, bitrate_kbps, bit_depth, plan, quit, trace) { let first_frame_budget = (attempt == 1).then_some(FIRST_ATTEMPT_FRAME_BUDGET);
match build_pipeline(
vd,
mode,
bitrate_kbps,
bit_depth,
plan,
quit,
first_frame_budget,
trace,
) {
Ok(pipe) => { Ok(pipe) => {
if attempt > 1 { if attempt > 1 {
tracing::info!(attempt, "pipeline up after retry"); tracing::info!(attempt, "pipeline up after retry");
@@ -2729,7 +2809,7 @@ fn build_pipeline_with_retry(
Err(e) => { Err(e) => {
let chain = format!("{e:#}"); let chain = format!("{e:#}");
let permanent = is_permanent_build_error(&chain); let permanent = is_permanent_build_error(&chain);
if permanent || attempt == MAX_ATTEMPTS { if permanent || attempt == max_attempts {
let why = if permanent { let why = if permanent {
"permanent" "permanent"
} else { } else {
@@ -2741,7 +2821,7 @@ fn build_pipeline_with_retry(
} }
tracing::warn!( tracing::warn!(
attempt, attempt,
max = MAX_ATTEMPTS, max = max_attempts,
backoff_ms = backoff.as_millis() as u64, backoff_ms = backoff.as_millis() as u64,
error = %chain, error = %chain,
"pipeline build failed — retrying" "pipeline build failed — retrying"
@@ -2802,6 +2882,10 @@ fn build_pipeline(
bit_depth: u8, bit_depth: u8,
plan: crate::session_plan::SessionPlan, plan: crate::session_plan::SessionPlan,
quit: &Arc<AtomicBool>, quit: &Arc<AtomicBool>,
// First-frame wait override (`None` = the backend's default 10 s): the retry loop shortens
// its FIRST attempt so a stream stuck in the gamescope takeover race fails over to the
// reconnect that fixes it (see FIRST_ATTEMPT_FRAME_BUDGET in `build_pipeline_with_retry`).
first_frame_budget: Option<std::time::Duration>,
// Transition trace (P0.1): stamps the build's stages (display acquire, capture attach, first // Transition trace (P0.1): stamps the build's stages (display acquire, capture attach, first
// frame, encoder open) into the bring-up/resize timeline. `None` on untraced rebuilds. // frame, encoder open) into the bring-up/resize timeline. `None` on untraced rebuilds.
trace: Option<&crate::bringup::Trace>, trace: Option<&crate::bringup::Trace>,
@@ -2857,7 +2941,11 @@ fn build_pipeline(
t.mark("capture_attached"); t.mark("capture_attached");
} }
capturer.set_active(true); capturer.set_active(true);
let frame = match capturer.next_frame().context("first frame") { let first = match first_frame_budget {
Some(budget) => capturer.next_frame_within(budget),
None => capturer.next_frame(),
};
let frame = match first.context("first frame") {
Ok(f) => f, Ok(f) => f,
Err(e) => { Err(e) => {
// A reused kept display was dead — invalidate it so the next attempt creates fresh (A2). // A reused kept display was dead — invalidate it so the next attempt creates fresh (A2).
@@ -179,6 +179,13 @@ impl SessionPlan {
// Vulkan device; on Linux the capture facade flips the zero-copy policy to the // Vulkan device; on Linux the capture facade flips the zero-copy policy to the
// raw-dmabuf passthrough (see above). // raw-dmabuf passthrough (see above).
pyrowave: self.codec == crate::encode::Codec::PyroWave, pyrowave: self.codec == crate::encode::Codec::PyroWave,
// Producer-native NV12 (gamescope) is consumable only by the Linux Vulkan Video
// backend — resolved HERE from the plan's codec so the capturer never reaches back
// into encode (the same one-way edge as `gpu` above).
#[cfg(target_os = "linux")]
nv12_native: crate::encode::linux_native_nv12_ok(self.codec),
#[cfg(not(target_os = "linux"))]
nv12_native: false,
} }
} }
} }
+60 -9
View File
@@ -168,12 +168,29 @@ The canonical "decide, don't just observe" pattern — approve pairing from your
## Recipe: full controller passthrough (VirtualHere) ## Recipe: full controller passthrough (VirtualHere)
To get a controller's *native* features on the host — DualSense gyro, touchpad, adaptive To get a controller's *native* features on the host — DualSense gyro, touchpad, adaptive
triggers, USB rumble — instead of the emulated pad, share the physical device from the couch with triggers, USB rumble — instead of the emulated pad, hand the physical device from the couch to the
[VirtualHere](https://www.virtualhere.com/) (USB-over-IP) and bind it to the host only while a host over [VirtualHere](https://www.virtualhere.com/) (USB-over-IP), bound only while a client is
client is connected. The couch runs the VirtualHere **server** (sharing the pad); the host runs connected so the couch keeps its own controller the rest of the time.
the VirtualHere **client** and this automation drives its `-t` IPC.
Zero-code, bracketed on the stream with two hooks: **The two sides.** VirtualHere is a server/client pair, and you run both:
- **Server — on the couch** (where the pad is physically plugged in). Run the VirtualHere USB
Server there; it shares the pad on the LAN. Leave it running.
- **Client — on the host** (where the game and this automation run). Install the VirtualHere
Client (as a service, or the tray app). It auto-discovers the couch's shared pad on the same
LAN; across subnets, add it once with `<VH_CLIENT> -t "MANUAL HUB ADD,<couch-ip>:7575"`. The
client binary is `vhclientx86_64` on Linux, `vhui64.exe` on Windows, `vhclientosx` on macOS,
`vhclientarm64` on ARM Linux.
The client's `-t` flag is a one-shot IPC to the already-running client: `-t LIST` prints every
visible device with its address (`server.port`, e.g. `couch-deck.11`); `-t "USE,<addr>"` mounts it
onto the host; `-t "STOP USING,<addr>"` hands it back. The automation just brackets
`USE` / `STOP USING` around a session.
### Zero-code: two hooks
Bracket it on the stream with two [hooks](#hooks-hooksjson) — mount when video starts, release
when it stops. This is all most setups need:
```json ```json
{ {
@@ -184,8 +201,42 @@ Zero-code, bracketed on the stream with two hooks:
} }
``` ```
`couch-deck.11` is the device's VirtualHere address (`vhclientx86_64 -t LIST`); same-LAN setups `couch-deck.11` is the device's VirtualHere address from `vhclientx86_64 -t LIST`.
auto-discover it, otherwise `MANUAL HUB ADD,<couch-ip>:7575` once. For a version that resolves the
device by name, filters to one couch, and releases the pad on a clean shutdown, see the ### Scripted: resolve by name, release on shutdown
The hooks above hard-code the address, and can strand the pad on the host if it's stopped
mid-stream (the `stream.stopped` hook never fires). The
[`virtualhere-dualsense.ts`](https://git.unom.io/unom/punktfunk/src/branch/main/sdk/examples/virtualhere-dualsense.ts) [`virtualhere-dualsense.ts`](https://git.unom.io/unom/punktfunk/src/branch/main/sdk/examples/virtualhere-dualsense.ts)
SDK example. SDK example is the robust version: it resolves the device by **name substring** (survives the
address changing), can filter to **one couch** in a multi-client setup, and **releases the pad on
a clean shutdown** (`systemctl stop`, ^C) so the couch always gets its controller back.
It's a standalone [`@punktfunk/host` script](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
— run it in its own directory. The one edit is its import: the in-repo copy imports from
`../src/index.js`; outside the repo it's the published package:
```sh
mkdir ~/punktfunk-scripts && cd ~/punktfunk-scripts
bun init -y
bun add @punktfunk/host # point the @punktfunk scope at the registry first — see the SDK README
# save the example as virtualhere-dualsense.ts, and change its first import to the package:
# - import { connect } from "../src/index.js";
# + import { connect } from "@punktfunk/host";
VH_DEVICE=DualSense bun virtualhere-dualsense.ts
```
`VH_DEVICE` is required — a VirtualHere address (`couch-deck.11`) or a device-name substring
(`DualSense`). Optional: `VH_CLIENT` overrides the client binary (default `vhclientx86_64`);
`VH_ONLY_CLIENT` binds only for one punktfunk client label. Running on the host box the SDK needs
no token or URL — it reads the host's loopback credentials itself (see
[Connection resolution](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#connection-resolution)).
Keep it running as a
[systemd user unit](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
(its default `SIGTERM` triggers the script's own release step — so `systemctl stop` hands the pad
back), or drop it under the [scripting runner](/docs/plugins) with your other units.
> The example brackets on `client.connected` / `client.disconnected` — the pad returns to the
> couch the moment they disconnect. Switch to `stream.started` / `stream.stopped` if you'd rather
> pass it through only while video is actually flowing; both are noted in the file's header.
+1
View File
@@ -90,6 +90,7 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
| `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. | | `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. |
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. | | `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. |
| `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. | | `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
| `PUNKTFUNK_CHACHA20` | `1` · `0` *(default on)* | ChaCha20-Poly1305 session encryption for clients without hardware AES (old ARM TVs, e.g. webOS), lifting their ~100 Mbps software-AES decrypt ceiling. **On by default** on the host; a session uses it only when the client requests it — everyone else stays on AES-GCM. Purely a performance choice (both ciphers are full-strength); set `0` to force AES-GCM for all sessions. |
| `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. | | `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. |
| `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. | | `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. |
| `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. | | `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. |
+18 -2
View File
@@ -58,8 +58,10 @@ It is idempotent — safe to re-run. In one pass it:
1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain, 1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain,
2. builds `punktfunk-host` (and the web console), 2. builds `punktfunk-host` (and the web console),
3. writes config to `~/.config/punktfunk/` (a generated web-console login password), 3. writes config to `~/.config/punktfunk/` (a generated web-console login password),
4. raises the UDP socket buffers to 32 MB and adds you to the `input` group (needs `sudo`; skipped 4. raises the UDP socket buffers to 32 MB, installs the gamepad udev rule + the `vhci-hcd` autoload
with a warning if unavailable), and adds you to the `input` group (virtual gamepads / **native Steam Deck controller passthrough**),
and seeds the KDE RemoteDesktop grant for Desktop-mode input — this step **prompts for your `sudo`
password** (a stock Steam Deck requires one; without it gamepad passthrough and the UDP tuning are skipped),
5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger, 5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger,
so they run without a login session). so they run without a login session).
@@ -82,6 +84,14 @@ When it finishes it prints the web-console URL and how to pair.
> If you only ever use native clients, install with `--no-gamestream` for a host with no GameStream > If you only ever use native clients, install with `--no-gamestream` for a host with no GameStream
> surface at all. > surface at all.
> **First install — reboot once before streaming.** KWin only authorizes Desktop-mode screen capture
> on a fresh session, and the new `input` group (native Steam Deck controller passthrough) only takes
> effect on a new login — so after the **first** install, **reboot the Deck** (a re-run that changes
> nothing doesn't need it). Streaming **Game Mode** with a generic Xbox pad works right away; **Desktop
> capture and the native Steam Deck controller need the reboot.** If a client connects and every
> session ends with `KWin does not expose zkde_screencast_unstable_v1` or the pad shows up as an Xbox
> 360 controller, you haven't rebooted yet.
## 3. Pair a device ## 3. Pair a device
By default the host **requires PIN pairing** (secure). Two ways to pair: By default the host **requires PIN pairing** (secure). Two ways to pair:
@@ -128,6 +138,12 @@ bash ~/punktfunk/scripts/steamdeck/update.sh
thrash the managed session. Pick one mode per session. thrash the managed session. Pick one mode per session.
- **Keep the device awake.** On handhelds, Game Mode auto-suspends on idle, which drops the host off - **Keep the device awake.** On handhelds, Game Mode auto-suspends on idle, which drops the host off
the network mid stream — disable auto-suspend (Settings → Power) for a headless host. the network mid stream — disable auto-suspend (Settings → Power) for a headless host.
- **Native Steam Deck controller passthrough** presents the client's pad as a real Steam Deck
controller (paddles, trackpads, gyro) via a virtual USB device — that needs the `input` group and the
`vhci-hcd` module live, so it only works **after the first-install reboot** above; until then the pad
degrades to a generic Xbox 360 controller (still fully playable). If you're streaming *to* another
Steam Deck, also set Steam Input to **Off** for Punktfunk on that Deck — see
[Stream to a Steam Deck](/docs/steam-deck).
- **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to - **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to
start after an update, just re-run `update.sh` to rebuild against the new base. start after an update, just re-run `update.sh` to rebuild against the new base.
- Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md). - Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md).
+85 -1
View File
@@ -272,7 +272,7 @@
#define INBOUND_REQ_FLAG 2147483648 #define INBOUND_REQ_FLAG 2147483648
#endif #endif
// 16-byte AEAD authentication tag appended by GCM. // 16-byte AEAD authentication tag appended by either session cipher.
#define TAG_LEN 16 #define TAG_LEN 16
// Wire tag distinguishing an input datagram from a video packet. // Wire tag distinguishing an input datagram from a video packet.
@@ -465,6 +465,20 @@
#define VIDEO_CAP_STREAMED_AU 32 #define VIDEO_CAP_STREAMED_AU 32
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
// control channel, so there is no downgrade surface.
#define VIDEO_CAP_CHACHA20 64
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] // [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder // (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -484,6 +498,28 @@
#define HOST_CAP_CLIPBOARD 2 #define HOST_CAP_CLIPBOARD 2
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::client_caps`] bit: the client renders the host cursor LOCALLY
// (design/remote-desktop-sweep.md M2). It consumes [`CursorShape`](super::control::CursorShape)
// control messages (RGBA bitmap + hotspot, cached by serial) and per-frame
// [`CursorState`](super::datagram::CursorState) `0xD0` datagrams (position/visibility), and
// draws the pointer itself — so the host must STOP compositing the cursor into the video
// (`SessionPlan.cursor_blend = false`) or the user sees it twice. Active only when the host
// answers with [`HOST_CAP_CURSOR`] (capable-and-agreed, the 444/clipboard precedent); toward
// an older or incapable host nothing changes.
#define CLIENT_CAP_CURSOR 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
// whose capture carries no cursor, and NOT Windows yet, where DWM composites into the IDD
// frame). Set only when the client asked via [`CLIENT_CAP_CURSOR`]; when both bits agree the
// host stops blending and ships [`CursorShape`](super::control::CursorShape) +
// [`CursorState`](super::datagram::CursorState) instead.
#define HOST_CAP_CURSOR 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** // [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST // encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -736,6 +772,20 @@
#define CLIP_FILE_INDEX_NONE UINT32_MAX #define CLIP_FILE_INDEX_NONE UINT32_MAX
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
#define MSG_CURSOR_SHAPE 80
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
// real cursors (typically ≤ 64 px, ≤ 96 px at HiDPI scale); the HOST downscales anything
// larger before forwarding, so the cap is invisible to clients.
#define CURSOR_SHAPE_MAX_SIDE 120
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// Datagram wire tags. Video rides UDP; everything low-rate rides QUIC datagrams, // Datagram wire tags. Video rides UDP; everything low-rate rides QUIC datagrams,
// demultiplexed by the first byte: input = [`crate::input::INPUT_MAGIC`] (0xC8, client→host), // demultiplexed by the first byte: input = [`crate::input::INPUT_MAGIC`] (0xC8, client→host),
@@ -824,6 +874,28 @@
#define HOST_TIMING_MAGIC 207 #define HOST_TIMING_MAGIC 207
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Cursor-state datagram tag, host → client (design/remote-desktop-sweep.md M2). Next tag after
// [`HOST_TIMING_MAGIC`]. Sent once per captured frame while the cursor channel is negotiated
// ([`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) ∧
// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR)) — per-frame resend makes the plane
// self-healing under loss (latest-wins, no refresh timer). The bitmap itself rides the
// reliable control stream ([`CursorShape`](super::control::CursorShape)); this 14-byte
// datagram only moves/hides the pointer.
#define CURSOR_STATE_MAGIC 208
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`CursorState::flags`] bit: the host cursor is visible.
#define CURSOR_VISIBLE 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`CursorState::flags`] bit: a host app captured/hid the pointer — the client SHOULD run
// relative/captured (M3 auto-flip; advisory, user override always wins).
#define CURSOR_RELATIVE_HINT 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// QUIC application error code a punktfunk/1 client closes the control connection with on a // QUIC application error code a punktfunk/1 client closes the control connection with on a
// **deliberate quit** (a user "stop", not a network drop). The host reads it off the connection's // **deliberate quit** (a user "stop", not a network drop). The host reads it off the connection's
@@ -855,6 +927,18 @@
#define HELLO_LAUNCH_MAX 128 #define HELLO_LAUNCH_MAX 128
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
// only one pre-cipher builds know).
#define CIPHER_AES_128_GCM 0
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
#define CIPHER_CHACHA20_POLY1305 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// Type byte of [`PairRequest`]. // Type byte of [`PairRequest`].
#define MSG_PAIR_REQUEST 16 #define MSG_PAIR_REQUEST 16
+106 -11
View File
@@ -48,6 +48,9 @@ BIN="$TARGET_DIR/release/punktfunk-host"
CONFIG="$HOME/.config/punktfunk" CONFIG="$HOME/.config/punktfunk"
UNITS="$HOME/.config/systemd/user" UNITS="$HOME/.config/systemd/user"
XRD="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}" XRD="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"
# Set when this run does something that only a fresh login picks up (input-group add, first-time
# KWin .desktop grant). Drives the loud "reboot before streaming" note in the summary.
NEED_RELOGIN=0
# --- 0. preflight ---------------------------------------------------------- # --- 0. preflight ----------------------------------------------------------
log "Preflight" log "Preflight"
@@ -66,6 +69,33 @@ fi
DISTROBOX="$(command -v distrobox)" # baked into the web unit (may be /usr/bin or ~/.local/bin) DISTROBOX="$(command -v distrobox)" # baked into the web unit (may be /usr/bin or ~/.local/bin)
ok "distrobox: $DISTROBOX" ok "distrobox: $DISTROBOX"
# --- acquire sudo up front (before the ~15-min build) ----------------------
# Steps 4-5 (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger) need root. Prompt NOW,
# not after the build — so you authorize once and walk away, and a non-interactive run fails LOUDLY
# here instead of silently skipping the tuning at the very end. A stock SteamOS 'deck' has no
# password, so sudo can't work until you set one.
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo is needed once (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger):"
if sudo -v; then
SUDO_OK=1
# keep the sudo timestamp warm across the long build so steps 4-5 don't re-prompt / expire
( while sudo -n -v 2>/dev/null; do sleep 50; done ) &
_pf_sudo_keepalive=$!
trap '[ -n "${_pf_sudo_keepalive:-}" ] && kill "$_pf_sudo_keepalive" 2>/dev/null || true' EXIT
fi
fi
if [ "$SUDO_OK" != 1 ]; then
if [ -t 0 ]; then
warn "No sudo — a stock SteamOS 'deck' account has no password. Set one and re-run: passwd"
else
warn "No TTY for the sudo prompt (non-interactive run) — system tuning + linger will be SKIPPED."
warn "Run in Konsole or an interactive 'ssh -t' session (or pre-authorize sudo) to enable them."
fi
fi
# --- 1. build container + toolchain --------------------------------------- # --- 1. build container + toolchain ---------------------------------------
log "Build container '$BOX' ($BOX_IMAGE)" log "Build container '$BOX' ($BOX_IMAGE)"
if distrobox list 2>/dev/null | awk -F'|' '{gsub(/ /,"",$2); print $2}' | grep -qx "$BOX"; then if distrobox list 2>/dev/null | awk -F'|' '{gsub(/ /,"",$2); print $2}' | grep -qx "$BOX"; then
@@ -84,7 +114,7 @@ set -e
export DEBIAN_FRONTEND=noninteractive export DEBIAN_FRONTEND=noninteractive
sudo apt-get update -qq sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends \ sudo apt-get install -y -qq --no-install-recommends \
build-essential pkg-config clang curl git ca-certificates \ build-essential pkg-config clang cmake curl git ca-certificates \
libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev libswscale-dev libavdevice-dev \ libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev libswscale-dev libavdevice-dev \
libpipewire-0.3-dev libspa-0.2-dev \ libpipewire-0.3-dev libspa-0.2-dev \
libgbm-dev libegl-dev libgl-dev libdrm-dev libva-dev \ libgbm-dev libegl-dev libgl-dev libdrm-dev libva-dev \
@@ -101,10 +131,13 @@ ok "build deps ready"
# --- 2. build host (+ web) ------------------------------------------------- # --- 2. build host (+ web) -------------------------------------------------
log "Building punktfunk-host (release) — first build is slow (~10-15 min)" log "Building punktfunk-host (release) — first build is slow (~10-15 min)"
# vulkan-encode matches the packaged builds (deb/arch): the raw Vulkan Video HEVC/AV1 backend
# (real RFI loss recovery). Pure-Rust ash — no extra system dep. A featureless hand build would
# silently fall back to libav VAAPI.
distrobox enter "$BOX" -- bash -lc " distrobox enter "$BOX" -- bash -lc "
set -e set -e
export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR' export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'
cd '$SRC' && cargo build -r -p punktfunk-host cd '$SRC' && cargo build -r -p punktfunk-host --features punktfunk-host/vulkan-encode
" "
[ -x "$BIN" ] || die "build did not produce $BIN" [ -x "$BIN" ] || die "build did not produce $BIN"
ok "host binary: $BIN" ok "host binary: $BIN"
@@ -126,8 +159,12 @@ mkdir -p "$CONFIG"
if [ ! -f "$CONFIG/host.env" ]; then if [ ! -f "$CONFIG/host.env" ]; then
cat > "$CONFIG/host.env" <<'EOF' cat > "$CONFIG/host.env" <<'EOF'
# punktfunk Steam Deck host config (sourced by the punktfunk-host user service). # punktfunk Steam Deck host config (sourced by the punktfunk-host user service).
# Auto encoder: VAAPI on the Deck's AMD GPU, NVENC on NVIDIA. # Auto encoder: Vulkan Video (or VAAPI fallback) on the Deck's AMD GPU, NVENC on NVIDIA.
PUNKTFUNK_ENCODER=auto PUNKTFUNK_ENCODER=auto
# Van Gogh (LCD/OLED Deck) RADV still gates VK_KHR_video_encode_* behind this perftest flag;
# without it the Vulkan backend can't open and sessions fall back to libav VAAPI. Harmless on
# GPUs where encode is exposed by default.
RADV_PERFTEST=video_encode
# The host auto-detects the live session (Game Mode gamescope / Desktop KDE) per connect. # The host auto-detects the live session (Game Mode gamescope / Desktop KDE) per connect.
# Override the compositor only if detection misbehaves: PUNKTFUNK_COMPOSITOR=gamescope # Override the compositor only if detection misbehaves: PUNKTFUNK_COMPOSITOR=gamescope
EOF EOF
@@ -136,6 +173,34 @@ else
ok "host.env exists (left as-is)" ok "host.env exists (left as-is)"
fi fi
# KWin authorization for Desktop-Mode streaming (and mid-stream Game↔Desktop switches): KWin
# resolves a connecting client's /proc/<pid>/exe against a .desktop `Exec=` and only then grants
# the restricted Wayland globals it lists (see packaging/linux/io.unom.Punktfunk.Host.desktop).
# Exec must therefore be THIS install's binary path, not the packaged /usr/bin one. KWin reads
# grants at session start — after first install, restart the Desktop session (Game Mode and back).
DESKTOP_DST="$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
# First-time install of the grant: KWin only reads it at session start, so a fresh login is required
# before Desktop-mode capture works. A re-run that just rewrites it needs no relogin.
[ -f "$DESKTOP_DST" ] || NEED_RELOGIN=1
mkdir -p "$HOME/.local/share/applications"
sed "s|^Exec=.*|Exec=$BIN|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" > "$DESKTOP_DST"
ok "KWin desktop-capture authorization (io.unom.Punktfunk.Host.desktop → $BIN)"
# KDE Desktop-mode INPUT: a normal Plasma login lacks the RemoteDesktop portal grant the host's libei
# input path needs, so it would pop an "Allow remote control?" dialog a headless host can't answer.
# Seed it once (per-user, no root) — mirrors packaging/bazzite/kde-desktop-setup.sh. Game Mode
# (gamescope) needs none of this; the .desktop above already grants org_kde_kwin_fake_input.
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ -s "$GRANT_DST" ]; then
ok "KDE RemoteDesktop grant already present"
elif [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
systemctl --user restart xdg-permission-store 2>/dev/null || true
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
if [ "$WITH_WEB" = 1 ] && [ ! -f "$CONFIG/web.env" ]; then if [ "$WITH_WEB" = 1 ] && [ ! -f "$CONFIG/web.env" ]; then
# Random login password + session secret for the web console, generated once. # Random login password + session secret for the web console, generated once.
# `|| true` swallows the SIGPIPE `tr` takes when `head` closes the pipe (pipefail would abort). # `|| true` swallows the SIGPIPE `tr` takes when `head` closes the pipe (pipefail would abort).
@@ -151,9 +216,11 @@ else
[ "$WITH_WEB" = 1 ] && ok "web.env exists (login password unchanged)" [ "$WITH_WEB" = 1 ] && ok "web.env exists (login password unchanged)"
fi fi
# --- 4. system tuning (needs sudo; skipped gracefully if unavailable) ------ # --- 4. system tuning (needs sudo: UDP buffers + gamepad udev rule + vhci-hcd + input group) --------
log "System tuning (UDP buffers + input group) — needs sudo" log "System tuning (UDP buffers + gamepad rules + vhci-hcd + input group)"
if sudo -n true 2>/dev/null; then # sudo was acquired up front in preflight (SUDO_OK) so this never stalls behind the long build; a
# skip here (no password / no TTY) was already reported loudly there.
if [ "$SUDO_OK" = 1 ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' \ printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' \
| sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null | sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
@@ -161,13 +228,34 @@ if sudo -n true 2>/dev/null; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules && sudo udevadm trigger || true sudo udevadm control --reload-rules && sudo udevadm trigger || true
ok "installed udev rule (virtual gamepads)" ok "installed udev rule (virtual gamepads + native Steam Deck controller)"
fi
# vhci-hcd: the usbip transport that makes the virtual Steam Deck pad a *real* USB device so Steam
# Input adopts it (else it degrades to plain UHID, which Steam ignores — "no controller appears").
# Persist the autoload AND load it now so passthrough works without waiting for a reboot.
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || warn "could not load vhci-hcd now (loads on next boot) — needed for the native Steam Deck pad"
ok "vhci-hcd autoload installed (native Steam Deck controller transport)"
fi
if id -nG "$USER" | grep -qw input; then
ok "already in the 'input' group"
else
sudo usermod -aG input "$USER"
NEED_RELOGIN=1
warn "added $USER to the 'input' group (applies on next login)"
fi fi
id -nG "$USER" | grep -qw input || { sudo usermod -aG input "$USER"; warn "added $USER to 'input' group — log out/in (or reboot) for gamepad support"; }
else else
warn "passwordless sudo unavailable — skipping UDP-buffer + udev tuning." warn "no usable sudo — SKIPPED system tuning. Gamepad passthrough + clean streaming need root (udev"
warn "Without it, high-bitrate streaming drops packets. Apply manually later:" warn "rule, 'input' group, vhci-hcd, UDP buffers) — there is no user-space way to do these."
warn " echo -e 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf && sudo sysctl --system" warn "A stock SteamOS 'deck' account has NO password, so sudo can't work until you set one:"
warn " passwd # set a sudo password once, then re-run this script"
warn "Or apply it by hand (then reboot):"
warn " sudo install -m644 $SRC/scripts/60-punktfunk.rules /etc/udev/rules.d/ &&"
warn " sudo install -m644 $SRC/scripts/punktfunk-modules.conf /etc/modules-load.d/punktfunk.conf &&"
warn " sudo usermod -aG input $USER &&"
warn " printf 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432\\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf &&"
warn " sudo sysctl --system && sudo udevadm control --reload-rules && sudo udevadm trigger"
fi fi
# --- 5. systemd user services --------------------------------------------- # --- 5. systemd user services ---------------------------------------------
@@ -248,3 +336,10 @@ else
echo " • Pairing required (secure default). From a client, pick this host and enter the PIN the host shows." echo " • Pairing required (secure default). From a client, pick this host and enter the PIN the host shows."
fi fi
echo " • Update later: bash $SRC/scripts/steamdeck/update.sh" echo " • Update later: bash $SRC/scripts/steamdeck/update.sh"
if [ "$NEED_RELOGIN" = 1 ]; then
echo
warn "ONE MORE STEP before streaming — reboot the Deck (or fully log out and back in)."
echo " KWin only authorizes Desktop-mode screen capture on a fresh session, and the new 'input'"
echo " group (native Steam Deck controller passthrough) only applies to a new login. Streaming"
echo " Game Mode with a generic Xbox pad works now; Desktop capture + the native Deck pad need the reboot."
fi
+65 -1
View File
@@ -21,7 +21,8 @@ if [ "${1:-}" = "--pull" ]; then
fi fi
log "Rebuilding host (release)" log "Rebuilding host (release)"
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'; cd '$SRC' && cargo build -r -p punktfunk-host" # vulkan-encode matches the packaged builds (deb/arch) — see install.sh.
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'; cd '$SRC' && cargo build -r -p punktfunk-host --features punktfunk-host/vulkan-encode"
ok "host rebuilt" ok "host rebuilt"
if [ "$WEB" = 1 ]; then if [ "$WEB" = 1 ]; then
log "Rebuilding web console" log "Rebuilding web console"
@@ -29,6 +30,69 @@ if [ "$WEB" = 1 ]; then
ok "web rebuilt" ok "web rebuilt"
fi fi
# Retrofit config that install.sh now writes but older installs predate (both idempotent):
# RADV_PERFTEST — Van Gogh RADV still gates VK_KHR_video_encode_* behind it; without it the
# Vulkan backend can't open and sessions silently fall back to libav VAAPI. The KWin .desktop —
# KWin only grants the restricted capture/input globals to the exe a .desktop authorizes.
HOST_ENV="$HOME/.config/punktfunk/host.env"
if [ -f "$HOST_ENV" ] && ! grep -q '^RADV_PERFTEST=' "$HOST_ENV"; then
printf '\n# Van Gogh RADV gates VK_KHR_video_encode_* behind this (Vulkan Video encode).\nRADV_PERFTEST=video_encode\n' >> "$HOST_ENV"
ok "host.env: added RADV_PERFTEST=video_encode"
fi
mkdir -p "$HOME/.local/share/applications"
sed "s|^Exec=.*|Exec=$TARGET_DIR/release/punktfunk-host|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" \
> "$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
ok "KWin desktop-capture authorization refreshed"
# Retrofit the system bits install.sh now sets up but older installs predate (idempotent). vhci-hcd =
# usbip transport for the native Steam Deck pad; 60-punktfunk.rules = /dev/uhid + vhci access; input
# group = uhid write; the kde-authorized grant (per-user, no root) = Desktop-mode input. A stock Deck
# needs a sudo PASSWORD, so PROMPT for it rather than silently skipping (skipping = gamepads stay dead).
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo needs your password to (re)apply the gamepad udev rule, vhci-hcd, input group, and UDP buffers:"
sudo -v && SUDO_OK=1 || true
fi
if [ "$SUDO_OK" = 1 ]; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules >/dev/null 2>&1 || true
sudo udevadm trigger >/dev/null 2>&1 || true
ok "gamepad udev rule ensured"
fi
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || true
ok "vhci-hcd autoload ensured (native Steam Deck controller)"
fi
# UDP buffers: older installs (or sudo-skipped ones) still run the stock 416 KB cap.
if [ ! -f /etc/sysctl.d/99-punktfunk-net.conf ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null 2>&1 || true
ok "UDP socket buffers raised to 32 MB (persisted)"
fi
if id -nG "$USER" | grep -qw input; then :; else
sudo usermod -aG input "$USER"
warn "added $USER to the 'input' group — REBOOT (or log out/in) for it to apply"
fi
else
warn "no usable sudo — SKIPPED gamepad/udev/vhci/UDP tuning (all root-only; no user-space alternative)."
warn "A stock SteamOS 'deck' account has NO password — set one with 'passwd', then re-run. Gamepads stay"
warn "Xbox-360 until this runs and you reboot."
fi
echo
warn "If the controller still shows as an Xbox 360 pad, REBOOT the Deck once — the 'input' group and the"
warn "vhci-hcd module only become live for the host service on a fresh login."
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ ! -s "$GRANT_DST" ] && [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
log "Restarting services" log "Restarting services"
systemctl --user restart punktfunk-host.service systemctl --user restart punktfunk-host.service
ok "punktfunk-host restarted" ok "punktfunk-host restarted"
+4 -1
View File
@@ -89,7 +89,10 @@ A complexity ladder in [`examples/`](./examples) — start at the top:
4. [`couch-preset.effect.ts`](./examples/couch-preset.effect.ts) — **advanced, Effect-native**: only if you're composing Effect programs. 4. [`couch-preset.effect.ts`](./examples/couch-preset.effect.ts) — **advanced, Effect-native**: only if you're composing Effect programs.
Examples 13 are the plain Promise facade and cover most automation; you only need example 4's Examples 13 are the plain Promise facade and cover most automation; you only need example 4's
Effect surface for composed, interruptible programs. Run any with `bun examples/<file>.ts`. Effect surface for composed, interruptible programs. Run any **in the repo** with
`bun examples/<file>.ts`. To **deploy** one on a host, install the package into its own directory
(`bun add @punktfunk/host`) and change its `../src/…` import to `@punktfunk/host` — see
[Running a single script as a service](#running-a-single-script-as-a-service).
Plus a real-world recipe: Plus a real-world recipe:
+5
View File
@@ -15,6 +15,11 @@
// VH_DEVICE=couch-deck.11 bun examples/virtualhere-dualsense.ts # address from `-t LIST` // VH_DEVICE=couch-deck.11 bun examples/virtualhere-dualsense.ts # address from `-t LIST`
// VH_DEVICE=DualSense bun examples/virtualhere-dualsense.ts # …or match by name substring // VH_DEVICE=DualSense bun examples/virtualhere-dualsense.ts # …or match by name substring
// //
// To run this *outside* the repo (the normal case), drop it in its own dir, `bun add
// @punktfunk/host`, and change the import below from `../src/index.js` to `@punktfunk/host`; then
// keep it alive as a systemd user unit (its SIGTERM release hands the pad back on `systemctl
// stop`). Full walkthrough: docs → Events & hooks → "full controller passthrough (VirtualHere)".
//
// Env: VH_DEVICE required — a VirtualHere address (`server.port`) or a device-name substring. // Env: VH_DEVICE required — a VirtualHere address (`server.port`) or a device-name substring.
// VH_CLIENT client binary. Default `vhclientx86_64` (Linux); Windows `vhui64.exe`, // VH_CLIENT client binary. Default `vhclientx86_64` (Linux); Windows `vhui64.exe`,
// macOS `vhclientosx`, ARM Linux `vhclientarm64`. // macOS `vhclientosx`, ARM Linux `vhclientarm64`.
+2 -1
View File
@@ -6,6 +6,7 @@
//! harness adds `tc netem` jitter/reorder on the UDP path. //! harness adds `tc netem` jitter/reorder on the UDP path.
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionKey;
use punktfunk_core::error::PunktfunkError; use punktfunk_core::error::PunktfunkError;
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
use punktfunk_core::transport::loopback_pair; use punktfunk_core::transport::loopback_pair;
@@ -25,7 +26,7 @@ fn config(role: Role, scheme: FecScheme, drop_period: u32) -> Config {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: drop_period, loopback_drop_period: drop_period,
} }