Compare commits

..

12 Commits

Author SHA1 Message Date
enricobuehler 1efb8aef74 feat(core+host+client): cursor channel — remote-desktop sweep M2a+M2b
ci / web (pull_request) Successful in 1m8s
ci / docs-site (pull_request) Successful in 1m10s
apple / swift (pull_request) Successful in 1m23s
apple / screenshots (pull_request) Has been skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Failing after 6m44s
ci / bench (pull_request) Successful in 7m24s
android / android (pull_request) Failing after 7m47s
ci / rust (pull_request) Failing after 8m36s
windows / build (x86_64-pc-windows-msvc) (pull_request) Failing after 4m36s
The host cursor stops riding the video and becomes a real OS cursor on
the client (the Parsec/RDP model): pointer feel no longer pays the
capture→encode→network→decode→present round trip.

Wire (M2a):
- Hello grows a client_caps trailing byte (CLIENT_CAP_CURSOR) after the
  fixed display_hdr block — presence disambiguated by remaining length,
  which caps the post-HDR tail at 27 bytes (documented); Welcome answers
  HOST_CAP_CURSOR (capable-and-asked, the 444/clipboard precedent).
- CursorShape (0x50, control stream): serial + dims + hotspot + straight
  RGBA, ≤120px/side so the u16 frame always fits (128² would overshoot);
  client caches by serial — re-showing a known shape costs 14 bytes, not
  a bitmap (RDP pointer-cache for free).
- CursorState (0xD0 datagram): serial + visible/relative_hint flags +
  position, sent once per encode-loop tick — latest-wins, self-healing
  under loss, no refresh timer. relative_hint is reserved for M3.
- Client core: two new planes (control-task + datagram-task arms) →
  next_cursor_shape/next_cursor_state; connect() grows client_caps
  (C ABI passes 0 until the v11 cursor poll fns exist).

Host (M2b, Linux portal only):
- handshake::cursor_forward is THE predicate (client asked ∧ Linux ∧
  compositor ≠ gamescope) — Welcome bit and session wiring both read it.
- SessionPlan.cursor_blend goes false for a forwarding session; the
  encode loop ticks a CursorForwarder every iteration: shape-serial diff
  → control-task bridge (mirrors probe_result), state datagram → conn.
- CursorOverlay/capture CursorState carry the hotspot through
  (nearest-neighbor downscale backstop for XL cursors, unit-tested).

Presenter:
- CursorChannel drains both planes per loop iteration; shapes become
  SDL color cursors (from_surface + hotspot), applied while the desktop
  mouse model is engaged; visibility follows the host; capture/released
  hands back the system cursor. Sessions advertise the cap when they
  START in desktop mode.

Verified on .21: fmt + clippy -D warnings (7 crates) + tests green
(core 218 incl. new wire roundtrips, host 245 incl. e2e + forwarder
downscale tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 01:09:51 +02:00
enricobuehler 12df1388de feat(apple/input): desktop (absolute) mouse mode on macOS — remote-desktop sweep M1
ci / web (pull_request) Successful in 53s
ci / docs-site (pull_request) Successful in 53s
apple / swift (pull_request) Successful in 1m21s
apple / screenshots (pull_request) Has been skipped
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 4m15s
ci / bench (pull_request) Successful in 5m35s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 1m40s
android / android (pull_request) Successful in 11m14s
ci / rust (pull_request) Successful in 21m49s
Folds the parked client-side-cursor machinery (cursorMode auto/always/
never, hidden while disabled) into the cross-client mouse model:
MouseInputMode capture|desktop under DefaultsKey.mouseMode, picked in
Settings ▸ Keyboard & mouse (macOS), resolved at session start and
gated off on gamescope hosts (relative-only EIS).

Desktop model = the un-neutered absolute path with the SDL cursor
policy: pointer never disassociated (enters/leaves the stream freely),
monitor forwards letterboxed absolute positions, and the local cursor
hides only while over the view via an invisible-cursor rect (the
host's composited cursor is the one you see; AppKit manages the rect,
so no hide/unhide balancing). ⌘⇧C becomes ⌃⌥⇧M — the same chord as
the SDL clients — flipping the model live with an atomic
release/re-engage.

Verified: swift build + full test suites green (macOS arm64).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 00:03:19 +02:00
enricobuehler 7295ae70f9 feat(client/input): desktop (absolute) mouse mode — remote-desktop sweep M1
New physical-mouse model beside capture: Desktop leaves the pointer
uncaptured and sends absolute positions through the letterbox
(MouseMoveAbs — every host injector already consumes it). Capture
stays the default and the game model.

- pf-client-core: MouseMode (capture|desktop) persisted in Settings,
  default capture so existing stores are unchanged.
- pf-presenter: Capture grows a desktop model — latest-wins pending
  abs position coalesced per loop iteration (same 1000 Hz discipline
  as relative), flushed before clicks/keys/wheel so they land where
  the cursor is; Ctrl+Alt+Shift+M flips the model live; local cursor
  stays hidden over the window (the host's composited cursor is the
  one you see until the M2 cursor channel); Windows keyboard grab
  only engages for capture (a desktop stream is something you
  Alt-Tab away from).
- gamescope gating: its EIS is relative-only, so desktop mode is
  pinned off there (resolved_compositor), with a log note.
- Settings surfaces: GTK row (dynamic caption), WinUI combo,
  console-UI row + step test, capture-hint line.

Verified: fmt + clippy -D warnings + tests (33/40/19) on Linux .21;
clippy -D warnings for all five crates incl. punktfunk-client-windows
native on .173.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 23:52:16 +02:00
enricobuehler 87e7c82cbc chore(release): bump workspace version to 0.17.2
audit / bun-audit (push) Failing after 18s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
apple / swift (push) Successful in 1m23s
audit / cargo-audit (push) Successful in 2m44s
ci / bench (push) Successful in 6m26s
apple / screenshots (push) Successful in 6m38s
ci / rust (push) Successful in 24m53s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m10s
decky / build-publish (push) Successful in 22s
android-screenshots / screenshots (push) Successful in 3m7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m38s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9m3s
release / apple (push) Successful in 10m50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m41s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m4s
deb / build-publish-host (push) Successful in 12m44s
android / android (push) Successful in 14m35s
docker / deploy-docs (push) Successful in 23s
arch / build-publish (push) Successful in 15m35s
web-screenshots / screenshots (push) Successful in 3m1s
linux-client-screenshots / screenshots (push) Successful in 7m31s
windows-host / package (push) Successful in 13m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m24s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m32s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m39s
flatpak / build-publish (push) Successful in 6m40s
2026-07-21 23:33:53 +02:00
enricobuehler b2e3d1b540 fix(steamdeck/install): add cmake dep + acquire sudo in preflight
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m52s
apple / swift (push) Successful in 1m22s
android / android (push) Successful in 13m4s
ci / bench (push) Successful in 8m2s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
deb / build-publish (push) Successful in 9m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m14s
apple / screenshots (push) Successful in 6m51s
arch / build-publish (push) Successful in 18m54s
docker / deploy-docs (push) Successful in 27s
ci / rust (push) Successful in 19m17s
deb / build-publish-host (push) Successful in 9m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
pyrowave-sys's build.rs runs cmake to build the vendored PyroWave C library,
but cmake wasn't in the distrobox apt list — so `cargo build -p punktfunk-host`
dies on every SteamOS host with "is cmake not installed?". Add it.

Also move sudo acquisition from step 4 (after the ~15-min build) into preflight
and keep the timestamp warm across the build. Before, a non-interactive run
(`ssh host 'bash install.sh'`) silently skipped the UDP-buffer/udev/vhci-hcd/
input-group tuning AND enable-linger — a "successful" host with no gamepad
passthrough, lossy streaming, and no headless persistence (dies on logout).
Now sudo is prompted once up front (authorize and walk away), and a no-TTY /
no-password run fails loudly before the build instead of silently at the end.

Validated end-to-end on SteamOS 3.8 in a VM with Radeon 780M passthrough:
full host + web build, all planes listening, VAAPI render node, headless
auto-start via linger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehler d36bec6e9d feat(core+host): negotiate ChaCha20-Poly1305 as the session cipher for soft-AES clients
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Failing after 13s
audit / cargo-audit (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Successful in 10m1s
windows-host / package (push) Successful in 11m20s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m39s
Lifts the ~100 Mbps decrypt ceiling on clients without hardware AES — the
armv7 soft-AES targets (webOS TVs), where AES-128-GCM resolves to fixsliced
software AES + software GHASH (~50-100 cpb) while ChaCha20-Poly1305's ARX
construction runs ~10-17 cpb portable, a 4-7x lift that PyroWave-on-TV needs
(design/chacha20-session-cipher.md).

Phase 1 (core crypto, no wire change): SessionKey merges cipher choice and
key material (invalid combinations unrepresentable, zeroize + redacted-Debug
discipline kept); SessionCrypto dispatches both aead-0.5 ciphers per call —
the salt||seq nonce scheme, per-direction salts, seq-as-AAD and replay
window carry over verbatim (same 96-bit nonce / 16-byte tag, const-asserted).
Config.key becomes SessionKey; validate's zero-key rejection follows the
active variant. The C ABI keeps its fixed 16-byte key mapped to AES — no
ABI_VERSION bump.

Phase 2 (negotiation): VIDEO_CAP_CHACHA20 (0x40) — support-plus-request in
one bit, the VIDEO_CAP_444 precedent. Welcome grows cipher@68 +
key_chacha@69..101, emitted only when non-zero so an AES session's Welcome
stays byte-identical to the pre-cipher form; decode is fail-closed (short
key or unknown id -> Err, never a silent AES fallback). No WIRE_VERSION
bump; downgrade resistance inherited from the pinned-TLS control channel.

Phase 3 (host): grant only when the client advertised the bit and the
PUNKTFUNK_CHACHA20 kill-switch (default on, documented) allows; fresh
32-byte per-session key from the same RNG discipline, legacy key field
stays independently random; resolved cipher logged at session start.

Verification: seal/open suites parameterized over both ciphers + a
cross-cipher tamper case; Welcome roundtrip/truncation/fail-closed tests;
ChaCha lossy-loopback soak (loss/replay is cipher-independent); bench
gains _chacha20 series (AES ids unchanged for CI history) — host-side
sealing line-rate-trivial on both x86 (~640 MiB/s) and Apple Silicon
(~535 MiB/s). punktfunk-probe drives the interop matrix via
PUNKTFUNK_CLIENT_CHACHA20=1 and logs the negotiated cipher.

Phase 4 (pf-webos pin bump + unconditional cap bit) follows the next
core release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehler abc54a7d13 fix(decky): actually ship shortcut artwork + Steam Input layout in the published plugin
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m58s
The CI workflow assembled the store zip by hand and silently drifted from
scripts/package.sh when 60d46530 added assets/ and controller_config/ — every
published build (canary and stable through 0.17.1) was missing both, so the
Steam shortcut a fresh install creates had no images and no controller layout
(shortcut_art() found nothing to send; apply_controller_config failed with
template-missing). CI now stages through scripts/package.sh — the plugin file
list lives in one place — and a backstop verifies the runtime-loaded pieces are
in the zip before publishing.

Frontend hardening for installs poisoned by those builds: applyArtwork stamped
its per-appId ART_VERSION marker even when zero assets were applied, so
shipping the files alone would never heal existing shortcuts — only mark done
when something actually landed, and bump ART_VERSION 2→3 to force one re-apply.
Same guard for the controller-config marker (ok with zero account configset
dirs is not done), plus one deferred artwork retry for the fresh-AddShortcut
registration race setShortcutHidden already defers around.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehler 309e37f1e1 fix(host/linux): scope the GPU clock pin to live client sessions
The PUNKTFUNK_PIN_CLOCKS clock pin (AMD power_dpm_force_performance_level=high / NVIDIA nvmlDeviceSetGpuLockedClocks) was armed once at host start and released only at host exit, so the box's clocks stayed pinned the whole time the host ran — even with no client connected.

Move the pin behind a box-wide refcount (gpuclocks::session_pin()) shared across both streaming planes (native + GameStream): it arms on the first live client and releases on the last disconnect. on_host_start() now only installs the process-scoped NVIDIA P2-cap application profile. No behavior change when PUNKTFUNK_PIN_CLOCKS is unset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehler 56d21f9445 fix(client/wol): retry the QUIC dial across the connect budget so wake-from-sleep launches survive
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 40s
ci / bench (push) Successful in 6m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish (push) Successful in 9m54s
release / apple (push) Successful in 9m5s
deb / build-publish-host (push) Successful in 9m35s
flatpak / build-publish (push) Failing after 8m19s
android / android (push) Successful in 16m57s
apple / screenshots (push) Successful in 6m29s
arch / build-publish (push) Successful in 17m35s
windows-host / package (push) Successful in 18m46s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
docker / deploy-docs (push) Successful in 26s
ci / rust (push) Successful in 25m24s
Launching from the Decky plugin against a sleeping host fired the
Wake-on-LAN packet and dialed immediately — but the dial was a single
quinn attempt that gave up after the transport's 8 s idle window, far
shorter than a suspend-to-RAM resume. The host woke to find the client
already gone (its retransmitted Initials show up host-side as
"QUIC accept failed error=timed out"), the stream shortcut exited, and
the launch looked cancelled.

- punktfunk-core: dial in a loop until the embedder's connect budget is
  spent — short 3 s attempts keep the Initial cadence dense, so the
  connect lands within ~a second of the host's network returning. Only
  silence is retried: pin mismatches, typed rejections, and any real
  answer surface immediately, and the shutdown flag stops the loop.
- decky: stretch the budget to 75 s when a magic packet actually went
  out (PF_CONNECT_TIMEOUT → --connect-timeout via the wrapper), so the
  retry window covers the resume; an awake host still connects in <1 s.
- host: a clean close before the control handshake is a reachability
  probe (hosts-page online pips), not a failed session — log it at
  debug instead of WARN "session ended with error", which buried the
  real failures in the reporter's log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:26:10 +02:00
enricobuehler f36d13e371 fix(steamdeck/install): prompt for sudo instead of requiring passwordless
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m5s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m43s
docker / deploy-docs (push) Successful in 48s
deb / build-publish-host (push) Successful in 10m9s
android / android (push) Successful in 18m18s
arch / build-publish (push) Successful in 19m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m16s
ci / rust (push) Successful in 28m27s
The system-tuning block (UDP buffers, gamepad udev rule, vhci-hcd autoload,
input group) was gated on `sudo -n true` — passwordless sudo — so on a stock
SteamOS box (the deck account needs a password) the whole block was silently
skipped: /dev/uhid stayed root-only so the gamepad degraded to an Xbox 360 pad,
and the UDP buffers stayed at the 416 KB default. Video was unaffected because
Game Mode/gamescope needs none of it, which masked the skip on-glass.

- install.sh + update.sh: acquire sudo interactively (`sudo -v` prompt on a TTY)
  instead of requiring passwordless; skip only when there's no TTY or auth fails,
  and then print the exact manual block plus a `passwd` hint — a stock SteamOS
  'deck' account has no password, so sudo can't work until one is set.
- update.sh: also retrofit the UDP-buffer sysctl for older/skipped installs, and
  loudly nag to reboot when the input group was just added (a --user restart does
  not pick up the new group; only a fresh login does).
- steamos-host.md: note this step prompts for the sudo password.

Root is unavoidable: the udev rule, input group, and vhci-hcd module are all
kernel operations with no user-space alternative.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:56:20 +02:00
enricobuehler aacd610b68 fix(steamdeck/install): set up controller passthrough + surface the required reboot
ci / docs-site (push) Successful in 1m12s
ci / web (push) Successful in 1m13s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m35s
apple / screenshots (push) Successful in 6m13s
deb / build-publish (push) Successful in 10m21s
docker / deploy-docs (push) Successful in 34s
deb / build-publish-host (push) Successful in 9m58s
android / android (push) Successful in 17m7s
arch / build-publish (push) Successful in 17m47s
ci / rust (push) Successful in 19m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m2s
A fresh SteamOS host install failed on-glass two ways: KWin denied Desktop-mode
capture (zkde_screencast_unstable_v1) so every session died in the pipeline build,
and the native Steam Deck pad degraded to an Xbox 360 controller. Both stem from
setup that only a fresh login applies — the host user-service keeps the
pre-install group set and KWin reads its .desktop grant at session start — plus a
missing vhci-hcd autoload the pad's USB/IP transport needs (the deb/rpm/arch
packages ship it; the Deck installer did not).

- install.sh: install punktfunk-modules.conf (+ modprobe vhci-hcd now) for the
  native Deck controller transport, seed the KDE RemoteDesktop grant
  (kde-authorized) for Desktop-mode input, and print a loud "reboot before
  streaming" notice when a relogin is actually required (input group added or the
  .desktop grant first installed).
- update.sh: retrofit the udev rule, vhci-hcd autoload, input group, and grant so
  existing installs pick them up on the next update without a reinstall.
- steamos-host.md: document the first-install reboot and the native controller
  passthrough requirements (input group + vhci-hcd + client-side Steam Input Off).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:22:08 +02:00
enricobuehler 081ff64087 docs: explain how to actually run the VirtualHere passthrough example
apple / swift (push) Successful in 1m22s
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m39s
apple / screenshots (push) Successful in 6m38s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
deb / build-publish (push) Successful in 9m0s
deb / build-publish-host (push) Successful in 9m20s
windows-host / package (push) Successful in 9m46s
android / android (push) Successful in 18m37s
arch / build-publish (push) Successful in 18m48s
docker / deploy-docs (push) Successful in 28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m44s
ci / rust (push) Successful in 26m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m2s
The VirtualHere DualSense recipe linked the `virtualhere-dualsense.ts` SDK
example as a deployable recipe but skipped everything needed to run it: that
VirtualHere is a server (couch) + client (host) pair, and that the example —
like every example — imports `../src/index.js`, which only resolves inside the
SDK repo. A user copying it out had no way to know they must
`bun add @punktfunk/host` and swap that import.

- automation.md: rewrite the recipe into a full walkthrough — the two-sided
  VirtualHere setup, the `-t` verbs, the zero-code hooks version, and a new
  scripted section with exact deploy steps, env vars, and running it as a
  service (its own SIGTERM handler makes `systemctl stop` release the pad).
- sdk/README.md: say how to run an example in-repo vs deployed on a host.
- virtualhere-dualsense.ts: header note on the import swap + service setup,
  since that file is where the doc link lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:28:23 +02:00
63 changed files with 2523 additions and 389 deletions
+24 -30
View File
@@ -6,17 +6,12 @@
# #
# The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT # The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT
# need the Decky CLI (which requires Docker + rust-nightly only to compile native backends). # need the Decky CLI (which requires Docker + rust-nightly only to compile native backends).
# We build the frontend with pnpm and assemble the store-layout zip by hand: # We build the frontend with pnpm and stage the store-layout tree with the SAME script local
# # builds use (clients/decky/scripts/package.sh) — the plugin's file list lives in exactly ONE
# punktfunk.zip # place, so a file added there (bin/, assets/, controller_config/, …) can never be silently
# punktfunk/ <- single top-level dir == plugin.json "name" # missing from the published build. (Hand-assembling the zip here is how the shipped plugin
# plugin.json [required] # lost the shortcut artwork + Steam Input layout for a while.) CI only adds `update.json` on
# package.json [required; CI stamps "version" — Decky reads the installed version here] # top: the {channel, manifest} pointer the plugin's self-update check polls.
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
# #
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel # SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed # `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
@@ -90,28 +85,27 @@ jobs:
- name: Assemble store-layout zip - name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }} working-directory: ${{ gitea.workspace }}
run: | run: |
apt-get update && apt-get install -y --no-install-recommends zip >/dev/null # node:22-bookworm ships python3 (a package.sh dep) but not zip; install both anyway
STAGE="$RUNNER_TEMP/decky" # so an image change can't silently break the build.
DEST="$STAGE/$PLUGIN" apt-get update && apt-get install -y --no-install-recommends zip python3 >/dev/null
rm -rf "$STAGE"; mkdir -p "$DEST/dist" "$DEST/bin" # Stage the canonical plugin tree (dist/, main.py, bin/, assets/, controller_config/,
cp clients/decky/plugin.json "$DEST/" # LICENSE, …) with the same script local/sideload builds use — see the header comment.
cp clients/decky/package.json "$DEST/" # Runs AFTER the version stamp, so the staged package.json carries $VERSION.
cp clients/decky/main.py "$DEST/" bash clients/decky/scripts/package.sh
cp clients/decky/dist/index.js "$DEST/dist/" DEST="clients/decky/out/$PLUGIN"
cp clients/decky/README.md "$DEST/" # CI-only addition: the self-update channel pointer the backend reads (main.py
# The stream-launch wrapper (target of the Steam shortcut); keep it executable # check_update). It points at THIS channel's manifest.json (published below); that
# (runner_info() also re-chmods at runtime in case the zip/extract drops the bit). # manifest in turn points at the immutable per-version zip, so its sha256 stays valid
cp clients/decky/bin/punktfunkrun.sh "$DEST/bin/" # across future alias re-uploads.
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json" printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" ) ( cd clients/decky/out && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip" ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip" unzip -l "$RUNNER_TEMP/punktfunk.zip"
# Backstop against packaging drift: the runtime-loaded pieces MUST be in the zip.
for f in main.py dist/index.js bin/punktfunkrun.sh assets/grid.png \
controller_config/punktfunk.vdf update.json; do
unzip -l "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN/$f" >/dev/null || { echo "MISSING $f" >&2; exit 1; }
done
# The update manifest the plugin polls: the immutable per-version artifact + its # The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on # sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias). # mismatch — so it MUST be the per-version URL, never the mutable alias).
Generated
+64 -27
View File
@@ -656,6 +656,30 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]] [[package]]
name = "ciborium" name = "ciborium"
version = "0.2.2" version = "0.2.2"
@@ -691,6 +715,7 @@ checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [ dependencies = [
"crypto-common", "crypto-common",
"inout", "inout",
"zeroize",
] ]
[[package]] [[package]]
@@ -2159,7 +2184,7 @@ dependencies = [
[[package]] [[package]]
name = "latency-probe" name = "latency-probe"
version = "0.17.1" version = "0.17.2"
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
@@ -2264,7 +2289,7 @@ dependencies = [
[[package]] [[package]]
name = "libvpl-sys" name = "libvpl-sys"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -2299,7 +2324,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]] [[package]]
name = "loss-harness" name = "loss-harness"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"punktfunk-core", "punktfunk-core",
] ]
@@ -2788,7 +2813,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]] [[package]]
name = "pf-capture" name = "pf-capture"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2808,7 +2833,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-client-core" name = "pf-client-core"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2832,7 +2857,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-clipboard" name = "pf-clipboard"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2850,7 +2875,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-console-ui" name = "pf-console-ui"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2871,7 +2896,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-encode" name = "pf-encode"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2895,7 +2920,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-ffvk" name = "pf-ffvk"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"ash", "ash",
"bindgen", "bindgen",
@@ -2904,7 +2929,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-frame" name = "pf-frame"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"libc", "libc",
@@ -2916,7 +2941,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-gpu" name = "pf-gpu"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-host-config", "pf-host-config",
@@ -2930,11 +2955,11 @@ dependencies = [
[[package]] [[package]]
name = "pf-host-config" name = "pf-host-config"
version = "0.17.1" version = "0.17.2"
[[package]] [[package]]
name = "pf-inject" name = "pf-inject"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2962,14 +2987,14 @@ dependencies = [
[[package]] [[package]]
name = "pf-paths" name = "pf-paths"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "pf-presenter" name = "pf-presenter"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2984,7 +3009,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-vdisplay" name = "pf-vdisplay"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -3014,7 +3039,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-win-display" name = "pf-win-display"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-paths", "pf-paths",
@@ -3026,7 +3051,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-zerocopy" name = "pf-zerocopy"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3137,6 +3162,17 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]] [[package]]
name = "polyval" name = "polyval"
version = "0.6.2" version = "0.6.2"
@@ -3222,7 +3258,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-android" name = "punktfunk-client-android"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"jni", "jni",
@@ -3238,7 +3274,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-linux" name = "punktfunk-client-linux"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -3254,7 +3290,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-session" name = "punktfunk-client-session"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-client-core", "pf-client-core",
@@ -3269,7 +3305,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-windows" name = "punktfunk-client-windows"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"async-channel", "async-channel",
"ffmpeg-next", "ffmpeg-next",
@@ -3288,11 +3324,12 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-core" name = "punktfunk-core"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"bytes", "bytes",
"cbindgen", "cbindgen",
"chacha20poly1305",
"criterion", "criterion",
"fec-rs", "fec-rs",
"hmac", "hmac",
@@ -3319,7 +3356,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-host" name = "punktfunk-host"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
@@ -3403,7 +3440,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-probe" name = "punktfunk-probe"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"mdns-sd", "mdns-sd",
@@ -3417,7 +3454,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-tray" name = "punktfunk-tray"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ksni", "ksni",
@@ -3440,7 +3477,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]] [[package]]
name = "pyrowave-sys" name = "pyrowave-sys"
version = "0.17.1" version = "0.17.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" } ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package] [workspace.package]
version = "0.17.1" version = "0.17.2"
edition = "2021" edition = "2021"
rust-version = "1.82" rust-version = "1.82"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0", "name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0" "identifier": "MIT OR Apache-2.0"
}, },
"version": "0.17.1" "version": "0.17.2"
}, },
"paths": { "paths": {
"/api/v1/clients": { "/api/v1/clients": {
@@ -447,6 +447,14 @@ extension SettingsView {
/// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path). /// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path).
@ViewBuilder var inputSection: some View { @ViewBuilder var inputSection: some View {
Section("Keyboard & mouse") { Section("Keyboard & mouse") {
#if os(macOS)
described(mouseModeDescription) {
Picker("Mouse input", selection: $mouseMode) {
Text("Capture (games)").tag(MouseInputMode.capture.rawValue)
Text("Desktop (absolute)").tag(MouseInputMode.desktop.rawValue)
}
}
#endif
described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) { described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) {
Picker("Modifier keys", selection: $modifierLayout) { Picker("Modifier keys", selection: $modifierLayout) {
ForEach(ModifierLayout.allCases, id: \.self) { layout in ForEach(ModifierLayout.allCases, id: \.self) { layout in
@@ -459,6 +467,20 @@ extension SettingsView {
} }
} }
} }
#if os(macOS)
/// The SELECTED mouse model explained dynamic, like the touch-mode caption.
private var mouseModeDescription: String {
switch MouseInputMode(rawValue: mouseMode) ?? .capture {
case .capture:
return "The pointer locks to the stream and sends relative motion — best for "
+ "games. ⌃⌥⇧M switches live; applies from the next capture otherwise."
case .desktop:
return "The pointer moves freely in and out of the stream and sends absolute "
+ "positions — best for remote desktop work. Unavailable on gamescope hosts."
}
}
#endif
#endif #endif
// MARK: - Audio // MARK: - Audio
@@ -89,6 +89,7 @@ struct SettingsView: View {
@State var customMode = false @State var customMode = false
#endif #endif
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.mouseMode) var mouseMode = MouseInputMode.capture.rawValue
@AppStorage(DefaultsKey.speakerUID) var speakerUID = "" @AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = "" @AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0 @AppStorage(DefaultsKey.micChannel) var micChannel = 0
@@ -110,11 +110,11 @@ public final class InputCapture {
/// event itself is swallowed). Main queue. /// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)? public var onToggleCapture: (() -> Void)?
/// Fired on C (the client-side-cursor toggle flips between the captured/disassociated /// Fired on M (the mouse-model flip, capture desktop cross-client parity with the
/// relative path and the visible-cursor absolute path; detected here, like , so it works /// SDL clients' Ctrl+Alt+Shift+M; detected here, like , so it works regardless of the
/// regardless of the current capture state and the event itself is swallowed). macOS only; /// current capture state and the event itself is swallowed). macOS only; the
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue. /// absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)? public var onToggleMouseMode: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS /// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which /// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
@@ -245,13 +245,14 @@ public final class InputCapture {
self.onToggleCapture?() self.onToggleCapture?()
return nil return nil
} }
// C toggles the client-side cursor (visible-cursor absolute path vs the // M flips the mouse model (capture desktop the SDL clients' identical
// captured relative path). keyCode 8 = kVK_ANSI_C; layout-independent so it // chord). Detected in both capture states, like , so the model can be set
// fires the same on any keyboard. Suppress the C (latched like 's Esc) so it // before engaging. keyCode 46 = kVK_ANSI_M; layout-independent. Suppress the M
// doesn't type into the host, and swallow the event so it doesn't beep. // (latched like 's Esc) so it doesn't type into the host, and swallow the
if event.keyCode == 8 /* C */, flags == [.command, .shift] { // event so it doesn't beep.
self.suppressedVK = 0x43 // VK_C the same physical C is en route via GC if event.keyCode == 46 /* M */, flags == [.control, .option, .shift] {
self.onToggleCursor?() self.suppressedVK = 0x4D // VK_M the same physical M is en route via GC
self.onToggleMouseMode?()
return nil return nil
} }
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other // The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
@@ -0,0 +1,12 @@
/// How a physical mouse drives the host the cross-client mouse model (the SDL clients'
/// `MouseMode` / `Settings::mouse_mode`, design/remote-desktop-sweep.md M1). Stored stringly
/// under `DefaultsKey.mouseMode`.
public enum MouseInputMode: String, CaseIterable, Sendable {
/// Pointer capture (disassociated, hidden cursor, relative deltas) the game model,
/// and the default: the only cursor you see is the host's.
case capture
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion is forwarded as absolute positions through the letterbox. The remote desktop
/// model. Requires a host injector with absolute support (not gamescope).
case desktop
}
@@ -38,10 +38,11 @@ private let streamInputDebug =
/// dragged deltas become the relative motion StreamLayerView forwards), and hide it. /// dragged deltas become the relative motion StreamLayerView forwards), and hide it.
/// hide/unhide and associate are balanced via `captured`. /// hide/unhide and associate are balanced via `captured`.
/// ///
/// In CLIENT-SIDE-CURSOR mode (gamescope, whose capture carries no host cursor) this is a /// In the DESKTOP mouse model (absolute pointer, remote-desktop-sweep M1) this is a no-op:
/// no-op: the local cursor stays visible and free, and StreamLayerView forwards ABSOLUTE /// the pointer stays free (entering and leaving the stream at will) and StreamLayerView
/// positions instead the visible system cursor IS the on-screen cursor. `disassociate` /// forwards ABSOLUTE positions instead; the local cursor is hidden only while over the view
/// selects between the two; `release()` only undoes what `capture` actually did. /// (cursor rects). `disassociate` selects between the two; `release()` only undoes what
/// `capture` actually did.
private final class CursorCapture { private final class CursorCapture {
private var captured = false private var captured = false
/// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode), /// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode),
@@ -207,14 +208,17 @@ public final class StreamLayerView: NSView {
/// forwarded). Main-thread only. /// forwarded). Main-thread only.
public private(set) var captured = false public private(set) var captured = false
/// Client-side-cursor mode: when true the local system cursor stays VISIBLE over the /// Desktop (absolute) mouse model remote-desktop-sweep M1: when true the pointer is
/// stream and the mouse monitor forwards ABSOLUTE positions (the visible cursor is the /// never disassociated (it enters and leaves the stream freely) and the mouse monitor
/// on-screen cursor gamescope draws none, so no double cursor); when false the existing /// forwards ABSOLUTE positions through the letterbox; the local cursor is hidden only
/// captured/disassociated relative path runs unchanged. Initialized at session start from /// while over this view (cursor rects the host's composited cursor, tracking our
/// the `cursorMode` setting + the host's resolved compositor, toggled live by C. A live /// sends, is the one you see) and reappears the moment it leaves. When false the
/// flip re-engages capture in the new mode so disassociation + the abs/rel choice swap /// captured/disassociated relative path runs unchanged. Initialized at session start
/// atomically. Main-thread only. /// from the `mouseMode` setting gated by the host's resolved compositor (gamescope's
private var cursorVisible = false /// EIS is relative-only absolute sends would be dropped, so it pins to capture);
/// flipped live by M. A live flip re-engages capture in the new model so
/// disassociation + the abs/rel choice swap atomically. Main-thread only.
private var desktopMouse = false
/// One-shot auto-engage request (stream start, trust confirmed) attempted as soon /// One-shot auto-engage request (stream start, trust confirmed) attempted as soon
/// as the view is in a window with real bounds, then dropped, so it can never fire /// as the view is in a window with real bounds, then dropped, so it can never fire
/// surprisingly later (e.g. on a resize). /// surprisingly later (e.g. on a resize).
@@ -440,9 +444,9 @@ public final class StreamLayerView: NSView {
// If the cursor grab is refused (e.g. the reactivating click arrives before the app is // If the cursor grab is refused (e.g. the reactivating click arrives before the app is
// frontmost), stay released so the NEXT click retries never latch captured=true over // frontmost), stay released so the NEXT click retries never latch captured=true over
// a free cursor, which would make mouseDown's `!captured` guard reject every later click. // a free cursor, which would make mouseDown's `!captured` guard reject every later click.
// In client-side-cursor mode there is no grab (the cursor stays visible) capture // In the desktop mouse model there is no grab (the pointer stays free) capture
// always engages and the monitor forwards absolute positions instead. // always engages and the monitor forwards absolute positions instead.
guard cursorCapture.capture(in: self, disassociate: !cursorVisible) else { return } guard cursorCapture.capture(in: self, disassociate: !desktopMouse) else { return }
inputCapture?.setForwarding(true, suppressClick: fromClick) inputCapture?.setForwarding(true, suppressClick: fromClick)
// Install AFTER the warp + setForwarding: the engage warp generates no forwarded // Install AFTER the warp + setForwarding: the engage warp generates no forwarded
// delta (the monitor isn't up yet), and the engage click's suppression latch is // delta (the monitor isn't up yet), and the engage click's suppression latch is
@@ -450,6 +454,7 @@ public final class StreamLayerView: NSView {
installMouseMonitor() installMouseMonitor()
captured = true captured = true
window?.makeFirstResponder(self) window?.makeFirstResponder(self)
window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages
notifyCaptureChange(true) notifyCaptureChange(true)
} }
@@ -459,9 +464,28 @@ public final class StreamLayerView: NSView {
cursorCapture.release() cursorCapture.release()
inputCapture?.setForwarding(false) inputCapture?.setForwarding(false)
captured = false captured = false
window?.invalidateCursorRects(for: self)
notifyCaptureChange(false) notifyCaptureChange(false)
} }
/// A fully transparent cursor for the desktop mouse model's hide-over-view rect
/// an empty 1×1 image draws nothing.
private static let invisibleCursor = NSCursor(
image: NSImage(size: NSSize(width: 1, height: 1)), hotSpot: .zero)
/// Desktop mouse model: the local cursor is hidden while over the stream (the host's
/// composited cursor, tracking our absolute sends, is the one you see) and reappears
/// the moment it leaves the view AppKit applies/removes the rect's cursor for us,
/// so there is no hide/unhide balancing to get wrong. Capture model instead hides
/// globally via `CursorCapture` (the pointer can't leave the view there).
override public func resetCursorRects() {
if captured && desktopMouse {
addCursorRect(bounds, cursor: Self.invisibleCursor)
} else {
super.resetCursorRects()
}
}
/// A single local monitor for motion + buttons, installed only while captured. A local /// A single local monitor for motion + buttons, installed only while captured. A local
/// monitor is more robust than view overrides for relative motion: it sidesteps the /// monitor is more robust than view overrides for relative motion: it sidesteps the
/// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and /// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and
@@ -473,12 +497,12 @@ public final class StreamLayerView: NSView {
/// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're /// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're
/// inert locally. /// inert locally.
/// ///
/// In client-side-cursor mode the cursor is NOT frozen, so bare `.mouseMoved` events are /// In the desktop mouse model the cursor is NOT frozen, so bare `.mouseMoved` events are
/// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and /// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and
/// restore it on removal so absolute hover-motion keeps flowing without a click held. /// restore it on removal so absolute hover-motion keeps flowing without a click held.
private func installMouseMonitor() { private func installMouseMonitor() {
guard mouseEventMonitor == nil else { return } guard mouseEventMonitor == nil else { return }
if cursorVisible { if desktopMouse {
savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents
window?.acceptsMouseMovedEvents = true window?.acceptsMouseMovedEvents = true
} }
@@ -490,8 +514,8 @@ public final class StreamLayerView: NSView {
guard let self, self.captured, let ic = self.inputCapture else { return event } guard let self, self.captured, let ic = self.inputCapture else { return event }
switch event.type { switch event.type {
case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged: case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged:
if self.cursorVisible { if self.desktopMouse {
// Client-side cursor: forward the ABSOLUTE position (mapped through the // Desktop mouse model: forward the ABSOLUTE position (mapped through the
// aspect-fit letterbox into host pixels), the same path the iPad pointer // aspect-fit letterbox into host pixels), the same path the iPad pointer
// fallback uses. Events in the letterbox bars are dropped (nil host point). // fallback uses. Events in the letterbox bars are dropped (nil host point).
if let p = self.hostPoint(from: event) { if let p = self.hostPoint(from: event) {
@@ -609,14 +633,27 @@ public final class StreamLayerView: NSView {
// be a cursor trap with dead input. // be a cursor trap with dead input.
self?.releaseCapture() self?.releaseCapture()
} }
// C flips the client-side cursor live. Only the key window's stream owns it (same // M flips the mouse model (capture desktop) live the SDL clients' identical
// guard as the capture toggle). Re-engage capture in the new mode so disassociation // chord. Only the key window's stream owns it (same guard as the capture toggle).
// and the absolute/relative forwarding choice swap atomically releaseCapture restores // Re-engage capture in the new model so disassociation and the absolute/relative
// the old mode's grab (if any), engageCapture installs the new one. // forwarding choice swap atomically releaseCapture restores the old model's grab
// C would flip the client-side cursor live NEUTERED while the feature is disabled // (if any), engageCapture installs the new one. On a gamescope host the chord is a
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only // no-op: its EIS grants only a relative pointer, so the desktop model's absolute
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands. // sends would be silently dropped (pointer stuck = "all input dead").
capture.onToggleCursor = {} capture.onToggleMouseMode = { [weak self] in
guard let self, self.window?.isKeyWindow == true,
let conn = self.connection else { return }
guard conn.resolvedCompositor != .gamescope else {
streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only")
return
}
let wasCaptured = self.captured
if wasCaptured { self.releaseCapture() }
self.desktopMouse.toggle()
if wasCaptured { self.engageCapture(fromClick: false) }
self.window?.invalidateCursorRects(for: self)
streamInputLog.info("mouse mode: \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by // The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout. // the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in capture.onReleaseCapture = { [weak self] in
@@ -643,15 +680,18 @@ public final class StreamLayerView: NSView {
capture.start() capture.start()
inputCapture = capture inputCapture = capture
// Client-side cursor is TEMPORARILY DISABLED. It positions the host cursor with ABSOLUTE // Desktop (absolute) mouse model resolved at session start from the mouseMode
// events, but gamescope's input socket (EIS) grants only a relative pointer, so those are // setting, gated by the host's compositor: gamescope's input socket (EIS) grants
// silently dropped the pointer never moves and clicks/scroll land on the stuck position // only a relative pointer, so absolute sends would be silently dropped there
// (looks like "all input dead"). gamescope is exactly the compositor Auto enabled it for. // (pointer stuck = "all input dead") pinned to capture. M flips it live.
// Forced off until per-compositor gating (KWin/GNOME/Sway have absolute) or a synthetic- let mode = MouseInputMode(
// cursor-over-relative path lands; the resolution logic below is kept for that. See the rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? ""
// C handler (also neutered) and the cursorMode setting (hidden). ) ?? .capture
cursorVisible = false let absOK = connection.resolvedCompositor != .gamescope
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent) desktopMouse = mode == .desktop && absOK
if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
}
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2 // Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by // (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
@@ -94,8 +94,11 @@ public enum DefaultsKey {
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth. /// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444" public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts" public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never". /// Physical-mouse model (macOS): "capture" (pointer lock + relative, the default) or
public static let cursorMode = "punktfunk.cursorMode" /// "desktop" (uncaptured absolute pointer) the cross-client `mouse_mode`. Replaces the
/// never-shipped "punktfunk.cursorMode" (auto/always/never client-side-cursor setting,
/// which was hidden while disabled and had no readers).
public static let mouseMode = "punktfunk.mouseMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by /// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is /// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across /// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
+11 -2
View File
@@ -16,6 +16,8 @@
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games) # PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect) # PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990) # PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_CONNECT_TIMEOUT connect budget in seconds (optional; the plugin stretches it after
# firing Wake-on-LAN so the connect survives the host's resume)
# PF_APPID flatpak app id (default io.unom.Punktfunk) # PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH) # PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
# #
@@ -61,10 +63,17 @@ if [ -z "${PF_HOST:-}" ]; then
echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2 echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2
exit 2 exit 2
fi fi
# Trailing args shared by both streaming execs. A stretched connect budget rides along when the
# plugin set one (it just fired Wake-on-LAN, so the host may still be resuming); an older flatpak
# without --connect-timeout ignores the flag harmlessly (hand-scanned argv).
set -- --fullscreen
if [ -n "${PF_CONNECT_TIMEOUT:-}" ]; then
set -- --connect-timeout "$PF_CONNECT_TIMEOUT" "$@"
fi
if [ -n "${PF_LAUNCH:-}" ]; then if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title. # A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" "$@"
fi fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" "$@"
+32 -7
View File
@@ -70,7 +70,9 @@ function setShortcutHidden(appId: number, hidden: boolean): void {
}; };
// Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId). // Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId).
const ART_VERSION = 2; // v3: CI zips through 0.17.1 shipped no assets/ at all, yet v2 was still recorded as applied
// on those installs — the bump makes them re-apply once on the first build that has the files.
const ART_VERSION = 3;
function artKey(appId: number): string { function artKey(appId: number): string {
return `punktfunk:shortcutArt:${appId}`; return `punktfunk:shortcutArt:${appId}`;
} }
@@ -79,7 +81,7 @@ function artKey(appId: number): string {
* Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per * Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per
* appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call. * appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call.
*/ */
async function applyArtwork(appId: number): Promise<void> { async function applyArtwork(appId: number, isRetry = false): Promise<void> {
try { try {
if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) { if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) {
return; return;
@@ -91,16 +93,29 @@ async function applyArtwork(appId: number): Promise<void> {
[art.logo, 2], [art.logo, 2],
[art.gridwide, 3], [art.gridwide, 3],
]; ];
let applied = false;
for (const [data, assetType] of assets) { for (const [data, assetType] of assets) {
if (data) { if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType); await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
applied = true;
} }
} }
if (art.icon_path) { if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path); SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
applied = true;
} }
// Only record "done" when something actually landed — a plugin build whose assets/ is
// missing/empty must keep retrying on later mounts instead of poisoning the marker.
if (applied) {
localStorage.setItem(artKey(appId), `${ART_VERSION}`); localStorage.setItem(artKey(appId), `${ART_VERSION}`);
}
} catch (e) { } catch (e) {
// A shortcut fresh out of AddShortcut may not be registered yet (the same race
// setShortcutHidden defers around) — one deferred second attempt, then leave it to
// the next mount.
if (!isRetry) {
setTimeout(() => void applyArtwork(appId, true), 2500);
}
console.warn("punktfunk: shortcut artwork not applied", e); console.warn("punktfunk: shortcut artwork not applied", e);
} }
} }
@@ -157,7 +172,9 @@ async function ensureControllerConfig(): Promise<void> {
return; return;
} }
const r = await applyControllerConfig(SHORTCUT_NAME); const r = await applyControllerConfig(SHORTCUT_NAME);
if (r?.ok) { // `ok` alone isn't done: with zero account configset dirs (fresh Steam) the backend
// succeeds without pointing any account at the template — keep retrying until one lands.
if (r?.ok && (r.applied ?? []).some((a) => a.startsWith("configset:"))) {
localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`); localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`);
} else { } else {
console.warn("punktfunk: controller config not fully applied", r); console.warn("punktfunk: controller config not fully applied", r);
@@ -283,13 +300,21 @@ export async function launchStream(
opts: LaunchOpts = {}, opts: LaunchOpts = {},
): Promise<void> { ): Promise<void> {
// Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now // Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now
// so it races with the shortcut setup (near-zero added latency), and awaited just before RunGame. // so it races with the shortcut setup (near-zero added latency); its outcome is needed below
// (the connect budget), and RunGame follows the await either way, so nothing is slower for it.
// Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is // Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is
// known), and the connect that follows has its own retry window, so a failure never blocks launch. // known), and the connect that follows has its own retry window, so a failure never blocks launch.
const waking = wake(host, port).catch(() => ({ ok: false })); const waking = wake(host, port).catch(() => ({ ok: false }));
const { appId, runner } = await ensureStreamShortcut(); const [{ appId, runner }, woke] = await Promise.all([ensureStreamShortcut(), waking]);
const target = port && port !== 9777 ? `${host}:${port}` : host; const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`]; const env = [`PF_HOST=${target}`];
// A magic packet actually went out (a MAC was known), so the host may be mid-resume from
// suspend — that takes far longer than the client's default 15 s connect budget. Stretch the
// budget so the client's wake-tolerant dial keeps retrying across the resume; against an
// already-awake host the connect still lands in under a second, so this costs nothing.
if (woke.ok) {
env.push("PF_CONNECT_TIMEOUT=75");
}
if (opts.browse) { if (opts.browse) {
env.push("PF_BROWSE=1"); env.push("PF_BROWSE=1");
if (opts.mgmt) { if (opts.mgmt) {
@@ -303,9 +328,9 @@ export async function launchStream(
env.push(`PF_LAUNCH=${opts.launchId}`); env.push(`PF_LAUNCH=${opts.launchId}`);
} }
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper // KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_* from the environment. // script rides behind it as an argument and reads PF_* from the environment. The wake was
// awaited above, so the magic packet is out before the connect attempt.
SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`); SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
await waking; // ensure the magic packet is out before the connect attempt
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100); SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
} }
+31
View File
@@ -69,6 +69,14 @@ const TOUCH_MODE_CAPTIONS: &[&str] = &[
"The cursor jumps to your finger — a tap clicks there", "The cursor jumps to your finger — a tap clicks there",
"Real multi-touch reaches the host — for touch-native apps", "Real multi-touch reaches the host — for touch-native apps",
]; ];
/// Physical-mouse model values (persisted) + labels + dynamic captions — same idiom as
/// the touch rows. Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[&str] = &["capture", "desktop"];
const MOUSE_MODE_LABELS: &[&str] = &["Capture (games)", "Desktop (absolute)"];
const MOUSE_MODE_CAPTIONS: &[&str] = &[
"Pointer locks to the stream — relative motion, best for games",
"Pointer moves freely in and out — best for remote desktop work",
];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page. /// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page.
const APP_LICENSE: &str = concat!( const APP_LICENSE: &str = concat!(
@@ -542,6 +550,20 @@ pub fn show(
set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]); set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]);
}); });
} }
let mouse_row = ChoiceRow::new(
&dialog,
inline,
"Mouse input",
MOUSE_MODE_CAPTIONS[0],
MOUSE_MODE_LABELS,
);
{
let w = mouse_row.widget().clone();
mouse_row.connect_changed(move |i| {
let i = (i as usize).min(MOUSE_MODE_CAPTIONS.len() - 1);
set_row_subtitle(&w, MOUSE_MODE_CAPTIONS[i]);
});
}
let inhibit_row = adw::SwitchRow::builder() let inhibit_row = adw::SwitchRow::builder()
.title("Capture system shortcuts") .title("Capture system shortcuts")
.subtitle("Forward Alt+Tab, Super, … to the host while input is captured") .subtitle("Forward Alt+Tab, Super, … to the host while input is captured")
@@ -718,6 +740,12 @@ pub fn show(
touch_row.set_selected(touch_i as u32); touch_row.set_selected(touch_i as u32);
// set_selected never fires the changed hook, so seed the dynamic caption directly. // set_selected never fires the changed hook, so seed the dynamic caption directly.
set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]); set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]);
let mouse_i = MOUSE_MODES
.iter()
.position(|&m| m == s.mouse_mode)
.unwrap_or(0);
mouse_row.set_selected(mouse_i as u32);
set_row_subtitle(mouse_row.widget(), MOUSE_MODE_CAPTIONS[mouse_i]);
let comp_i = COMPOSITORS let comp_i = COMPOSITORS
.iter() .iter()
.position(|&c| c == s.compositor) .position(|&c| c == s.compositor)
@@ -788,6 +816,7 @@ pub fn show(
touch_group.add(touch_row.widget()); touch_group.add(touch_row.widget());
// Group titles are Pango markup — the ampersand must be an entity. // Group titles are Pango markup — the ampersand must be an entity.
let kbm_group = group("Keyboard &amp; mouse", ""); let kbm_group = group("Keyboard &amp; mouse", "");
kbm_group.add(mouse_row.widget());
kbm_group.add(&inhibit_row); kbm_group.add(&inhibit_row);
kbm_group.add(&invert_row); kbm_group.add(&invert_row);
input.add(&touch_group); input.add(&touch_group);
@@ -867,6 +896,8 @@ pub fn show(
} }
s.touch_mode = s.touch_mode =
TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string(); TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string();
s.mouse_mode =
MOUSE_MODES[(mouse_row.selected() as usize).min(MOUSE_MODES.len() - 1)].to_string();
s.forward_pad = chosen_pin.borrow().clone(); s.forward_pad = chosen_pin.borrow().clone();
s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)] s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)]
.to_string(); .to_string();
+12
View File
@@ -498,6 +498,13 @@ async fn session(args: Args) -> Result<()> {
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() { if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444; caps |= punktfunk_core::quic::VIDEO_CAP_444;
} }
// PUNKTFUNK_CLIENT_CHACHA20=1 advertises VIDEO_CAP_CHACHA20 — drives the
// host's ChaCha20-Poly1305 session-cipher resolution (the soft-AES armv7
// negotiation, design/chacha20-session-cipher.md §7) without a webOS build;
// the negotiated cipher is reported in the welcome log line below.
if std::env::var_os("PUNKTFUNK_CLIENT_CHACHA20").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_CHACHA20;
}
caps caps
}, },
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the // `--audio-channels` (default stereo); the probe multistream-decodes + validates the
@@ -535,6 +542,11 @@ async fn session(args: Args) -> Result<()> {
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444, chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format, chroma_format_idc = welcome.chroma_format,
codec = codec_ext(welcome.codec), codec = codec_ext(welcome.codec),
cipher = if welcome.cipher == punktfunk_core::quic::CIPHER_CHACHA20_POLY1305 {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
"session offer" "session offer"
); );
+1
View File
@@ -158,6 +158,7 @@ pub fn run(target: Option<&str>) -> u8 {
v => v, v => v,
}, },
touch_mode: settings_at_start.touch_mode(), touch_mode: settings_at_start.touch_mode(),
mouse_mode: settings_at_start.mouse_mode(),
invert_scroll: settings_at_start.invert_scroll, invert_scroll: settings_at_start.invert_scroll,
json_status, json_status,
on_connected: Some(Box::new(move |fingerprint: [u8; 32]| { on_connected: Some(Box::new(move |fingerprint: [u8; 32]| {
+6
View File
@@ -172,6 +172,11 @@ mod session_main {
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session // defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually. // pump) pins one manually.
display_hdr: None, display_hdr: None,
// The presenter renders the host cursor locally in desktop mouse mode (M2 cursor
// channel); capture-mode sessions keep the composited cursor, so only advertise
// when the session STARTS in desktop mode. The host gates further (Linux portal
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled, mic_enabled: settings.mic_enabled,
clipboard, clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter // The Settings preference (auto → VAAPI where it exists; the presenter
@@ -429,6 +434,7 @@ mod session_main {
v => v, v => v,
}, },
touch_mode: settings.touch_mode(), touch_mode: settings.touch_mode(),
mouse_mode: settings.mouse_mode(),
invert_scroll: settings.invert_scroll, invert_scroll: settings.invert_scroll,
json_status: true, json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| { on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
+18
View File
@@ -90,6 +90,13 @@ const TOUCH_MODES: &[(&str, &str)] = &[
("pointer", "Direct pointer"), ("pointer", "Direct pointer"),
("touch", "Touch passthrough"), ("touch", "Touch passthrough"),
]; ];
/// Physical-mouse presets: `(stored value, display label)` — capture (pointer lock,
/// relative, for games) vs desktop (uncaptured absolute pointer, for remote desktop
/// work). Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[(&str, &str)] = &[
("capture", "Capture (games)"),
("desktop", "Desktop (absolute)"),
];
/// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to /// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to
/// auto-detect when the choice is unavailable. Only meaningful against a Linux host. /// auto-detect when the choice is unavailable. Only meaningful against a Linux host.
const COMPOSITORS: &[(&str, &str)] = &[ const COMPOSITORS: &[(&str, &str)] = &[
@@ -394,6 +401,10 @@ pub(crate) fn settings_page(
let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| { let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| {
s.touch_mode = TOUCH_MODES[i].0.to_string(); s.touch_mode = TOUCH_MODES[i].0.to_string();
}); });
let (mouse_names, mouse_i) = presets(MOUSE_MODES, |v| *v == s.mouse_mode);
let mouse_combo = setting_combo(ctx, "Mouse input", mouse_names, mouse_i, |s, i| {
s.mouse_mode = MOUSE_MODES[i].0.to_string();
});
let invert_scroll_toggle = let invert_scroll_toggle =
setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| { setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| {
s.invert_scroll = on s.invert_scroll = on
@@ -542,6 +553,13 @@ pub(crate) fn settings_page(
out.extend(group( out.extend(group(
Some("Keyboard & mouse"), Some("Keyboard & mouse"),
vec![ vec![
described(
mouse_combo,
"Capture locks the pointer to the stream and sends relative motion — \
best for games. Desktop leaves the pointer free to enter and leave \
the stream and sends absolute positions best for remote desktop \
work. Ctrl+Alt+Shift+M switches live.",
),
described( described(
shortcuts_toggle, shortcuts_toggle,
"Alt+Tab, the Windows key and friends reach the host while the stream \ "Alt+Tab, the Windows key and friends reach the host while the stream \
+8
View File
@@ -868,6 +868,10 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves, /// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change. /// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64, serial: u64,
/// The compositor-reported hotspot — carried on the overlay for the cursor-forward
/// channel (the blend path uses the pre-adjusted `x`/`y` and never reads it).
hot_x: i32,
hot_y: i32,
} }
impl CursorState { impl CursorState {
@@ -884,6 +888,8 @@ mod pipewire {
h: self.bh, h: self.bh,
rgba: self.rgba.clone(), rgba: self.rgba.clone(),
serial: self.serial, serial: self.serial,
hot_x: self.hot_x.max(0) as u32,
hot_y: self.hot_y.max(0) as u32,
}) })
} }
} }
@@ -1380,6 +1386,8 @@ mod pipewire {
cursor.visible = true; cursor.visible = true;
cursor.x = pos_x - hot_x; cursor.x = pos_x - hot_x;
cursor.y = pos_y - hot_y; cursor.y = pos_y - hot_y;
cursor.hot_x = hot_x;
cursor.hot_y = hot_y;
if bmp_off == 0 { if bmp_off == 0 {
// Position-only update — keep the cached bitmap. // Position-only update — keep the cached bitmap.
return; return;
+12
View File
@@ -44,6 +44,13 @@ pub struct SessionParams {
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The /// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`. /// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool, pub clipboard: bool,
/// Advertise `quic::CLIENT_CAP_CURSOR`: this embedder renders the host cursor locally
/// (the presenter's cursor channel, design/remote-desktop-sweep.md M2), so the host may
/// stop compositing the pointer into the video. Only set when the embedder actually
/// draws it (the SDL presenter in desktop mouse mode) — a session that advertises it
/// without rendering streams with NO visible cursor. The host answers `HOST_CAP_CURSOR`
/// when its capture can forward (Linux portal, not gamescope/Windows).
pub cursor_forward: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see /// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
/// `video::Decoder::new`). /// `video::Decoder::new`).
pub decoder: String, pub decoder: String,
@@ -255,6 +262,11 @@ fn pump(
// This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an // This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an
// A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600). // A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600).
punktfunk_core::client::display_hdr_env_override().or(params.display_hdr), punktfunk_core::client::display_hdr_env_override().or(params.display_hdr),
if params.cursor_forward {
punktfunk_core::quic::CLIENT_CAP_CURSOR
} else {
0
},
params.launch.clone(), params.launch.clone(),
params.pin, params.pin,
Some(params.identity), Some(params.identity),
+57
View File
@@ -456,6 +456,48 @@ impl TouchMode {
} }
} }
/// How a physical mouse drives the host — the desktop-sweep mouse model
/// (design/remote-desktop-sweep.md M1). Stored stringly in [`Settings::mouse_mode`] so the
/// file stays readable; parsed with [`MouseMode::from_name`].
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum MouseMode {
/// Pointer lock (relative deltas, hidden cursor) — the game model, and the default:
/// the only cursor you see is the host's.
Capture,
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion goes on the wire as absolute positions through the letterbox. The remote
/// desktop model. Requires a host injector with absolute support (not gamescope).
Desktop,
}
impl MouseMode {
/// Cycle/picker order (also the settings pickers' option order).
pub const ALL: [MouseMode; 2] = [MouseMode::Capture, MouseMode::Desktop];
/// Parse the persisted name, defaulting to `Capture` for unset/unknown values.
pub fn from_name(s: &str) -> MouseMode {
match s {
"desktop" => MouseMode::Desktop,
_ => MouseMode::Capture,
}
}
/// The persisted name (the inverse of [`from_name`](Self::from_name)).
pub fn as_name(self) -> &'static str {
match self {
MouseMode::Capture => "capture",
MouseMode::Desktop => "desktop",
}
}
pub fn label(self) -> &'static str {
match self {
MouseMode::Capture => "Capture (games)",
MouseMode::Desktop => "Desktop (absolute)",
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file /// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time. /// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)] #[derive(Clone, Serialize, Deserialize)]
@@ -490,6 +532,12 @@ pub struct Settings {
/// stores load as trackpad. /// stores load as trackpad.
#[serde(default = "default_touch_mode")] #[serde(default = "default_touch_mode")]
pub touch_mode: String, pub touch_mode: String,
/// How a physical mouse drives the host: a [`MouseMode`] name — `"capture"` (default,
/// pointer lock + relative) or `"desktop"` (uncaptured absolute pointer). Read at
/// connect via [`Settings::mouse_mode`]. `default` so pre-existing stores load as
/// capture — today's behavior.
#[serde(default = "default_mouse_mode")]
pub mouse_mode: String,
/// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured. /// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured.
pub inhibit_shortcuts: bool, pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source. /// Stream the default microphone to the host's virtual mic source.
@@ -577,6 +625,10 @@ fn default_touch_mode() -> String {
"trackpad".into() "trackpad".into()
} }
fn default_mouse_mode() -> String {
"capture".into()
}
fn default_true() -> bool { fn default_true() -> bool {
true true
} }
@@ -604,6 +656,10 @@ impl Settings {
TouchMode::from_name(&self.touch_mode) TouchMode::from_name(&self.touch_mode)
} }
pub fn mouse_mode(&self) -> MouseMode {
MouseMode::from_name(&self.mouse_mode)
}
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto). /// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 { pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() { match self.codec.as_str() {
@@ -631,6 +687,7 @@ impl Default for Settings {
forward_pad: String::new(), forward_pad: String::new(),
compositor: "auto".into(), compositor: "auto".into(),
touch_mode: "trackpad".into(), touch_mode: "trackpad".into(),
mouse_mode: "capture".into(),
inhibit_shortcuts: true, inhibit_shortcuts: true,
mic_enabled: false, mic_enabled: false,
audio_channels: 2, audio_channels: 2,
+42 -2
View File
@@ -10,7 +10,7 @@ use crate::screens::{Ctx, Outbox};
use crate::theme::{Fonts, DIM, W}; use crate::theme::{Fonts, DIM, W};
use crate::widgets::{ListMsg, MenuList, RowSpec}; use crate::widgets::{ListMsg, MenuList, RowSpec};
use pf_client_core::gamepad::{MenuEvent, MenuPulse}; use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use skia_safe::{Canvas, Rect}; use skia_safe::{Canvas, Rect};
/// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale /// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale
@@ -29,10 +29,11 @@ enum RowId {
Pad, Pad,
PadType, PadType,
Touch, Touch,
Mouse,
Stats, Stats,
} }
const ROWS: [RowId; 13] = [ const ROWS: [RowId; 14] = [
RowId::Resolution, RowId::Resolution,
RowId::Refresh, RowId::Refresh,
RowId::Bitrate, RowId::Bitrate,
@@ -45,6 +46,7 @@ const ROWS: [RowId; 13] = [
RowId::Pad, RowId::Pad,
RowId::PadType, RowId::PadType,
RowId::Touch, RowId::Touch,
RowId::Mouse,
RowId::Stats, RowId::Stats,
]; ];
@@ -251,6 +253,7 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
"Touch mode", "Touch mode",
s.touch_mode().label().into(), s.touch_mode().label().into(),
), ),
RowId::Mouse => (None, "Mouse mode", s.mouse_mode().label().into()),
RowId::Stats => ( RowId::Stats => (
Some("Interface"), Some("Interface"),
"Statistics overlay", "Statistics overlay",
@@ -292,6 +295,11 @@ fn detail(id: RowId) -> &'static str {
"How the touchscreen drives the host: Trackpad (relative cursor), \ "How the touchscreen drives the host: Trackpad (relative cursor), \
Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)." Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)."
} }
RowId::Mouse => {
"How a physical mouse drives the host: Capture locks the pointer (relative, \
for games), Desktop leaves it free and sends absolute positions. \
Ctrl+Alt+Shift+M switches live while streaming."
}
RowId::Stats => { RowId::Stats => {
"How much the overlay shows: Compact (one line) → Normal → Detailed. \ "How much the overlay shows: Compact (one line) → Normal → Detailed. \
Ctrl+Alt+Shift+S cycles it live while streaming." Ctrl+Alt+Shift+S cycles it live while streaming."
@@ -367,6 +375,11 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, TouchMode::ALL.len(), delta, wrap) step_option(cur, TouchMode::ALL.len(), delta, wrap)
.map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string()) .map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string())
} }
RowId::Mouse => {
let cur = MouseMode::ALL.iter().position(|m| *m == s.mouse_mode());
step_option(cur, MouseMode::ALL.len(), delta, wrap)
.map(|i| s.mouse_mode = MouseMode::ALL[i].as_name().to_string())
}
RowId::Stats => { RowId::Stats => {
let cur = StatsVerbosity::ALL let cur = StatsVerbosity::ALL
.iter() .iter()
@@ -510,6 +523,33 @@ mod tests {
assert_eq!(ctx.settings.touch_mode, "trackpad"); assert_eq!(ctx.settings.touch_mode, "trackpad");
} }
#[test]
fn mouse_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
assert_eq!(settings.mouse_mode, "capture");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
// Capture → Desktop, then a step past the end is a boundary.
assert!(
!adjust(RowId::Mouse, -1, false, &mut ctx),
"already first = thud"
);
assert!(adjust(RowId::Mouse, 1, false, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "desktop");
assert!(!adjust(RowId::Mouse, 1, false, &mut ctx), "last = thud");
// A wraps back to the first.
assert!(adjust(RowId::Mouse, 1, true, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "capture");
}
#[test] #[test]
fn unknown_value_snaps_to_first() { fn unknown_value_snaps_to_first() {
let (mut settings, pads) = ctx_parts(); let (mut settings, pads) = ctx_parts();
+5
View File
@@ -196,6 +196,11 @@ pub struct CursorOverlay {
pub rgba: std::sync::Arc<Vec<u8>>, pub rgba: std::sync::Arc<Vec<u8>>,
/// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves. /// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves.
pub serial: u64, pub serial: u64,
/// Hotspot (the pixel that IS the pointer position) within `w`×`h`. The blend paths ignore
/// it (`x`/`y` are already hotspot-adjusted); the cursor-forward channel ships it to the
/// client so a locally-drawn OS cursor points with the right pixel.
pub hot_x: u32,
pub hot_y: u32,
} }
/// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of /// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of
+17
View File
@@ -63,6 +63,13 @@ pub struct HostConfig {
/// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0. /// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0.
/// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth). /// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool, pub four_four_four: bool,
/// `PUNKTFUNK_CHACHA20` — host policy gate for the negotiated ChaCha20-Poly1305 session
/// cipher (design/chacha20-session-cipher.md). **Default ON** (pure rollout safety — perf-only,
/// both AEADs are full-strength): the host merely *allows* it — a session only seals with
/// ChaCha when the client advertised `VIDEO_CAP_CHACHA20` (set by soft-AES armv7 clients,
/// e.g. webOS TVs, whose GCM decrypt caps at ~100 Mbps); everyone else stays AES-128-GCM.
/// `PUNKTFUNK_CHACHA20=0`/`false`/`off`/`no` disables.
pub chacha20: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation. /// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool, pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic). /// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -147,6 +154,16 @@ impl HostConfig {
) )
}) })
.unwrap_or(true), .unwrap_or(true),
// Default ON, explicit-off grammar (the client's VIDEO_CAP_CHACHA20 bit is the real
// per-session switch; see the field doc).
chacha20: val("PUNKTFUNK_CHACHA20")
.map(|s| {
!matches!(
s.trim().to_ascii_lowercase().as_str(),
"0" | "false" | "off" | "no"
)
})
.unwrap_or(true),
perf: flag("PUNKTFUNK_PERF"), perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"), video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"), compositor: val("PUNKTFUNK_COMPOSITOR"),
+119
View File
@@ -0,0 +1,119 @@
//! Client-side cursor rendering (design/remote-desktop-sweep.md M2): the host forwards the
//! pointer's SHAPE (reliable control stream, cached by serial) and per-frame STATE (lossy
//! `0xD0` — position/visibility), and WE draw it as a real OS cursor — pointer feel stops
//! paying the video round-trip (the Parsec/RDP model). Active only when the session
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use std::collections::HashMap;
use std::time::Duration;
/// Shape serials cached at most — cursors cycle through a handful of shapes (arrow, I-beam,
/// resize…); a runaway host can't grow the map past this (the cache resets, shapes re-arrive
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → built OS cursor. An SDL `Cursor` must outlive its `set()`, so the cache owns
/// every shape ever applied this session (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, Cursor>,
/// The serial currently installed via `Cursor::set` (`None` = default/system cursor).
installed: Option<u32>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
impl CursorChannel {
pub fn new(connector: &NativeClient) -> CursorChannel {
let negotiated = connector.host_caps() & HOST_CAP_CURSOR != 0;
if negotiated {
tracing::info!("cursor channel negotiated — host cursor renders locally");
}
CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
state: None,
}
}
/// Whether the host forwards the cursor this session (it no longer composites one).
pub fn negotiated(&self) -> bool {
self.negotiated
}
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal.
pub fn pump(&mut self, connector: &NativeClient, mouse: &MouseUtil, desktop_active: bool) {
if !self.negotiated {
return;
}
while let Ok(shape) = connector.next_cursor_shape(Duration::ZERO) {
if self.shapes.len() >= SHAPE_CACHE_MAX {
// Degenerate host: reset — live shapes re-install via the serial-miss path.
self.shapes.clear();
self.installed = None;
}
let mut data = shape.rgba;
let built = sdl3::surface::Surface::from_data(
&mut data,
shape.w as u32,
shape.h as u32,
shape.w as u32 * 4,
sdl3::pixels::PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())
.and_then(|surf| {
Cursor::from_surface(&surf, shape.hot_x as i32, shape.hot_y as i32)
.map_err(|e| e.to_string())
});
match built {
Ok(cursor) => {
// A re-sent serial replaces its entry; force re-install if it's current.
if self.installed == Some(shape.serial) {
self.installed = None;
}
self.shapes.insert(shape.serial, cursor);
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
}
if !desktop_active {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
Cursor::from_system(SystemCursor::Arrow)
.map(|c| c.set())
.ok();
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some(st.serial) {
if let Some(cursor) = self.shapes.get(&st.serial) {
cursor.set();
self.installed = Some(st.serial);
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
}
// Visibility follows the host (a host app hid its pointer ⇒ ours hides too). Queried,
// not shadowed, so apply_capture's own show/hide calls can never desync us.
if mouse.is_cursor_showing() != st.visible() {
mouse.show_cursor(st.visible());
}
}
}
+63 -3
View File
@@ -14,10 +14,17 @@
//! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are //! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are
//! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would //! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would
//! otherwise send a datagram per event). //! otherwise send a datagram per event).
//!
//! The DESKTOP mouse model (design/remote-desktop-sweep.md M1) reuses this same engage/
//! release state but never locks the pointer: the local cursor moves freely (hidden over
//! the window — the host's composited cursor is the one you see) and motion goes on the
//! wire as absolute positions through the letterbox (`MouseMoveAbs`, latest-wins per loop
//! iteration). Requires a host injector with absolute support — gamescope's EIS is
//! relative-only, so sessions there are pinned to capture ([`Capture::new`] `abs_ok`).
use crate::keymap_sdl; use crate::keymap_sdl;
use crate::touch::{Abs, Act, Gestures}; use crate::touch::{Abs, Act, Gestures};
use pf_client_core::trust::TouchMode; use pf_client_core::trust::{MouseMode, TouchMode};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
@@ -41,6 +48,13 @@ pub struct Capture {
held_buttons: HashSet<u32>, held_buttons: HashSet<u32>,
/// Relative motion not yet on the wire, summed per loop iteration. /// Relative motion not yet on the wire, summed per loop iteration.
pending_rel: (i32, i32), pending_rel: (i32, i32),
/// Desktop-model position not yet on the wire, latest-wins per loop iteration.
pending_abs: Option<Abs>,
/// The desktop (absolute, uncaptured) mouse model is active. Flipped live by the
/// Ctrl+Alt+Shift+M chord; never true unless `abs_ok`.
desktop: bool,
/// The host injector accepts `MouseMoveAbs` (any compositor but gamescope).
abs_ok: bool,
/// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space — /// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space —
/// precision surfaces deliver sub-unit deltas; truncating each event drops the tail. /// precision surfaces deliver sub-unit deltas; truncating each event drops the tail.
scroll_acc: (f64, f64), scroll_acc: (f64, f64),
@@ -70,10 +84,14 @@ fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, fl
} }
impl Capture { impl Capture {
/// `abs_ok` = the host injector accepts absolute pointer events; without it the
/// desktop model is unavailable and `mouse_mode` silently resolves to capture.
pub fn new( pub fn new(
connector: Arc<NativeClient>, connector: Arc<NativeClient>,
touch_mode: TouchMode, touch_mode: TouchMode,
invert_scroll: bool, invert_scroll: bool,
mouse_mode: MouseMode,
abs_ok: bool,
) -> Capture { ) -> Capture {
Capture { Capture {
connector, connector,
@@ -82,6 +100,9 @@ impl Capture {
held_keys: HashSet::new(), held_keys: HashSet::new(),
held_buttons: HashSet::new(), held_buttons: HashSet::new(),
pending_rel: (0, 0), pending_rel: (0, 0),
pending_abs: None,
desktop: abs_ok && mouse_mode == MouseMode::Desktop,
abs_ok,
scroll_acc: (0.0, 0.0), scroll_acc: (0.0, 0.0),
touch_slots: HashMap::new(), touch_slots: HashMap::new(),
touch_mode, touch_mode,
@@ -94,6 +115,24 @@ impl Capture {
self.captured self.captured
} }
/// The desktop (absolute, uncaptured) mouse model is active.
pub fn desktop(&self) -> bool {
self.desktop
}
/// Flip capture ⇄ desktop (the Ctrl+Alt+Shift+M chord). `None` = the host can't take
/// absolute pointer events (gamescope), so the chord has nothing to offer; otherwise
/// the new desktop state. Motion gathered under the old model never crosses modes.
pub fn toggle_desktop(&mut self) -> Option<bool> {
if !self.abs_ok {
return None;
}
self.desktop = !self.desktop;
self.pending_rel = (0, 0);
self.pending_abs = None;
Some(self.desktop)
}
/// Whether a regained focus should re-engage: yes unless the user released /// Whether a regained focus should re-engage: yes unless the user released
/// deliberately (the chord keeps its meaning across an Alt-Tab). /// deliberately (the chord keeps its meaning across an Alt-Tab).
pub fn should_reengage(&self) -> bool { pub fn should_reengage(&self) -> bool {
@@ -117,6 +156,7 @@ impl Capture {
return false; return false;
} }
self.pending_rel = (0, 0); // never flush motion gathered while captured self.pending_rel = (0, 0); // never flush motion gathered while captured
self.pending_abs = None;
for vk in self.held_keys.drain() { for vk in self.held_keys.drain() {
send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0); send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0);
} }
@@ -132,22 +172,42 @@ impl Capture {
true true
} }
/// Forward the coalesced motion delta, if any — one datagram per loop iteration. /// Forward the coalesced motion, if any — one datagram per loop iteration. Only one
/// of the two stores is ever populated (the run loop routes by [`desktop`](Self::desktop)).
pub fn flush_motion(&mut self) { pub fn flush_motion(&mut self) {
let (dx, dy) = std::mem::take(&mut self.pending_rel); let (dx, dy) = std::mem::take(&mut self.pending_rel);
if dx != 0 || dy != 0 { if dx != 0 || dy != 0 {
send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0); send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0);
} }
if let Some(a) = self.pending_abs.take() {
send(
&self.connector,
InputKind::MouseMoveAbs,
0,
a.x,
a.y,
Self::touch_flags(a.w, a.h),
);
}
} }
/// Relative motion (SDL relative mouse mode delivers raw deltas while locked). /// Relative motion (SDL relative mouse mode delivers raw deltas while locked).
pub fn on_motion(&mut self, xrel: f32, yrel: f32) { pub fn on_motion(&mut self, xrel: f32, yrel: f32) {
if self.captured { if self.captured && !self.desktop {
self.pending_rel.0 += xrel as i32; self.pending_rel.0 += xrel as i32;
self.pending_rel.1 += yrel as i32; self.pending_rel.1 += yrel as i32;
} }
} }
/// Desktop-model motion: the cursor's position mapped into the letterboxed content
/// rect. Latest-wins — intermediate positions carry no information the final one
/// doesn't (unlike deltas, which must sum).
pub fn on_motion_abs(&mut self, abs: Abs) {
if self.captured && self.desktop {
self.pending_abs = Some(abs);
}
}
pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) { pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) {
if !self.captured { if !self.captured {
return; return;
+2
View File
@@ -16,6 +16,8 @@
#[cfg(any(target_os = "linux", windows))] #[cfg(any(target_os = "linux", windows))]
pub mod csc; pub mod csc;
#[cfg(any(target_os = "linux", windows))]
pub mod cursor;
#[cfg(windows)] #[cfg(windows)]
pub mod d3d11; pub mod d3d11;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
+113 -20
View File
@@ -20,11 +20,11 @@ use crate::vk::{FrameInput, Presenter};
use anyhow::{Context as _, Result}; use anyhow::{Context as _, Result};
use pf_client_core::gamepad::GamepadService; use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats}; use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use pf_client_core::video::VulkanDecodeDevice; use pf_client_core::video::VulkanDecodeDevice;
use pf_client_core::video::{DecodedFrame, DecodedImage}; use pf_client_core::video::{DecodedFrame, DecodedImage};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::config::Mode; use punktfunk_core::config::{CompositorPref, Mode};
use sdl3::event::{Event, WindowEvent}; use sdl3::event::{Event, WindowEvent};
use sdl3::keyboard::Mod; use sdl3::keyboard::Mod;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
@@ -48,6 +48,11 @@ pub struct SessionOpts {
/// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per /// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per
/// session — a mouse-only client leaves this at the default and never sees a finger. /// session — a mouse-only client leaves this at the default and never sees a finger.
pub touch_mode: TouchMode, pub touch_mode: TouchMode,
/// Physical-mouse model: `Capture` (pointer lock + relative, the default) or `Desktop`
/// (uncaptured absolute pointer — design/remote-desktop-sweep.md M1). Ctrl+Alt+Shift+M
/// flips it live; silently resolves to capture on hosts without absolute injection
/// (gamescope).
pub mouse_mode: MouseMode,
/// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]). /// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]).
pub invert_scroll: bool, pub invert_scroll: bool,
/// Emit the `{"ready":true}` stdout line after the first presented frame. /// Emit the `{"ready":true}` stdout line after the first presented frame.
@@ -228,6 +233,9 @@ struct StreamState {
/// window-normalized position must be re-based onto the content rect). `None` until /// window-normalized position must be re-based onto the content rect). `None` until
/// the first frame; touches before then have nothing to map onto and are dropped. /// the first frame; touches before then have nothing to map onto and are dropped.
last_video: Option<(u32, u32)>, last_video: Option<(u32, u32)>,
/// Client-side cursor rendering (M2 cursor channel) — created with the connector; inert
/// when the host didn't negotiate the channel.
cursor_chan: Option<crate::cursor::CursorChannel>,
} }
impl StreamState { impl StreamState {
@@ -258,6 +266,7 @@ impl StreamState {
frames: wake_rx, frames: wake_rx,
connector: None, connector: None,
capture: None, capture: None,
cursor_chan: None,
force_software, force_software,
canceled: false, canceled: false,
ready_announced: false, ready_announced: false,
@@ -490,7 +499,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
WindowEvent::FocusLost => { WindowEvent::FocusLost => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(false) { if cap.release(false) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
tracing::info!("focus lost — input released"); tracing::info!("focus lost — input released");
} }
} }
@@ -501,7 +510,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.should_reengage() { if cap.should_reengage() {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
tracing::info!("focus gained — input recaptured"); tracing::info!("focus gained — input recaptured");
} }
} }
@@ -537,20 +546,39 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.captured() { if cap.captured() {
cap.release(true); cap.release(true);
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} else { } else {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} }
tracing::info!(captured = cap.captured(), "chord: release/engage"); tracing::info!(captured = cap.captured(), "chord: release/engage");
} }
continue; continue;
} }
// Mouse model flip (capture ⇄ desktop) — applies immediately when
// engaged; a released stream just changes what the next engage does.
if chord && sc == Scancode::M {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
match cap.toggle_desktop() {
Some(desktop) => {
if cap.captured() {
apply_capture(&mut window, &mouse, true, desktop);
}
tracing::info!(desktop, "chord: mouse mode");
}
None => tracing::info!(
"chord: mouse mode — host has no absolute pointer \
(gamescope), staying captured"
),
}
}
continue;
}
if chord && sc == Scancode::D { if chord && sc == Scancode::D {
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("chord: disconnect"); tracing::info!("chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
// The pump emits Ended(None); the end path routes per mode. // The pump emits Ended(None); the end path routes per mode.
} }
continue; continue;
@@ -583,17 +611,42 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.on_key_up(sc); cap.on_key_up(sc);
} }
} }
Event::MouseMotion { xrel, yrel, .. } => { Event::MouseMotion {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { x, y, xrel, yrel, ..
} => {
if let Some(st) = stream.as_mut() {
let video = st.last_video;
if let Some(cap) = st.capture.as_mut() {
if cap.desktop() {
// Desktop model: the cursor's window position through the
// letterbox (same mapping as a pointer-mode finger).
// Before the first decoded frame there is nothing to map
// onto — dropped, like touch.
if let Some(video) = video {
let (lw, lh) = window.size();
let nx = x / lw.max(1) as f32;
let ny = y / lh.max(1) as f32;
let (ax, ay, aw, ah) =
finger_to_content(window.size_in_pixels(), video, nx, ny);
cap.on_motion_abs(Abs {
x: ax,
y: ay,
w: aw,
h: ah,
});
}
} else {
cap.on_motion(xrel, yrel); cap.on_motion(xrel, yrel);
} }
} }
}
}
Event::MouseButtonDown { mouse_btn, .. } => { Event::MouseButtonDown { mouse_btn, .. } => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if !cap.captured() { if !cap.captured() {
// The engaging click is suppressed toward the host. // The engaging click is suppressed toward the host.
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} else { } else {
cap.on_button_down(mouse_btn); cap.on_button_down(mouse_btn);
} }
@@ -695,6 +748,17 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.flush_motion(); cap.flush_motion();
} }
// Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor —
// only meaningful in the desktop mouse model (capture's relative lock hides it).
if let Some(st) = stream.as_mut() {
if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) {
let desktop_active = st
.capture
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active);
}
}
// Text input follows the overlay's editing state (edge-triggered). // Text input follows the overlay's editing state (edge-triggered).
let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active()); let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active());
@@ -714,7 +778,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
while escape_rx.try_recv().is_ok() { while escape_rx.try_recv().is_ok() {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(true) { if cap.release(true) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
if fullscreen && !opts.fullscreen { if fullscreen && !opts.fullscreen {
@@ -727,7 +791,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("controller chord: disconnect"); tracing::info!("controller chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
@@ -818,10 +882,28 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
.ok(); .ok();
gamepad.attach(c.clone()); gamepad.attach(c.clone());
st.clock_offset = Some(c.clock_offset_shared()); st.clock_offset = Some(c.clock_offset_shared());
let mut cap = Capture::new(c.clone(), opts.touch_mode, opts.invert_scroll); // gamescope's EIS grants only a relative pointer — absolute sends
// would be dropped, so the desktop model is pinned off there. Auto
// (an older host that didn't say) stays allowed: Windows hosts and
// pre-Welcome-compositor Linux hosts both take absolute.
let abs_ok = c.resolved_compositor != CompositorPref::Gamescope;
if opts.mouse_mode == MouseMode::Desktop && !abs_ok {
tracing::info!(
"desktop mouse mode unavailable on a gamescope host \
(relative-only input) using capture"
);
}
let mut cap = Capture::new(
c.clone(),
opts.touch_mode,
opts.invert_scroll,
opts.mouse_mode,
abs_ok,
);
cap.engage(); // capture engages when the stream starts (ui_stream parity) cap.engage(); // capture engages when the stream starts (ui_stream parity)
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
st.capture = Some(cap); st.capture = Some(cap);
st.cursor_chan = Some(crate::cursor::CursorChannel::new(&c));
st.connector = Some(c); st.connector = Some(c);
if let Some(f) = opts.on_connected.as_mut() { if let Some(f) = opts.on_connected.as_mut() {
f(fingerprint); f(fingerprint);
@@ -870,7 +952,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = stream.take() { if let Some(st) = stream.take() {
st.shutdown(); st.shutdown();
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
if let Some(o) = overlay.as_mut() { if let Some(o) = overlay.as_mut() {
// A user-canceled dial ends silently — no error scene. // A user-canceled dial ends silently — no error scene.
if canceled { if canceled {
@@ -887,7 +969,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = &mut st.capture { if let Some(cap) = &mut st.capture {
cap.release(true); cap.release(true);
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
match &mode { match &mode {
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)), ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),
ModeCtl::Browse(_) => { ModeCtl::Browse(_) => {
@@ -1477,11 +1559,22 @@ impl ResizeIndicator {
/// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process /// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process
/// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor /// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor
/// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts). /// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts).
fn apply_capture(window: &mut sdl3::video::Window, mouse: &sdl3::mouse::MouseUtil, on: bool) { ///
mouse.set_relative_mouse_mode(window, on); /// The `desktop` mouse model never locks: the pointer roams (and leaves the window)
/// freely, the local cursor is hidden over the window — the host's composited cursor,
/// tracking our absolute sends, is the one you see (until the M2 cursor channel flips
/// who draws it) — and system chords stay local (a remote desktop is something you
/// Alt-Tab away from, not into). `desktop` only matters while `on`.
fn apply_capture(
window: &mut sdl3::video::Window,
mouse: &sdl3::mouse::MouseUtil,
on: bool,
desktop: bool,
) {
mouse.set_relative_mouse_mode(window, on && !desktop);
mouse.show_cursor(!on); mouse.show_cursor(!on);
#[cfg(windows)] #[cfg(windows)]
window.set_keyboard_grab(on); window.set_keyboard_grab(on && !desktop);
} }
/// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)? /// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)?
@@ -1599,7 +1692,7 @@ struct PresentedWindow {
/// The capture hints (`ui_stream` parity — the words the user reads while released). /// The capture hints (`ui_stream` parity — the words the user reads while released).
const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats"; Ctrl+Alt+Shift+M mouse mode · Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats";
const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave"; Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave";
+5
View File
@@ -33,6 +33,11 @@ reed-solomon-simd = "3.1" # GF(2^16) Leopard-RS, SIMD, O(n log n) — the w
# NOT interoperable.) See vendor/fec-rs/LICENSE (BSD-2-Clause). # NOT interoperable.) See vendor/fec-rs/LICENSE (BSD-2-Clause).
fec-rs = { path = "vendor/fec-rs" } fec-rs = { path = "vendor/fec-rs" }
aes-gcm = "0.10" # AES-128-GCM session crypto, matches GameStream aes-gcm = "0.10" # AES-128-GCM session crypto, matches GameStream
# ChaCha20-Poly1305 session crypto, negotiated by clients without hardware AES (the soft-AES
# armv7 targets — webOS TVs — where GCM caps decrypt at ~100 Mbps; ARX runs 4-7x faster there).
# Same RustCrypto `aead 0.5` generation as aes-gcm: identical trait/nonce/tag shapes, pure Rust,
# cross-compiles like aes-gcm (no cmake). See design/chacha20-session-cipher.md.
chacha20poly1305 = "0.10"
zerocopy = { version = "0.8", features = ["derive"] } zerocopy = { version = "0.8", features = ["derive"] }
bytes = "1" bytes = "1"
socket2 = { version = "0.6", features = [ socket2 = { version = "0.6", features = [
+22 -12
View File
@@ -1,7 +1,8 @@
//! Tier-1 microbenchmarks for the punktfunk/1 hot path — GPU-free, so they run in normal CI. //! Tier-1 microbenchmarks for the punktfunk/1 hot path — GPU-free, so they run in normal CI.
//! //!
//! Two layers: //! Two layers:
//! - `crypto/*` — the isolated AES-128-GCM primitives on one ~MTU shard. //! - `crypto/*` — the isolated AEAD primitives (AES-128-GCM + the negotiated
//! ChaCha20-Poly1305) on one ~MTU shard.
//! - `pipeline/*`— a whole frame through the real per-frame path end to end over the in-process //! - `pipeline/*`— a whole frame through the real per-frame path end to end over the in-process
//! loopback transport: FEC encode → AES-GCM seal → packetize → (loopback) → reassemble → //! loopback transport: FEC encode → AES-GCM seal → packetize → (loopback) → reassemble →
//! FEC decode → open. This is what a throughput/latency regression in the core would show up in. //! FEC decode → open. This is what a throughput/latency regression in the core would show up in.
@@ -11,11 +12,11 @@
use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput}; use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionCrypto; use punktfunk_core::crypto::{SessionCrypto, SessionKey};
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
use punktfunk_core::transport::loopback_pair; use punktfunk_core::transport::loopback_pair;
const TAG_LEN: usize = 16; // AES-GCM authentication tag const TAG_LEN: usize = 16; // AEAD authentication tag (GCM and Poly1305 share the size)
const SHARD: usize = punktfunk_core::config::mtu1500_shard_payload(); // one MTU-safe data shard const SHARD: usize = punktfunk_core::config::mtu1500_shard_payload(); // one MTU-safe data shard
fn cfg(role: Role, scheme: FecScheme) -> Config { fn cfg(role: Role, scheme: FecScheme) -> Config {
@@ -38,21 +39,29 @@ fn cfg(role: Role, scheme: FecScheme) -> Config {
shard_payload: SHARD, shard_payload: SHARD,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: true, // bench the real path — crypto is always on for punktfunk/1 encrypt: true, // bench the real path — crypto is always on for punktfunk/1
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4], salt: [1, 2, 3, 4],
loopback_drop_period: 0, // throughput run: no induced loss (loss-harness covers recovery) loopback_drop_period: 0, // throughput run: no induced loss (loss-harness covers recovery)
} }
} }
fn bench_crypto(c: &mut Criterion) { fn bench_crypto(c: &mut Criterion) {
let host = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Host); let mut g = c.benchmark_group("crypto");
let client = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Client); g.throughput(Throughput::Bytes(SHARD as u64));
// Both negotiated session AEADs. On the x86 / Apple Silicon this runs on, both must be
// line-rate-trivial — the chacha20 series is the host-side sealing-cost check for the
// negotiated soft-AES-armv7 path (design/chacha20-session-cipher.md §7). The AES series
// keeps its unsuffixed names so the CI regression compare retains its history.
for (suffix, key) in [
("", SessionKey::Aes128Gcm([7u8; 16])),
("_chacha20", SessionKey::ChaCha20Poly1305([7u8; 32])),
] {
let host = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Host);
let client = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Client);
let payload = vec![0xABu8; SHARD]; let payload = vec![0xABu8; SHARD];
let sealed = host.seal(0, &payload).unwrap(); let sealed = host.seal(0, &payload).unwrap();
let mut g = c.benchmark_group("crypto"); g.bench_function(format!("seal{suffix}"), |b| {
g.throughput(Throughput::Bytes(SHARD as u64));
g.bench_function("seal", |b| {
let mut seq = 0u64; let mut seq = 0u64;
b.iter(|| { b.iter(|| {
let ct = host.seal(seq, black_box(&payload)).unwrap(); let ct = host.seal(seq, black_box(&payload)).unwrap();
@@ -60,7 +69,7 @@ fn bench_crypto(c: &mut Criterion) {
black_box(ct) black_box(ct)
}) })
}); });
g.bench_function("seal_in_place", |b| { g.bench_function(format!("seal_in_place{suffix}"), |b| {
let mut seq = 0u64; let mut seq = 0u64;
let mut buf = vec![0xABu8; SHARD + TAG_LEN]; let mut buf = vec![0xABu8; SHARD + TAG_LEN];
b.iter(|| { b.iter(|| {
@@ -68,10 +77,10 @@ fn bench_crypto(c: &mut Criterion) {
seq += 1; seq += 1;
}) })
}); });
g.bench_function("open", |b| { g.bench_function(format!("open{suffix}"), |b| {
b.iter(|| black_box(client.open(0, black_box(&sealed)).unwrap())) b.iter(|| black_box(client.open(0, black_box(&sealed)).unwrap()))
}); });
g.bench_function("open_in_place", |b| { g.bench_function(format!("open_in_place{suffix}"), |b| {
// In-place open consumes the buffer, so each iteration restores the ciphertext first — // In-place open consumes the buffer, so each iteration restores the ciphertext first —
// one memcpy, mirroring what the recv ring does when the next datagram lands in the slot. // one memcpy, mirroring what the recv ring does when the next datagram lands in the slot.
let mut buf = sealed.clone(); let mut buf = sealed.clone();
@@ -80,6 +89,7 @@ fn bench_crypto(c: &mut Criterion) {
black_box(client.open_in_place(0, &mut buf).unwrap()); black_box(client.open_in_place(0, &mut buf).unwrap());
}) })
}); });
}
g.finish(); g.finish();
} }
+9 -1
View File
@@ -11,6 +11,7 @@
//! - Panics never cross the boundary: every entry point is wrapped in `catch_unwind`. //! - Panics never cross the boundary: every entry point is wrapped in `catch_unwind`.
use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use crate::crypto::SessionKey;
use crate::error::PunktfunkStatus; use crate::error::PunktfunkStatus;
use crate::input::InputEvent; use crate::input::InputEvent;
use crate::reanchor::{GateVerdict, ReanchorGate}; use crate::reanchor::{GateVerdict, ReanchorGate};
@@ -94,7 +95,10 @@ impl PunktfunkConfig {
shard_payload: self.shard_payload as usize, shard_payload: self.shard_payload as usize,
max_frame_bytes, max_frame_bytes,
encrypt: self.encrypt != 0, encrypt: self.encrypt != 0,
key: self.key, // The C ABI keeps its fixed 16-byte key and always selects AES-128-GCM — no
// ABI_VERSION bump. Raw-`Config` C embedders can't negotiate ChaCha; the Swift/
// Kotlin clients are aarch64 with AES CE and never want it.
key: SessionKey::Aes128Gcm(self.key),
salt: self.salt, salt: self.salt,
loopback_drop_period: self.loopback_drop_period, loopback_drop_period: self.loopback_drop_period,
}; };
@@ -1570,6 +1574,10 @@ unsafe fn connect_ex_impl(
// themselves (EDR / MediaCodec), so the host's EDID defaults are fine there. An `ex8` // themselves (EDR / MediaCodec), so the host's EDID defaults are fine there. An `ex8`
// variant can carry it if a passthrough embedder ever needs it. // variant can carry it if a passthrough embedder ever needs it.
None, None,
// No client_caps in the C ABI yet either: cursor-channel opt-in for Apple/Android
// arrives with the ABI v11 cursor poll fns — until an embedder can RENDER the
// forwarded cursor it must not ask the host to stop compositing it.
0,
launch, launch,
pin, pin,
identity, identity,
+50 -2
View File
@@ -42,8 +42,8 @@ pub use self::rumble::{ActuatorQuirks, RumbleCommand};
use self::control::{CtrlRequest, Negotiated}; use self::control::{CtrlRequest, Negotiated};
use self::frame_channel::{DecodeLatAcc, FrameChannel, FramePop}; use self::frame_channel::{DecodeLatAcc, FrameChannel, FramePop};
use self::planes::{ use self::planes::{
RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE, RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, CURSOR_SHAPE_QUEUE, CURSOR_STATE_QUEUE,
RUMBLE_QUEUE, HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE, RUMBLE_QUEUE,
}; };
use self::probe::ProbeState; use self::probe::ProbeState;
use self::pump::run_pump; use self::pump::run_pump;
@@ -93,6 +93,12 @@ pub struct NativeClient {
/// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises /// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises
/// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any). /// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any).
host_timing: Mutex<Receiver<crate::quic::HostTiming>>, host_timing: Mutex<Receiver<crate::quic::HostTiming>>,
/// Inbound cursor shapes (control-stream [`crate::quic::CursorShape`]) — only a session
/// that advertised [`quic::CLIENT_CAP_CURSOR`] against a [`quic::HOST_CAP_CURSOR`] host
/// ever receives any.
cursor_shape: Mutex<Receiver<crate::quic::CursorShape>>,
/// Inbound per-frame cursor state — `0xD0` datagrams (same negotiation gate as shapes).
cursor_state: Mutex<Receiver<crate::quic::CursorState>>,
input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>, input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>,
/// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker. /// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker.
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing /// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
@@ -316,6 +322,12 @@ impl NativeClient {
// display's EDID so host apps tone-map to the client's real panel; `None` = unknown/SDR // display's EDID so host apps tone-map to the client's real panel; `None` = unknown/SDR
// (the host keeps its built-in EDID defaults). See [`crate::quic::Hello::display_hdr`]. // (the host keeps its built-in EDID defaults). See [`crate::quic::Hello::display_hdr`].
display_hdr: Option<HdrMeta>, display_hdr: Option<HdrMeta>,
// Non-video client capabilities ([`crate::quic::Hello::client_caps`]) — set
// [`crate::quic::CLIENT_CAP_CURSOR`] ONLY if this embedder actually renders the host
// cursor locally (shape + state planes): the host stops compositing the pointer into
// the video for a session that advertises it, so a non-rendering embedder that sets it
// streams with NO visible cursor at all. `0` = today's composited behavior.
client_caps: u8,
launch: Option<String>, launch: Option<String>,
pin: Option<[u8; 32]>, pin: Option<[u8; 32]>,
identity: Option<(String, String)>, identity: Option<(String, String)>,
@@ -337,6 +349,10 @@ impl NativeClient {
let (clip_event_tx, clip_event_rx) = let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE); std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>(); let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>();
let (cursor_shape_tx, cursor_shape_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorShape>(CURSOR_SHAPE_QUEUE);
let (cursor_state_tx, cursor_state_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorState>(CURSOR_STATE_QUEUE);
let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>(); let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>();
let shutdown = Arc::new(AtomicBool::new(false)); let shutdown = Arc::new(AtomicBool::new(false));
let quit = Arc::new(AtomicBool::new(false)); let quit = Arc::new(AtomicBool::new(false));
@@ -390,9 +406,11 @@ impl NativeClient {
video_codecs, video_codecs,
preferred_codec, preferred_codec,
display_hdr, display_hdr,
client_caps,
launch, launch,
pin, pin,
identity, identity,
connect_timeout: timeout,
frames: frame_chan_w, frames: frame_chan_w,
audio_tx, audio_tx,
rumble_tx, rumble_tx,
@@ -400,6 +418,8 @@ impl NativeClient {
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx, host_timing_tx,
cursor_shape_tx,
cursor_state_tx,
input_rx, input_rx,
mic_rx, mic_rx,
rich_input_rx, rich_input_rx,
@@ -444,6 +464,8 @@ impl NativeClient {
hidout: Mutex::new(hidout_rx), hidout: Mutex::new(hidout_rx),
hdr_meta: Mutex::new(hdr_meta_rx), hdr_meta: Mutex::new(hdr_meta_rx),
host_timing: Mutex::new(host_timing_rx), host_timing: Mutex::new(host_timing_rx),
cursor_shape: Mutex::new(cursor_shape_rx),
cursor_state: Mutex::new(cursor_state_rx),
input_tx, input_tx,
mic_tx, mic_tx,
rich_input_tx, rich_input_tx,
@@ -891,6 +913,32 @@ impl NativeClient {
} }
} }
/// Pull the next host cursor shape (design/remote-desktop-sweep.md M2): RGBA bitmap +
/// hotspot, sent on pointer-bitmap change over the reliable control stream. The embedder
/// caches by `serial` and builds an OS cursor from it; [`NativeClient::next_cursor_state`]
/// references shapes by serial. Only a session that advertised
/// [`crate::quic::CLIENT_CAP_CURSOR`] against a capable host receives any. Same
/// timeout/closed semantics as [`NativeClient::next_hidout`].
pub fn next_cursor_shape(&self, timeout: Duration) -> Result<crate::quic::CursorShape> {
match self.cursor_shape.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-frame cursor state (`0xD0`): position, visibility and the M3
/// relative-mode hint, referencing a shape by serial. Latest-wins — an embedder should
/// drain the queue and apply only the newest. Same negotiation gate and timeout/closed
/// semantics as [`NativeClient::next_cursor_shape`].
pub fn next_cursor_state(&self, timeout: Duration) -> Result<crate::quic::CursorState> {
match self.cursor_state.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access /// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access
/// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network` /// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network`
/// split (`network = (received + clock_offset pts) host_us`); a stats consumer should /// split (`network = (received + clock_offset pts) host_us`); a stats consumer should
@@ -35,6 +35,16 @@ pub(crate) const HOST_TIMING_QUEUE: usize = 512;
/// a dropped fetch-request makes the serving stream time out and reset cleanly. /// a dropped fetch-request makes the serving stream time out and reset cleanly.
pub(crate) const CLIP_EVENT_QUEUE: usize = 32; pub(crate) const CLIP_EVENT_QUEUE: usize = 32;
/// Cursor-shape plane depth (control-stream [`crate::quic::CursorShape`], one per pointer-bitmap
/// change — human-paced). Overflow drops the newest (try_send); the next shape change or a
/// serial mismatch against `0xD0` state heals it visually within a shape-change period.
pub(crate) const CURSOR_SHAPE_QUEUE: usize = 8;
/// Cursor-state plane depth (`0xD0`, one datagram per captured frame). Latest-wins state — the
/// embedder drains per present; a tiny ring only bridges scheduling jitter. Overflow drops the
/// newest (try_send), healed by the very next frame's datagram.
pub(crate) const CURSOR_STATE_QUEUE: usize = 8;
/// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames). /// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames).
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct AudioPacket { pub struct AudioPacket {
+4
View File
@@ -51,6 +51,8 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx, host_timing_tx,
cursor_shape_tx,
cursor_state_tx,
input_rx, input_rx,
mut mic_rx, mut mic_rx,
mut rich_input_rx, mut rich_input_rx,
@@ -123,6 +125,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
clock_offset: clock_offset.clone(), clock_offset: clock_offset.clone(),
clock_gen: clock_gen.clone(), clock_gen: clock_gen.clone(),
clip_event_tx: clip_event_tx.clone(), clip_event_tx: clip_event_tx.clone(),
cursor_shape_tx,
} }
.run(), .run(),
); );
@@ -136,6 +139,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx, host_timing_tx,
cursor_state_tx,
)); ));
// Clipboard task: the fetch-stream accept loop (host pulls what we offered) + outbound fetches // Clipboard task: the fetch-stream accept loop (host pulls what we offered) + outbound fetches
@@ -21,6 +21,9 @@ pub(super) struct ControlTask {
/// Clipboard metadata events (ClipState/ClipOffer) feed the same event plane the /// Clipboard metadata events (ClipState/ClipOffer) feed the same event plane the
/// clipboard task uses for fetch data. /// clipboard task uses for fetch data.
pub(super) clip_event_tx: std::sync::mpsc::SyncSender<ClipEventCore>, pub(super) clip_event_tx: std::sync::mpsc::SyncSender<ClipEventCore>,
/// Host cursor shapes ([`CursorShape`], sent on pointer-bitmap change) → the embedder's
/// shape plane ([`NativeClient::next_cursor_shape`]).
pub(super) cursor_shape_tx: std::sync::mpsc::SyncSender<crate::quic::CursorShape>,
} }
impl ControlTask { impl ControlTask {
@@ -36,6 +39,7 @@ impl ControlTask {
clock_offset, clock_offset,
clock_gen, clock_gen,
clip_event_tx, clip_event_tx,
cursor_shape_tx,
} = self; } = self;
// Mid-stream clock re-sync (see [`ClockResync`]): a batch runs every // Mid-stream clock re-sync (see [`ClockResync`]): a batch runs every
// CLOCK_RESYNC_INTERVAL and whenever the pump asks (CtrlRequest::ClockResync after // CLOCK_RESYNC_INTERVAL and whenever the pump asks (CtrlRequest::ClockResync after
@@ -167,6 +171,10 @@ impl ControlTask {
seq: offer.seq, seq: offer.seq,
kinds: offer.kinds, kinds: offer.kinds,
}); });
} else if let Ok(shape) = crate::quic::CursorShape::decode(&msg) {
// Pointer bitmap changed (cursor channel, only when negotiated). try_send:
// an overflowing ring drops the newest shape — the next change resends.
let _ = cursor_shape_tx.try_send(shape);
} else { } else {
tracing::warn!( tracing::warn!(
tag = ?msg.first(), tag = ?msg.first(),
@@ -3,6 +3,9 @@
use super::*; use super::*;
// One parameter per demuxed plane — grouping them into a struct would just move the field
// list one hop away from the single call site.
#[allow(clippy::too_many_arguments)]
pub(super) async fn run( pub(super) async fn run(
conn: quinn::Connection, conn: quinn::Connection,
audio_tx: std::sync::mpsc::SyncSender<AudioPacket>, audio_tx: std::sync::mpsc::SyncSender<AudioPacket>,
@@ -11,6 +14,7 @@ pub(super) async fn run(
hidout_tx: std::sync::mpsc::SyncSender<crate::quic::HidOutput>, hidout_tx: std::sync::mpsc::SyncSender<crate::quic::HidOutput>,
hdr_meta_tx: std::sync::mpsc::SyncSender<crate::quic::HdrMeta>, hdr_meta_tx: std::sync::mpsc::SyncSender<crate::quic::HdrMeta>,
host_timing_tx: std::sync::mpsc::SyncSender<crate::quic::HostTiming>, host_timing_tx: std::sync::mpsc::SyncSender<crate::quic::HostTiming>,
cursor_state_tx: std::sync::mpsc::SyncSender<crate::quic::CursorState>,
) { ) {
// Per-pad reorder gate for v2 rumble envelopes (the seq analog of the host's gamepad-state // Per-pad reorder gate for v2 rumble envelopes (the seq analog of the host's gamepad-state
// gate): a datagram the network reordered must not roll a stopped motor back on. Legacy v1 // gate): a datagram the network reordered must not roll a stopped motor back on. Legacy v1
@@ -73,6 +77,11 @@ pub(super) async fn run(
let _ = host_timing_tx.try_send(t); let _ = host_timing_tx.try_send(t);
} }
} }
Some(&crate::quic::CURSOR_STATE_MAGIC) => {
if let Some(s) = crate::quic::decode_cursor_state_datagram(&d) {
let _ = cursor_state_tx.try_send(s);
}
}
_ => {} // unknown tag — a newer host; ignore _ => {} // unknown tag — a newer host; ignore
} }
} }
@@ -32,21 +32,65 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
identity.as_ref().map(|(c, k)| (c.as_str(), k.as_str())), identity.as_ref().map(|(c, k)| (c.as_str(), k.as_str())),
); );
let ep = ep.map_err(|e| PunktfunkError::Io(std::io::Error::other(e.to_string())))?; let ep = ep.map_err(|e| PunktfunkError::Io(std::io::Error::other(e.to_string())))?;
let conn = ep // Dial with retry across the connect budget, not a single attempt: one quinn dial gives
// up after the transport's idle window (~8 s of silence), which is shorter than a
// suspend-to-RAM resume — the Steam Deck flow fires Wake-on-LAN and connects
// immediately, so the host is still waking while the first Initials go out, and a
// single-shot dial died just before the host came up. Short attempts keep the Initial
// cadence dense (quinn's per-attempt retransmits back off toward multi-second gaps), so
// the connect lands within ~a second of the host's network returning. Only SILENCE is
// retried: a host that answers and rejects us (pin mismatch, ALPN/version, typed close)
// must surface immediately, and the embedder's shutdown flag (budget expiry in
// `connect`, or a user cancel) stops the loop between attempts.
const DIAL_ATTEMPT: std::time::Duration = std::time::Duration::from_secs(3);
// Redial headroom: leave room for the control handshake (Hello/Welcome/clock sync)
// after a late dial success, so it still completes inside the embedder's budget.
const CONTROL_HEADROOM: std::time::Duration = std::time::Duration::from_secs(2);
let start = tokio::time::Instant::now();
let deadline = start + args.connect_timeout;
let redial_until = start + args.connect_timeout.saturating_sub(CONTROL_HEADROOM);
let conn = loop {
let connecting = ep
.connect(remote, "punktfunk") .connect(remote, "punktfunk")
.map_err(|_| PunktfunkError::InvalidArg("connect"))? .map_err(|_| PunktfunkError::InvalidArg("connect"))?;
.await // Cap the attempt to the remaining budget so a success never lands after the
.map_err(|e| { // embedder's `ready_rx` wait has already given up and flagged a teardown.
let now = tokio::time::Instant::now();
let attempt = DIAL_ATTEMPT.min(deadline.saturating_duration_since(now));
let gave_up = || {
tokio::time::Instant::now() >= redial_until
|| shutdown.load(std::sync::atomic::Ordering::SeqCst)
};
match tokio::time::timeout(attempt, connecting).await {
Ok(Ok(conn)) => break conn,
Ok(Err(e)) => {
// A pin mismatch surfaces as a TLS failure; report it as a crypto error so // A pin mismatch surfaces as a TLS failure; report it as a crypto error so
// the embedder can distinguish "wrong host identity" from plain IO trouble. // the embedder can distinguish "wrong host identity" from plain IO trouble.
let fp_mismatch = let fp_mismatch = pin.is_some()
pin.is_some() && observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true); && observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true);
if fp_mismatch { if fp_mismatch {
PunktfunkError::Crypto return Err(PunktfunkError::Crypto);
} else {
PunktfunkError::Io(std::io::Error::other(e.to_string()))
} }
})?; // The transport's own idle expiry — the host never answered — is the one
// retryable outcome; everything else is a real answer or a local failure.
let host_silent = matches!(e, quinn::ConnectionError::TimedOut);
if !host_silent {
return Err(PunktfunkError::Io(std::io::Error::other(e.to_string())));
}
if gave_up() {
return Err(PunktfunkError::Timeout);
}
}
// Attempt window elapsed with the host still silent; dropping `connecting`
// abandoned that dial — go again unless the budget is spent.
Err(_) => {
if gave_up() {
return Err(PunktfunkError::Timeout);
}
}
}
tracing::debug!(%remote, "host silent — re-dialing (wake/resume tolerant connect)");
};
let fingerprint = observed.lock().unwrap().unwrap_or([0u8; 32]); let fingerprint = observed.lock().unwrap().unwrap_or([0u8; 32]);
// The rest of the handshake runs in an inner future so a failure can consult // The rest of the handshake runs in an inner future so a failure can consult
// `conn.close_reason()`: a host that turned us away with a typed application close // `conn.close_reason()`: a host that turned us away with a typed application close
@@ -99,6 +143,10 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
// The client display's HDR volume → the host's virtual-display EDID (host apps // The client display's HDR volume → the host's virtual-display EDID (host apps
// tone-map to the client's real panel). `None` = unknown/SDR. // tone-map to the client's real panel). `None` = unknown/SDR.
display_hdr, display_hdr,
// NOT unconditional like HOST_TIMING above: CLIENT_CAP_CURSOR makes the host
// stop compositing the pointer, so only an embedder that actually renders the
// cursor locally may set it (the embedder decides, we pass through).
client_caps: args.client_caps,
} }
.encode(), .encode(),
) )
@@ -22,9 +22,14 @@ pub(crate) struct WorkerArgs {
pub(crate) video_codecs: u8, pub(crate) video_codecs: u8,
pub(crate) preferred_codec: u8, pub(crate) preferred_codec: u8,
pub(crate) display_hdr: Option<HdrMeta>, pub(crate) display_hdr: Option<HdrMeta>,
pub(crate) client_caps: u8,
pub(crate) launch: Option<String>, pub(crate) launch: Option<String>,
pub(crate) pin: Option<[u8; 32]>, pub(crate) pin: Option<[u8; 32]>,
pub(crate) identity: Option<(String, String)>, pub(crate) identity: Option<(String, String)>,
/// The embedder's connect budget (the same value `connect` bounds `ready_rx` with): the
/// dial loop re-dials a silent host within it, so a host still resuming from Wake-on-LAN
/// is caught the moment its network comes back instead of failing on the first attempt.
pub(crate) connect_timeout: std::time::Duration,
pub(crate) frames: Arc<FrameChannel>, pub(crate) frames: Arc<FrameChannel>,
pub(crate) audio_tx: SyncSender<AudioPacket>, pub(crate) audio_tx: SyncSender<AudioPacket>,
pub(crate) rumble_tx: SyncSender<RumbleUpdate>, pub(crate) rumble_tx: SyncSender<RumbleUpdate>,
@@ -34,6 +39,8 @@ pub(crate) struct WorkerArgs {
pub(crate) hidout_tx: SyncSender<HidOutput>, pub(crate) hidout_tx: SyncSender<HidOutput>,
pub(crate) hdr_meta_tx: SyncSender<HdrMeta>, pub(crate) hdr_meta_tx: SyncSender<HdrMeta>,
pub(crate) host_timing_tx: SyncSender<crate::quic::HostTiming>, pub(crate) host_timing_tx: SyncSender<crate::quic::HostTiming>,
pub(crate) cursor_shape_tx: SyncSender<crate::quic::CursorShape>,
pub(crate) cursor_state_tx: SyncSender<crate::quic::CursorState>,
pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>, pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>, pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>,
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>, pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
+15 -6
View File
@@ -1,5 +1,6 @@
//! Session configuration and protocol/FEC parameters. //! Session configuration and protocol/FEC parameters.
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
use crate::packet::{CRYPTO_OVERHEAD, HEADER_LEN, MAX_DATAGRAM_BYTES}; use crate::packet::{CRYPTO_OVERHEAD, HEADER_LEN, MAX_DATAGRAM_BYTES};
use zeroize::Zeroize; use zeroize::Zeroize;
@@ -355,9 +356,11 @@ pub struct Config {
/// hostile/corrupt headers; see [`Session`](crate::session::Session)). /// hostile/corrupt headers; see [`Session`](crate::session::Session)).
pub max_frame_bytes: usize, pub max_frame_bytes: usize,
pub encrypt: bool, pub encrypt: bool,
/// AES-128 session key established during pairing. MUST be unique per session when /// The negotiated session AEAD + its key, established during pairing/handshake —
/// AES-128-GCM for every peer by default, ChaCha20-Poly1305 when the client negotiated it
/// (soft-AES armv7 targets; see [`SessionKey`]). MUST be unique per session when
/// `encrypt` is set (see the nonce-uniqueness contract in [`crate::crypto`]). /// `encrypt` is set (see the nonce-uniqueness contract in [`crate::crypto`]).
pub key: [u8; 16], pub key: SessionKey,
/// Per-session nonce salt, established alongside `key` during pairing. MUST be /// Per-session nonce salt, established alongside `key` during pairing. MUST be
/// unique per (key, session). /// unique per (key, session).
pub salt: [u8; 4], pub salt: [u8; 4],
@@ -382,7 +385,8 @@ impl std::fmt::Debug for Config {
.field("shard_payload", &self.shard_payload) .field("shard_payload", &self.shard_payload)
.field("max_frame_bytes", &self.max_frame_bytes) .field("max_frame_bytes", &self.max_frame_bytes)
.field("encrypt", &self.encrypt) .field("encrypt", &self.encrypt)
.field("key", &"<redacted>") // SessionKey's own Debug redacts the material but keeps the cipher choice visible.
.field("key", &self.key)
.field("salt", &"<redacted>") .field("salt", &"<redacted>")
.field("loopback_drop_period", &self.loopback_drop_period) .field("loopback_drop_period", &self.loopback_drop_period)
.finish() .finish()
@@ -426,7 +430,7 @@ impl Config {
"max_frame_bytes too large for this shard/block configuration (block count overflows u16)", "max_frame_bytes too large for this shard/block configuration (block count overflows u16)",
)); ));
} }
if self.encrypt && self.key == [0u8; 16] { if self.encrypt && self.key.is_zero() {
return Err(PunktfunkError::InvalidArg( return Err(PunktfunkError::InvalidArg(
"encrypt requires a non-zero session key (see crypto nonce-uniqueness contract)", "encrypt requires a non-zero session key (see crypto nonce-uniqueness contract)",
)); ));
@@ -449,7 +453,7 @@ impl Config {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 64 * 1024 * 1024, max_frame_bytes: 64 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
@@ -465,7 +469,12 @@ mod tests {
let mut c = Config::p1_defaults(Role::Host); let mut c = Config::p1_defaults(Role::Host);
c.encrypt = true; // key is still all-zero c.encrypt = true; // key is still all-zero
assert!(c.validate().is_err()); assert!(c.validate().is_err());
c.key = [1u8; 16]; c.key = SessionKey::Aes128Gcm([1u8; 16]);
assert!(c.validate().is_ok());
// The rejection follows whichever cipher variant is active.
c.key = SessionKey::ChaCha20Poly1305([0u8; 32]);
assert!(c.validate().is_err());
c.key = SessionKey::ChaCha20Poly1305([1u8; 32]);
assert!(c.validate().is_ok()); assert!(c.validate().is_ok());
} }
+194 -34
View File
@@ -1,9 +1,10 @@
//! AES-128-GCM session sealing, matching GameStream's video crypto in P1. //! Session sealing with the negotiated AEAD — AES-128-GCM (matching GameStream's video
//! crypto in P1) by default, ChaCha20-Poly1305 (RFC 8439) for clients without hardware AES.
//! //!
//! ## Nonce uniqueness (the GCM safety requirement) //! ## Nonce uniqueness (the AEAD safety requirement)
//! //!
//! The 96-bit nonce is `salt (4 bytes) || sequence (8 bytes, big-endian)`. Reusing a //! The 96-bit nonce is `salt (4 bytes) || sequence (8 bytes, big-endian)`. Reusing a
//! `(key, nonce)` pair under AES-GCM is catastrophic, so two precautions apply: //! `(key, nonce)` pair is catastrophic under either AEAD, so two precautions apply:
//! //!
//! 1. **Per-direction salts.** Host and client share one `key` and `salt`, and each //! 1. **Per-direction salts.** Host and client share one `key` and `salt`, and each
//! counts its sequence from 0. To stop the host's video stream and the client's input //! counts its sequence from 0. To stop the host's video stream and the client's input
@@ -17,17 +18,96 @@
//! The sequence number is also passed as AEAD associated data, so tampering with the //! The sequence number is also passed as AEAD associated data, so tampering with the
//! on-wire sequence is detected (the tag check fails) rather than silently shifting the //! on-wire sequence is detected (the tag check fails) rather than silently shifting the
//! nonce. Note: this layer does not provide anti-replay — see `Session`. //! nonce. Note: this layer does not provide anti-replay — see `Session`.
//!
//! ## Why two ciphers
//!
//! Both AEADs are full-strength; the choice (negotiated via `Welcome::cipher`) is purely a
//! performance one. On targets without hardware AES — the soft-AES armv7 clients (webOS TVs) —
//! GCM's fixsliced AES + software GHASH costs ~50100 cycles/byte and caps decrypt at
//! ~100 Mbps, while ChaCha20-Poly1305's ARX construction runs ~1017 cycles/byte in portable
//! software (design/chacha20-session-cipher.md). Same 96-bit nonce, 16-byte tag, and AAD
//! shape, so the entire nonce discipline above carries over verbatim.
use crate::config::Role; use crate::config::Role;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
use aes_gcm::aead::{Aead, AeadInPlace, KeyInit, Payload}; use aes_gcm::aead::{Aead, AeadInPlace, KeyInit, Payload};
use aes_gcm::{Aes128Gcm, Key, Nonce}; use aes_gcm::{Aes128Gcm, Key, Nonce};
use chacha20poly1305::ChaCha20Poly1305;
use zeroize::Zeroize;
/// 16-byte AEAD authentication tag appended by GCM. /// 16-byte AEAD authentication tag appended by either session cipher.
pub const TAG_LEN: usize = 16; pub const TAG_LEN: usize = 16;
// The wire (CRYPTO_OVERHEAD) and every in-place split assume both negotiated AEADs append
// exactly TAG_LEN bytes — a different-tag cipher can never slip in behind this constant.
const _: () = assert!(std::mem::size_of::<aes_gcm::Tag>() == TAG_LEN);
const _: () = assert!(std::mem::size_of::<chacha20poly1305::Tag>() == TAG_LEN);
/// The negotiated session AEAD together with its key material — merged so the invalid state
/// (a ChaCha cipher with an AES-sized key, or vice versa) is unrepresentable. AES-128-GCM is
/// the default every peer speaks; ChaCha20-Poly1305 is granted to clients that advertised
/// [`VIDEO_CAP_CHACHA20`](crate::quic::VIDEO_CAP_CHACHA20) (the soft-AES armv7 targets —
/// see the module docs). 256 bits for ChaCha is what RFC 8439 requires.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum SessionKey {
Aes128Gcm([u8; 16]),
ChaCha20Poly1305([u8; 32]),
}
impl SessionKey {
/// Canonical lowercase cipher name for session-start logs.
pub fn cipher_name(&self) -> &'static str {
match self {
SessionKey::Aes128Gcm(_) => "aes-128-gcm",
SessionKey::ChaCha20Poly1305(_) => "chacha20-poly1305",
}
}
/// True when the key material is all zeros — the pairing-layer footgun `Config::validate`
/// rejects when encryption is on (see the nonce-uniqueness contract in the module docs).
pub fn is_zero(&self) -> bool {
match self {
SessionKey::Aes128Gcm(k) => k == &[0u8; 16],
SessionKey::ChaCha20Poly1305(k) => k == &[0u8; 32],
}
}
}
/// Key material never appears in logs, whichever variant is active — only the cipher choice
/// (`Config`'s hand-written `Debug` relies on this).
impl std::fmt::Debug for SessionKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SessionKey::Aes128Gcm(_) => f.write_str("Aes128Gcm(<redacted>)"),
SessionKey::ChaCha20Poly1305(_) => f.write_str("ChaCha20Poly1305(<redacted>)"),
}
}
}
/// Same zeroize-on-drop discipline the raw key array had (`Config`'s `Drop`).
impl Zeroize for SessionKey {
fn zeroize(&mut self) {
match self {
SessionKey::Aes128Gcm(k) => k.zeroize(),
SessionKey::ChaCha20Poly1305(k) => k.zeroize(),
}
}
}
/// The two negotiated AEADs behind one seal/open surface. Both are the same RustCrypto
/// `aead 0.5` generation (identical trait shapes, nonce/tag types), so each call below is a
/// two-arm match right next to the cipher work itself.
// AES's precomputed round keys (~0.7 KB) dwarf ChaCha's 32-byte state, but there is exactly
// one long-lived `SessionCrypto` per session — boxing the variant would trade that one-off
// slack for a pointer chase on every per-datagram seal/open.
#[allow(clippy::large_enum_variant)]
enum Cipher {
Aes128Gcm(Aes128Gcm),
ChaCha20Poly1305(ChaCha20Poly1305),
}
pub struct SessionCrypto { pub struct SessionCrypto {
cipher: Aes128Gcm, cipher: Cipher,
/// Salt for nonces we seal with (our direction). /// Salt for nonces we seal with (our direction).
send_salt: [u8; 4], send_salt: [u8; 4],
/// Salt for nonces we open with (the peer's direction). /// Salt for nonces we open with (the peer's direction).
@@ -35,11 +115,18 @@ pub struct SessionCrypto {
} }
impl SessionCrypto { impl SessionCrypto {
pub fn new(key: &[u8; 16], salt: [u8; 4], role: Role) -> Self { pub fn new(key: &SessionKey, salt: [u8; 4], role: Role) -> Self {
let key = Key::<Aes128Gcm>::from_slice(key); let cipher = match key {
SessionKey::Aes128Gcm(k) => {
Cipher::Aes128Gcm(Aes128Gcm::new(Key::<Aes128Gcm>::from_slice(k)))
}
SessionKey::ChaCha20Poly1305(k) => Cipher::ChaCha20Poly1305(ChaCha20Poly1305::new(
Key::<ChaCha20Poly1305>::from_slice(k),
)),
};
let own = direction(role); let own = direction(role);
SessionCrypto { SessionCrypto {
cipher: Aes128Gcm::new(key), cipher,
send_salt: dir_salt(salt, own), send_salt: dir_salt(salt, own),
recv_salt: dir_salt(salt, own ^ 1), recv_salt: dir_salt(salt, own ^ 1),
} }
@@ -49,14 +136,15 @@ impl SessionCrypto {
/// authenticated as associated data. /// authenticated as associated data.
pub fn seal(&self, seq: u64, plaintext: &[u8]) -> Result<Vec<u8>> { pub fn seal(&self, seq: u64, plaintext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.send_salt, seq); let nonce = nonce(self.send_salt, seq);
self.cipher let aad = seq.to_be_bytes();
.encrypt( let payload = Payload {
Nonce::from_slice(&nonce),
Payload {
msg: plaintext, msg: plaintext,
aad: &seq.to_be_bytes(), aad: &aad,
}, };
) match &self.cipher {
Cipher::Aes128Gcm(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
Cipher::ChaCha20Poly1305(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
}
.map_err(|_| PunktfunkError::Crypto) .map_err(|_| PunktfunkError::Crypto)
} }
@@ -69,9 +157,15 @@ impl SessionCrypto {
let nonce = nonce(self.send_salt, seq); let nonce = nonce(self.send_salt, seq);
let split = buf.len() - TAG_LEN; let split = buf.len() - TAG_LEN;
let (plaintext, tag_slot) = buf.split_at_mut(split); let (plaintext, tag_slot) = buf.split_at_mut(split);
let tag = self let aad = seq.to_be_bytes();
.cipher let tag = match &self.cipher {
.encrypt_in_place_detached(Nonce::from_slice(&nonce), &seq.to_be_bytes(), plaintext) Cipher::Aes128Gcm(c) => {
c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
Cipher::ChaCha20Poly1305(c) => {
c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
}
.map_err(|_| PunktfunkError::Crypto)?; .map_err(|_| PunktfunkError::Crypto)?;
tag_slot.copy_from_slice(&tag); tag_slot.copy_from_slice(&tag);
Ok(()) Ok(())
@@ -80,20 +174,21 @@ impl SessionCrypto {
/// Open `ciphertext || tag` for sequence `seq` (also bound as associated data). /// Open `ciphertext || tag` for sequence `seq` (also bound as associated data).
pub fn open(&self, seq: u64, ciphertext: &[u8]) -> Result<Vec<u8>> { pub fn open(&self, seq: u64, ciphertext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.recv_salt, seq); let nonce = nonce(self.recv_salt, seq);
self.cipher let aad = seq.to_be_bytes();
.decrypt( let payload = Payload {
Nonce::from_slice(&nonce),
Payload {
msg: ciphertext, msg: ciphertext,
aad: &seq.to_be_bytes(), aad: &aad,
}, };
) match &self.cipher {
Cipher::Aes128Gcm(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
Cipher::ChaCha20Poly1305(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
}
.map_err(|_| PunktfunkError::Crypto) .map_err(|_| PunktfunkError::Crypto)
} }
/// Open in place, no per-packet allocation: `buf` holds `[ciphertext .. ][tag]` on entry and /// Open in place, no per-packet allocation: `buf` holds `[ciphertext .. ][tag]` on entry and
/// the plaintext in its first `buf.len() - TAG_LEN` bytes on success (returned as the length) /// the plaintext in its first `buf.len() - TAG_LEN` bytes on success (returned as the length)
/// — byte-identical to `open`, just written in place. GCM verifies the tag *before* /// — byte-identical to `open`, just written in place. Both AEADs verify the tag *before*
/// decrypting, so on failure `buf` still holds the ciphertext (the caller drops the packet /// decrypting, so on failure `buf` still holds the ciphertext (the caller drops the packet
/// either way). The hot-path receiver (`Session::poll_frame`) uses this to avoid the `Vec` /// either way). The hot-path receiver (`Session::poll_frame`) uses this to avoid the `Vec`
/// that `open`'s convenience API allocates for every datagram at line rate — the receive /// that `open`'s convenience API allocates for every datagram at line rate — the receive
@@ -105,13 +200,21 @@ impl SessionCrypto {
let nonce = nonce(self.recv_salt, seq); let nonce = nonce(self.recv_salt, seq);
let split = buf.len() - TAG_LEN; let split = buf.len() - TAG_LEN;
let (ciphertext, tag) = buf.split_at_mut(split); let (ciphertext, tag) = buf.split_at_mut(split);
self.cipher let aad = seq.to_be_bytes();
.decrypt_in_place_detached( match &self.cipher {
Cipher::Aes128Gcm(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce), Nonce::from_slice(&nonce),
&seq.to_be_bytes(), &aad,
ciphertext, ciphertext,
aes_gcm::Tag::from_slice(tag), aes_gcm::Tag::from_slice(tag),
) ),
Cipher::ChaCha20Poly1305(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce),
&aad,
ciphertext,
chacha20poly1305::Tag::from_slice(tag),
),
}
.map_err(|_| PunktfunkError::Crypto)?; .map_err(|_| PunktfunkError::Crypto)?;
Ok(split) Ok(split)
} }
@@ -145,6 +248,13 @@ pub fn random_key() -> [u8; 16] {
k k
} }
/// Generate a fresh random ChaCha20-Poly1305 session key (RFC 8439's 256-bit size).
pub fn random_key32() -> [u8; 32] {
let mut k = [0u8; 32];
rand::RngCore::fill_bytes(&mut rand::rng(), &mut k);
k
}
/// Generate a fresh random per-session nonce salt. /// Generate a fresh random per-session nonce salt.
pub fn random_salt() -> [u8; 4] { pub fn random_salt() -> [u8; 4] {
let mut s = [0u8; 4]; let mut s = [0u8; 4];
@@ -156,9 +266,17 @@ pub fn random_salt() -> [u8; 4] {
mod tests { mod tests {
use super::*; use super::*;
/// One fresh key per negotiated cipher — every sealing test below must hold for both.
fn both_keys() -> [SessionKey; 2] {
[
SessionKey::Aes128Gcm(random_key()),
SessionKey::ChaCha20Poly1305(random_key32()),
]
}
#[test] #[test]
fn seal_open_roundtrip_cross_direction() { fn seal_open_roundtrip_cross_direction() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -175,10 +293,11 @@ mod tests {
// open its own outbound packet → distinct nonce spaces per direction. // open its own outbound packet → distinct nonce spaces per direction.
assert!(host.open(42, &sealed).is_err()); assert!(host.open(42, &sealed).is_err());
} }
}
#[test] #[test]
fn directions_use_distinct_nonce_spaces() { fn directions_use_distinct_nonce_spaces() {
let key = random_key(); for key in both_keys() {
let salt = [0u8; 4]; // even an all-zero base salt must separate the directions let salt = [0u8; 4]; // even an all-zero base salt must separate the directions
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -188,10 +307,11 @@ mod tests {
client.seal(0, b"abc").unwrap() client.seal(0, b"abc").unwrap()
); );
} }
}
#[test] #[test]
fn open_in_place_matches_open_and_rejects_tampering() { fn open_in_place_matches_open_and_rejects_tampering() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -221,10 +341,11 @@ mod tests {
let mut runt = vec![0u8; TAG_LEN - 1]; let mut runt = vec![0u8; TAG_LEN - 1];
assert!(client.open_in_place(0, &mut runt).is_err()); assert!(client.open_in_place(0, &mut runt).is_err());
} }
}
#[test] #[test]
fn seal_in_place_matches_seal_and_opens() { fn seal_in_place_matches_seal_and_opens() {
let key = random_key(); for key in both_keys() {
let salt = random_salt(); let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host); let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client); let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -245,4 +366,43 @@ mod tests {
assert_eq!(client.open(7, &buf).unwrap(), msg); assert_eq!(client.open(7, &buf).unwrap(), msg);
} }
} }
}
#[test]
fn ciphers_are_not_interchangeable() {
// A packet sealed under one AEAD must not open under the other — negotiation skew has
// to fail loudly (a tag mismatch), never decode garbage. The ChaCha key repeats the AES
// key bytes so even overlapping key material can't accidentally interoperate.
let salt = random_salt();
let aes = SessionKey::Aes128Gcm([7u8; 16]);
let chacha = SessionKey::ChaCha20Poly1305([7u8; 32]);
let sealed = SessionCrypto::new(&aes, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&chacha, salt, Role::Client)
.open(1, &sealed)
.is_err());
let sealed = SessionCrypto::new(&chacha, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&aes, salt, Role::Client)
.open(1, &sealed)
.is_err());
}
#[test]
fn session_key_zero_check_and_debug_redaction() {
assert!(SessionKey::Aes128Gcm([0u8; 16]).is_zero());
assert!(SessionKey::ChaCha20Poly1305([0u8; 32]).is_zero());
assert!(!SessionKey::Aes128Gcm([1u8; 16]).is_zero());
assert!(!SessionKey::ChaCha20Poly1305([1u8; 32]).is_zero());
// Key bytes must never reach a log, whichever variant — only the cipher choice.
for key in both_keys() {
let dbg = format!("{key:?}");
assert!(dbg.contains("<redacted>"), "{dbg}");
}
let mut k = SessionKey::ChaCha20Poly1305([9u8; 32]);
k.zeroize();
assert!(k.is_zero());
}
} }
+3 -2
View File
@@ -1,6 +1,7 @@
use super::reassemble::LOSS_WINDOW_NS; use super::reassemble::LOSS_WINDOW_NS;
use super::*; use super::*;
use crate::config::{Config, FecScheme}; use crate::config::{Config, FecScheme};
use crate::crypto::SessionKey;
use crate::fec::coder_for; use crate::fec::coder_for;
use crate::stats::StatsCounters; use crate::stats::StatsCounters;
use zerocopy::{FromBytes, IntoBytes}; use zerocopy::{FromBytes, IntoBytes};
@@ -182,7 +183,7 @@ fn explicit_frame_index_is_stamped_and_internal_counter_untouched() {
shard_payload: 16, shard_payload: 16,
max_frame_bytes: 4096, max_frame_bytes: 4096,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
}; };
@@ -292,7 +293,7 @@ fn e2e_config(scheme: FecScheme, fec_percent: u8) -> Config {
shard_payload: 16, shard_payload: 16,
max_frame_bytes: 4096, max_frame_bytes: 4096,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
+31
View File
@@ -44,6 +44,17 @@ pub const VIDEO_CAP_PROBE_SEQ: u8 = 0x10;
/// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so /// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so
/// the fallback is zero-risk. /// the fallback is zero-risk.
pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20; pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20;
/// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
/// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
/// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
/// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
/// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
/// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
/// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
/// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
/// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
/// control channel, so there is no downgrade surface.
pub const VIDEO_CAP_CHACHA20: u8 = 0x40;
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] /// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder /// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -60,6 +71,24 @@ pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// trailing `host_caps` byte — no wire-layout change. /// trailing `host_caps` byte — no wire-layout change.
pub const HOST_CAP_CLIPBOARD: u8 = 0x02; pub const HOST_CAP_CLIPBOARD: u8 = 0x02;
/// [`Hello::client_caps`] bit: the client renders the host cursor LOCALLY
/// (design/remote-desktop-sweep.md M2). It consumes [`CursorShape`](super::control::CursorShape)
/// control messages (RGBA bitmap + hotspot, cached by serial) and per-frame
/// [`CursorState`](super::datagram::CursorState) `0xD0` datagrams (position/visibility), and
/// draws the pointer itself — so the host must STOP compositing the cursor into the video
/// (`SessionPlan.cursor_blend = false`) or the user sees it twice. Active only when the host
/// answers with [`HOST_CAP_CURSOR`] (capable-and-agreed, the 444/clipboard precedent); toward
/// an older or incapable host nothing changes.
pub const CLIENT_CAP_CURSOR: u8 = 0x01;
/// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
/// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
/// whose capture carries no cursor, and NOT Windows yet, where DWM composites into the IDD
/// frame). Set only when the client asked via [`CLIENT_CAP_CURSOR`]; when both bits agree the
/// host stops blending and ships [`CursorShape`](super::control::CursorShape) +
/// [`CursorState`](super::datagram::CursorState) instead.
pub const HOST_CAP_CURSOR: u8 = 0x04;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** /// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST /// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this. /// advertise this.
@@ -225,6 +254,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_HEVC, codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD, host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD,
cipher: 0,
key_chacha: None,
}; };
let got = Welcome::decode(&w.encode()).unwrap(); let got = Welcome::decode(&w.encode()).unwrap();
assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD); assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD);
+115
View File
@@ -783,6 +783,83 @@ impl ClipFetchHdr {
} }
} }
// --- Cursor channel (design/remote-desktop-sweep.md M2) --------------------------------------
// The host cursor, forwarded out-of-band so the CLIENT draws it as a real OS cursor (the
// Parsec/RDP model) instead of paying the video round-trip. Shape (rare, needs reliability)
// rides here on the control stream; per-frame position/visibility rides the lossy `0xD0`
// datagram plane ([`super::datagram::CursorState`]). Active only when the client's
// [`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) met the host's
// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR) — the host stops compositing then.
// ---------------------------------------------------------------------------------------------
/// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
pub const MSG_CURSOR_SHAPE: u8 = 0x50;
/// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
/// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
/// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
/// real cursors (typically ≤ 64 px, ≤ 96 px at HiDPI scale); the HOST downscales anything
/// larger before forwarding, so the cap is invisible to clients.
pub const CURSOR_SHAPE_MAX_SIDE: u16 = 120;
/// `host → client` ([`MSG_CURSOR_SHAPE`]): one cursor shape, sent when the pointer's bitmap
/// changes (never per-frame — [`super::datagram::CursorState`] carries the motion). The client
/// caches shapes by `serial` and re-installs a cached one without any bitmap crossing again
/// (the RDP pointer-cache idea for free: re-showing a known serial is a 14-byte
/// [`super::datagram::CursorState`], not a resend).
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct CursorShape {
/// Bitmap identity — bumped by the host's capture layer only on shape change; position
/// moves keep the serial stable. [`super::datagram::CursorState::serial`] references it.
pub serial: u32,
/// Bitmap dimensions in pixels, `1..=`[`CURSOR_SHAPE_MAX_SIDE`] each.
pub w: u16,
pub h: u16,
/// Hotspot (the pixel that IS the pointer position), within `w`×`h`.
pub hot_x: u16,
pub hot_y: u16,
/// Straight-alpha RGBA8, exactly `w * h * 4` bytes, no padding.
pub rgba: Vec<u8>,
}
impl CursorShape {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] serial[5..9] w[9..11] h[11..13] hot_x[13..15] hot_y[15..17] rgba…
let mut b = Vec::with_capacity(17 + self.rgba.len());
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CURSOR_SHAPE);
b.extend_from_slice(&self.serial.to_le_bytes());
b.extend_from_slice(&self.w.to_le_bytes());
b.extend_from_slice(&self.h.to_le_bytes());
b.extend_from_slice(&self.hot_x.to_le_bytes());
b.extend_from_slice(&self.hot_y.to_le_bytes());
b.extend_from_slice(&self.rgba);
b
}
pub fn decode(b: &[u8]) -> Result<CursorShape> {
if b.len() < 17 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CURSOR_SHAPE {
return Err(PunktfunkError::InvalidArg("bad CursorShape"));
}
let u16at = |o: usize| u16::from_le_bytes([b[o], b[o + 1]]);
let (w, h) = (u16at(9), u16at(11));
if w == 0 || h == 0 || w > CURSOR_SHAPE_MAX_SIDE || h > CURSOR_SHAPE_MAX_SIDE {
return Err(PunktfunkError::InvalidArg("bad CursorShape dims"));
}
if b.len() != 17 + (w as usize) * (h as usize) * 4 {
return Err(PunktfunkError::InvalidArg("bad CursorShape len"));
}
Ok(CursorShape {
serial: u32::from_le_bytes(b[5..9].try_into().unwrap()),
w,
h,
hot_x: u16at(13),
hot_y: u16at(15),
rgba: b[17..].to_vec(),
})
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use crate::config::Mode; use crate::config::Mode;
@@ -1147,4 +1224,42 @@ mod tests {
assert!(ClipFetchHdr::decode(&[bytes.as_slice(), &[0]].concat()).is_err()); assert!(ClipFetchHdr::decode(&[bytes.as_slice(), &[0]].concat()).is_err());
assert!(ClipFetchHdr::decode(&bytes[..bytes.len() - 1]).is_err()); assert!(ClipFetchHdr::decode(&bytes[..bytes.len() - 1]).is_err());
} }
#[test]
fn cursor_shape_roundtrip() {
let s = CursorShape {
serial: 7,
w: 2,
h: 3,
hot_x: 1,
hot_y: 2,
rgba: (0..2 * 3 * 4).map(|i| i as u8).collect(),
};
assert_eq!(CursorShape::decode(&s.encode()).unwrap(), s);
// Max-side shape still fits the u16 control frame with headroom.
let side = CURSOR_SHAPE_MAX_SIDE;
let big = CursorShape {
serial: u32::MAX,
w: side,
h: side,
hot_x: side - 1,
hot_y: 0,
rgba: vec![0xAB; side as usize * side as usize * 4],
};
let bytes = big.encode();
assert!(bytes.len() <= u16::MAX as usize, "must fit a control frame");
assert_eq!(CursorShape::decode(&bytes).unwrap(), big);
// Rejections: zero / oversize dims, and a length that disagrees with them.
let mut zero = s.encode();
zero[9] = 0;
zero[10] = 0;
assert!(CursorShape::decode(&zero).is_err());
let mut oversize = s.encode();
oversize[9..11].copy_from_slice(&(CURSOR_SHAPE_MAX_SIDE + 1).to_le_bytes());
assert!(CursorShape::decode(&oversize).is_err());
let mut short = s.encode();
short.pop();
assert!(CursorShape::decode(&short).is_err());
// Distinct from the neighboring vocabulary.
assert!(ClipState::decode(&s.encode()).is_err());
}
} }
@@ -606,6 +606,75 @@ pub fn decode_host_timing_datagram(b: &[u8]) -> Option<HostTiming> {
}) })
} }
/// Cursor-state datagram tag, host → client (design/remote-desktop-sweep.md M2). Next tag after
/// [`HOST_TIMING_MAGIC`]. Sent once per captured frame while the cursor channel is negotiated
/// ([`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) ∧
/// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR)) — per-frame resend makes the plane
/// self-healing under loss (latest-wins, no refresh timer). The bitmap itself rides the
/// reliable control stream ([`CursorShape`](super::control::CursorShape)); this 14-byte
/// datagram only moves/hides the pointer.
pub const CURSOR_STATE_MAGIC: u8 = 0xD0;
/// [`CursorState::flags`] bit: the host cursor is visible.
pub const CURSOR_VISIBLE: u8 = 0x01;
/// [`CursorState::flags`] bit: a host app captured/hid the pointer — the client SHOULD run
/// relative/captured (M3 auto-flip; advisory, user override always wins).
pub const CURSOR_RELATIVE_HINT: u8 = 0x02;
/// Per-frame host-cursor state (position, visibility, mode hint). `x`/`y` are the pointer
/// position (hotspot point, not bitmap top-left) in the host OUTPUT's pixel space — the same
/// space the video mode describes, so the client maps through its letterbox exactly like it
/// maps touches, in reverse.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct CursorState {
/// The [`CursorShape`](super::control::CursorShape) serial this state refers to. A client
/// that has no cached shape for it keeps its previous cursor until the (reliable) shape
/// message lands — at worst one control-stream RTT of stale shape, never a wrong position.
pub serial: u32,
/// Bitfield of [`CURSOR_VISIBLE`] / [`CURSOR_RELATIVE_HINT`].
pub flags: u8,
pub x: i32,
pub y: i32,
}
impl CursorState {
pub fn visible(&self) -> bool {
self.flags & CURSOR_VISIBLE != 0
}
pub fn relative_hint(&self) -> bool {
self.flags & CURSOR_RELATIVE_HINT != 0
}
}
/// Wire length of a [`CURSOR_STATE_MAGIC`] datagram: tag + u32 serial + flags + 2 × i32 = 14.
const CURSOR_STATE_LEN: usize = 1 + 4 + 1 + 8;
/// Encode a [`CursorState`] into a [`CURSOR_STATE_MAGIC`] datagram.
pub fn encode_cursor_state_datagram(s: &CursorState) -> Vec<u8> {
let mut b = Vec::with_capacity(CURSOR_STATE_LEN);
b.push(CURSOR_STATE_MAGIC);
b.extend_from_slice(&s.serial.to_le_bytes());
b.push(s.flags);
b.extend_from_slice(&s.x.to_le_bytes());
b.extend_from_slice(&s.y.to_le_bytes());
b
}
/// Parse a [`CURSOR_STATE_MAGIC`] datagram → [`CursorState`]. `None` on bad tag or a short
/// buffer (the fixed length bounds every read before it happens; a longer buffer is tolerated
/// for append-extension, like 0xCF).
pub fn decode_cursor_state_datagram(b: &[u8]) -> Option<CursorState> {
if b.len() < CURSOR_STATE_LEN || b[0] != CURSOR_STATE_MAGIC {
return None;
}
Some(CursorState {
serial: u32::from_le_bytes(b[1..5].try_into().unwrap()),
flags: b[5],
x: i32::from_le_bytes(b[6..10].try_into().unwrap()),
y: i32::from_le_bytes(b[10..14].try_into().unwrap()),
})
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use crate::quic::*; use crate::quic::*;
@@ -922,4 +991,32 @@ mod tests {
) )
.is_none()); .is_none());
} }
#[test]
fn cursor_state_roundtrip() {
for (flags, x, y) in [
(CURSOR_VISIBLE, 0i32, 0i32),
(CURSOR_VISIBLE | CURSOR_RELATIVE_HINT, -5, 2160),
(0, i32::MIN, i32::MAX),
] {
let s = CursorState {
serial: 42,
flags,
x,
y,
};
let d = encode_cursor_state_datagram(&s);
assert_eq!(decode_cursor_state_datagram(&d), Some(s));
assert_eq!(s.visible(), flags & CURSOR_VISIBLE != 0);
assert_eq!(s.relative_hint(), flags & CURSOR_RELATIVE_HINT != 0);
// Append-extensible like 0xCF: a longer buffer still parses the known prefix.
let mut ext = d.clone();
ext.push(0xFF);
assert_eq!(decode_cursor_state_datagram(&ext), Some(s));
// Short / wrong tag are rejected before any read.
assert_eq!(decode_cursor_state_datagram(&d[..d.len() - 1]), None);
let mut bad = d.clone();
bad[0] = HOST_TIMING_MAGIC;
assert_eq!(decode_cursor_state_datagram(&bad), None);
}
}
} }
+268 -13
View File
@@ -4,6 +4,7 @@ use super::*;
use crate::config::{ use crate::config::{
CompositorPref, Config, FecConfig, FecScheme, GamepadPref, Mode, ProtocolPhase, Role, CompositorPref, Config, FecConfig, FecScheme, GamepadPref, Mode, ProtocolPhase, Role,
}; };
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
/// `client → host`: open the session, requesting a display mode (the host creates its /// `client → host`: open the session, requesting a display mode (the host creates its
@@ -82,6 +83,15 @@ pub struct Hello {
/// forcing the earlier placeholders. Omitted by older clients / when the client has no HDR /// forcing the earlier placeholders. Omitted by older clients / when the client has no HDR
/// display (decodes to `None` — the host keeps its built-in EDID defaults). /// display (decodes to `None` — the host keeps its built-in EDID defaults).
pub display_hdr: Option<HdrMeta>, pub display_hdr: Option<HdrMeta>,
/// Non-video client capabilities — a bitfield of [`CLIENT_CAP_CURSOR`] (the client renders
/// the host cursor locally; the host stops compositing it and forwards shape + state
/// instead). Appended as a single byte AFTER `display_hdr`; because that block is a fixed
/// [`super::datagram::HDR_META_BODY_LEN`]-byte optional with no placeholder form, presence is
/// disambiguated by REMAINING LENGTH at decode: fewer than `HDR_META_BODY_LEN` bytes after
/// `preferred_codec` ⇒ no HDR block, the tail bytes are the post-HDR fields directly. This
/// caps everything after `display_hdr` at `HDR_META_BODY_LEN 1` bytes total — document any
/// future field here and mind the budget. Omitted when zero and by older clients (→ `0`).
pub client_caps: u8,
} }
/// QUIC application error code a punktfunk/1 client closes the control connection with on a /// QUIC application error code a punktfunk/1 client closes the control connection with on a
@@ -107,6 +117,13 @@ pub const HELLO_NAME_MAX: usize = 64;
/// (`steam:<appid>` / `custom:<12 hex>`); the cap just bounds an attacker-controlled field. /// (`steam:<appid>` / `custom:<12 hex>`); the cap just bounds an attacker-controlled field.
pub const HELLO_LAUNCH_MAX: usize = 128; pub const HELLO_LAUNCH_MAX: usize = 128;
/// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
/// only one pre-cipher builds know).
pub const CIPHER_AES_128_GCM: u8 = 0;
/// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
/// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
pub const CIPHER_CHACHA20_POLY1305: u8 = 1;
/// `host → client`: the complete session offer. /// `host → client`: the complete session offer.
#[derive(Clone, Copy, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Welcome { pub struct Welcome {
@@ -173,6 +190,22 @@ pub struct Welcome {
/// per-transition events otherwise). Appended after `codec` as a single trailing byte; an /// per-transition events otherwise). Appended after `codec` as a single trailing byte; an
/// older host that omits it decodes to `0` (no capabilities — legacy events only). /// older host that omits it decodes to `0` (no capabilities — legacy events only).
pub host_caps: u8, pub host_caps: u8,
/// The session AEAD the data plane seals with — [`CIPHER_AES_128_GCM`] (`0`, the default
/// every peer speaks) or [`CIPHER_CHACHA20_POLY1305`] (`1`). The host sets `1` ONLY toward
/// a client that advertised [`VIDEO_CAP_CHACHA20`] (the soft-AES armv7 targets). Appended
/// after `host_caps` at offset 68 and — unlike the earlier trailing fields — emitted only
/// when non-zero, so an AES session's Welcome stays **byte-identical** to the pre-cipher
/// wire form; an older host omits it (→ `0`, AES). Decode is fail-closed: an unknown id is
/// an `Err`, never a silent AES fallback — the host only picks a cipher this client
/// advertised, so an unknown id reaching us is a bug, and falling back would yield an
/// undecryptable session with a confusing failure signature.
pub cipher: u8,
/// The 256-bit ChaCha20-Poly1305 session key (RFC 8439 requires the full 32 bytes; wire
/// cost is once per handshake) — present iff `cipher == 1`, at offsets 69..101. The legacy
/// 16-byte `key` keeps its offset and stays independently random, so nothing downstream
/// ever observes an all-zero key. Decode rejects `cipher == 1` with fewer than 32 key
/// bytes following.
pub key_chacha: Option<[u8; 32]>,
} }
/// `client → host`: data plane is bound, begin streaming. /// `client → host`: data plane is bound, begin streaming.
@@ -220,8 +253,13 @@ impl Hello {
let vcodecs_present = self.video_codecs != 0; let vcodecs_present = self.video_codecs != 0;
let pref_present = self.preferred_codec != 0; let pref_present = self.preferred_codec != 0;
let hdr_present = self.display_hdr.is_some(); let hdr_present = self.display_hdr.is_some();
let need_placeholders = let ccaps_present = self.client_caps != 0;
self.video_caps != 0 || ac_present || vcodecs_present || pref_present || hdr_present; let need_placeholders = self.video_caps != 0
|| ac_present
|| vcodecs_present
|| pref_present
|| hdr_present
|| ccaps_present;
match (&self.name, &self.launch) { match (&self.name, &self.launch) {
(None, None) if !need_placeholders => {} (None, None) if !need_placeholders => {}
(name, _) => { (name, _) => {
@@ -242,21 +280,27 @@ impl Hello {
b.push(self.video_caps); b.push(self.video_caps);
} }
// audio_channels: emitted when non-stereo OR a later field follows. // audio_channels: emitted when non-stereo OR a later field follows.
if ac_present || vcodecs_present || pref_present || hdr_present { if ac_present || vcodecs_present || pref_present || hdr_present || ccaps_present {
b.push(self.audio_channels); b.push(self.audio_channels);
} }
// video_codecs: emitted when non-zero OR a later field follows. // video_codecs: emitted when non-zero OR a later field follows.
if vcodecs_present || pref_present || hdr_present { if vcodecs_present || pref_present || hdr_present || ccaps_present {
b.push(self.video_codecs); b.push(self.video_codecs);
} }
// preferred_codec: emitted when non-zero OR display_hdr follows. // preferred_codec: emitted when non-zero OR a later field follows.
if pref_present || hdr_present { if pref_present || hdr_present || ccaps_present {
b.push(self.preferred_codec); b.push(self.preferred_codec);
} }
// display_hdr: fixed HDR_META_BODY_LEN-byte HdrMeta body. Last field; omitted when `None`. // display_hdr: fixed HDR_META_BODY_LEN-byte HdrMeta body; omitted when `None` even if
// later fields follow (no placeholder form — the decoder disambiguates by remaining
// length, which caps the post-HDR tail at HDR_META_BODY_LEN 1 bytes).
if let Some(m) = &self.display_hdr { if let Some(m) = &self.display_hdr {
super::datagram::write_hdr_meta_body(m, &mut b); super::datagram::write_hdr_meta_body(m, &mut b);
} }
// client_caps: single byte after the (optional) HDR block. Emitted when non-zero.
if ccaps_present {
b.push(self.client_caps);
}
b b
} }
@@ -322,9 +366,26 @@ impl Hello {
preferred_codec: b.get(tail + 3).copied().unwrap_or(0), preferred_codec: b.get(tail + 3).copied().unwrap_or(0),
// Optional trailing HdrMeta body (fixed length) — absent on an older client / a // Optional trailing HdrMeta body (fixed length) — absent on an older client / a
// client without an HDR display → `None` (the host keeps its EDID defaults). // client without an HDR display → `None` (the host keeps its EDID defaults).
display_hdr: b // Presence is decided by REMAINING LENGTH (there is no placeholder form for the
.get(tail + 4..tail + 4 + super::datagram::HDR_META_BODY_LEN) // fixed block): ≥ HDR_META_BODY_LEN bytes after `preferred_codec` ⇒ the block is
.map(super::datagram::read_hdr_meta_body), // there and post-HDR fields follow it; fewer ⇒ no block, the bytes ARE the post-HDR
// fields. Sound as long as the post-HDR tail stays under HDR_META_BODY_LEN bytes.
display_hdr: (b.len().saturating_sub(tail + 4) >= super::datagram::HDR_META_BODY_LEN)
.then(|| {
b.get(tail + 4..tail + 4 + super::datagram::HDR_META_BODY_LEN)
.map(super::datagram::read_hdr_meta_body)
})
.flatten(),
// client_caps: the byte after the HDR block when present, else directly at tail+4.
client_caps: {
let off = if b.len().saturating_sub(tail + 4) >= super::datagram::HDR_META_BODY_LEN
{
tail + 4 + super::datagram::HDR_META_BODY_LEN
} else {
tail + 4
};
b.get(off).copied().unwrap_or(0)
},
}) })
} }
} }
@@ -366,6 +427,21 @@ impl Welcome {
b.push(self.codec); b.push(self.codec);
// Host input caps at offset 67 — older clients stop before this → 0 (legacy input only). // Host input caps at offset 67 — older clients stop before this → 0 (legacy input only).
b.push(self.host_caps); b.push(self.host_caps);
// Session cipher at offset 68 + the 32-byte ChaCha key at 69..101 — emitted ONLY when a
// non-default cipher was negotiated, so an AES session's Welcome stays byte-identical
// to the pre-cipher wire form. The host only sets cipher toward a client that
// advertised VIDEO_CAP_CHACHA20, so an old client never sees these bytes at all.
debug_assert_eq!(
self.cipher == CIPHER_CHACHA20_POLY1305,
self.key_chacha.is_some(),
"key_chacha present iff cipher == 1"
);
if self.cipher != CIPHER_AES_128_GCM {
b.push(self.cipher);
if let Some(k) = &self.key_chacha {
b.extend_from_slice(k);
}
}
b b
} }
@@ -374,8 +450,10 @@ impl Welcome {
// scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45] // scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45]
// salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59] // salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59]
// bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63] // bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63]
// chroma_format[64] audio_channels[65] codec[66] (everything from compositor on is an // chroma_format[64] audio_channels[65] codec[66] host_caps[67] cipher[68]
// optional trailing byte; an older host stops earlier). // key_chacha[69..101] (everything from compositor on is an optional trailing byte; an
// older host stops earlier; cipher/key_chacha are present only when ChaCha was
// negotiated).
if b.len() < 53 || &b[0..4] != MAGIC { if b.len() < 53 || &b[0..4] != MAGIC {
return Err(PunktfunkError::InvalidArg("bad Welcome")); return Err(PunktfunkError::InvalidArg("bad Welcome"));
} }
@@ -385,6 +463,24 @@ impl Welcome {
key.copy_from_slice(&b[29..45]); key.copy_from_slice(&b[29..45]);
let mut salt = [0u8; 4]; let mut salt = [0u8; 4];
salt.copy_from_slice(&b[45..49]); salt.copy_from_slice(&b[45..49]);
// Session cipher at 68 — absent on an older host → AES-128-GCM. Fail-closed on
// anything else: `cipher == 1` with fewer than 32 key bytes must be an error (a silent
// AES fallback would yield an undecryptable session with a confusing failure
// signature), and an unknown id (≥ 2) reaching us is a bug — a host only picks a
// cipher this client advertised — never a legitimate negotiation.
let cipher = b.get(68).copied().unwrap_or(CIPHER_AES_128_GCM);
let key_chacha = match cipher {
CIPHER_AES_128_GCM => None,
CIPHER_CHACHA20_POLY1305 => {
let bytes = b
.get(69..101)
.ok_or(PunktfunkError::InvalidArg("bad Welcome"))?;
let mut k = [0u8; 32];
k.copy_from_slice(bytes);
Some(k)
}
_ => return Err(PunktfunkError::InvalidArg("bad Welcome")),
};
Ok(Welcome { Ok(Welcome {
abi_version: u32at(4), abi_version: u32at(4),
udp_port: u16at(8), udp_port: u16at(8),
@@ -452,6 +548,8 @@ impl Welcome {
// Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state // Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state
// snapshots; the client keeps sending legacy per-transition events). // snapshots; the client keeps sending legacy per-transition events).
host_caps: b.get(67).copied().unwrap_or(0), host_caps: b.get(67).copied().unwrap_or(0),
cipher,
key_chacha,
}) })
} }
@@ -462,7 +560,12 @@ impl Welcome {
c.fec = self.fec; c.fec = self.fec;
c.shard_payload = self.shard_payload as usize; c.shard_payload = self.shard_payload as usize;
c.encrypt = self.encrypt; c.encrypt = self.encrypt;
c.key = self.key; // The negotiated AEAD: the ChaCha key when cipher == 1 (guaranteed present by decode —
// the `(1, None)` shape is unreachable off the wire), the legacy AES key otherwise.
c.key = match (self.cipher, self.key_chacha) {
(CIPHER_CHACHA20_POLY1305, Some(k)) => SessionKey::ChaCha20Poly1305(k),
_ => SessionKey::Aes128Gcm(self.key),
};
c.salt = self.salt; c.salt = self.salt;
// Client-side reassembler ceiling: p1_defaults' 64 MiB hostile-header memory bound is // Client-side reassembler ceiling: p1_defaults' 64 MiB hostile-header memory bound is
// ~10x larger than any real access unit. Derive it from the negotiated rate instead: // ~10x larger than any real access unit. Derive it from the negotiated rate instead:
@@ -531,6 +634,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, // exercise a non-default codec through the roundtrip codec: CODEC_H264, // exercise a non-default codec through the roundtrip
host_caps: HOST_CAP_GAMEPAD_STATE, host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
}; };
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w); assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
@@ -564,6 +669,81 @@ mod tests {
assert!(derived > (8 << 20) && derived < (64 << 20)); assert!(derived > (8 << 20) && derived < (64 << 20));
} }
#[test]
fn welcome_cipher_negotiation_wire_and_back_compat() {
use crate::crypto::SessionKey;
let base = Welcome {
abi_version: 2,
udp_port: 7000,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 20,
max_data_per_block: 4096,
},
shard_payload: 1200,
encrypt: true,
key: [7u8; 16],
salt: [9, 8, 7, 6],
frames: 0,
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 50_000,
bit_depth: 8,
color: ColorInfo::SDR_BT709,
chroma_format: CHROMA_IDC_420,
audio_channels: 2,
codec: CODEC_HEVC,
host_caps: 0,
cipher: CIPHER_AES_128_GCM,
key_chacha: None,
};
// An AES session's Welcome is byte-identical to the pre-cipher wire form (68 bytes) —
// the old-client × new-host interop guarantee.
let enc = base.encode();
assert_eq!(enc.len(), 68);
assert_eq!(Welcome::decode(&enc).unwrap(), base);
// ChaCha roundtrip: cipher byte at 68, the 32-byte key at 69..101.
let k32: [u8; 32] = core::array::from_fn(|i| i as u8 + 1);
let cha = Welcome {
cipher: CIPHER_CHACHA20_POLY1305,
key_chacha: Some(k32),
..base
};
let cenc = cha.encode();
assert_eq!(cenc.len(), 68 + 1 + 32);
assert_eq!(Welcome::decode(&cenc).unwrap(), cha);
// A truncated old-host Welcome (no cipher byte) decodes to the AES default.
let old_host = Welcome::decode(&cenc[..68]).unwrap();
assert_eq!(old_host.cipher, CIPHER_AES_128_GCM);
assert_eq!(old_host.key_chacha, None);
// cipher == 1 with a missing / short key → Err, fail-closed (a silent AES fallback
// would yield an undecryptable session with a confusing failure signature).
assert!(Welcome::decode(&cenc[..69]).is_err());
assert!(Welcome::decode(&cenc[..100]).is_err());
// An unknown cipher id (≥ 2) → Err: the host only picks a cipher we advertised, so an
// unknown id reaching us is a bug, never a legitimate negotiation.
let mut bad = cenc.clone();
bad[68] = 2;
assert!(Welcome::decode(&bad).is_err());
// session_config maps both variants onto the data-plane key, and both validate.
let aes_cfg = base.session_config(Role::Client);
assert_eq!(aes_cfg.key, SessionKey::Aes128Gcm([7u8; 16]));
aes_cfg.validate().expect("AES config validates");
let cha_cfg = cha.session_config(Role::Client);
assert_eq!(cha_cfg.key, SessionKey::ChaCha20Poly1305(k32));
cha_cfg.validate().expect("ChaCha config validates");
}
#[test] #[test]
fn codec_negotiation_and_back_compat() { fn codec_negotiation_and_back_compat() {
// resolve_codec precedence (HEVC > AV1 > H.264), no preference (0). // resolve_codec precedence (HEVC > AV1 > H.264), no preference (0).
@@ -656,6 +836,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_PYROWAVE, codec: CODEC_PYROWAVE,
host_caps: 0, host_caps: 0,
cipher: 0,
key_chacha: None,
} }
.encode(), .encode(),
) )
@@ -684,6 +866,7 @@ mod tests {
video_codecs: CODEC_H264 | CODEC_HEVC, video_codecs: CODEC_H264 | CODEC_HEVC,
preferred_codec: CODEC_H264, preferred_codec: CODEC_H264,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
let enc = h.encode(); let enc = h.encode();
let dec = Hello::decode(&enc).unwrap(); let dec = Hello::decode(&enc).unwrap();
@@ -726,6 +909,8 @@ mod tests {
audio_channels: 2, audio_channels: 2,
codec: CODEC_H264, codec: CODEC_H264,
host_caps: 0, host_caps: 0,
cipher: 0,
key_chacha: None,
} }
.encode(), .encode(),
) )
@@ -758,6 +943,7 @@ mod tests {
video_codecs: CODEC_H264 | CODEC_HEVC, // exercise the codec bitfield roundtrip video_codecs: CODEC_H264 | CODEC_HEVC, // exercise the codec bitfield roundtrip
preferred_codec: CODEC_HEVC, preferred_codec: CODEC_HEVC,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
assert_eq!(Hello::decode(&h.encode()).unwrap(), h); assert_eq!(Hello::decode(&h.encode()).unwrap(), h);
let s = Start { let s = Start {
@@ -788,6 +974,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
let enc = h.encode(); let enc = h.encode();
assert_eq!(enc.len(), 26); assert_eq!(enc.len(), 26);
@@ -831,6 +1018,8 @@ mod tests {
audio_channels: 6, // 5.1 — exercises the non-default trailing byte audio_channels: 6, // 5.1 — exercises the non-default trailing byte
codec: CODEC_HEVC, codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE, host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
}; };
let wenc = w.encode(); let wenc = w.encode();
assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps
@@ -903,6 +1092,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
let enc = base.encode(); let enc = base.encode();
assert_eq!( assert_eq!(
@@ -954,6 +1144,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
// launch alone (no name): a zero-length name placeholder keeps the offset deterministic. // launch alone (no name): a zero-length name placeholder keeps the offset deterministic.
let with_launch = Hello { let with_launch = Hello {
@@ -1013,6 +1204,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
}; };
// A real client-panel volume (P3 primaries, 800-nit peak, 0.05-nit floor, 400-nit FALL). // A real client-panel volume (P3 primaries, 800-nit peak, 0.05-nit floor, 400-nit FALL).
let vol = HdrMeta { let vol = HdrMeta {
@@ -1080,6 +1272,7 @@ mod tests {
video_codecs: 0, video_codecs: 0,
preferred_codec: 0, preferred_codec: 0,
display_hdr: None, display_hdr: None,
client_caps: 0,
} }
.encode(); .encode();
assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair"); assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair");
@@ -1093,4 +1286,66 @@ mod tests {
.encode(); .encode();
assert!(Hello::decode(&pr).is_err()); assert!(Hello::decode(&pr).is_err());
} }
#[test]
fn hello_client_caps_roundtrip_and_back_compat() {
let base = Hello {
abi_version: 2,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 0,
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
video_codecs: 0,
preferred_codec: 0,
display_hdr: None,
client_caps: 0,
};
let vol = HdrMeta {
display_primaries: [[13250, 34500], [7500, 3000], [34000, 16000]],
white_point: [15635, 16450],
max_display_mastering_luminance: 8_000_000,
min_display_mastering_luminance: 500,
max_cll: 0,
max_fall: 400,
};
// caps WITHOUT an HDR block: the single byte after preferred_codec (remaining < the
// fixed block length, so the decoder must NOT read it as a truncated HdrMeta).
let caps_only = Hello {
client_caps: CLIENT_CAP_CURSOR,
..base.clone()
};
assert_eq!(Hello::decode(&caps_only.encode()).unwrap(), caps_only);
// caps AND the HDR block: caps lands after the fixed block.
let both = Hello {
display_hdr: Some(vol),
client_caps: CLIENT_CAP_CURSOR,
..base.clone()
};
assert_eq!(Hello::decode(&both.encode()).unwrap(), both);
// HDR without caps stays byte-identical to the pre-caps wire form and decodes caps 0.
let hdr_only = Hello {
display_hdr: Some(vol),
..base.clone()
};
assert_eq!(Hello::decode(&hdr_only.encode()).unwrap(), hdr_only);
// An older client (no trailing byte at all) decodes to 0.
assert_eq!(Hello::decode(&base.encode()).unwrap().client_caps, 0);
// An older HOST reading a caps-bearing Hello: its decode simply never looks past the
// fields it knows — nothing before the caps byte moved.
let enc = both.encode();
assert_eq!(
Hello::decode(&enc[..enc.len() - 1]).unwrap(),
Hello {
client_caps: 0,
..both.clone()
}
);
}
} }
+3 -2
View File
@@ -781,6 +781,7 @@ impl Session {
mod wire_equivalence_tests { mod wire_equivalence_tests {
use super::*; use super::*;
use crate::config::{FecConfig, FecScheme, ProtocolPhase}; use crate::config::{FecConfig, FecScheme, ProtocolPhase};
use crate::crypto::SessionKey;
use crate::transport::loopback_pair; use crate::transport::loopback_pair;
fn host_cfg(scheme: FecScheme, fec_percent: u8, encrypt: bool) -> Config { fn host_cfg(scheme: FecScheme, fec_percent: u8, encrypt: bool) -> Config {
@@ -798,7 +799,7 @@ mod wire_equivalence_tests {
shard_payload: 64, shard_payload: 64,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt, encrypt,
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [3, 1, 4, 1], salt: [3, 1, 4, 1],
loopback_drop_period: 0, loopback_drop_period: 0,
} }
@@ -930,7 +931,7 @@ mod wire_equivalence_tests {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: 0, loopback_drop_period: 0,
}; };
+26 -1
View File
@@ -5,6 +5,7 @@
use proptest::prelude::*; use proptest::prelude::*;
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionKey;
use punktfunk_core::fec::coder_for; use punktfunk_core::fec::coder_for;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
@@ -25,7 +26,7 @@ fn config(role: Role, scheme: FecScheme, encrypt: bool, drop_period: u32) -> Con
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt, encrypt,
key: [7u8; 16], key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4], salt: [1, 2, 3, 4],
loopback_drop_period: drop_period, loopback_drop_period: drop_period,
} }
@@ -101,6 +102,30 @@ fn encrypted_stream_recovers_under_loss() {
assert_eq!(stats.frames_completed, frames.len() as u64); assert_eq!(stats.frames_completed, frames.len() as u64);
} }
/// The negotiated ChaCha20-Poly1305 session cipher through the same lossy full-stream path:
/// loss/replay behavior is cipher-independent (the replay window keys off the authenticated
/// seq), so recovery must be byte-identical to the AES run above.
#[test]
fn chacha20_encrypted_stream_recovers_under_loss() {
let frames = sample_frames();
let mk = |role| {
let mut c = config(role, FecScheme::Gf16, true, 8);
c.key = SessionKey::ChaCha20Poly1305([7u8; 32]);
c
};
let (host_tp, client_tp) = loopback_pair(8, 0);
let mut host = Session::new(mk(Role::Host), Box::new(host_tp)).unwrap();
let mut client = Session::new(mk(Role::Client), Box::new(client_tp)).unwrap();
for (i, frame) in frames.iter().enumerate() {
host.submit_frame(frame, i as u64 * 1_000_000, 0).unwrap();
let got = client
.poll_frame()
.expect("frame should recover despite loss");
assert_eq!(&got.data, frame, "frame {i} mismatched after recovery");
}
assert!(client.stats().fec_recovered_shards > 0);
}
#[test] #[test]
fn lossless_stream_is_exact() { fn lossless_stream_is_exact() {
let frames = sample_frames(); let frames = sample_frames();
@@ -86,6 +86,12 @@ pub fn start(
crate::events::emit(crate::events::EventKind::ClientConnected { crate::events::emit(crate::events::EventKind::ClientConnected {
client: event_client.clone(), client: event_client.clone(),
}); });
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock
// floor while this compat-plane stream runs, refcounted with every other live session
// across both planes. Released when the closure exits (stream stopped) — so idle clocks
// aren't pinned between Moonlight sessions. No-op off Linux / when the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
let result = run( let result = run(
cfg, cfg,
app.as_ref(), app.as_ref(),
+96 -20
View File
@@ -2,11 +2,15 @@
//! plan Tier 1B): the driver's adaptive P-state ramps clocks down between bursty encode frames, //! plan Tier 1B): the driver's adaptive P-state ramps clocks down between bursty encode frames,
//! so every frame re-pays a spin-up. This is NOT theoretical — measured on the 780M (VCN 4), //! so every frame re-pays a spin-up. This is NOT theoretical — measured on the 780M (VCN 4),
//! a 1440p HEVC encode takes ~4.4 ms/frame with the clocks hot (120 fps pacing) but ~8 ms/frame //! a 1440p HEVC encode takes ~4.4 ms/frame with the clocks hot (120 fps pacing) but ~8 ms/frame
//! at a 60 fps duty cycle: the sag doubles per-frame encode latency. //! at a 60 fps duty cycle: the sag doubles per-frame encode latency. So the pin is worth holding
//! only while a client is actively streaming: it is armed on the first live client and released
//! when the last one disconnects (refcounted across both streaming planes — see [`session_pin`]),
//! leaving the driver's idle power management alone the rest of the time.
//! //!
//! **AMD** (`PUNKTFUNK_PIN_CLOCKS=1`, root-gated by sysfs ownership): write `high` into each //! **AMD** (`PUNKTFUNK_PIN_CLOCKS=1`, root-gated by sysfs ownership): write `high` into each
//! amdgpu card's `power_dpm_force_performance_level` for the host lifetime, restoring the prior //! amdgpu card's `power_dpm_force_performance_level` while a client is streaming, restoring the
//! value on exit. Non-root gets EACCES → logged once with the privilege recipe. Deliberately //! prior value when the last client disconnects. Non-root gets EACCES → logged once with the
//! privilege recipe. Deliberately
//! opt-in: it defeats power management box-wide and is wrong on battery (Steam Deck!). //! opt-in: it defeats power management box-wide and is wrong on battery (Steam Deck!).
//! //!
//! **NVIDIA** — two independent halves, both no-ops off NVIDIA: //! **NVIDIA** — two independent halves, both no-ops off NVIDIA:
@@ -28,14 +32,16 @@
//! while leaving boost headroom — NVIDIA's own latency guidance is "raise the floor, don't pin //! while leaving boost headroom — NVIDIA's own latency guidance is "raise the floor, don't pin
//! the max" (locking above base just gets throttled; a max pin only burns idle watts). Non-root //! the max" (locking above base just gets throttled; a max pin only burns idle watts). Non-root
//! callers get `NVML_ERROR_NO_PERMISSION` — logged once with the privilege recipe, then the //! callers get `NVML_ERROR_NO_PERMISSION` — logged once with the privilege recipe, then the
//! host runs unpinned. The pin is undone on drop (host exit); after a crash it persists until //! host runs unpinned. The pin is undone on drop (when the last client disconnects); after a
//! driver reload/reboot, which the reset-before-pin on the next start self-heals. Deliberately //! crash it persists until driver reload/reboot, which the reset-before-pin on the next arm
//! self-heals. Deliberately
//! NOT default-on: it defeats idle downclocking for the whole box and is wrong on //! NOT default-on: it defeats idle downclocking for the whole box and is wrong on
//! battery-powered hosts. //! battery-powered hosts.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program). // Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use std::os::raw::{c_char, c_int, c_uint, c_void}; use std::os::raw::{c_char, c_int, c_uint, c_void};
use std::sync::{Mutex, OnceLock};
/// `nvmlDevice_t` — an opaque driver handle. /// `nvmlDevice_t` — an opaque driver handle.
type NvmlDevice = *mut c_void; type NvmlDevice = *mut c_void;
@@ -133,8 +139,9 @@ struct AmdPin {
restore: String, restore: String,
} }
/// Host-lifetime guard: holds the armed clock pins (NVML floor and/or amdgpu perf level) and /// Holds the armed clock pins (NVML floor and/or amdgpu perf level) and undoes them on drop. Owned
/// undoes them on drop. /// by the [`session_pin`] refcount: constructed when the first live client arms the pin, dropped
/// (clocks restored) when the last one disconnects.
pub struct ClockGuard { pub struct ClockGuard {
nvml: Option<NvmlPin>, nvml: Option<NvmlPin>,
amd: Vec<AmdPin>, amd: Vec<AmdPin>,
@@ -142,9 +149,10 @@ pub struct ClockGuard {
// SAFETY: `ClockGuard` holds opaque NVML device handles + resolved fn pointers from the loaded // SAFETY: `ClockGuard` holds opaque NVML device handles + resolved fn pointers from the loaded
// driver library (plus plain sysfs paths/strings). NVML is documented thread-safe, the handles are // driver library (plus plain sysfs paths/strings). NVML is documented thread-safe, the handles are
// plain driver tokens with no thread affinity, and the guard is only ever *moved* (held in `main`, // plain driver tokens with no thread affinity, and the guard is only ever *moved* (into the
// dropped once at exit) and used through `&mut`/ownership — never shared. Transfer across threads // `pin_refcount` mutex when armed, taken back out and dropped when the last client disconnects) and
// is therefore sound. // used through exclusive ownership behind that mutex — never shared. Transfer across threads is
// therefore sound.
unsafe impl Send for ClockGuard {} unsafe impl Send for ClockGuard {}
impl Drop for ClockGuard { impl Drop for ClockGuard {
@@ -182,22 +190,89 @@ impl Drop for ClockGuard {
} }
/// Startup hook for the host subcommands (`serve` / `punktfunk1-host`): install the NVIDIA P2-cap /// Startup hook for the host subcommands (`serve` / `punktfunk1-host`): install the NVIDIA P2-cap
/// application profile and, when `PUNKTFUNK_PIN_CLOCKS` is set, arm the vendor clock pin (NVML /// application profile — the process-scoped, no-root half of the NVIDIA lever, which the driver
/// core-clock floor / amdgpu `high` performance level). Returns the guard keeping the pins for /// only acts on once the host holds a live CUDA/NVENC context (i.e. during a session).
/// the host lifetime. `None` when nothing was armed. ///
pub fn on_host_start() -> Option<ClockGuard> { /// The vendor clock *pin* is deliberately NOT armed here anymore: held for the whole host lifetime
/// it kept the box's clocks hot even with no client connected. It is now refcounted per live client
/// via [`session_pin`] (armed on both streaming planes), so idle clocks are left to the driver's
/// power management until someone actually streams.
pub fn on_host_start() {
if nvidia_present() { if nvidia_present() {
ensure_cuda_perf_profile(); ensure_cuda_perf_profile();
} }
}
/// The box-wide clock-pin refcount, shared across BOTH streaming planes (native + GameStream): the
/// vendor pin is a single global GPU setting, so N concurrent sessions share ONE pin — armed when
/// the first client goes live, released when the last one leaves.
struct PinRefcount {
/// Number of live [`SessionClockPin`] handles.
live: usize,
/// The armed pins, present iff `live > 0` and something was actually pinnable.
guard: Option<ClockGuard>,
}
fn pin_refcount() -> &'static Mutex<PinRefcount> {
static STATE: OnceLock<Mutex<PinRefcount>> = OnceLock::new();
STATE.get_or_init(|| {
Mutex::new(PinRefcount {
live: 0,
guard: None,
})
})
}
/// RAII handle that keeps the box-wide clock pin armed while it is alive. Obtain one per live client
/// session on either plane via [`session_pin`]; when the last outstanding handle drops, the pin is
/// released and the driver's idle downclocking resumes. A no-op handle when `PUNKTFUNK_PIN_CLOCKS`
/// is unset (the opt-in gate) — the refcount only ticks for sessions that asked for pinning.
pub struct SessionClockPin {
/// Whether this handle actually incremented the refcount (false = opt-in gate off → no-op).
counted: bool,
}
/// Arm the box-wide clock pin for one live client session, refcounted across every plane. Returns
/// an RAII handle; the pin is released when the last handle drops. A no-op (returns immediately,
/// touches no GPU state) unless `PUNKTFUNK_PIN_CLOCKS` is set.
pub fn session_pin() -> SessionClockPin {
if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") { if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") {
return None; return SessionClockPin { counted: false };
} }
let mut state = pin_refcount().lock().unwrap();
state.live += 1;
if state.live == 1 {
// 0 → 1: the first live client — arm the vendor pin (reset-before-pin inside `pin_nvidia`
// heals a stale pin from a crashed previous run).
let nvml = if nvidia_present() { pin_nvidia() } else { None }; let nvml = if nvidia_present() { pin_nvidia() } else { None };
let amd = pin_amdgpu(); let amd = pin_amdgpu();
if nvml.is_none() && amd.is_empty() { state.guard = if nvml.is_none() && amd.is_empty() {
return None; None // nothing pinnable (no perms / no supported GPU) — the session streams unpinned
} } else {
Some(ClockGuard { nvml, amd }) Some(ClockGuard { nvml, amd })
};
}
SessionClockPin { counted: true }
}
impl Drop for SessionClockPin {
fn drop(&mut self) {
if !self.counted {
return;
}
// Take the guard out under the lock but drop it *outside* — releasing the pin does NVML +
// sysfs I/O we don't want to hold the refcount lock across.
let release = {
let mut state = pin_refcount().lock().unwrap();
state.live = state.live.saturating_sub(1);
if state.live == 0 {
state.guard.take()
} else {
None
}
};
drop(release);
}
} }
/// Force every amdgpu card's DPM performance level to `high` for the session — the encode-latency /// Force every amdgpu card's DPM performance level to `high` for the session — the encode-latency
@@ -238,7 +313,7 @@ fn pin_amdgpu() -> Vec<AmdPin> {
card = %name, card = %name,
was = %prev, was = %prev,
"amdgpu performance level pinned to high (encode clock sag removed) — \ "amdgpu performance level pinned to high (encode clock sag removed) — \
restored on host exit" restored when the last client disconnects"
); );
pins.push(AmdPin { pins.push(AmdPin {
path, path,
@@ -327,7 +402,8 @@ fn pin_nvidia() -> Option<NvmlPin> {
} }
tracing::info!( tracing::info!(
devices = pinned.len(), devices = pinned.len(),
"NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released on host exit" "NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released when the last \
client disconnects"
); );
Some(NvmlPin { nvml, pinned }) Some(NvmlPin { nvml, pinned })
} }
+10 -7
View File
@@ -246,14 +246,17 @@ fn real_main() -> Result<()> {
crate::capture::dxgi::install_gpu_pref_hook(); crate::capture::dxgi::install_gpu_pref_hook();
} }
// NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile and, // NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile. The
// under PUNKTFUNK_PIN_CLOCKS, hold the NVML core-clock floor for the host lifetime (reset on // vendor clock *pin* (PUNKTFUNK_PIN_CLOCKS) is no longer held for the host lifetime — it is
// exit via the guard's Drop). No-op off NVIDIA / on the tool subcommands. // armed per live client via `gpuclocks::session_pin()` on both streaming planes, so idle clocks
// are left alone while nobody is connected. No-op off NVIDIA / on the tool subcommands.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
let _nv_clocks = match args.first().map(String::as_str) { if matches!(
Some("serve") | Some("punktfunk1-host") => gpuclocks::on_host_start(), args.first().map(String::as_str),
_ => None, Some("serve") | Some("punktfunk1-host")
}; ) {
gpuclocks::on_host_start();
}
match args.first().map(String::as_str) { match args.first().map(String::as_str) {
// The host: the native punktfunk/1 plane + management API by default (secure), and — with // The host: the native punktfunk/1 plane + management API by default (secure), and — with
+55 -6
View File
@@ -76,6 +76,8 @@ mod handshake;
/// The mid-stream control task (plan §W1); `serve_session` spawns `control::run` after the /// The mid-stream control task (plan §W1); `serve_session` spawns `control::run` after the
/// handshake to multiplex renegotiation / speed-test control messages onto the data-plane channels. /// handshake to multiplex renegotiation / speed-test control messages onto the data-plane channels.
mod control; mod control;
/// Cursor-forward channel (M2): the encode loop's shape/state emission.
mod cursor_fwd;
/// The capture→encode→send data plane (plan §W1); `serve_session` dispatches the synthetic or /// The capture→encode→send data plane (plan §W1); `serve_session` dispatches the synthetic or
/// virtual source here (`synthetic_stream` / `virtual_stream`) and hands the latter a /// virtual source here (`synthetic_stream` / `virtual_stream`) and hands the latter a
@@ -439,7 +441,11 @@ pub(crate) async fn serve(
) )
.await .await
{ {
Ok(()) => tracing::info!(%peer, "session complete"), Ok(Served::Session) => tracing::info!(%peer, "session complete"),
Ok(Served::ProbeClose) => tracing::debug!(
%peer,
"closed before the control handshake (reachability probe)"
),
Err(e) => { Err(e) => {
tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error") tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error")
} }
@@ -629,6 +635,15 @@ type AudioCapSlot = Arc<std::sync::Mutex<Option<Box<dyn crate::audio::AudioCaptu
/// connection (the host stops waiting at once). /// connection (the host stops waiting at once).
const PENDING_APPROVAL_WAIT: std::time::Duration = std::time::Duration::from_secs(180); const PENDING_APPROVAL_WAIT: std::time::Duration = std::time::Duration::from_secs(180);
/// How a served connection ended. A peer that completes the QUIC handshake and closes cleanly
/// (code 0) without ever opening the control stream is a reachability probe (the clients'
/// hosts-page "online" pips / `--reachable`) or an abandoned connect — routine, and logged
/// quietly: as a WARN it buried the real failures in a wake-on-LAN triage log.
enum Served {
Session,
ProbeClose,
}
/// One client session: handshake → input/audio planes → data plane until done/disconnect. /// One client session: handshake → input/audio planes → data plane until done/disconnect.
/// Everything torn down on return (RAII: virtual output, encoder, threads via channel close). /// Everything torn down on return (RAII: virtual output, encoder, threads via channel close).
/// A connection whose first message is a PairRequest runs the pairing ceremony instead. /// A connection whose first message is a PairRequest runs the pairing ceremony instead.
@@ -651,14 +666,23 @@ async fn serve_session(
// parked knock can't hold a streaming slot. `sem` is the pool it re-acquires from. // parked knock can't hold a streaming slot. `sem` is the pool it re-acquires from.
mut permit: tokio::sync::OwnedSemaphorePermit, mut permit: tokio::sync::OwnedSemaphorePermit,
sem: Arc<tokio::sync::Semaphore>, sem: Arc<tokio::sync::Semaphore>,
) -> Result<()> { ) -> Result<Served> {
let peer = conn.remote_address(); let peer = conn.remote_address();
// First message decides what this connection is: a pairing ceremony or a session. // First message decides what this connection is: a pairing ceremony or a session.
let (mut send, mut recv) = tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi()) let (mut send, mut recv) = match tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi())
.await .await
.map_err(|_| anyhow!("control stream timeout"))? .map_err(|_| anyhow!("control stream timeout"))?
.context("accept control stream")?; {
// A clean close before any control stream: a reachability probe / abandoned connect,
// not a failed session (see [`Served::ProbeClose`]).
Err(quinn::ConnectionError::ApplicationClosed(ref ac))
if ac.error_code == quinn::VarInt::from_u32(0) =>
{
return Ok(Served::ProbeClose);
}
r => r.context("accept control stream")?,
};
let first = tokio::time::timeout(HANDSHAKE_TIMEOUT, io::read_msg(&mut recv)) let first = tokio::time::timeout(HANDSHAKE_TIMEOUT, io::read_msg(&mut recv))
.await .await
.map_err(|_| anyhow!("first message timeout"))??; .map_err(|_| anyhow!("first message timeout"))??;
@@ -709,7 +733,9 @@ async fn serve_session(
} }
*last = Some(std::time::Instant::now()); *last = Some(std::time::Instant::now());
} }
return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin).await; return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin)
.await
.map(|()| Served::Session);
} }
// Pairing gate for a session Hello (a PairRequest was handled above). Lifted OUT of the // Pairing gate for a session Hello (a PairRequest was handled above). Lifted OUT of the
@@ -919,6 +945,15 @@ async fn serve_session(
// accepted ack as "the active mode is now X" and fixes itself; old clients just log it. // accepted ack as "the active mode is now X" and fixes itself; old clients just log it.
let (reconfig_result_tx, reconfig_result_rx) = let (reconfig_result_tx, reconfig_result_rx) =
tokio::sync::mpsc::unbounded_channel::<Reconfigured>(); tokio::sync::mpsc::unbounded_channel::<Reconfigured>();
// Cursor-forward bridge (M2): the encode loop diffs each frame's cursor serial and hands
// changed SHAPES here; the control task (the control stream's sole writer) sends them.
// Same shape as `probe_result_tx`. Wired even when the channel wasn't negotiated — it
// just never fires then.
let (cursor_shape_tx, cursor_shape_rx) =
tokio::sync::mpsc::unbounded_channel::<punktfunk_core::quic::CursorShape>();
// Negotiated cursor forwarding: MUST match the HOST_CAP_CURSOR bit the Welcome advertised
// (handshake::cursor_forward is the single predicate both read).
let cursor_forward = handshake::cursor_forward(hello.client_caps, compositor);
// Adaptive FEC: the control task maps each client LossReport to a recovery percent and publishes // Adaptive FEC: the control task maps each client LossReport to a recovery percent and publishes
// it here; the data-plane send loop reads + applies it per frame. Disabled (pinned) when // it here; the data-plane send loop reads + applies it per frame. Disabled (pinned) when
// PUNKTFUNK_FEC_PCT is set. Seeded with the session's starting FEC so it's a no-op until a report. // PUNKTFUNK_FEC_PCT is set. Seeded with the session's starting FEC so it's a no-op until a report.
@@ -954,6 +989,7 @@ async fn serve_session(
probe_tx, probe_tx,
probe_result_rx, probe_result_rx,
reconfig_result_rx, reconfig_result_rx,
cursor_shape_rx,
clip_enabled, clip_enabled,
clip, clip,
)); ));
@@ -1175,6 +1211,13 @@ async fn serve_session(
launch: hello.launch.clone(), launch: hello.launch.clone(),
plane: crate::events::Plane::Native, plane: crate::events::Plane::Native,
}); });
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock floor for
// as long as THIS session streams, refcounted with every other live session across both planes.
// RAII like the marker above — armed on the first live client, released when the last one
// disconnects, so idle clocks aren't pinned while nobody is connected. No-op off Linux / when
// the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
// The session's launch, threaded into the data plane. Windows carries the store-qualified id // The session's launch, threaded into the data plane. Windows carries the store-qualified id
// (spawned into the interactive user session once capture is live); other hosts resolve the id // (spawned into the interactive user session once capture is live); other hosts resolve the id
// to its shell command HERE against the host's own library — a client can only ever pick an // to its shell command HERE against the host's own library — a client can only ever pick an
@@ -1332,6 +1375,8 @@ async fn serve_session(
fec_target: fec_target_dp, fec_target: fec_target_dp,
conn: conn_stream, conn: conn_stream,
timing_conn, timing_conn,
cursor_forward,
cursor_shape_tx,
probe_seq, probe_seq,
streamed_au, streamed_au,
stats: stats_dp, stats: stats_dp,
@@ -1392,7 +1437,7 @@ async fn serve_session(
// host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the // host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the
// TV's gaming session back so it's the default when no one is streaming. // TV's gaming session back so it's the default when no one is streaming.
crate::vdisplay::restore_managed_session(); crate::vdisplay::restore_managed_session();
result result.map(|()| Served::Session)
} }
/// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails /// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails
@@ -1998,6 +2043,7 @@ mod tests {
0, // video_codecs (HEVC-only) 0, // video_codecs (HEVC-only)
0, // preferred_codec 0, // preferred_codec
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
None, // pin (TOFU) None, // pin (TOFU)
None, // identity (host doesn't require pairing) None, // identity (host doesn't require pairing)
@@ -2168,6 +2214,7 @@ mod tests {
0, // video_codecs (0 → HEVC-only) 0, // video_codecs (0 → HEVC-only)
0, // preferred_codec (auto) 0, // preferred_codec (auto)
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
None, // pin: TOFU — the operator's approval (not a PIN) authorizes this client None, // pin: TOFU — the operator's approval (not a PIN) authorizes this client
Some((cert, key)), Some((cert, key)),
@@ -2235,6 +2282,7 @@ mod tests {
0, // video_codecs 0, // video_codecs
0, // preferred_codec 0, // preferred_codec
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
None, None,
None, None,
@@ -2264,6 +2312,7 @@ mod tests {
0, // video_codecs 0, // video_codecs
0, // preferred_codec 0, // preferred_codec
None, // display_hdr None, // display_hdr
0, // client_caps
None, // launch None, // launch
Some(host_fp), Some(host_fp),
Some((cert.clone(), key.clone())), Some((cert.clone(), key.clone())),
@@ -30,6 +30,7 @@ pub(super) async fn run(
probe_tx: std::sync::mpsc::Sender<ProbeRequest>, probe_tx: std::sync::mpsc::Sender<ProbeRequest>,
mut probe_result_rx: tokio::sync::mpsc::UnboundedReceiver<ProbeResult>, mut probe_result_rx: tokio::sync::mpsc::UnboundedReceiver<ProbeResult>,
mut reconfig_result_rx: tokio::sync::mpsc::UnboundedReceiver<Reconfigured>, mut reconfig_result_rx: tokio::sync::mpsc::UnboundedReceiver<Reconfigured>,
mut cursor_shape_rx: tokio::sync::mpsc::UnboundedReceiver<punktfunk_core::quic::CursorShape>,
clip_enabled: Arc<AtomicBool>, clip_enabled: Arc<AtomicBool>,
clip: pf_clipboard::ClipCoord, clip: pf_clipboard::ClipCoord,
) { ) {
@@ -262,6 +263,15 @@ pub(super) async fn run(
break; break;
} }
} }
shape = cursor_shape_rx.recv() => {
// Cursor-forward bridge (M2): the encode loop diffed a new pointer bitmap.
// Rare (shape changes are human-paced); ≤ ~58 KiB fits the u16 frame by
// construction (cursor_fwd downscales).
let Some(shape) = shape else { break }; // data plane gone
if io::write_msg(&mut ctrl_send, &shape.encode()).await.is_err() {
break;
}
}
offer = clip_offer_rx.recv(), if !clip_offer_closed => { offer = clip_offer_rx.recv(), if !clip_offer_closed => {
// Host copied → the coordinator minted a `ClipOffer`; forward it to the client // Host copied → the coordinator minted a `ClipOffer`; forward it to the client
// (only while sync is on — a race with a just-received disable would otherwise // (only while sync is on — a race with a just-received disable would otherwise
@@ -0,0 +1,144 @@
//! Cursor-forward channel, host side (design/remote-desktop-sweep.md M2).
//!
//! When the session negotiated the cursor channel (client `CLIENT_CAP_CURSOR` met our
//! `HOST_CAP_CURSOR`), the encoder stops blending the pointer into the video
//! (`SessionPlan::cursor_blend = false`) and the encode loop forwards it out-of-band instead:
//! the SHAPE (bitmap + hotspot, rare) rides the reliable control stream via the control-task
//! bridge, per-tick STATE (position/visibility, 14 B) rides a lossy `0xD0` datagram — resent
//! every iteration so loss self-heals with no refresh timer.
use punktfunk_core::quic::{
encode_cursor_state_datagram, CursorShape, CursorState, CURSOR_SHAPE_MAX_SIDE, CURSOR_VISIBLE,
};
/// Per-session forward state, owned by the encode loop (the thread that binds frames).
pub(super) struct CursorForwarder {
/// Serial of the last shape handed to the control-task bridge (`None` = none yet).
sent_serial: Option<u64>,
/// Last visible pointer position (hotspot point, frame px) — held across hidden spans so
/// a hide still states WHERE the pointer was (the M3 reappear position).
last_pos: (i32, i32),
}
impl CursorForwarder {
pub(super) fn new() -> CursorForwarder {
CursorForwarder {
sent_serial: None,
last_pos: (0, 0),
}
}
/// Called once per encode-loop iteration with the bound frame's overlay (also on repeat
/// iterations — the state datagram is the plane's loss heal, so it goes out every tick).
/// `None` overlay = hidden pointer (or no bitmap yet): state only, `visible` clear.
pub(super) fn tick(
&mut self,
cursor: Option<&pf_frame::CursorOverlay>,
conn: &quinn::Connection,
shape_tx: &tokio::sync::mpsc::UnboundedSender<CursorShape>,
) {
let flags = match cursor {
Some(ov) => {
if self.sent_serial != Some(ov.serial) {
if let Some(shape) = shape_from_overlay(ov) {
// Bridge full ⇒ control task gone ⇒ session is tearing down anyway.
let _ = shape_tx.send(shape);
self.sent_serial = Some(ov.serial);
}
}
self.last_pos = (ov.x + ov.hot_x as i32, ov.y + ov.hot_y as i32);
CURSOR_VISIBLE
}
None => 0,
};
let state = CursorState {
serial: self.sent_serial.unwrap_or(0) as u32,
flags,
x: self.last_pos.0,
y: self.last_pos.1,
};
let _ = conn.send_datagram(encode_cursor_state_datagram(&state).into());
}
}
/// Build the wire shape from a capture overlay, integer-downscaling (nearest-neighbor) anything
/// over [`CURSOR_SHAPE_MAX_SIDE`] so the message always fits the u16-length control frame.
/// Real cursors are far under the cap — the scale path is a correctness backstop for XL
/// accessibility cursors, not a quality path. `None` on a malformed overlay (short buffer).
fn shape_from_overlay(ov: &pf_frame::CursorOverlay) -> Option<CursorShape> {
let px = (ov.w as usize).checked_mul(ov.h as usize)?.checked_mul(4)?;
if ov.w == 0 || ov.h == 0 || ov.rgba.len() < px {
return None;
}
let max = CURSOR_SHAPE_MAX_SIDE as u32;
let f = ov.w.max(ov.h).div_ceil(max).max(1);
let (w, h) = (ov.w.div_ceil(f), ov.h.div_ceil(f));
let rgba = if f == 1 {
ov.rgba.as_ref().clone()
} else {
let mut out = Vec::with_capacity((w * h * 4) as usize);
for y in 0..h {
for x in 0..w {
let (sx, sy) = ((x * f).min(ov.w - 1), (y * f).min(ov.h - 1));
let o = ((sy * ov.w + sx) * 4) as usize;
out.extend_from_slice(&ov.rgba[o..o + 4]);
}
}
out
};
Some(CursorShape {
serial: ov.serial as u32,
w: w as u16,
h: h as u16,
hot_x: (ov.hot_x / f).min(w - 1) as u16,
hot_y: (ov.hot_y / f).min(h - 1) as u16,
rgba,
})
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
fn overlay(w: u32, h: u32, hot: (u32, u32)) -> pf_frame::CursorOverlay {
pf_frame::CursorOverlay {
x: 10,
y: 20,
w,
h,
rgba: Arc::new((0..w * h * 4).map(|i| i as u8).collect()),
serial: 3,
hot_x: hot.0,
hot_y: hot.1,
}
}
#[test]
fn small_shape_passes_through() {
let s = shape_from_overlay(&overlay(32, 32, (4, 5))).unwrap();
assert_eq!((s.w, s.h, s.hot_x, s.hot_y, s.serial), (32, 32, 4, 5, 3));
assert_eq!(s.rgba.len(), 32 * 32 * 4);
// Encodes within the u16 control-frame cap.
assert!(s.encode().len() <= u16::MAX as usize);
}
#[test]
fn oversize_shape_downscales_with_hotspot() {
// 256² → f = ceil(256/120) = 3 → 86² (256.div_ceil(3)), hotspot scales with it.
let s = shape_from_overlay(&overlay(256, 256, (255, 0))).unwrap();
assert!(s.w <= CURSOR_SHAPE_MAX_SIDE && s.h <= CURSOR_SHAPE_MAX_SIDE);
assert_eq!(s.rgba.len(), s.w as usize * s.h as usize * 4);
assert!(s.hot_x < s.w && s.hot_y < s.h);
assert!(s.encode().len() <= u16::MAX as usize);
// The scaled message must decode (dims within the cap).
assert_eq!(CursorShape::decode(&s.encode()).unwrap(), s);
}
#[test]
fn short_buffer_rejected() {
let mut ov = overlay(8, 8, (0, 0));
ov.rgba = Arc::new(vec![0; 8]);
assert!(shape_from_overlay(&ov).is_none());
}
}
@@ -8,6 +8,22 @@
use super::*; use super::*;
/// Whether this session forwards the cursor out-of-band (design/remote-desktop-sweep.md M2):
/// the client asked ([`CLIENT_CAP_CURSOR`](punktfunk_core::quic::CLIENT_CAP_CURSOR)) AND the
/// capture path can deliver cursor metadata separately from the frame — today that is the
/// Linux portal `SPA_META_Cursor` path only: not gamescope (its capture paints no cursor at
/// all), not Windows (DWM composites into the IDD frame — M2c). THE single predicate: the
/// Welcome's `HOST_CAP_CURSOR` bit and the session's forwarding/blend-off wiring both read it,
/// so they can never disagree.
pub(super) fn cursor_forward(
client_caps: u8,
compositor: Option<crate::vdisplay::Compositor>,
) -> bool {
cfg!(target_os = "linux")
&& client_caps & punktfunk_core::quic::CLIENT_CAP_CURSOR != 0
&& compositor.is_some_and(|c| c != crate::vdisplay::Compositor::Gamescope)
}
/// Run the Hello→Welcome→Start negotiation. Borrows the control streams (the caller keeps them for /// Run the Hello→Welcome→Start negotiation. Borrows the control streams (the caller keeps them for
/// mid-stream renegotiation afterwards). `first` is the already-read first control message. /// mid-stream renegotiation afterwards). `first` is the already-read first control message.
#[allow(clippy::type_complexity, clippy::too_many_arguments)] #[allow(clippy::type_complexity, clippy::too_many_arguments)]
@@ -354,6 +370,28 @@ pub(super) async fn negotiate(
// just follow. // just follow.
let mut salt = [0u8; 4]; let mut salt = [0u8; 4];
rand::thread_rng().fill_bytes(&mut salt); rand::thread_rng().fill_bytes(&mut salt);
// Session AEAD: ChaCha20-Poly1305 when the client asked for it (VIDEO_CAP_CHACHA20 — the
// soft-AES armv7 targets, whose GCM decrypt caps at ~100 Mbps) and the operator
// kill-switch allows (PUNKTFUNK_CHACHA20, default on — pure rollout safety; perf-only,
// both AEADs are full-strength). The fresh-per-session discipline above applies to this
// key identically; the legacy 16-byte `key` stays independently random so nothing
// downstream ever observes an all-zero key.
let client_wants_chacha = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_CHACHA20 != 0;
let chacha = client_wants_chacha && pf_host_config::config().chacha20;
let key_chacha = chacha.then(|| {
let mut k = [0u8; 32];
rand::thread_rng().fill_bytes(&mut k);
k
});
tracing::info!(
cipher = if chacha {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
client_wants_chacha,
"session cipher"
);
let welcome = Welcome { let welcome = Welcome {
abi_version: punktfunk_core::WIRE_VERSION, abi_version: punktfunk_core::WIRE_VERSION,
udp_port, udp_port,
@@ -422,7 +460,25 @@ pub(super) async fn negotiate(
punktfunk_core::quic::HOST_CAP_CLIPBOARD punktfunk_core::quic::HOST_CAP_CLIPBOARD
} else { } else {
0 0
}
// Cursor channel granted (client asked + this capture path can deliver cursor
// metadata out of the frame) — the client turns its local renderer on ONLY when
// it sees this bit, and serve_session wires forwarding from the same predicate.
| if cursor_forward(hello.client_caps, compositor) {
punktfunk_core::quic::HOST_CAP_CURSOR
} else {
0
}, },
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
// pre-cipher wire form. The host's own data plane picks the cipher up via
// `welcome.session_config` — no other host change.
cipher: if chacha {
punktfunk_core::quic::CIPHER_CHACHA20_POLY1305
} else {
punktfunk_core::quic::CIPHER_AES_128_GCM
},
key_chacha,
}; };
io::write_msg(send, &welcome.encode()).await?; io::write_msg(send, &welcome.encode()).await?;
bringup.mark("welcome"); bringup.mark("welcome");
+27 -1
View File
@@ -938,6 +938,14 @@ pub(super) struct SessionContext {
/// thread emits one 0xCF datagram per AU (capture→sent µs) on it, so the client can split its /// thread emits one 0xCF datagram per AU (capture→sent µs) on it, so the client can split its
/// `host+network` latency stage. `None` = older client, no emission. /// `host+network` latency stage. `None` = older client, no emission.
pub(super) timing_conn: Option<quinn::Connection>, pub(super) timing_conn: Option<quinn::Connection>,
/// The session negotiated the cursor channel (design/remote-desktop-sweep.md M2 —
/// `handshake::cursor_forward`): the encoder does NOT blend the pointer into the video;
/// the encode loop forwards shape (via `cursor_shape_tx`) + per-tick `0xD0` state instead.
pub(super) cursor_forward: bool,
/// SHAPE bridge to the control task (the control stream's sole writer) — mirrors
/// `probe_result_tx`. Inert when `cursor_forward` is false.
pub(super) cursor_shape_tx:
tokio::sync::mpsc::UnboundedSender<punktfunk_core::quic::CursorShape>,
/// The client advertised [`punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ`]: speed-test bursts may /// The client advertised [`punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ`]: speed-test bursts may
/// run mid-session in the probe index space (its reassembler keeps a separate probe window). /// run mid-session in the probe index space (its reassembler keeps a separate probe window).
/// `false` = older client whose single-window reassembler would drop probe-space frames as /// `false` = older client whose single-window reassembler would drop probe-space frames as
@@ -987,7 +995,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
ctx.bit_depth, ctx.bit_depth,
ctx.chroma, ctx.chroma,
ctx.codec, ctx.codec,
ctx.compositor != pf_vdisplay::Compositor::Gamescope, // Blend the pointer into the video only where the capture HAS one (not gamescope) AND
// the client is not drawing it locally (the M2 cursor channel — blending too would
// show it twice).
ctx.compositor != pf_vdisplay::Compositor::Gamescope && !ctx.cursor_forward,
); );
// PyroWave rides the datagram-aligned wire mode (§4.4): every encoder this session opens // PyroWave rides the datagram-aligned wire mode (§4.4): every encoder this session opens
// packetizes at the negotiated shard payload, so a lost datagram costs blocks, not frames. // packetizes at the negotiated shard payload, so a lost datagram costs blocks, not frames.
@@ -1021,6 +1032,8 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
fec_target, fec_target,
conn, conn,
timing_conn, timing_conn,
cursor_forward,
cursor_shape_tx,
probe_seq, probe_seq,
streamed_au, streamed_au,
stats, stats,
@@ -1034,6 +1047,13 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// reverts to whole-AU sends without touching the encoder's slicing knobs). The third gate — // reverts to whole-AU sends without touching the encoder's slicing knobs). The third gate —
// whether the ENCODER actually chunks — is dynamic (`supports_chunked_poll`, per AU). // whether the ENCODER actually chunks — is dynamic (`supports_chunked_poll`, per AU).
let streamed_wire = streamed_au && std::env::var("PUNKTFUNK_STREAMED_AU").as_deref() != Ok("0"); let streamed_wire = streamed_au && std::env::var("PUNKTFUNK_STREAMED_AU").as_deref() != Ok("0");
// Cursor-forward state (M2): shape-serial diffing + the per-tick 0xD0 state send. The
// encoder was told not to blend (SessionPlan above), so from the first frame the client's
// locally-drawn cursor is the only one.
let mut cursor_fwd = cursor_forward.then(super::cursor_fwd::CursorForwarder::new);
if cursor_forward {
tracing::info!("cursor channel negotiated — forwarding shape/state, encoder blend off");
}
if streamed_wire { if streamed_wire {
tracing::info!( tracing::info!(
"client accepts streamed AUs (VIDEO_CAP_STREAMED_AU) — chunked encoder output \ "client accepts streamed AUs (VIDEO_CAP_STREAMED_AU) — chunked encoder output \
@@ -1985,6 +2005,12 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
); );
} }
} }
// Cursor channel (M2): every iteration — new frame OR repeat — states the pointer
// (self-healing under datagram loss) and forwards a changed shape via the control
// bridge. `frame` is the newest bound frame either way.
if let Some(fwd) = cursor_fwd.as_mut() {
fwd.tick(frame.cursor.as_ref(), &conn, &cursor_shape_tx);
}
if perf && diag_at.elapsed() >= std::time::Duration::from_secs(2) { if perf && diag_at.elapsed() >= std::time::Duration::from_secs(2) {
let secs = diag_at.elapsed().as_secs_f64(); let secs = diag_at.elapsed().as_secs_f64();
tracing::info!( tracing::info!(
+60 -9
View File
@@ -168,12 +168,29 @@ The canonical "decide, don't just observe" pattern — approve pairing from your
## Recipe: full controller passthrough (VirtualHere) ## Recipe: full controller passthrough (VirtualHere)
To get a controller's *native* features on the host — DualSense gyro, touchpad, adaptive To get a controller's *native* features on the host — DualSense gyro, touchpad, adaptive
triggers, USB rumble — instead of the emulated pad, share the physical device from the couch with triggers, USB rumble — instead of the emulated pad, hand the physical device from the couch to the
[VirtualHere](https://www.virtualhere.com/) (USB-over-IP) and bind it to the host only while a host over [VirtualHere](https://www.virtualhere.com/) (USB-over-IP), bound only while a client is
client is connected. The couch runs the VirtualHere **server** (sharing the pad); the host runs connected so the couch keeps its own controller the rest of the time.
the VirtualHere **client** and this automation drives its `-t` IPC.
Zero-code, bracketed on the stream with two hooks: **The two sides.** VirtualHere is a server/client pair, and you run both:
- **Server — on the couch** (where the pad is physically plugged in). Run the VirtualHere USB
Server there; it shares the pad on the LAN. Leave it running.
- **Client — on the host** (where the game and this automation run). Install the VirtualHere
Client (as a service, or the tray app). It auto-discovers the couch's shared pad on the same
LAN; across subnets, add it once with `<VH_CLIENT> -t "MANUAL HUB ADD,<couch-ip>:7575"`. The
client binary is `vhclientx86_64` on Linux, `vhui64.exe` on Windows, `vhclientosx` on macOS,
`vhclientarm64` on ARM Linux.
The client's `-t` flag is a one-shot IPC to the already-running client: `-t LIST` prints every
visible device with its address (`server.port`, e.g. `couch-deck.11`); `-t "USE,<addr>"` mounts it
onto the host; `-t "STOP USING,<addr>"` hands it back. The automation just brackets
`USE` / `STOP USING` around a session.
### Zero-code: two hooks
Bracket it on the stream with two [hooks](#hooks-hooksjson) — mount when video starts, release
when it stops. This is all most setups need:
```json ```json
{ {
@@ -184,8 +201,42 @@ Zero-code, bracketed on the stream with two hooks:
} }
``` ```
`couch-deck.11` is the device's VirtualHere address (`vhclientx86_64 -t LIST`); same-LAN setups `couch-deck.11` is the device's VirtualHere address from `vhclientx86_64 -t LIST`.
auto-discover it, otherwise `MANUAL HUB ADD,<couch-ip>:7575` once. For a version that resolves the
device by name, filters to one couch, and releases the pad on a clean shutdown, see the ### Scripted: resolve by name, release on shutdown
The hooks above hard-code the address, and can strand the pad on the host if it's stopped
mid-stream (the `stream.stopped` hook never fires). The
[`virtualhere-dualsense.ts`](https://git.unom.io/unom/punktfunk/src/branch/main/sdk/examples/virtualhere-dualsense.ts) [`virtualhere-dualsense.ts`](https://git.unom.io/unom/punktfunk/src/branch/main/sdk/examples/virtualhere-dualsense.ts)
SDK example. SDK example is the robust version: it resolves the device by **name substring** (survives the
address changing), can filter to **one couch** in a multi-client setup, and **releases the pad on
a clean shutdown** (`systemctl stop`, ^C) so the couch always gets its controller back.
It's a standalone [`@punktfunk/host` script](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
— run it in its own directory. The one edit is its import: the in-repo copy imports from
`../src/index.js`; outside the repo it's the published package:
```sh
mkdir ~/punktfunk-scripts && cd ~/punktfunk-scripts
bun init -y
bun add @punktfunk/host # point the @punktfunk scope at the registry first — see the SDK README
# save the example as virtualhere-dualsense.ts, and change its first import to the package:
# - import { connect } from "../src/index.js";
# + import { connect } from "@punktfunk/host";
VH_DEVICE=DualSense bun virtualhere-dualsense.ts
```
`VH_DEVICE` is required — a VirtualHere address (`couch-deck.11`) or a device-name substring
(`DualSense`). Optional: `VH_CLIENT` overrides the client binary (default `vhclientx86_64`);
`VH_ONLY_CLIENT` binds only for one punktfunk client label. Running on the host box the SDK needs
no token or URL — it reads the host's loopback credentials itself (see
[Connection resolution](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#connection-resolution)).
Keep it running as a
[systemd user unit](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
(its default `SIGTERM` triggers the script's own release step — so `systemctl stop` hands the pad
back), or drop it under the [scripting runner](/docs/plugins) with your other units.
> The example brackets on `client.connected` / `client.disconnected` — the pad returns to the
> couch the moment they disconnect. Switch to `stream.started` / `stream.stopped` if you'd rather
> pass it through only while video is actually flowing; both are noted in the file's header.
+1
View File
@@ -90,6 +90,7 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
| `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. | | `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. |
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. | | `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. |
| `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. | | `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
| `PUNKTFUNK_CHACHA20` | `1` · `0` *(default on)* | ChaCha20-Poly1305 session encryption for clients without hardware AES (old ARM TVs, e.g. webOS), lifting their ~100 Mbps software-AES decrypt ceiling. **On by default** on the host; a session uses it only when the client requests it — everyone else stays on AES-GCM. Purely a performance choice (both ciphers are full-strength); set `0` to force AES-GCM for all sessions. |
| `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. | | `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. |
| `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. | | `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. |
| `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. | | `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. |
+18 -2
View File
@@ -58,8 +58,10 @@ It is idempotent — safe to re-run. In one pass it:
1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain, 1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain,
2. builds `punktfunk-host` (and the web console), 2. builds `punktfunk-host` (and the web console),
3. writes config to `~/.config/punktfunk/` (a generated web-console login password), 3. writes config to `~/.config/punktfunk/` (a generated web-console login password),
4. raises the UDP socket buffers to 32 MB and adds you to the `input` group (needs `sudo`; skipped 4. raises the UDP socket buffers to 32 MB, installs the gamepad udev rule + the `vhci-hcd` autoload
with a warning if unavailable), and adds you to the `input` group (virtual gamepads / **native Steam Deck controller passthrough**),
and seeds the KDE RemoteDesktop grant for Desktop-mode input — this step **prompts for your `sudo`
password** (a stock Steam Deck requires one; without it gamepad passthrough and the UDP tuning are skipped),
5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger, 5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger,
so they run without a login session). so they run without a login session).
@@ -82,6 +84,14 @@ When it finishes it prints the web-console URL and how to pair.
> If you only ever use native clients, install with `--no-gamestream` for a host with no GameStream > If you only ever use native clients, install with `--no-gamestream` for a host with no GameStream
> surface at all. > surface at all.
> **First install — reboot once before streaming.** KWin only authorizes Desktop-mode screen capture
> on a fresh session, and the new `input` group (native Steam Deck controller passthrough) only takes
> effect on a new login — so after the **first** install, **reboot the Deck** (a re-run that changes
> nothing doesn't need it). Streaming **Game Mode** with a generic Xbox pad works right away; **Desktop
> capture and the native Steam Deck controller need the reboot.** If a client connects and every
> session ends with `KWin does not expose zkde_screencast_unstable_v1` or the pad shows up as an Xbox
> 360 controller, you haven't rebooted yet.
## 3. Pair a device ## 3. Pair a device
By default the host **requires PIN pairing** (secure). Two ways to pair: By default the host **requires PIN pairing** (secure). Two ways to pair:
@@ -128,6 +138,12 @@ bash ~/punktfunk/scripts/steamdeck/update.sh
thrash the managed session. Pick one mode per session. thrash the managed session. Pick one mode per session.
- **Keep the device awake.** On handhelds, Game Mode auto-suspends on idle, which drops the host off - **Keep the device awake.** On handhelds, Game Mode auto-suspends on idle, which drops the host off
the network mid stream — disable auto-suspend (Settings → Power) for a headless host. the network mid stream — disable auto-suspend (Settings → Power) for a headless host.
- **Native Steam Deck controller passthrough** presents the client's pad as a real Steam Deck
controller (paddles, trackpads, gyro) via a virtual USB device — that needs the `input` group and the
`vhci-hcd` module live, so it only works **after the first-install reboot** above; until then the pad
degrades to a generic Xbox 360 controller (still fully playable). If you're streaming *to* another
Steam Deck, also set Steam Input to **Off** for Punktfunk on that Deck — see
[Stream to a Steam Deck](/docs/steam-deck).
- **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to - **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to
start after an update, just re-run `update.sh` to rebuild against the new base. start after an update, just re-run `update.sh` to rebuild against the new base.
- Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md). - Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md).
+85 -1
View File
@@ -272,7 +272,7 @@
#define INBOUND_REQ_FLAG 2147483648 #define INBOUND_REQ_FLAG 2147483648
#endif #endif
// 16-byte AEAD authentication tag appended by GCM. // 16-byte AEAD authentication tag appended by either session cipher.
#define TAG_LEN 16 #define TAG_LEN 16
// Wire tag distinguishing an input datagram from a video packet. // Wire tag distinguishing an input datagram from a video packet.
@@ -465,6 +465,20 @@
#define VIDEO_CAP_STREAMED_AU 32 #define VIDEO_CAP_STREAMED_AU 32
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
// control channel, so there is no downgrade surface.
#define VIDEO_CAP_CHACHA20 64
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] // [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder // (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -484,6 +498,28 @@
#define HOST_CAP_CLIPBOARD 2 #define HOST_CAP_CLIPBOARD 2
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::client_caps`] bit: the client renders the host cursor LOCALLY
// (design/remote-desktop-sweep.md M2). It consumes [`CursorShape`](super::control::CursorShape)
// control messages (RGBA bitmap + hotspot, cached by serial) and per-frame
// [`CursorState`](super::datagram::CursorState) `0xD0` datagrams (position/visibility), and
// draws the pointer itself — so the host must STOP compositing the cursor into the video
// (`SessionPlan.cursor_blend = false`) or the user sees it twice. Active only when the host
// answers with [`HOST_CAP_CURSOR`] (capable-and-agreed, the 444/clipboard precedent); toward
// an older or incapable host nothing changes.
#define CLIENT_CAP_CURSOR 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
// whose capture carries no cursor, and NOT Windows yet, where DWM composites into the IDD
// frame). Set only when the client asked via [`CLIENT_CAP_CURSOR`]; when both bits agree the
// host stops blending and ships [`CursorShape`](super::control::CursorShape) +
// [`CursorState`](super::datagram::CursorState) instead.
#define HOST_CAP_CURSOR 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** // [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST // encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -736,6 +772,20 @@
#define CLIP_FILE_INDEX_NONE UINT32_MAX #define CLIP_FILE_INDEX_NONE UINT32_MAX
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Type byte of [`CursorShape`] (host → client): the pointer's bitmap + hotspot changed.
#define MSG_CURSOR_SHAPE 80
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Per-side pixel cap for a forwarded cursor bitmap. The control-stream frame is length-prefixed
// with a `u16`, so a whole message must fit 65535 bytes — 128×128 RGBA (65536 B) already
// overshoots before the 17-byte header. 120² (57.6 KiB + header) fits with headroom and covers
// real cursors (typically ≤ 64 px, ≤ 96 px at HiDPI scale); the HOST downscales anything
// larger before forwarding, so the cap is invisible to clients.
#define CURSOR_SHAPE_MAX_SIDE 120
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// Datagram wire tags. Video rides UDP; everything low-rate rides QUIC datagrams, // Datagram wire tags. Video rides UDP; everything low-rate rides QUIC datagrams,
// demultiplexed by the first byte: input = [`crate::input::INPUT_MAGIC`] (0xC8, client→host), // demultiplexed by the first byte: input = [`crate::input::INPUT_MAGIC`] (0xC8, client→host),
@@ -824,6 +874,28 @@
#define HOST_TIMING_MAGIC 207 #define HOST_TIMING_MAGIC 207
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Cursor-state datagram tag, host → client (design/remote-desktop-sweep.md M2). Next tag after
// [`HOST_TIMING_MAGIC`]. Sent once per captured frame while the cursor channel is negotiated
// ([`CLIENT_CAP_CURSOR`](super::caps::CLIENT_CAP_CURSOR) ∧
// [`HOST_CAP_CURSOR`](super::caps::HOST_CAP_CURSOR)) — per-frame resend makes the plane
// self-healing under loss (latest-wins, no refresh timer). The bitmap itself rides the
// reliable control stream ([`CursorShape`](super::control::CursorShape)); this 14-byte
// datagram only moves/hides the pointer.
#define CURSOR_STATE_MAGIC 208
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`CursorState::flags`] bit: the host cursor is visible.
#define CURSOR_VISIBLE 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`CursorState::flags`] bit: a host app captured/hid the pointer — the client SHOULD run
// relative/captured (M3 auto-flip; advisory, user override always wins).
#define CURSOR_RELATIVE_HINT 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// QUIC application error code a punktfunk/1 client closes the control connection with on a // QUIC application error code a punktfunk/1 client closes the control connection with on a
// **deliberate quit** (a user "stop", not a network drop). The host reads it off the connection's // **deliberate quit** (a user "stop", not a network drop). The host reads it off the connection's
@@ -855,6 +927,18 @@
#define HELLO_LAUNCH_MAX 128 #define HELLO_LAUNCH_MAX 128
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
// only one pre-cipher builds know).
#define CIPHER_AES_128_GCM 0
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
#define CIPHER_CHACHA20_POLY1305 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// Type byte of [`PairRequest`]. // Type byte of [`PairRequest`].
#define MSG_PAIR_REQUEST 16 #define MSG_PAIR_REQUEST 16
+89 -11
View File
@@ -48,6 +48,9 @@ BIN="$TARGET_DIR/release/punktfunk-host"
CONFIG="$HOME/.config/punktfunk" CONFIG="$HOME/.config/punktfunk"
UNITS="$HOME/.config/systemd/user" UNITS="$HOME/.config/systemd/user"
XRD="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}" XRD="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"
# Set when this run does something that only a fresh login picks up (input-group add, first-time
# KWin .desktop grant). Drives the loud "reboot before streaming" note in the summary.
NEED_RELOGIN=0
# --- 0. preflight ---------------------------------------------------------- # --- 0. preflight ----------------------------------------------------------
log "Preflight" log "Preflight"
@@ -66,6 +69,33 @@ fi
DISTROBOX="$(command -v distrobox)" # baked into the web unit (may be /usr/bin or ~/.local/bin) DISTROBOX="$(command -v distrobox)" # baked into the web unit (may be /usr/bin or ~/.local/bin)
ok "distrobox: $DISTROBOX" ok "distrobox: $DISTROBOX"
# --- acquire sudo up front (before the ~15-min build) ----------------------
# Steps 4-5 (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger) need root. Prompt NOW,
# not after the build — so you authorize once and walk away, and a non-interactive run fails LOUDLY
# here instead of silently skipping the tuning at the very end. A stock SteamOS 'deck' has no
# password, so sudo can't work until you set one.
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo is needed once (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger):"
if sudo -v; then
SUDO_OK=1
# keep the sudo timestamp warm across the long build so steps 4-5 don't re-prompt / expire
( while sudo -n -v 2>/dev/null; do sleep 50; done ) &
_pf_sudo_keepalive=$!
trap '[ -n "${_pf_sudo_keepalive:-}" ] && kill "$_pf_sudo_keepalive" 2>/dev/null || true' EXIT
fi
fi
if [ "$SUDO_OK" != 1 ]; then
if [ -t 0 ]; then
warn "No sudo — a stock SteamOS 'deck' account has no password. Set one and re-run: passwd"
else
warn "No TTY for the sudo prompt (non-interactive run) — system tuning + linger will be SKIPPED."
warn "Run in Konsole or an interactive 'ssh -t' session (or pre-authorize sudo) to enable them."
fi
fi
# --- 1. build container + toolchain --------------------------------------- # --- 1. build container + toolchain ---------------------------------------
log "Build container '$BOX' ($BOX_IMAGE)" log "Build container '$BOX' ($BOX_IMAGE)"
if distrobox list 2>/dev/null | awk -F'|' '{gsub(/ /,"",$2); print $2}' | grep -qx "$BOX"; then if distrobox list 2>/dev/null | awk -F'|' '{gsub(/ /,"",$2); print $2}' | grep -qx "$BOX"; then
@@ -84,7 +114,7 @@ set -e
export DEBIAN_FRONTEND=noninteractive export DEBIAN_FRONTEND=noninteractive
sudo apt-get update -qq sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends \ sudo apt-get install -y -qq --no-install-recommends \
build-essential pkg-config clang curl git ca-certificates \ build-essential pkg-config clang cmake curl git ca-certificates \
libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev libswscale-dev libavdevice-dev \ libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev libswscale-dev libavdevice-dev \
libpipewire-0.3-dev libspa-0.2-dev \ libpipewire-0.3-dev libspa-0.2-dev \
libgbm-dev libegl-dev libgl-dev libdrm-dev libva-dev \ libgbm-dev libegl-dev libgl-dev libdrm-dev libva-dev \
@@ -148,11 +178,29 @@ fi
# the restricted Wayland globals it lists (see packaging/linux/io.unom.Punktfunk.Host.desktop). # the restricted Wayland globals it lists (see packaging/linux/io.unom.Punktfunk.Host.desktop).
# Exec must therefore be THIS install's binary path, not the packaged /usr/bin one. KWin reads # Exec must therefore be THIS install's binary path, not the packaged /usr/bin one. KWin reads
# grants at session start — after first install, restart the Desktop session (Game Mode and back). # grants at session start — after first install, restart the Desktop session (Game Mode and back).
DESKTOP_DST="$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
# First-time install of the grant: KWin only reads it at session start, so a fresh login is required
# before Desktop-mode capture works. A re-run that just rewrites it needs no relogin.
[ -f "$DESKTOP_DST" ] || NEED_RELOGIN=1
mkdir -p "$HOME/.local/share/applications" mkdir -p "$HOME/.local/share/applications"
sed "s|^Exec=.*|Exec=$BIN|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" \ sed "s|^Exec=.*|Exec=$BIN|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" > "$DESKTOP_DST"
> "$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
ok "KWin desktop-capture authorization (io.unom.Punktfunk.Host.desktop → $BIN)" ok "KWin desktop-capture authorization (io.unom.Punktfunk.Host.desktop → $BIN)"
# KDE Desktop-mode INPUT: a normal Plasma login lacks the RemoteDesktop portal grant the host's libei
# input path needs, so it would pop an "Allow remote control?" dialog a headless host can't answer.
# Seed it once (per-user, no root) — mirrors packaging/bazzite/kde-desktop-setup.sh. Game Mode
# (gamescope) needs none of this; the .desktop above already grants org_kde_kwin_fake_input.
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ -s "$GRANT_DST" ]; then
ok "KDE RemoteDesktop grant already present"
elif [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
systemctl --user restart xdg-permission-store 2>/dev/null || true
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
if [ "$WITH_WEB" = 1 ] && [ ! -f "$CONFIG/web.env" ]; then if [ "$WITH_WEB" = 1 ] && [ ! -f "$CONFIG/web.env" ]; then
# Random login password + session secret for the web console, generated once. # Random login password + session secret for the web console, generated once.
# `|| true` swallows the SIGPIPE `tr` takes when `head` closes the pipe (pipefail would abort). # `|| true` swallows the SIGPIPE `tr` takes when `head` closes the pipe (pipefail would abort).
@@ -168,9 +216,11 @@ else
[ "$WITH_WEB" = 1 ] && ok "web.env exists (login password unchanged)" [ "$WITH_WEB" = 1 ] && ok "web.env exists (login password unchanged)"
fi fi
# --- 4. system tuning (needs sudo; skipped gracefully if unavailable) ------ # --- 4. system tuning (needs sudo: UDP buffers + gamepad udev rule + vhci-hcd + input group) --------
log "System tuning (UDP buffers + input group) — needs sudo" log "System tuning (UDP buffers + gamepad rules + vhci-hcd + input group)"
if sudo -n true 2>/dev/null; then # sudo was acquired up front in preflight (SUDO_OK) so this never stalls behind the long build; a
# skip here (no password / no TTY) was already reported loudly there.
if [ "$SUDO_OK" = 1 ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' \ printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' \
| sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null | sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
@@ -178,13 +228,34 @@ if sudo -n true 2>/dev/null; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules && sudo udevadm trigger || true sudo udevadm control --reload-rules && sudo udevadm trigger || true
ok "installed udev rule (virtual gamepads)" ok "installed udev rule (virtual gamepads + native Steam Deck controller)"
fi
# vhci-hcd: the usbip transport that makes the virtual Steam Deck pad a *real* USB device so Steam
# Input adopts it (else it degrades to plain UHID, which Steam ignores — "no controller appears").
# Persist the autoload AND load it now so passthrough works without waiting for a reboot.
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || warn "could not load vhci-hcd now (loads on next boot) — needed for the native Steam Deck pad"
ok "vhci-hcd autoload installed (native Steam Deck controller transport)"
fi
if id -nG "$USER" | grep -qw input; then
ok "already in the 'input' group"
else
sudo usermod -aG input "$USER"
NEED_RELOGIN=1
warn "added $USER to the 'input' group (applies on next login)"
fi fi
id -nG "$USER" | grep -qw input || { sudo usermod -aG input "$USER"; warn "added $USER to 'input' group — log out/in (or reboot) for gamepad support"; }
else else
warn "passwordless sudo unavailable — skipping UDP-buffer + udev tuning." warn "no usable sudo — SKIPPED system tuning. Gamepad passthrough + clean streaming need root (udev"
warn "Without it, high-bitrate streaming drops packets. Apply manually later:" warn "rule, 'input' group, vhci-hcd, UDP buffers) — there is no user-space way to do these."
warn " echo -e 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf && sudo sysctl --system" warn "A stock SteamOS 'deck' account has NO password, so sudo can't work until you set one:"
warn " passwd # set a sudo password once, then re-run this script"
warn "Or apply it by hand (then reboot):"
warn " sudo install -m644 $SRC/scripts/60-punktfunk.rules /etc/udev/rules.d/ &&"
warn " sudo install -m644 $SRC/scripts/punktfunk-modules.conf /etc/modules-load.d/punktfunk.conf &&"
warn " sudo usermod -aG input $USER &&"
warn " printf 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432\\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf &&"
warn " sudo sysctl --system && sudo udevadm control --reload-rules && sudo udevadm trigger"
fi fi
# --- 5. systemd user services --------------------------------------------- # --- 5. systemd user services ---------------------------------------------
@@ -265,3 +336,10 @@ else
echo " • Pairing required (secure default). From a client, pick this host and enter the PIN the host shows." echo " • Pairing required (secure default). From a client, pick this host and enter the PIN the host shows."
fi fi
echo " • Update later: bash $SRC/scripts/steamdeck/update.sh" echo " • Update later: bash $SRC/scripts/steamdeck/update.sh"
if [ "$NEED_RELOGIN" = 1 ]; then
echo
warn "ONE MORE STEP before streaming — reboot the Deck (or fully log out and back in)."
echo " KWin only authorizes Desktop-mode screen capture on a fresh session, and the new 'input'"
echo " group (native Steam Deck controller passthrough) only applies to a new login. Streaming"
echo " Game Mode with a generic Xbox pad works now; Desktop capture + the native Deck pad need the reboot."
fi
+49
View File
@@ -44,6 +44,55 @@ sed "s|^Exec=.*|Exec=$TARGET_DIR/release/punktfunk-host|" "$SRC/packaging/linux/
> "$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop" > "$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
ok "KWin desktop-capture authorization refreshed" ok "KWin desktop-capture authorization refreshed"
# Retrofit the system bits install.sh now sets up but older installs predate (idempotent). vhci-hcd =
# usbip transport for the native Steam Deck pad; 60-punktfunk.rules = /dev/uhid + vhci access; input
# group = uhid write; the kde-authorized grant (per-user, no root) = Desktop-mode input. A stock Deck
# needs a sudo PASSWORD, so PROMPT for it rather than silently skipping (skipping = gamepads stay dead).
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo needs your password to (re)apply the gamepad udev rule, vhci-hcd, input group, and UDP buffers:"
sudo -v && SUDO_OK=1 || true
fi
if [ "$SUDO_OK" = 1 ]; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules >/dev/null 2>&1 || true
sudo udevadm trigger >/dev/null 2>&1 || true
ok "gamepad udev rule ensured"
fi
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || true
ok "vhci-hcd autoload ensured (native Steam Deck controller)"
fi
# UDP buffers: older installs (or sudo-skipped ones) still run the stock 416 KB cap.
if [ ! -f /etc/sysctl.d/99-punktfunk-net.conf ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null 2>&1 || true
ok "UDP socket buffers raised to 32 MB (persisted)"
fi
if id -nG "$USER" | grep -qw input; then :; else
sudo usermod -aG input "$USER"
warn "added $USER to the 'input' group — REBOOT (or log out/in) for it to apply"
fi
else
warn "no usable sudo — SKIPPED gamepad/udev/vhci/UDP tuning (all root-only; no user-space alternative)."
warn "A stock SteamOS 'deck' account has NO password — set one with 'passwd', then re-run. Gamepads stay"
warn "Xbox-360 until this runs and you reboot."
fi
echo
warn "If the controller still shows as an Xbox 360 pad, REBOOT the Deck once — the 'input' group and the"
warn "vhci-hcd module only become live for the host service on a fresh login."
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ ! -s "$GRANT_DST" ] && [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
log "Restarting services" log "Restarting services"
systemctl --user restart punktfunk-host.service systemctl --user restart punktfunk-host.service
ok "punktfunk-host restarted" ok "punktfunk-host restarted"
+4 -1
View File
@@ -89,7 +89,10 @@ A complexity ladder in [`examples/`](./examples) — start at the top:
4. [`couch-preset.effect.ts`](./examples/couch-preset.effect.ts) — **advanced, Effect-native**: only if you're composing Effect programs. 4. [`couch-preset.effect.ts`](./examples/couch-preset.effect.ts) — **advanced, Effect-native**: only if you're composing Effect programs.
Examples 13 are the plain Promise facade and cover most automation; you only need example 4's Examples 13 are the plain Promise facade and cover most automation; you only need example 4's
Effect surface for composed, interruptible programs. Run any with `bun examples/<file>.ts`. Effect surface for composed, interruptible programs. Run any **in the repo** with
`bun examples/<file>.ts`. To **deploy** one on a host, install the package into its own directory
(`bun add @punktfunk/host`) and change its `../src/…` import to `@punktfunk/host` — see
[Running a single script as a service](#running-a-single-script-as-a-service).
Plus a real-world recipe: Plus a real-world recipe:
+5
View File
@@ -15,6 +15,11 @@
// VH_DEVICE=couch-deck.11 bun examples/virtualhere-dualsense.ts # address from `-t LIST` // VH_DEVICE=couch-deck.11 bun examples/virtualhere-dualsense.ts # address from `-t LIST`
// VH_DEVICE=DualSense bun examples/virtualhere-dualsense.ts # …or match by name substring // VH_DEVICE=DualSense bun examples/virtualhere-dualsense.ts # …or match by name substring
// //
// To run this *outside* the repo (the normal case), drop it in its own dir, `bun add
// @punktfunk/host`, and change the import below from `../src/index.js` to `@punktfunk/host`; then
// keep it alive as a systemd user unit (its SIGTERM release hands the pad back on `systemctl
// stop`). Full walkthrough: docs → Events & hooks → "full controller passthrough (VirtualHere)".
//
// Env: VH_DEVICE required — a VirtualHere address (`server.port`) or a device-name substring. // Env: VH_DEVICE required — a VirtualHere address (`server.port`) or a device-name substring.
// VH_CLIENT client binary. Default `vhclientx86_64` (Linux); Windows `vhui64.exe`, // VH_CLIENT client binary. Default `vhclientx86_64` (Linux); Windows `vhui64.exe`,
// macOS `vhclientosx`, ARM Linux `vhclientarm64`. // macOS `vhclientosx`, ARM Linux `vhclientarm64`.
+2 -1
View File
@@ -6,6 +6,7 @@
//! harness adds `tc netem` jitter/reorder on the UDP path. //! harness adds `tc netem` jitter/reorder on the UDP path.
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role}; use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionKey;
use punktfunk_core::error::PunktfunkError; use punktfunk_core::error::PunktfunkError;
use punktfunk_core::session::Session; use punktfunk_core::session::Session;
use punktfunk_core::transport::loopback_pair; use punktfunk_core::transport::loopback_pair;
@@ -25,7 +26,7 @@ fn config(role: Role, scheme: FecScheme, drop_period: u32) -> Config {
shard_payload: 1024, shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024, max_frame_bytes: 8 * 1024 * 1024,
encrypt: false, encrypt: false,
key: [0u8; 16], key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4], salt: [0u8; 4],
loopback_drop_period: drop_period, loopback_drop_period: drop_period,
} }